Giter VIP home page Giter VIP logo

qems2's Introduction

QEMS2 - a question submission and editing system

What is it?

QEMS2 is a successor to a system called, intuitively enough, QEMS, which was used by High School Academic Pyramidal Questions to produce quizbowl questions for various high school quizbowl competitions. The original QEMS was written in PHP and broke down after the maintainer stopped working on it; it was also completely hand-coded and made no use of modern web development frameworks or standards. QEMS2 is an attempt to remedy all that, by rebuilding the system from the ground up using modern tools.

Stack

QEMS2 is based on a technology stack that uses MySQL for storage, Python (in the form of Django) for the backend, and jQuery/Foundation for frontend manipulations.

Packetization is currently (sort of) handled with a separate C# app available here: https://github.com/mbentley00/QemsPacketizer

Installation

Grab the code:

git clone https://github.com/grapesmoker/qems2

Because the Django secret key is not checked in, you will need to create a new file called "secret" in the root directory that you cloned. This file will look something like this:

SECRET_KEY = 'your secret key'

Generic *nix (including OS X)

To run QEMS2, you'll need the following prerequisites.

python2 >= 2.7, MySQL, nodejs

Once you have those installed, you should use pip to get the necessary Python packages. It's generally recommended that you use virtualenv to set up a virtual environment for your project.

pip install django<2
pip install beautifulsoup4
pip install django-bower
pip install django-contrib-comments
pip install django-haystack==2.3.1
pip install django-allauth
pip install whoosh
pip install mysql-python
pip install unicodecsv

Next, grab bower using npm for front-end package management:

sudo npm install -g bower

Set up your MySQL connection as, for example, mysql -u root -p:

CREATE USER django@localhost IDENTIFIED BY 'django';
CREATE DATABASE qems2_stable;
GRANT ALL PRIVILEGES ON qems2_stable.* TO django@localhost;
GRANT ALL PRIVILEGES ON test_qems2_stable.* TO django@localhost;

If you used a different database than "qems2_stable" in the above steps, open settings.py and change the value of NAME to be whatever you used above.

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
        'NAME': 'qems2_stable',                      # Or path to database file if using sqlite3.
        'USER': 'django',                      # Not used with sqlite3.
        'PASSWORD': 'django',                  # Not used with sqlite3.
        'HOST': '',                      # Set to empty string for localhost. Not used with sqlite3.
        'PORT': '',                      # Set to empty string for default. Not used with sqlite3.
    }
}

While you're in settings.py, you'll probably want to disable the e-mail settings to make debugging easier:

# Account and registration settings
ACCOUNT_EMAIL_REQUIRED = False
ACCOUNT_EMAIL_VERIFICATION='none'

Finally, use manage.py to populate the database, install the front-end packages, collect static files, and start the development server:

python manage.py syncdb
python manage.py bower install
python manage.py collectstatic
python manage.py runserver

If the bower install command is failing because node can't be found, you may need to install nodejs-legacy.

Windows

Install the 32-bit version of Python 2.7.9 to C:\Python27; make sure to install the component that modifies your path. If the msi fails due to a missing DLL component, navigate to C:\Users{username}\AppData\Local, right-click on Temp, go to Properties > Security and give "Everyone" "Full Control" (you'll get a lot of errors, just power through w/ the okay button).

Install the 32-bit MySQL installer from http://dev.mysql.com/downloads/windows/ and the 32-bit msi from http://nodejs.org/download/.

Run an instance of Windows PowerShell as an administrator and:

pip install django<2
pip install beautifulsoup4
pip install django-bower
pip install django-contrib-comments
pip install django-haystack
pip install django-registration-redux
pip install whoosh
pip install unicodecsv

Install the latest from https://pypi.python.org/pypi/MySQL-python/, make sure git is installed, and make sure all of the above programs are in your path, incl. variations on: C:\Program Files (x86)\MySQL\MySQL Workbench 6.2 CE, C:\Program Files (x86)\Git\cmd, C:\Python27, C:\Python27\Scripts, C:\Program Files (x86)\nodejs, %LocalAppData%\..\Roaming\npm. Then npm install -g bower.

Then follow the above instructions to set up the MySQL connection and the server, except make sure to add BOWER_PATH = os.path.normpath(r'C:\Users\{username}\AppData\Roaming\npm\bower.cmd') below the BOWER_COMPONENTS_ROOT line in settings.py before bower install'ing. If you see an error about being unable to connect to port 10061, update the HOST and PORT properties in DATABASES in settings.py. Note that the installation of git may require you to reference python as C:\Python27\python.exe instead of python in PowerShell.

Running QEMS2

As with any Django project, you should now be able to access the website at http://localhost:8000.

You can populate the database with some default data by running python manage.py shell < qems2/qsub/populate_db_with_default_data.py.

Usage

QEMS2 is currently in heavy development. Although the core parts of the application are stable, features are being added all the time, so it's hard to give an exhaustive description of them all here. The foregoing will summarize the basic concepts involved in the question submission and editing workflow.

Roles

QEMS2 is structured around the concept of roles. There are three types of roles in the system: owners, editors, and writers (there are also administrators who can access the admin site, but that is not strictly speaking a part of the application itself). The hierarchy of powers is strictly inclusive from the top down, i.e. anything that an editor can do, an owner can do, and anything that a writer can do, an editor can do.

  1. Owners: Anyone who creates a tournament is automatically the owner of that tournament and has complete freedom to do anything they would like to it. This includes assigning other users to be editors or writers of their tournament, modifying the tournament distribution, and editing any question in the system.
  2. Editors: these users have the power to edit or delete any question in the system. They do not have the power to change the distribution or the core information about the set. Editors can also lock questions to prevent writers from editing those questions.
  3. Writers: these users are empowered only to submit questions, edit (unlocked) questions that they have submitted, and see (but not edit) questions submitted by others.

In addition, any user can leave comments on packets, individual questions, or the set as a whole.

Distributions

The number and type of questions that a question set requires is controlled by assigning a distribution to a set. You can create a distribution by clicking on the Distributions link on the sidebar, giving your distribution a name, and adding category/subcategory entries, in addition to numbers specifying both maximum and minimum tossups and bonuses in that category per packet.

Question Sets

The question set is the basic unit of operation in QEMS2. Question sets are created by clicking on the Question Sets link on the sidebar and then Create Question Set. Question sets are required to have some distribution, but picking a distribution does not preclude you from editing that distribution later. Once a question set is created, you can manage that question set through various tabs:

  • Question set info: this tab contains the basic information about the set (name, date, and distribution), as well as a status report on the set's completion. The status screen lets you know how many questions of each kind have been written and how many still remain to be written. Green check marks indicate that the minimum number of questions in each category/subcategory entry have been written.
  • Editor assignments: this tab allows the owner to add writers and editors to a set. Clicking on either Add Editor or Add Writer brings up a screen with a list of users who can be added to the set. Users can only be either writers or editors.
  • Set-wide distribution: this is one of the most important tabs. The set-wide distribution controls the total number of questions in each category/subcategory entry. You cannot change the categories themselves (that is only possible via the Distributions screen); you can only change the total number of tossups and bonuses in each. This number covers the entire set as opposed to the main distribution which applies at the packet level. For example, the a distribution may call for a maximum of 1 tossup and 1 bonus and a minimum of 0 of each in the category/subcategory Science/Math. Those constraints apply on a per packet basis. If you have 10 packets in your set, you may only want 5 tossups and 5 bonuses in Science/Math, in which case you would set the numbers for the set-wide distribution 5 each.
  • Tiebreaker distribution: the tiebreaker distribution functions exactly like the set-wide distribution, but only applies to tiebreaker questions. The numbers indicate the total number of tiebreakers in each category/subcategory entry on a per set basis.
  • Questions: this tab gives users access to the questions that are part of the set. From here, a user can either add tossups and bonuses to the set manually or by uploading a file. It also shows users the questions that are part of the set. From here, an editor can click on the Edit link under the Actions column to edit the questions. The editorial workflow will be covered in a different section.
  • Packets: questions are collected into packets. This tab allows the owner to create a packet, or multiple packets to be part of the set. Clicking on a packet link will bring up a page that indicates the completion status of the packet, which is similar to the completion status page of the set. The completion status of a packet is controlled by distribution assigned to the set. This tab also allows the owner to add questions either manually or from an existing stock of questions that are part of the set but have not been assigned to a packet.

Editing Workflow

The editing workflow of QEMS2 starts with writers and editors being assigned to a question set by the set owner. After that, writers and editors are free to start writing questions; they can do so either by adding questions manually or uploading questions in a text file. Once a writer has submitted a question, it can be edited by an editor. Usually, once the editor has completed their editing, they can lock a question from further editing by a writer. This does not prevent the editor or the set owner from editing these questions; it only locks out the original question writer. At any point, a question may be moved by an editor or the set owner into a packet.

The submission/editing cycle continues until both the set-level constraints and the packet-level constraints have been satisfied. Once the constraints have been satisfied and the questions have been assigned to packets, the packets may be exported to PDF form. At that point, the set is ready to be played.

Format for Question Uploads

If you are uploading questions, they have to be formatted in a certain way. An uploaded file must be in plain text, and the questions must have this form:

Tossups:

This scientist noted the difference between Asiatic and Oceanic fauna, hypothesizing that their geographical isolation played a role in their different developments; the boundary between the two types of fauna is now known as his namesake line. For 10 points, identify this scientist who proposed a theory of evolution contemporaneously with Charles Darwin.

ANSWER: Alfred Russell _Wallace_

Note that the question text must be all on a single line, followed by a carriage return, with the answer prefaced by the word "ANSWER:" and the required parts of the answer set off by the underscore marks.

Bonuses:

At one point in this film, two characters are arrested after they mistake dust-covered Union soldiers for Confederates. For 10 points each:

[10] Identify this film in which those two characters later help a Union colonel blow up a bridge across a river that they need to cross, while searching for Bill Carson's gold.

ANSWER: The _Good, the Bad, and the Ugly_

[10] The Good, the Bad, and the Ugly is probably the most famous of the spaghetti westerns of Sergio Leone, starring this man as "Blondie."

ANSWER: Clint _Eastwood_

[10] Eastwood's other quasi-historical roles include this role in Unforgiven, in which he kills the Sherrif played by Gene Hackman, as revenge for Hackman's killing of his companion, played by Morgan Freeman.

ANSWER: _William_ [or _Bill_] _Munny_ [accept either name]

The leadin must be on a single line, followed by a newline. Each part must be prefaced with the value in square brackets, e.g. [10], followed by the text of the bonus part, followed by a newline. As with tossups, the answer must be set off with "ANSWER:" followed by the answer. In the prompt, emphasis can be applied using underscores (or asterisks), while in the answer, underscores are used to indicate required parts. You do not need blank lines between question parts and answers; those are just here for readability and will be ignored.

Testing

To run the included unit tests (under qems2/qsub/tests/):

python manage.py test qems2/qsub/tests/

If a test fails, that is probably a bad thing.

Upcoming Features

QEMS2 is evolving fairly rapidly. Features are being added frequently at the request of the users. There are a number of upcoming additions that will be made to QEMS2 in the next few months:

  1. Bulk question writing will allow users to write directly into a text-box instead of having to upload a file.
  2. Question set status that takes tiebreakers into account.
  3. Customizable PDF generation.
  4. Pagination in long lists
  5. Full-text search

qems2's People

Contributors

alopezlago avatar ashvinsrivatsa avatar epicfaace avatar grapesmoker avatar jonathannoah avatar mbentley00 avatar sephirothrr avatar theta91 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

qems2's Issues

Users can go to "add editors" page without admin/editor permission

REPRO:

  1. Add a random integer to the end of http://qems2.grapesmoker.net/add_editor/, such as http://qems2.grapesmoker.net/add_editor/2/

RESULTS:
I don't have editor permissions for this set, but I'm still allowed to add editors to the set:
image

I have no idea what hitting the submit button will do in this case. Didn't want to break anything by clicking it.

Note that if I went to the NSC set in the conventional way and then clicked on "Editor Assignments" link, I get the proper "you don't have permissions" error.

EXPECTED:
Users without permission should see a permissions error page when viewing that page.

Moving between pages on Questions tab changes vertical position of page change dialog

REPRO:

  1. Go to the "Questions" tab for a set with multiple pages of tossups and bonuses.
  2. Hit the "next" button for tossups. Also do things like change the number of questions to display by default on one page.

RESULTS:
The vertical scroll position of the page will change making it hard to stay focused on where you are. Additionally, by doing a combination of changing pages and number of questions on a page, you can get into weird states like this:
image

I've redacted the questions in the image, but what's happening here is that the tossup's back/forward buttons are displayed in the middle of the list of bonuses.

EXPECTED:
Back/forward buttons don't have weird scrolling issues

tags

Need a tag system to assign various tags to questions. Would be super if we could also generate tag clouds by frequency or something.

navigation pillboxes/links

Each page within the set > packet > question hierarchy needs links back to its parents for easy navigation.

"Errors in your form! All question numbers are required!" when new distribution created

If you create a new distribution, you will see the label mentioned in the title. You won't be able to add anything to the distribution from your current screen. If you go back to the distributions page and click on the link, you will now be able to edit it.

I think it's complaining about the missing entry_id, as adding that directly to the URL seems to fix the issue.

Questions uploaded through the type questions dialog don't have their types classified

REPRO:

  1. Upload a question like this in the "Type Questions" dialog:

"Peace comes dripping slow" for this man, whose debut solo album featured tracks like "Cry Me a River" and "Senorita". Inspired by his nostalgia for a place in County Sligo, this man appeared in the "Motherlover" digital short and played Sean Parker in The Social Network. For 10 points, name this William Butler Yeats poem and former member of 'N Sync who recorded "SexyBack".
ANSWER: Justin Timberlake Isle of Innisfree

  1. Go to the edit question dialog for this question.

RESULTS:
The tossup's "Question Type" is still set to blank.

EXPECTED:
The tossup's "Question Type" should be set to tossup in this case (or bonus if a bonus was uploaded).

NOTES:
Not sure that this matters all too much behind the scenes--the parser was smart enough to correctly place tossups/bonuses in the correct category on the questions view.

Require e-mail verification of accounts

REPRO:

  1. Register a new account with someone else's e-mail address.
  2. Alternatively, change the e-mail address on an existing account to someone else's e-mail address.

RESULTS:
No verification/notification that this username is linked to this e-mail address.

EXPECTED:
At the very least we should send an e-mail to that e-mail address. Probably we should require e-mail confirmation.

NOTES:
This could be used for phishing/spoofing.

admin panel

Admins need to be able to perform site admin actions like approve pending registrations, ban users, etc.

settings panel

Users need to be able to perform profile-management tasks like changing personal info and password.

Remove "delete tossup" button; make it an option from "edit question" screen

The "delete tossup" button is too close to the "edit question" button. More importantly, there's little reason for it to be there at all. This option should be moved to the "edit question" screen (and restricted solely to editors, if this is not already the case).

priority: very low (i.e. after packetizing has been implemented).

Implement QEMS1.0 'Type Questions' box

A RTE is nice, but QEMS1.0 had the 'type questions' box that allowed you to paste anywhere from one to as many questions as you wanted in it and submit, then tag each separately w/ type (auto-detected)/category/subcategory/time/place.

This is pretty much the # 1 feature we need with regards to question submission for QEMS2.0 because it was the easiest way to submit questions: write wherever you want (word document, google doc) to get the line length right, then submit in small batches whenever you felt like it. No creating multiple files for upload, just copy/paste every several questions.

html in question/answer content

Right now, we sanitize the questions and answers before they're saved into the db. We do this because, I suspect, having HTML in question text will screw up the fulltext search that we will need to have. Maybe this is crazy, but I'm thinking that perhaps we can save each text field twice, once as a sanitized "search" version without any html, and once as the "display" version with html. Obviously this doubles per-question overhead and so is not ideal; for now we won't do this but will see if it causes an issue when fulltext indexing is enabled.

sort long lists / make long lists sortable

in particular, the completion percentages list is in a really wacky order (alphabetical would do fine, or the same order as the distribution page would be dandy). may also be very useful to sort the long list of questions by author or by category rather than by date submitted. How doable is it to auto-sort the table by clicking on a column header, like the sortable tables on QBWiki?

Use HTTPS in production

Everything on the production server is currently served through HTTP. This means that a man-in-the-middle could steal credentials and pretend to be someone else. This is probably beyond the reach of most high schoolers, but it is doable if the user connects to a public hotspot (e.g. campus or restaurant network).

Type question returning errors

I tried to enter the text of a tossup (with, to my knowledge, no soft breaks) followed by ANSWER: xxx on a separate line and got an IntegrityError with value "Column 'question_number' cannot be null" at /complete_upload/. I was directed to the error page after getting to the confirmation page and hitting "submit". Is there some field that doesn't have a value somewhere?

After submitting a question, clear the text from the fields

REPRO:

  1. Submit a new tossup using the regular "Add a Tossup" dialog.

RESULTS:
You get a results page that says your tossup has been added. You also see the same Tossup Text and Tossup Answer content as in your submitted question. It's not clear if you're now on an edit screen or if you delete this text whether it will submit a new question or not.

EXPECTED:
Clear the tossup and answer textboxes after submitting a question to make it clearer you can use this form to submit another tossup.

No way to log out

At the very least I can't figure out how to do it. Profile page or clicking on the "QEMS - Logged in as [username]" don't seem to have this feature.

Allow more formatting in comment fields

I'd like to be able to separate different points within the same comment with line/paragraph breaks. Couldn't hurt to allow full formatting (italics etc) to the extent that it's allowed in questions, since we may use the comment field to explain how part of a question is supposed to look.

Don't reference packages from js/ unless they are custom scripts

FYI: any files that you might find in the js/ subfolder of static are not to be used because they are likely old. The only thing in there that's current are the custom scripts like qsub.js and other such things. All bower-installable things should be srced from their bower-installed dir, e.g. for tinymce, src=tinymce/tinymce.jquery.js. I'll have to clean up the js/ subdir at some point to prevent this.

Accent Mark Errors

I was editing some of my questions and came across a problem where I was not able to save my edits unless I removed the accent marks from the answer line (for example, it would only accept changes made to answer lines that read "Republica de Nicaragua" without the accented "u" in the word "Republica," and so on). I did so in hopes of clicking the "edit" button again and inserting the correctly punctuated answer line, but I kept getting the wacky error page whenever I tried to re-insert the symbols that contained accent marks. Am I doing something wrong here?

Install instructions

Might be a good idea to document this somewhere. Here's what I did to get it running on my Mac:

git clone https://github.com/grapesmoker/qems2
cd qems2/
sudo pip install django-debug-toolbar
sudo pip install django-bower
sudo npm install -g bower
python manage.py syncdb
python manage.py bower install
python manage.py runserver

MySQL

create user django@localhost identified by 'django';
create database qems2;
grant all privileges on qems2.* to django@localhost;

Required parts of answers need to have a distinctive mark

Having people MANUALLY FORMAT the required part of an answer as bold and underline on the writing side is CRAAAAZY! We need to go back to using underscores or another single-purpose mark and handling the formatting in the packet display routine. Without this, question search and repeat checking is going to be completely impossible, and we are locked into a particular decision on how to display final packets instead of being able to set it later or change it without changing every individual question.

This is not even the first thing that needs to be addressed, it's more like the zeroth.

Standardize name display

I like the feature that shows real names rather than login IDs in the "add writers" screen (though with the caveats about security that I mentioned elsewhere). This should be implemented everywhere -- seeing the real name of the person who wrote a question rather than a possibly unintuitive ID would help.

Better initial landing screen: just a list of question sets

The QEMS1 landing screen was good; QEMS2 much less so. You have to click at least three times just to get to a question set, which is really inefficient.

A new landing screen should be designed that simply lists all question sets you are on -- there should be no distinction between a set you own, are editing, or are writing for on the landing screen; everything should be listed in one place.

priority: after packetizing

Add a .gitignore to the repo

Jerry - I'm sure you have a .gitignore on the box where you're doing your development; can you add it to the repo?

Add test coverage

There's currently no test coverage. We should add tests and make it a requirement to have all tests passing before we can push our changes.

role-based security

Use Django's role-based security mechanism to restrict what users can do. E.g. admins are allowed to police users, remove comments, etc., but ordinary users are not.

Can't "assign categories to editors"

Not sure if this is a known issue yet, clicking this button just leads to an error page. I assume the eventual function of this will be to restrict editors to certain categories, and that they currently can edit any question in the set?

search

See possible problems with issue #10. We need fulltext search.

Not logging in?

This may be a trivial issue, but for some reason whenever I go to the qems2 site it says "QEMS 2- logged in as" and then nothing. I can't see any question sets, and the only distribution I can see is NSC. I registered with the username "lasy."

Back button doesn't preserve the page you were on

REPRO:

  1. Go to the Questions tab.
  2. Click on a specific question to edit.
  3. Hit the back button in the browser.

RESULT:
You are taken back to the main page.

EXPECTED:
You should be taken back to the page you were last on, i.e. the questions page.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.