Repository containing various templates to be used with Amazon MTurk for pairwise preference annoations with custom questions.
Copy the desired template into Design Layout when creating a new Project on Amazon MTurk
Template assumes a csv with the following columns as input:
poem1, poem2, question1, question1_id, question2, question2_id, question3, question3_id, question4, question4_id
question4 and question4_id are optional and can be left blank. In this case only the first 3 questions are displayed to the annotator.
Similar setup as the previous template but with an additional n/a field in the form to allow annotators not only decide between poem1 and poem2 but also left a question undecided.
Template with 5 questions and no blank fields allowed, displaying a binary choice for each question.
poem1, poem2, question1, question1_id, question2, question2_id, question3, question3_id, question4, question4_id, question5, question5_id