CODEWITHSUNDEEP
amazon-web-servicesmechanicalturk
abuccts
abuccts

Reputation: 339

Amazon Mechanical Turk: Random Questions in One Qualification Type

I am using CreateQualificationType operation to create a new qualification type in MTurk, so that I can require this qualification test in my HITs.

Now I have a set of questions for this qualification test, but I only want MTurk workers who request this qualification test to answer just a small part of those questions, and different works answer different questions randomly. I didn't find any useful answers in AWS Document about Qualifications. Is there a method to achieve that?

Upvotes: 1

Views: 627

Answers (2)

mcwizard
mcwizard

Reputation: 461

a piece of cake, actually. what you need to do is create the question xml file inside a loop and then create the hit from that question xml. In the example below I hand crafted a question xml and verified that it worked on the sandbox. then I split the question xml file into parts a,b,c corresponding to the parts that did not change, then in my loop I write the xml code to a text file.

  • parta
  • part of the question that changes, in this case the image
  • partb part of the question that changes, in this case the multiple choice
  • partc

at the conclusion of the progress I have a valid xml file that I open and use in the create hit. Now I know this question is about a qualification, but the idea is exactly the same.

for hit_number in range(how_many):
    print(hit_number)
    my_emotions = sample(emotion_names,5)
    my_emoji = sample(emoji_numbers,1)
    HIT_image = <a url on S3 is convenient>

    f_out = open('data/dynamic_question.xml', 'w')
    f_out.write(parta)
    f_out.close()

    f_out = open('data/dynamic_question.xml', 'a')
    f_out.write('    <DataURL>' + HIT_image + '</DataURL>' + '\n')
    f_out.write(partb)

    for emotion in my_emotions:
        f_out.write('          <Selection>' + '\n')
        f_out.write('            <SelectionIdentifier>' + emotion + '</SelectionIdentifier>' + '\n')
        f_out.write('            <Text>' + emotion.capitalize() + '</Text>' + '\n')
        f_out.write('          </Selection>' + '\n')

    f_out.write(partc)
    f_out.close()

    with open("data/dynamic_question.xml") as f:
        question = f.read()

    # Create the HIT 
    response = client.create_hit(

Upvotes: 2

Thomas
Thomas

Reputation: 44555

This is unfortunately not possible. A Qualification test can only be setup using QuestionForm XML, which does not allow randomization of any kind. The most commonly recommended alternative is to create a HIT with the content you want and then assign qualification scores to workers based upon their responses to the HIT. Less than ideal, but the only option.

Upvotes: 1

Related Questions