By Aaron Moss, PhD & Tommy Roggina
At CloudResearch, we advocate for requesters to treat workers fairly when posting HITs on Amazon’s Mechanical Turk (MTurk). Workers are, after all, the people who make the research possible. Sometimes situations arise in which an MTurk worker is unable to receive payment, despite having completed a survey. Below are two common scenarios in which a worker may not be paid, despite completing a survey:
Problem
Imagine a worker taking your survey on a poor internet connection. The worker submits their secret code at the end of the survey, but because of the poor connection or a disruption in service, the HIT registers on CloudResearch but not on MTurk. In this situation, you may see the HIT listed in your .csv file as “submitted to CloudResearch.” You may also notice that your HIT has completed, but the progress bar is one worker short of the requested total. For more on this, read “Why has my study been marked as “Complete” when it is short of the requested number of participants?” knowledge base article. In this situation, the worker may email you saying, “I took your survey, but I do not see it pending on my dashboard. Can you please compensate me for my time?” or “I submitted the HIT, but I’m not sure it went through. Can you confirm my payment?”
Imagine a worker is taking your survey and they are concentrating diligently on it. The worker attempts to submit the HIT, but cannot because they have taken too much time and the HIT has expired. You then receive an email from the worker stating, “I took your entire survey, but, when I went to submit the HIT, it had already expired. Could you please compensate me?” Note: CloudResearch recommends setting the maximum time to complete a HIT to at least double the amount of time the survey is expected to take. Ideally, you should give workers three or four times as long as you expect the study to take because some people will be slower than others.
Solution
Compensation HIT Feature
In scenarios like the ones described above, you can use CloudResearch’s Compensation HIT feature to pay workers. Compensation HITs are regular MTurk HITs that only require the worker to accept and submit the HIT to receive payment. Once a worker submits your Compensation HIT, CloudResearch will automatically approve and pay the worker. You could manually create compensation HITs on your own, but our feature greatly simplifies and speeds up the process.
Please note: For workers who are taking one of your HITs for the first time, have them search for the Compensation HIT on their MTurk Dashboard. If they cannot find it, please reach out to support@cloudresearch.com and ask a representative to post the Compensation HIT for you.
To create a Compensation HIT, go to the “Manage Workers” tab and select “Create Compensation HIT” from the drop-down menu.
Once you choose to “Create a Compensation HIT,” you will be taken to the Create Compensation HIT page.
When compensating workers, please keep in mind that regular CloudResearch fees apply in addition to the 20% Amazon fee and the CloudResearch pro features fee. Visit the Pricing page for more details.
In the large textbox, enter the Amazon Mechanical Turk IDs or CloudResearch IDs of the workers you would like to compensate. Enter each ID on a separate line.
Next, enter the amount you would like to compensate the worker(s) and save the HIT.
After saving your HIT, you will be returned to the study dashboard where you will see your Compensation HIT on the top of your study list.
After launching your study, we recommend emailing workers to notify them that you have created a compensation HIT for them. You must wait 6 minutes from the time the HIT is launched before emailing workers.
The Email Included Workers button will light up orange when you can email the workers. Once you have emailed workers, it will keep track of how long it has been since your last mass email notification.
Bonusing Workers
An alternative method of compensating workers is to issue them a bonus by clicking on the green “Grant Bonuses” button in your original HIT. You can issue a bonus to any study a worker has taken, even if it has already completed.
Please note: You can only grant a bonus to a worker who has completed at least one HIT for you previously. For workers who are taking one of your HITs for the first time, bonuses are not an option.
You will be taken to the grant bonus page (see below).
For bonus HITs, only a 20% Amazon fee is applied.
Please note that worker bonuses are cumulative. If you have given a worker a bonus of $2 in the past and would like to grant them a $3 bonus now, you must enter $5 into the amount text box.
You may check how much you have bonused a worker in the past by entering their Amazon ID into the Check Bonus textbox.
To bonus a worker, enter the worker’s MTurk or CloudResearch ID into the “Worker IDs” textbox on separate lines. You may also include a message.
New Bonus Feature
Alternatively, you can also bonus workers using our new feature that allows you to send different bonus amounts to different workers all at once.
To use this feature you must input a .csv file that has the MTurk IDs in the first column and the payment amount in the second column. You can download a .csv file from your study dashboard that includes MTurk IDs, as well as other information on the study.
Next, drag and drop the file you just created into the drag and drop field.
Success! You have just compensated the workers on your list!
While there are some scenarios that can engender a MTurk worker losing pay, CloudResearch offers several options for requesters to compensate workers. As always, we encourage requesters to be considerate of workers’ time and to compensate workers fairly when possible.
