Blog

Read our latest articles for tips, tricks, advice and news about research techniques and solutions.

Creating Mechanical Turk Custom Panels With CloudResearch Worker Groups

Jonathan Robinson, PhD
March 23, 2015

Creating Mechanical Turk Custom Panels With CloudResearch Worker Groups

Problem: Suppose you need to run a group of HITs open only to participants who are women under 50. You previously ran a HIT and know the Worker IDs that you want to reach, but have no way to email...

Read More >
IRB Template for Mechanical Turk and TurkPrime

Leib Litman, PhD
March 13, 2015

IRB Template for Mechanical Turk and TurkPrime

An IRB will generally request a description of how participants will be recruited, reimbursed and interacted with. Additionally IRBs always request information about how anonymity of the participants is protected. Members of the IRB board may not be familiar with Amazon Turk, and it may be helpful to include a brief description of MTurk in your IRB application.

Read More >
The New New Demographics on Mechanical Turk: Is there Still a Gender Gap?

Leib Litman, PhD
March 12, 2015

The New New Demographics on Mechanical Turk: Is there Still a Gender Gap?

In this blog, we report on the demographics of Mechanical Turk workers by reviewing 75 studies conducted with US-based Workers in 2013 and 2014. From a total of 32,595 Workers, we found 15,324 were female (47%).

Read More >
A Simple Formula for Predicting the Time to Complete a Study on Mechanical Turk

CloudResearch
March 9, 2015

A Simple Formula for Predicting the Time to Complete a Study on Mechanical Turk

We describe a general formula for predicting the time it takes Workers to complete survey studies on MTurk. The average Worker takes 10.3 seconds to answer a single question. This means that a study with 60 questions should take approximately 10 minutes.

Read More >
How to Minimize Dropout Rate on Mechanical Turk

Leib Litman, PhD
March 8, 2015

How to Minimize Dropout Rate on Mechanical Turk

It is generally thought that pay rate does not affect data quality on Mechanical Turk. For example (Buhrmester, Kwang, & Gosling, 2011) showed that whether Workers are paid 5 cents or one dollar for a survey study, the internal reliability of the surveys does not change. They did show however that fewer Workers will take the surveys that pay less. We recently replicated these findings for both US and India-based Workers (Litman et al, 2014).

Read More >
Determining Completion Rate and Dropout Rate on Mechanical Turk

Leib Litman, PhD
March 6, 2015

Determining Completion Rate and Dropout Rate on Mechanical Turk

What is the completion rate and dropout rate? Dropout rate is defined as the percentage of participants who start taking a study but do not complete it. Dropout rate is sometimes referred to as attrition rate, and is the opposite...

Read More >
Why Use CloudResearch Panels?

Leib Litman, PhD
February 20, 2015

Why Use CloudResearch Panels?

MTurk Requesters are often interested in studying specific groups of people. For example, a researcher may be interested in men over 40, Republicans, people who are concerned about the cleanliness of sponges, or cancer survivors. CloudResearch Panels utilizes various techniques...

Read More >
System Qualification Enhancements – US State Qualifications

Jonathan Robinson, PhD
January 30, 2015

System Qualification Enhancements – US State Qualifications

Amazon just announced that their Worker Qualification now supports US State locations. It is currently available through their API and is also available through their Web Interface. It is great to see that Amazon is adding features to their API;...

Read More >

SUBSCRIBE TO RECEIVE UPDATES

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.