Millisecond Forums

Running Studies with Inquisit and MTurk (UPDATED)


http://www.millisecond.com/forums/Topic15757.aspx

By seandr - Thursday, March 12, 2015

UPDATE: Good news for Inquisit Web researchers - downloading the Inquisit Web app is no longer against MTurk policy.  Amazon has loosened it's policy around software downloads. Rather than prohibiting software downloads in general, the new policy prohibits "HITs that require Workers to download software that contains any malware, spyware, viruses, or other harmful code". Amazon's complete terms of service policies can be found here
-----------------------------------
Amazon's MTurk is a popular and effective way to quickly recruit research participants for online studies. Unfortunately, Inquisit studies are technically not allowed on MTurk because its application model violates the site's no download policy. MTurk policies are enforced by the workers, so if a worker reports an Inquisit study, Amazon will shut it down. To be clear, Inquisit is a benevolent app. However, Amazon does not maintain an application white list or otherwise make exceptions to its policy.

Nevertheless, researchers can and do successfully run Inquisit studies with MTurk. The key is to keep the workers happy. If you are planning to run an MTurk study with Inquisit, we suggest the following guidelines:
1) Pay the workers well.
2) Pay the workers regardless of what happens.

3) Pay the workers promptly.
4) Restrict your study to master workers if you can afford to.
5) Keep studies down to 20 minutes or less, and if you need more time, compensate more generously.
6) Do not use your personal Amazon account for doing research.

[UPDATE]: We have received the following feedback from a customer who spoke with a representative at Amazon about the download policy. The customer was advised that linking to the Inquisit Web app as a prerequisite for the HIT was not a violation of their policy. (We have not been able to reach Amazon to confirm.) If you choose to use this strategy, you may link to the following download page:
http://www.millisecond.com/download/inquisitweb4.aspx

Generally speaking, workers appreciate HITS that respect the value of their time, and they are disinclined to take away well-paying opportunities from themselves and their fellow workers. 

If you receive questions about Inquisit from workers or other participants, please direct them to the following page:
http://www.millisecond.com/web/participants.aspx