Announcing psiTurk 2.0!

The lab is proud to announce the release psiTurk 2.0, the latest version of our open framework for running experiments online using Amazon Mechanical Turk. This represents a large effort in the lab featuring nearly 1000 commits on github from 9 contributors, and feedback from over 35 early beta testers.

Along with the release of the software we have launched psiturk.org, a web service that facilitates online data collection. The website is home to extensive documentation of the psiTurk code as well as a new “experiment exchange” where you can find psiTurk-compatible experiments, download their code, replicate them, or extend them for your own research purposes.

psiTurk enables entirely new ways of developing complex scientific experiments online. At the same time, it aims to significantly reduce the wasted effort across academic labs and those in industry when programmers must re-implement the same designs over and over. By enabling a unified interface and new ways to exchange code, psiTurk helps students and researchers leverage very best software ideas in their own science projects.

Major features in psiTurk 2.0 include:

  • Interactive command-line. Provides a simple way to interact with Amazon Mechanical Turk to pay participants, check your balance, create HITs, etc… In addition to making it much easier to use AMT, the new command line exposes features not present on the Amazon website including the ability to “unreject” workers who are accidentally rejected.
  • Prevent repeat workers. Prevents Mechanical Turk workers from doing your task more than once (even across multiple HITs).
  • Block particular browsers. It is easy to configure psiTurk to block users using particular types of browsers with a known incompatibility with your experiment code.
  • Automatically record info about browser state. psiTurk automatically (i.e., no coding required) records information about the browser state of people in your experiment including if people are switching between windows on their computer during a task.
  • Experiment Exchange. The psiturk.org Experiment Exchange provides a searchable public index of open-sourced psiTurk-compatible experiments. Use these to replicate someone else’s design and to learn from their code. Via your psiturk.org account you can share you experiment with others and get credit for your contributions (via github).
  • Secure Ad Hosting. Due to new security restrictions in modern browsers if you aren’t using an officially signed-SSL certificate to display your web experiment, many workers on Amazon Mechanical Turk will not be able to see your task in the HIT search page. psiturk.org provides a free, Secure Ad Hosting service which you can use to host an “ad” for your experiment which is properly signed.
  • High availability, zero-config databases. psiTurk allows you to easily spin up MySQL databases on Amazon’s cloud using the same account credentials you use to connect to Mechanical Turk. No need to install and manage a database on your university or company network.

These are really just a subset of the new features and bug fixes which appear in version 2.0. We are still limiting enrollment on psiturk.org to ensure the system can scale with more users. If you would like to request to be a part of our initial roll out please visit https://psiturk.org/invite and we will let you know as slots become available.

The psiturk tool is always freely available via the python package manager, pip:

pip install psiturk

Further instruction, documentation, hands-on videos, and guides are available here.







  1. [...] preparation for the release of psiTurk 2.0, we sent out a short survey to gauge behavioral researchers’ interest in online data [...]

    thinking about thinking » Online data collection survey results