How bad is your experiment computer at measuring reaction time? A simple hack to find out.

This is something that former lab programmer Louis Tur and I worked out a couple years ago before we had a blog. I was digging through some old files and I thought I’d share because I’ve had never come across a similarly cheap/simple way to quantify RT measurement error on a experiment computer.

The time it takes for a person to make a decision following the presentation of a stimulus (or reaction time) is a very common measure in behavioral psychology. I suppose decades ago reaction time was a very complicated thing to measure. Fortunately, the internal clocks on modern computers make it pretty easy to time-stamp key presses with millisecond resolution. However, the time recorded by a standard computer is always subject to variable measurement error.

At any given point while a computer is running, the CPU is receiving data from many sources: the video card, peripheral devices (mouse, keyboard and other USB devices), the hard drive, etc. The result is that data can be sent to the CPU but it is not processed until a previous request is fulfilled (this is handled using what are known as interrupts). It’s a bit like a traffic jam (loosely speaking). Many factors can effect the probability and duration of these traffic jams including the speed of the computer, the type and number of peripheral devices attached to the computer, the software that is currently running, etc… The end result is that even if you press a key on a keyboard, the computer may not process it right away, and that delay is “error” in your measurement.

Some psychology experiments require reaction times to be accurate to the millisecond. Note that it is usually a mistake to worry about this when a mean RT or mean-of-median RT is being computed per subject since the between-subject error is usually much larger than the device error and these two things sum. However, some analyses, like spectral analyses of RT or inferences based on distributional properties of RT distributions, may be more sensitive. Thus, it would be very useful to know how much the error from the computer contributes to measurements. This is also interesting to know because some researchers spend a lot of extra money on fancy dedicated reaction time measuring electronics. Are they really worth it?

The only way to know for sure it try to measure the noise on your experiment testing computer!
Here’s a simple and cheap way to do it. For this setup you will need:

  1. A contact microphone ( used to have them available for purchase but apparently now they just give you instructions on how to build your own. Other online shops make them as well to save you time/hassle.)
  2. Sound editing software (we recommend Audacity open source and free)
  3. Two computers… first, a PC to run the test program (experimental PC) and a PC to record the audio data (recording PC)

The idea of the set up is summarized in this picture:

Basically, the contact microphone is connected to the keyboard of the computer you use to run experiments. The mic is plugged into the audio jack on a separate computer, perhaps a laptop. A contact microphone picks up vibrations coming through solid materials (e.g., wooden tabletops) but not through the air. Thus, they are not very sensitive to ambient noise in a room. However, if you stick a contact mic on a table and gently touch the table with your finger, the contact mic will register a relatively robust sound wave.

The idea here is to record an audio file on the laptop using the contact microphone while you perform a simple tapping task on the experiment computer’s keyboard. The mic will register when your fingers hit the keyboard and not much else (if you are careful not to touch the table with your palms, etc…). The sound card on your computer usually samples sound at at least 44.1KHz, meaning that there are about 44 samples recorded per millisecond. This sub-millisecond resolution is pretty great for getting accurate keystroke time measurements. In addition, incoming sound data to the audio card on your laptop or computer is buffered in a small amount of on-board memory. This buffering allows for uninterrupted flow of sound data (i.e., it is not influenced by interrupts or other software that might be running on the laptop computer).

After you set this up you can measure the time between key presses estimated by the experiment computer and compare those to the onsets in the sound waves (which are pretty obvious to pick out because the contact mic records silence punctuated by a really loud “clang” when your finger touches the keyboard). It helps to press the keys kind of “vigorously.” The time from the onset of one “clang” to the next should line up (to the nearest millisecond) to the time recorded by your experiment software. If not, too bad, but at least you can quantify how badly your particular experiment setup is performing.

The only problem here is making the RT measurements on a sound file. You could spend hours carefully making the measurement in a sound editing software package converting samples to time using the sampling rate. But don’t you worry… we wrote a little python script that can process an uncompressed WAV file and will spit out the time between loud “click” events and will save these values to a text file for further analysis! You can download our scripts here (check which has the code for comparing data collected from an experiment to a WAV file… you might have to hack it a bit for your own use).

Problem solved!

  1. If your stimuli are auditory, you could even use this to measure RTs very accurately during the experiment itself :O) I actually don’t think you need a contact mic specifically; if set up properly, a regular mic should be sufficient.

    Mike Lawrence
  2. Yeah, the contact mic just meant less signal processing of the recorded .WAV file since it is silence with loud “pops.” I agree may not be necessary.

    Todd Gureckis
  3. Thanks Todd, this seems like a really useful procedure.

    Jake Westfall
  4. Hi Todd,

    If this is an auditory experiment, could we just have participants tap on the mike itself, and record during that time? Does recording wav files slow down the stimuli presentation?

    Thanks heaps!

    Li-Ann Leow
  5. Li-Ann: Yeah that could definitely work. Notice the setup in the figure… a separate computer/laptop is used for recording the sound, so it shouldn’t effect stimulus presentation on the main computer.

    Todd Gureckis
  6. Great idea, this is exactly what I’m trying to do. One question: you say we can “measure the time between key presses estimated by the experiment computer”. If I don’t have (somewhat expensive) software like DirectRT, can I still do this? Is there any relatively cheap software out there that will measure keystroke times?

  7. Seth, yeah there is code mentioned in the post that will help you do this for free. It requires a bit of knowledge of python programming but shouldn’t be too difficult.

    Todd Gureckis