OpenSesame had slightly less precision across the board, but most notably in audio stimuli and Expyriment had rather poor precision. Among the lab-based experiments, Psychtoolbox, PsychoPy, Presentation and E-Prime provided the best timing, all with mean precision under 1 millisecond across the visual, audio and response measures. Where possible, the packages were tested on Windows, macOS, and Ubuntu, and in a range of browsers for the online studies, to try to identify common patterns in performance.
We compared a range of popular packages: PsychoPy, E-Prime®, NBS Presentation®, Psychophysics Toolbox, OpenSesame, Expyriment, Gorilla, jsPsych, Lab.js and Testable. Here we report a wide-ranging study looking at the precision and accuracy of visual and auditory stimulus timing and response times, measured with a Black Box Toolkit. Very little information is available, however, on what timing performance they achieve in practice. There are a large number of software packages with which to conduct these behavioral experiments and measure response times and performance of participants. Many researchers in the behavioral sciences depend on research software that presents stimuli, and records response times, with sub-millisecond precision.
