___________________________________________________________________________________________________________________ *Cognitive Reflection Test* ___________________________________________________________________________________________________________________ Script Author: Katja Borchert, Ph.D. (katjab@millisecond.com) for Millisecond Software, LLC Date: 05-08-2020 last updated: 02-18-2022 by K. Borchert (katjab@millisecond.com) for Millisecond Software, LLC Script Copyright © 02-18-2022 Millisecond Software ___________________________________________________________________________________________________________________ BACKGROUND INFO ___________________________________________________________________________________________________________________ This script implements the Cognitive Reflection Test (CRT). The CRT is a simple 3-problem test. The correct solution to each problem is easily understood once explained but the problems require the suppression of 'foil' responses that easily come to mind. Reference Frederick, S. (2005). Cognitive Reflection and Decision Making, The Journal of Economic Perspectives, 19, pp. 25-42. ___________________________________________________________________________________________________________________ TASK DESCRIPTION ___________________________________________________________________________________________________________________ Participants see three problems on the same page with response boxes placed next to them. They can select to respond to each problem by pressing on the response box next to each problem. Once the response box is pressed, number buttons are provided and participants can select their response. Participants can correct their responses whenever they want (they also can go back and forth between problems). The computer records any responses participants make (including pressing the correction button) for future analyses. The script provides the option to run this task as a timed vs. an untimed test. Note: By default, participants have to provide a response to each problem before the submit button becomes activated (if the timelimit has been reached in the timed condition, the task terminates regardless) ___________________________________________________________________________________________________________________ DURATION ___________________________________________________________________________________________________________________ the default set-up of the script takes appr. 8 minutes to complete (depends on how much time participants spend on the problems) ___________________________________________________________________________________________________________________ DATA FILE INFORMATION ___________________________________________________________________________________________________________________ The default data stored in the data files are: (1) Raw data file: 'cognitivereflectiontest_raw*.iqdat' (a separate file for each participant)* build: The specific Inquisit version used (the 'build') that was run computer.platform: the platform the script was run on (win/mac/ios/android) date, time: date and time script was run subject, group: with the current subject/groupnumber session: with the current session id blockcode, blocknum: the name and number of the current block (built-in Inquisit variable) trialcode, trialnum: the name and number of the currently recorded trial (built-in Inquisit variable) Note: trialnum is a built-in Inquisit variable; it counts all trials run; even those that do not store data to the data file. runTimedTest: true (1) = the test is timed; false (0) = the test is untimed timeLimit: the time limit for a timed test (in ms) response: the participant's response (scancode of response buttons) latency: the response latency (in ms) of the current trial; measured from: onset of trial problem1_allResponses: a string variable that collects all button presses made when working on problem 1 (will show corrections) problem1Response: the current response to problem 1 problem1CorrectAnswer: the correct answer to problem 1 problem1Correct: 1 = current provided solution is correct, 0 = otherwise problem1FoilAnswer: the foil answer to problem 1 problem1Foil: 1 = the current provided solution is the foil answer; 0 = otherwise problem1FoilDetected: 1 = participant has provided foil answer at some point; 0 = no foil answer detected for problem 1 problem1_countCorrections: counts the number of corrections made for problem 1 problem1_duration: sums up all trial.problem1 latencies (in ms) up to this point (same for problem2/problem3) (2) Summary data file: 'cognitivereflectiontest_summary*.iqdat' (a separate file for each participant)* inquisit.version: Inquisit version run computer.platform: the platform the script was run on (win/mac/ios/android) startDate: date script was run startTime: time script was started subjectid: assigned subject id number groupid: assigned group id number sessionid: assigned session id number elapsedTime: time it took to run script (in ms); measured from onset to offset of script completed: 0 = script was not completed (prematurely aborted); 1 = script was completed (all conditions run) runTimedTest: true (1) = the test is timed; false (0) = the test is untimed timeLimit: the time limit for a timed test (in ms) testDuration_s: the duration of the test (in seconds) countCorrect: counts all correct responses (Max: 3) problem1_allResponses: a string variable that collects all button presses made when working on problem 1 (will show corrections) Example: |10|done|5|correction|5|done| => first time participant responded to problem 1, participant typed a 1 and a 0, then clicked 'done' => second time participant responded to problem 1, participant added a 5 to current response (it appeared as 105) then clicked correction and then typed a single 5 again and clicked 'done' problem1Response: the final response to problem 1 problem1CorrectAnswer: the correct answer to problem 1 problem1Correct: 1 = final provided solution is correct, 0 = otherwise problem1FoilAnswer: the foil answer to problem 1 problem1Foil: 1 = the final provided solution is the foil answer; 0 = otherwise problem1FoilDetected: 1 = participant has provided foil answer at some point; 0 = no foil answer detected for problem 1 Note: this variable is conservative in detecting foil responses. It might miss subtle Foil Responses such as: |105|correction|5|done => participant first typed 10, then might have changed his mind and typed 5 but then noticed that the 5 was just added to the 10 (appearing as 105). Thus, participant hit correction button and re-typed a single 5 and was done. The current algorithm to detect the foil looks for |10| sequences in 'problem1_all Responses' and thus would have missed the subtle foil response. problem1_countCorrections: counts the number of corrections made for problem 1 problem1_duration: sums up all trial.problem1 latencies (in ms) (same for problem2/problem3) * separate data files: to change to one data file for all participants (on Inquisit Lab only), go to section "DATA" and follow further instructions ___________________________________________________________________________________________________________________ EXPERIMENTAL SET-UP ___________________________________________________________________________________________________________________ trial.problempage: screen presents all three problems and their response boxes in black as well as the 'Submit' button (Submit button becomes active once all three problems have a solution) Participants can select the response box of each problem to activate the problem. trial.problem1: 'activates' problem1 by highlightening it in blue font (problem2/3 turn gray) Response number buttons (0-9) as well as a 'correction' and 'done' button are provided. Participants can select any number button, the correction button (in this case the previous response gets erased) or the 'done' button. Once the 'done' button is pressed, participants return to trial.problempage. (same for trial.problem2/trial.problem3) The computer tracks any of the responses made. For a timed test, the task terminates after the maximum time allowed. ___________________________________________________________________________________________________________________ STIMULI ___________________________________________________________________________________________________________________ provided by Millisecond Software - can be edited under section Editable Stimuli ___________________________________________________________________________________________________________________ INSTRUCTIONS ___________________________________________________________________________________________________________________ provided by Millisecond Software - can be edited under section Editable Instructions ___________________________________________________________________________________________________________________ EDITABLE CODE ___________________________________________________________________________________________________________________ check below for (relatively) easily editable parameters, stimuli, instructions etc. Keep in mind that you can use this script as a template and therefore always "mess" with the entire code to further customize your experiment. The parameters you can change are (see below):