Script Author: Katja Borchert, Ph.D. (katjab@millisecond.com), Millisecond
Created: January 08, 2020
Last Modified: January 14, 2025 by K. Borchert (katjab@millisecond.com), Millisecond
Script Copyright © Millisecond Software, LLC
This script implements the Cognitive Reflection Test (CRT).
The CRT is a simple 3-problem test. The correct solution to each problem is easily understood once explained but the problems require the suppression of 'foil' responses that easily come to mind.
Frederick, S. (2005). Cognitive Reflection and Decision Making, The Journal of Economic Perspectives, 19, pp. 25-42.
8 minutes
Participants see three problems on the same page with response boxes placed next to them. They can select to respond to each problem by pressing on the response box next to each problem. Once the response box is pressed, number buttons are provided and participants can select their response.
Participants can correct their responses whenever they want (they also can go back and forth between problems). The computer records any responses participants make (including pressing the correction button) for future analyses.
The script provides the option to run this task as a timed vs. an untimed test. By default, participants have to provide a response to each problem before the submit button becomes activated (if the timelimit has been reached in the timed condition, the task terminates regardless)
trial.problemPage: screen presents all three problems and their response boxes in black as well
as the 'Submit' button (Submit button becomes active once all three problems have a solution)
Participants can select the response box of each problem to activate the problem.
trial.problem1: 'activates' problem1 by highlightening it in blue font (problem2/3 turn gray)
Response number buttons (0-9) as well as a 'correction' and 'done' button are provided.
Participants can select any number button, the correction button (in this case the previous
response gets erased) or the 'done' button. Once the 'done' button is pressed,
participants return to trial.problemPage.
(same for trial.problem2/trial.problem3)
The computer tracks any of the responses made.
For a timed test, the task terminates after the maximum time allowed.
provided by Millisecond - can be edited under section Editable Stimuli
provided by Millisecond - can be edited under section Editable Instructions
File Name: cognitivereflectiontest_summary*.iqdat
| Name | Description |
|---|---|
| inquisit.version | Inquisit version number |
| computer.platform | Device platform: win | mac |ios | android |
| startDate | Date the session was run |
| startTime | Time the session was run |
| subjectId | Participant ID |
| groupId | Group number |
| sessionId | Session number |
| elapsedTime | Session duration in ms |
| completed | 0 = Test was not completed 1 = Test was completed |
| rUntimedTest | True (1) = the test is timed; false (0) = the test is untimed |
| timeLimit | The time limit for a timed test (in ms) |
| testDurationS | The duration of the test (in seconds) |
| countCorrect | Counts all correct responses (Max: 3) |
| problem1AllResponses | A string variable that collects all button presses made when working on problem 1 (will show corrections) Example: |10|done|5|correction|5|done| => first time participant responded to problem 1, participant typed a 1 and a 0, then clicked 'done' => second time participant responded to problem 1, participant added a 5 to current response (it appeared as 105) then clicked correction and then typed a single 5 again and clicked 'done' |
| problem1Response | The final response to problem 1 |
| problem1CorrectAnswer | The correct answer to problem 1 |
| problem1Correct | 1 = final provided solution is correct, 0 = otherwise |
| problem1FoilAnswer | The foil answer to problem 1 |
| problem1Foil | 1 = the final provided solution is the foil answer; 0 = otherwise |
| problem1FoilDetected | 1 = participant has provided foil answer at some point; 0 = no foil answer detected for problem 1 this variable is conservative in detecting foil responses. It might miss subtle Foil Responses such as: |105|correction|5|done => participant first typed 10, then might have changed his mind and typed 5 but then noticed that the 5 was just added to the 10 (appearing as 105). Thus, participant hit correction button and re-typed a single 5 and was done. The current algorithm to detect the foil looks for |10| sequences in 'problem1_all Responses' and thus would have missed the subtle foil response. |
| problem1CountCorrections | Counts the number of corrections made for problem 1 |
| problem1Duration | Sums up all trial.problem1 latencies (in ms) (same for problem2/problem3) |
File Name: cognitivereflectiontest_raw*.iqdat
| Name | Description |
|---|---|
| build | Inquisit version number |
| computer.platform | Device platform: win | mac |ios | android |
| date | Date the session was run |
| time | Time the session was run |
| subject | Participant ID |
| group | Group number |
| session | Session number |
| blockCode | Name of the current block |
| blockNum | Number of the current block |
| trialCode | Name of the current trial |
| trialNum | Number of the current trial |
| rUntimedTest | True (1) = the test is timed; false (0) = the test is untimed |
| timeLimit | The time limit for a timed test (in ms) |
| response | The participant's response (scancode of response buttons) |
| latency | The response latency (in ms) of the current trial; measured from: onset of trial |
| problem1AllResponses | A string variable that collects all button presses made when working on problem 1 (will show corrections) |
| problem1Response | The current response to problem 1 |
| problem1CorrectAnswer | The correct answer to problem 1 |
| problem1Correct | 1 = current provided solution is correct, 0 = otherwise |
| problem1FoilAnswer | The foil answer to problem 1 |
| problem1Foil | 1 = the current provided solution is the foil answer; 0 = otherwise |
| problem1FoilDetected | 1 = participant has provided foil answer at some point; 0 = no foil answer detected for problem 1 |
| problem1CountCorrections | Counts the number of corrections made for problem 1 |
| problem1Duration | Sums up all trial.problem1 latencies (in ms) up to this point (same for problem2/problem3) |
The procedure can be adjusted by setting the following parameters.
| Name | Description | Default |
|---|---|---|
| problem1CorrectAnswer | Stores the officially correct response to problem1 | "5" |
| problem2CorrectAnswer | Stores the officially correct response to problem2 | "5" |
| problem3CorrectAnswer | Stores the officially correct response to problem3 | "47" |
| problem1FoilAnswer | Stores the foil response to problem1 | "10" |
| problem2FoilAnswer | Stores the foil response to problem2 | "100" |
| problem3FoilAnswer | Stores the foil response to problem3 | "24" |
| rUntimedTest | True = the test is timed; false = the test is not timed | false |
| timeLimit | The time limit (in ms) if parameters.runTimedTest = true | 300000 |