Script Author: Katja Borchert, Ph.D. (katjab@millisecond.com), Millisecond
Created: January 14, 2025
Last Modified: January 14, 2025 by K. Borchert (katjab@millisecond.com), Millisecond
Script Copyright © Millisecond Software, LLC
This script implements the Cognitive Reflection Test2 (CRT2), using 4 items suggested by Thomson & Oppenheimer (2016).
The original CRT (Frederick, 2005) is a simple 3-problem test. The correct solution to each problem is easily understood once explained but the problems require the suppression of 'foil' responses that easily come to mind.
8 minutes
Participants see four problems on the same page with response boxes placed next to them. They can select to respond to each problem by pressing on the response box next to each problem. Once the response box is pressed, a textbox is provided to enter the responses.
Participants can correct their responses whenever they want (they also can go back and forth between problems). The computer records any responses participants submits for future analyses.
The script provides the option to run this task as a timed vs. an untimed test. By default, participants have to provide a response to each problem before the submit button becomes activated (if the timelimit has been reached in the timed condition, the task terminates regardless)
trial.problemPage: screen presents all three problems and their response boxes in black as well
as the 'Submit' button (Submit button becomes active once all three problems have a solution)
Participants can select the response box of each problem to activate the problem.
openended.problem1: 'activates' problem1 by highlightening it in blue font (problem2/3/4 turn gray).
An active textbox with a continue button replaces the previous responsebox next to problem1.
Participants can enter their response and change it as often as they want. Once they
press the 'continue' button, they submit their response and return to trial.problemPage.
(same for problem2/problem3)
The computer tracks any of the submitted responses made.
For a timed test, the task terminates after the maximum time allowed.
provided by Thomson & Oppenheimer (2016) - can be edited under section Editable Stimuli
provided by Millisecond - can be edited under section Editable Instructions
File Name: cognitivereflectiontest_summary*.iqdat
| Name | Description |
|---|---|
| inquisit.version | Inquisit version number |
| computer.platform | Device platform: win | mac |ios | android |
| startDate | Date the session was run |
| startTime | Time the session was run |
| subjectId | Participant ID |
| groupId | Group number |
| sessionId | Session number |
| elapsedTime | Session duration in ms |
| completed | 0 = Test was not completed 1 = Test was completed |
| runTimedTest | True (1) = the test is timed; false (0) = the test is untimed |
| timeLimit | The time limit for a timed test (in ms) |
| testDurationS | The duration of the test (in seconds) |
| countCorrect | Counts all correct responses (Max: 3) |
| problem1ResponseOrig | The entered current response to problem 1 |
| problem1Response | The potentially cleaned current response to problem 1 (Cleaned = if given response is part of the acceptable correct/foil responses, the cleaned response will hold the correct/foil response instead of the entered response) Example problem1ResponseOrig = 'SECOND' (entered response) problem1Response = 2 (cleaned) |
| problem1CorrectAnswer | The correct answer to problem 1 |
| problem1Correct | 1 = current provided solution is correct, 0 = otherwise |
| problem1FoilAnswer | The foil answer to problem 1 |
| problem1Foil | 1 = the current provided solution is the foil answer; 0 = otherwise |
| problem1FoilDetected | 1 = participant has provided foil answer at some point; 0 = no foil answer detected for problem 1 |
| problem1CountCorrections | Counts the number of corrections made for problem 1 |
| problem1Duration | Sums up all trial.problem1 latencies (in ms) up to this point (same for problem2/problem3/problem4) |
File Name: cognitivereflectiontest_raw*.iqdat
| Name | Description |
|---|---|
| build | Inquisit version number |
| computer.platform | Device platform: win | mac |ios | android |
| date | Date the session was run |
| time | Time the session was run |
| subject | Participant ID |
| group | Group number |
| session | Session number |
| blockCode | Name of the current block |
| blockNum | Number of the current block |
| trialCode | Name of the current trial |
| trialNum | Number of the current trial |
| runTimedTest | True (1) = the test is timed; false (0) = the test is untimed |
| timeLimit | The time limit for a timed test (in ms) |
| response | The participant's response (scancode of response buttons) |
| latency | The response latency (in ms) of the current trial; measured from: onset of trial |
| problem1AllResponses | A string variable that collects all submitted responses for problem 1 (will show corrections) * submitted: after pressing 'continue' button |
| problem1ResponseOrig | The entered current response to problem 1 |
| problem1Response | The potentially cleaned current response to problem 1 (Cleaned = if given response is part of the acceptable correct/foil responses, the cleaned response will hold the correct/foil response instead of the entered response) Example problem1ResponseOrig = 'SECOND' (entered response) problem1Response = 2 (cleaned) |
| problem1CorrectAnswer | The correct answer to problem 1 |
| problem1Correct | 1 = current provided solution is correct, 0 = otherwise |
| problem1FoilAnswer | The foil answer to problem 1 |
| problem1Foil | 1 = the current provided solution is the foil answer; 0 = otherwise |
| problem1FoilDetected | 1 = participant has provided foil answer at some point; 0 = no foil answer detected for problem 1 |
| problem1CountCorrections | Counts the number of corrections made for problem 1 |
| problem1Duration | Sums up all trial.problem1 latencies (in ms) up to this point (same for problem2/problem3/problem4) |
The procedure can be adjusted by setting the following parameters.
| Name | Description | Default |
|---|---|---|
| runTimedTest | false //true = the test is timed; false = the test is not timed | |
| timeLimit | 300000 //the time limit (in ms) if parameters.runTimedTest = true (default: 5 minutes) |