Return to the Cognitive Reflection Test page
___________________________________________________________________________________________________________________	

								*Cognitive Reflection Test*
___________________________________________________________________________________________________________________	

Script Author: Katja Borchert, Ph.D. (katjab@millisecond.com) for Millisecond Software, LLC
Date: 05-08-2020
last updated:  05-08-2020 by K. Borchert (katjab@millisecond.com) for Millisecond Software, LLC

Script Copyright © 05-08-2020 Millisecond Software

___________________________________________________________________________________________________________________
BACKGROUND INFO 	
___________________________________________________________________________________________________________________	

This script implements the Cognitive Reflection Test (CRT).

The CRT is a simple 3-problem test. The correct solution to each problem is easily understood once 
explained but the problems require the suppression of 'foil' responses that easily come to mind.

Reference

Frederick, S. (2005). Cognitive Reflection and Decision Making, 
The Journal of Economic Perspectives, 19, pp. 25-42.

___________________________________________________________________________________________________________________
TASK DESCRIPTION	
___________________________________________________________________________________________________________________	

Participants see three problems on the same page with response boxes placed next to them.
They can select to respond to each problem by pressing on the response box next to each problem.
Once the response box is pressed, number buttons are provided and participants can select their
response.

Participants can correct their responses whenever they want (they also can go back and forth between problems).
The computer records any responses participants make (including pressing the correction button) 
for future analyses.

The script provides the option to run this task as a timed vs. an untimed test.
Note: By default, participants have to provide a response to each problem before the 
submit button becomes activated
(if the timelimit has been reached in the timed condition, the task terminates regardless)

___________________________________________________________________________________________________________________	
DURATION 
___________________________________________________________________________________________________________________	
the default set-up of the script takes appr. 8 minutes to complete
(depends on how much time participants spend on the problems)
___________________________________________________________________________________________________________________	
DATA FILE INFORMATION 
___________________________________________________________________________________________________________________		
The default data stored in the data files are:

(1) Raw data file: 'cognitivereflectiontest_raw*.iqdat' (a separate file for each participant)*


build:								The specific Inquisit version used (the 'build') that was run
computer.platform:					the platform the script was run on (win/mac/ios/android)
date, time, 						date and time script was run 
subject, group, 					with the current subject/groupnumber
script.sessionid:					with the current session id

blockcode, blocknum:				the name and number of the current block (built-in Inquisit variable)
trialcode, trialnum: 				the name and number of the currently recorded trial (built-in Inquisit variable)
										Note: trialnum is a built-in Inquisit variable; it counts all trials run; even those
										that do not store data to the data file. 
										
parameters.runTimedTest:			true (1) = the test is timed; false (0) = the test is untimed 
parameters.timeLimit:				the time limit for a timed test (in ms)										
																				
response:							the participant's response (scancode of response buttons)
latency: 							the response latency (in ms) of the current trial; measured from: onset of trial 

problem1_allResponses:				a string variable that collects all button presses made when working on problem 1 (will show corrections) 
problem1Response:					the current response to problem 1

problem1CorrectAnswer:				the correct answer to problem 1 
problem1Correct:					1 = current provided solution is correct, 0 = otherwise 

problem1FoilAnswer:					the foil answer to problem 1
problem1Foil:						1 = the current provided solution is the foil answer; 0 = otherwise
problem1FoilDetected:				1 = participant has provided foil answer at some point; 0 = no foil answer detected for problem 1
problem1_countCorrections:			counts the number of corrections made for problem 1

problem1_duration:					sums up all trial.problem1 latencies (in ms) up to this point

(same for problem2/problem3)


(2) Summary data file: 'cognitivereflectiontest_summary*.iqdat' (a separate file for each participant)*

computer.platform:					the platform the script was run on (win/mac/ios/android)
inquisit.build:						the version/build of Inquisit
script.startdate:					date script was run
script.starttime:					time script was started
script.subjectid:					assigned subject id number
script.groupid:						assigned group id number
script.sessionid:					assigned session id number
script.elapsedtime:					time it took to run script (in ms); measured from onset to offset of script
script.completed:					0 = script was not completed (prematurely aborted); 
									1 = script was completed (all conditions run)
									
parameters.runTimedTest:			true (1) = the test is timed; false (0) = the test is untimed 
parameters.timeLimit:				the time limit for a timed test (in ms)	

testDuration_s:						the duration of the test (in seconds)									
									
countCorrect:						counts all correct responses (Max: 3)
									
problem1_allResponses:				a string variable that collects all button presses made when working on problem 1 (will show corrections) 
									Example: |10|done|5|correction|5|done|
									=> first time participant responded to problem 1, participant typed a 1 and a 0, then clicked 'done'
									=> second time participant responded to problem 1, participant added a 5 to current response (it appeared as 105)
									then clicked correction and then typed a single 5 again and clicked 'done'

problem1Response:					the final response to problem 1

problem1CorrectAnswer:				the correct answer to problem 1 
problem1Correct:					1 = final provided solution is correct, 0 = otherwise 

problem1FoilAnswer:					the foil answer to problem 1
problem1Foil:						1 = the final provided solution is the foil answer; 0 = otherwise
problem1FoilDetected:				1 = participant has provided foil answer at some point; 0 = no foil answer detected for problem 1
									Note: this variable is conservative in detecting foil responses. It might miss subtle Foil Responses 
									such as: |105|correction|5|done 
									=> participant first typed 10, then might have changed his mind and typed 5
									but then noticed that the 5 was just added to the 10 (appearing as 105). Thus, participant hit
									correction button and re-typed a single 5 and was done.
									The current algorithm to detect the foil looks for |10| sequences in 'problem1_all Responses'
									and thus would have missed the subtle foil response.


problem1_countCorrections:			counts the number of corrections made for problem 1

problem1_duration:					sums up all trial.problem1 latencies (in ms)

(same for problem2/problem3)
																								
							
* separate data files: to change to one data file for all participants (on Inquisit Lab only), go to section
"DATA" and follow further instructions

___________________________________________________________________________________________________________________	
EXPERIMENTAL SET-UP 
___________________________________________________________________________________________________________________	

trial.problempage: screen presents all three problems and their response boxes in black as well 
as the 'Submit' button (Submit button becomes active once all three problems have a solution)
Participants can select the response box of each problem to activate the problem.

trial.problem1: 'activates' problem1 by highlightening it in blue font (problem2/3 turn gray)
Response number buttons (0-9) as well as a 'correction' and 'done' button are provided.
Participants can select any number button, the correction button (in this case the previous
response gets erased) or the 'done' button. Once the 'done' button is pressed,
participants return to trial.problempage.
(same for trial.problem2/trial.problem3)

The computer tracks any of the responses made.
For a timed test, the task terminates after the maximum time allowed. 

___________________________________________________________________________________________________________________
STIMULI
___________________________________________________________________________________________________________________
provided by Millisecond Software - can be edited under section Editable Stimuli

___________________________________________________________________________________________________________________	
INSTRUCTIONS 
___________________________________________________________________________________________________________________	
provided by Millisecond Software - can be edited under section Editable Instructions

___________________________________________________________________________________________________________________	
EDITABLE CODE 
___________________________________________________________________________________________________________________	
check below for (relatively) easily editable parameters, stimuli, instructions etc. 
Keep in mind that you can use this script as a template and therefore always "mess" with the entire code 
to further customize your experiment.

The parameters you can change are (see below):