___________________________________________________________________________________________________________________
*Multi-Step Decision-Making Task (MSDM)*
___________________________________________________________________________________________________________________
Script Author: Katja Borchert, Ph.D. (katjab@millisecond.com) for Millisecond Software, LLC
Date: 11-30-2021
last updated: 03-13-2024 by K. Borchert (katjab@millisecond.com) for Millisecond Software, LLC
Script Copyright © 03-13-2024 Millisecond Software
___________________________________________________________________________________________________________________
BACKGROUND INFO
___________________________________________________________________________________________________________________
The Multi-Step Decision-Making Task (MSDM) task (e.g. Huys et al, 2012) investigates people's decision making
of multi-step problems within a reinforcement paradigm.
This script implements Millisecond's version of the MSDM task. The implemented Inquisit procedure is
based on Faulkner et al (2021) and is Millisecond's best guess effort of the implemented trainings and tests.
References
Faulkner P, Huys QJM, Renz D, Eshel N, Pilling S, Dayan P, Roiser JP.
A comparison of 'pruning' during multi-step planning in depressed and healthy individuals.
Psychol Med. 2021 Mar 12:1-9.
doi: 10.1017/S0033291721000799.
Epub ahead of print. PMID: 33706833.
Huys QJM, Eshel N, O’Nions E, Sheridan L, Dayan P & Roiser JP (2012). Bonsai Trees in
your head: How the Pavlovian system sculpts goal-directed choices by pruning decision
trees. PLoS Computational Biology, 8(3), 1-13.
___________________________________________________________________________________________________________________
TASK DESCRIPTION
___________________________________________________________________________________________________________________
The MSDM task is a sequential decision making task that requires participants to
employ a 'plan ahead' strategy.
Specifically, participants are given a transition matrix with 6 states in which to move around.
Some of the moves result in gains and some in losses.
The 6 states are interconnected in the following way
- each state can be reached by only 2 other states
Example: state1 can be reached from state5 and state6 only
- once on a state you can move to only two other states using the response keys [U] and [I].
Example: from state1 you can reach state2 (using [U]) and state4 (using [I])
Participants work through 2 training sessions:
Training I:
- learn how to navigate the transition matrix using the keys [U] and [I]
Training II:
- learn how rewards and losses are connected to each transition
Test:
- participants have to plan moves (2-8) around the transition matrix to maximize gains
___________________________________________________________________________________________________________________
DURATION
___________________________________________________________________________________________________________________
the default set-up of the script takes appr. 45 minutes to complete
___________________________________________________________________________________________________________________
DATA FILE INFORMATION
___________________________________________________________________________________________________________________
The default data stored in the data files are:
(1) Raw data file: 'msdm_raw*.iqdat' (a separate file for each participant)*
build: The specific Inquisit version used (the 'build') that was run
computer.platform: the platform the script was run on (win/mac/ios/android)
date, time: date and time script was run
subject, group, with the current subject/groupnumber
session: with the current session id
blockcode, blocknum: the name and number of the current block (built-in Inquisit variable)
trialcode, trialnum: the name and number of the currently recorded trial (built-in Inquisit variable)
Note: trialnum is a built-in Inquisit variable; it counts all trials run; even those
that do not store data to the data file.
runCounter: only for training sessions: keeps track of repeated training(training_test) sessions
testCounter: only for final test session: keeps track of number of test blocks run
trialCounter: counts the number of trials in a given block
startstate: the start state (1-6), from which to start moving around (used during all trainings and tests)
nextstate: the next state that a move results into (if applicable)
endstate: the end state that should be reached (used in 'training1_test2' only)
//test only:
numberMoves: the number of moves that need to be made during the test
maxGain: the maximum gain that could be collected given the start state and the number of moves
selectedGain: the gains collected through the sequence of selected moves
bestMoves: a variable that stores the sequence of I and U moves associated with maxGain
selectedMoves: stores the sequence of I and U responses made by participant
bestStateSequences: a variable that stores the sequential orders of the visited states associated with maxGain given the current start state (contains startstate!)
selectedStateSequence: stores the sequence of states that participants moved through (contains startstate!)
bestStateSequenceSelected: 1 (true): the selectedStateSequence is one of the bestStateSequences
0 (false): the selectedStateSequence is not one of the optimal ones
response: built-in response variable: the participant's response (scancode of response buttons)
responseText: built-in response variable: the letter of the pressed response key
resp: custom response variable
correctResponse: the established correct response for a given trial (if applicable)
correct: accuracy of response: 1 = correct response; 0 = otherwise (if applicable)
latency: the response latency (in ms); measured from: onset of trial
win: reward structure training: shows the gain/loss given a particular move during training2
key: used during training1 (transition training): stores the key that participant is asked to press
(2) Summary data file: 'msdm_summary*.iqdat' (a separate file for each participant)*
inquisit.version: Inquisit version run
computer.platform: the platform the script was run on (win/mac/ios/android)
startDate: date script was run
startTime: time script was started
subjectid: assigned subject id number
groupid: assigned group id number
sessionid: assigned session id number
elapsedTime: time it took to run script (in ms); measured from onset to offset of script
completed: 0 = script was not completed (prematurely aborted);
1 = script was completed (all conditions run)
ACC_statestest: proportion correct responses during the states test
ACC_training1_test1: proportion correct during training1 (test1)
ACC_training1_test2: proportion correct during training1 (test1)
ACC_training2_test: proportion correct during training2 (test)
___________________________________________________________________________________________________________________
EXPERIMENTAL SET-UP
___________________________________________________________________________________________________________________
(I) Training Session I:
A. Introduction of the environment and the 6 states
-> test of state knowledge ('which state is this?')
B. Introduction of the transition matrix with explicit images of the interconnected states
and pathways (see Faulkner et al, 2021).
Note: The implemented trainings and tests are not originals. They are Millisecond's best
guess efforts.
-> implemented Training: move around the transition matrix using keys [I] and [U].
-> implemented Test1: given a start state and a particular key (I or U), which state do you end up on?
-> implemented Test2: given a start state and a particular end state, how could you get there
(enter the appropriate key response sequences)?
Note: can enter any way, not just the fastest
(II) Training Session II:
A. Introduction of the reward states associated with moving from state to state.
Note: The implemented trainings and tests are not originals. They are Millisecond's best
guess efforts.
-> implemented Training: As done by Faulkner et al (2021), participants are given hints in the form of (+) and (-)
under each state but have to learn by trial and error which move (the U or the I move)
is associated with which reward (loss).
-> implemented Test: given a start state and a particular key (I or U), enter the associated gain (loss)
(III) Test: 48 trials total
For the test, participants are told that the goal is to maximize the gains given
a particular start state AND a particular number of moves (2, 4, 6, 8) that need to be made
The Test runs 2 different types of test trials:
a. 'Immediate': participants are given gain/loss-feedback after each U/I response
b. 'Delayed' (plan-ahead): participants enter the planned complete movement sequence (e.g. 'UUII').
Once the movement sequence is entered, the program shows the selected transitions and provides
associated gain/loss-feedback to the participants.
A. 24 trials (12 'immediate' and 12 'delayed' trials, order randomized)
- immediate and delayed trials run 3 repetitions of the number of moves (2, 4, 6, 8) each
- start states are selected randomly without replacement for each trial
Note: this first block of 24 trials is still considered training by Faulkner et al (2021)
B. 24 trials (12 'immediate' and 12 'delayed' trials, order randomized)
- immediate and delayed trials run 3 repetitions of the number of moves (2, 4, 6, 8) each
- start states are selected randomly without replacement for each trial
Note:
Faulkner et al (2021) report that they tested 2-8 number of moves.
This script uses the even number of 4 possible number of moves (2, 4, 6, 8) as this number works nicely
with the number 12 (trials per delayed/immediate condition), 24 (number of trials per block) and
48 (total number of trials run).
The script determines for each trial
- the maximum gain possible
- all possible best move sequences that would lead to this maximum gain
- the possible best move sequences WITHOUT any extreme loss paths (--)
- the number of possible best move sequences
- the number of possible best move sequences that contain at least one extreme loss path (--)
- the number of possible best move sequences that do NOT contain any extreme loss path (--)
for each test move sequence made, the script determines:
- the obtained gain
- whether the selected moves contained at least one extreme loss path (--)
- whether the selected moves were optimal
___________________________________________________________________________________________________________________
STIMULI
___________________________________________________________________________________________________________________
provided by Millisecond Software
___________________________________________________________________________________________________________________
INSTRUCTIONS
___________________________________________________________________________________________________________________
provided by Millisecond Software
___________________________________________________________________________________________________________________
EDITABLE CODE
___________________________________________________________________________________________________________________
check below for (relatively) easily editable parameters, stimuli, instructions etc.
Keep in mind that you can use this script as a template and therefore always "mess" with the entire code
to further customize your experiment.
The parameters you can change are:
