Return to the Two-Armed Bandit Task page
Four Armed Bandit Task
Script Author: Katja Borchert, Ph.D. (email@example.com) for Millisecond Software, LLC
last updated: 02-15-2018 by K.Borchert (firstname.lastname@example.org) for Millisecond Software LLC
Script Copyright © 02-15-2018 Millisecond Software
This script implements a 'Four Armed Bandit Task' similarly to the one described in:
Daw, N.D., O'Doherty, J.P., Dayan, P., Seymour, B. & Dolan, R.J. (2006).
Cortical substrates for exploratory decisions in humans.
Nature. 2006 June 15; 441(7095): 876–879.
and Supplemental Materials (Refer to Web version on PubMed Central for supplementary material)
Participants have to choose between four slots that are each tied to different payoffs.
The payoffs of each slot fluctuate from trial to trial. Specifically,
the payoffs for each slot are drawn from a Gaussian distribution (standard deviation = 4)
around mean M, rounded to the nearest integer (enforced range 1-100).
At the beginning of each selection trial, the mean for each slot diffuses in a decaying
Gaussian random walk and the new payoffs are drawn from their respective (updated) distributions (Daw et al, 2006).
Participants are instructed to maximize their payoffs.
DATA FILE INFORMATION:
The default data stored in the data files are:
(1) Raw data file: 'fourarmedbandittask_raw*.iqdat' (a separate file for each participant)*
build: Inquisit build
computer.platform: the platform the script was run on
date, time, subject, group: date and time script was run with the current subject/groupnumber
blockcode, blocknum: the name and number of the current block
trialcode, trialnum: the name and number of the currently recorded trial
(Note: not all trials that are run might record data; by default data is collected unless /recorddata = false is set for a particular trial/block)
/countRounds: counts the number of rounds played
/TotaltrialCount: running count of all test trials run across rounds
/trialCount_perRound: running count of test trials per round
/noResponseCount: running count of all test trials where no selection was made (in time) across rounds
response: the participant's response
trial.selection: the selected slot
latency: the response latency (in ms); measured from:
/selectionRT: the latency (ms) of selecting the current slot; measured from onset of all four 'slots'
/selectedSlot: the selected slot: 1, 2, 3, 4
/choice: 0 = no selection made (timed-out)
1 = exploitative selection made (participant selected the slot with the highest known payoff at this point)
2 = exploratory selection (participant selected an slot that did not have the highest known payoff at this point)
Note: values.choice is determined in 'trial.selection' based on the last known payoffs seen before making the choice
/currenthighestSeenPayOffSlot: the slot that has the currently known highest payoff of the four options
/currenthighestSeenPayOff: the currently known highest payoff
trial.selection stores the value that is known when the selection is made (but before the result has been revealed)
trial.iti stores the potentially updated value that is known at the end of the trial
/lastSeenPayOff1: stores the last seen payOff for 'slot1'
/lastSeenPayOff2: stores the last seen payOff for 'slot2'
/lastSeenPayOff3: stores the last seen payOff for 'slot3'
/lastSeenPayOff4: stores the last seen payOff for 'slot4'
Note: values.currenthighestSeenPayOff is calculated as the highest of these 4 values
Note: all 'lastSeenPayOffs' are updated at the END of each trial sequence
trial.selection: stores the known payOffs at the start of the trial sequence
trial.iti: stores the known payOffs at the end of the trial sequence
/total: the total points won
/currentPayoff: stores the current payoff based on the selected slot
/highestPayOffSelected: 1 = the slot with the currently highest payoff was selected
2 = an slot with lesser payoff was selected
0 = no choice was made
Note: at time of making their choice, participants were not aware of any potential changes in relative
/currentHighestPayOffSlot: the slot with the currently highest payoff (is not necessarily known by player)
/payOff1: current payoff for selecting 'slot1'
/payOff2: current payoff for selecting 'slot2'
/payOff3: current payoff for selecting 'slot3'
/payOff4: current payoff for selecting 'slot4'
/mean: helper variable to calculate the current means for the four slots
/mean1: the current mean payoff for 'slot1'; used to calculate payoff1
/mean2: the current mean payoff for 'slot2'; used to calculate payoff2
/mean3: the current mean payoff for 'slot3'; used to calculate payoff3
/mean4: the current mean payoff for 'slot4'; used to calculate payoff4
(2) Summary data file: 'fourarmedbandittask_summary*.iqdat' (a separate file for each participant)*
script.startdate: date script was run
script.starttime: time script was started
script.subjectid: subject id number
script.groupid: group id number
script.elapsedtime: time it took to run script (in ms)
computer.platform: the platform the script was run on
/completed: 0 = script was not completed (prematurely aborted); 1 = script was completed (all conditions run)
/TotaltrialCount: number of test trials run across rounds
/noResponseCount: number of test trials where no selection was made (in time) across rounds
/propNoResponses: proportion no responses (no choice was made)
/prop_highestPayOff: proportion highest payOff option selected (of all trials in which a choice was made)
/prop_exploitative: proportion exploitative choices (of all trials in which a choice was made)
Note: 'exploitative' selection (in this script): participant selected the slot with the highest known payoff at this point
* separate data files: to change to one data file for all participants (on Inquisit Lab only), go to section
"DATA" and follow further instructions
(1) Demo: 5 trials (runs with different payoff values than test trials)
(2) Test: 2 rounds with 150 trials each; break in between
4 slots: represented by a red, green, blue and yellow box;
slots are selected by mouse click
slot selection (max. 1500ms) -> animated slot (3000ms) -> reveal of selected slot's payoff (1000ms) -> blank screen (2000ms)
(if no slot is selected within 1500ms -> error feedback (4200ms) -> blank screen (2000ms))
PayOffs for each slot calculated at beginning of each trial.selection (with i = 'slots' 1-4):
payOff(i) = round(randgaussian(values.newmean(i), 4)); rounded to the nearest integer
with Max PayOff = 100 and Min PayOff = 1 (see Daw et al, 2006, Supplementary Methods),
additional constraint implemented in this script: the calculated payOffs have to be different from each other
(new payoffs are calculated based on the same means if two of the payoffs end up being the same)
newmean(i) = 0.9836*previousmean(i) + (1-0.9836)*50 + randgaussian(0, 2.8) (see Daw et al, 2006 for further explanation, Supplementary Methods)
* function randgaussian(mean, standarddeviation) samples a value from the normal distribution with the given mean
* initial means of 20, 40, 60, 80 (editable parameters) are assigned randomly to the four 'slots'
provided by Millisecond Software - can be edited under section Editable Stimuli
provided by Millisecond Software (not original to Daw et al, 2006)- can be edited under section Editable Instructions
* main instructions are provided via *.htm files. To edit instructions replace existing htm files or edit
provided ones using simple text editors such as Notepad (win) or TextEdit (Mac)
check below for (relatively) easily editable parameters, stimuli, instructions etc.
Keep in mind that you can use this script as a template and therefore always "mess" with the entire code to further customize
The parameters you can change are:
/skipTotal: true (1): the total points won are NOT presented on screen (default)
false (0): the total points won are presented on screen
/startMean1: the first mean value to use to calculate payoffs for the first trial (default: 20)
/startMean2: the second mean value to use to calculate payoffs for the first trial (default: 40)
/startMean3: the third mean value to use to calculate payoffs for the first trial (default: 60)
/startMean4: the fourth mean value to use to calculate payoffs for the first trial (default: 80)
Note: these means are randomly assigned to 'slot1'-'slot4'
/slotSize: the proportional size of the four 'slots' (proportional to canvas) (default: 40%)
/selectionTimeout: the response timeout (in ms) for making a choice (default: 1500ms)
/timeoutWarningDuration: the duration (in ms) of the red X to signal a no response timeout (default: 4200ms)
/animationDuration: the duration (in ms) of the animated slot (the wait time until result is revealed) (default: 3000ms)
/outcomeDuration: the duration (in ms) of the result (the reveal of the points won) (default: 1000ms)
/iti: the intertrial interval in ms (default: 2000)
Note: in this script, the iti is fixed (compare to: Daw et al, 2006, Supplementary Methods)
/breakDuration: btw. round rest duration (in ms) (default: 60000ms)
/readyDuration: 'get ready' duration (in ms) (default: 2000ms)