Script Author: Katja Borchert, Ph.D. (katjab@millisecond.com), Millisecond
Created: January 14, 2018
Last Modified: January 24, 2025 by K. Borchert (katjab@millisecond.com), Millisecond
Script Copyright © Millisecond Software, LLC
This script implements a 'Four Armed Bandit Task'; a paradigm to study the conflict between the opposing demands of gathering new information and exploiting known information to maximize profits in human decision making. The implemented procedure is similar to the one described in Daw et al (2006).
Daw, N.D., O'Doherty, J.P., Dayan, P., Seymour, B. & Dolan, R.J. (2006). Cortical substrates for exploratory decisions in humans. Nature. 2006 June 15; 441(7095): 876–879. and Supplemental Materials (Refer to Web version on PubMed Central for supplementary material)
30 minutes
Participants have to choose between four slots that are each tied to different payoffs. The payoffs of each slot fluctuate from trial to trial. Specifically, the payoffs for each slot are drawn from a Gaussian distribution (standard deviation = 4) around mean M, rounded to the nearest integer (enforced range 1-100). At the beginning of each selection trial, the mean for each slot diffuses in a decaying Gaussian random walk and the new payoffs are drawn from their respective (updated) distributions (Daw et al, 2006). Participants are instructed to maximize their payoffs. Participants use keyboard keys to make their choice.
(1) Demo: 5 trials (runs with different payoff values than test trials)
(2) Test: 2 rounds with 150 trials each; break in between
Trial Set-Up:
4 slots: represented by a red (1), green (2), blue (3) and yellow (4) box;
slots are selected by keyboard press
Trial Sequence:
slot selection (max. 1500ms) -> animated slot (2000ms) -> reveal of selected slot's payoff (1000ms) -> blank screen (1000ms)
(if no slot is selected within 1500ms -> error feedback (4200ms) -> blank screen (1000ms))
PayOff Calculations:
PayOffs for each slot calculated at beginning of each trial.selection (with i = 'slots' 1-4):
payOff(i) = round(randGaussian(values.newmean(i), 4)); rounded to the nearest integer
with Max PayOff = 100 and Min PayOff = 1 (see Daw et al, 2006, Supplementary Methods),
additional constraint implemented in this script: the calculated payOffs have to be different from each other
(new payoffs are calculated based on the same means if two of the payoffs end up being the same)
with:
newmean(i) = 0.9836*previousmean(i) + (1-0.9836)*50 + randGaussian(0, 2.8) (see Daw et al, 2006 for further explanation, Supplementary Methods)
• function randGaussian(mean, standarddeviation) samples a value from the normal distribution with the given mean
and standarddeviation
• initial means of 20, 40, 60, 80 (editable parameters) are assigned randomly to the four 'slots'
provided by Millisecond - can be edited under section Editable Stimuli
provided by Millisecond (not original to Daw et al, 2006)- can be edited under section Editable Instructions
• main instructions are provided via *.htm files. To edit instructions replace existing htm files or edit
provided ones using simple text editors such as Notepad (win) or TextEdit (Mac)
File Name: fourarmedbandittask_summary*.iqdat
| Name | Description |
|---|---|
| inquisit.version | Inquisit version number |
| computer.platform | Device platform: win | mac |ios | android |
| computer.touch | 0 = device has no touchscreen capabilities; 1 = device has touchscreen capabilities |
| computer.hasKeyboard | 0 = no external keyboard detected; 1 = external keyboard detected |
| startDate | Date the session was run |
| startTime | Time the session was run |
| subjectId | Participant ID |
| groupId | Group number |
| sessionId | Session number |
| elapsedTime | Session duration in ms |
| completed | 0 = Test was not completed 1 = Test was completed |
| totalTrialCount | Number of test trials run across rounds |
| noResponseCount | Number of test trials where no selection was made (in time) across rounds |
| propNoResponses | Proportion no responses (no choice was made) |
| propHighestPayOff | Proportion highest payOff option selected (of all trials in which a choice was made) |
| propExploitative | Proportion exploitative choices (of all trials in which a choice was made) 'exploitative' selection (in this script) participant selected the slot with the highest known payoff at this point |
File Name: fourarmedbandittask_raw*.iqdat
| Name | Description |
|---|---|
| build | Inquisit version number |
| computer.platform | Device platform: win | mac |ios | android |
| computer.touch | 0 = device has no touchscreen capabilities; 1 = device has touchscreen capabilities |
| computer.hasKeyboard | 0 = no external keyboard detected; 1 = external keyboard detected |
| date | Date the session was run |
| time | Time the session was run |
| subject | Participant ID |
| group | Group number |
| session | Session number |
| blockcode | The name the current block (built-in Inquisit variable) |
| blocknum | The number of the current block (built-in Inquisit variable) |
| trialcode | The name of the currently recorded trial (built-in Inquisit variable) |
| trialnum | The number of the currently recorded trial (built-in Inquisit variable) trialnum is a built-in Inquisit variable; it counts all trials run even those that do not store data to the data file. |
| countRounds | Counts the number of rounds played |
| totalTrialCount | Running count of all test trials run across rounds |
| trialCountPerRound | Running count of test trials per round |
| noResponseCount | Running count of all test trials where no selection was made (in time) across rounds |
| response | The participant's response => trial.selection: the selected slot ('slot1' (red), 'slot2' (green), 'slot3' (blue), 'slot4' (yellow) ) |
| latency | The response latency (in ms); measured from |
| selectionRT | The latency (ms) of selecting the current slot; measured from onset of all four 'slots' |
| selectedSlot | The selected slot: 1, 2, 3, 4 |
| choice | 0 = no selection made (timed-out) 1 = exploitative selection made (participant selected the slot with the highest known payoff at this point) 2 = exploratory selection (participant selected an slot that did not have the highest known payoff at this point) choice is determined in 'trial.selection' based on the last known payoffs seen before making the choice |
| currentHighestSeenPayOffSlot | The slot that has the currently known highest payoff of the four options |
| currentHighestSeenPayOff | The currently known highest payoff Note trial.selection stores the value that is known when the selection is made (but before the result has been revealed) trial.iti stores the potentially updated value that is known at the end of the trial |
| lastSeenPayOff1 | Stores the last seen payOff for 'slot1' |
| lastSeenPayOff2 | Stores the last seen payOff for 'slot2' |
| lastSeenPayOff3 | Stores the last seen payOff for 'slot3' |
| lastSeenPayOff | Stores the last seen payOff for 'slot4' currenthighestSeenPayOff is calculated as the highest of these 4 values all 'lastSeenPayOffs' are updated at the END of each trial sequence trial.selection: stores the known payOffs at the start of the trial sequence trial.iti: stores the known payOffs at the end of the trial sequence |
| total | The total points won |
| currentPayoff | Stores the current payoff 'paid' based on the selected slot |
| highestPayOffSelected | 1 = the slot with the currently highest payoff was selected 2 = a slot with a lesser payoff was selected 0 = no choice was made at time of making their choice, participants were not aware of any potential changes in relative payoffs. |
| currentHighestPayOffSlot | The slot with the currently highest payoff (is not necessarily known by player) |
| payOff1 | Current payoff for selecting 'slot1' |
| payOff2 | Current payoff for selecting 'slot2' |
| payOff3 | Current payoff for selecting 'slot3' |
| payOff4 | Current payoff for selecting 'slot4' |
| mean | Helper variable to calculate the current means for the four slots |
| mean1 | The current mean payoff for 'slot1'; used to calculate payoff1 |
| mean2 | The current mean payoff for 'slot2'; used to calculate payoff2 |
| mean3 | The current mean payoff for 'slot3'; used to calculate payoff3 |
| mean4 | The current mean payoff for 'slot4'; used to calculate payoff4 |
The procedure can be adjusted by setting the following parameters.
| Name | Description | Default |
|---|---|---|
Design Parameters |
||
| skipTotal | True (1): the total points won are NOT presented on screen (default) false (0): the total points won are presented on screen | true |
| numberOfTestTrialsGame | Number of test trials | 150 |
| countRounds | Number of rounds to play | 2 |
| startMean1 | The first mean value to use to calculate payoffs for the first trial | 20 |
| startMean2 | The second mean value to use to calculate payoffs for the first trial | 40 |
| startMean3 | The third mean value to use to calculate payoffs for the first trial | 60 |
| startMean4 | The fourth mean value to use to calculate payoffs for the first trial these means are randomly assigned to 'slot1'-'slot4' | 80 |
Sizing Parameters |
||
| slotSize | The proportional size of the four 'slots' (proportional to canvas) | 35% |
Timing Parameters |
||
| selectionTimeout | The response timeout (in ms) for making a choice | 1500 |
| timeoutWarningDuration | 4200:the duration (in ms) of the red X to signal a no response timeout (default: 4200ms) | |
| animationDuration | The duration (in ms) of the animated slot (the wait time until result is revealed) | 2000 |
| outcomeDuration | The duration (in ms) of the result (the reveal of the points won) | 1000 |
| iti | The intertrial interval in ms in this script, the iti is fixed (compare to: Daw et al, 2006, Supplementary Methods) | 2000 |
| breakDuration | Btw. round rest duration (in ms) | 60000 |
| readyDuration | 'get ready' duration (in ms) | 2000 |
Reponse Keys |
||
| slot1TopLeftKey | Slot1 key | "E" |
| slot2TopRightKey | Slot2 key | "I" |
| slot3BottomRightKey | Slot3 key | "M" |
| slot4BottomLeftKey | Slot4 key | "C" |