___________________________________________________________________________________________________________________
The Mixed-Trial Implicit Relational Assessment Procedure (MT-IRAP): Template
___________________________________________________________________________________________________________________
Script Author: Katja Borchert, Ph.D. (katjab@millisecond.com) for Millisecond Software, LLC
Date: 08-06-2018
last updated: 03-08-2024 by K. Borchert (katjab@millisecond.com) for Millisecond Software, LLC
Script Copyright © 03-08-2024 Millisecond Software
___________________________________________________________________________________________________________________
BACKGROUND INFO
___________________________________________________________________________________________________________________
The IRAP is a procedure suggested by Barnes-Holmes et al (e.g. 2006, 2010A, 2010B) to measure implicit attitudes.
Participants are asked to quickly categorize pairs of stimuli according to pre-determined rules.
The rules are either hypothesis-consistent (e.g. 'Flowers are pleasant; Insects are unpleasant')
or hypothesis-inconsistent (e.g. 'Insects are pleasant, Flowers are unpleasant').
From one block to the next, the rule flips from being hypothesis-consistent to hypothesis-inconsistent.
Latency difference between hypothesis-consistent and hypothesis-inconsistent blocks are then used to
to assess implicit attitudes.
The mixed trial IRAP (MT-IRAP, Levin et al, 2010) presents consistent and inconsistent trials in the same block
using the cues "truth" or "lie" as 'rule' cues. Participants are given the cue to 'lie' or 'tell the truth'
1s before each pair of stimuli is presented.
Note that instead of using pre-determined rules and thus predetermined error response (see IRAP), the MT-IRAP uses
the participant's response pattern to determine correct and incorrect responses for each stimuli.
Specifically, the MT-IRAP uses the more infrequent response pattern to a stimuli as the error response
(see Levin et al, 2010, p.248). For example, a consistent response pattern would be:
to say 'yes' 100% of the time to 'daffodils-good' under the Truth-condition and
to say 'no' 100% of the time to 'daffodils-good' under the Lie-condition.
For a participant who responds
90% of the time 'yes' to 'daffodils-good' under the Truth-condition and
70% of the time 'yes' to 'daffodils-good' under the Lie-condition, the less frequent response pattern (90% > 70%)
is the one consistent with disagreeing with 'daffodils-good'. In this particular case,
the participant thus made a number of 'errors' (70%) in the 'daffodil-good' Lie-condition and only 10% errors
in the 'daffodils-good' Truth-condition.
MT-IRAP D-scores in this script are calculated as the difference between mean response latencies on
Truth and Lie trials for each type of target and individual target words.
procedure to calculate MT-IRAP scores in this script follows the
guidelines outlined in Levin et al (2010, p.250). Direction of responding is set so that a positive MT-IRAP score
indicates that responses on the Lie trials took longer than the Truth trials*.
This script provides MT-IRAP D-scores for 3 words per target category.
*Note: while it's not explicitly noted by Levin et al (2010), due to the fact that the error response
is not pre-determined in this procedure, positive D-scores simply indicate that lieing
took longer than telling the truth.
Since the participant's personal truth might not align with the 'assumed truth' (e.g. 'flowers-good'),
the interpretation of the D-scores ideally takes into account the participants' individual response patterns.
References:
Barnes-Holmes, D., Barnes-Holmes, Y., Power, P., Hayden, E., Milne, R., & Stewart, I. (2006).
Do you know what you really believe? Developing the Implicit Relational Evaluation Procedure (IRAP) as a direct measure
of implicit beliefs. The Irish Psychologist, 32(7), 169–177.
Barnes-Holmes, D., Barnes-Holmes, Y., Stewart, I., & Boles, S. (2010A). A sketch of the
implicit relational assessment procedure (IRAP) and the relational elaboration
and coherence (REC) model. The Psychological Record, 60(3), 527.
Barnes-Holmes, D., Murtagh, L., Barnes-Holmes, Y., & Stewart, I. (2010B). Using the Implicit Association
Test and the Implicit Relational Assessment Procedure to measure attitudes towards meat and
vegetables in vegetarians and meat-eaters. The Psychological Record, 60, 287-306.
Hussey, I., Mhaoileoin, D. N., Barnes-Holmes, D., Ohtsuki, T., Kishita, N., Hughes, S., & Murphy, C. (2016).
The IRAP Is Nonrelative but notAcontextual: Changes to the Contrast Category Influence Men’s
Dehumanization of Women. The Psychological Record, 66(2), 291– 299.
http://doi.org/10.1007/s40732-016-0171-6
open source IRAP procedure by I. Hussey:
https://contextualscience.org/implicit_relational_assessment_procedure_irap_website.
Levin, M. E., Hayes, S. C., & Waltz, T. (2010). Creating an implicit measure of cognition more suited to applied research:
A test of the Mixed Trial - Implicit Relational Assessment Procedure (MT-IRAP).
International Journal of Behavioral Consultation and Therapy, 6, 245-261.
___________________________________________________________________________________________________________________
TASK DESCRIPTION
___________________________________________________________________________________________________________________
In each trial, participants receive pairs of items such as the word "Tulip" (belonging to the target 'Flower' category)
paired with the word "Good" (belonging to the positive attribute category).
Before each item pair is presented, participants are told to tell the 'truth' or 'lie'.
For example, for Truth-trials a participant would say 'yes' to the item pair 'tulip-good' (assuming
that participant associates flowers with something positive). For Lie-trials, the participant would respond
'no' to the same item pair (for a consistent response pattern).
This script runs 4 trial types under the Truth and Lie condition:
* AA trials: good + flower
* BA trials: bad + flower
* AB trials: good + insect
* BB trials: bad + insect
___________________________________________________________________________________________________________________
DURATION
___________________________________________________________________________________________________________________
default set-up: appr. 15minutes
Practice: appr. 4 minutes
1 Test Round: appr. 5 minutes
=> repeated test rounds will increase the duration accordingly
___________________________________________________________________________________________________________________
DATA FILE INFORMATION
___________________________________________________________________________________________________________________
The default data stored in the data files are:
(1) Raw data file: 'mt_irap_raw*.iqdat' (a separate file for each participant)*
build: The specific Inquisit version used (the 'build') that was run
computer.platform: the platform the script was run on (win/mac/ios/android)
date, time: date and time script was run
subject, group: with the current subject/groupnumber
session: with the current session id
blockcode, blocknum: the name and number of the current block (built-in Inquisit variable)
trialcode, trialnum: the name and number of the currently recorded trial (built-in Inquisit variable)
Note: trialnum is a built-in Inquisit variable; it counts all trials run;
even those that do not store data to the data file such as feedback trials
phase: "practice" vs. "test"
countPractice1Rounds: counts the number of practice1 rounds
countPractice2Rounds: counts the number of practice2 rounds
countAllTestRounds: counts the number of test rounds (one round = two blocks)
repeatBlock: 1 = this block repeats the previous block due to a low consistency measure
0 = not a repeat block
countTestRounds: counts number of test blocks (disregarding repeated ones)
targetCat: FLOWERS vs INSECTS
attributeCat: GOOD vs. BAD
cueCat: Truth vs. Lie
target: the presented target stimulus (e.g. 'daffodil')
attribute: the presented attribute stimulus (here: good vs. bad)
latency: measures response time (in ms) from onset of stimulus pair until response is made
response: the scancode of the response button pressed
18 = E
23 = I
responseText: the label of the responsekey (E or I)
resp: the interpreted response: "Yes" vs. "No"
respA: 1 = yes response; 0 = otherwise
correct: ! Note: this column stores the accuracy of the response based on a pre-determined response pattern
consistent with the hypothesis "flowers are good and insects are bad"
HOWEVER: for summary variables, the participant's response pattern is taken into account to
determine the actual error response (= non-frequent response for a target)
responseOptionA: 'yes' key (18 = E; 23 = I)
responseOptionB: 'no' key (18 = E; 23 = I)
Block-Consisteny Measures:
in general all values that fall within 0.25 and 0.75 are considered inconsistent (with 0.5 ~ random response pattern)
list.A1_consistentResp.mean: porportion of 'yes'-equivalent responses for agreeing with 'daffodils are good'
Note: a participant who always responds 'no' to 'daffodils are good' under the truth-cue
and 'yes' under the 'lie' cue, will end up with a value of 0. However, the response pattern
is very consistent.
list.A2_consistentResp.mean: porportion of 'yes'-equivalent responses for agreeing with 'roses are good'
list.A3_consistentResp.mean: porportion of 'yes'-equivalent responses for agreeing with 'lillys are good'
list.B1_consistentResp.mean: porportion of 'yes'-equivalent responses for agreeing with 'spiders are good'
Note: a participant who always responds 'no' to 'spiders are good' under the truth-cue
and 'yes' under the 'lie' cue, will end up with a value of 0. However, the response pattern
is very consistent.
list.B2_consistentResp.mean: porportion of 'yes'-equivalent responses for agreeing with 'wasps are good'
list.B3_consistentResp.mean: porportion of 'yes'-equivalent responses for agreeing with 'maggots are good'
(2) Summary data file: 'mt_irap_summary*.iqdat' (a separate file for each participant)*
inquisit.version: Inquisit version run
computer.platform: the platform the script was run on (win/mac/ios/android)
startDate: date script was run
startTime: time script was started
subjectid: assigned subject id number
groupid: assigned group id number
sessionid: assigned session id number
elapsedTime: time it took to run script (in ms); measured from onset to offset of script
completed: 0 = script was not completed (prematurely aborted);
1 = script was completed (all conditions run)
The following variables concerns Practice Performance:
countPractice1Rounds: counts the number of practice1 rounds run
practice1pass: 1 = practice1 passed; 0 = otherwise
ACC_practice1: proportion correct during the practice1 block that passed performance criteria
RT_practice1: median correct response time (in ms) during the practice1 block that passed
countPractice2Rounds: counts the number of practice2 rounds run
practice2pass: 1 = practice2 passed; 0 = otherwise
ACC_practice2: proportion correct during the practice2 block that passed performance criteria
RT_practice2: median correct response time (in ms) during the practice2 block that passed
The following variables concerns Test Performance:
Note: this script provides summary statistics for 3 words per target
countAllTestRounds: counts the number of test blocks run (including the blocks that are repeated due to low response consistency)
countTestRounds: counts the number of test blocks run that are used for summary variable calculations (aka the blocks with reasonable response consistency)
Test Response Consistency Measures:
Consistency per word:
measured as the mean of individual block consistency percentages for each target word across ALL test blocks
(regardless of whether the testblock was repeated due to a low consistency measure or not)
In this script, the consistency measure provides directional information:
0% = CONSISTENT ('item is bad') <- 50% = INCONSISTENT -> 100% = CONSISTENT ('item is good')
100%: participant CONSISTENTLY responded to the item as if the statement 'item is good' was the participant's truth (direction towards goodness of item)
50%: participant randomly responded to the item as good or bad (inconsistent responding)
0%: participant CONSISTENTLY responded to the item as if the statement 'item is bad' was the participant's truth (direction towards badness of item)
Levin et al (2010, p.250): "In addition, if a stimulus was not sorted consistently at least 65% of the time it was excluded
from analyses [...]"
meanDirectionalConsistency_A1: mean block consistency percentage for target A1 (here: daffodil)
meanDirectionalConsistency_A2: mean block consistency percentage for target A2 (here: rose)
meanDirectionalConsistency_A3: mean block consistency percentage for target A3 (here: lilly)
meanDirectionalConsistency_B1: mean block consistency percentage for target B1 (here: spider)
meanDirectionalConsistency_B2: mean block consistency percentage for target B2 (here: wasp)
meanDirectionalConsistency_B3: mean block consistency percentage for target B3 (here: maggot)
!NOTE: The remaining summary variables are only calculated based on responses of test blocks that all passed the
consistency criteria for each word (75% consistent):
Error-Analysis: Levin et al (2010, p.248): 'the less frequent response direction is counted as an error'.
The script assesses the direction of the error using the algorithm described under section
Levin et al (2010, p.250) suggest to remove all data of participants with errorrates > 25% (0.25) from
analyses.
overallErrorRate: proportion errors across the test blocks
AA1_ErrorRate: proportion errors across AA test trials presenting stimulus A1 (here: daffodils)
AA2_ErrorRate: proportion errors across AA test trials presenting stimulus A2 (here: roses)
AA3_ErrorRate: proportion errors across AA test trials presenting stimulus A3 (here: lillies)
BA1_ErrorRate: proportion errors across BA test trials presenting stimulus A1 (here: daffodils)
BA2_ErrorRate: proportion errors across BA test trials presenting stimulus A2 (here: roses)
BA3_ErrorRate: proportion errors across BA test trials presenting stimulus A3 (here: lillies)
AB1_ErrorRate: proportion errors across AB test trials presenting stimulus B1 (here: spider)
AB2_ErrorRate: proportion errors across AB test trials presenting stimulus B2 (here: wasp)
AB3_ErrorRate: proportion errors across AB test trials presenting stimulus B3 (here: maggot)
BB1_ErrorRate: proportion errors across BB test trials presenting stimulus B1 (here: spider)
BB2_ErrorRate: proportion errors across BB test trials presenting stimulus B2 (here: wasp)
BB3_ErrorRate: proportion errors across BB test trials presenting stimulus B3 (here: maggot)
Latency Measures:
NOTE: all following latency measures are based on
- correct responses (correct = frequent response pattern for the participant for that particular stimulus)
- latencies <= 10000ms
(see Levin et al, 2010, p.250)
medianCorrTestRT: median response latency (in ms) across all trials
Levin et al (2010, p.250) suggest to remove data of participants with "average response latency above 3 seconds"
D-Scores:
Direction of responding is set so that a positive MT-IRAP score indicates that responses on the Lie trials took longer than the Truth trials*.
*Note: while it's not explicitly noted by Levin et al (2010), due to the fact that the error is not pre-determined in this
procedure, positive D-scores simply indicate that lieing took longer than telling the truth.
Since the participant's personal truth might not align with the conventional truth (e.g. 'flowers-good'),
the interpretation of the D-scores ideally takes into account the participant's direction of errors.
D_MT_IRAP_AA1: D-score for difference in responding to truth vs. lie trials in AA trials (here: good-daffodil)
D_MT_IRAP_AA2: D-score for difference in responding to truth vs. lie trials in AA trials (here: good-rose)
D_MT_IRAP_AA3: D-score for difference in responding to truth vs. lie trials in AA trials (here: good-lilly)
D_MT_IRAP_BA1: D-score for difference in responding to truth vs. lie trials in BA trials (here: bad-daffodil)
D_MT_IRAP_BA2: D-score for difference in responding to truth vs. lie trials in BA trials (here: bad-rose)
D_MT_IRAP_BA3: D-score for difference in responding to truth vs. lie trials in BA trials (here: bad-lilly)
D_MT_IRAP_AB1: D-score for difference in responding to truth vs. lie trials in AB trials (here: good-spider)
D_MT_IRAP_AB2: D-score for difference in responding to truth vs. lie trials in AB trials (here: good-wasp)
D_MT_IRAP_AB3: D-score for difference in responding to truth vs. lie trials in AB trials (here: good-maggot)
D_MT_IRAP_BB1: D-score for difference in responding to truth vs. lie trials in BB trials (here: bad-spider)
D_MT_IRAP_BB2: D-score for difference in responding to truth vs. lie trials in BB trials (here: bad-wasp)
D_MT_IRAP_BB3: D-score for difference in responding to truth vs. lie trials in BB trials (here: bad-maggot)
D_MT_IRAP_A1: mean of the two D-Scores regarding target A1 (here: daffodil), Example: (D_MT_IRAP_AA1 + D_MT_IRAP_BA1)/2
D_MT_IRAP_A2: mean of the two D-Scores regarding target A2 (here: rose), Example: (D_MT_IRAP_AA2 + D_MT_IRAP_BA2)/2
D_MT_IRAP_A3: mean of the two D-Scores regarding target A3 (here: lilly), Example: (D_MT_IRAP_AA3 + D_MT_IRAP_BA3)/2
D_MT_IRAP_B1: mean of the two D-Scores regarding target B1 (here: spider), Example: (D_MT_IRAP_BB1 + D_MT_IRAP_AB1)/2
D_MT_IRAP_B2: mean of the two D-Scores regarding target B2 (here: wasp), Example: (D_MT_IRAP_BB2 + D_MT_IRAP_AB2)/2
D_MT_IRAP_B3: mean of the two D-Scores regarding target B3 (here: maggot), Example: (D_MT_IRAP_BB3 + D_MT_IRAP_AB3)/2
D_MT_IRAP_AA: mean of the three D-Scores for A targets (here: flowers) used in AA trials
D_MT_IRAP_BA: mean of the three D-Scores for A targets (here: flowers) used in BA trials
D_MT_IRAP_AB: mean of the three D-Scores for B targets (here: insects) used in AB trials
D_MT_IRAP_BB: mean of the three D-Scores for B targets (here: insects) used in BB trials
D_MT_IRAP_A: mean of the D-Scores for target A (here: flowers)
D_MT_IRAP_B: mean of the D-Scores for target B (here: insects)
D-MT-IRAP: mean of the two D-Scores for target A and target B
* separate data files: to change to one data file for all participants (on Inquisit Lab only), go to section
"DATA" and follow further instructions
___________________________________________________________________________________________________________________
EXPERIMENTAL SET-UP
___________________________________________________________________________________________________________________
(1) Practice: 2 stages
Practice1: 16 trials with only Truth-Trials (uses 2 items for practice target A (desserts)
and 2 items for practice target B (diseases))
- AA, BB, AB, BA trials; random order (each target word runs twice in its respective truth-trials)
Pass Criteria:
- Proportion Correct >= 80%, median RT <= 2000ms
=> Participants get the chance to repeat practice1 6 times (after failing 6 times, the script terminates)
Practice2: 16 trials with Truth and Lie Trials
- AA, BB, AB, BA trials; random order (each target word runs once in its respective trials)
Pass Criteria:
- Proportion Correct >= 80%, median RT <= 2000ms
=> Participants get the chance to repeat practice1 6 times (after failing 6 times, the script terminates)
9 each of AA/BA/AB/BB-truth trials and 9 each of AA/BA/AB/BB-lie trials (order is randomly determined)
AA = good paired with flowers (here 3 flowers: daffodil, rose, lilly)
AB = good paired with insects (here 3 insects: spider, wasp, maggot)
BA = bad paired with flowers
BB = bad paired with insects
- each 'flower' stim is equally often selected by AA truth, AA lie, BA truth, BA lie (3 times each)
- each 'insect' stim is equally often selected by BB truth, BB lie, AB truth, AB lie (3 times each)
Consistency Criteria:
at the end of each test block, the program checks whether the participant has met the consistency criteria
of 75% consistent responding for each item:
* checks whether the responses to each individual words were at least 75% consistent
(Example: participant consistently responded 'yes' to 'daffodils are good' under truth condition and 'no'
to 'daffodils are good' under lie condition as well as responded 'no' to 'daffodils are bad' under truth condition
and 'yes' to 'daffodils are bad' under lie condition)
* if not all consistency criterium are met, the block gets repeated and the data are not used for summary statistics
* if all consistency criteriums are met, the block terminates and the data are used for summary statistics
Trialsequence:
Instructions (Truth/Lie) and response button assignment for 1000ms
-> Item pair gets added until correct response
-> ITI
Data:
in this script:
- latencies of error responses are removed (error response = infrequent response for a given stimulus)
Note: whether a response is an error response is determined at the end of each block, not on the individual trial level
- latencies > 10000ms are removed
___________________________________________________________________________________________________________________
STIMULI
___________________________________________________________________________________________________________________
provided by Millisecond Software - can be edited under section Editable Stimuli
The default script provides:
practice: 2 dessert words, 2 disease words
test: 3 flower words, 3 insect words
___________________________________________________________________________________________________________________
INSTRUCTIONS
___________________________________________________________________________________________________________________
provided by Millisecond Software - can be edited under section Editable Instructions
(they are not original to Levin et al, 2010)
___________________________________________________________________________________________________________________
EDITABLE CODE
___________________________________________________________________________________________________________________
check below for (relatively) easily editable parameters, stimuli, instructions etc.
Keep in mind that you can use this script as a template and therefore always "mess" with the entire code
to further customize your experiment.
The parameters you can change are:
Experimental Design Parameters:
/maxNumberPractice1Rounds: max number of practice1 rounds (default: 6)
/maxNumberPractice2Rounds: max number of practice1 rounds (default: 6)
/numberOfTestRounds: number of test rounds (default: 2)
/fixedResponseKeyLocation: true (1): the response key assignment is fixed throughout the task
(the assignment is randomly decided at the start of the task) (default)
false (0): the response key assignment is randomly decided for each trial
(in this case participants not only have to make a categorization choice but
also pay attention to the location of the correct response key)
Timing Parameters:
/cueDuration: the duration (in ms) of the trial cue (here: truth vs. lie) before the pair of words appears (default: 1000ms)
/readyDuration: the duration (in ms) of the 'get-ready' trial (default: 3000ms)
/iti: the intertrial interval in ms (default: 400)
/minPracACC: mastery accuracy criterion: the minimum proportion correct that participants have to reach
in each practice phase (practice1 + practice2) before they can move on to the test (default: 0.8)
/maxPracRT: mastery response time criterion: the maximum medium latency (in ms) that is permitted
in each practice phase (practice1 + practice2) before participants might be able to move on to the test (default: 2000ms)
/maxTestRT: The scoring algorithm for expressions.D_MT_IRAP_excludes all test responses with latencies (in ms) > 10000ms
from expressions.D_MT_IRAP_calculations (default: 10000ms)
Sizing Parameters:
/stimSize: proportional sizing (relative to canvas height) of the presented words on the screen (default: 8%)
Response Key Parameters:
/leftkey: the left response key (default: "E")
/rightkey: the right response key (default: "I")
