___________________________________________________________________________________________________________________
The Implicit Relational Assessment Procedure (IRAP): Template
___________________________________________________________________________________________________________________
Script Author: Katja Borchert, Ph.D. (katjab@millisecond.com) for Millisecond Software, LLC
Date: 07-31-2018
last updated: 02-23-2022 by K. Borchert (katjab@millisecond.com) for Millisecond Software, LLC
Script Copyright © 02-23-2022 Millisecond Software
___________________________________________________________________________________________________________________
BACKGROUND INFO
___________________________________________________________________________________________________________________
The IRAP is a procedure suggested by Barnes-Holmes et al (e.g. 2006, 2010A, 2010B) to measure implicit attitudes.
Participants are asked to quickly categorize pairs of stimuli according to pre-determined rules.
The rules are either hypothesis-consistent (e.g. 'Flowers are pleasant; Insects are unpleasant')
or hypothesis-inconsistent (e.g. 'Insects are pleasant, Flowers are unpleasant').
From one block to the next, the rule flips from being hypothesis-consistent to hypothesis-inconsistent.
The IRAP is based on the assumption that participants are faster to categorize item pairs
(e.g. the positive word 'beautiful' paired with the flower 'daisy')
when the categorization rule coincides with the participant's implicit attitudes towards the targets
(e.g. "I like flowers") and slower when the categorization rule works against the participant's implicit attitude.
The difference in mean reaction btw. hypothesis-consistent and hypothesis-inconsistent trials are
computed and captured in several D-IRAP scores which can be used to assess implicit attitudes.
This script uses the D-IRAP algorithms outlined in Barnes-Holmes et al (2010A, p.533, 2010B, p.298).
In general, positive D-IRAP scores suggest implicit attitudes in line with the hypothesis-consistent
categorization rule. Negative D-IRAP scores suggest implicit attitudes in line with the hypothesis-inconsistent
categorization rule.
In contrast to the Implicit Association Task (IAT), the IRAP generates D-IRAP scores separately for
each target category.
This script provides a general IRAP framework to assess implicit attitudes using the target categories
"FLOWERS" and "INSECTS" as examples.
The hypothesis-consistent rule is: 'Flowers are pleasant'-'Insects are unpleasant'.
=> positive D-IRAP scores reflect a hypothesis-consistent bias ('flowers are pleasant, insects are unpleasant')
=> negative D-IRAP scores reflect a hypothesis-inconsistent bias ('insects are pleasant, flowers are unpleasant')
The zero-point reflects no bias.
This script implements The Implicit Relational Assessment Procedure (IRAP) similarly to
the open source IRAP procedure by I. Hussey:
https://contextualscience.org/implicit_relational_assessment_procedure_irap_website.
References:
Barnes-Holmes, D., Barnes-Holmes, Y., Power, P., Hayden, E., Milne, R., & Stewart, I. (2006).
Do you know what you really believe? Developing the Implicit Relational Evaluation Procedure (IRAP) as a direct measure
of implicit beliefs. The Irish Psychologist, 32(7), 169–177.
Barnes-Holmes, D., Barnes-Holmes, Y., Stewart, I., & Boles, S. (2010A). A sketch of the
implicit relational assessment procedure (IRAP) and the relational elaboration
and coherence (REC) model. The Psychological Record, 60(3), 527.
Barnes-Holmes, D., Murtagh, L., Barnes-Holmes, Y., & Stewart, I. (2010B). Using the Implicit Association
Test and the Implicit Relational Assessment Procedure to measure attitudes towards meat and
vegetables in vegetarians and meat-eaters. The Psychological Record, 60, 287-306.
Hussey, I., Mhaoileoin, D. N., Barnes-Holmes, D., Ohtsuki, T., Kishita, N., Hughes, S., & Murphy, C. (2016).
The IRAP Is Nonrelative but notAcontextual: Changes to the Contrast Category Influence Men’s
Dehumanization of Women. The Psychological Record, 66(2), 291– 299.
http://doi.org/10.1007/s40732-016-0171-6
___________________________________________________________________________________________________________________
TASK DESCRIPTION
___________________________________________________________________________________________________________________
Participants are asked to assess the similarity of two items such as the word "Tulip"
(belonging to the target 'Flower' category)
paired with the word "Superb" (belonging to the positive attribute category).
The rule to assess 'similarity' is: 'Flowers are pleasant, insects are unpleasant'.
According to this rule, words belonging to the category 'FLOWER' that are paired with a pleasant word are
'similar' whereasflower words that are paired with unpleasant words are 'different'.
Likewise, insects that are paired with pleasant words
are 'different' and insects that are paired with unpleasant words are 'similar'.
Participants work through several rounds of testing whereas each round consists of 2 blocks:
the first blocks presents the hypothesis-consistent rule, the second rule presents the hypothesis-inconsistent rule
(or vice versa).
This script counterbalances whether the hypothesis consistent rule ('flowers are pleasant, insects are unpleasant')
or the hypothesis inconsistent rule ('flowers are unpleasant, insects are pleasant') is presented first in each round
of testing. Use odd groupnumber to run the order 'consistent -> inconsistent'; use even groupnumbers to run the order
'inconsistent -> consistent' order.
__________________________________________________________________________________________________________________
DURATION
___________________________________________________________________________________________________________________
the default set-up of the script takes appr. ~10-15min (depending on speed and accuracy) to complete
___________________________________________________________________________________________________________________
DATA FILE INFORMATION
___________________________________________________________________________________________________________________
The default data stored in the data files are:
(1) Raw data file: 'irap_raw*.iqdat' (a separate file for each participant)*
build: The specific Inquisit version used (the 'build') that was run
computer.platform: the platform the script was run on (win/mac/ios/android)
date, time: date and time script was run
subject, group, with the current subject/groupnumber
session: with the current session id
blockcode, blocknum: the name and number of the current block (built-in Inquisit variable)
trialcode, trialnum: the name and number of the currently recorded trial (built-in Inquisit variable)
Note: trialnum is a built-in Inquisit variable; it counts all trials run;
even those that do not store data to the data file such as feedback trials
conditionOrder: "c-ic" - 'hypothesis-consistent' rule run before 'hypothesis-inconsistent' rule
"ic-c" - 'hypothesis-inconsistent' rule run before 'hypothesis-consistent' rule
countPracticeRounds: counts the number of practice rounds (one round = two blocks)
countTestRounds: counts the number of test rounds (one round = two blocks)
phase: "practice" vs. "test"
ruleConsistency: 1 = block runs the hypothesis-consistent rule
2 = block runs the hypothesis-inconsistent rule
targetCat: the currently tested target category (here: FLOWERS or INSECTS)
attributeCat: the currently tested attribute category (here: PLEASANT or UNPLEASANT)
stimulusitem.1: the attribute word presented
stimulusitem.2: the target image presented
latency: the response latency (in ms); measured from: onset of words until CORRECT response is made
(!Note: trials only advance with a correct response, so if an error is made a latency-penalty
is automatically added until the response is corrected)
response: scancode of the participant's (last) response button
18 = E
21 = I
corrResp: the correct response in words "similar" vs. "different"
corrKey: stores the key of the currently correct response
correct: accuracy of INITIAL response: 1 = correct initial response; 0 = otherwise
responseOptionA: the response Option A key (here: response option A = 'similar' key)
responseOptionB: the response Option B key (here: response option B = 'different' key)
ACC_practice: proportion correct (based on initial response) during the current practice block
(resets after each practice block)
medianRT_practice: median response latency (in ms) of correct or corrected responses during
the current practice block (resets after each practice block)
practicePass: 1 = practice mastery criteria have been met
0 = practice mastery criteria have not been met
(2) Summary data file: 'irap_images_summary*.iqdat' (a separate file for each participant)*
inquisit.version: Inquisit version run
computer.platform: the platform the script was run on (win/mac/ios/android)
startDate: date script was run
startTime: time script was started
subjectid: assigned subject id number
groupid: assigned group id number
sessionid: assigned session id number
elapsedTime: time it took to run script (in ms); measured from onset to offset of script
completed: 0 = script was not completed (prematurely aborted);
1 = script was completed (all conditions run)
conditionOrder: "c-ic" - 'hypothesis-consistent' rule run before 'hypothesis-inconsistent' rule
"ic-c" - 'hypothesis-inconsistent' rule run before 'hypothesis-consistent' rule
practicePass: 1 = practice mastery criteria have been met
0 = practice mastery criteria have not been met
countPracticeRounds: counts the number of practice rounds (one round = two blocks)
minTestblockACC: tracks the lowest proportion correct (based on initial response) in any test block run
Note: Barnes-Holmes et al (2010A, p.532): "No performance criteria are applied during the test blocks
in order to proceed, but if a participant’s performance falls below the practice accuracy criterion
(e.g., 80%) for any test block, the data for that participant typically are discarded."
propBelowMinRT: proportion of test trials with response latencies < 300ms
Note: Barnes-Holmes et al (2010A, p.533), suggest to eliminate all data of participants
with propBelowMinRT > 0.1 (10%)
Notes:
(1) the D_IRAP scores are calculated based on 3 or fewer test rounds (this script does not automatically
adapt to running more than 3 test rounds - however, it could be edited to do so):
(2) D-IRAP interpretation: in general
positive D-IRAP scores reflect a hypothesis-consistent bias ('flowers are pleasant, insects are unpleasant')
negative D-IRAP scores reflect a hypothesis-inconsistent bias ('insects are pleasant, flowers are unpleasant')
The zero-point reflects no bias.
D_IRAP_AA: D_IRAP score for AA (here: pleasant word-flower) trials - (D_IRAP_AA1 + D_IRAP_AA2 + D_IRAP_AA3)/2
(calculated as the mean of the D-IRAP scores of the individual test rounds 1-3,
automatically adapts if fewer test rounds are run)
positive D_IRAP_AA: on average, participant was faster to correctly categorize trials that paired
'pleasant words with flowers'
if the rule was 'flowers are good' (=similar) than under the rule 'flowers are bad' (=different)
negative D_IRAP_AA: on average, participant was faster to correctly categorize trials that paired
'pleasant words with flowers'
if the rule was 'flowers are bad' (=different) than under the rule 'flowers are good' (=similar)
D_IRAP_AB: D_IRAP score for AB (here: pleasant word-insect) trials - (D_IRAP_AB1 + D_IRAP_AB2 + D_IRAP_AB3)/2
(calculated as the mean of the D-IRAP scores of the individual test rounds 1-3,
automatically adapts if fewer test rounds are run)
positive D_IRAP_AB: on average, participant was faster to correctly categorize trials that paired
'pleasant words with insects'
if the rule was 'insects are bad' (=different) than under the rule 'insects are good' (=similar)
negative D_IRAP_AB: on average, participant was faster to correctly categorize trials that paired
'pleasant words with insects'
if the rule was 'insects are good' (=similar) than under the rule 'insects are bad' (=different)
D_IRAP_BA: D_IRAP score for BA (here: unpleasant word-flower) trials - (D_IRAP_BA1 + D_IRAP_BA2 + D_IRAP_BA3)/2
(calculated as the mean of the D-IRAP scores of the individual test rounds 1-3, automatically adapts if fewer test rounds are run)
positive D_IRAP_BA: on average, participant was faster to correctly categorize trials that paired
'unpleasant words with flowers'
if the rule was 'flowers are good' (=different) than under the rule 'flowers are bad' (=similar)
negative D_IRAP_BA: on average, participant was faster to correctly categorize trials that paired
'unpleasant words with flowers'
if the rule was 'flowers are bad' (=similar) than under the rule 'flowers are good' (=different)
D_IRAP_BB: D_IRAP score for BB (here: unpleasant word-insect) trials- (D_IRAP_BB1 + D_IRAP_BB2 + D_IRAP_BB3)/2
(calculated as the mean of the D-IRAP scores of the individual test rounds 1-3,
automatically adapts if fewer test rounds are run)
positive D_IRAP_BB: on average, participant was faster to correctly categorize trials that paired 'unpleasant words with insects'
if the rule was 'insects are bad' (=similar) than under the rule 'insects are good' (=different)
negative D_IRAP_BB: on average, participant was faster to correctly categorize trials that paired 'unpleasant words with insects'
if the rule was 'insects are good' (=different) than under the rule 'insects are bad' (=similar)
D_IRAP_A: mean of the two IRAP D-scores for target A (here: flowers) - (D_IRAP_AA+D_IRAP_BA)/2
positive D_IRAP_A:
averaged across trials that presented flowers ('pleasant-flower'; 'unpleasant-flower')
participant was faster to correctly categorize the item pairs under the rule
'flowers are good'.
negative D_IRAP_A:
averaged across trials that presented flowers ('pleasant-flower'; 'unpleasant-flower')
participant was faster to correctly categorize the item pairs under the rule
'flowers are bad'.
D_IRAP_B: mean of the two IRAP D-scores for target B (here: insects) - (D_IRAP_AB+D_IRAP_BB)/2
positive D_IRAP_B:
averaged across trials that presented insects ('pleasant-insect'; 'unpleasant-insect')
participant was faster to correctly categorize the item pairs under the rule
'insects are bad'.
negative D_IRAP_B:
averaged across trials that presented insects ('pleasant-insect'; 'unpleasant-insect')
participant was faster to correctly categorize the item pairs under the rule
'insects are good'.
D_IRAP: overall D_IRAP score as the mean of the D_IRAP scores for target A and target B - (D_IRAP_A + D_IRAP_B)/2
positive D_IRAP:
averaged across all trial types participant was faster to correctly categorize the item pairs under the
rule 'flowers are good - insects are bad'
=> bias towards flowers over insects
negative D_IRAP:
averaged across all trial types participant was faster to correctly categorize the item pairs under the
rule 'flowers are bad - insects are good'
=> bias towards insects over flowers
* separate data files: to change to one data file for all participants (on Inquisit Lab only), go to section
"DATA" and follow further instructions
___________________________________________________________________________________________________________________
EXPERIMENTAL SET-UP
___________________________________________________________________________________________________________________
(1) Practice: max. 4 practice rounds (one round = one hypothesis-consistent and one hypothesis-inconsistent block)
- each practice block presents 32 trials: 8 repeats of each of the four trialtypes:
AA = pleasant attributes paired with flowers
AB = pleasant attributes paired with insects
BA = unpleasant attributes paired with flowers
BB = unpleasant attributes paired with insects
- the order of the trials is determined randomly by the computer
- each trial type randomly samples from the pool of available items (sampling = random without replacement).
In this script: there are 4 examplars per each target and attribute category,
thus each trialtype uses each of the four attributes and targets twice, randomly assembling the pair.
- all trials terminate with the correct response (if an error is made a red X
appears on screen until the response is corrected)
- by default, location of the 'similar'/'different' response keys changes randomly
(see Editable Parameters to set it to a fixed location)
Practice End:
- if the practice performance in both blocks of any given round is 80% or higher AND the median response latency in
both blocks is 2000ms or lower (=mastery criteria), practice terminates and the test is started.
- practice rounds are repeated for a maximum of 4 practice rounds IF
- the performance in one or both of the blocks in any given round is below 80% (based on initial response) OR
- the median response latency is greater than 2000ms in one or both of the blocks in any given round
- The script terminates prematurely if the 'mastery criteria' is not reached after 4 practice rounds.
(2) Test: 3 test rounds (one round = one hypothesis-consistent and one hypothesis-inconsistent block)
- each practice block presents 32 trials: 8 repeats of each of the four trialtypes:
AA = pleasant attributes paired with flowers
AB = pleasant attributes paired with insects
BA = unpleasant attributes paired with flowers
BB = unpleasant attributes paired with insects
- the order of the trials is determined randomly by the computer
- each trial type randomly samples from the pool of available items (sampling = random without replacement)
In this script: there are 4 examplars per each target and attribute category,
thus each trialtype uses each of the four attributes and targets twice, randomly assembling the pair.
- all trials terminate with the correct response (if an error is made a red X appears on screen until the response is corrected)
- by default, location of the 'similar'/'different' response keys changes randomly (see Editable Parameters to set it to a fixed location)
- Barnes-Holmes et al (2010A, p.532): "No performance criteria are applied during the test blocks in order to proceed, but if
a participant’s performance falls below the practice accuracy criterion (e.g., 80%) for any test block, the data for that participant typically are discarded."
-> this script tracks the lowest test performance of each participant by values.minTestblockACC (see summary data file)
Trialsequence:
Item pair until correct response -> ITI
Note: the order of consistent-inconsistent rules are counterbalanced by groupnumbers
odd groupnumbers: start with hypothesis-consistent rule (here: flowers are pleasant, insects are unpleasant)
even groupnumbers: start with hypothesis-inconsistent rule (here: flowers are unpleasant, insects are pleasant)
___________________________________________________________________________________________________________________
STIMULI
___________________________________________________________________________________________________________________
provided by Millisecond Software - can be edited under section Editable Stimuli
The default script provides:
4 flower words, 4 insect words, 4 positive attribute words, 4 negative attribute words
___________________________________________________________________________________________________________________
INSTRUCTIONS
___________________________________________________________________________________________________________________
provided by Millisecond Software - can be edited under section Editable Instructions
(Note: some of the instructions are modelled on the open source IRAP procedure by I. Hussey)
___________________________________________________________________________________________________________________
EDITABLE CODE
___________________________________________________________________________________________________________________
check below for (relatively) easily editable parameters, stimuli, instructions etc.
Keep in mind that you can use this script as a template and therefore always "mess" with the entire code
to further customize your experiment.
The parameters you can change are:
Experimental Design Parameters:
/maxNumberPracticeRounds: max number of practice rounds (default: 4)
Notes:
a) one round = one hypothesis-consistent and one hypothesis-inconsistent block
b) Participants have to get 80% correct (see parameters.minPracACC) in each block of a practice round as well as
as perform with a median latency <= 2000ms (see parameters.maxPracRT) in each block of a practice round to
advance to the test.
/numberOfTestRounds: number of test rounds run (default: 3)
Notes:
a) one round = one hypothesis-consistent and one hypothesis-inconsistent block
b) D-IRAP scores in this script are calculated for 3 or fewer test rounds
/fixedResponseKeyLocation: true (1): the response key assignment is fixed throughout the task
(the assignment is randomly decided at the start of the task) (default)
false (0): the response key assignment is randomly decided for each trial
(in this case participants not only have to make a categorization choice but
also pay attention to the location of the correct response key)
Timing Parameters:
/readyDuration: the duration (in ms) of the 'get-ready' trial (default: 3000ms)
/iti: the intertrial interval in ms (default: 400)
/minPracACC: mastery accuracy criterion: the minimum proportion correct that participants have to reach
in each of the two blocks of a practice round before they can move on to the test (default: 0.8)
/maxPracRT: mastery response time criterion: the maximum medium latency (in ms) that is permitted
in each of the two blocks of a round before participants might be able to move on to the test (default: 2000ms)
/minTestRT: this script tracks the number of test responses with latencies smaller than minTestRT (in ms)
and stores the proportion (expressions.propBelowMinRT) in the summary data file.
The scoring algorithm for D-IRAP (e.g. Barnes-Holmes et al, 2010, suggests to exclude all
participants from further analyses whose expressions.propBelowMinRT > 0.1 - or 10% )
(default: 300ms)
/maxTestRT: The scoring algorithm for D-IRAP excludes all test responses with latencies (in ms) > 10000ms
from D-IRAP calculations (default: 10000ms)
Sizing Parameters:
/stimSize: proportional sizing (relative to canvas height) of the presented words on the screen (default: 8%)
Response Key Parameters:
/leftkey: the left response key (default: "E")
/rightkey: the right response key (default: "I")
