Return to the Mixed-Trial Implicit Relational Assessment Procedure (MT-IRAP) page
___________________________________________________________________________________________________________________	

						The Mixed-Trial Implicit Relational Assessment Procedure (MT-IRAP): Template
___________________________________________________________________________________________________________________

Script Author: Katja Borchert, Ph.D. (katjab@millisecond.com) for Millisecond Software, LLC
Date: 08-06-2018
last updated:  07-22-2020 by K. Borchert (katjab@millisecond.com) for Millisecond Software, LLC

Script Copyright © 07-22-2020 Millisecond Software

___________________________________________________________________________________________________________________
BACKGROUND INFO 	
___________________________________________________________________________________________________________________

The IRAP is a procedure suggested by Barnes-Holmes et al (e.g. 2006, 2010A, 2010B) to measure implicit attitudes.
Participants are asked to quickly categorize pairs of stimuli according to pre-determined rules. 
The rules are either hypothesis-consistent (e.g. 'Flowers are pleasant; Insects are unpleasant')
or hypothesis-inconsistent (e.g. 'Insects are pleasant, Flowers are unpleasant').
From one block to the next, the rule flips from being hypothesis-consistent to hypothesis-inconsistent.
Latency difference between hypothesis-consistent and hypothesis-inconsistent blocks are then used to 
to assess implicit attitudes.

The mixed trial IRAP (MT-IRAP, Levin et al, 2010) presents consistent and inconsistent trials in the same block
using the cues "truth" or "lie" as 'rule' cues. Participants are given the cue to 'lie' or 'tell the truth' 
1s before each pair of stimuli is presented.

Note that instead of using pre-determined rules and thus predetermined error response (see IRAP), the MT-IRAP uses 
the participant's response pattern to determine correct and incorrect responses for each stimuli.
Specifically, the MT-IRAP uses the more infrequent response pattern to a stimuli as the error response
(see Levin et al, 2010, p.248). For example, a consistent response pattern would be:
to say 'yes' 100% of the time to 'daffodils-good' under the Truth-condition and 
to say 'no' 100% of the time to 'daffodils-good' under the Lie-condition.
For a participant who responds 
90% of the time 'yes' to 'daffodils-good' under the Truth-condition and 
70% of the time 'yes' to 'daffodils-good' under the Lie-condition, the less frequent response pattern (90% > 70%)
is the one consistent with disagreeing with 'daffodils-good'. In this particular case, 
the participant thus made a number of  'errors' (70%) in the 'daffodil-good' Lie-condition and only 10% errors 
in the 'daffodils-good' Truth-condition.

MT-IRAP D-scores in this script are calculated as the difference between mean response latencies on 
Truth and Lie trials for  each type of target and individual target words.  
 procedure to calculate MT-IRAP scores in this script follows the 
guidelines outlined in Levin et al (2010, p.250). Direction of responding is set so that a positive MT-IRAP score  
indicates that responses on the Lie trials took longer than the Truth trials*.

This script provides MT-IRAP D-scores for 3 words per target category.

*Note: while it's not explicitly noted by Levin et al (2010), due to the fact that the error response 
is not pre-determined in this procedure, positive D-scores simply indicate that lieing 
took longer than telling the truth.
Since the participant's personal truth might not align with the 'assumed truth' (e.g. 'flowers-good'), 
the interpretation of the D-scores ideally takes into account the participants' individual response patterns.

References:

Barnes-Holmes, D., Barnes-Holmes, Y., Power, P., Hayden, E., Milne, R., & Stewart, I. (2006). 
Do you know what you really believe? Developing the Implicit Relational Evaluation Procedure (IRAP) as a direct measure
of implicit beliefs. The Irish Psychologist, 32(7), 169–177.

Barnes-Holmes, D., Barnes-Holmes, Y., Stewart, I., & Boles, S. (2010A). A sketch of the
implicit relational assessment procedure (IRAP) and the relational elaboration
and coherence (REC) model. The Psychological Record, 60(3), 527.

Barnes-Holmes, D., Murtagh, L., Barnes-Holmes, Y., & Stewart, I. (2010B). Using the Implicit Association
Test and the Implicit Relational Assessment Procedure to measure attitudes towards meat and
vegetables in vegetarians and meat-eaters. The Psychological Record, 60, 287-306.

Hussey, I., Mhaoileoin, D. N., Barnes-Holmes, D., Ohtsuki, T., Kishita, N., Hughes, S., & Murphy, C. (2016). 
The IRAP Is Nonrelative but notAcontextual: Changes to the Contrast Category Influence Men’s
Dehumanization of Women. The Psychological Record, 66(2), 291– 299. 
http://doi.org/10.1007/s40732-016-0171-6
open source IRAP procedure by I. Hussey:
https://contextualscience.org/implicit_relational_assessment_procedure_irap_website.

Levin, M. E., Hayes, S. C., & Waltz, T. (2010). Creating an implicit measure of cognition more suited to applied research: 
A test of the Mixed Trial - Implicit Relational Assessment Procedure (MT-IRAP). 
International Journal of Behavioral Consultation and Therapy, 6, 245-261. 


___________________________________________________________________________________________________________________
TASK DESCRIPTION	
___________________________________________________________________________________________________________________	

In each trial, participants receive pairs of items such as the word "Tulip" (belonging to the target 'Flower' category)
paired with the word "Good" (belonging to the positive attribute category).
Before each item pair is presented, participants are told to tell the 'truth' or 'lie'.
For example, for Truth-trials a participant would say 'yes' to the item pair 'tulip-good' (assuming 
that participant associates flowers with something positive). For Lie-trials, the participant would respond
'no' to the same item pair (for a consistent response pattern).
This script runs 4 trial types under the Truth and Lie condition:
* AA trials: good + flower
* BA trials: bad + flower
* AB trials: good + insect
* BB trials: bad + insect

___________________________________________________________________________________________________________________	
DURATION 
___________________________________________________________________________________________________________________	
default set-up: appr. 15minutes
Practice: appr. 4 minutes
1 Test Round: appr. 5 minutes
=> repeated test rounds will increase the duration accordingly

___________________________________________________________________________________________________________________	
DATA FILE INFORMATION 
___________________________________________________________________________________________________________________	 
The default data stored in the data files are:

(1) Raw data file: 'mt_irap_raw*.iqdat' (a separate file for each participant)*

build:								The specific Inquisit version used (the 'build') that was run
computer.platform:					the platform the script was run on (win/mac/ios/android)
date, time, 						date and time script was run 
subject, group, 					with the current subject/groupnumber
script.sessionid:					with the current session id

blockcode, blocknum:				the name and number of the current block (built-in Inquisit variable)
trialcode, trialnum: 				the name and number of the currently recorded trial (built-in Inquisit variable)
									Note: trialnum is a built-in Inquisit variable; it counts all trials run; 
									even those that do not store data to the data file such as feedback trials

values.phase:						"practice" vs. "test"											
values.countPractice1Rounds:		counts the number of practice1 rounds
values.countPractice2Rounds:		counts the number of practice2 rounds										
values.countAllTestRounds:			counts the number of test rounds (one round = two blocks)
values.repeatBlock:					1 = this block repeats the previous block due to a low consistency measure
									0 = not a repeat block
values.countTestRounds:				counts number of test blocks (disregarding repeated ones)

values.targetCat:					FLOWERS vs INSECTS
values.attributeCat: 				GOOD vs. BAD
values.cueCat:						Truth vs. Lie
values.target:						the presented target stimulus (e.g. 'daffodil')
values.attribute:					the presented attribute stimulus (here: good vs. bad)
latency:							measures response time (in ms) from onset of stimulus pair until response is made

response:							the scancode of the response button pressed
										18 = E
										23 = I
										
responsetext:						the label of the responsekey (E or I)											
										
										
values.resp:						the interpreted response: "Yes" vs. "No"
values.respA:						1 = yes response; 0 = otherwise

correct:							! Note: this column stores the accuracy of the response based on a pre-determined response pattern
										consistent with the hypothesis "flowers are good and insects are bad"
										HOWEVER: for summary variables, the participant's response pattern is taken into account to 
										determine the actual error response (= non-frequent response for a target)
								
values.responseOptionA:					'yes' key (18 = E; 23 = I)
values.responseOptionB:					'no' key (18 = E; 23 = I)								
								
Block-Consisteny Measures:
in general all values that fall within 0.25 and 0.75 are considered inconsistent (with 0.5 ~ random response pattern)
								
list.A1_consistentResp.mean:			porportion of 'yes'-equivalent responses for agreeing with 'daffodils are good'
										Note: a participant who always responds 'no' to 'daffodils are good' under the truth-cue
										and 'yes' under the 'lie' cue, will end up with a value of 0. However, the response pattern
										is very consistent.
										
list.A2_consistentResp.mean:			porportion of 'yes'-equivalent responses for agreeing with 'roses are good'
list.A3_consistentResp.mean:			porportion of 'yes'-equivalent responses for agreeing with 'lillys are good'

list.B1_consistentResp.mean:			porportion of 'yes'-equivalent responses for agreeing with 'spiders are good'
										Note: a participant who always responds 'no' to 'spiders are good' under the truth-cue
										and 'yes' under the 'lie' cue, will end up with a value of 0. However, the response pattern
										is very consistent.
										
list.B2_consistentResp.mean:			porportion of 'yes'-equivalent responses for agreeing with 'wasps are good'
list.B3_consistentResp.mean:			porportion of 'yes'-equivalent responses for agreeing with 'maggots are good'


(2) Summary data file: 'mt_irap_summary*.iqdat' (a separate file for each participant)*

computer.platform:					the platform the script was run on (win/mac/ios/android)
script.startdate:					date script was run
script.starttime:					time script was started
script.subjectid:					assigned subject id number
script.groupid:						assigned group id number
script.sessionid:					assigned session id number
script.elapsedtime:					time it took to run script (in ms); measured from onset to offset of script
script.completed:					0 = script was not completed (prematurely aborted); 
									1 = script was completed (all conditions run)

The following variables concerns Practice Performance:

values.countPractice1Rounds:		counts the number of practice1 rounds run 
values.practice1pass:				1 = practice1 passed; 0 = otherwise 
values.ACC_practice1:				proportion correct during the practice1 block that passed performance criteria
values.RT_practice1:				median correct response time (in ms) during the practice1 block that passed
values.countPractice2Rounds:		counts the number of practice2 rounds run 
values.practice2pass:				1 = practice2 passed; 0 = otherwise 
values.ACC_practice2:				proportion correct during the practice2 block that passed performance criteria
values.RT_practice2:				median correct response time (in ms) during the practice2 block that passed


The following variables concerns Test Performance: 
Note: this script provides summary statistics for 3 words per target

values.countAllTestRounds: 			counts the number of test blocks run (including the blocks that are repeated due to low response consistency)
values.countTestRounds: 			counts the number of test blocks run that are used for summary variable calculations (aka the blocks with reasonable response consistency)

Test Response Consistency Measures:
Consistency per word: 
measured as the mean of individual block consistency percentages for each target word across ALL test blocks 
(regardless of whether the testblock was repeated due to a low consistency measure or not)

In this script, the consistency measure provides directional information:

0% = CONSISTENT ('item is bad') <- 50% = INCONSISTENT -> 100% = CONSISTENT ('item is good')

100%: participant CONSISTENTLY responded to the item as if the statement 'item is good' was the participant's truth (direction towards goodness of item)
50%: participant randomly responded to the item as good or bad (inconsistent responding)
0%: participant CONSISTENTLY responded to the item as if the statement 'item is bad' was the participant's truth (direction towards badness of item)

Levin et al (2010, p.250): "In addition, if a stimulus was not sorted consistently at least 65% of the time it was excluded 
from analyses [...]"

expressions.meanDirectionalConsistency_A1:		mean block consistency percentage for target A1 (here: daffodil)
expressions.meanDirectionalConsistency_A2:		mean block consistency percentage for target A2 (here: rose)  
expressions.meanDirectionalConsistency_A3:		mean block consistency percentage for target A3 (here: lilly) 
expressions.meanDirectionalConsistency_B1:		mean block consistency percentage for target B1 (here: spider)  
expressions.meanDirectionalConsistency_B2:		mean block consistency percentage for target B2 (here: wasp)  
expressions.meanDirectionalConsistency_B3:		mean block consistency percentage for target B3 (here: maggot) 

!NOTE: The remaining summary variables are only calculated based on responses of test blocks that all passed the 
consistency criteria for each word (75% consistent):

Error-Analysis: Levin et al (2010, p.248): 'the less frequent response direction is counted as an error'.
The script assesses the direction of the error using the algorithm described under section EXPRESSIONS.
Levin et al (2010, p.250) suggest to remove all data of participants with errorrates > 25% (0.25) from
analyses.

expressions.overallErrorRate:	proportion errors across the test blocks
expressions.AA1_ErrorRate:      proportion errors across AA test trials presenting stimulus A1 (here: daffodils)
expressions.AA2_ErrorRate:      proportion errors across AA test trials presenting stimulus A2 (here: roses)
expressions.AA3_ErrorRate:      proportion errors across AA test trials presenting stimulus A3 (here: lillies)
expressions.BA1_ErrorRate:      proportion errors across BA test trials presenting stimulus A1 (here: daffodils)
expressions.BA2_ErrorRate:      proportion errors across BA test trials presenting stimulus A2 (here: roses)
expressions.BA3_ErrorRate:      proportion errors across BA test trials presenting stimulus A3 (here: lillies)
expressions.AB1_ErrorRate:      proportion errors across AB test trials presenting stimulus B1 (here: spider)
expressions.AB2_ErrorRate:      proportion errors across AB test trials presenting stimulus B2 (here: wasp)
expressions.AB3_ErrorRate:      proportion errors across AB test trials presenting stimulus B3 (here: maggot)
expressions.BB1_ErrorRate:      proportion errors across BB test trials presenting stimulus B1 (here: spider)
expressions.BB2_ErrorRate:      proportion errors across BB test trials presenting stimulus B2 (here: wasp)
expressions.BB3_ErrorRate:      proportion errors across BB test trials presenting stimulus B3 (here: maggot)


Latency Measures:

NOTE: all following latency measures are based on
- correct responses (correct = frequent response pattern for the participant for that particular stimulus)
- latencies <= 10000ms
(see Levin et al, 2010, p.250)

expressions.medianCorrTestRT:		median response latency (in ms) across all trials
									Levin et al (2010, p.250) suggest to remove data of participants with "average response latency above 3 seconds"
														
D-Scores: 								
Direction of responding is set so that a positive MT-IRAP score  indicates that responses on the Lie trials took longer than the Truth trials*.

*Note: while it's not explicitly noted by Levin et al (2010), due to the fact that the error is not pre-determined in this
procedure, positive D-scores simply indicate that lieing took longer than telling the truth.
Since the participant's personal truth might not align with the conventional truth (e.g. 'flowers-good'), 
the interpretation of the D-scores ideally takes into account the participant's direction of errors.								
								
expressions.D_MT_IRAP_AA1:		D-score for difference in responding to truth vs. lie trials in AA trials (here: good-daffodil)
expressions.D_MT_IRAP_AA2:		D-score for difference in responding to truth vs. lie trials in AA trials (here: good-rose)
expressions.D_MT_IRAP_AA3:		D-score for difference in responding to truth vs. lie trials in AA trials (here: good-lilly)
expressions.D_MT_IRAP_BA1:		D-score for difference in responding to truth vs. lie trials in BA trials (here: bad-daffodil)
expressions.D_MT_IRAP_BA2:		D-score for difference in responding to truth vs. lie trials in BA trials (here: bad-rose)
expressions.D_MT_IRAP_BA3:		D-score for difference in responding to truth vs. lie trials in BA trials (here: bad-lilly)
expressions.D_MT_IRAP_AB1:		D-score for difference in responding to truth vs. lie trials in AB trials (here: good-spider)
expressions.D_MT_IRAP_AB2:		D-score for difference in responding to truth vs. lie trials in AB trials (here: good-wasp)
expressions.D_MT_IRAP_AB3:		D-score for difference in responding to truth vs. lie trials in AB trials (here: good-maggot)
expressions.D_MT_IRAP_BB1:		D-score for difference in responding to truth vs. lie trials in BB trials (here: bad-spider)
expressions.D_MT_IRAP_BB2:		D-score for difference in responding to truth vs. lie trials in BB trials (here: bad-wasp)
expressions.D_MT_IRAP_BB3:		D-score for difference in responding to truth vs. lie trials in BB trials (here: bad-maggot)

expressions.D_MT_IRAP_A1:		mean of the two D-Scores regarding target A1 (here: daffodil), Example: (expressions.D_MT_IRAP_AA1 + expressions.D_MT_IRAP_BA1)/2
expressions.D_MT_IRAP_A2:		mean of the two D-Scores regarding target A2 (here: rose), Example: (expressions.D_MT_IRAP_AA2 + expressions.D_MT_IRAP_BA2)/2
expressions.D_MT_IRAP_A3:		mean of the two D-Scores regarding target A3 (here: lilly), Example: (expressions.D_MT_IRAP_AA3 + expressions.D_MT_IRAP_BA3)/2
expressions.D_MT_IRAP_B1:		mean of the two D-Scores regarding target B1 (here: spider), Example: (expressions.D_MT_IRAP_BB1 + expressions.D_MT_IRAP_AB1)/2
expressions.D_MT_IRAP_B2:		mean of the two D-Scores regarding target B2 (here: wasp), Example: (expressions.D_MT_IRAP_BB2 + expressions.D_MT_IRAP_AB2)/2
expressions.D_MT_IRAP_B3:		mean of the two D-Scores regarding target B3 (here: maggot), Example: (expressions.D_MT_IRAP_BB3 + expressions.D_MT_IRAP_AB3)/2

expressions.D_MT_IRAP_AA:		mean of the three D-Scores for A targets (here: flowers) used in AA trials
expressions.D_MT_IRAP_BA:		mean of the three D-Scores for A targets (here: flowers) used in BA trials
expressions.D_MT_IRAP_AB:		mean of the three D-Scores for B targets (here: insects) used in AB trials
expressions.D_MT_IRAP_BB:		mean of the three D-Scores for B targets (here: insects) used in BB trials

expressions.D_MT_IRAP_A:		mean of the D-Scores for target A (here: flowers)
expressions.D_MT_IRAP_B:		mean of the D-Scores for target B (here: insects)

D-MT-IRAP:						mean of the two D-Scores for target A and target B

* separate data files: to change to one data file for all participants (on Inquisit Lab only), go to section
"DATA" and follow further instructions


___________________________________________________________________________________________________________________	
EXPERIMENTAL SET-UP 
___________________________________________________________________________________________________________________	

(1) Practice: 2 stages
Practice1: 16 trials with only Truth-Trials (uses 2 items for practice target A (desserts) 
and 2 items for practice target B (diseases))
- AA, BB, AB, BA trials; random order (each target word runs twice in its respective truth-trials)
Pass Criteria:
- Proportion Correct >= 80%, median RT <= 2000ms
=> Participants get the chance to repeat practice1 6 times (after failing 6 times, the script terminates)

Practice2: 16 trials with Truth and Lie Trials
- AA, BB, AB, BA trials; random order (each target word runs once in its respective trials)
Pass Criteria:
- Proportion Correct >= 80%, median RT <= 2000ms
=> Participants get the chance to repeat practice1 6 times (after failing 6 times, the script terminates)

 9 each of AA/BA/AB/BB-truth trials and 9 each of AA/BA/AB/BB-lie trials (order is randomly determined)
AA = good paired with flowers (here 3 flowers: daffodil, rose, lilly)
AB = good paired with insects (here 3 insects: spider, wasp, maggot)
BA = bad paired with flowers
BB = bad paired with insects
- each 'flower' stim is equally often selected by AA truth, AA lie, BA truth, BA lie (3 times each)
- each 'insect' stim is equally often selected by BB truth, BB lie, AB truth, AB lie (3 times each)

Consistency Criteria:
at the end of each test block, the program checks whether the participant has met the consistency criteria
of 75% consistent responding for each item:
		* checks whether the responses to each individual words were at least 75% consistent
		(Example: participant consistently responded 'yes' to 'daffodils are good' under truth condition and 'no' 
		to 'daffodils are good' under lie condition as well as responded 'no' to 'daffodils are bad' under truth condition 
		and 'yes' to 'daffodils are bad' under lie condition)
		* if not all consistency criterium are met, the block gets repeated and the data are not used for summary statistics
		* if all consistency criteriums are met, the block terminates and the data are used for summary statistics
		
		
Trialsequence: 
Instructions (Truth/Lie) and response button assignment for 1000ms
-> Item pair gets added until correct response
-> ITI

Data: 
in this script:

- latencies of error responses are removed (error response = infrequent response for a given stimulus)
Note: whether a response is an error response is determined at the end of each block, not on the individual trial level
- latencies > 10000ms are removed


___________________________________________________________________________________________________________________
STIMULI
___________________________________________________________________________________________________________________
provided by Millisecond Software - can be edited under section Editable Stimuli
The default script provides:
practice: 2 dessert words, 2 disease words
test: 3 flower words, 3 insect words

___________________________________________________________________________________________________________________	
INSTRUCTIONS 
___________________________________________________________________________________________________________________
provided by Millisecond Software - can be edited under section Editable Instructions
(they are not original to Levin et al, 2010)

___________________________________________________________________________________________________________________	
EDITABLE CODE 
___________________________________________________________________________________________________________________	
check below for (relatively) easily editable parameters, stimuli, instructions etc. 
Keep in mind that you can use this script as a template and therefore always "mess" with the entire code 
to further customize your experiment.

The parameters you can change are:


Experimental Design Parameters:

/maxNumberPractice1Rounds:				max number of practice1 rounds (default: 6)
/maxNumberPractice2Rounds:				max number of practice1 rounds (default: 6)
												
/numberOfTestRounds:					number of test rounds (default: 2)
												
/fixedResponseKeyLocation:				true (1): the response key assignment is fixed throughout the task
										(the assignment is randomly decided at the start of the task) (default)
										false (0): the response key assignment is randomly decided for each trial
										(in this case participants not only have to make a categorization choice but
										also pay attention to the location of the correct response key)
																								
Timing Parameters:												

/cueDuration:							the duration (in ms) of the trial cue (here: truth vs. lie) before the pair of words appears (default: 1000ms)												

/readyDuration:							the duration (in ms) of the 'get-ready' trial (default: 3000ms)
/iti:									the intertrial interval in ms (default: 400)



/minPracACC:							mastery accuracy criterion: the minimum proportion correct that participants have to reach 
										in each practice phase (practice1 + practice2) before they can move on to the test (default: 0.8)
												
/maxPracRT:								mastery response time criterion: the maximum medium latency (in ms) that is permitted 
										in each practice phase (practice1 + practice2) before participants might be able to move on to the test (default: 2000ms)

										
/maxTestRT:								The scoring algorithm for expressions.D_MT_IRAP_excludes all test responses with latencies (in ms) > 10000ms
										from expressions.D_MT_IRAP_calculations (default: 10000ms)
												

Sizing Parameters:

/stimSize:								proportional sizing (relative to canvas height) of the presented words on the screen (default: 8%)


Response Key Parameters: 
/leftkey:								the left response key (default: "E")
/rightkey:								the right response key (default: "I")