___________________________________________________________________________________________________________________ The Mixed-Trial Implicit Relational Assessment Procedure (MT-IRAP): Template ___________________________________________________________________________________________________________________ Script Author: Katja Borchert, Ph.D. (katjab@millisecond.com) for Millisecond Software, LLC Date: 08-06-2018 last updated: 02-23-2022 by K. Borchert (katjab@millisecond.com) for Millisecond Software, LLC Script Copyright © 02-23-2022 Millisecond Software ___________________________________________________________________________________________________________________ BACKGROUND INFO ___________________________________________________________________________________________________________________ The IRAP is a procedure suggested by Barnes-Holmes et al (e.g. 2006, 2010A, 2010B) to measure implicit attitudes. Participants are asked to quickly categorize pairs of stimuli according to pre-determined rules. The rules are either hypothesis-consistent (e.g. 'Flowers are pleasant; Insects are unpleasant') or hypothesis-inconsistent (e.g. 'Insects are pleasant, Flowers are unpleasant'). From one block to the next, the rule flips from being hypothesis-consistent to hypothesis-inconsistent. Latency difference between hypothesis-consistent and hypothesis-inconsistent blocks are then used to to assess implicit attitudes. The mixed trial IRAP (MT-IRAP, Levin et al, 2010) presents consistent and inconsistent trials in the same block using the cues "truth" or "lie" as 'rule' cues. Participants are given the cue to 'lie' or 'tell the truth' 1s before each pair of stimuli is presented. Note that instead of using pre-determined rules and thus predetermined error response (see IRAP), the MT-IRAP uses the participant's response pattern to determine correct and incorrect responses for each stimuli. Specifically, the MT-IRAP uses the more infrequent response pattern to a stimuli as the error response (see Levin et al, 2010, p.248). For example, a consistent response pattern would be: to say 'yes' 100% of the time to 'daffodils-good' under the Truth-condition and to say 'no' 100% of the time to 'daffodils-good' under the Lie-condition. For a participant who responds 90% of the time 'yes' to 'daffodils-good' under the Truth-condition and 70% of the time 'yes' to 'daffodils-good' under the Lie-condition, the less frequent response pattern (90% > 70%) is the one consistent with disagreeing with 'daffodils-good'. In this particular case, the participant thus made a number of 'errors' (70%) in the 'daffodil-good' Lie-condition and only 10% errors in the 'daffodils-good' Truth-condition. MT-IRAP D-scores in this script are calculated as the difference between mean response latencies on Truth and Lie trials for each type of target and individual target words. procedure to calculate MT-IRAP scores in this script follows the guidelines outlined in Levin et al (2010, p.250). Direction of responding is set so that a positive MT-IRAP score indicates that responses on the Lie trials took longer than the Truth trials*. This script provides MT-IRAP D-scores for 3 words per target category. *Note: while it's not explicitly noted by Levin et al (2010), due to the fact that the error response is not pre-determined in this procedure, positive D-scores simply indicate that lieing took longer than telling the truth. Since the participant's personal truth might not align with the 'assumed truth' (e.g. 'flowers-good'), the interpretation of the D-scores ideally takes into account the participants' individual response patterns. References: Barnes-Holmes, D., Barnes-Holmes, Y., Power, P., Hayden, E., Milne, R., & Stewart, I. (2006). Do you know what you really believe? Developing the Implicit Relational Evaluation Procedure (IRAP) as a direct measure of implicit beliefs. The Irish Psychologist, 32(7), 169–177. Barnes-Holmes, D., Barnes-Holmes, Y., Stewart, I., & Boles, S. (2010A). A sketch of the implicit relational assessment procedure (IRAP) and the relational elaboration and coherence (REC) model. The Psychological Record, 60(3), 527. Barnes-Holmes, D., Murtagh, L., Barnes-Holmes, Y., & Stewart, I. (2010B). Using the Implicit Association Test and the Implicit Relational Assessment Procedure to measure attitudes towards meat and vegetables in vegetarians and meat-eaters. The Psychological Record, 60, 287-306. Hussey, I., Mhaoileoin, D. N., Barnes-Holmes, D., Ohtsuki, T., Kishita, N., Hughes, S., & Murphy, C. (2016). The IRAP Is Nonrelative but notAcontextual: Changes to the Contrast Category Influence Men’s Dehumanization of Women. The Psychological Record, 66(2), 291– 299. http://doi.org/10.1007/s40732-016-0171-6 open source IRAP procedure by I. Hussey: https://contextualscience.org/implicit_relational_assessment_procedure_irap_website. Levin, M. E., Hayes, S. C., & Waltz, T. (2010). Creating an implicit measure of cognition more suited to applied research: A test of the Mixed Trial - Implicit Relational Assessment Procedure (MT-IRAP). International Journal of Behavioral Consultation and Therapy, 6, 245-261. ___________________________________________________________________________________________________________________ TASK DESCRIPTION ___________________________________________________________________________________________________________________ In each trial, participants receive pairs of items such as the word "Tulip" (belonging to the target 'Flower' category) paired with the word "Good" (belonging to the positive attribute category). Before each item pair is presented, participants are told to tell the 'truth' or 'lie'. For example, for Truth-trials a participant would say 'yes' to the item pair 'tulip-good' (assuming that participant associates flowers with something positive). For Lie-trials, the participant would respond 'no' to the same item pair (for a consistent response pattern). This script runs 4 trial types under the Truth and Lie condition: * AA trials: good + flower * BA trials: bad + flower * AB trials: good + insect * BB trials: bad + insect ___________________________________________________________________________________________________________________ DURATION ___________________________________________________________________________________________________________________ default set-up: appr. 15minutes Practice: appr. 4 minutes 1 Test Round: appr. 5 minutes => repeated test rounds will increase the duration accordingly ___________________________________________________________________________________________________________________ DATA FILE INFORMATION ___________________________________________________________________________________________________________________ The default data stored in the data files are: (1) Raw data file: 'mt_irap_raw*.iqdat' (a separate file for each participant)* build: The specific Inquisit version used (the 'build') that was run computer.platform: the platform the script was run on (win/mac/ios/android) date, time: date and time script was run subject, group: with the current subject/groupnumber session: with the current session id blockcode, blocknum: the name and number of the current block (built-in Inquisit variable) trialcode, trialnum: the name and number of the currently recorded trial (built-in Inquisit variable) Note: trialnum is a built-in Inquisit variable; it counts all trials run; even those that do not store data to the data file such as feedback trials phase: "practice" vs. "test" countPractice1Rounds: counts the number of practice1 rounds countPractice2Rounds: counts the number of practice2 rounds countAllTestRounds: counts the number of test rounds (one round = two blocks) repeatBlock: 1 = this block repeats the previous block due to a low consistency measure 0 = not a repeat block countTestRounds: counts number of test blocks (disregarding repeated ones) targetCat: FLOWERS vs INSECTS attributeCat: GOOD vs. BAD cueCat: Truth vs. Lie target: the presented target stimulus (e.g. 'daffodil') attribute: the presented attribute stimulus (here: good vs. bad) latency: measures response time (in ms) from onset of stimulus pair until response is made response: the scancode of the response button pressed 18 = E 23 = I responsetext: the label of the responsekey (E or I) resp: the interpreted response: "Yes" vs. "No" respA: 1 = yes response; 0 = otherwise correct: ! Note: this column stores the accuracy of the response based on a pre-determined response pattern consistent with the hypothesis "flowers are good and insects are bad" HOWEVER: for summary variables, the participant's response pattern is taken into account to determine the actual error response (= non-frequent response for a target) responseOptionA: 'yes' key (18 = E; 23 = I) responseOptionB: 'no' key (18 = E; 23 = I) Block-Consisteny Measures: in general all values that fall within 0.25 and 0.75 are considered inconsistent (with 0.5 ~ random response pattern) list.A1_consistentResp.mean: porportion of 'yes'-equivalent responses for agreeing with 'daffodils are good' Note: a participant who always responds 'no' to 'daffodils are good' under the truth-cue and 'yes' under the 'lie' cue, will end up with a value of 0. However, the response pattern is very consistent. list.A2_consistentResp.mean: porportion of 'yes'-equivalent responses for agreeing with 'roses are good' list.A3_consistentResp.mean: porportion of 'yes'-equivalent responses for agreeing with 'lillys are good' list.B1_consistentResp.mean: porportion of 'yes'-equivalent responses for agreeing with 'spiders are good' Note: a participant who always responds 'no' to 'spiders are good' under the truth-cue and 'yes' under the 'lie' cue, will end up with a value of 0. However, the response pattern is very consistent. list.B2_consistentResp.mean: porportion of 'yes'-equivalent responses for agreeing with 'wasps are good' list.B3_consistentResp.mean: porportion of 'yes'-equivalent responses for agreeing with 'maggots are good' (2) Summary data file: 'mt_irap_summary*.iqdat' (a separate file for each participant)* inquisit.version: Inquisit version run computer.platform: the platform the script was run on (win/mac/ios/android) startdate: date script was run starttime: time script was started subjectid: assigned subject id number groupid: assigned group id number sessionid: assigned session id number elapsedtime: time it took to run script (in ms); measured from onset to offset of script completed: 0 = script was not completed (prematurely aborted); 1 = script was completed (all conditions run) The following variables concerns Practice Performance: countPractice1Rounds: counts the number of practice1 rounds run practice1pass: 1 = practice1 passed; 0 = otherwise ACC_practice1: proportion correct during the practice1 block that passed performance criteria RT_practice1: median correct response time (in ms) during the practice1 block that passed countPractice2Rounds: counts the number of practice2 rounds run practice2pass: 1 = practice2 passed; 0 = otherwise ACC_practice2: proportion correct during the practice2 block that passed performance criteria RT_practice2: median correct response time (in ms) during the practice2 block that passed The following variables concerns Test Performance: Note: this script provides summary statistics for 3 words per target countAllTestRounds: counts the number of test blocks run (including the blocks that are repeated due to low response consistency) countTestRounds: counts the number of test blocks run that are used for summary variable calculations (aka the blocks with reasonable response consistency) Test Response Consistency Measures: Consistency per word: measured as the mean of individual block consistency percentages for each target word across ALL test blocks (regardless of whether the testblock was repeated due to a low consistency measure or not) In this script, the consistency measure provides directional information: 0% = CONSISTENT ('item is bad') <- 50% = INCONSISTENT -> 100% = CONSISTENT ('item is good') 100%: participant CONSISTENTLY responded to the item as if the statement 'item is good' was the participant's truth (direction towards goodness of item) 50%: participant randomly responded to the item as good or bad (inconsistent responding) 0%: participant CONSISTENTLY responded to the item as if the statement 'item is bad' was the participant's truth (direction towards badness of item) Levin et al (2010, p.250): "In addition, if a stimulus was not sorted consistently at least 65% of the time it was excluded from analyses [...]" meanDirectionalConsistency_A1: mean block consistency percentage for target A1 (here: daffodil) meanDirectionalConsistency_A2: mean block consistency percentage for target A2 (here: rose) meanDirectionalConsistency_A3: mean block consistency percentage for target A3 (here: lilly) meanDirectionalConsistency_B1: mean block consistency percentage for target B1 (here: spider) meanDirectionalConsistency_B2: mean block consistency percentage for target B2 (here: wasp) meanDirectionalConsistency_B3: mean block consistency percentage for target B3 (here: maggot) !NOTE: The remaining summary variables are only calculated based on responses of test blocks that all passed the consistency criteria for each word (75% consistent): Error-Analysis: Levin et al (2010, p.248): 'the less frequent response direction is counted as an error'. The script assesses the direction of the error using the algorithm described under section Levin et al (2010, p.250) suggest to remove all data of participants with errorrates > 25% (0.25) from analyses. overallErrorRate: proportion errors across the test blocks AA1_ErrorRate: proportion errors across AA test trials presenting stimulus A1 (here: daffodils) AA2_ErrorRate: proportion errors across AA test trials presenting stimulus A2 (here: roses) AA3_ErrorRate: proportion errors across AA test trials presenting stimulus A3 (here: lillies) BA1_ErrorRate: proportion errors across BA test trials presenting stimulus A1 (here: daffodils) BA2_ErrorRate: proportion errors across BA test trials presenting stimulus A2 (here: roses) BA3_ErrorRate: proportion errors across BA test trials presenting stimulus A3 (here: lillies) AB1_ErrorRate: proportion errors across AB test trials presenting stimulus B1 (here: spider) AB2_ErrorRate: proportion errors across AB test trials presenting stimulus B2 (here: wasp) AB3_ErrorRate: proportion errors across AB test trials presenting stimulus B3 (here: maggot) BB1_ErrorRate: proportion errors across BB test trials presenting stimulus B1 (here: spider) BB2_ErrorRate: proportion errors across BB test trials presenting stimulus B2 (here: wasp) BB3_ErrorRate: proportion errors across BB test trials presenting stimulus B3 (here: maggot) Latency Measures: NOTE: all following latency measures are based on - correct responses (correct = frequent response pattern for the participant for that particular stimulus) - latencies <= 10000ms (see Levin et al, 2010, p.250) medianCorrTestRT: median response latency (in ms) across all trials Levin et al (2010, p.250) suggest to remove data of participants with "average response latency above 3 seconds" D-Scores: Direction of responding is set so that a positive MT-IRAP score indicates that responses on the Lie trials took longer than the Truth trials*. *Note: while it's not explicitly noted by Levin et al (2010), due to the fact that the error is not pre-determined in this procedure, positive D-scores simply indicate that lieing took longer than telling the truth. Since the participant's personal truth might not align with the conventional truth (e.g. 'flowers-good'), the interpretation of the D-scores ideally takes into account the participant's direction of errors. D_MT_IRAP_AA1: D-score for difference in responding to truth vs. lie trials in AA trials (here: good-daffodil) D_MT_IRAP_AA2: D-score for difference in responding to truth vs. lie trials in AA trials (here: good-rose) D_MT_IRAP_AA3: D-score for difference in responding to truth vs. lie trials in AA trials (here: good-lilly) D_MT_IRAP_BA1: D-score for difference in responding to truth vs. lie trials in BA trials (here: bad-daffodil) D_MT_IRAP_BA2: D-score for difference in responding to truth vs. lie trials in BA trials (here: bad-rose) D_MT_IRAP_BA3: D-score for difference in responding to truth vs. lie trials in BA trials (here: bad-lilly) D_MT_IRAP_AB1: D-score for difference in responding to truth vs. lie trials in AB trials (here: good-spider) D_MT_IRAP_AB2: D-score for difference in responding to truth vs. lie trials in AB trials (here: good-wasp) D_MT_IRAP_AB3: D-score for difference in responding to truth vs. lie trials in AB trials (here: good-maggot) D_MT_IRAP_BB1: D-score for difference in responding to truth vs. lie trials in BB trials (here: bad-spider) D_MT_IRAP_BB2: D-score for difference in responding to truth vs. lie trials in BB trials (here: bad-wasp) D_MT_IRAP_BB3: D-score for difference in responding to truth vs. lie trials in BB trials (here: bad-maggot) D_MT_IRAP_A1: mean of the two D-Scores regarding target A1 (here: daffodil), Example: (D_MT_IRAP_AA1 + D_MT_IRAP_BA1)/2 D_MT_IRAP_A2: mean of the two D-Scores regarding target A2 (here: rose), Example: (D_MT_IRAP_AA2 + D_MT_IRAP_BA2)/2 D_MT_IRAP_A3: mean of the two D-Scores regarding target A3 (here: lilly), Example: (D_MT_IRAP_AA3 + D_MT_IRAP_BA3)/2 D_MT_IRAP_B1: mean of the two D-Scores regarding target B1 (here: spider), Example: (D_MT_IRAP_BB1 + D_MT_IRAP_AB1)/2 D_MT_IRAP_B2: mean of the two D-Scores regarding target B2 (here: wasp), Example: (D_MT_IRAP_BB2 + D_MT_IRAP_AB2)/2 D_MT_IRAP_B3: mean of the two D-Scores regarding target B3 (here: maggot), Example: (D_MT_IRAP_BB3 + D_MT_IRAP_AB3)/2 D_MT_IRAP_AA: mean of the three D-Scores for A targets (here: flowers) used in AA trials D_MT_IRAP_BA: mean of the three D-Scores for A targets (here: flowers) used in BA trials D_MT_IRAP_AB: mean of the three D-Scores for B targets (here: insects) used in AB trials D_MT_IRAP_BB: mean of the three D-Scores for B targets (here: insects) used in BB trials D_MT_IRAP_A: mean of the D-Scores for target A (here: flowers) D_MT_IRAP_B: mean of the D-Scores for target B (here: insects) D-MT-IRAP: mean of the two D-Scores for target A and target B * separate data files: to change to one data file for all participants (on Inquisit Lab only), go to section "DATA" and follow further instructions ___________________________________________________________________________________________________________________ EXPERIMENTAL SET-UP ___________________________________________________________________________________________________________________ (1) Practice: 2 stages Practice1: 16 trials with only Truth-Trials (uses 2 items for practice target A (desserts) and 2 items for practice target B (diseases)) - AA, BB, AB, BA trials; random order (each target word runs twice in its respective truth-trials) Pass Criteria: - Proportion Correct >= 80%, median RT <= 2000ms => Participants get the chance to repeat practice1 6 times (after failing 6 times, the script terminates) Practice2: 16 trials with Truth and Lie Trials - AA, BB, AB, BA trials; random order (each target word runs once in its respective trials) Pass Criteria: - Proportion Correct >= 80%, median RT <= 2000ms => Participants get the chance to repeat practice1 6 times (after failing 6 times, the script terminates) 9 each of AA/BA/AB/BB-truth trials and 9 each of AA/BA/AB/BB-lie trials (order is randomly determined) AA = good paired with flowers (here 3 flowers: daffodil, rose, lilly) AB = good paired with insects (here 3 insects: spider, wasp, maggot) BA = bad paired with flowers BB = bad paired with insects - each 'flower' stim is equally often selected by AA truth, AA lie, BA truth, BA lie (3 times each) - each 'insect' stim is equally often selected by BB truth, BB lie, AB truth, AB lie (3 times each) Consistency Criteria: at the end of each test block, the program checks whether the participant has met the consistency criteria of 75% consistent responding for each item: * checks whether the responses to each individual words were at least 75% consistent (Example: participant consistently responded 'yes' to 'daffodils are good' under truth condition and 'no' to 'daffodils are good' under lie condition as well as responded 'no' to 'daffodils are bad' under truth condition and 'yes' to 'daffodils are bad' under lie condition) * if not all consistency criterium are met, the block gets repeated and the data are not used for summary statistics * if all consistency criteriums are met, the block terminates and the data are used for summary statistics Trialsequence: Instructions (Truth/Lie) and response button assignment for 1000ms -> Item pair gets added until correct response -> ITI Data: in this script: - latencies of error responses are removed (error response = infrequent response for a given stimulus) Note: whether a response is an error response is determined at the end of each block, not on the individual trial level - latencies > 10000ms are removed ___________________________________________________________________________________________________________________ STIMULI ___________________________________________________________________________________________________________________ provided by Millisecond Software - can be edited under section Editable Stimuli The default script provides: practice: 2 dessert words, 2 disease words test: 3 flower words, 3 insect words ___________________________________________________________________________________________________________________ INSTRUCTIONS ___________________________________________________________________________________________________________________ provided by Millisecond Software - can be edited under section Editable Instructions (they are not original to Levin et al, 2010) ___________________________________________________________________________________________________________________ EDITABLE CODE ___________________________________________________________________________________________________________________ check below for (relatively) easily editable parameters, stimuli, instructions etc. Keep in mind that you can use this script as a template and therefore always "mess" with the entire code to further customize your experiment. The parameters you can change are: Experimental Design Parameters: /maxNumberPractice1Rounds: max number of practice1 rounds (default: 6) /maxNumberPractice2Rounds: max number of practice1 rounds (default: 6) /numberOfTestRounds: number of test rounds (default: 2) /fixedResponseKeyLocation: true (1): the response key assignment is fixed throughout the task (the assignment is randomly decided at the start of the task) (default) false (0): the response key assignment is randomly decided for each trial (in this case participants not only have to make a categorization choice but also pay attention to the location of the correct response key) Timing Parameters: /cueDuration: the duration (in ms) of the trial cue (here: truth vs. lie) before the pair of words appears (default: 1000ms) /readyDuration: the duration (in ms) of the 'get-ready' trial (default: 3000ms) /iti: the intertrial interval in ms (default: 400) /minPracACC: mastery accuracy criterion: the minimum proportion correct that participants have to reach in each practice phase (practice1 + practice2) before they can move on to the test (default: 0.8) /maxPracRT: mastery response time criterion: the maximum medium latency (in ms) that is permitted in each practice phase (practice1 + practice2) before participants might be able to move on to the test (default: 2000ms) /maxTestRT: The scoring algorithm for expressions.D_MT_IRAP_excludes all test responses with latencies (in ms) > 10000ms from expressions.D_MT_IRAP_calculations (default: 10000ms) Sizing Parameters: /stimSize: proportional sizing (relative to canvas height) of the presented words on the screen (default: 8%) Response Key Parameters: /leftkey: the left response key (default: "E") /rightkey: the right response key (default: "I")