Millisecond Forums

Don't even get me started...;-)

Which is not true. While Cohen's d may trail off into infinity in theory, D actually varies between -2 and +2. See Sriram, Greenwald, & Nosek (2006), p. 57, footnote 2, or Nosek & Sriram (2007), p. 396, footnote 2.

This will be my last post on the topic of IAT for a while. I'll leave the rest to Sean.

~Dave

Whoops, just edited my post while you were answering :).

Has this been resolved off-forum?

Apologies for letting this drop.

The scoring algorithm used in the ST-IAT was taken directly from the IAT, which in turn was taken from the SPSS script, which in turn was based on Tony's IAT script, which in turn reflects the improved scoring algorithm reported in Greenwald, Nosek, and Banaji (2003).

Latencies above 10000 ms are indeed discarded by both the Inquisit and SPSS script. I may be misunderstanding something, but I'm not aware of any part of the improved algorithm that involves calculating "the mean latencies for some practice trials to get the latency scale for an individual user". Do you have a reference for this?

It's certainly possible that in adapting the IAT score to the ST-IAT that something isn't right, but I'm not aware of any mistakes. Is there anything in particular that seems fishy here?

-Sean

Hi,

I must have said that in the wrong way :). The mentioned article Greenwald, Nosek, and Banaji (2003, p. 42)  summarises the difference in this way:

The improved algorithm has three substantial changes from the conventional procedure: (a) use of practice-block data (Step 1 in Table 7), (b)
use of error penalties (computed in Steps 5 and 7), and (c) use of individual-respondent standard deviations to provide the measure’s scale unit (computed in Step 6 and applied in Step 11).

Now, as you might have already guessed, I'm not too knowledgeable in IAT scoring and I'm interested in this because I'd like to know if whtether I can report the running D scores as ones calculated by the improved algorithm or as something else. I'm also not even completely sure how the imporved alogrithm  is applied with a ST-IAT, since the article is obviously advising the scoring of conventional IATs, which has a different number of block etc.

Basically, I'm just trying to grasp what the template calculates from the user latencies, but I'm not trying to take this to any higher level or start a methodological discussion on different procedures which would be way beyond my skills :).

Best,

Kaarinen

Hi! I see there's been a lot of clarification of what the D score actually is and how it's calculated, but I have one more question.

Is there a formal way to put the D scores into categories of effect size, like "Small", "Medium" and "Large" effects as is commonly done with Cohen's d scores? I'm hesitant to just jump in and use the same system as is used with Cohen's d scores because IAT D scores go from -2 to +2. I'd be grateful for any advice! Katherine

Katherine,

you will find your question answered in the basic IAT literature such as Greenwald, McGhee & Schwartz (1998) or Greenwald, Nosek & Banaji (2003) ,which anyone doing IATs *must* read.

Regards,

~Dave

Hi Dave,

Thanks for the tips. I had read the 2003 paper, but not the 1998 one. But, I must confess, that now, even after reading both articles, I still do not have the answer to my question. In the 1998 article, I only see that they present the conventional effect sizes for the d measure and in the later article in which they discuss the IAT D measure, I don't see anything about conventional effect sizes or how to report effect sizes. 

Basically, what I need to know is how to report and discuss my D measures an how to understand when to expect that a difference between groups' D measures might be significant or worth looking into. For example, if one group has a D score of .30 and another has .17, how can I discuss this and think about it before doing a significance test? I wonder if it's possible to say that a D score of .30 represents a large association, while a score of .20 represents a small association. Or is this only possible after calculating a d score?

Thanks for any help! Katherine

Katherine,

as an example see e.g. https://implicit.harvard.edu/implicit/demo/background/raceinfo.html.

You can glean the same info by simply looking at any of the IAT scripts:

<trial summary>
[...]
/ ontrialbegin = [if( abs(expressions.d) > 0.15 ) values.magnitude = "a slight"]
/ ontrialbegin = [if( abs(expressions.d) > 0.35 ) values.magnitude = "a moderate"]
/ ontrialbegin = [if( abs(expressions.d) >= 0.65 ) values.magnitude = "a strong"]
[...]
</trial>

Regards,

~Dave

Aha, thank you!