The best way to randomize in Inquisit


Author
Message
valengrandelski
valengrandelski
Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)
Group: Forum Members
Posts: 10, Visits: 76
I'm trying to create a survey with 5 different conditions as well as 2 separate IATs and I'm having difficulty with the best/easiest way to put all these pieces together and randomize to participants.

Here's how I would like the entirety of the experiment administered (counterbalanced across 5 conditions):

--Survey, part 1 (5 conditions)
--IAT #1, Sexual Risk (would like to counterbalance random assignment of order of IATs--half of participants receive #1 first, half receive #2 first)
--IAT #2, Racial Prejudice (see IAT #1 note)
--Survey, part 2 (5 conditions, must be matched to the condition received in part 1)

Another complicating factor is within each IAT, I have another 2 conditions--one where the compatible test is received first and one where the incompatible test is received first.  Is there an easier way to change this order within the IAT so it is randomly assigned?  

Here are parts of my script--it's ridiculously long, so I've included only the different surveys and the different IATs.  I can attach more if needed.  I really appreciate any help or feedback.

<survey PrEP_public_perceptions_CONDITION_1a> 
/ pages = [1=ppp1; 2=ppp2; 3=ppp3; 4=ppp4; 5=ppp5; 6=ppp7_condition_1p1; 7=ppp7_condition_1p2; 8=ppp7_condition_1p3; 9=ppp7_condition_1p4; 10=ppp7_condition_1p5; 11=ppp7_condition_1p6; 12=ppp8_condition_1p1; 13=ppp8_condition_1p2; 14=ppp8_condition_1p3; 15=ppp8_condition_1p4; 16=ppp9_p1; 17=ppp9_p2; 18=ppp10_p1; 19=ppp10_p2; 20=ppp10_p3; 21=ppp11; 22=ppp12_p1; 23=ppp12_p2; 24=ppp12_p3; 25=ppp13; 26=ppp14_p1; 27=ppp14_p3; 28=ppp14_p5; 29=ppp14_p7; 30=ppp14_p9; 31=ppp14_p11a; 32=ppp14_p13; 33=ppp14_p15; 34=ppp14_p16; 35=ppp14_p17; 36=ppp14_p18;] 
/ itemspacing = 5% 
/ showpagenumbers = false 
/ showquestionnumbers = false
</survey>

<survey PrEP_public_perceptions_CONDITION_1b> 
/ pages = [1=ppp16_p1; 2=ppp16_p2; 3=ppp17_p1; 4=ppp17_p2; 5=ppp18_p1; 6=ppp18_p2; 7=ppp19_condition1.2.3.5;]
/ itemspacing = 5% 
/ showpagenumbers = false 
/ showquestionnumbers = false
</survey>

<survey PrEP_public_perceptions_CONDITION_2a> 
/ pages = [1=ppp1; 2=ppp2; 3=ppp3; 4=ppp4; 5=ppp5; 6=ppp7_condition_2p1; 7=ppp7_condition_2p2; 8=ppp7_condition_2p3; 9=ppp7_condition_2p4; 10=ppp7_condition_2p5; 11=ppp7_condition_2p6; 12=ppp8_condition_2p1; 13=ppp8_condition_2p2; 14=ppp8_condition_2p3; 15=ppp8_condition_2p4; 16=ppp9_p1; 17=ppp9_p2; 18=ppp10_p1; 19=ppp10_p2; 20=ppp10_p3; 21=ppp11; 22=ppp12_p1; 23=ppp12_p2; 24=ppp12_p3; 25=ppp13; 26=ppp14_p1; 27=ppp14_p3; 28=ppp14_p5; 29=ppp14_p7; 30=ppp14_p9; 31=ppp14_p11a; 32=ppp14_p13; 33=ppp14_p15; 34=ppp14_p16; 35=ppp14_p17; 36=ppp14_p18;] 
/ itemspacing = 5% 
/ showpagenumbers = false 
/ showquestionnumbers = false
</survey>

<survey PrEP_public_perceptions_CONDITION_2b> 
/ pages = [1=ppp16_p1; 2=ppp16_p2; 3=ppp17_p1; 4=ppp17_p2; 5=ppp18_p1; 6=ppp18_p2; 7=ppp19_condition1.2.3.5;]
/ itemspacing = 5% 
/ showpagenumbers = false 
/ showquestionnumbers = false
</survey>

<survey PrEP_public_perceptions_CONDITION_3a> 
/ pages = [1=ppp1; 2=ppp2; 3=ppp3; 4=ppp4; 5=ppp5; 6=ppp7_condition_3p1; 7=ppp7_condition_3p2; 8=ppp7_condition_3p3; 9=ppp7_condition_3p4; 10=ppp7_condition_3p5; 11=ppp7_condition_3p6; 12=ppp8_condition_3p1; 13=ppp8_condition_3p2; 14=ppp8_condition_3p3; 15=ppp8_condition_3p4; 16=ppp9_p1; 17=ppp9_p2; 18=ppp10_p1; 19=ppp10_p2; 20=ppp10_p3; 21=ppp11; 22=ppp12_p1; 23=ppp12_p2; 24=ppp12_p3; 25=ppp13; 26=ppp14_p1; 27=ppp14_p3; 28=ppp14_p5; 29=ppp14_p7; 30=ppp14_p9; 31=ppp14_p11a; 32=ppp14_p13; 33=ppp14_p15; 34=ppp14_p16; 35=ppp14_p17; 36=ppp14_p18;] 
/ itemspacing = 5% 
/ showpagenumbers = false 
/ showquestionnumbers = false
</survey>

<survey PrEP_public_perceptions_CONDITION_3b> 
/ pages = [1=ppp16_p1; 2=ppp16_p2; 3=ppp17_p1; 4=ppp17_p2; 5=ppp18_p1; 6=ppp18_p2; 7=ppp19_condition1.2.3.5;]
/ itemspacing = 5% 
/ showpagenumbers = false 
/ showquestionnumbers = false
</survey>

<survey PrEP_public_perceptions_CONDITION_4a> 
/ pages = [1=ppp1; 2=ppp2; 3=ppp3; 4=ppp4; 5=ppp5; 6=ppp7_condition_4p1; 7=ppp7_condition_4p2; 8=ppp7_condition_4p3; 9=ppp7_condition_4p4; 10=ppp7_condition_4p5; 11=ppp7_condition_4p6; 12=ppp7_condition_4p7; 13=ppp8_condition_4p1; 14=ppp8_condition_4p2; 15=ppp8_condition_4p3; 16=ppp8_condition_4p4; 17=ppp9_p1; 18=ppp9_p2; 19=ppp10_p1; 20=ppp10_p2; 21=ppp10_p3; 22=ppp11; 23=ppp12_p1; 24=ppp12_p2; 25=ppp12_p3; 26=ppp13; 27=ppp14_p1; 28=ppp14_p3; 29=ppp14_p5; 30=ppp14_p7; 31=ppp14_p9; 32=ppp14_p11a; 33=ppp14_p13; 34=ppp14_p15; 35=ppp14_p16; 36=ppp14_p17; 37=ppp14_p18;] 
/ itemspacing = 5% 
/ showpagenumbers = false 
/ showquestionnumbers = false
</survey>

<survey PrEP_public_perceptions_CONDITION_4b> 
/ pages = [1=ppp16_p1; 2=ppp16_p1a_condition4; 3=ppp16_p2; 4=ppp17_p1; 5=ppp17_p2; 6=ppp18_p1; 7=ppp18_p2; 8=ppp19_condition4;]
/ itemspacing = 5% 
/ showpagenumbers = false 
/ showquestionnumbers = false
</survey>

<survey PrEP_public_perceptions_CONDITION_5a> 
/ pages = [1=ppp1; 2=ppp2; 3=ppp3; 4=ppp4; 5=ppp5; 6=ppp7_condition_5p1; 7=ppp7_condition_5p2; 8=ppp7_condition_5p3; 9=ppp7_condition_5p4; 10=ppp7_condition_5p5; 11=ppp7_condition_5p6; 12=ppp7_condition_5p7; 13=ppp8_condition_5p1; 14=ppp8_condition_5p2; 15=ppp8_condition_5p3; 16=ppp8_condition_5p4; 17=ppp9_p1; 18=ppp9_p2; 19=ppp10_p1; 20=ppp10_p2; 21=ppp10_p3; 22=ppp11; 23=ppp12_p1; 24=ppp12_p2; 25=ppp12_p3; 26=ppp13; 27=ppp14_p1; 28=ppp14_p3; 29=ppp14_p5; 30=ppp14_p7; 31=ppp14_p9; 32=ppp14_p11a; 33=ppp14_p13; 34=ppp14_p15; 35=ppp14_p16; 36=ppp14_p17; 37=ppp14_p18;] 
/ itemspacing = 5% 
/ showpagenumbers = false 
/ showquestionnumbers = false
</survey>

<survey PrEP_public_perceptions_CONDITION_5b> 
/ pages = [1=ppp16_p1; 2=ppp16_p1a_condition5; 3=ppp16_p2; 4=ppp17_p1; 5=ppp17_p2; 6=ppp18_p1; 7=ppp18_p2; 8=ppp19_condition1.2.3.5;]
/ itemspacing = 5% 
/ showpagenumbers = false 
/ showquestionnumbers = false
</survey>


<expt Sexual_Risk_IAT1>
/ subjects = (1 of 2)
/ groupassignment = random
/ blocks = [1=attributepractice_sexrisk; 2=targetcompatiblepractice_sexrisk; 3=compatiblepractice_sexrisk; 4=compatibletestinstructions_sexrisk; 5=compatibletest_sexrisk; 6=targetincompatiblepractice_sexrisk; 7=incompatiblepractice_sexrisk; 8=incompatibletestinstructions_sexrisk; 9=incompatibletest_sexrisk]
/ postinstructions = (summary_sexrisk)
</expt>

<expt Sexual_Risk_IAT2>
/ subjects = (2 of 2)
/ groupassignment = random
/ blocks = [1=attributepractice_sexrisk; 2=targetincompatiblepractice_sexrisk; 3=incompatiblepractice_sexrisk; 4=incompatibletestinstructions_sexrisk; 5=incompatibletest_sexrisk; 6=targetcompatiblepractice_sexrisk; 7=compatiblepractice_sexrisk; 8=compatibletestinstructions_sexrisk; 9=compatibletest_sexrisk]
/ postinstructions = (summary_sexrisk)
</expt>

<expt Race_Prej_IAT1>
/ subjects = (1 of 2)
/ groupassignment = random
/ blocks = [
            1=targetcompatiblepractice_raceprej; 2=attributepractice_raceprej; 3=compatibletest1_raceprej; 4=compatibletestinstructions_raceprej;
            5=compatibletest2_raceprej; 6=targetincompatiblepractice_raceprej; 7=incompatibletest1_raceprej; 8=incompatibletestinstructions_raceprej; 
            9=incompatibletest2_raceprej; 10=summary_raceprej]
</expt>

<expt Race_Prej_IAT2>
/subjects = (2 of 2)
/ groupassignment = random
/ blocks = [
            1=targetincompatiblepractice_raceprej; 2=attributepractice_raceprej; 3=incompatibletest1_raceprej; 4=incompatibletestinstructions_raceprej;
            5=incompatibletest2_raceprej; 6=targetcompatiblepractice_raceprej; 7=compatibletest1_raceprej; 8=compatibletestinstructions_raceprej; 
            9=compatibletest2_raceprej; 10=summary_raceprej]
</expt>


Dave
Dave
Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)
Group: Administrators
Posts: 10K, Visits: 59K
I would argue that (a) separating the various parts into individual scripts and (b) using <batch> elements to administer the desired order / do condition assignment is the best way to handle this. I.e.

<batch>
/ subjects = (1,2 of 10)
/ groupassignment = groupnumber
/ file = "start_survey_condition_1.iqx"
/ file = "IAT_1.iqx"
/ file = "IAT_2.iqx"
/ file = "end_survey_condition_1.iqx"
</batch>

<batch>
/ subjects = (3,4 of 10)
/ groupassignment = groupnumber
/ file = "start_survey_condition_2.iqx"
/ file = "IAT_1.iqx"
/ file = "IAT_2.iqx"
/ file = "end_survey_condition_2.iqx"
</batch>

...

<batch>
/ subjects = (9,10 of 10)
/ groupassignment = groupnumber
/ file = "start_survey_condition_5.iqx"
/ file = "IAT_1.iqx"
/ file = "IAT_2.iqx"
/ file = "end_survey_condition_5.iqx"
</batch>

> Another complicating factor is within each IAT, I have another 2 conditions--one where the compatible test is received first and one where the incompatible test is received first.  Is there an easier way to change this order within the IAT so it is randomly assigned?

Yes, if you want to you can set the /groupassignment attributes to 'random' instead of 'groupnumber' or 'subjectnumber'.

valengrandelski
valengrandelski
Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)
Group: Forum Members
Posts: 10, Visits: 76
Thanks so much, Dave!  This is super helpful.  A few follow-up questions:

  • If I use /groupassignment = random both within each IAT and in the batch elements, is this counterbalanced?  Will there be an equal number of participants in each condition?  If not, how do I ensure that it's counterbalanced?

  • For using batch elements, I know I have to separate the components into individual scripts, so if I want to circulate the file for testing to colleagues, do they need ALL the scripts or just the one containing the batch elements?  What's the easiest way to package the files?

  • Finally, for uploading my experiment to Inquisit Web, am I allowed to upload multiple scripts or can I just upload the script with the batch elements?  How would that work?

Dave
Dave
Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)
Group: Administrators
Posts: 10K, Visits: 59K
Q: If I use /groupassignment = random both within each IAT and in the batch elements, is this counterbalanced?  Will there be an equal number of participants in each condition?  If not, how do I ensure that it's counterbalanced?

A: No, it's indeed completely random, not counterbalanced. How do you counterbalance? Use groupnumber and assign an equal number of participants to those groups (e.g. on the web pass groupnumber in as a query parameter and distribute different links among your participants).

Q:
For using batch elements, I know I have to separate the components into individual scripts, so if I want to circulate the file for testing to colleagues, do they need ALL the scripts or just the one containing the batch elements?  What's the easiest way to package the files?

A: They need all the scripts and supporting files (e.g. images, etc.). If your colleagues conduct their tests in Inquisit Lab, put everything in a ZIP archive (or the like) and email it to them or put it up somewhere for them to download. If they test using Inquisit Web, simply upload all the files and configure everything properly via the Web Script Wizard and distribute the link(s) to the experiment's launch page.

Q:
Finally, for uploading my experiment to Inquisit Web, am I allowed to upload multiple scripts or can I just upload the script with the batch elements?  How would that work?

A: The script containing the <batch> elements is the one you register as your active script in the 1st step. You upload all the remaining (actual) scripts and supporting files in the 2nd step.

valengrandelski
valengrandelski
Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)
Group: Forum Members
Posts: 10, Visits: 76
Ok, so it sounds like Inquisit Web takes care of the having multiple scripts thing.

So, if I set the / groupassignment = groupnumber in order to counterbalance, Inquisit Web will automatically generate different links for the different groups/conditions, correct?  So, if I have 5 conditions, will there be 5 different survey links?  Is there any way to have 1 link or no?

Another / groupassignment question:  if I set it to "groupnumber" within each IAT script (to switch order of compatible vs. incompatible received first by participants), would that generate additional links/conditions or no?

Also, taking another look at the batch elements coding, I realized that I don't understand why / subjects = (1,2 of 10) instead of (1 of 10), if I want to flip the order of the IATs.  Can you clarify?  Should I write it like this:

<batch>
/ subjects = (1 of 10)
/ groupassignment = groupnumber
/ file = "start_survey_condition_1.iqx"
/ file = "IAT_1.iqx"
/ file = "IAT_2.iqx"
/ file = "end_survey_condition_1.iqx"
</batch>

<batch>
/ subjects = (2 of 10)
/ groupassignment = groupnumber
/ file = "start_survey_condition_1.iqx"
/ file = "IAT_2.iqx"
/ file = "IAT_1.iqx"
/ file = "end_survey_condition_1.iqx"
</batch>

<batch>
/ subjects = (3 of 10)
/ groupassignment = groupnumber
/ file = "start_survey_condition_2.iqx"
/ file = "IAT_1.iqx"
/ file = "IAT_2.iqx"
/ file = "end_survey_condition_2.iqx"
</batch> 

<batch>
/ subjects = (4 of 10)
/ groupassignment = groupnumber
/ file = "start_survey_condition_2.iqx"
/ file = "IAT_2.iqx"
/ file = "IAT_1.iqx"
/ file = "end_survey_condition_2.iqx"
</batch> 

Dave
Dave
Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)
Group: Administrators
Posts: 10K, Visits: 59K
> So, if I set the / groupassignment = groupnumber in order to counterbalance, Inquisit Web will automatically generate different links for the different groups/conditions, correct?

No. What you can do is specify the groupnumber via a URL query parameter (you do so when you upload and configure the experiment). You can then explicitly distribute links containing the query parameter to your participants. Suppose you have three conditions. Then you would give a 3rd of your participants a link looking like this

http://research.millisecond.com/username/experiment.web?condition=1

another 3rd

http://research.millisecond.com/username/experiment.web?condition=2

and the final 3rd

http://research.millisecond.com/username/experiment.web?condition=3

> Another / groupassignment question:  if I set it to "groupnumber" within each IAT script (to switch order of compatible vs. incompatible received first by participants), would that generate additional links/conditions or no?

No.

<batch>
/ subjects = (1 of 10)
/ groupassignment = groupnumber
/ file = "start_survey_condition_1.iqx"
/ file = "IAT_1.iqx"
/ file = "IAT_2.iqx"
/ file = "end_survey_condition_1.iqx"
</batch>

<batch>
/ subjects = (2 of 10)
/ groupassignment = groupnumber
/ file = "start_survey_condition_1.iqx"
/ file = "IAT_2.iqx"
/ file = "IAT_1.iqx"
/ file = "end_survey_condition_1.iqx"
</batch>
...
Yes. If you want to flip the order of IATs in addition, that's what you do. (You did not mention that in your original post.) However, if you leave the IATs set to /groupassingnment=groupnumber you need to make sure to feed both odd end even numbers in -- otherwise the order of  blocks *within* the IAT will not be counterbalanced between subjects. That's why the <batch> elements I provided as examples were set up in the way (1,2 of X) etc.

valengrandelski
valengrandelski
Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)
Group: Forum Members
Posts: 10, Visits: 76
Ok, I have a changed a few things in my script and I'm using Inquisit Web to collect my data.

In 1 script, I have 20 <expt> elements and I have /groupassignment = group number in order to counterbalance.  For each expt, /subjects = (1 of 20), (2 of 20), etc.

When I upload to Inquisit Web, how should the groupID be determined?  Sequential?  Random with replacement?  All I want is an equal number of participants (total sample size 500) in each of my 20 expts.  I set the subjectIDs to be determined sequentially.

Advice?  Thanks so much!

Dave
Dave
Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)
Group: Administrators
Posts: 10K, Visits: 59K
For the most control over group Ns, pass the groupid in via query parameter and give 1/20th of your participants a link with the parameter set to 1, 1/20th with parameter set to 2, etc.

For all other options, there is no strict guarantee that group Ns will be equal. Sequential generation seems like an okay compromise, though you should expect some gaps.

Hope this helps.

valengrandelski
valengrandelski
Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)Expert (1.2K reputation)
Group: Forum Members
Posts: 10, Visits: 76
I'm sorry, but I don't know how to do what you're suggesting.

Once I choose "URL Query Parameter" for groupID, what should I enter for this prompt:  Enter the name of the url query parameter that contains the group id. 
Not sure about that one...
How does it know to create 20 links?
Will the links appear on my Web account, just underneath the script?


We might just go with sequential with the understanding it might not be equal across all groups.

Dave
Dave
Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)Supreme Being (854K reputation)
Group: Administrators
Posts: 10K, Visits: 59K
We've already covered the query parameter scenario in this thread:

https://www.millisecond.com/forums/FindPost13899.aspx

It does not create separate links -- all that differs is the value of the parameter.

GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Reading This Topic

Explore
Messages
Mentions
Search