|
AnthonyFCollinsSussex
|
|
|
Group: Forum Members
Posts: 52,
Visits: 249
|
+x+x+x+x+x+xOne of those days, my bad :// order RTA -> Positive IBT -> RTB / subjects = (1 of 4)/ groupassignment = groupnumber / file = "RTA.iqx"/ file = "Positive IBT.iqx"/ file = "RTB.iqx// order RTA -> Control IBT -> RTB / subjects = (2 of 4)/ groupassignment = groupnumber/ file = "RTA.iqx"/ file = "Control IBT.iqx"/ file = "RTB.iqx// order RTB -> Positive IBT -> RTA / subjects = (3 of 4)/ groupassignment = groupnumber/ file = "RTB.iqx"/ file = "Positive IBT.iqx"/ file = "RTA.iqx// order RTB -> Control IBT -> RTA / subjects = (4 of 4)/ groupassignment = groupnumber/ file = "RTB.iqx"/ file = "Control IBT.iqx"/ file = "RTA.iqxThis is to ensure that we can identify that positive IBT does exert an effect that is not confounded by materials Set presentation (to rule out a material presentation effect, in this case A and B, we very much don’t want this , hence the above Okay, that looks correct to me now. Group ID will be logged in the data files (unless you deliberatey excluded it), and you can derive order (RTA first vs RTB first) from that. As for how condition assignment works generally, see https://www.millisecond.com/forums/Topic13856.aspx Hello Dave Hope all is well. So it turns out this set-up is not quite right and may still impose an element of order effect (as it is sequential). I need to check whether the following is implementable within Inquisit, or whether this needs to be done in Qualtrics . Given there are 4 'conditions' in the above, we need to create the following array, where the length is determined by maximum number of participants (we hope this to be 150). I have gone for 20 below for sake of ease: Condition = [ 1 3 2 4 4 3 1 2 2 3 4 1 1 2 3 4 4 2 1 3] The condition for each participant would be based on the order in which they are run, such that the first participant would be allocated to condition 1 (the first number in the condition array), the second person would be allocate to condition 3 (the second number in the condition array) etc. the array is constructed such that the order of every 4 participants is random but includes one in each condition i.e. above you have 1 3 2 4 followed by 4 3 1 2 etc. In this way you know that provided you run a multiple of 4 participants you will always have an equal number in each of the 4 conditions. Is there a way I can achieve this? If you select random (without replacement) group number generation in the Inquisit Web experiment's settings and indicate the number of groups as 4, then for each set of four participants that visit the start page, one will be randomly assigned to condition 1, one will be randomly assigned to condition 2, one to condition 3, one to condition 4. If this is not to your liking, you can have Qualtrics perform the condition assignment and have it send group ID over via URL parameter, just like subject ID. Sounds like this set-up achieves what we need it to, in other words participant 1 could be assigned to any one of the 4, let's say at random, 3rd condition, the second participant gets assigned to the 1st condition, and then the final two participants will be randomly allocated to one of the final remaining conditions (e.g. 2nd and 4th condition in this case). Then this randomizing process restarts for the next 4 participants (with a different randomised order/allocation) and so on. If I have understood this correctly, then we should not need Qualtrics, and this achieves the array above (random 50% of each training condition get RTA followed by RTB, and the other 50% the other way around) Thank you Dave again, this is very much appreciated > in other words participant 1 could be assigned to any one of the 4, let's say at random, 3rd condition, the second participant gets assigned to the 1st condition, and then the final two participants will be randomly allocated to one of the final remaining conditions (e.g. 2nd and 4th condition in this case) Yes, that's how it works. Note, though, that group ID generation happens when the participant visits the start page, regardless of whether that particpant ends up actually launching the study at that point or not. So you will see gaps due to dropouts, that's just a reality of online research Hello Dave I hope you are well and that you have had a pleasant week so far. I do need to check the algorithm concerning random without replacement on the Inquisit web-player. Just to re-cap: My batch is the following: <batch> / subjects = (1 of 4) / groupassignment = groupnumber / file = RTA.iqx / file = Positive IBT.iqx / file = RTB.iqx </batch> <batch> / subjects = (2 of 4) / groupassignment = groupnumber / file = RTB.iqx / file = Control IBT.iqx / file = RTA.iqx </batch> / subjects = (3 of 4) / groupassignment = groupnumber / file = RTB.iqx / file = Positive IBT.iqx / file = RTA.iqx </batch> / subjects = (4 of 4) / groupassignment = groupnumber / file = RTA.iqx / file = Control IBT.iqx / file = RTB.iqx </batch> <defaults> / screencolor = white / txcolor = black / txbgcolor = white / fontstyle = ("Arial", 2.8%) </defaults> <data> /separatefiles = true </data> Thus, the order of every 4 participants is random but includes one in each condition, and therefore each condition ought only be summoned once in every set of 4 participants (array). To do so, accordingly as instructed above, I ensured that I selected random (without replacement) group number generation, as well as for subject ID generation, in the Inquisit Web experiment's settings, and indicated the number of groups as 4. I conducted some runs to ensure the allocation worked correctly. This simply entails me ctrl+Q'ing my through my experiment each time (for speed) and checking one of my script's group id variable in the data output. I have so far conducted many runs and found some weird results. This is the first lot when I first uploaded the experiment. 1, 3, 2, 4 2, 2, 2, 2, <-- I now realise that this was because I stupidly used the same start-up screen for those 4 consecutive trials. 3, 1, 4, 1 3, 2, 2, 4 2, 1, 3, 4 2, 1, 3, 4 3, 2, 1, 4 2, 3, 1, 4 2, 3, 1, 4 1, 3, 2, 4 1, 2, 3, 4 2, 3, 1, 4 1, 2, 3, 4 2, 1, 3, 4 2, 3, 1, 4 After unregistering/deleting the exp, and merely re-uploading the exact same one again, I obtained the following: 2, 1, 3, 4 2, 1, 3, 4 3, 2, 1, 4 3, 2, 1, 4 3, 1, 2, 4 1, 2, 3, 4 2, 1, 3, 4 3, 2, 4, 1 I proceeded to upload the same experiment again, and finally got the following: 3, 1, 2, 4 3, 2, 1, 4 3, 2, 1, 4 1, 3, 2, 4 3, 1, 4, 3 <-- On the last turn here, I took a longer pause from the previous launch (4th condition), and weirdly...it used the same condition as at the start, 3 2, 1, 4, 2 <-- And again the same as above, merely for a slightly longer time interval between run 3 and run 4. Finally, I notice from all my test-runs, not one of them starts with the 4th condition. Is that coincidental? Is there something in the algorithm that is leading to this? There is nothing in my code to suggest why this is happening, and I am genuinely very stumped! I just wondered if these 'random' sequences looked normal. Thank you very much in advance... Anthony
|
|
|
|
|
Dave
|
|
|
Group: Administrators
Posts: 12K,
Visits: 98K
|
+x+x+x+x+x+x+xOne of those days, my bad :// order RTA -> Positive IBT -> RTB / subjects = (1 of 4)/ groupassignment = groupnumber / file = "RTA.iqx"/ file = "Positive IBT.iqx"/ file = "RTB.iqx// order RTA -> Control IBT -> RTB / subjects = (2 of 4)/ groupassignment = groupnumber/ file = "RTA.iqx"/ file = "Control IBT.iqx"/ file = "RTB.iqx// order RTB -> Positive IBT -> RTA / subjects = (3 of 4)/ groupassignment = groupnumber/ file = "RTB.iqx"/ file = "Positive IBT.iqx"/ file = "RTA.iqx// order RTB -> Control IBT -> RTA / subjects = (4 of 4)/ groupassignment = groupnumber/ file = "RTB.iqx"/ file = "Control IBT.iqx"/ file = "RTA.iqxThis is to ensure that we can identify that positive IBT does exert an effect that is not confounded by materials Set presentation (to rule out a material presentation effect, in this case A and B, we very much don’t want this , hence the above Okay, that looks correct to me now. Group ID will be logged in the data files (unless you deliberatey excluded it), and you can derive order (RTA first vs RTB first) from that. As for how condition assignment works generally, see https://www.millisecond.com/forums/Topic13856.aspx Hello Dave Hope all is well. So it turns out this set-up is not quite right and may still impose an element of order effect (as it is sequential). I need to check whether the following is implementable within Inquisit, or whether this needs to be done in Qualtrics . Given there are 4 'conditions' in the above, we need to create the following array, where the length is determined by maximum number of participants (we hope this to be 150). I have gone for 20 below for sake of ease: Condition = [ 1 3 2 4 4 3 1 2 2 3 4 1 1 2 3 4 4 2 1 3] The condition for each participant would be based on the order in which they are run, such that the first participant would be allocated to condition 1 (the first number in the condition array), the second person would be allocate to condition 3 (the second number in the condition array) etc. the array is constructed such that the order of every 4 participants is random but includes one in each condition i.e. above you have 1 3 2 4 followed by 4 3 1 2 etc. In this way you know that provided you run a multiple of 4 participants you will always have an equal number in each of the 4 conditions. Is there a way I can achieve this? If you select random (without replacement) group number generation in the Inquisit Web experiment's settings and indicate the number of groups as 4, then for each set of four participants that visit the start page, one will be randomly assigned to condition 1, one will be randomly assigned to condition 2, one to condition 3, one to condition 4. If this is not to your liking, you can have Qualtrics perform the condition assignment and have it send group ID over via URL parameter, just like subject ID. Sounds like this set-up achieves what we need it to, in other words participant 1 could be assigned to any one of the 4, let's say at random, 3rd condition, the second participant gets assigned to the 1st condition, and then the final two participants will be randomly allocated to one of the final remaining conditions (e.g. 2nd and 4th condition in this case). Then this randomizing process restarts for the next 4 participants (with a different randomised order/allocation) and so on. If I have understood this correctly, then we should not need Qualtrics, and this achieves the array above (random 50% of each training condition get RTA followed by RTB, and the other 50% the other way around) Thank you Dave again, this is very much appreciated > in other words participant 1 could be assigned to any one of the 4, let's say at random, 3rd condition, the second participant gets assigned to the 1st condition, and then the final two participants will be randomly allocated to one of the final remaining conditions (e.g. 2nd and 4th condition in this case) Yes, that's how it works. Note, though, that group ID generation happens when the participant visits the start page, regardless of whether that particpant ends up actually launching the study at that point or not. So you will see gaps due to dropouts, that's just a reality of online research Hello Dave I hope you are well and that you have had a pleasant week so far. I do need to check the algorithm concerning random without replacement on the Inquisit web-player. Just to re-cap: My batch is the following: <batch> / subjects = (1 of 4) / groupassignment = groupnumber / file = RTA.iqx / file = Positive IBT.iqx / file = RTB.iqx </batch> <batch> / subjects = (2 of 4) / groupassignment = groupnumber / file = RTB.iqx / file = Control IBT.iqx / file = RTA.iqx </batch> / subjects = (3 of 4) / groupassignment = groupnumber / file = RTB.iqx / file = Positive IBT.iqx / file = RTA.iqx </batch> / subjects = (4 of 4) / groupassignment = groupnumber / file = RTA.iqx / file = Control IBT.iqx / file = RTB.iqx </batch> <defaults> / screencolor = white / txcolor = black / txbgcolor = white / fontstyle = ("Arial", 2.8%) </defaults> <data> /separatefiles = true </data> Thus, the order of every 4 participants is random but includes one in each condition, and therefore each condition ought only be summoned once in every set of 4 participants (array). To do so, accordingly as instructed above, I ensured that I selected random (without replacement) group number generation, as well as for subject ID generation, in the Inquisit Web experiment's settings, and indicated the number of groups as 4. I conducted some runs to ensure the allocation worked correctly. This simply entails me ctrl+Q'ing my through my experiment each time (for speed) and checking one of my script's group id variable in the data output. I have so far conducted many runs and found some weird results. This is the first lot when I first uploaded the experiment. 1, 3, 2, 4 2, 2, 2, 2, <-- I now realise that this was because I stupidly used the same start-up screen for those 4 consecutive trials. 3, 1, 4, 1 3, 2, 2, 4 2, 1, 3, 4 2, 1, 3, 4 3, 2, 1, 4 2, 3, 1, 4 2, 3, 1, 4 1, 3, 2, 4 1, 2, 3, 4 2, 3, 1, 4 1, 2, 3, 4 2, 1, 3, 4 2, 3, 1, 4 After unregistering/deleting the exp, and merely re-uploading the exact same one again, I obtained the following: 2, 1, 3, 4 2, 1, 3, 4 3, 2, 1, 4 3, 2, 1, 4 3, 1, 2, 4 1, 2, 3, 4 2, 1, 3, 4 3, 2, 4, 1 I proceeded to upload the same experiment again, and finally got the following: 3, 1, 2, 4 3, 2, 1, 4 3, 2, 1, 4 1, 3, 2, 4 3, 1, 4, 3 <-- On the last turn here, I took a longer pause from the previous launch (4th condition), and weirdly...it used the same condition as at the start, 3 2, 1, 4, 2 <-- And again the same as above, merely for a slightly longer time interval between run 3 and run 4. Finally, I notice from all my test-runs, not one of them starts with the 4th condition. Is that coincidental? Is there something in the algorithm that is leading to this? There is nothing in my code to suggest why this is happening, and I am genuinely very stumped! I just wondered if these 'random' sequences looked normal. Thank you very much in advance... Anthony I fail to see any issue here. As explained previously, any visit to the start page, by yourself or somebody else, will trigger generation of a group number, as will any extraneous refresh of the page. In the given set of 4, that ID then is out of the picture and will only re-occur in the next set of 4.
|
|
|
|
|
AnthonyFCollinsSussex
|
|
|
Group: Forum Members
Posts: 52,
Visits: 249
|
+x+x+x+x+x+x+x+xOne of those days, my bad :// order RTA -> Positive IBT -> RTB / subjects = (1 of 4)/ groupassignment = groupnumber / file = "RTA.iqx"/ file = "Positive IBT.iqx"/ file = "RTB.iqx// order RTA -> Control IBT -> RTB / subjects = (2 of 4)/ groupassignment = groupnumber/ file = "RTA.iqx"/ file = "Control IBT.iqx"/ file = "RTB.iqx// order RTB -> Positive IBT -> RTA / subjects = (3 of 4)/ groupassignment = groupnumber/ file = "RTB.iqx"/ file = "Positive IBT.iqx"/ file = "RTA.iqx// order RTB -> Control IBT -> RTA / subjects = (4 of 4)/ groupassignment = groupnumber/ file = "RTB.iqx"/ file = "Control IBT.iqx"/ file = "RTA.iqxThis is to ensure that we can identify that positive IBT does exert an effect that is not confounded by materials Set presentation (to rule out a material presentation effect, in this case A and B, we very much don’t want this , hence the above Okay, that looks correct to me now. Group ID will be logged in the data files (unless you deliberatey excluded it), and you can derive order (RTA first vs RTB first) from that. As for how condition assignment works generally, see https://www.millisecond.com/forums/Topic13856.aspx Hello Dave Hope all is well. So it turns out this set-up is not quite right and may still impose an element of order effect (as it is sequential). I need to check whether the following is implementable within Inquisit, or whether this needs to be done in Qualtrics . Given there are 4 'conditions' in the above, we need to create the following array, where the length is determined by maximum number of participants (we hope this to be 150). I have gone for 20 below for sake of ease: Condition = [ 1 3 2 4 4 3 1 2 2 3 4 1 1 2 3 4 4 2 1 3] The condition for each participant would be based on the order in which they are run, such that the first participant would be allocated to condition 1 (the first number in the condition array), the second person would be allocate to condition 3 (the second number in the condition array) etc. the array is constructed such that the order of every 4 participants is random but includes one in each condition i.e. above you have 1 3 2 4 followed by 4 3 1 2 etc. In this way you know that provided you run a multiple of 4 participants you will always have an equal number in each of the 4 conditions. Is there a way I can achieve this? If you select random (without replacement) group number generation in the Inquisit Web experiment's settings and indicate the number of groups as 4, then for each set of four participants that visit the start page, one will be randomly assigned to condition 1, one will be randomly assigned to condition 2, one to condition 3, one to condition 4. If this is not to your liking, you can have Qualtrics perform the condition assignment and have it send group ID over via URL parameter, just like subject ID. Sounds like this set-up achieves what we need it to, in other words participant 1 could be assigned to any one of the 4, let's say at random, 3rd condition, the second participant gets assigned to the 1st condition, and then the final two participants will be randomly allocated to one of the final remaining conditions (e.g. 2nd and 4th condition in this case). Then this randomizing process restarts for the next 4 participants (with a different randomised order/allocation) and so on. If I have understood this correctly, then we should not need Qualtrics, and this achieves the array above (random 50% of each training condition get RTA followed by RTB, and the other 50% the other way around) Thank you Dave again, this is very much appreciated > in other words participant 1 could be assigned to any one of the 4, let's say at random, 3rd condition, the second participant gets assigned to the 1st condition, and then the final two participants will be randomly allocated to one of the final remaining conditions (e.g. 2nd and 4th condition in this case) Yes, that's how it works. Note, though, that group ID generation happens when the participant visits the start page, regardless of whether that particpant ends up actually launching the study at that point or not. So you will see gaps due to dropouts, that's just a reality of online research Hello Dave I hope you are well and that you have had a pleasant week so far. I do need to check the algorithm concerning random without replacement on the Inquisit web-player. Just to re-cap: My batch is the following: <batch> / subjects = (1 of 4) / groupassignment = groupnumber / file = RTA.iqx / file = Positive IBT.iqx / file = RTB.iqx </batch> <batch> / subjects = (2 of 4) / groupassignment = groupnumber / file = RTB.iqx / file = Control IBT.iqx / file = RTA.iqx </batch> / subjects = (3 of 4) / groupassignment = groupnumber / file = RTB.iqx / file = Positive IBT.iqx / file = RTA.iqx </batch> / subjects = (4 of 4) / groupassignment = groupnumber / file = RTA.iqx / file = Control IBT.iqx / file = RTB.iqx </batch> <defaults> / screencolor = white / txcolor = black / txbgcolor = white / fontstyle = ("Arial", 2.8%) </defaults> <data> /separatefiles = true </data> Thus, the order of every 4 participants is random but includes one in each condition, and therefore each condition ought only be summoned once in every set of 4 participants (array). To do so, accordingly as instructed above, I ensured that I selected random (without replacement) group number generation, as well as for subject ID generation, in the Inquisit Web experiment's settings, and indicated the number of groups as 4. I conducted some runs to ensure the allocation worked correctly. This simply entails me ctrl+Q'ing my through my experiment each time (for speed) and checking one of my script's group id variable in the data output. I have so far conducted many runs and found some weird results. This is the first lot when I first uploaded the experiment. 1, 3, 2, 4 2, 2, 2, 2, <-- I now realise that this was because I stupidly used the same start-up screen for those 4 consecutive trials. 3, 1, 4, 1 3, 2, 2, 4 2, 1, 3, 4 2, 1, 3, 4 3, 2, 1, 4 2, 3, 1, 4 2, 3, 1, 4 1, 3, 2, 4 1, 2, 3, 4 2, 3, 1, 4 1, 2, 3, 4 2, 1, 3, 4 2, 3, 1, 4 After unregistering/deleting the exp, and merely re-uploading the exact same one again, I obtained the following: 2, 1, 3, 4 2, 1, 3, 4 3, 2, 1, 4 3, 2, 1, 4 3, 1, 2, 4 1, 2, 3, 4 2, 1, 3, 4 3, 2, 4, 1 I proceeded to upload the same experiment again, and finally got the following: 3, 1, 2, 4 3, 2, 1, 4 3, 2, 1, 4 1, 3, 2, 4 3, 1, 4, 3 <-- On the last turn here, I took a longer pause from the previous launch (4th condition), and weirdly...it used the same condition as at the start, 3 2, 1, 4, 2 <-- And again the same as above, merely for a slightly longer time interval between run 3 and run 4. Finally, I notice from all my test-runs, not one of them starts with the 4th condition. Is that coincidental? Is there something in the algorithm that is leading to this? There is nothing in my code to suggest why this is happening, and I am genuinely very stumped! I just wondered if these 'random' sequences looked normal. Thank you very much in advance... Anthony I fail to see any issue here. As explained previously, any visit to the start page, by yourself or somebody else, will trigger generation of a group number, as will any extraneous refresh of the page. In the given set of 4, that ID then is out of the picture and will only re-occur in the next set of 4. It just looks as though for example condition 4 is always being landed at the end of most sets, and I was wondering if the algorithm was truly random, should this happen, as it looks so systematic
|
|
|
|