Issues with counterbalancing and sequential groupid


Author
Message
annievu94
annievu94
New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)
Group: Forum Members
Posts: 4, Visits: 22
I linked my study with SONA so I set my participants ID as URL parameter and groupid as sequential, hoping that students will be counterbalanced into 1 of the 8 conditions per the sequence that they click on the link (so each of my condition will be counterbalanced and has even number of participants). However, the order is somewhat random? Please see attached and advise how I should proceed! 
Attachments
Dave
Dave
Supreme Being (871K reputation)Supreme Being (871K reputation)Supreme Being (871K reputation)Supreme Being (871K reputation)Supreme Being (871K reputation)Supreme Being (871K reputation)Supreme Being (871K reputation)Supreme Being (871K reputation)Supreme Being (871K reputation)
Group: Administrators
Posts: 10K, Visits: 62K
annievu94 - 11/21/2020
I linked my study with SONA so I set my participants ID as URL parameter and groupid as sequential, hoping that students will be counterbalanced into 1 of the 8 conditions per the sequence that they click on the link (so each of my condition will be counterbalanced and has even number of participants). However, the order is somewhat random? Please see attached and advise how I should proceed! 

Sort your logs ascending by Event Time and it won't look so random:



Note that a group number will be generated whenever someone visits the study's start page, regardless of whether that participant then starts and completes the study at that point or not. That is why you will see gaps between the sequential IDs (e.g. 3 and 4 missing between 2 and 5 in the above).

annievu94
annievu94
New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)
Group: Forum Members
Posts: 4, Visits: 22
Thank you Dave for the clarification!
So does set the group setting as either Randomization With Replacement or Randomization Without Replacement will solve this problem? I just want equal sample sizes for each condition! 
Dave
Dave
Supreme Being (871K reputation)Supreme Being (871K reputation)Supreme Being (871K reputation)Supreme Being (871K reputation)Supreme Being (871K reputation)Supreme Being (871K reputation)Supreme Being (871K reputation)Supreme Being (871K reputation)Supreme Being (871K reputation)
Group: Administrators
Posts: 10K, Visits: 62K
annievu94 - 11/23/2020
Thank you Dave for the clarification!
So does set the group setting as either Randomization With Replacement or Randomization Without Replacement will solve this problem? I just want equal sample sizes for each condition! 

Neither will solve the problem. You're going to have dropouts or people who take a peak at the start page, but don't start the experiment right then and there and instead return later. That will lead to imbalances no matter which group number generation method you choose.
annievu94
annievu94
New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)
Group: Forum Members
Posts: 4, Visits: 22
Dave - 11/23/2020
annievu94 - 11/23/2020
Thank you Dave for the clarification!
So does set the group setting as either Randomization With Replacement or Randomization Without Replacement will solve this problem? I just want equal sample sizes for each condition! 

Neither will solve the problem. You're going to have dropouts or people who take a peak at the start page, but don't start the experiment right then and there and instead return later. That will lead to imbalances no matter which group number generation method you choose.

Thank you Dave!

Last question, since this is my first time running online experiment on Inquisit and I couldn't find the answer on the forum, what is the difference between "Random Assignment With Replacement" and "Random Assignment without Replacement"?
Dave
Dave
Supreme Being (871K reputation)Supreme Being (871K reputation)Supreme Being (871K reputation)Supreme Being (871K reputation)Supreme Being (871K reputation)Supreme Being (871K reputation)Supreme Being (871K reputation)Supreme Being (871K reputation)Supreme Being (871K reputation)
Group: Administrators
Posts: 10K, Visits: 62K
annievu94 - 11/23/2020
Dave - 11/23/2020
annievu94 - 11/23/2020
Thank you Dave for the clarification!
So does set the group setting as either Randomization With Replacement or Randomization Without Replacement will solve this problem? I just want equal sample sizes for each condition! 

Neither will solve the problem. You're going to have dropouts or people who take a peak at the start page, but don't start the experiment right then and there and instead return later. That will lead to imbalances no matter which group number generation method you choose.

Thank you Dave!

Last question, since this is my first time running online experiment on Inquisit and I couldn't find the answer on the forum, what is the difference between "Random Assignment With Replacement" and "Random Assignment without Replacement"?

Same thing as drawing from a deck of cards (conditions) with or without putting the drawn card back into the deck after the draw.

Suppose you have 4 groups. Then, for every set of 4 participants that hit the start page, random assignment without replacement will assign one of those participants to group 1, one to group 2, one to group 3, and one to group 4.

With random assignment with replacement, every participant that hits the start page has en equal chance of being assigned to any of the 4 groups, essentially as if flipping a coin.

annievu94
annievu94
New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)New Member (11 reputation)
Group: Forum Members
Posts: 4, Visits: 22
Thank you so much Dave!!! I really appreciate all the help!!!
GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Reading This Topic

Explore
Messages
Mentions
Search