Millisecond Forums

+x

 

sparklab - 7/6/2021

I'm running a study on MTurk and am directing participants to Qualtrics first and then to Inquisit Web. On the last page of the Inquisit survey, participants see a code that they have to put into MTurk.

Almost everything is going without a hitch, but some responses are puzzling. I'm having trouble interpreting my study log. What do the launch, license, data, and finish event times mean?

1) What could be happening if someone has a finish timestamp but only 1 of the 2 data files?

2) What could be happening if someone has a finish timestamp but no data files?

3) Is it correct to assume that if someone only has a launch or license timestamp, they did not complete the study?

launch: Time(s) the start button on the experiment's start page was clicked. Note this does not mean the study was actually launched. Clicking the start button without having the player installed (as per the instructions on the start page) will do nothing.
license: time the player executed the license check. This, then, at least means the player was installed.
finish: time redirection to the specified finish page occurred.

> 1) What could be happening if someone has a finish timestamp but only 1 of the 2 data files?

Participant quit the study before one of the data files was created or an upload problem. Any not yet uploaded data files remain on the device and can be manually uploaded via the player's "Data" tab.

> 2) What could be happening if someone has a finish timestamp but no data files?

Quitting the study before there was any data, upload problem, or disabled automatic data file upload in the player (controls for that are on the "Data" tab in the player app).

> 3) Is it correct to assume that if someone only has a launch or license timestamp, they did not complete the study?

Typically yes, although there can be edge cases (e.g. internet connection went down while the participant was completing the scripts -- they do not require persistent internet connectivity), so redirection to the finish page may not have occurred or could not be logged.

+x

 

Dave - 7/6/2021

+x

 

sparklab - 7/6/2021

I'm running a study on MTurk and am directing participants to Qualtrics first and then to Inquisit Web. On the last page of the Inquisit survey, participants see a code that they have to put into MTurk.

Almost everything is going without a hitch, but some responses are puzzling. I'm having trouble interpreting my study log. What do the launch, license, data, and finish event times mean?

1) What could be happening if someone has a finish timestamp but only 1 of the 2 data files?

2) What could be happening if someone has a finish timestamp but no data files?

3) Is it correct to assume that if someone only has a launch or license timestamp, they did not complete the study?

launch: Time(s) the start button on the experiment's start page was clicked. Note this does not mean the study was actually launched. Clicking the start button without having the player installed (as per the instructions on the start page) will do nothing.
license: time the player executed the license check. This, then, at least means the player was installed.
finish: time redirection to the specified finish page occurred.

> 1) What could be happening if someone has a finish timestamp but only 1 of the 2 data files?

Participant quit the study before one of the data files was created or an upload problem. Any not yet uploaded data files remain on the device and can be manually uploaded via the player's "Data" tab.

> 2) What could be happening if someone has a finish timestamp but no data files?

Quitting the study before there was any data, upload problem, or disabled automatic data file upload in the player (controls for that are on the "Data" tab in the player app).

> 3) Is it correct to assume that if someone only has a launch or license timestamp, they did not complete the study?

Typically yes, although there can be edge cases (e.g. internet connection went down while the participant was completing the scripts -- they do not require persistent internet connectivity), so redirection to the finish page may not have occurred or could not be logged.

Thank you! Do participants generally know how to quit the study? I assume that the data will not be created if they do not click "continue" on the very last page of my script?

Do people have to manually go in and disable automatic data file upload (is it automatically turned on when people install the player)?

This isn't happening for a lot of people, but I want to make sure I adjust my instructions so that people will know in advance what they need to do/need not to do for study compensation.

+x

 

sparklab - 7/6/2021

+x

 

Dave - 7/6/2021

+x

 

sparklab - 7/6/2021

I'm running a study on MTurk and am directing participants to Qualtrics first and then to Inquisit Web. On the last page of the Inquisit survey, participants see a code that they have to put into MTurk.

Almost everything is going without a hitch, but some responses are puzzling. I'm having trouble interpreting my study log. What do the launch, license, data, and finish event times mean?

1) What could be happening if someone has a finish timestamp but only 1 of the 2 data files?

2) What could be happening if someone has a finish timestamp but no data files?

3) Is it correct to assume that if someone only has a launch or license timestamp, they did not complete the study?

launch: Time(s) the start button on the experiment's start page was clicked. Note this does not mean the study was actually launched. Clicking the start button without having the player installed (as per the instructions on the start page) will do nothing.
license: time the player executed the license check. This, then, at least means the player was installed.
finish: time redirection to the specified finish page occurred.

> 1) What could be happening if someone has a finish timestamp but only 1 of the 2 data files?

Participant quit the study before one of the data files was created or an upload problem. Any not yet uploaded data files remain on the device and can be manually uploaded via the player's "Data" tab.

> 2) What could be happening if someone has a finish timestamp but no data files?

Quitting the study before there was any data, upload problem, or disabled automatic data file upload in the player (controls for that are on the "Data" tab in the player app).

> 3) Is it correct to assume that if someone only has a launch or license timestamp, they did not complete the study?

Typically yes, although there can be edge cases (e.g. internet connection went down while the participant was completing the scripts -- they do not require persistent internet connectivity), so redirection to the finish page may not have occurred or could not be logged.

Thank you! Do participants generally know how to quit the study? I assume that the data will not be created if they do not click "continue" on the very last page of my script?

Do people have to manually go in and disable automatic data file upload (is it automatically turned on when people install the player)?

This isn't happening for a lot of people, but I want to make sure I adjust my instructions so that people will know in advance what they need to do/need not to do for study compensation.

> Do participants generally know how to quit the study?

If they're experienced MTurk workers who have participated in other studies previously, at least some of them will know.

> I assume that the data will not be created if they do not click "continue" on the very last page of my script?

The answer to that really depends on the script. For most scripts (any RT tasks), there should be some amount of data created and uploadable by that point, but things are a bit different for <survey> scripts and will also depend on any /recorddata settings in the given script (e.g. wheter practice blocks/trials are logged; if not, then there'd be no data files created yet when quitting some time during practice.)

> Do people have to manually go in and disable automatic data file upload (is it automatically turned on when people install the player)?

Automatic upload is enabled by default, so typically it has to be disabled either deliberately or by accident by the user (barring an unknown bug that somehow randomly disables it on some systems).

+x

 

Dave - 7/6/2021

+x

 

sparklab - 7/6/2021

+x

 

Dave - 7/6/2021

+x

 

sparklab - 7/6/2021

I'm running a study on MTurk and am directing participants to Qualtrics first and then to Inquisit Web. On the last page of the Inquisit survey, participants see a code that they have to put into MTurk.

Almost everything is going without a hitch, but some responses are puzzling. I'm having trouble interpreting my study log. What do the launch, license, data, and finish event times mean?

1) What could be happening if someone has a finish timestamp but only 1 of the 2 data files?

2) What could be happening if someone has a finish timestamp but no data files?

3) Is it correct to assume that if someone only has a launch or license timestamp, they did not complete the study?

launch: Time(s) the start button on the experiment's start page was clicked. Note this does not mean the study was actually launched. Clicking the start button without having the player installed (as per the instructions on the start page) will do nothing.
license: time the player executed the license check. This, then, at least means the player was installed.
finish: time redirection to the specified finish page occurred.

> 1) What could be happening if someone has a finish timestamp but only 1 of the 2 data files?

Participant quit the study before one of the data files was created or an upload problem. Any not yet uploaded data files remain on the device and can be manually uploaded via the player's "Data" tab.

> 2) What could be happening if someone has a finish timestamp but no data files?

Quitting the study before there was any data, upload problem, or disabled automatic data file upload in the player (controls for that are on the "Data" tab in the player app).

> 3) Is it correct to assume that if someone only has a launch or license timestamp, they did not complete the study?

Typically yes, although there can be edge cases (e.g. internet connection went down while the participant was completing the scripts -- they do not require persistent internet connectivity), so redirection to the finish page may not have occurred or could not be logged.

Thank you! Do participants generally know how to quit the study? I assume that the data will not be created if they do not click "continue" on the very last page of my script?

Do people have to manually go in and disable automatic data file upload (is it automatically turned on when people install the player)?

This isn't happening for a lot of people, but I want to make sure I adjust my instructions so that people will know in advance what they need to do/need not to do for study compensation.

> Do participants generally know how to quit the study?

If they're experienced MTurk workers who have participated in other studies previously, at least some of them will know.

> I assume that the data will not be created if they do not click "continue" on the very last page of my script?

The answer to that really depends on the script. For most scripts (any RT tasks), there should be some amount of data created and uploadable by that point, but things are a bit different for <survey> scripts and will also depend on any /recorddata settings in the given script (e.g. wheter practice blocks/trials are logged; if not, then there'd be no data files created yet when quitting some time during practice.)

> Do people have to manually go in and disable automatic data file upload (is it automatically turned on when people install the player)?

Automatic upload is enabled by default, so typically it has to be disabled either deliberately or by accident by the user (barring an unknown bug that somehow randomly disables it on some systems).

Sorry, one more question! What about someone for whom there's no launch but there's a finish? (worker ID ends in DVYS in my study)

+x

 

sparklab - 7/6/2021

+x

 

Dave - 7/6/2021

+x

 

sparklab - 7/6/2021

+x

 

Dave - 7/6/2021

+x

 

sparklab - 7/6/2021

I'm running a study on MTurk and am directing participants to Qualtrics first and then to Inquisit Web. On the last page of the Inquisit survey, participants see a code that they have to put into MTurk.

Almost everything is going without a hitch, but some responses are puzzling. I'm having trouble interpreting my study log. What do the launch, license, data, and finish event times mean?

1) What could be happening if someone has a finish timestamp but only 1 of the 2 data files?

2) What could be happening if someone has a finish timestamp but no data files?

3) Is it correct to assume that if someone only has a launch or license timestamp, they did not complete the study?

launch: Time(s) the start button on the experiment's start page was clicked. Note this does not mean the study was actually launched. Clicking the start button without having the player installed (as per the instructions on the start page) will do nothing.
license: time the player executed the license check. This, then, at least means the player was installed.
finish: time redirection to the specified finish page occurred.

> 1) What could be happening if someone has a finish timestamp but only 1 of the 2 data files?

Participant quit the study before one of the data files was created or an upload problem. Any not yet uploaded data files remain on the device and can be manually uploaded via the player's "Data" tab.

> 2) What could be happening if someone has a finish timestamp but no data files?

Quitting the study before there was any data, upload problem, or disabled automatic data file upload in the player (controls for that are on the "Data" tab in the player app).

> 3) Is it correct to assume that if someone only has a launch or license timestamp, they did not complete the study?

Typically yes, although there can be edge cases (e.g. internet connection went down while the participant was completing the scripts -- they do not require persistent internet connectivity), so redirection to the finish page may not have occurred or could not be logged.

Thank you! Do participants generally know how to quit the study? I assume that the data will not be created if they do not click "continue" on the very last page of my script?

Do people have to manually go in and disable automatic data file upload (is it automatically turned on when people install the player)?

This isn't happening for a lot of people, but I want to make sure I adjust my instructions so that people will know in advance what they need to do/need not to do for study compensation.

> Do participants generally know how to quit the study?

If they're experienced MTurk workers who have participated in other studies previously, at least some of them will know.

> I assume that the data will not be created if they do not click "continue" on the very last page of my script?

The answer to that really depends on the script. For most scripts (any RT tasks), there should be some amount of data created and uploadable by that point, but things are a bit different for <survey> scripts and will also depend on any /recorddata settings in the given script (e.g. wheter practice blocks/trials are logged; if not, then there'd be no data files created yet when quitting some time during practice.)

> Do people have to manually go in and disable automatic data file upload (is it automatically turned on when people install the player)?

Automatic upload is enabled by default, so typically it has to be disabled either deliberately or by accident by the user (barring an unknown bug that somehow randomly disables it on some systems).

Sorry, one more question! What about someone for whom there's no launch but there's a finish? (worker ID ends in DVYS in my study)

Looking at that log entry, it's probably due to the use of a Chromebook (the only one in the set, it appears), and launching directly from the player app.
Things get a little confusing there, because that'll make use of the Android app on a system that is not technically Android.