Missing Data


Author
Message
nashby
nashby
Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)
Group: Forum Members
Posts: 78, Visits: 159
So I have been running a study on Mturk and while the normal problems are found (missing data, participants cant get it to run or it breaks halfway through) I am having a huge problem with data not being uploaded/saved to inquists server so when I go to retrieve the data its just not there. Anyone have this problem and know of any fixes? I am wondering where the file is saved on the persons computer before the upload, maybe its just sitting there and if they search for it they could email it to me? This is a huge problem of course as I am either having to pay for work that is missing or getting a ton of complaints and a very bad rep for not paying for the work.
Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 12K, Visits: 98K
Once a web experiment finishes, the Inquisit Web runtime will attempt to contact the server and upload the data. If it cannot contact the server for some reason (e.g., because a local firewall on the participant's machine blocked communication or the internet connection went down entirely), it will prompt the participant to save the file locally and contact the researcher. If a participant declines to save the file, the data is lost.

I unfortunately cannot tell you why some (many?) of your participants may be experiencing problems without you providing a link to the respective launch page. In general, you should make sure it's using the latest Inquisit Web version and that the script is properly activated.

nashby
nashby
Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)
Group: Forum Members
Posts: 78, Visits: 159
Yes I am doing all of that and they are not being prompted to save the file locally it just ends doesnt show the end page and thats it. It works for maybe 70% of people and fails for the other 30%....

Here is a link to the study.

http://research.millisecond.com/nashby/Bundled.web

Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 12K, Visits: 98K
Thanks for the link. I did a couple of quick test runs and it's working as expected for me. It appears to properly save data and redirect to the finish page *if* a participant fully completes the procedure.

It will not do so if a given participant triggers one of your various script.abort() conditions, which is also the expected behavior.

It would seem plausible that the above is the case for at least *some* of your participants reporting an apparent breakdown / not being redirected. As for the others experiencing issues, if you have any additional details (error messages and the like), please don't hesitate to share them.

nashby
nashby
Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)
Group: Forum Members
Posts: 78, Visits: 159
Actually it does upload the data on the script abort as well, just doesnt show the finish page.

So its some other problem and its costing me dearly. I dont have any other details to go off of people report using a variety of browsers and many report that inquist has worked for them in the past. This is really frustrating.
Dave
Dave
Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)Supreme Being (1M reputation)
Group: Administrators
Posts: 12K, Visits: 98K
Hmm, offhand I have no idea what other problem that may be. In case you have participant tracking enabled, the logs accessible via your account at https://www.millisecond.com/myaccount/logs.aspx may provide some hint(s).

nashby
nashby
Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)
Group: Forum Members
Posts: 78, Visits: 159
Thanks had a look only 1 error reported and so dont think thats going to shed much light. I think I would then simply say if you are going to run a study online don't use Inquisit because it doesn't work so well and will just get you in trouble :( Back to using java I guess.
seandr
seandr
Supreme Being (142K reputation)Supreme Being (142K reputation)Supreme Being (142K reputation)Supreme Being (142K reputation)Supreme Being (142K reputation)Supreme Being (142K reputation)Supreme Being (142K reputation)Supreme Being (142K reputation)Supreme Being (142K reputation)
Group: Administrators
Posts: 1.3K, Visits: 5.6K
I just tried your script, and I was unable to proceed past the following trial:


<trial genintro>
/ stimulusframes = [1 = intro2, proceed2, box]
/ validresponse = (proceed, box)
/ recorddata = false
/ ontrialend = [if(trial.genintro.response == "proceed"){values.kickem = 1}]
</trial>


The problem is that "proceed" is listed as a valid response, but it isn't actually presented anywhere on the screen, so there's no way to actually proceed. Instead, the trial presents a stimulus called "proceed2". I'm guessing your participants click about 10 times in vain on the "proceed2" stimulus, and then bail out by hitting Ctrl+Alt+Delete.

You'll likely see higher completion rates if you fix this problem. 

-Sean


nashby
nashby
Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)Guru (8.6K reputation)
Group: Forum Members
Posts: 78, Visits: 159
Thanks for catching this but that is actually an attention check. If they click proceed they fail and it kicks them out anyways so if they ctrl quit out that would be for the best. 

The problem with the data not uploading still remains since this is people who did the whole experiment and then it didnt upload.
seandr
seandr
Supreme Being (142K reputation)Supreme Being (142K reputation)Supreme Being (142K reputation)Supreme Being (142K reputation)Supreme Being (142K reputation)Supreme Being (142K reputation)Supreme Being (142K reputation)Supreme Being (142K reputation)Supreme Being (142K reputation)
Group: Administrators
Posts: 1.3K, Visits: 5.6K
Ok, doing some more digging here.

I'm counting data files, and what I'm seeing is 96 data files in your account right now. Another 18 data files were deleted, one on 10/28 and the others on 10/30, so the cumulative total is 114. The log for the experiment reports there should be 112 - not sure why the discrepancy, but possibly the log was cleared after 2 test runs, or there were 2 duplicate uploads (which can happen if the server returns an error after the first attempt, so Inquisit tries again, and then both attempts succeed). In any case, all of the data files that made it to the server seem to be accounted for.

What makes you think there are additional data files that didn't make it to the server?

Regarding the attention check, it's clever, but there's no way to differentiate
1) someone who misses or is confused by the instructions, repeatedly clicks the "Let's get started" link, assumes Inquisit is hung when nothing happens, and angrily shuts down Inquisit using Ctrl+Alt+Delete.
2) someone who encountered a legitimate bug.

Also note that every participant who misses the instructions is going to blame you and/or Inquisit for their inability to proceed, which ruins both yours and Inquisit's reputation. If you're going to use this kind of check, you should allow them to respond with either the link or the invisible box, and show them a message that says "Sorry, you weren't reading instructions, the experiment will now end" if they click the wrong one. That would also give you a record in the data file that they did, in fact, click the wrong link. It looks like 18 people dropped out after successfully starting the experiment - I'm guessing most or all of them got caught in your trap without knowing it.

If you have any additional info regarding data files that didn't get sent to the server, let me know. I'm looking into the missing finish page entries in the log now...

-Sean
Edited 10 Years Ago by seandr
GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Reading This Topic

Explore
Messages
Mentions
Search