Character at the end of IQDAT files


Author
Message
lakeside
lakeside
Guru (5.9K reputation)Guru (5.9K reputation)Guru (5.9K reputation)Guru (5.9K reputation)Guru (5.9K reputation)Guru (5.9K reputation)Guru (5.9K reputation)Guru (5.9K reputation)Guru (5.9K reputation)
Group: Forum Members
Posts: 34, Visits: 121
I am trying to create an automated workflow to import IQDAT files for a task (one file per subject), combine them into a single and output them to a variety of formats (SAS, SPPS, TAB). First, Inquisit's multi-file import/combine function isn't command line accessible, is it?

Assuming not, I am running into import errors when I reach the end of the IQDAT files. Apparently, there is an extra character at the end of the final line. Looks like a space. Is there a reason for this extra character?

Tags
Dave
Dave
Supreme Being (784K reputation)Supreme Being (784K reputation)Supreme Being (784K reputation)Supreme Being (784K reputation)Supreme Being (784K reputation)Supreme Being (784K reputation)Supreme Being (784K reputation)Supreme Being (784K reputation)Supreme Being (784K reputation)
Group: Administrators
Posts: 10K, Visits: 51K
Data files end with line termination markers applicable to the platform. Under Windows, those markers are 0D 0A in hex (carriage return followed by line feed; CRLF). Under Unix-like operating systems, the marker is a single 0A (line feed; LF). See e.g. http://www.maxi-pedia.com/line+termination+line+feed+versus+carriage+return+0d0a for details.

While Inquisit's merge functionality is not exposed via the command line, *.iqdat files are simple, tab-delimited plain text files, their encoding is UTF-8.

Edited 5 Years Ago by Dave
lakeside
lakeside
Guru (5.9K reputation)Guru (5.9K reputation)Guru (5.9K reputation)Guru (5.9K reputation)Guru (5.9K reputation)Guru (5.9K reputation)Guru (5.9K reputation)Guru (5.9K reputation)Guru (5.9K reputation)
Group: Forum Members
Posts: 34, Visits: 121
The last character I'm seeing is a hexadecimal value of '0xd', which displays as a space. Inquisit is installed on a Windows 7 machine and is writing data to a Windows server via a network share.

Dave
Dave
Supreme Being (784K reputation)Supreme Being (784K reputation)Supreme Being (784K reputation)Supreme Being (784K reputation)Supreme Being (784K reputation)Supreme Being (784K reputation)Supreme Being (784K reputation)Supreme Being (784K reputation)Supreme Being (784K reputation)
Group: Administrators
Posts: 10K, Visits: 51K
Hmm, can you attach that data file and the script that generates it to this thread? (Click on +Insert when posting a reply to upload files.) Thanks!

Dave
Dave
Supreme Being (784K reputation)Supreme Being (784K reputation)Supreme Being (784K reputation)Supreme Being (784K reputation)Supreme Being (784K reputation)Supreme Being (784K reputation)Supreme Being (784K reputation)Supreme Being (784K reputation)Supreme Being (784K reputation)
Group: Administrators
Posts: 10K, Visits: 51K
Okay. After a quick test, I can (If I understood your setup correctly) sort of reproduce the issue.

(1) If you save to an UNC path as in

<data>
...
/ file = "\\myserver\myfolder\"
...
</data>

The final line terminator will be a single carriage return (0xD; CR). This should not cause any problems when merging files with Inquisit (and doesn't in my tests). Not sure which program would interpret a carriage return as a space (0x20). Most programs also provide a way to specify which characters to interpret as representing a new line.

(2) If you map your share to a drive letter, e.g.,

> net use z: \\myserver\myfolder

and save to that drive letter (instead of using the raw UNC path)

<data>
...
/ file = "Z:\"
...
</data>

the data file will end with the Windows standard 0xD 0xA CRLF- tuple.

I believe (1) is a Windows-quirk with regards to UNC paths, but I can't find the reference right now; I will double-check and add it here (or post a correction).

lakeside
lakeside
Guru (5.9K reputation)Guru (5.9K reputation)Guru (5.9K reputation)Guru (5.9K reputation)Guru (5.9K reputation)Guru (5.9K reputation)Guru (5.9K reputation)Guru (5.9K reputation)Guru (5.9K reputation)
Group: Forum Members
Posts: 34, Visits: 121
I am opening the IQDAT file in UltraEdit to determine the hexadecimal value. I am using SAS to import the files and discovered the issue when the hitting the final value in the final row. That column was importing as numeric but the space in that final slot was leading SAS to interpret the last value as a string. For the record, I'm working around this by importing the last column as a string and then casting it to numeric.

GO

Merge Selected

Merge into selected topic...



Merge into merge target...



Merge into a specific topic ID...




Reading This Topic

Explore
Messages
Mentions
Search