Faux Pas Test

Technical Manual

Script Author: Katja Borchert, Ph.D. (katjab@millisecond.com), Millisecond

Created: January 08, 2024

Last Modified: January 03, 2025 by K. Borchert (katjab@millisecond.com), Millisecond

Script Copyright © Millisecond Software, LLC

Background

This script implements Millisecond's version of the Faux Pas Test (FPT), a test of Theory-of-Mind (ToM) abilities that require social sensitivity (Stone et al, 1998, Gregory et al, 2002). Millisecond's test uses Pennington's et al (2023) materials, made available under the Commons Attribution 4.0 International License (CC BY 4.0; https://creativecommons.org/licenses/by/4.0).

The materials (as well as the original psychopy script) can be downloaded here: https://osf.io/2bhmy/

References

Stone, V. E., Baron-Cohen, S., & Knight, R. T. (1998). Frontal Lobe Contributions to Theory of Mind. Journal of Cognitive Neuroscience, 10(5), 640–656. https://doi.org/10.1162/089892998562942

Gregory, C., Lough, S., Stone, V., Erzinclioglu, S., Martin, L., Baron-Cohen, S., & Hodges, J. R. (2002). Theory of mind in patients with frontal variant frontotemporal dementia and Alzheimer’s disease: Theoretical and practical implications. Brain: A Journal of Neurology, 125(4), 752–764. https:// doi.org/10.1093/brain/awf079

Pennington, C. R., Oxtoby, M. C.-S.-Y., & Shaw, D. J. (2023). Social cognitive disruptions in multiple sclerosis: The role of executive (dys)function. Neuropsychology. https://doi.org/10.1037~neu0000917

Duration

7 minutes

Description

Participants read 20 social scenarios. Half of these scenarios contain a 'faux pas' (e.g. something said unintentionally that should have been confidential or could be perceived as hurtful or inappropriate). Participants are asked several questions about the potential faux pas as well as several comprehension questions about the scenarios.

Millisecond converted the openended comprehension questions used by Pennington et al (2023) into 4-choice multiple choice questions to simplify accuracy assessments.

Procedure

Test: 20 scenarios (Pennington et al, 2023)
- by default, the order of the scenarios is set to be sequential in this script.

Scenario Sequence

Question1: "Did anyone say something they shouldn't have said or something awkward? (Yes/No Answer)"
-> scenario is presented alone for 8000ms
-> the scenario question as well as the response buttons are added to the scenario (Scenario still visible)
-> no timelimit

IF Answer is 'YES' (faux pas detected): Question2-Question6

Question2: "Who said something they shouldn't have said or something awkward?"
-> openended question (with scenario still visible, to keep demand on working memory minimal (Gregory et al, 2002))
-> no timelimit
see item.q2CorrAnswers for all answers that are accepted as correct in this script.
All responses are converted to lowercase letters (as are the correct answers), so capitalization is ignored
in the accuracy assessment

Question3: "Why shouldn't he/she have said it or why was it awkward?"
Question4: "Why do you think he/she said it?"
Question5: specific to scenario, asks about the 'insulted' person's knowledge of the situation
Question6: "How do you think X felt? "
-> no timelimit; all questions appear on the same page
these questions have no bearings on performance assessments

ALL participants are asked the following COMPREHENSION questions q7-q8:
-> each comprehension question appears on its own page
-> no timelimits
in contrast to Pennington et al (2023), the comprehension questions in this script are provided as
multiple-choice questions with 4 answer choices (as opposed to open-ended questions as done by
Pennington et al, 2023) to simplify automatic scoring

-> q8 is followed by a blank screen of 500ms (iti)

Faux-Pas Performance Assessment:
- q1/q2: if both responses are correct, faux pas detection is set to 1 (see values.fpACC) - otherwise 0
for correctly identifying non faux-pas situations, q2 is automatically set to be correctly answered

Comprehension Performance Assessment:
- if answers to both comprehension questions q7/q8 are correct, comprehension performance is set ot 1 (see values.compACC) - otherwise 0

Overall Scenario Accuracy:
- only if both Faux Pas Performance AND Comprehension Performance are 1, is the scenario accuracy set to 1 (see values.scenarioACC)

An overall Faux Pas Detection Score is calculated as the ratio of
number of correct FP detections with correct comprehension / number of correct FP rejections with correct comprehension

Stimuli

Scenarios provided by Pennington's et al (2023) under the
Commons Attribution 4.0 International License
(CC BY 4.0; https://creativecommons.org/licenses/by/4.0).

The materials can be downloaded here:
https://osf.io/2bhmy/ - can be edited under section 'Editable Stimuli'

Instructions

provided by Millisecond - can be edited under section 'Editable Instructions'

Summary Data

File Name: fauxpastest_summary*.iqdat

Data Fields

NameDescription
inquisit.version Inquisit version number
computer.platform Device platform: win | mac |ios | android
startDate Date the session was run
startTime Time the session was run
subjectId Participant ID
groupId Group number
sessionId Session number
elapsedTime Session duration in ms
completed 0 = Test was not completed
1 = Test was completed
scenarioCounter Counts the number of scenarios run
countCatFCorr Number of CategoryF scenarios with scenarioACC = 1
countCatCCorr Number of CategoryC scenarios with scenarioACC = 1
fauxpasDetectionScore Ratio of 'countCatFCorr/countCatCCorr'
=> higher fauxpasDetectionScore indicate better ToM (Theory of Mind)

Raw Data

File Name: fauxpastest_raw*.iqdat

Data Fields

NameDescription
build Inquisit version number
computer.platform Device platform: win | mac |ios | android
date Date the session was run
time Time the session was run
subject Participant ID
group Group number
session Session number
blockCode Name of the current block
blockNum Number of the current block
trialCode Name of the current trial
trialNum Number of the current trial
scenarioCounter Counts the number of scenarios run
scenarioIndex The index number of the currently presented scenario
scenario The actually presented scenario
category "F" (faux pas) vs. "C" (control)
Overall Scenario Performance
fpAcc 1 = q1 and q2 were answered correctly; 0 = otherwise
compAcc 1 = q7 and q8 were answered correctly; 0 = otherwise
scenarioAcc = fpACC * compACC => 1 = both fpACC AND compACC are correct; 0 = otherwise
countCatFCorr Number of CategoryF scenarios with scenarioACC = 1
countCatCCorr Number of CategoryC scenarios with scenarioACC = 1
fauxpasDetectionScore Ratio of 'countCatFCorr/countCatCCorr'
=> higher fauxpasDetectionScore indicate better ToM (Theory of Mind)
Faux Pas Questions
q1CorrAnswer Stores the currently correct answer to q1 (yes or no)
q1Resp Participant's response to q1
q1Acc 1 = participant's response to q1 was correct; 0 = otherwise
q2CorrAnswers Stores the currently correct answer(s) to q2 ( answers try to anticipate spelling errors)
q2Resp Participant's response to q2
q2Acc 1 = participant's response to q2 was correct; 0 = otherwise
q3Resp- q6Resp Responses given to q3-q6 (responses are not further evaluated in this script)
Comprehension Questions (In This Script: Multiple Choice With 4 Answer Choices)
q7CorrAnswer Stores the currently correct answer to q7
q7Resp Participant's response to q7
q7Acc 1 = participant's response to q7 was correct; 0 = otherwise
q8CorrAnswer Stores the currently correct answer to q8
q8Resp Participant's response to q8
q8Acc 1 = participant's response to q8 was correct; 0 = otherwise
Built-In Dv
response The response of participant in the current trial
correct Correctness of response (1 = correct, 0 = error)
latency Response latency (in ms)

Parameters

The procedure can be adjusted by setting the following parameters.

NameDescriptionDefault
scenarioDurationMS 8000 //duration (in ms) that the scenario is presented before the response buttons are added to the screen
itiMS The intertrial duration (in ms)500