Design Fluency Task - DF

Technical Manual

Script Author: Katja Borchert, Ph.D. (katjab@millisecond.com), Millisecond

Created: January 03, 2025

Last Modified: January 31, 2025 by K. Borchert (katjab@millisecond.com), Millisecond

Script Copyright © Millisecond Software, LLC

Background

This script implements the Millisecond Design Fluency Task (DFT); a computerized version of a design fluency task in which participants are asked to generate unique line drawings between a provided display of dots.

Design fluency tasks are generally viewed as the nonverbal analog to Verbal Fluency tasks and are used to assess executive functions such as planning: the ability to plan a new line design cognitive flexibility: ability to switch between patterns fluency: the speed of generating new designs inhibition: the ability to inhibit/repeat previously generated designs and ignore distractors

The DFT tests participants under three task conditions similarly to the ones from the D-KEFS Design Fluency task (Kaplan et al, 2001) with the difference that participants in the DFT work on one display at a time with patterns being erased after submitting them. This aspect of the DFT procedure is in line with the Computerized Design Fluency (C-DF) task published by Woods et al (2016).

The DFT requires the use of a computer mouse on non-touchscreen devices. Default instructions are for touchscreens but revert to mouse use if no touchscreen is detected.

1. Condition 1: draw 4-line patterns in 5-dot (all black) displays 2. Condition 2: draw 4-line patterns in 10-dot displays (5 white dots, 5 black dots), ignoring the black dots (distractor condition) 3. Condition 3: draw 4-line patterns in 10-dot displays (5 white dots, 5 black dots), alternating btw. white and black dots (switching condition)

References

Delis DC, Kaplan E, Kramer JH (2001) Delis-Kaplan Executive Function System (D-KEFS). San Antonio, TX: The Psychological Corporation.

Woods DL, Wyma JM, Herron TJ, Yund EW (2016). A Computerized Test of Design Fluency. PLoS ONE 11(5): e0153952. doi:10.1371/journal.pone.0153952

Duration

7 minutes

Description

Participants are asked to draw as many unique 4 'continuous' line patterns connecting (up to 5) dots as they can within 1 minute. The line patterns can be drawn with a computer mouse (non-touchscreens) of with fingers/stylus pens (touchscreens). Default instructions are for touchscreens but revert to mouse use if no touchscreen is detected.

Mouse: Trial starts with participants being asked to click on the screen. Without lifting the finger from the mouse button, they can then start drawing. When they lift their finger, they can either click the 'done' button or continue drawing by clicking once again on the screen.

Touch: Trial starts with participants being asked to press down on the screen to start drawing. When they lift their finger, they can either press the 'done' button or continue drawing by pressing once again the screen.

in this implementation of the task: - lines that NOT not connect two dots are simply ignored by the program. - non-straight/wavy lines btw. dots are interpreted as straight lines

Participants work on one dot display at a time with the solution being erased once it's submitted. All starting dot patterns (within the same condition) provide the same dot placements.

3 conditions are tested:

1. Condition 1 (mandatory): draw unique 4-line patterns in 5-dot (all black) displays 2. Condition 2 (optional)*: draw unique line patterns in 10-dot displays (5 white dots, 5 black dots), ignoring the black dots (distractor condition) 3. Condition 3 (optional)*: draw unique 4-line patterns in 10-dot displays (5 white dots, 5 black dots), alternating btw. white and black dots (switching condition)

• optional via parameters setting

Procedure

(1) Intro: participants learn about the rules for the DFT using several examples
of correct and incorrect solutions with a simplified dot pattern

(2) Practice (optional, see parameter settings)
Participants work on 5 (see parameter settings) simplified dot patterns and receive
accuracy/error feedback.
- All dot patterns are provided one-by-one with the solution being erased after it's submitted.
- All dot displays are the same

(3) Task1 (mandatory): timeLimit = 1minute (see parameter settings)
=> Draw unique 4-line patterns in 5-dot (all black) displays
- All dot patterns are provided one-by-one with the solution being erased after it's submitted.
- All dot displays are the same
- No feedback

(4) Task2 (optional, see parameter settings): timeLimit = 1minute (see parameter settings)
=> Draw unique line patterns in 10-dot displays (5 white dots, 5 black dots), ignoring the black dots (distractors)
- All dot patterns are provided one-by-one with the solution being erased after it's submitted.
- All dot displays are the same
- No feedback

(4) Task3 (optional, see parameter settings): timeLimit = 1minute (see parameter settings)
=> Draw unique 4-line patterns in 10-dot displays (5 white dots, 5 black dots), alternating btw. white and black dots (switching condition)
- All dot patterns are provided one-by-one with the solution being erased after it's submitted.
- All dot displays are the same
- No feedback


Scoring Algorithm

TrialSolutions are stored as follows:

- dot segments name 'smallest' dot first, then 'larger' dot (based on dot identifying numbers)
=> path direction unimportant: segment '15' and segment '51' get both stored as segment '15'
- solutions store segments from 'smallest' to 'largest'
=> participant draws: 2->3->4->5->1 => trialSolution: 15,23,34,45

Step1: Check for 'empty' solutions
The script checks if a trialSolution is empty (no path segments added)
-> empty solutions are automatically scored as incorrect
-> errorCategory: "noSolution" (accuracy analysis ends)

Step2: Check for Repetition
The script stores all new solutions in a storage array 'AllSolutions' (resets for each condition).
If a trialSolution is already part of this storage array, the solution is scored as a repeat.
-> repetitions are automatically scored as incorrect (accuracy analysis ends)
-> errorCategory: "repeat violation"

Step3: Check for Number Of Segments drawn:
The script tracks the number of new segments drawn during each trial (repeated segments are NOT counted as new lines)
-> designs with fewer or more segments than 4 are automatically scored as incorrect (accuracy analysis ends)
-> errorCategory: "lineCount violation"

Step4: Check for Continuation of Lines drawn (see below)
-> designs that fail the continuation check are scored as incorrect
-> errorCategory: "continuation violation"

Step5: Check for Dot Sequence
-> this step depends on task:
- Task1: no constraints for Task1
- Task2: connected dots need to be all white (thus only 'odd' numbered dots)
- Task3: connected dots need to alternate btw. black (even) and white (odd) dots
-> the sequence check is performed on the theoretical solution path that leads to a continuous line design
-> error Category: "sequence violation"

All correct solutions are further analyzed for a detectable pattern:
"4quadrilateral": 4 dot 'closed' solution (forms a quadrilateral)
"4triangle+1": 4 dot solution that forms a closed triangle with one outside dot connected to the triangle at one of the endpoints
"5open": currently any 5 dot solution


Continuation Check:
There are 24 possible segment comparisons to conduct to decide if 4 lines using 5 dots
could be drawn in one continous path:

paths = ["12,23,34","12,24,41","13,32,24","13,34,42","14,42,23","14,43,32","21,13,34","21,14,43","23,31,14","23,34,41","24,41,13","24,43,31","31,12,24","31,14,42","32,21,14","32,24,41","34,41,12","34,42,21","41,12,23","41,13,32","42,21,13","42,23,31","43,31,12","43,32,21"]
"12,23,34" -> compare 1st segment with 2nd segment for overlapping dots -> compare 2nd segment with 3 segment for overlapping dots -> etc.

Example:
solution with 4 unique lines: "24,26,28,46" (4 segments, ordered from 'smallest' to 'largest')
"24" -> refers to first segment stored
"26" -> second segment
"28" -> third segment
"46" -> fourth segment

24 possible segment comparisons to test for each solution:
test1: 12,23,34 12: '24' vs.'26' (pass,they share '2' -> '2' cannot be used in next comp)-> 23: '26' vs. '28' (dots don't share a number other than 2): FAIL
test2: 12,24,43 12: '24' vs.'26' (pass,they share '2' -> '2' cannot be used in next comp)-> 24: '26' vs. '46' (pass:'46' contains '6' ->'6' cannot be used for next comp)-> 43: '46' vs '28' (dots don't share any numbers): FAIL
test3: 13,32,24 13: '24' vs.'28' (pass,they share '2' -> '2' cannot be used in next comp)-> 32: '28' vs. '26' (dots don't share a number other than 2): FAIL
test4: 13,34,42 13: '24' vs.'28' (pass,they share '2' -> '2' cannot be used in next comp)-> 34: '28' vs. '46' (dots do not share any numbers): FAIL
test5: 14,42,23 14: '24' vs.'46' (pass,they share '4' -> '4' cannot be used in next comp)-> 42: '46' vs. '26' (pass,they share '6' -> '6' cannot be used in next comp)-> 23: '26' vs. '28' (pass,they share '2'): PASS => continuation!
!!!This solution can be drawn continuously using the path: 2->4->6->2->8
no further evaluations needed

Stimuli

provided by Millisecond

black dots: dots with even numbers (0,2,4,6,8) -
white dots: dots with odd numbers (1,3,5,7,9)

-the numbering is used as a way to identify each dot (and its color);
-the numbering is done clockwise (dot 0 is located on the bottom left)

DOT LOCATIONS:
the default locations of the dots in task1/task2/task3 were chosen to simulate the locations used in the
paper and pencil D-KEFS Design Fluency Task
-> pictures task1setup.png; task2setup.png; task3setup.png
show the default dot locations (see parameters.dotLocations) run by this script in each of the three tasks
(locations can be edited under section Editable Parameters)

Instructions

provided by Millisecond - can be edited in script "designfluencytask_instructions_inc.iqjs"

Scoring

By default, this script does NOT store screenshots of every solution submitted.
If you'd like to store each solution as a separate screenshot, go to trial.eval and
activate / screenCapture.

Summary Data

File Name: designfluencytask_summary*.iqdat

Data Fields

NameDescription
inquisit.version Inquisit version number
computer.platform Device platform: win | mac |ios | android
computer.touch 0 = device has no touchscreen capabilities; 1 = device has touchscreen capabilities
computer.hasKeyboard 0 = no external keyboard detected; 1 = external keyboard detected
startDate Date the session was run
startTime Time the session was run
subjectId Participant ID
groupId Group number
sessionId Session number
elapsedTime Session duration in ms
completed 0 = Test was not completed
1 = Test was completed
parameters.taskDurationMS The time (in ms) allocated to each task
compositeScore The combined score of correct number of solutions submitted in task1,task2 and task3
Per Task (Explained For Task1)
numberTrials1 The number of trials COMPLETED in task1
numberCorrect1 The number of correct trials submitted in task1
noSolutions1 The number of 'noSolution' errors made in task1
repeatViolations1 The number of 'repeatViolations' errors made in task1
lineCountViolations1 The number of 'lineCountViolations' errors made in task1
continueViolations1 The number of 'continueViolations' errors made in task1
sequenceViolations1 The number of 'sequenceViolations' errors made in task1
patternSwitches1 The number of detected pattern switches from one correct pattern to the next correct pattern in task1
All Stored Solutions For Task1-Task3
allSolutions1-allSolutions3 Stores the array variable containing all provided solutions for each task
even numbers: refer to black dots
odd numbers: refer to white dots

Raw Data

File Name: designfluencytask_raw*.iqdat

Data Fields

NameDescription
build Inquisit version number
computer.platform Device platform: win | mac |ios | android
computer.touch 0 = device has no touchscreen capabilities; 1 = device has touchscreen capabilities
computer.hasKeyboard 0 = no external keyboard detected; 1 = external keyboard detected
date Date the session was run
time Time the session was run
subject Participant ID
group Group number
session Session number
blockcode The name the current block (built-in Inquisit variable)
blocknum The number of the current block (built-in Inquisit variable)
trialcode The name of the currently recorded trial (built-in Inquisit variable)
trialnum The number of the currently recorded trial (built-in Inquisit variable)
trialnum is a built-in Inquisit variable; it counts all trials run
even those that do not store data to the data file.
parameters.taskDurationMS The time (in ms) allocated to each task
task The current task worked on: "0" (practice),"1","2","3"
trialCounterPerTask Tracks the number of trials started for each task
response The valid trial response made ( for task trials this is either a dot or the done button)
correct Correctness of response (1 = correct, 0 = error) ( does not apply)
latency Response latency (in ms); measured from: onset of current trial until valid response was made
prevDot The previously connected dot in the current design
currentDot The currently visited dot
currentPathStr A string variable that stores the current path between 2 dots
-> contains direction information
Examples
24 -> moved from dot2 to dot4
42 -> moved from dot4 to dot2
trialPaths Contains all trial paths added (contains direction information)
Example: 02,80,40,42, (all segments drawn, not necessarily in a continuous manner)
trialSolution Contains the final design segments
- does not contain path direction information
- ordered from 'smallest' to 'largest' segment
Example: 02,04,08,24,
repeatSolution 1 = provided solution is a repeat (based on trialSolution)
0 = provided solution is new
lineCounter Tracks the number of drawn lines between dots
continuation Empty: line was not tested for continuation as an error was already encountered
-1 = the provided design could NOT be drawn with a continuous line
0-23 = a continuous solution path was found for the provided design
(number refers to first successful solution path tested from array variable 'paths')
for this analysis it does not matter whether the design was
drawn in a continuous fashion
it just matters that the final design is evaluated as 'continuous'.
conSolPath Stores the possible continued solution path found
sequenceAcc Empty: sequenceAcc was not tested (an error was already encountered before this analysis)
1 = the sequence of connected dots in the continuous solution path
conforms to the constraints of the current tasks
task1: no sequence constraints (only black dots provided)
task2: connect only white (odd) dots (black dots need to be ignored)
task3: connect dots in alternating colors (...odd->even->odd...)
0 = sequence violation found
trialAcc 1 = the provided solution is evaluated as correct after the solution has passed all error checks
0 = the provided solution contains an error
trialError Stores the FIRST error found (empty for correct solution)
solutions are checked for errors in the following sequence
(1) noSolution (contains some additional info in case of a completely empty design, a one-dot design or scribbles that never touched a dot)
(2) repeatViolation
(3) lineViolation
(4) continueViolation
(5) sequenceViolation
trialRT Time (in ms) from trialStart until 'done' button is pressed
Pattern Analysis Of Correct Solutions
patternCat The patterns (as much as the computer could analyse it) of the correct solution
"" (empty) => solution was incorrect
"4quadrilateral": 4 dot 'closed' solution (forms a quadrilateral)
"4triangle+1": 4 dot solution that forms a closed triangle with one outside dot connected to triangle at one of the endpoints
"5open": currently any 5 dot solution
patternSwitch 1 = the currently presented correct solution differs in its pattern from the last presented correct solution
0 = otherwise
patternSwitchCounter Tracks the number of detected (correct) pattern switches
Error Counts Per Task: Error Analysis Stops At First Error Encountered
noSolutions Tracks the number of 'noSolution' errors per task
repeatViolations Track the number of 'repeatViolation' errors per task
lineCountViolations Track the number of 'lineViolation' errors per task
continueViolations Track the number of 'continueViolation' errors per task
sequenceViolations Track the number of 'sequenceViolation' errors per task
liftCounter Counts number of times (per trial) that participants made a buttonUp response
(lifted finger from mouse button or lifted finger from screen)

Parameters

The procedure can be adjusted by setting the following parameters.

NameDescriptionDefault
Color Parameter
canvasColor Display color of the actively used portion of the screen (the 'canvas')
if set to a color other than the screenColor, you can distinguish the active canvas
from the inactive portion of the screen
white
screenColor Color of the screen not used by the canvas ('inactive screen')black
defaultTextColor Default color of text items (presented on the canvas)black
frameHeightPct Sets the frameHeight of the dot setup images
dot locations and dot sizes are coded relative to the height of the frame (dot size= 10% of the frameHeight)
thus changing the frameHeight will automatically resize the dots and place them
at comparable locations
50%
Exp Design Parameters
runTask2 True: task2 is run after task1 (task1 = mandatory)
false: task2 is skipped
true
runTask3 True: task3 is run as the last task (task1 = mandatory)
false: task3 is skipped
true
runPractice True: practice trials (for task1) are run
false: practice block is skipped
true
practiceTrials Number of practice Trials (for task1) to run5
Duration Parameters
taskDurationMS The assigned timeout (in ms) for each task (60000 = 1 minute recommended)60000
practiceFeedbackMS The duration (in ms) that practice feedback is provided3000