Using logLime (UseR! 2026 workshop)
Tomasz Żółtak
2026-07-03
Data and setup
Have you already ran the the script ‘prepare-for-the-workshop!.R’? If no, do it now before proceeding further!
Loading packages
Reading the data
For the purpose of this workshop, you can read the data in two ways:
- The harder way: reading from a huge CSV file (in the form in which the data is exported from the web-surveying platform), which will take a little more time and perhaps may cause some technical issues;
- The easier way: simply opening the .RData with the data already imported to R data frames.
Please, choose the one you prefer, or try the first and switch to the second if you encounter problems.
The data comes from the second wave of the survey experiment conducted as a part of the research grant Understanding response styles in self-report data: consequences, remedies and sources conducted in the Institute of Philosophy and Sociology of the Polish Academy of Sciences by the team of prof. Artur Pokropek.
The hard way
The main data file contains survey results exported from the Lime Survey platform to a CSV file (encoded in UTF-8 with BOM):
- In fact, this file was created by combining two such files from technically two different surveys - each with a differerent version of the order of items in the questionnaire (counterbalance).
- We need to read this file to R by ourselves.
- Such a huge text files (hundreds of MBs) should not be read using base R import functions, but rather specialized packages, like arrow or vroom.
- Below this file is imported into a data frame named
surveyResults.
In the folllowing sections it is shown how to read the data using either the arrow or the vroom package. Please, choose the one you prefer or try booth (and compare execution time, if you want).
Using the arrow package
With the Apache arrow library you can read the data this way (in my experience the most efficient, although with the data about the size we use, it does not make much difference):
surveyResults <-
arrow::read_csv_arrow("dataExp5BOM.csv",
read_options =
arrow::CsvReadOptions$create(block_size = 10^8))Caution! The non-default settings are necessary to
enable reading data containing such long records, as those storing the
collected log-data using read_csv_arrow() (set above using
tha argument read_options).
Using the vroom package
surveyResults <- vroom::vroom("dataExp5BOM.csv")
#> Rows: 332 Columns: 270
#> ── Column specification ────────────────────────────────────────────────────────
#> Delimiter: ","
#> chr (233): condition, variant, gender, education, place_of_residence, age, ...
#> dbl (33): respid, lastpage, n_interviews_last_year, gtrust1.SQ001, gtrust3...
#> dttm (3): submitdate, startdate, datestamp
#> date (1): panel_registration_date
#>
#> ℹ Use `spec()` to retrieve the full column specification for this data.
#> ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.The easy way
If something goes wrong or you don’t want to deal with large text files by yourself, simply run:
surveyResults <- readRDS("dataExp5.RDS")Structure of the data
Survey results include different types of variables:
names(surveyResults)
#> [1] "respid" "condition"
#> [3] "submitdate" "lastpage"
#> [5] "startdate" "datestamp"
#> [7] "variant" "gender"
#> [9] "education" "place_of_residence"
#> [11] "age" "n_interviews_last_year"
#> [13] "panel_registration_date" "status"
#> [15] "vac.SQ001" "vac.SQ002"
#> [17] "vac.SQ003" "vac.SQ004"
#> [19] "vac.SQ005" "vac.SQ006"
#> [21] "vac.SQ007" "vac.SQ008"
#> [23] "vac.SQ009" "vac.SQ010"
#> [25] "immig.SQ001" "immig.SQ002"
#> [27] "immig.SQ003" "immig.SQ004"
#> [29] "immig.SQ005" "immig.SQ006"
#> [31] "immig.SQ007" "immig.SQ008"
#> [33] "read1.SQ001" "read1.SQ002"
#> [35] "read1.SQ003" "read1.SQ004"
#> [37] "read1.SQ005" "read1.SQ006"
#> [39] "read2.SQ001" "read2.SQ002"
#> [41] "read2.SQ003" "read2.SQ004"
#> [43] "read2.SQ005" "read2.SQ006"
#> [45] "read2.SQ007" "read2.SQ008"
#> [47] "read2.SQ009" "read2.SQ010"
#> [49] "read2.SQ011" "read3"
#> [51] "itrust1.SQ001" "itrust1.SQ002"
#> [53] "itrust1.SQ003" "itrust1.SQ004"
#> [55] "itrust1.SQ005" "itrust1.SQ006"
#> [57] "itrust1.SQ007" "itrust1.SQ008"
#> [59] "itrust1.SQ009" "itrust1.SQ010"
#> [61] "itrust1.SQ011" "itrust1.SQ012"
#> [63] "itrust1.SQ013" "itrust2.SQ001"
#> [65] "itrust2.SQ002" "itrust2.SQ003"
#> [67] "itrust2.SQ004" "itrust2.SQ005"
#> [69] "itrust2.SQ006" "itrust2.SQ007"
#> [71] "itrust2.SQ008" "itrust2.SQ009"
#> [73] "itrust2.SQ010" "itrust2.SQ011"
#> [75] "itrust2.SQ012" "gtrust1.SQ001"
#> [77] "gtrust3.SQ001" "gtrust2.SQ001"
#> [79] "BIS11a.SQ001" "BIS11a.SQ002"
#> [81] "BIS11a.SQ003" "BIS11a.SQ004"
#> [83] "BIS11a.SQ005" "BIS11a.SQ006"
#> [85] "BIS11a.SQ007" "BIS11a.SQ008"
#> [87] "BIS11a.SQ009" "BIS11a.SQ010"
#> [89] "BIS11b.SQ011" "BIS11b.SQ012"
#> [91] "BIS11b.SQ013" "BIS11b.SQ014"
#> [93] "BIS11b.SQ015" "BIS11b.SQ016"
#> [95] "BIS11b.SQ017" "BIS11b.SQ018"
#> [97] "BIS11b.SQ019" "BIS11b.SQ020"
#> [99] "BIS11c.SQ021" "BIS11c.SQ022"
#> [101] "BIS11c.SQ023" "BIS11c.SQ024"
#> [103] "BIS11c.SQ025" "BIS11c.SQ026"
#> [105] "BIS11c.SQ027" "BIS11c.SQ028"
#> [107] "BIS11c.SQ029" "BIS11c.SQ030"
#> [109] "DIS19a.SQ001" "DIS19a.SQ002"
#> [111] "DIS19a.SQ003" "DIS19a.SQ004"
#> [113] "DIS19a.SQ005" "DIS19a.SQ006"
#> [115] "DIS19a.SQ007" "DIS19a.SQ008"
#> [117] "DIS19a.SQ009" "DIS19a.SQ010"
#> [119] "DIS19b.SQ011" "DIS19b.SQ012"
#> [121] "DIS19b.SQ013" "DIS19b.SQ014"
#> [123] "DIS19b.SQ015" "DIS19b.SQ016"
#> [125] "DIS19b.SQ017" "DIS19b.SQ018"
#> [127] "DIS19b.SQ019" "NC01a.SQ001"
#> [129] "NC01a.SQ002" "NC01a.SQ003"
#> [131] "NC01a.SQ004" "NC01a.SQ005"
#> [133] "NC01a.SQ006" "NC01a.SQ007"
#> [135] "NC01a.SQ008" "NC01a.SQ009"
#> [137] "NC01a.SQ010" "NC01a.SQ011"
#> [139] "NC01a.SQ012" "NC01b.SQ013"
#> [141] "NC01b.SQ014" "NC01b.SQ015"
#> [143] "NC01b.SQ016" "NC01b.SQ017"
#> [145] "NC01b.SQ018" "NC01b.SQ019"
#> [147] "NC01b.SQ020" "NC01b.SQ021"
#> [149] "NC01b.SQ022" "NC01b.SQ023"
#> [151] "NC01b.SQ024" "NC01c.SQ025"
#> [153] "NC01c.SQ026" "NC01c.SQ027"
#> [155] "NC01c.SQ028" "NC01c.SQ029"
#> [157] "NC01c.SQ030" "NC01c.SQ031"
#> [159] "NC01c.SQ032" "NC01c.SQ033"
#> [161] "NC01c.SQ034" "NC01c.SQ035"
#> [163] "NC01c.SQ036" "IPIP20a.SQ001"
#> [165] "IPIP20a.SQ002" "IPIP20a.SQ003"
#> [167] "IPIP20a.SQ004" "IPIP20a.SQ005"
#> [169] "IPIP20a.SQ006" "IPIP20a.SQ007"
#> [171] "IPIP20a.SQ008" "IPIP20a.SQ009"
#> [173] "IPIP20a.SQ010" "IPIP20b.SQ011"
#> [175] "IPIP20b.SQ012" "IPIP20b.SQ013"
#> [177] "IPIP20b.SQ014" "IPIP20b.SQ015"
#> [179] "IPIP20b.SQ016" "IPIP20b.SQ017"
#> [181] "IPIP20b.SQ018" "IPIP20b.SQ019"
#> [183] "IPIP20b.SQ020" "device01"
#> [185] "device02" "device03"
#> [187] "multi01.SQ001" "multi01.SQ002"
#> [189] "multi01.SQ003" "multi01.SQ004"
#> [191] "multi01.SQ005" "multi01.SQ006"
#> [193] "multi01.SQ007" "multi01.SQ008"
#> [195] "multi01.SQ009" "multi01.other"
#> [197] "mcatt.SQ001" "mcatt.SQ002"
#> [199] "mcatt.SQ003" "mcatt.SQ004"
#> [201] "mcatt.SQ005" "mcatt.SQ006"
#> [203] "mcatt.SQ007" "mcatt.SQ008"
#> [205] "mcatt.SQ009" "mcint.SQ001"
#> [207] "mcint.SQ002" "mcint.SQ003"
#> [209] "mcint.SQ004" "mcint.SQ005"
#> [211] "mcint.SQ006" "mcint.SQ007"
#> [213] "mcint.SQ008" "mcburd.SQ001"
#> [215] "mcburd.SQ002" "mcburd.SQ003"
#> [217] "mcburd.SQ004" "mcburd.SQ005"
#> [219] "mcburd.SQ006" "lag"
#> [221] "lag_comment" "comments"
#> [223] "interviewtime" "grTime_instruction"
#> [225] "grTime_vac" "grTime_immig"
#> [227] "grTime_read1" "grTime_read2"
#> [229] "grTime_read3" "grTime_itrust1"
#> [231] "grTime_itrust2" "grTime_gtrust"
#> [233] "grTime_bis11a" "grTime_bis11b"
#> [235] "grTime_bis11c" "grTime_dis19a"
#> [237] "grTime_dis19b" "grTime_nc01a"
#> [239] "grTime_nc01b" "grTime_nc01c"
#> [241] "grTime_ipip20a" "grTime_ipip20b"
#> [243] "grTime_device" "grTime_multi"
#> [245] "grTime_mcatt" "grTime_mcint"
#> [247] "grTime_mcburd" "grTime_comments"
#> [249] "grTime_debriefing" "logvac"
#> [251] "logimmig" "logread1"
#> [253] "logread2" "logread3"
#> [255] "logitrust1" "logitrust2"
#> [257] "loggtrust" "logbis11a"
#> [259] "logbis11b" "logbis11c"
#> [261] "logdis19a" "logdis19b"
#> [263] "lognc01a" "lognc01b"
#> [265] "lognc01c" "logipip20a"
#> [267] "logipip20b" "logmcatt"
#> [269] "logmcint" "logmcburd"respidis respondent id.condition:statusare respondent’s characteristics got from the Internet panel provider and variables (paradata) describing the interview status.vac.SQ001:IPIP20b.SQ020are responses to survey questions; variable names are created in such a way that the part before the dot describes question (which is - in most cases - the equivalent of a survey screen) and the part after the dot describes the subquestion (item).- On these survey screens log-data has been collected.
- Variables
vac.SQ001:gtrust2.SQ001contain opinion scales while variablesBIS11a.SQ001:IPIP20b.SQ020contain personality scales - depending on the version of the survey these two blocks of question may be presented in a different order (compare values of the variablevariant). - Exported from the Lime Survey platform in the form of answer labels and simply read from a CSV file these responses are not analytically useful unless further transformed to factors (specyfing the order of levels in a reasonable way).
device01:device03are responses to questions about pointing device respondent used while answering the survey.- Log-data was not collected on this survey screen.
multi01.SQ001:multi01.otherare responses to the question about multitasking during the interview.- Log-data was not collected on this survey screen.
mcatt.SQ001:mcburd.SQ006are responses to questions that were aimed to measure respondent’s involvement and burden.- During these screens log-data has been collected.
- Exported from the Lime Survey platform in the form of answer labels and simply read from a CSV file these responses are not analytically useful unless further transformed to factors (specyfing the order of levels in a reasonable way).
lag:commentsare responses to the questions on the last survey screen that regarded technical difficulties during the survey.- Log-data was not collected on this survey screen.
interviewtime:grTime_debriefingare time of the whole interview along with screen times recorded by the Lime Survey platform (itself).logvac:logmcburdare variables storing recorded log-streams.
At the moment log-data is stored in the form of very long character strings, compare:
substr(surveyResults$logvac[1], 1, 1000)
#> [1] "-1;browser;'Mozilla/5.0 (Windows NT 10.0, Win64, x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36';pl-PL;;;;1920;929;|-1;screen;;;;;;1920;1080;|-1;input_position;INPUT;;;0;0,0;0;|-1;input_position;INPUT;fieldnames;;0;0,0;0;|-1;input_position;INPUT;thisstep;;0;0,0;0;|-1;input_position;INPUT;sid;;0;0,0;0;|-1;input_position;INPUT;start_time;;0;0,0;0;|-1;input_position;INPUT;LEMpostKey;;0;0,0;0;|-1;input_position;INPUT;token;;0;0,0;0;|-1;input_position;INPUT;relevance8388;;0;0,0;0;|-1;input_position;INPUT;relevance8389;;0;0,0;0;|-1;input_position;INPUT;relevanceG11;;0;0,0;0;|-1;input_position;INPUT;aQuestionsWithDependencies;;0;0,0;0;|-1;input_position;INPUT;java745195X1852X8388SQ001;;0;0,0;0;|-1;input_position;INPUT;answer745195X1852X8388SQ001-1;;17;17,763.796875;321.390625;|-1;input_position;INPUT;answer745195X1852X8388SQ001-2;;17;17,949.21875;321.390625;|-1;input_position;INPUT;answer745195X1852X8388SQ001-3;;17;17,1134.640625;321.390625;|-1;input_position;"
nchar(surveyResults$logvac[1])
#> [1] 45228(However, a single log-data record typically wouldn’t exceed 524,288 characters, as this is the limit of the HTML TEXTAREA field used to store it on the web-surveying platform.)
Preprocessing log-data
Transforming log-data streams into data frames
To transform log-data into analytically useful form, you need to use
function separate_logdata_types() providing it with a data
frame containing respondent id along with columns storing recorded
log-data streams. The additional argument
questionNamesPrefix enables to remove a prefix (“log” in
our case) from names of columns storing log-data streams, when placing
them into the variable identifying survey screen in the output data.
logData <- separate_logdata_types(surveyResults |>
select(respid, starts_with("log")),
questionNamesPrefix = "log")However, you will get much more useful data, if you provide
separate_logdata_types() also with information about survey
structure, using the surveyStructure argument. Often the
simplest way to do so is to provide names of files storing exported
survey structure using this argument (as below), but you can also read
such a file (or files) yourself and provide a resulting data frame using
the same argument.
In the case of our data, there are two survey structure files exported to TXT files (tab-separated) from the Lime Survey platform. - There are two such files because results come from technically two different surveys - each with a different variant of order of items.
logData <-
separate_logdata_types(surveyResults |>
select(respid, starts_with("log")),
surveyStructure = c("limesurvey_survey_745195.txt",
"limesurvey_survey_817417.txt"),
questionNamesPrefix = "log")
#> Processing log-data streams with:
#> - respondent's id(s) stored in columns: respid;
#> - log-data streams stored in columns: logvac, logimmig, logread1, logread2, logread3, logitrust1, logitrust2, loggtrust, logbis11a, logbis11b, logbis11c, logdis19a, logdis19b, lognc01a, lognc01b, lognc01c, logipip20a, logipip20b, logmcatt, logmcint, logmcburd.
#>
#> Preprocessing log-data streams:
#> Pivoting data with log-streams in many columns to put it into one column...
#> Separating log-streams into rows...
#> Separating log-streams into columns...
#> Marking broken records...
#> Done.
#> Separating returns to survey screens.
#> Warning in separate_logdata_types(select(surveyResults, respid,
#> starts_with("log")), : Separating returns to survey screens is not possible for
#> log-data collected using 'logdataLimeSurvey' applet in versions earlier than
#> 1.1. Argument `separateReturns` was set to 'no' automatically.
#> Processing input positions.
#> Processing system information.
#> Processing actions:
#> Computing scroll lengths...
#> Transforming mousemove events into actual moves...
#> Labeling clicks and hovers...
#> Labelling types of elements...
#> Done.
#>
#> Separated data consists of:
#> - 326 respondents on 21 screens,
#> - 6'524 respondent-screen-entries,
#> - 1'814'090 actions,
#> out of which data about 153 is somehow broken.Function separate_logdata_types() returns a list of
three data frames:
systemInfo - storing access-related paradata regarding each respondent-screen (i.e. a single row describes a specific survey screen for a specific respondent):
logData$systemInfo #> # A tibble: 6,524 × 21 #> respid screen userAgent language browserWidth browserHeight screenWidth #> <int> <chr> <chr> <chr> <chr> <chr> <chr> #> 1 58172 bis11a 'Mozilla/5.0 … pl-PL 1920 929 1920 #> 2 58172 bis11b 'Mozilla/5.0 … pl-PL 1920 929 1920 #> 3 58172 bis11c 'Mozilla/5.0 … pl-PL 1920 929 1920 #> 4 58172 dis19a 'Mozilla/5.0 … pl-PL 1920 929 1920 #> 5 58172 dis19b 'Mozilla/5.0 … pl-PL 1920 929 1920 #> 6 58172 gtrust 'Mozilla/5.0 … pl-PL 1920 929 1920 #> 7 58172 immig 'Mozilla/5.0 … pl-PL 1920 929 1920 #> 8 58172 ipip20a 'Mozilla/5.0 … pl-PL 1920 929 1920 #> 9 58172 ipip20b 'Mozilla/5.0 … pl-PL 1920 929 1920 #> 10 58172 itrust1 'Mozilla/5.0 … pl-PL 1920 929 1920 #> # ℹ 6,514 more rows #> # ℹ 14 more variables: screenHeight <chr>, inputsMinPageX <dbl>, #> # inputsMinPageY <dbl>, inputsWidth <dbl>, inputsHeight <dbl>, boxType <chr>, #> # lastTimeStampRel <dbl>, problemsAnyBroken <int>, problemsLeftBrowser <int>, #> # problemsResized <int>, problemsNoPageLoaded <int>, problemsNoSubmit <int>, #> # problemsTimeStamps <int>, problemsNoActions <int>- respondent’s web browser self-identification string,
- respondent’s screen and web browser window width and height,
- data on position and size of a rectangular area spanned by INPUT
elements that are used to mark answers in table-format questions,
- this information is useful to standardize cursor moves indicators
between different respondents; it may be also used to identify
respondents who have been shown nominally table-format questions in a
list layout because of too narrow browser window
(
inputsWidthis 0 in list layout),
- this information is useful to standardize cursor moves indicators
between different respondents; it may be also used to identify
respondents who have been shown nominally table-format questions in a
list layout because of too narrow browser window
(
- timestamp of the last recorded event,
- columns identifying some common problems that were diagnosed during
preprocessing of log-data:
problemsAnyBroken- whether any log-data regarding a given respondent-screen has been broken - if so, this typically means that problems were encountered during saving log-data stream to the Lime Survey database and actions element is probably incomplete, with some events not being recorded,problemsLeftBrowser- whether respondent left browser window (card) while answering a given screen,problemsResized- whether respondent changed the size of a browser window while answering a given screen,problemsNoPageLoaded- whether log-data for a given respondent-screen does not contain a pageLoaded event (if so, it is most probably because an old version of the JavaScript applet - that not recorded this type of events - was used to collect the log-data),problemsNoSubmit- whether log-data for a given respondent-screen does not contain a submit event (if so, it is most probably because an old version of the JavaScript applet - that not recorded this type of events - was used to collect the log-data),problemsTimeStamps- whether there is a discontinuity in values of timestamps for a given respondent-screen that results in a negative duration of some mousemove events (if so, it is most probably because an old version of the JavaScript applet - that used an unreliable method of getting timestamps - was used to collect the log-data),problemsNoActions- whether there are no actions at all in the recorded log-data for a given respondent-screen.
inputPositions - storing position of the INPUT elements which are used to mark answers to questions on the survey screen; this data is useful to draw backgrounds of plots visualizing cursor traces; also, some information included in systemInfo were computed using this data.
logData$inputPositions #> # A tibble: 343,319 × 18 #> respid screen target.tagName target.id target.class width height pageX pageY #> <int> <chr> <chr> <chr> <chr> <dbl> <dbl> <dbl> <dbl> #> 1 58172 vac INPUT answer745… "" 17 17 865. 350. #> 2 58172 vac INPUT answer745… "" 17 17 1050. 350. #> 3 58172 vac INPUT answer745… "" 17 17 1236. 350. #> 4 58172 vac INPUT answer745… "" 17 17 1421. 350. #> 5 58172 vac INPUT answer745… "" 17 17 865. 398. #> 6 58172 vac INPUT answer745… "" 17 17 1050. 398. #> 7 58172 vac INPUT answer745… "" 17 17 1236. 398. #> 8 58172 vac INPUT answer745… "" 17 17 1421. 398. #> 9 58172 vac INPUT answer745… "" 17 17 865. 455. #> 10 58172 vac INPUT answer745… "" 17 17 1050. 455. #> # ℹ 343,309 more rows #> # ℹ 9 more variables: questionId <int>, questionCode <chr>, #> # questionFormat <chr>, subquestionCode <chr>, answerCode <chr>, #> # width_rel <dbl>, height_rel <dbl>, pageX_rel <dbl>, pageY_rel <dbl>- If survey structure files were provided to
separate_logdata_types()INPUT element are matched with question code, subquestion code (if applies) and answer code, as well as with question format code (used internally by the Lime Survey platform). - This data may be used to compute number of questions and
items on a given respondent-screen by calling
count_number_of_items().
- If survey structure files were provided to
actions - storing response-related paradata, i.e. information about actual events triggered by respondent’s actions:
logData$actions #> # A tibble: 1,814,090 × 26 #> respid screen timeStamp timeStampRel type target.tagName target.id #> <int> <chr> <dbl> <dbl> <chr> <chr> <chr> #> 1 58172 vac 729. 0 mousemove TD answer745195X1… #> 2 58172 vac 968. 240. mousemove TD answer745195X1… #> 3 58172 vac 1040. 312. mouseout TD answer745195X1… #> 4 58172 vac 1040. 312. mouseover LABEL answer745195X1… #> 5 58172 vac 1080. 352. mouseout LABEL answer745195X1… #> 6 58172 vac 1080. 352. mouseover TD answer745195X1… #> 7 58172 vac 1080. 352. mousemove TD answer745195X1… #> 8 58172 vac 1112. 384. mouseout TD answer745195X1… #> 9 58172 vac 1112. 384. mouseover TD answer745195X1… #> 10 58172 vac 1128. 400. mouseout TD answer745195X1… #> # ℹ 1,814,080 more rows #> # ℹ 19 more variables: target.class <chr>, which <dbl>, metaKey <dbl>, #> # pageX <dbl>, pageY <dbl>, broken <dbl>, moveX <dbl>, moveY <dbl>, #> # duration <dbl>, moveXScrollCorrected <dbl>, moveYScrollCorrected <dbl>, #> # surveyId <chr>, questionId <int>, questionFormat <chr>, SGQA <chr>, #> # questionCode <chr>, subquestionCode <chr>, answerCode <chr>, #> # elementType <chr>- There are quite many variables in this data frame but some of them
relate only to specific types of events.
- See logDataLimeSurvey applet README for more specific information.
- If survey structure files were provided to
separate_logdata_types(), most events are matched with question code, subquestion code and answer code (if apply). - Mousemove events reported here were already transformed to describe moves (i.e. vectors) instead of only cursor positions. Moreover, distance of move is reported in two versions: with and without correction for scrolling.
- There are quite many variables in this data frame but some of them
relate only to specific types of events.
Why it is important to provide survey structure?
Providing survey structure information is important, as it enables
separate_logdata_types() to map actions (events) onto
question, subquestion and answer codes of the elements that triggered
them. Without these information your ability to compute editing
and hovering indices, and even answering speed indices will be
considerably limited.
Non-standard respondent id
In the calls to separate_logdata_types() above there
were no indicated which variable(s) in the input data constitute
respondent’s id. It is possible unless name of respondent’s id is one
(or a combination of several) of the listed below:
- id,
- token,
- respid.
Two first are default columns used to identify respondents in data exported from the Lime Survey platform. The last is my personal choice.
If in your data another column is respondent’s id, you will need to
indicate this in your call to separate_logdata_types() by
using additional respId argument. It works in a tidy-selection
way, so you may give name of the variable unquoted in a call, for
example:
anotherLogData <-
separate_logdata_types(anotherSurveyResults,
respId = someAnotherRespondentId,
surveyStructure = anotherSurveyStructurFiles)Be aware, that ability to identify respondent’s id is necessary for most of the package’s functions. So if you use non-standard variable as respondent’s id, you will need to provide it in the same way as above also in calls to another functions of the package.
Separating stagnations
The mousemove Java Script event reports cursor position, but
is triggered by the cursor being moved. As a consequence, a respondent
who does not move the cursor will be recorded in log-data returned by
separate_logdata_types() as a (relatively) long-lasting but
(probably) rather short-distanced mousemove. However, you may
want to separate such actions into two distinct ones: one
representing only a stagnation (not-moving) period and another
representing actual move (as it already started). You can do so by using
function separate_stagnations().
Unfortunately, this can only be done by dividing action using an arbitrarily chosen duration threshold, i.e. dividing all the mousemove events lasting longer than a specified value, assuming that they cover an initial period of stagnation followed by the actual move lasting for a duration equal to the chosen threshold. It is reasonable to set this threshold to the value of frequency of collecting mousemove events that was used in the Lime Survey log-data collecting applet (typically 100 ms), or perhaps to a slightly higher value.
Below we will create a copy of our log-data object and perform separation of stagnations on it using a threshold of 120 ms (the frequency of collecting mousemove events on the surveying platform was 100 ms):
logDataStag <- logData
logDataStag$actions <- separate_stagnations(logDataStag$actions, 120)
# checking how the number of mousemove actions has changed:
sum(logData$actions$type %in% "mousemove")
#> [1] 619824
sum(logDataStag$actions$type %in% "mousemove")
#> [1] 970627Let’s compare the data for some specific respondent-screen:
logData$actions |>
filter(type %in% "mousemove", respid == 58172, screen == "vac") |>
slice(1:5) |>
select(respid:type, pageX:duration)
#> # A tibble: 5 × 11
#> respid screen timeStamp timeStampRel type pageX pageY broken moveX moveY
#> <int> <chr> <dbl> <dbl> <chr> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 58172 vac 729. 0 mousemove 1053 598 0 1053 598
#> 2 58172 vac 968. 240. mousemove 1050 599 0 -3 1
#> 3 58172 vac 1080. 352. mousemove 1025 554 0 -25 -45
#> 4 58172 vac 1192. 464. mousemove 895 392 0 -130 -162
#> 5 58172 vac 3529. 2801. mousemove 892 387 0 -3 -5
#> # ℹ 1 more variable: duration <dbl>
logDataStag$actions |>
filter(type %in% "mousemove", respid == 58172, screen == "vac") |>
slice(1:7) |>
select(respid:type, pageX:duration)
#> # A tibble: 7 × 11
#> respid screen timeStamp timeStampRel type pageX pageY broken moveX moveY
#> <int> <chr> <dbl> <dbl> <chr> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 58172 vac 729. 0 mousemove 1053 598 0 1053 598
#> 2 58172 vac 848. 120. mousemove 1050 599 0 0 0
#> 3 58172 vac 968. 240. mousemove 1050 599 0 -3 1
#> 4 58172 vac 1080. 352. mousemove 1025 554 0 -25 -45
#> 5 58172 vac 1192. 464. mousemove 895 392 0 -130 -162
#> 6 58172 vac 3409. 2681. mousemove 892 387 0 0 0
#> 7 58172 vac 3529. 2801. mousemove 892 387 0 -3 -5
#> # ℹ 1 more variable: duration <dbl>Be aware that separating stagnations affects values of the cursor move average absolute accelerations computed afterwards! With stagnations separated values of these indices are higher. However, other cursor indices remain unaffected by whether stagnations were separated or not.
You will need separate_logdata_types() if you are
interested in computing process indicators like the longest period
of stagnation - it is not implemented in the package (at least at
the moment), but its computation is straightforward after separating
stagnations.
Removing problematic data
As it was discussed above, systemInfo element of a list
returned by separate_logdata_types() includes some
variables that identify common problems found in the processed log-data.
The purpose of the remove_problems() function is to assist
you in deciding whether to keep or to remove this problematic records
from a given log-data object. For each variable identifying problems it
gives you a short description of the problem, provides suggestions
whether to remove or to keep the problematic records (given what process
indicators you are going to compute further), and allows you to decide
what to do with these records.
logData <- remove_problems(logData)Be aware that the problems identified by variables created by
separate_logdata_types() may not be the only ones you
should consider!
Removing data manually
Whether you identified some other type of problem in the data or want to perform further analysis uing only a subgroup of respondents and/or screens, the preferred workflow looks as follows:
If you want to analyze only some subset of respondent-screens, for example only respondents who used a mouse as a pointing device:
- Create an object identifying the subgroup you want to analyze:
mouseOnly <- surveyResults |> filter(device02 %in% "Mouse") |> select(respid, device02) # this is not necessary but lowers memory footprint- Perform
semi_join()with this object on each element of your log-data object:
logDataMouseOnly <- lapply(logData, semi_join, y = mouseOnly, by = "respid")If you identified some problematic group and want to remove it from the analysis, for example respondents who did not successfully complete the survey:
- Create an object identifying problematic records:
dropouts <- surveyResults |> filter(status != "passed") |> # those, who did not complete the questionnaire select(respid, status) # this is not necessary but lowers memory footprint- Perform
anti_join()with this object on each element of your log-data object:
logDataPassOnly <- lapply(logData, anti_join, y = dropouts, by = "respid")
You may identify subgroups/problems also on the screen or
respondent-screen level - then you should simply accordingly modify the
set of columns you select and provide using the by
argument.
For example, below we will manually remove some data using
information stored in the systemInfo element of the
logData object:
(noProblems <- logData$systemInfo |>
filter(problemsAnyBroken != 1,
problemsLeftBrowser != 1,
problemsResized != 1,
problemsTimeStamps != 1) |>
select(respid, screen, starts_with("problems")))
#> # A tibble: 5,444 × 9
#> respid screen problemsAnyBroken problemsLeftBrowser problemsResized
#> <int> <chr> <int> <int> <int>
#> 1 58172 bis11a 0 0 0
#> 2 58172 bis11b 0 0 0
#> 3 58172 bis11c 0 0 0
#> 4 58172 dis19a 0 0 0
#> 5 58172 dis19b 0 0 0
#> 6 58172 gtrust 0 0 0
#> 7 58172 immig 0 0 0
#> 8 58172 ipip20a 0 0 0
#> 9 58172 ipip20b 0 0 0
#> 10 58172 itrust1 0 0 0
#> # ℹ 5,434 more rows
#> # ℹ 4 more variables: problemsNoPageLoaded <int>, problemsNoSubmit <int>,
#> # problemsTimeStamps <int>, problemsNoActions <int>
logData <- lapply(logData, semi_join, y = noProblems, by = c("respid", "screen"))Other log-data transforming functions
There are two other functions included in the package that perform some transformation of log-data in order to prepare it for further analysis:
compute_relative_positions()- it enhances data on actions (events), specifically mosemoves and clicks, by computing positions and move distances relative to the position of a given survey screen (extremely located) form INPUT elements. Positions and distances recalculated this way are more comparable between respondents with different browser window sizes. See section on cursor moves indices for examples of using this function.compute_cursor_positions()- it enables to compute cursor position at evenly spaced time points. Data in this form is necessary, for example, to prepare heat-map graphs illustrating the amount of time spent by the respondent over different regions of the survey screen. Example of using this function you will find in the vignette regarding drawing graphs using log-data.
Storing log-data objects
Log-data objects are large and potentially very large. To assure efficient storage and analysis, please consider the following solutions:
- Filter out and keep only these types of actions (events) you are interested in. Especially if you are not interested in cursor moves or hovering, removing mousemove or mouseover and mouseout events will greatly reduce the amount of data.
- Do not store results in CSV files. Use either .RData format (it enables to store log-data list as a single object) or Apache Arrow’s feather or parquet format to save the space and speed up reading the data.
Computing process indicators
With the log-data prepared as described in the previous section, you can easily calculate the process indicators using the package functions. In most functions you will provide actions element of a log-data object as the first argument.
| Type of process indicators | Functions | Types of events used |
|---|---|---|
| Response editing | compute_editing() |
change |
| Hovering time | compute_hovering() |
mouseover, mouseout |
| Answering time | compute_aat(), compute_aht() |
change or (mouseover, mouseout) |
| Cursor moves | compute_cursor_indices() |
mousemove |
Response editing
Use function compute_editing() to compute number of
response edits:
(edits <- compute_editing(logData$actions))
#> Preprocessing log-data streams.
#> Counting edits.
#> # A tibble: 46,873 × 5
#> respid screen questionCode subquestionCode edits
#> <int> <chr> <chr> <chr> <int>
#> 1 58172 bis11a BIS11a SQ001 1
#> 2 58172 bis11a BIS11a SQ002 1
#> 3 58172 bis11a BIS11a SQ003 1
#> 4 58172 bis11a BIS11a SQ004 1
#> 5 58172 bis11a BIS11a SQ005 1
#> 6 58172 bis11a BIS11a SQ006 1
#> 7 58172 bis11a BIS11a SQ007 1
#> 8 58172 bis11a BIS11a SQ008 1
#> 9 58172 bis11a BIS11a SQ009 2
#> 10 58172 bis11a BIS11a SQ010 1
#> # ℹ 46,863 more rowsIf you prefer to get results in a wide format with only one
row for each respondent, you should use
returnFormat = "wide" argument (be aware that because both
survey screen and question code are used to construct column names, in
one-question-per-screen survey designs column names may look somewhat
redundant).
compute_editing(logData$actions, returnFormat = "wide")
#> Preprocessing log-data streams.
#> Counting edits.
#> Pivoting results to wide format.
#> # A tibble: 324 × 193
#> respid edits_bis11a_BIS11a_SQ…¹ edits_bis11a_BIS11a_…² edits_bis11a_BIS11a_…³
#> <int> <int> <int> <int>
#> 1 58172 1 1 1
#> 2 86337 1 1 1
#> 3 90905 1 1 1
#> 4 91641 1 1 1
#> 5 93237 1 1 1
#> 6 93279 1 1 1
#> 7 95537 NA NA NA
#> 8 102147 1 1 1
#> 9 105241 1 1 1
#> 10 108285 1 1 1
#> # ℹ 314 more rows
#> # ℹ abbreviated names: ¹edits_bis11a_BIS11a_SQ001, ²edits_bis11a_BIS11a_SQ002,
#> # ³edits_bis11a_BIS11a_SQ003
#> # ℹ 189 more variables: edits_bis11a_BIS11a_SQ004 <int>,
#> # edits_bis11a_BIS11a_SQ005 <int>, edits_bis11a_BIS11a_SQ006 <int>,
#> # edits_bis11a_BIS11a_SQ007 <int>, edits_bis11a_BIS11a_SQ008 <int>,
#> # edits_bis11a_BIS11a_SQ009 <int>, edits_bis11a_BIS11a_SQ010 <int>, …Please note, that marking answer for the first times is also counted
as edit by compute_editing(), so changes
of respondent’s answer is indicated by number of reported edits greater
than 1.
Also, be aware that computing number of edits is possible only if you
have provided survey structure files while constructing log-data object
using separate_logdata_types() (and consequently if there
are columns questionCode and subquestionCode available
in your actions).
Example analysis
Now, we may check, whether the average number of edits depends on whether a given item was presented a given respondent in the first or in the second part of the questionnaire:
edits |>
inner_join(surveyResults |>
select(respid, variant),
by = "respid") |>
summarise(meanEdits = mean(edits),
.by = c(variant, screen, questionCode)) |>
mutate(typeOfScale = recode_values(screen,
c("bis11a", "bis11b", "bis11c",
"dis19a", "dis19b",
"nc01a", "nc01b", "nc01c",
"ipip20a", "ipip20b") ~ "personality",
c("gtrust", "immig", "itrust1", "itrust2",
"read1", "read2", "read3",
"vac") ~ "opinion",
c("mcatt", "mcburd",
"mcint") ~ "engagement")) |>
select(typeOfScale, screen, questionCode, variant, meanEdits) |>
arrange(typeOfScale, questionCode) |>
tidyr::pivot_wider(names_from = "variant", values_from = "meanEdits") |>
print(n = Inf)
#> # A tibble: 23 × 5
#> typeOfScale screen questionCode Personality scales f…¹ `Opinion scales first`
#> <chr> <chr> <chr> <dbl> <dbl>
#> 1 engagement mcatt mc3 1.06 1.06
#> 2 engagement mcint mc4 1.06 1.05
#> 3 engagement mcburd mc5 1.07 1.06
#> 4 opinion read1 ccz1 1.09 1.06
#> 5 opinion read2 ccz2 1.06 1.06
#> 6 opinion read3 ccz3 1.08 1.17
#> 7 opinion immig cmi 1.09 1.06
#> 8 opinion vac csz1 1.07 1.06
#> 9 opinion gtrust ctr1 1.09 1.16
#> 10 opinion gtrust ctr2 1.07 1.04
#> 11 opinion gtrust ctr3 1.09 1.10
#> 12 opinion itrus… itr1 1.06 1.06
#> 13 opinion itrus… itr2 1.06 1.06
#> 14 personality bis11a BIS11a 1.06 1.05
#> 15 personality bis11b BIS11b 1.06 1.07
#> 16 personality bis11c BIS11c 1.07 1.06
#> 17 personality dis19a DIS19a 1.07 1.09
#> 18 personality dis19b DIS19b 1.09 1.07
#> 19 personality ipip2… IPIP20a 1.06 1.06
#> 20 personality ipip2… IPIP20b 1.04 1.06
#> 21 personality nc01a NC01a 1.08 1.08
#> 22 personality nc01b NC01b 1.05 1.07
#> 23 personality nc01c NC01c 1.06 1.08
#> # ℹ abbreviated name: ¹`Personality scales first`Hovering time
Hovering indices describe how much time the cursor spent over specific elements of a given survey screen. If only questionCode, subquestionCode and answerCode columns are available in the input data, they will be used to define the structure of the returned data frame.
Please note, that computing hovering time is lengthy (here it takes
ofer one minute). By default, the function shows a progress bar, but to
avoid a mess in a static output of this vignette, the following call
uses the argument showPB=interactive(), so the progress bar
is shown only during interactive R session (and not while the .Rmd file
is compiled).
(hovering <- compute_hovering(logData$actions, showPB = interactive()))
#> Preprocessing log-data streams.
#> Calculating hovering times:
#> # A tibble: 136,885 × 7
#> respid screen questionCode subquestionCode answerCode elementType hoverTime
#> <int> <chr> <chr> <chr> <chr> <chr> <dbl>
#> 1 58172 bis11a BIS11a SQ001 2 answer cell 0.0958
#> 2 58172 bis11a BIS11a SQ001 3 answer cell 2.10
#> 3 58172 bis11a BIS11a SQ002 1 answer cell 0.888
#> 4 58172 bis11a BIS11a SQ002 2 answer cell 0.464
#> 5 58172 bis11a BIS11a SQ002 3 answer cell 0.144
#> 6 58172 bis11a BIS11a SQ003 1 answer cell 1.75
#> 7 58172 bis11a BIS11a SQ003 2 answer cell 0.536
#> 8 58172 bis11a BIS11a SQ003 3 answer cell 0.143
#> 9 58172 bis11a BIS11a SQ004 1 answer cell 0.384
#> 10 58172 bis11a BIS11a SQ004 2 answer cell 0.576
#> # ℹ 136,875 more rowsHovering times are reported in seconds.
Results returned by compute_hovering() will be typically
used for further transformations - most notably for an aggregation.
Please note, that if you are interested in the results computed on the
higher level of aggregation, you may get them directly from
compute_hovering() by removing some columns from the input
data. For example if you are interested only in total time spent over
different type of the survey interface elements, you may exclude columns
questionCode, subquestionCode and answerCode
from the input data:
compute_hovering(select(logData$actions,
-c(questionCode, subquestionCode, answerCode)),
showPB = interactive())
#> Preprocessing log-data streams.
#> Calculating hovering times:
#> # A tibble: 23,100 × 4
#> respid screen elementType hoverTime
#> <int> <chr> <chr> <dbl>
#> 1 58172 bis11a answer cell 22.9
#> 2 58172 bis11a other 1.76
#> 3 58172 bis11a question content 1.69
#> 4 58172 bis11a response scale (arrays) 0.919
#> 5 58172 bis11a submit button 0.568
#> 6 58172 bis11b answer cell 23.5
#> 7 58172 bis11b other 1.51
#> 8 58172 bis11b question content 1.66
#> 9 58172 bis11b response scale (arrays) 1.72
#> 10 58172 bis11b submit button 0.248
#> # ℹ 23,090 more rowsThere is no possibility to get hovering results in a wide format - if you want it, you must pivot the results by yourself.
Example analysis
We may check, whether there is a relationship between the length of the question content (stem) - i.e. how many characters there are in the instructions shown to the respondent at the beginning of the question - and mean hovering time over the question content (length of the question content is coded by hand in the code below):
hoveringResults <- hovering |>
filter(elementType %in% "question content") |>
inner_join(surveyResults |>
select(respid, variant),
by = "respid") |>
summarise(meanHoverTime = mean(hoverTime),
.by = c(screen, questionCode)) |>
mutate(typeOfScale = recode_values(screen,
c("bis11a", "bis11b", "bis11c",
"dis19a", "dis19b",
"nc01a", "nc01b", "nc01c",
"ipip20a", "ipip20b") ~ "personality",
c("gtrust", "immig", "itrust1", "itrust2",
"read1", "read2", "read3",
"vac") ~ "opinion",
c("mcatt", "mcburd",
"mcint") ~ "engagement"),
lengthOfQHeader = case_when(screen %in% c("bis11a", "bis11b", "bis11c") ~ 346,
screen %in% c("dis19a", "dis19b") ~ 381,
screen %in% c("nc01a", "nc01b", "nc01c") ~ 297,
screen %in% c("ipip20a", "ipip20b") ~ 545,
questionCode %in% "ctr1" ~ 135,
questionCode %in% "ctr2" ~ 148,
questionCode %in% "ctr3" ~ 137,
screen %in% "immig" ~ 626,
screen %in% c("itrust1", "itrust2") ~ 111,
screen %in% "read1" ~ 87,
screen %in% "read2" ~ 87,
screen %in% "read3" ~ 189,
screen %in% "vac" ~ 87,
screen %in% "mcatt" ~ 82,
screen %in% "mcburd" ~ 70,
screen %in% "mcint" ~ 82)) |>
select(typeOfScale, screen, questionCode, meanHoverTime, lengthOfQHeader) |>
arrange(desc(lengthOfQHeader), screen, questionCode)
print(hoveringResults, n = Inf)
#> # A tibble: 23 × 5
#> typeOfScale screen questionCode meanHoverTime lengthOfQHeader
#> <chr> <chr> <chr> <dbl> <dbl>
#> 1 opinion immig <NA> 0.572 626
#> 2 personality ipip20a IPIP20a 0.146 545
#> 3 personality ipip20b IPIP20b 0.165 545
#> 4 personality dis19a DIS19a 0.126 381
#> 5 personality dis19b DIS19b 0.146 381
#> 6 personality bis11a BIS11a 1.48 346
#> 7 personality bis11b BIS11b 0.373 346
#> 8 personality bis11c BIS11c 0.304 346
#> 9 personality nc01a NC01a 0.487 297
#> 10 personality nc01b NC01b 0.921 297
#> 11 personality nc01c NC01c 0.120 297
#> 12 opinion read3 <NA> 0.247 189
#> 13 opinion gtrust ctr2 0.278 148
#> 14 opinion gtrust ctr3 0.342 137
#> 15 opinion gtrust ctr1 0.170 135
#> 16 opinion itrust1 <NA> 0.120 111
#> 17 opinion itrust2 <NA> 0.116 111
#> 18 opinion read1 <NA> 0.0909 87
#> 19 opinion read2 <NA> 0.104 87
#> 20 opinion vac <NA> 0.175 87
#> 21 engagement mcatt <NA> 0.137 82
#> 22 engagement mcint <NA> 0.147 82
#> 23 engagement mcburd <NA> 0.0833 70
hoveringResults |>
select(meanHoverTime, lengthOfQHeader) |>
cor()
#> meanHoverTime lengthOfQHeader
#> meanHoverTime 1.000000 0.306122
#> lengthOfQHeader 0.306122 1.000000
with(hoveringResults,
{
plot(lengthOfQHeader, meanHoverTime, type = "p", cex = 2, pch = 19,
xlim = c(0, max(lengthOfQHeader)), ylim = c(0, max(meanHoverTime)))
grid()
text(lengthOfQHeader, meanHoverTime,
labels = ifelse(is.na(questionCode), screen, questionCode),
cex = 1.3, pos = 1)
abline(lm(meanHoverTime ~ lengthOfQHeader))
})
Answering time
Indices computed using changes
Several indices describing respondent’s answering speed can be
computed by calling function compute_aat() (aat stands for
Average Answering Time):
(answeringTime <- compute_aat(logData$actions))
#> Preprocessing log-data streams.
#> Computing number of items on respondent-screens.
#> Computing answering time indicators.
#> # A tibble: 5,315 × 7
#> respid screen timeToFirstAnswer averageAnsweringTime pageTimeIndex
#> <int> <chr> <dbl> <dbl> <dbl>
#> 1 58172 bis11a 4.79 2.42 2.79
#> 2 58172 bis11b 5.31 2.51 2.90
#> 3 58172 bis11c 5.34 2.80 3.18
#> 4 58172 dis19a 8.64 3.49 4.10
#> 5 58172 dis19b 5.75 4.34 4.58
#> 6 58172 gtrust 5.36 NA NA
#> 7 58172 immig 8.19 3.86 4.87
#> 8 58172 ipip20a 11.9 2.41 3.53
#> 9 58172 ipip20b 3.32 2.58 2.76
#> 10 58172 itrust1 7.41 2.33 2.80
#> # ℹ 5,305 more rows
#> # ℹ 2 more variables: averageAnsweringTimeAll <dbl>, pageTimeIndexAll <dbl>The indices differ with respect to whether they:
- Include all the time spent on the survey screen - pageTimeIndex, pageTimeIndexAll;
- Estimate only the time spent on answering the items, ruling
out the time spent on reading the question stem -
averageAnsweringTime, averageAnsweringTimeAll;
- The idea of the index is illustrated on the Ulf Kröhne’s website.
Morevoer, the indices differ with respect to whether they:
- Include only the items (subquestions) that the respondent answered (ignoring those that were omitted without providing an answer) - pageTimeIndex, averageAnsweringTime;
- Include all the items (subquestions), irrespective they
were answered or not - pageTimeIndexAll,
averageAnsweringTimeAll.
- Please note, that by default the number of all the items
(subquestions) is estimated from the input data, what means that if all
the respondents omitted some item it won’t be taken into
account. To solve this problem you may use additional argument
numberOfItemsalong with results of a call tocount_number_of_items(logData$inputPositions).
- Please note, that by default the number of all the items
(subquestions) is estimated from the input data, what means that if all
the respondents omitted some item it won’t be taken into
account. To solve this problem you may use additional argument
By default if there were several questions on the same survey screen,
NA are returned as values of the aforementioned indices
(that is because they were designed for tabular-format questions). If
you want to change this behavior and get values of indices also for
screens containing several questions, you should set argument
multipleQuestionsAction="keep" (compare values in the 6th
row of the results with the call above):
(answeringTime <- compute_aat(logData$actions, multipleQuestionsAction = "keep"))
#> Preprocessing log-data streams.
#> Computing number of items on respondent-screens.
#> Computing answering time indicators.
#> # A tibble: 5,315 × 7
#> respid screen timeToFirstAnswer averageAnsweringTime pageTimeIndex
#> <int> <chr> <dbl> <dbl> <dbl>
#> 1 58172 bis11a 4.79 2.42 2.79
#> 2 58172 bis11b 5.31 2.51 2.90
#> 3 58172 bis11c 5.34 2.80 3.18
#> 4 58172 dis19a 8.64 3.49 4.10
#> 5 58172 dis19b 5.75 4.34 4.58
#> 6 58172 gtrust 5.36 4.72 5.18
#> 7 58172 immig 8.19 3.86 4.87
#> 8 58172 ipip20a 11.9 2.41 3.53
#> 9 58172 ipip20b 3.32 2.58 2.76
#> 10 58172 itrust1 7.41 2.33 2.80
#> # ℹ 5,305 more rows
#> # ℹ 2 more variables: averageAnsweringTimeAll <dbl>, pageTimeIndexAll <dbl>To get results in a wide format with only one row for each
respondent, you may use returnFormat="wide" argument:
compute_aat(logData$actions, returnFormat = "wide")
#> Preprocessing log-data streams.
#> Computing number of items on respondent-screens.
#> Computing answering time indicators.
#> Pivoting results to wide format.
#> # A tibble: 324 × 106
#> respid timeToFirstAnswer_bis11a timeToFirstAnswer_bi…¹ timeToFirstAnswer_bi…²
#> <int> <dbl> <dbl> <dbl>
#> 1 58172 4.79 5.31 5.34
#> 2 86337 1.33 6.37 2.01
#> 3 90905 4.72 4.15 2.48
#> 4 91641 73.1 3.87 2.40
#> 5 93237 4.14 0.907 5.62
#> 6 93279 1.69 2.03 1.85
#> 7 95537 NA 9.74 1.55
#> 8 102147 9.21 5.24 1.20
#> 9 105241 13.6 NA 1.33
#> 10 108285 22.6 20.4 17.0
#> # ℹ 314 more rows
#> # ℹ abbreviated names: ¹timeToFirstAnswer_bis11b, ²timeToFirstAnswer_bis11c
#> # ℹ 102 more variables: timeToFirstAnswer_dis19a <dbl>,
#> # timeToFirstAnswer_dis19b <dbl>, timeToFirstAnswer_gtrust <dbl>,
#> # timeToFirstAnswer_immig <dbl>, timeToFirstAnswer_ipip20a <dbl>,
#> # timeToFirstAnswer_ipip20b <dbl>, timeToFirstAnswer_itrust1 <dbl>,
#> # timeToFirstAnswer_itrust2 <dbl>, timeToFirstAnswer_mcatt <dbl>, …Example analysis
Let’s check whether respondents (who got to the end of the questionnaire) of different age differ in the speed of answering:
answeringTime |>
inner_join(surveyResults |>
filter(!(status %in% "not completed")) |>
select(respid, age),
by = "respid") |>
# we have to substitute infinite values with NAs first
mutate(averageAnsweringTime = ifelse(is.infinite(averageAnsweringTime),
NA_real_, averageAnsweringTime),
averageAnsweringTimeAll = ifelse(is.infinite(averageAnsweringTimeAll),
NA_real_, averageAnsweringTimeAll)) |>
# and only then aggregate
summarise(across(timeToFirstAnswer:pageTimeIndexAll, ~mean(., na.rm = TRUE)),
.by = c(age))
#> # A tibble: 3 × 6
#> age timeToFirstAnswer averageAnsweringTime pageTimeIndex
#> <chr> <dbl> <dbl> <dbl>
#> 1 30-39 6.40 2.72 3.74
#> 2 40-50 7.65 3.42 4.69
#> 3 18-29 8.19 2.86 4.37
#> # ℹ 2 more variables: averageAnsweringTimeAll <dbl>, pageTimeIndexAll <dbl>Indices compute using hovering times
Alternatively, indices describing answering speed can be computed using previously prepared data on hovering times:
(answeringTimeHover <- compute_aht(hovering))
#> Computing number of items on respondent-screens.
#> Computing answering time indicators.
#> # A tibble: 5,410 × 5
#> respid screen totalTime averageHoverTimeAll pageTimeIndexAll
#> <int> <chr> <dbl> <dbl> <dbl>
#> 1 58172 bis11a 27.9 2.29 2.79
#> 2 58172 bis11b 28.8 2.35 2.88
#> 3 58172 bis11c 31.7 3.04 3.17
#> 4 58172 dis19a 41.0 3.80 4.10
#> 5 58172 dis19b 41.2 4.18 4.58
#> 6 58172 gtrust 15.6 NA NA
#> 7 58172 immig 38.9 3.57 4.87
#> 8 58172 ipip20a 35.3 2.32 3.53
#> 9 58172 ipip20b 27.6 2.66 2.76
#> 10 58172 itrust1 36.4 2.70 2.80
#> # ℹ 5,400 more rowsThis function returns alternative estimate of the pageTimeIndexAll along with the indicator averageHoverTimeAll reporting average time of mouse cursor hovering over a single item (either over a content or answers), taking into account all the items on a given survey screen irrespective they were answered or omitted. Also, survey screen (totalTime) is returned.
As in the case of compute_aat() you may use arguments
multipleQuestionsAction = "keep") and
returnFormat = "wide" while calling
compute_aht():
compute_aht(hovering, multipleQuestionsAction = "keep")
#> Computing number of items on respondent-screens.
#> Computing answering time indicators.
#> # A tibble: 5,410 × 5
#> respid screen totalTime averageHoverTimeAll pageTimeIndexAll
#> <int> <chr> <dbl> <dbl> <dbl>
#> 1 58172 bis11a 27.9 2.29 2.79
#> 2 58172 bis11b 28.8 2.35 2.88
#> 3 58172 bis11c 31.7 3.04 3.17
#> 4 58172 dis19a 41.0 3.80 4.10
#> 5 58172 dis19b 41.2 4.18 4.58
#> 6 58172 gtrust 15.6 1.66 5.18
#> 7 58172 immig 38.9 3.57 4.87
#> 8 58172 ipip20a 35.3 2.32 3.53
#> 9 58172 ipip20b 27.6 2.66 2.76
#> 10 58172 itrust1 36.4 2.70 2.80
#> # ℹ 5,400 more rows
compute_aht(hovering, returnFormat = "wide")
#> Computing number of items on respondent-screens.
#> Computing answering time indicators.
#> Pivoting results to wide format.
#> # A tibble: 324 × 64
#> respid totalTime_bis11a totalTime_bis11b totalTime_bis11c totalTime_dis19a
#> <int> <dbl> <dbl> <dbl> <dbl>
#> 1 58172 27.9 28.8 31.7 41.0
#> 2 86337 41.5 37.9 53.3 101.
#> 3 90905 45.8 28.0 40.9 NA
#> 4 91641 95.1 37.8 34.6 NA
#> 5 93237 26.7 19.7 35.5 119.
#> 6 93279 9.89 9.92 8.42 12.2
#> 7 95537 NA 20.1 10.4 45.9
#> 8 102147 51.4 56.4 48.9 46.8
#> 9 105241 37.7 NA 30.8 54.9
#> 10 108285 49.5 67.0 56.2 107.
#> # ℹ 314 more rows
#> # ℹ 59 more variables: totalTime_dis19b <dbl>, totalTime_gtrust <dbl>,
#> # totalTime_immig <dbl>, totalTime_ipip20a <dbl>, totalTime_ipip20b <dbl>,
#> # totalTime_itrust1 <dbl>, totalTime_itrust2 <dbl>, totalTime_mcatt <dbl>,
#> # totalTime_mcburd <dbl>, totalTime_mcint <dbl>, totalTime_nc01b <dbl>,
#> # totalTime_nc01c <dbl>, totalTime_read1 <dbl>, totalTime_read2 <dbl>,
#> # totalTime_read3 <dbl>, totalTime_vac <dbl>, totalTime_nc01a <dbl>, …Example analysis
It is interesting to check, how the aat and aht results match:
answeringTimeATTvsAHT <-
inner_join(answeringTimeHover |>
rename(pageTimeIndexAllAHT = pageTimeIndexAll),
answeringTime |>
select(respid, screen, averageAnsweringTimeAll,
pageTimeIndexAllATT = pageTimeIndexAll),
by = c("respid", "screen")) |>
# we have to substitute infinite values with NAs first
mutate(averageHoverTimeAll = ifelse(is.infinite(averageHoverTimeAll),
NA_real_, averageHoverTimeAll),
averageAnsweringTimeAll = ifelse(is.infinite(averageAnsweringTimeAll),
NA_real_, averageAnsweringTimeAll))
cor(answeringTimeATTvsAHT |>
select(averageHoverTimeAll, pageTimeIndexAllAHT),
answeringTimeATTvsAHT |>
select(averageAnsweringTimeAll, pageTimeIndexAllATT),
use = "complete.obs")
#> averageAnsweringTimeAll pageTimeIndexAllATT
#> averageHoverTimeAll 0.8876763 0.8568470
#> pageTimeIndexAllAHT 0.8412659 0.9375719
with(answeringTimeATTvsAHT,
{
plot(averageAnsweringTimeAll, averageHoverTimeAll,
type = "p", cex = 1.5, pch = 19)
grid()
abline(a = 0, b = 1, lty = 2, col = "darkblue")
abline(lm(averageHoverTimeAll ~ averageAnsweringTimeAll), col = "red")
})
with(answeringTimeATTvsAHT,
{
plot(pageTimeIndexAllATT, pageTimeIndexAllAHT,
type = "p", cex = 1.5, pch = 19)
grid()
abline(a = 0, b = 1, lty = 2, col = "darkblue")
abline(lm(pageTimeIndexAllAHT ~ pageTimeIndexAllATT), col = "red")
})
Cursor moves indices
To compute several indices describing the way the respondent moves a cursor on a survey screen you may run:
(mouseMoves <- compute_cursor_indices(logData$actions))
#> Computing indices for individual moves.
#> Computing scrolling-corrected versions.
#> Aggregating indices.
#> # A tibble: 5,401 × 18
#> respid screen dX dX_sc dY dY_sc flipsX flipsX_sc flipsY flipsY_sc
#> <int> <chr> <dbl> <dbl> <dbl> <dbl> <int> <int> <int> <int>
#> 1 58172 bis11a 4577 4577 1470 1470 40 40 48 48
#> 2 58172 bis11b 6205 6205 2293 2293 38 38 47 47
#> 3 58172 bis11c 4085 4085 1549 1549 33 33 49 49
#> 4 58172 dis19a 3445 3445 1970 1806 40 40 58 56
#> 5 58172 dis19b 3636 3636 2282 2285 28 28 52 52
#> 6 58172 gtrust 1534 1534 1064 1064 16 16 19 19
#> 7 58172 immig 3534 3534 2421 2340 33 33 45 43
#> 8 58172 ipip20a 3731 3731 2289 2310 40 40 38 36
#> 9 58172 ipip20b 3819 3819 2103 2140 32 32 36 34
#> 10 58172 itrust1 3485 3485 1516 1608 42 42 50 50
#> # ℹ 5,391 more rows
#> # ℹ 8 more variables: vX <dbl>, aX <dbl>, vX_sc <dbl>, aX_sc <dbl>, vY <dbl>,
#> # aY <dbl>, vY_sc <dbl>, aY_sc <dbl>There are 4 main types of cursor moves indices:
- dX, dY - total distance traveled on the horizontal and vertical axis, respectively [px];
- vX, vY - average vertical speed on the horizontal and vertical axis, respectively [px/s];
- aX, aY - average absolute vertical acceleration on the horizontal and vertical axis, respectively [px/s^2];
- flipsX, flipsY - number of flips (changing direction of move) on the horizontal and vertical axis, respectively.
Along with regular versions, there are also scrolling corrected versions returned:
- These indices are identified by the suffix *_sc*.
- These indices are computed ruling out cursor moves over the survey screen that occurred because of the other actions than moving a pointing device (in particular: because of scrolling - either using mouse wheel, touchpad gestures or scrolling bar - using arrows to scroll the page or using TAB key to switch between survey form INPUT fields).
Moreover, you can get relative versions of the indices:
- Values of this indices are computing by dividing a regular and/or scrolling corrected ones by the width or height of a rectangle spanned by the most upper-left and the most bottom-right survey form INPUT element’s position on a given survey screen. Values of these indices ones are better comparable between respondents with different browser window size.
To get also this versions, you need to call
compute_relative_positions() on the actions
element (you need to provide also the systemInfo element of a
log-data object as the second argument) before passing it into
compute_cursor_indices():
(logData$actions <- compute_relative_positions(logData$actions,
logData$systemInfo))
#> # A tibble: 1,472,857 × 32
#> respid screen timeStamp timeStampRel type target.tagName target.id
#> <int> <chr> <dbl> <dbl> <chr> <chr> <chr>
#> 1 58172 vac 729. 0 mousemove TD answer745195X1…
#> 2 58172 vac 968. 240. mousemove TD answer745195X1…
#> 3 58172 vac 1040. 312. mouseout TD answer745195X1…
#> 4 58172 vac 1040. 312. mouseover LABEL answer745195X1…
#> 5 58172 vac 1080. 352. mouseout LABEL answer745195X1…
#> 6 58172 vac 1080. 352. mouseover TD answer745195X1…
#> 7 58172 vac 1080. 352. mousemove TD answer745195X1…
#> 8 58172 vac 1112. 384. mouseout TD answer745195X1…
#> 9 58172 vac 1112. 384. mouseover TD answer745195X1…
#> 10 58172 vac 1128. 400. mouseout TD answer745195X1…
#> # ℹ 1,472,847 more rows
#> # ℹ 25 more variables: target.class <chr>, which <dbl>, metaKey <dbl>,
#> # pageX <dbl>, pageY <dbl>, broken <dbl>, moveX <dbl>, moveY <dbl>,
#> # duration <dbl>, moveXScrollCorrected <dbl>, moveYScrollCorrected <dbl>,
#> # surveyId <chr>, questionId <int>, questionFormat <chr>, SGQA <chr>,
#> # questionCode <chr>, subquestionCode <chr>, answerCode <chr>,
#> # elementType <chr>, pageX_rel <dbl>, pageY_rel <dbl>, moveX_rel <dbl>, …
(mouseMoves <- compute_cursor_indices(logData$actions))
#> Computing indices for individual moves.
#> Computing scrolling-corrected versions.
#> Computing relative variants.
#> Computing relative scrolling-corrected variants.
#> Aggregating indices.
#> # A tibble: 5,401 × 30
#> respid screen dX dX_sc dX_rel dX_screl dY dY_sc dY_rel dY_screl flipsX
#> <int> <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <int>
#> 1 58172 bis11a 4577 4577 8.23 8.23 1470 1470 3.95 3.95 40
#> 2 58172 bis11b 6205 6205 11.2 11.2 2293 2293 5.63 5.63 38
#> 3 58172 bis11c 4085 4085 7.34 7.34 1549 1549 3.98 3.98 33
#> 4 58172 dis19a 3445 3445 5.81 5.81 1970 1806 3.75 3.44 40
#> 5 58172 dis19b 3636 3636 6.13 6.13 2282 2285 5.21 5.21 28
#> 6 58172 gtrust 1534 1534 4.48 4.48 1064 1064 2.76 2.76 16
#> 7 58172 immig 3534 3534 5.24 5.24 2421 2340 8.26 7.98 33
#> 8 58172 ipip20a 3731 3731 6.29 6.29 2289 2310 6.46 6.52 40
#> 9 58172 ipip20b 3819 3819 6.44 6.44 2103 2140 5.93 6.04 32
#> 10 58172 itrust1 3485 3485 5.48 5.48 1516 1608 3.21 3.40 42
#> # ℹ 5,391 more rows
#> # ℹ 19 more variables: flipsX_sc <int>, flipsY <int>, flipsY_sc <int>,
#> # vX <dbl>, aX <dbl>, vX_sc <dbl>, aX_sc <dbl>, vY <dbl>, aY <dbl>,
#> # vY_sc <dbl>, aY_sc <dbl>, vX_rel <dbl>, aX_rel <dbl>, vX_screl <dbl>,
#> # aX_screl <dbl>, vY_rel <dbl>, aY_rel <dbl>, vY_screl <dbl>, aY_screl <dbl>Also mouse move indices can be returned in a wide format:
compute_cursor_indices(logData$actions, returnFormat = "wide")
#> Computing indices for individual moves.
#> Computing scrolling-corrected versions.
#> Computing relative variants.
#> Computing relative scrolling-corrected variants.
#> Aggregating indices.
#> Pivoting results to wide format.
#> # A tibble: 324 × 589
#> respid dX_bis11a dX_bis11b dX_bis11c dX_dis19a dX_dis19b dX_gtrust dX_immig
#> <int> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 58172 4577 6205 4085 3445 3636 1534 3534
#> 2 86337 2936 3169 3702 4301 3426 1369 1770
#> 3 90905 4690 4654 4661 NA NA 2381 NA
#> 4 91641 7554 6526 5956 NA 7251 3231 5779
#> 5 93237 3377 2049 2351 2053 2377 NA NA
#> 6 93279 844 687 663 827 417 962 NA
#> 7 95537 NA 1385 1273 2618 1687 1624 725
#> 8 102147 2650 4347 2495 4586 2573 2554 1694
#> 9 105241 3077 NA 2643 4094 3261 1868 3229
#> 10 108285 5421 7054 4657 9066 3603 4766 57
#> # ℹ 314 more rows
#> # ℹ 581 more variables: dX_ipip20a <dbl>, dX_ipip20b <dbl>, dX_itrust1 <dbl>,
#> # dX_itrust2 <dbl>, dX_mcatt <dbl>, dX_mcburd <dbl>, dX_mcint <dbl>,
#> # dX_nc01b <dbl>, dX_nc01c <dbl>, dX_read1 <dbl>, dX_read2 <dbl>,
#> # dX_read3 <dbl>, dX_vac <dbl>, dX_nc01a <dbl>, dX_sc_bis11a <dbl>,
#> # dX_sc_bis11b <dbl>, dX_sc_bis11c <dbl>, dX_sc_dis19a <dbl>,
#> # dX_sc_dis19b <dbl>, dX_sc_gtrust <dbl>, dX_sc_immig <dbl>, …Example analysis
Let’s compare the values of the relative, scroll-corrected cursor move indices between respondents who were among 25% most active web-panel members and among 25% less active panel members, and in between this thresholds:
mouseMoves |>
inner_join(surveyResults |>
select(respid, n_interviews_last_year) |>
mutate(webPanelActivity =
cut(n_interviews_last_year,
c(-Inf,
quantile(n_interviews_last_year, c(0.25, 0.75)),
Inf),
labels = c("Lowest 25%", "Mediocre", "Highest 25%"))),
by = "respid") |>
summarise(across(c(ends_with("_screl"), "flipsX", "flipsY"),
~mean(ifelse(is.infinite(.), NA_real_, .), na.rm = TRUE)),
.by = c(screen, webPanelActivity)) |>
arrange(screen, webPanelActivity)
#> # A tibble: 63 × 10
#> screen webPanelActivity dX_screl dY_screl vX_screl aX_screl vY_screl aY_screl
#> <chr> <fct> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 bis11a Lowest 25% 7.65 3.85 0.000226 1.99e-6 0.000128 1.20e-6
#> 2 bis11a Mediocre 5.78 3.57 0.000238 1.99e-6 0.000157 1.22e-6
#> 3 bis11a Highest 25% 4.95 3.09 0.000213 1.88e-6 0.000164 1.39e-6
#> 4 bis11b Lowest 25% 8.28 3.76 0.000264 2.33e-6 0.000135 1.15e-6
#> 5 bis11b Mediocre 6.71 3.60 0.000254 2.33e-6 0.000157 1.41e-6
#> 6 bis11b Highest 25% 5.45 3.41 0.000316 2.60e-6 0.000224 1.71e-6
#> 7 bis11c Lowest 25% 6.75 3.53 0.000234 2.02e-6 0.000159 1.22e-6
#> 8 bis11c Mediocre 5.53 3.35 0.000240 2.25e-6 0.000203 1.60e-6
#> 9 bis11c Highest 25% 5.29 3.64 0.000192 1.77e-6 0.000160 1.35e-6
#> 10 dis19a Lowest 25% 7.12 3.36 0.000172 1.95e-6 0.000106 1.47e-6
#> # ℹ 53 more rows
#> # ℹ 2 more variables: flipsX <dbl>, flipsY <dbl>