5 Studies and results
5.2 Study II: An action uncertainty model for car following
Figure 8: Sample time series showing time headway and the subsequent occlusion dura-tion at glance onset moments. The sample trials are, from top to bottom, 25th, 50th and 75th percentile by Spearman correlation between the detrended instantaneous time gaps and occlusion durations. The respective correlations are 0.40, 0.56 and 0.68.
mand can be also attained by compensating for increase in demand with increase in capability attained with allocating more attention to the driving task.
5.2 Study II: An action uncertainty model for car following
The experimental part of Study II (Pekkanen et al., 2018) is essentially a replica-tion of the Study I’s setting, but with more immersive virtual reality (VR) driving simulator and real cars on a test track (see figure 9) and a larger and more diverse sample of 40 participants (of which three could not finish the VR experiment due to discomfort). Study II also introduces and validates a model (see section 4.3) that proposes an explanation forwhy shorter THW situations demand more visual attention. The model is implemented computationally and used to produce simu-lated task performance to study how the experimentally found connection between THW and OD can emerge from control of action uncertainty.
The analyses of Study II focus only on the occluded driving task, although unoccluded trials were driven as well. This is partly for practical purposes, as the article is quite lengthy as is, but more importantly because the instruction given to the participants was slightly different. In this experiment drivers could not be directed to drive at the limit of their capability due to safety issues when driving with a real car, and were instead instructed to ”drive as they normally would” but at the same time to take as few ”glances” as possible. Drivers likely don’t drive
Figure 9: Participant view in the real car (top) and virtual reality (VR) simulator (bottom) car following experiments with the occluder off (left) and on (right) in Study II. The driving simulator is mostly the same as in Study I, but with some cosmetic differences and slightly different leading vehicle behavior.
using full capacity while driving normally and thus variation in total capacity allocated to the experimental task causes some difficult to measure variation in the driving behavior.
A strong and seemingly linear relationship, as predicted by the interpretation proposed in Study I, was found between participants’ median THWs and ODs (see figure 10). Although the exact parameterization proposed earlier was statistically ruled out in both experiments, it is still a reasonable approximation of the median THW-OD connection. Also the covariation between instantaneous THWs and ODs was replicated, although not as robustly: a positive Spearman correlation between instantaneous THWs and ODs was found for 28 of the 40 participants in the real car experiment (Binomial testp= 0.02) and for 31 of the 37 in the VR experiment (p = 0.00004). The correlation was also weaker than in Study I, with real car
grand median 0.15 and VR grand median 0.19.
In addition to the THW and OD measures used in Study I, Study II also addresses the acceleration behavior of the drivers, as it is required for calibrating the developed model. The virtual reality simulator and real car experiments are found to be in quite good agreement with each other for THW and OD, but the acceleration behavior differs substantially and is not systematically related between the VR and real car experiments (see figure 11). The reason for this is not entirely clear, but is likely a combination of much more responsive vehicle and
0 2 4 6 8 10 12
6B;m`2 Ry, "2ir22M@T`iB+BTMib `2HiBQMb?BTb Q7 K2/BM iBK2 ?2/rv M/ Q++HmbBQM /m`iBQM BM i?2 `2H +` UL49yV M/ bBKmHiQ` UL4jdV 2tT2`BK2MibX .Qib BM/B+i2 T2`@
bm#D2+i K2/BMb- /b?2/ HBM2 Bb QM2@iQ@QM2- #H+F HBM2 Bb SbbBM;@ "#HQF `2;`2bbBQM HBM2 M/ ;`v `2 Bib N8W +QM}/2M+2 BMi2`pHX :`v bQHB/ HBM2 Bb aim/v AǶb T`QTQbH i?i K2/BM P. +Q``2bTQM/b QM2@iQ@QM2 rBi? BM+`2b2 BM K2/BM h>q M/ i?i ivTB+H 7mHH@
+T+Biv K2/BM iBK2 ?2/rv Bb TT`QtBKi2Hv QM2 b2+QM/X h?2 `2HiBQMb?BT Bb [mBi2 bi`QM; M/ `2HiBp2Hv r2HH 2tTHBM2/ #v HBM2` `2HiBQMb?BT U`2H +` S2`bQMρP = 0.76
6B;m`2 RR, *QKT`BbQM Q7 i?2 jd T`iB+BTMibǶ ;;`2;i2 K2bm`2b BM i?2 o_ M/ `2H +`
2tT2`BK2MibX .Qib BM/B+i2 bm#D2+i K2/BM- /b?2/ HBM2 Bb QM2@iQ@QM2 `2HiBQMb?BT- #H+F HBM2 b?Qrb SbbBM;@"#HQF `2;`2bbBQM `2bmHi M/ i?2 ;`v `2 Bib N8W +QM}/2M+2 BMi2`@
pHX hBK2 ?2/rvb US2`bQMǶbρP=0.78V M/ Q++HmbBQM /m`iBQMb UρP=0.83V +Q``2bTQM/
[mBi2 +HQb2Hv #2ir22M i?2 irQ 2tT2`BK2Mib- rBi? HKQbi QM2@iQ@QM2 HBM2` `2HiBQMb?BTbX
*Q``2HiBQM Q7 NNi? T2`+2MiBH2 Q7 ++2H2`iBQMb Bb HQr iQ MQM@2tBbi2Mi UρP = 0.29-p= 0.09V M/ o_ bBKmHiQ` `2bmHib BM #Qmi irB+2 H`;2` ++2H2`iBQM T2Fb QM p2`;2X
lack of somatosensory acceleration information in the simulator.
To understand how drivers control their speed during visual occlusion and how they estimate when they have to remove the occlusion, Study II presents a computational model (see section 4.3) that can conduct the car following task using imperfect and controllably intermittent input, i.e. do essentially the same task that the participants did in the experiment. The model assumes that drivers maintain a cognitive representation of the task environment, namely own speed and speed and distance relative to the leading vehicle, and selects acceleration actions based on the estimate. These quantities can not be observed directly, but are estimated from perceptual quantities, namely optical flow of the ground and size of the visual projection of the leading car and its expansion or contraction. Observation of these quantities is imperfect, i.e. corrupted with noise, which the driver takes into account when using the observations to estimate the environment’s state.
The modeled driver also has some assumptions of how the world works: it knows that speed causes change in position and acceleration change in speed over time, assumes a statistical distribution of how the leading car accelerates on average and knows approximately how its control actions affect the speed of its own vehicle.
Notably for the question of online versus representation-based control the model acts based on the representation’s estimated state even when the input is available, and thus omission of input does not require additional strategies. The representation-based estimate is useful even when sensory information is available:
if the assumptions of the world’s functioning are correct enough, they lead to bet-ter estimates and betbet-ter control than using the noisy sensory input alone. When input is missing, it becomes critical: with the internal representation the driver can ”predict” with some accuracy how the scene evolves and base the control ac-tions on this prediction. Furthermore, because the state estimate is probabilistic, the driver knows how uncertainty in the estimate, and thus uncertainty in what action to take, evolves and can use this uncertainty to decide when it needs more sensory input to be confident enough in what action to take. This intermittent sampling was implemented in the model as removing the occlusion for the 300 ms glance whenever uncertainty in the acceleration selection, operationalized as standard deviation of the acceleration distribution, reached a threshold.
To compare the model’s behavior to the human participants the model was calibrated against each participant’s data and simulated trials were generated using the resulting parameterizations, separately for the real car and VR experiments.
Overall, the model can conduct the task with similar success as the human drivers:
in the simulated data trials end in a collision in approximately 1.1% of real car and 0.8% of VR trials whereas for human drives the respective rates were 0% (95%
0 2 4 6 8 10 12
6B;m`2 Rk, JQ/2H@T`2/B+i2/ `2HiBQMb?BTb Q7 K2/BM iBK2 ?2/rv M/ Q++HmbBQM /m@
`iBQM i /Bz2`2Mi mM+2`iBMiv i?`2b?QH/bX h?2 +QHQ`2/ HBM2b `2T`2b2Mi i?2 bBKmHi2/
`2HiBQMb?BT i 2biBKi2/ 8i?- 8yi? M/ N8i? T2`+2MiBH2 mM+2`iBMiv i?`2b?QH/b Q7 i?2 bm#D2+i TQTmHiBQM BM i?2 `2H +` 2tT2`BK2MiX .Qib `2T`2b2Mi T2`@bm#D2+i h>q M/ P.
K2/BMb Ui?2 pHm2b `2 B/2MiB+H rBi? i?2 H27i TM2H Q7 };m`2 RyV rBi? +QHQ` BM/B+iBM;
i?2 bm#D2+iǶb 2biBKi2/ mM+2`iBMiv i?`2b?QH/X h?2 /b?2/ HBM2 Bb QM2@iQ@QM2X
*A yĜkX9WV M/ jXjW UN8W *A yXNĜ3X9WVX h?2 bBKmHi2/ /i HbQ `2T`Q/m+2b i?2 K2/BM h>q@P. +QMM2+iBQM mbBM; i?2 +iBQM mM+2`iBMiv +QMi`QH K2+?MBbK M/
2t?B#Bib MQM@HBM2` h>q@P. +QMM2+iBQM- r?B+? Bb KQ/mHi2/ #v i?2 BM/BpB/mHǶb +iBQM mM+2`iBMiv i?`2b?QH/ U};m`2 RkVX
h?2 `2T`Q/m+iBQM Q7 i?2 K2/BM h>q@P. +QMM2+iBQM Bb `;m#Hv bQK2r?i i`BpBH `2bmHi- b i?2 KQ/2H rb +HB#`i2/ ;BMbi T2`@T`iB+BTMi K2/BM h>q M/ P.- Hi?Qm;? Bi Bb MQi Q#pBQmb i?i i?2 KQ/2H +M T2`7Q`K i?Bb rBi?Qmi 2t+2b@
bBp2 +`b?BM; rBi? i?2b2 T`K2i2`BxiBQMbX >Qr2p2`- i?2 KQ/2H HbQ [mHBiiBp2Hv
`2T`Q/m+2b i?2 +Qp`BiBQM #2ir22M BMbiMiM2Qmb H2p2H h>q M/ P.- rBi? ;`M/
K2/BM aT2`KM +Q``2HiBQMb yXe8 M/ yX98 7Q` i?2 `2H +` M/ o_ T`K2i2`@
BxiBQMb `2bT2+iBp2HvX h?Bb `2T`Q/m+iBQM /Q2b MQi b?`2 i?2 `;m#H2 i`BpBHBiv Q7 i?2 K2/BM H2p2H +QMM2+iBQMc i?2 +HB#`iBQM T`Q+2/m`2 /Q2b MQi mb2 BMbiMiM2Qmb H2p2H /i M/ MQ h>q@P. +QMM2+iBQM Bb 2tTHB+BiHv BM+Hm/2/ BM i?2 KQ/2H- #mi Bi 2K2`;2b 7`QK i?2 KQK2Mi@iQ@KQK2Mi +QMi`QH Q7 mM+2`iBMiv BM i?2 ++2H2`iBQM +iBQMX h?mb i?2 KQ/2H T`QTQb2b M 2tTHMiBQM r?v /`Bp2`b i2M/ iQ M22/ KQ`2 pBbmH BM7Q`KiBQM BM i?2 b?Q`i2` h>q bBimiBQMb, i?2 mM+2`iBMiv Q7 ?Qr Km+?
++2H2`iBQM Q` /2+2H2`iBQM b?QmH/ #2 TTHB2/ i2M/b iQ #2 ?B;?2` BM i?2b2 bBim@
iBQMbX
6B;m`2 Rj, M 2tKTH2 Q7 ;x2 #2?pBQ` /m`BM; /`BpBM;X h?2 /`Bp2` }ti2b 7Qm` /Bz2`2Mi i`;2ib BM mM/2` QM2 b2+QM/X //BiBQMHHv- HH i?2 i`;2ib `2 KQpBM; `2HiBp2 iQ i?2 /`Bp2`
M/ i?2 /`Bp2`Ƕb 2v2 Ki+?2b i?2 i`;2i KQiBQM- r?B+? +mb2b +QKTH2t 2v2 KQp2K2Mi Tii2`M Q7 bHQr2` i`;2i i`+FBM; KQiBQMb BMi2`bT2`b2/ rBi? 7bi i`;2i brBi+?BM; KQiBQMbX 6Q` M MMQii2/ pB/2Q Q7 i?2 b2[m2M+2- b22 amTTH2K2Mi`v JQpB2 k BM GTTB- _BMFFH-M/ S2FFM2M UkyRdVX
8Xj aim/v AAA, 1v2@KQp2K2Mi MHvbBb mbBM; b2;K2Mi2/ HBM@
2` `2;`2bbBQM
JQbi `2H@rQ`H/ BMi2`KBii2Mi pBbmH bKTHBM; Bb MQi BM i?2 7Q`K Q7 iQiH Q++HmbBQM b BM bim/B2b A M/ AA- #mi Bi `Bb2b 7`QK BM?2`2Mi T`QT2`iB2b Q7 ?Qr Qm` 2v2b rQ`FX
>mKMb M/ Qi?2` 7Qp2i2 MBKHb KQp2 i?2B` 2v2b- ivTB+HHv KmHiBTH2 iBK2b T2`
b2+QM/- rBi? 7bi b++/B+ KQp2K2Mib BM Q`/2` iQ #`BM; ;x2 i`;2ib iQ i?2 `2HiBp2Hv bKHH 7Qp2H ?B;?@+[mBiv `2X q?2M 2v2 }ti2b i`;2i- Bi Bb i biM/biBHH Q`
7QHHQrb KQpBM; i`;2i rBi? `2HiBp2Hv bHQr KQp2K2Mi Q7 i?2 2v2 Ub22 };m`2 RjVX h?2 i?B`/ bim/v US2FFM2M GTTB- kyRdV T`2b2Mib M2r MHvbBb K2i?Q/ 7Q`
MHvxBM; i?2b2 KQp2K2Mib 7`QK K2bm`2K2Mib Q7 i?2 2v2Ƕb TQbBiBQMX h?2 K2i?Q/
Bb 2bT2+BHHv bmBi#H2 7Q` +QKTH2t ;x2 Tii2`Mb `BbBM; BM Mim`HBbiB+ ibFb Ĝ bm+?
b rHFBM;- bTQ`ib M/ /`BpBM; p2?B+H2 Ĝ M/ `2HiBp2Hv ?B;? H2p2Hb Q7 K2bm`2K2Mi MQBb2 BM `2+Q`/BM;b rBi? mM`2bi`B+i2/ T`iB+BTMi KQp2K2MiX
h?2 Ǵ}ti2@b++/2@}ti2Ǵ 2v2@KQp2K2Mi Tii2`M T`Q/m+2b bB;MH r?2`2 #`mTi +?M;2b BM bT22/ `2 BMi2`bT2`b2/ rBi? `2HiBp2Hv +QMbiMi bT22/bX h?2 /2p2H@
QT2/ K2i?Q/ KF2b mb2 Q7 i?2 `2bmHiBM; 7Q`K Q7 i?2 2v2@KQp2K2Mi bB;MH #v b@
bmKBM; i?i bm+? bB;MHb +M #2 r2HH TT`QtBKi2/ b TB2+2rBb2@HBM2` 7mM+iBQMbX h?2 TB2+2rBb2@HBM2` TT`QtBKiBQM Bb T`Q/m+2/ mbBM; b2;K2Mi2/ `2;`2bbBQM
r?B+?-6B;m`2 R9, >Q`BxQMiH ;x2 TQbBiBQM iBK2 b2`B2b 2t+2`Ti Q7 `2+Q`/BM; r?2`2 i?2 T`@
iB+BTMi Bb ri+?BM; KQpB2 +HBT UoB;- .Q``- *Qt- kyRkV rBi? //BiBQMH bBKmHi2/
K2bm`2K2Mi MQBb2 UbiM/`/ /2pBiBQM 0.5◦VX _2/ HBM2b BM/B+i2 i?2 /2MQBbBM; `2bmHi Q7 LaG_ Ui?2 B/2MiB}2/ b2;K2MibV rBi? i?2 /27mHi T`K2i2`b M/ miQKiB+HHv@7`QK@
/i BM72``2/ MQBb2 H2p2HX
#`B2~v Tmi- i`B2b iQ Ki+? i?2 /i b r2HH b TQbbB#H2 rBi? b 72r b2;K2Mib b TQbbB#H2 Ub22 };m`2 R9 7Q` bKTH2 b2;K2MiiBQMVX >Qr2p2`- }M/BM; i?2 b2;K2M@
iiBQM i?i #2bi biBb}2b i?Bb i`/2Qz Bb MQi M 2MiB`2Hv i`BpBH ibF, i?2 bBKTH2 TT`Q+? Q7 i`vBM; 2p2`v TQbbB#H2 b2;K2MiiBQM `2[mB`2b 2pHmiBQM Q7 2n b2;K2M@
iiBQMb 7Q`nK2bm`2K2Mi bKTH2bX 6Q` 2v2@KQp2K2Mi bB;MHb i?BbnƘBb ivTB+HHv BM i?2 i?QmbM/b Q` 2p2M KBHHBQMb- 7Q` r?B+? i?2 2n b2;K2MiiBQMb Bb +H2`Hv BM72bB#H2 iQ +QKTmi2X
Gm+FBHv- B7 i?2 b2;K2Mib `2 bbmK2/ iQ #2 BM/2T2M/2Mi Q7 2+? Qi?2`- i?2 +QKTmiiBQMH #m`/2M +M #2 ;`2iHv `2/m+2/, rBi? i?2 bQ +HH2/ S`mM2/ 1t+i GBM2`@hBK2 US1GhV K2i?Q/ UEBHHB+F- 62`M?2/- 1+FH2v- kyRkV i?2 QTiBKH b2;K2MiiBQM +M #2 7QmM/ rBi? O(n) QT2`iBQMb- r?B+? KF2b Bi mb#H2 7Q` 2p2M HQM; M/ ?B;? bKTHBM; `i2 K2bm`2K2Mi b2`B2bX lM7Q`imMi2Hv i?2 bbmKTiBQM Q7 BM/2T2M/2Mi b2;K2Mib- r?B+? K2Mb /Bb+QMiBMmQmb 7mM+iBQM- /Q2b MQi }i i?2 2v2@KQp2K2Mi bB;MHb p2`v r2HH, Bi rQmH/ bbmK2 i?i i?2 2v2 +M KF2 BMbiMi DmKTb 7`QK QM2 i`;2i iQ MQi?2`X hQ F22T i?2 7pQ`#H2 +QKTmiiBQMH +QKTH2tBiv Q7 i?2 S1Gh K2i?Q/- #mi mb2 KQ`2 `2HBbiB+ bbmKTiBQMb- aim/v AAA /2p2HQTb i?2 LBp2 a2;K2Mi2/ GBM2` _2;`2bbBQM ULaG_V K2i?Q/X hQ }M/ M TT`QtBKi2Hv QTiBKH b2;K2MiiBQM- LaG_ ;`22/BHv 2M7Q`+2b +QMiBMmBiv #v bbmKBM; i?i b2;@
K2Mi Hrvb bi`ib 7`QK i?2 T`2pBQmb b2;K2MiǶb 2M/TQBMi i?i rb 2biBKi2/ QMHv mbBM; /i mT iQ M/ BM+Hm/BM; i?i b2;K2MiX
h?2 `KB}+iBQMb Q7 i?2 ;`22/v +QMiBMmBiv 2M7Q`+2K2Mi `2 MQi 7mHHv
mM/2`biQQ/-#mi BM bBKmHiBQM bim/B2b i?2 `2bmHiBM; b2;K2MiiBQM b22Kb iQ rQ`F [mBi2 r2HH i H2bi 7Q` 2v2@KQp2K2Mi bB;MHbX LaG_Ƕb ++m`+v BM `2+QMbi`m+iBM; i?2 Ǵi`m2Ǵ ;x2
6B;m`2 R8, .2MQBbBM; T2`7Q`KM+2 U?B;?2` Bb #2ii2`V Q7 LaG_ BM +QKT`BbQM iQ i?2 hQiH o`BiBQM M/ qB2M2` }Hi2`b i /Bz2`2Mi MQBb2 H2p2HbX _2bmHib `2 b?QrM 7Q` irQ ?mKM /ib2ib UEQKQ;Q`ib2p E`TQp- kyRjc oB;- .Q``- *Qt- kyRkV M/ QM2 bBKmHi2/
/ib2iX hQiH o`BiBQM M/ qB2M2` T`K2i2`b r2`2 MmK2`B+HHv QTiBKBx2/ ;BMbi 2+? `2+Q`/BM;X LaG_ rb `mM mbBM; Bib /27mHi T`K2i2`bX
/B`2+iBQMb 7`QK MQBbv K2bm`2K2Mib rb +QKT`2/ iQ irQ 2tBbiBM; }Hi2`b r?B+?
r2`2 QTiBKBx2/ mbBM; i?2 i`m2 /B`2+iBQMb- r?2`2b LaG_ rb `mM rBi? i?2 /27mHi T`K2i2`BxiBQM 7Q` 2v2@KQp2K2Mi /i U};m`2 R8VX
LaG_Ƕb T2`7Q`KM+2 +H2`Hv bm`Tbb2/ i?2 qB2M2` }Hi2` QM HH #mi i?2 p2`v HQr2bi MQBb2 H2p2HbX h?2 qB2M2`@}Hi2` Bb i?2 QTiBKH HBM2` iBK2@BMp`BMi }Hi2`
UGhAV 7Q` i?Bb #2M+?K`F- r?B+? bm;;2bib i?i BM ;2M2`H LaG_Ƕb /2MQBbBM; T2`7Q`@
KM+2 bm`Tbb2b i?i Q7 Mv GhA }Hi2`- r?B+? `2 +QKKQMHv mb2/ 7Q` ;x2 bB;MH /2MQBbBM; UJ+F- "2H7MiB- a+?r`x- kyRdVX LaG_ rb HbQ +QKT2iBiBp2 ;BMbi i?2 MQM@GhA hQiH o`BiBQM }Hi2` U_m/BM- Pb?2`- 6i2KB- RNNkVX h?2 hQiH o`BiBQM }Hi2` rb QTiBKBx2/ ;BMbi i?2 ;`QmM/ i`mi? /i M/ i?mb T`2b2Mib M TT`QtBKi2 mTT2` #QmM/ 7Q` i?2 }Hi2`Ƕb T2`7Q`KM+2X
1v2@KQp2K2Mi bB;MHb `2 mbmHHv MHvx2/ BM i2`Kb Q7 Q+mHQKQiQ` 2p2Mib bm+?
b b++/2b- }tiBQMb M/ bKQQi? Tm`bmBib UBX2X i`+FBM; Q7 KQpBM; i`;2ibV- r?B+?
`2 bbQ+Bi2/ rBi? b2;K2Mib Q7 i?2 ;x2 bB;MHX hvTB+HHv i?2 Q+mHQKQiQ` 2p2Mib
`2 2biBKi2/ #v }`bi bbB;MBM; 2+? bKTH2 BMiQ M Q+mHQKQiQ` 2p2Mi +Hbb M/
i?2M K2`;BM; /D+2Mi bKTH2b Q7 i?2 bK2 +Hbb BMiQ b2;K2MiX qBi? LaG_-i?2 b2;K2Mib `2 T`Q/m+2/ }`bi M/ LaG_-i?2M 7mHH b2;K2Mib +M #2 +HbbB}2/ BMiQ Q+m@
HQKQiQ` 2p2Mi +Hbb2bX *HbbB7vBM; b2;K2Mib BMbi2/ Q7 bKTH2b ?b i?2 #2M2}i i?i b2;K2Mib ?p2 Km+? H2bb MQBbv 2biBKi2b Q7 +HbbB}+iBQM 72im`2b HBF2 bT22/
M/ +?M;2 BM KQp2K2Mi /B`2+iBQMX hQ /2H rBi? MQBb2- bKTH2@H2p2H +HbbB}+iBQM b+?2K2b ivTB+HHv `2Hv QM bB;MH T`2}Hi2`BM; r?B+? +mb2b /BbiQ`iBQM BM i?2 +HbbB}@
+iBQM 72im`2b UJ+F 2i HX- kyRdVX
aim/v AAA T`2b2Mib HbQ M Q+mHQKQiQ` 2p2Mi +HbbB}+iBQM K2i?Q/ i?i iF2b /@
pMi;2 Q7 i?Bb #2M2}i Q7 b2;K2MiiBQM #27Q`2 +HbbB}+iBQMX h?2 T`QTQb2/ K2i?Q/
classifies NSLR produced segments using a hidden Markov model (HMM), and is named NSLR-HMM accordingly. The hidden Markov model includes statistical assumptions about how often transitions between classes, e.g. how often a saccade segment is followed by a fixation segment, and what features should be observed for which oculomotor event, e.g. what is the speed distribution of saccade seg-ments. These assumptions can be combined to compute the most likely sequence of oculomotor events for a given measurement series.
The distributions for the features used in the model, namely speed of a seg-ment and change in the moveseg-ment direction, were estimated using an expert an-notated eye-movement dataset by Andersson, Larsson, Holmqvist, Stridh, and Nyström (2016). The expert annotation includes four oculomotor event classes – saccades, fixations, smooth pursuits and post-saccadic oscillations (PSOs) – and the data-driven estimation produces a four-class NSLR-HMM classifier. The re-sulting method is the only current method to use all four classes and it compares favorably against the ten algorithms benchmarked in Andersson et al. (2016) for all classes except post-saccadic oscillations for which NSLR-HMM is the second after a method specifically designed to detect PSOs (Larsson, Nyström, & Stridh, 2013).
Despite the competitive results, there are various ways that the NSLR-HMM-method could be improved. The NSLR-HMM handles ”well-behaved” (i.e. ap-proximately Gaussian) measurement noise well, but both the segmentation and the oculomotor event classification are prone to make erroneous segmentations and classifications due to outliers, which are quite common in especially mobile eye-movement recordings, and tend to cause spurious saccade classifications (see Supplementary figures S8-S9 in Pekkanen & Lappi, 2017). Much of the outlier-problem can be remedied by simply omitting samples that the eye-tracking soft-ware labels as low quality, which most packages do. This is especially simple for NSLR-HMM as it does not assume a uniform sampling rate that is a common assumption in especially denoising algorithms. However, a more elegant and likely more efficient way would be to detect outliers during the segmented regression estimation. This was done in an earlier version of the segmentation algorithm pre-sented in Lappi, Pekkanen, and Itkonen (2013) and should be quite straightforward to incorporate in NSLR.
The assumption that the oculomotor events are conditionally independent given the model, i.e. they have the Markov-property, is a clear idealization of how oculomotor events are presented in eye-movement signals, and is made largely for mathematical simplicity and the following computational efficiency. In prac-tice this means that information from past segments, apart from their classification,
cannot be used as a classification feature. Perhaps most importantly this means that the durations of the events cannot be used very effectively as a classification feature (the Markov property implicitly assumes that the number of segments in a given event is geometrically distributed). Many oculomotor event identification methods assume for example a maximum duration for saccades or minimum du-ration for fixations (see e.g. Andersson et al., 2016) to a good effect, but such can not be used in the current NSLR-HMM formulation. This does sometimes cause somewhat obvious misclassifications (see Supplementary figures S10-S11 in Pekkanen & Lappi, 2017), which could be eliminated by using a more flexible, but computationally more complex, state structure, such as a hidden semi-Markov model (Yu, 2010).