Eye Tracking and Visualization

(1)

Eye Tracking and Visualization

Kuno Kurzhals

Eurographics 2015 Tutorial: Eye Tracking Visualization

(2)

 Quantitative (statistics)

 Descriptive, inferential statistics

 String editing algorithms

 Qualitative (visualizations)

2 How to evaluate large amount of eye

tracking data?

(3)

3 Classification of Eye Tracking Visualizations

Eye Tracking Visualizations

Point-based Visualizations

AOI-based Visualizations

Temporal Spatial

Spatio-Temporal

Temporal

Spatial

(4)

 Point-based Visualization Techniques

 Timeline Visualization

 Attention Map Visualization

 Scanpath Visualization

4 Classification of Eye Tracking Visualizations

Eye Tracking Visualizations

Point-based Visualizations

AOI-based Visualizations

Temporal Spatial Spatio-Temporal

Temporal Spatial

(5)

 Line chart, bar chart, scatter plot, box plot, star chart

 Represent eye tracking metrics

5 Statistical Graphics

(6)

 Time represented on one axis

 Eye tracking data represented on other axis

 Static or dynamic stimulus

 One or multiple participants

6 Timeline Visualizations

(7)

 Scanpath separated in x- and y-direction

 Time represented on other axis

 Designed for single participant

 Used with static stimuli

7 Timeline Visualizations: Example

[Goldberg and Helfman 10]

(8)

 Temporal evolution of scanpath

 Time represented on horizontal axis

 Vertical fixation position on vertical axis

 Designed for multiple participants

 Used with dynamic stimuli

8 Timeline Visualizations: Example 2

(9)

+ Reduce crossings and overlaps + Visual scanpath comparison

+ Overview of scanning tendencies

 Difficult to perceive combined behavior of both dimensions

9 Timeline Visualization

(10)

 Bee swarm

 Fixation position of multiple participants

 Marked on stimulus

 Different color for each participant

10 Attention Maps

(11)

 Attention map

 Aggregation of eye movement data

 Time and participants

 Color coding

 Green = few fixations

 Red = many fixations

11 Attention Maps

(12)

 3D stimulus

 Projection to 2D

 Object-based

 Surface-based

12 Attention Maps - 3D

(13)

13 Attention Maps - Video

Standard Motion-compensated

(14)

 Example

14 Attention Maps - Video

(15)

+ Overview of data

+ Identify regions of interest (AOIs) + Scalability

 Time aggregated data

15 Attention Maps

(16)

 Spatio-temporal visualization

 Consecutive fixations connected

 Duration coded by circle radius

 Problem: Visual clutter

 Difficult comparison of participants

16 Scanpath Visualization

(17)

 Examples

17 Scanpath Visualization

[Stellmach et al. 10]

(18)

 Extends spatial dimension by a temporal dimension

 Spatio-temporal overview

 Visualize gaze points, scanpaths, and cluster

18 Space-Time Cube

(19)

 Example

19 Space-Time Cube Example

[Kurzhals and Weiskopf 13]

(20)

+ Spatio-temporal overview

+ Static representation of animation

 3D (Occlusion, Perception)

 Visual Clutter

20 Space-Time Cube

(21)

21 Classification of Eye Tracking Visualizations

Eye Tracking Visualizations

Point-based Visualizations

AOI-based Visualizations

Temporal Spatial

Spatio-Temporal

Temporal

Spatial

(22)

 AOI-based Visualization Techniques

 Timeline AOI Visualization

 Relational AOI Visualization

22 Classification of Eye Tracking Visualizations

(23)

 Areas of Interest (AOI)

 Manual annotation

 Boundary shapes

 Fixation labeling

 Automatic annoation

 Grid

 Clustering

 Computer Vision

23 Definiton of AOIs

(24)

 Static AOIs

 Objects: trivial case

 Regions: identification required

 Dynamic AOIs

 Changing position, size, and shape

24 Static & Dynamic AOIs

(25)

 Examples

25 Annotation Examples

(26)

26 Dynamic AOIs

(27)

 Time represented as axis in a coordinate system

 Second axis

 Participants

 AOIs

27 AOI Timeline

Visualization

(28)

 Separate timeline per participant

 Attended AOIs visualized on timeline

28 Scarf Plots

1

2

(29)

 Timeline for AOIs

 Visualization of AOI visits along time

29 AOI Timelines

[Räihä et al. 05]

(30)

30 AOI Timelines - Examples

[Weibel et al. 12]

[Kurzhals et al. 14]

[Raschke et al. 12]

[Burch et al. 13]

(31)

+ Overview of temporal distribution of attention + Comparison of various participants

 Synchronization needed for comparison

 Often not in context

 Scalability

31 AOI Timelines

(32)

 Visualization of relations between AOIs

 Transition frequencies

 Transition patterns

 Analysis of strategies

 Important for user comparison

32 Relational AOI

Visualizations

(33)

 Examples

33 Relational AOI Visualizations

[Holmqvist et al. 03]

[Tory et al. 05]

[Blascheck et al. 13]

[Tsang et al. 10]

(34)

+ Synchronization of participants less important + Analysis of all stimuli possible

 Often not in context

 Scalability

34 Relational AOI Visualizations

(35)

 Visualization techniques for

 Point-based data

 AOI-based data

 Few interactive techniques

 Few spatio-temporal techniques for AOI-based data

35 Summary

(36)

 No semantic information

 Patterns over time hard to interpret

 Comparison of participants with active stimulus

content problematic

36 Limitations of Point-Based Methods

(37)

 Few in-context techniques

 Loss of mental map

 Scalability

 Representation of many AOIs

37 Limitations of AOI-Based Methods

?

(38)

 Participants can influence the stimulus

 Web pages

 Interactive tools

 How to compare participants?

38 Stimuli with Active Content

(39)

 Visualization techniques depend on the analysis task

 Visual Analytics approaches that combine

 Multiple Visualizations

 Interaction

 Statistics

39 The Best Solution

(40)

40 Future Perspectives

Future Perspectives

Multimodal Data

EEG Skin

Resistance Other Sensors

Active Content Analysis Task

Driven Visual Analysis Visual

Analytics Interaction

Spatio-Temporal Visualizations

Dynamic Stimuli

Multiple Participants 3D

Stimuli

Smooth Pursuit

(41)

 [Goldberg and Helfman 10] Goldberg, J. H. & Helfman, J. I. Visual Scanpath Representation Proceedings of the 2010 Symposium on Eye Tracking Research & Applications, 2010, 203-210

 [Stellmach et al. 10] Stellmach, S.; Nacke, L. & Dachselt, R. Advanced Gaze Visualizations for Three-dimensional Virtual Environments Proceedings of the 2010 Symposium on Eye Tracking Research & Applications, 2010, 109-112

 [Kurzhals and Weiskopf 13] Kurzhals, K. & Weiskopf, D. Space-Time visual analytics of eye-tracking data for dynamic stimuli IEEE Transactions on Visualization and Computer Graphics, 2013, 19, 2129 - 2138

 [Räihä et al. 05] Räihä, K.-J.; Aula, A.; Majaranta, Pä.; Rantala, H. & Koivunen, K. Costabile, M. F. & Paternò, F. (Eds.) Static Visualization of Temporal Eye-tracking data Human-Computer Interaction-INTERACT 2005, Springer, 2005, 3585, 946-949

 [Weibel et al. 12] Weibel, N.; Fouse, A.; Emmenegger, C.; Kimmich, S. & Hutchins, E. Let's look at the Cockpit: Exploring Mobile Eye- Tracking for Observational Research on the Flight Deck Proceedings of the 2012 Symposium on Eye Tracking Research &

Applications, 2012, 107-114

 [Kurzhals et al. 14] Kurzhals, K.; Heimerl, F. & Weiskopf, D. ISeeCube: Visual Analysis of Gaze Data for Video Proceedings of the 2014 Symposium on Eye Tracking Research & Applications, 2014, 43-50

 [Raschke et al. 12] Raschke, M.; Chen, X. & Ertl, T. Parallel scan-path visualization Proceedings of the 2012 Symposium on Eye Tracking Research & Applications, 2012, 165-168

 [Burch et al. 13] Burch, M.; Kull, A. & Weiskopf, D. AOI Rivers for Visualizing Dynamic Eye Gaze Frequencies Computer Graphics Forum, 2013, 32, 281-290

 [Holmqvist et al. 03] Holmqvist, K.; Holsanova, J.; Barthelson, M. & Lundqvist, D. Hyönä, J.; Radach, R. & Deubel, H. (Eds.) Reading or Scanning? A Study of Newspaper and Net Paper Reading The Mind's Eye, Elsevier Science BV, 2003, 657-670

 [Tory et al. 05] Tory, M.; Atkins, S.; Kirkpatrick, A.; Nicolaou, M. & Yang, G.-Z. Eyegaze Analysis of Displays with Combined 2D and 3D Views Visualization, 2005. VIS 05. IEEE, 2005, 519-526

 [Blascheck et al. 13] Blascheck, T.; Raschke, M. & Ertl, T. Circular Heat Map Transition Diagram Proceedings of the 2013 Conference on Eye Tracking South Africa, 2013, 58-61

 [Tsang et al. 10] Tsang, H. Y.; Tory, M. K. & Swindells, C. eSeeTrack - Visualizing Sequential Fixation Patterns IEEE Transactions on Visualization and Computer Graphics, 2010, 16, 953-962