We support medical doctors
such that they can
optimally care about their patients!

Eye Tracking Based Navigation
for Proton Beam Therapy

PhD Thesis Defense
Presented by Stephan Wyder


Faculty representative: Prof. Dr. Philippe C. Cattin
Co-referee: Prof. Dr. Raphael Sznitman
External expert: Prof. Dr. med. Francis Munier

Intraocular Tumors



Left $\rightarrow$ Based on image from: National Eye Institute, National Institutes of Health, USA
Right $\rightarrow$ Based on image from: PathologyOutlines.com

Incidence


Men, 2013, Switzerland

Women, 2013, Switzerland
Based on data from: NICER (Nationales Institut für Krebsepidemiologie und -registrierung)

Proton Therapy at PSI

  • Precise radiation dose delivery ✓
  • Iterative tumor alignment with X-rays ❗
  • Manual gazing control ❗


Left $\rightarrow$ Based on image from: PathologyOutlines.com
Right $\rightarrow$ Based on image from: Paul Scherrer Institut

Research Question



Is it possible to accurately localize and align the eye in a different way?

3D Eye Tracking

Model configuration 1:
  • single camera
  • single light source

$\Rightarrow$ Enables to localize eye center $\mathbf{c}$

Stereo Eye Tracking

Model configuration 2:
  • multiple cameras
  • multiple light sources

$\Rightarrow$ Better accuracy and robustness


Eye Tracking Workflow

System Calibration

  • Data from construction plan
  • Field of view optimization
  • Eye tracker pose estimation
  • Eye parameter estimation

Eye Tracking Workflow

Image Acquisition

Eye Tracking Workflow

Feature Detection

Eye Tracking Workflow

Eye Localization

All vectors in $\mathbb{R}_3$

Input data:
  • $ \definecolor{mygreen}{RGB}{123,171,5} \color{mygreen}\text{from system calibration} $
  • $ \definecolor{myblue}{RGB}{16,127,199} \color{myblue}\text{from feature detection} $

Output data:
  • $ \definecolor{myyellow}{RGB}{254,184,84} \color{myyellow}\text{eye location} $


Further:

  • Geometrical axis
  • Visual axis / point of gaze
$$ \definecolor{myblue}{RGB}{16,127,199} \definecolor{mygreen}{RGB}{123,171,5} \definecolor{myyellow}{RGB}{254,184,84} \underbrace{({\color{mygreen}\textbf{l}_i} - {\color{mygreen}\textbf{o}_j}) \times ({\color{myblue}\textbf{u}_{ij}} - {\color{mygreen}\textbf{o}_j})}_{w_{ij}} \bullet ({\color{myyellow}\textbf{c}} - {\color{mygreen}\textbf{o}_j}) = 0\\ $$

$$ \definecolor{myblue}{RGB}{16,127,199} \definecolor{mygreen}{RGB}{123,171,5} \definecolor{myyellow}{RGB}{254,184,84} w_{ij} \bullet {\color{myyellow}\textbf{c}} - w_{ij} \bullet {\color{mygreen}\textbf{o}_j} = 0\\ w_{ij}^T \cdot {\color{myyellow}\textbf{c}} = w_{ij} \bullet {\color{mygreen}\textbf{o}_j}\\ \underbrace{ \left[ \begin{array}{c} {[({\color{mygreen}\mathbf{l}_{A1}}-{\color{mygreen}\mathbf{o}_A}) \times ({\color{myblue}\mathbf{u}_{A4}}-{\color{mygreen}\mathbf{o}_A})]}^{T}\\ {[({\color{mygreen}\mathbf{l}_{A1}}-{\color{mygreen}\mathbf{o}_B}) \times ({\color{myblue}\mathbf{u}_{B4}}-{\color{mygreen}\mathbf{o}_B})]}^{T}\\ \vdots \\ {[({\color{mygreen}\mathbf{l}_{B2}}-{\color{mygreen}\mathbf{o}_A}) \times ({\color{myblue}\mathbf{u}_{A1}}-{\color{mygreen}\mathbf{o}_A})]}^{T}\\ {[({\color{mygreen}\mathbf{l}_{B2}}-{\color{mygreen}\mathbf{o}_B}) \times ({\color{myblue}\mathbf{u}_{B2}}-{\color{mygreen}\mathbf{o}_B})]}^{T}\\ \end{array} \right] }_{\text{M}_2} \cdot \, {\color{myyellow}\textbf{c}} = \underbrace{ \left[ \begin{array}{c} ({\color{mygreen}\mathbf{l}_{A1}}-{\color{mygreen}\mathbf{o}_A}) \times ({\color{myblue}\mathbf{u}_{A4}}-{\color{mygreen}\mathbf{o}_A}) \bullet {\color{mygreen}\mathbf{o_A}}\\ ({\color{mygreen}\mathbf{l}_{A1}}-{\color{mygreen}\mathbf{o}_B}) \times ({\color{myblue}\mathbf{u}_{B4}}-{\color{mygreen}\mathbf{o}_B}) \bullet {\color{mygreen}\mathbf{o_B}}\\ \vdots \\ ({\color{mygreen}\mathbf{l}_{B2}}-{\color{mygreen}\mathbf{o}_A}) \times ({\color{myblue}\mathbf{u}_{A1}}-{\color{mygreen}\mathbf{o}_A}) \bullet {\color{mygreen}\mathbf{o_A}}\\ ({\color{mygreen}\mathbf{l}_{B2}}-{\color{mygreen}\mathbf{o}_B}) \times ({\color{myblue}\mathbf{u}_{B2}}-{\color{mygreen}\mathbf{o}_B}) \bullet {\color{mygreen}\mathbf{o_B}}\\ \end{array} \right] }_{\mathbf{h}}\\ \mathbf{M}_2 \cdot {\color{myyellow}\mathbf{c}} = \mathbf{h}\\ {\color{myyellow}\mathbf{c}} = (\mathbf{M}^\text{T}_2 \mathbf{M}_2)^{-1} \cdot \mathbf{M}^\text{T}_2 \mathbf{h} $$

Eye Tracking Workflow

Point of Gaze Accuracy

Classical evaluation with volunteers.
Point of gaze accuracy: $0.96^{\circ}$

Eye Center Accuracy

Novel evaluation with an eye phantom.
Eye center accuracy (20 pos.): $0.68$mm

Research Question



Is it possible to accurately localize and align the eye in a different way?

Yes, it's possible!

Project Status

Eye localization system:

  • ✓ Integrated eye tracking system
  • ✓ Validated system accuracy
  • ⇒ Improving feature detection
  • ⇒ Finding the optimal field of view

Improved navigation:

  • ✓ Eye localization system, UniBasel
  • ✓ Patient specific eye model, UniBern
  • ⇒ Combination of both models, PSI
  • ⇒ Clinical evaluation, PSI

Acknowledgments

Project partners:

Paul Scherrer Institute:
    $\Rightarrow$ A. Lomax, J. Hrbacek, F. Hennings
University of Bern (Ophthalmic Technology Laboratory):
    $\Rightarrow$ R. Sznitman, J. Kowal, S. de Zanet, C. Ciller, S. Apostolopoulos

Project support:

University of Basel (CIAN):
    $\Rightarrow$ P. Cattin, S. Pezold
University of Basel (BMC):
    $\Rightarrow$ B. Müller, G. Schulz
University Hospital Basel (Radiological Physics Group):
    $\Rightarrow$ O. Bieri, F. Santini
AOT AG:
    $\Rightarrow$ W. Deibel, A. Schneider

Project funding:

Swiss National Science Foundation


We support medical doctors
such that they can
optimally care about their patients!