Head-mounted eye tracker communication device

Year Submitted: 2025
Published Date: 30 April 2025

Tags: Technology, Product Development, Care Process ＆ Redesign

About this Content

Aims

The aim of the project is to develop a new head mounted eye gaze device (proof of
concept) that:
1) enable patients to communicate via eye gaze that can be head mounted for those who have poor neck control or unable to sit upright
2) enable patients who have involuntary head movements to continue using eye gaze
3) enable patients who require frequent repositioning/oral suctioning to use eye gaze with minimal interruptions.
4) allow patients to use eye gaze devices outdoors, and with potential to control their wheelchairs with it
This will hopefully increase independence for this group of patients to communicate their needs and also improve quality of life.

Background

Internationally, 2 out of 100,000 people suffer from Motor Neuron Disease. It is a degenerative disease with average expected lifespan of 3 years, less than 20% will survive 5 years; 10% will survive 10 years and less than 5% will survive beyond 20 years. 20% of these patients will experience significant speech difficulties and 80% of these patients will lose their respiratory abilities and limb movements within 3 years of disease onset. These patients often require high nursing care needs however have intact cognition. Similarly, patients who suffer from significant brain-stem stroke with loss of speech yet have intact cognition or are “locked-in” have no other means of communication beyond their eye movements. Patients with cervical spine injury are also at high risk of being bedbound and lose their independence in access to computers and their environment. Speech Therapists are involved in assessing these patients' ability to communicate and provide alternative means of communication. The Occupational Therapists are also involved in advising on mounting systems or environment set up. Prosthetists and Orthotists then provide additional neck or truncal braces to support neck control or sitting posture of these patients. Traditionally, patients are offered alternative and augmentative communication methods such as picture/word charts, or eye gaze charts/devices depending on their dexterity. Limitations of current eye gaze devices includes devices are required to be positioned at eye level using stands/table, which may get in the way of nursing needs like oral suctioning or frequent repositioning, require additional stands which are bulky, risk of dropping the device on patients as it is large and will often be positioned over patients, unable to be used with patients with involuntary head movements or poor neck
control to hold their head up to look at a screen and unsuitable for outdoors usage given interference with the infrared transmitter used.

Methods

The study team and collaborator (NTU Professor) developed a software together with hardware equipment- Vivepro and Sony to come up with two head mounted eye gaze device prototypes. The hardware and software has its own features. For hardware, Virtual reality headset as head mount, single camera attached to the headset, optical smart glasses (Windows), regular desktop display (projection of what patient is viewing/doing with the smart glasses). For software, calibration, controlling the cursor via 4 directions – left, right, up and down, mouse click, zoom function, quick links to browser, exit, help, on screen keyboard. The study recruited 20 healthy healthcare professional volunteers who were aged 21 and above, able to read basic English and able to use basic computer function. Volunteers with visual impairment - unable to see text on computer despite use of spectacles and pregnant women was excluded from the study. Each volunteer was taught on how to use the software to control the movement of the mouse cursor and time took for individual to learn was recorded. Tasks performed were aim test, word test and typing a sentence in email - "Hello, how are you?". We will be looking into three measurements: 1) Text entry rate- words per minute 2) Accuracy and 3) Time taken. Based on literature, a starting speed with a dwell time of 900 milliseconds was in the order of 7 words per minute in the controlled experiments reported by Majaranta, MacKenzie, Aula, and Räihä (2006), which is in line with previous results, e.g., those by Frey et al. (1996). Error rates were also reported to be <1% of total characters entered. As such, we would expect that users will achieve similar to 7 words per minute and eventual error rates to be <1% of total characters to be a feasible device. There is also a subject-reported questionnaire to provide additional input regarding the comfort and ease of use of the prototypes.

Results

Median (Interquartile range) was used to summarise the continuous variables due to the small sample size (n=20). Tests of association via Wilcoxon sign rank test in comparing continuous outcomes between prototypes A and B, and via McNemar test in comparing categorical outcomes between prototypes A and B due to paired nature of the data. In terms of accuracy test, the data was categorised into 2 groups based on a cut-off of 90% accuracy. Result shows that there are higher percentage of accuracy for all 3 tests. Both aim test and typing of a sentence in Email has the same average of 82.5% between prototype A and B on accuracy of target selection. Word letter test has an average percent of 62.5% for ≥90% accuracy between both prototype A and B. The time taken to learn the devices was an average of 22.5 and 23.5 minutes for prototype A and B respectively. From Table 2, it also shows that the errors from task have more than 1% of total character. In the subject- reported questionnaire, where scale ranges from 1 (strongly disagree) to 7 (strongly agree), two main themes were identified, satisfactory and productivity. Overall, the mean rating for both themes were 3 and 4 which skewed towards strongly disagree. Therefore, demonstrating that the device is not yet feasible for actual patient users.

Conclusion

In conclusion, the study has proved that the concept of using a head mounted eye gaze device to perform basic computer functions for neurology patients to communication is possible. As our result shows high percentage of accuracy for basic computer functionalities (aim test and word test). However, further improvement and modifications on critical features such as ease of learning, ease of use and comfortability are required before moving on to the feasibility phase for actual clinical use.

Lessons Learnt

Lessons learned from our experience in clinical software design underscore the importance of effective communication with external collaborators to ensure the intuitive design of systems for clinical use. Despite our efforts to articulate the challenges specific to the patient population, our collaborator, who tested prototypes internally and with students, perceived them as user-friendly. However, practical trials with typical participants highlighted significant usability difficulties, particularly in executing basic mouse cursor movements. Our experience highlighted the significance of prioritizing hardware stability in the design process. We became aware that our collaborator, primarily specialized in software engineering, encountered difficulties in adapting the hardware to our needs. In retrospect, we understood the importance of including engineers with expertise in the technical intricacies of hardware design. We sought the assistance of an external 3D printing company to contribute their knowledge in 3D prototype design. Difficulty in securing a committed engineer for the project, given the short employment duration of six months, led us to rely on ad hoc employment of students by the collaborator, resulting in significant delays. COVID-19 restrictions further complicated hiring a part-time research assistant, prompting us to seek support from the CRIO team. COVID-19 restrictions constrained participant recruitment to TTSH staff, limiting inter-institutional interactions. While this provided valuable clinical insights into the challenges healthcare professionals may face when using the prototypes with patients, it also risked missing crucial feedback from potential engineers necessary for future prototype development. Challenges related to licensing and patenting underscores the importance of navigating legal and contractual agreements effectively and maintaining contingency plans for unexpected developments.

Keywords

Head-mounted eye tracking device, augmentative and alternative communication, speech therapy, eye gaze

Innovators' Details

Connect with this contributor!

Tan Xuet Ying - Xuet_ying_tan@ttsh.com.sg ↗ (opens in new tab)

Project Attachment