Development of a Hands-free, Voice Activated Communication Device for Doctor-Nurse-Allied Health (DNA) P5

Year Submitted: 2024
Published Date: 31 December 2024

Tags: Care Process ＆ Redesign, Access To Care, Waiting Time

About this Content

Aims

The aim of the project is to introduce a hands-free communication device activated by voice or/and gaze-based command, facilitating seamless communication among healthcare professionals. The objectives include enhancing efficiency in initiating secured voice and video calls, empowering staff to make informed and timely decisions on patient care and promoting patient-centered care by extending communication beyond hospital confines. Additionally, the project aims to streamline documentation processes through features like photo-taking and gaze-based command technology. Ultimately, the goal is to improve communication, collaboration, and decision-making in healthcare settings, thereby enhancing patient outcomes and overall care quality.

Background

The project introduces a groundbreaking hands-free communication device activated via voice commands or/and gaze-based command, heralding a new era in healthcare communication. Designed to address the challenges of the current care paradigm, it empowers staff to initiate secured voice and video calls effortlessly, fostering seamless communication within healthcare settings. Whether facilitating 1-to-1 consultations or enabling dynamic 1-to-many discussions, this innovative device represents a significant stride towards TTSH's vision of a hospital without walls. Aligned with TTSH's overarching goals, this initiative not only enhances communication efficiency but also promotes patient empowerment and caregiver engagement. By extending care beyond traditional hospital boundaries, it supports the principle of "Care is a Transition," guiding patients through their healthcare journey with continuity and collaboration. The integration of cutting-edge digital technologies not only streamlines communication processes but also enhances clinical decision-making. Real-time access to patient information and multidisciplinary consultations ensures timely interventions and optimized care delivery. By incorporating features such as photo-taking and gaze-command capabilities, the device simplifies documentation tasks, allowing healthcare professionals to focus more on patient care. Initial implementation at TTSH Ward 5C serves as a testbed for potential campus-wide deployment and expansion into other care settings, promising a transformative impact on healthcare communication and delivery.

Methods

The project endeavours to conduct a Proof-Of-Concept (POC) to evaluate the capabilities of a proposed hands-free communication device, encompassing voice activation, gaze-based command, automated contact search, hands-free voice/video calling and photo-taking functions. The POC, simulated at Moon Technologies’ environment and later in ward environments, prioritizes operational readiness and user experience with the hardware and software applications. POC's utilization of mock scenarios and absence of TTSH system interface, initial security assessments are deemed unnecessary, though ongoing collaboration with Moon Technologies on IT security persists. 20 staff members participated in the POC, the trial was conducted with voice/video call content unrecorded to safeguard privacy. The inclusion of photo-taking and speech-to-text documentation, securely stored within the software application, underscores meticulous attention to functionality, security, and healthcare compatibility, setting the stage for future implementation. The functionalities of the device are as follow: a) Voice activation and voice command b) Gaze-based command c) Automated search of contact details to call out desired recipients (by voice and gaze-based command) d) Hands-free user experience with voice and video call (by voice and gaze-based command) e) Photo-taking (by voice and gaze-based command)

Results

Separate Proof of Concepts (POCs) were conducted with two collaborators, ST Engineering and Moon Technologies, to validate the feasibility of hands-free communication devices in an inpatient ward setting. These tests aimed to improve staff communications at TTSH Ward 5C, designated as TTSH’s Smart Ward. The POCs confirmed the technical functionality and operational readiness of both devices for staff communications in an inpatient ward. User feedback was collected to determine which solution was best suited for the use cases. Twenty participants trialled the devices and provided feedback. Technical Evaluation Concept & Success Indicators: 1 to 1 voice and video call. 85% accuracy on voice and gaze command recognition 90% on Video Conferencing establishment & stability. Moon Technologies scored 91%, ST Engineering scored 82.50%. 1 to Many (at least 5) voice and video call. 85% accuracy on voice activation and gaze command recognition. 90% on Video Conferencing establishment & stability. Moon Technologies scored 96%, ST Engineering scored 82.50%. Photo-taking capability 90% in capturing of intended image. Reduction of the amount of time spent in nursing documentation by 20% for patients with deterioration by using voice-to-text documentation. Moon Technologies scored 92%, ST Engineering scored 80%. To test the users’ (caller and receiver) experience on using the devices on Comfort Level, Feasibility, Charging & Storage, Login & Logoff and Training. 80% Users (caller and receiver) express positive experience on the apps. 90% application stabilities. Moon Technologies scored 88.30%, ST Engineering scored 86.60%.

Conclusion

The conclusion of the Proof-of-Concept (POC) phase for Moon Technologies underscores their technological prowess and suitability as a partner for TTSH (Tan Tock Seng Hospital). Through rigorous testing and evaluation, Moon Technologies' hands-free communication device has demonstrated superior performance, scoring high marks in accuracy, stability, and user satisfaction compared to alternative solutions. This validation not only solidifies Moon Technologies' reputation as a leading innovator in healthcare technology but also positions them as the preferred collaborator for TTSH's future endeavors. With their modular design, willingness to accommodate change requests, and cost-effective solution, Moon Technologies emerges from the POC phase with confidence, poised to deliver impactful solutions that enhance communication and efficiency within healthcare environments.

Lessons Learnt

The Proof-of-Concept (POC) phase with Moon Technologies has provided invaluable insights into healthcare technology development, accompanied by lessons learned from encountered challenges. While the hands-free communication device showcased technical excellence, challenges such as the time-consuming eye calibration process for some users and the absence of full speech-to-text functionality during the POC stage highlighted areas for improvement. Additionally, the device's support mechanism, which holds on the side of the head, and the need for users to position objects closer to the camera lens presented usability challenges. These obstacles underscore the importance of addressing user experience considerations, refining technical functionality, and optimizing device ergonomics in future iterations. Despite these challenges, the collaboration between Moon Technologies and TTSH demonstrated resilience and adaptability, further emphasizing the value of effective communication and partnership in overcoming obstacles and driving innovation in healthcare technology solutions.

Additional Information

Following the successful Proof-of-Concept (POC) phase, Moon Technologies is now poised to transition to the Proof-of-Value (POV) stage with TTSH (Tan Tock Seng Hospital). Leveraging the technical superiority and flexibility of their hands-free communication device, Moon Technologies aims to demonstrate its potential to enhance staff productivity and improve clinical outcomes, with a specific focus on wound assessments. The POV will involve integrating the device into the workflow of Registered Nurses in Ward 5C, alongside actual patient interactions, over a six-month period. Unlike the POC, which was conducted in a controlled environment with mock scenarios, the POV will take place in real-world conditions, extensively testing the device's capabilities during morning and afternoon shifts on all General Medicine cases. With a strong alignment to TTSH's core principles and strategic thrusts, Moon Technologies' DNA Voice Activated Communications device represents a forward-thinking solution that empowers healthcare professionals and enhances communication across the continuum of care. The project's scope will encompass testing various functionalities of the device for specific use cases, including voice activation, eye gaze-based commands, automated contact search, hands-free user experience, photo-taking with digital zoom, speech-to-text technology, data storage, and patient wrist tag barcode scanning. The POV will adhere to the Innovation Trial Framework, ensuring data privacy and security while maximizing the device's potential impact on staff workflow and patient care within the inpatient ward setting.

Keywords

communication, device, healthcare, voice, video, calls

Innovators' Details

Connect with this contributor!

Lim Mei Ling - Mei_ling_lim@ttsh.com.sg

Project Attachment