Jixuan Wang 1, Jingbo Yang 1, Haochi Zhang 1, Helen Lu 1, Michael Brudno 2
1 Department of Computer Science, University of Toronto
2Centre for Computational Medicine, Hospital for Sick Children
Abstract:
Patient encounters is the primary way in which physicians collect patient information. While the majority of hospital appointments in the ambulatory setting as well as General Practitioners use EHR systems, these systems are not optimally designed for capturing clinical information during patient encounters. Previous work has shown that in the ambulatory setting physicians spent more time documenting in EHR systems than on patient interaction. To address this gap we developed PhenoPad — an AI-based clinical tool enables the digitization of highly structured patient data and facilitates patient interaction, while providing physicians enough freedom to perform their jobs efficiently.
PhenoPad is a note taking interface for free-form notes and standard phenotypic information capture. Information is captured via a variety of modalities (speech, stylus, or typing) and quickly and unobtrusively presented to a physician for validation on a mobile device (tablet). Physicians can hold it like a notebook and write on it like on paper. Since this is a more natural way to take notes compared to computers and keyboards, the communication between physicians and patients is more natural. Physicians using PhenoPad are thus able to pay more attention to patients and their concerns, and provide better quality health care.
The system consists of three components: a tablet with stylus, microphones and cameras, and a cloud-based server. Handwritten notes are transformed to text by handwriting recognition. Physicians can easily take photos using the tablet and insert them into notes seamlessly. Microphones and cameras are utilized to capture audio and video signals of patient-physician communications, which are uploaded to the cloud server. Real-time analysis including speech recognition, speaker diarization, phenotypic information identification and clinical decision support are then performed on the cloud server. Analysis results are sent back immediately for validation and decision support. All information captured is also be exported to EHR systems.