LG Telehealth
Project developed at: LG electronics ( in Santa Clara, California)
This project focused on developing a remote healthcare solutions through cutting-edge technology and user-centered design. Leveraging LG’s expertise in connected devices and interfaces, the initiative aimed to create a seamless telemedicine platform for both patients and healthcare providers in collaboration with Amwell.
A strong emphasis was placed on user-centered design, prioritizing ease of use and accessibility to bridge the gap between patients and providers in a digital healthcare setting. The team conducted extensive user and market research, including user interviews and visits to hospitals and e-Sitter centers, to gather insights that informed the platform’s features and design.
Key areas of innovation included intuitive interfaces for remote consultations, advanced patient monitoring systems enhanced by AI, and secure, reliable connectivity using WebRTC communication. The platform incorporated LG’s custom hardware and a cloud-based call management system (CMS) and device management system (DMS), offering scalable solutions tailored to real-world healthcare needs.
As Project Leader, I guided the platform from a proof-of-concept to LG’s first market-ready Telehealth product. Working with two cross-functional teams with a total of 18 members spread across 4 states and 2 countries, I led the development of a full-featured system with over 40 features, designed robust APIs for scalability, and contributed directly to the codebase in React and TypeScript. I also managed successful collaborations between our team and our go-to-market partner company Amwell, as well as LG CTO division and LG CSO division in HQ in Korea, and the LG NOVA Digital Health team in Santa Clara, CA, ensuring alignment across all stakeholders and contributing to the project’s success.
This work resulted in a commercialized product, two patent applications for innovative aspects of the system and lots of media attention - e.g. Hit Consultant, Fierce Healthcare
This image shows the Virtual Rounding call experience from the patient side. Patients have minimal controls, simplifying their interaction with the system. They can accept or reject incoming provider calls using voice commands, eliminating the need for specialized remotes or additional hardware that might be lost or damaged. For patients unable to speak, the system can be configured to automatically connect calls, ensuring accessibility and ease of use in various healthcare scenarios.
This image shows the Virtual Rounding call experience from the provider side. Virtual Rounding call system is meant for a provider to connect with a patient but additional participants can be added to the call - this includes other providers, family members or translators.
Providers have comprehensive controls during a Virtual Rounding call - they can manage participants, switch between available cameras, adjust camera angles using PTZ or ePTZ controls, and perform other advanced actions to enhance the call experience. In this example, the provider is sharing X-ray images with the patient and all other participants on the call, demonstrating the system's ability to facilitate clear and collaborative communication in a healthcare setting.
This image demonstrates how the eSitter platform enables PSM staff to place a specific patient in “focus mode,” enlarging their video feed to occupy a larger portion of the screen. This feature allows PSM professionals to closely monitor a patient’s activities, making it particularly useful in situations such as “pulling”, where a patient tries to pull on their IV lines or oxygen mask. In this image, the patient in focus mode is also in “interaction mode,” indicating that the PSM professional is actively communicating with them. This interaction is likely to involve providing directions, such as instructing the patient to not pull on the lines or return to bed, or offering remote assistance with other needs.
This image showcases one of the latest versions of the eSitter platform, highlighting a scenario where the PSM specialist is multitasking. In this instance, the PSM is in “interaction mode” with one patient, marked by the green boundaries around their video feed, while simultaneously triggering an alarm in another patient’s room. The alarm is used to signal that the second patient is engaging in a dangerous activity, requiring the immediate intervention from the on-floor staff.
The image also highlights several newly introduced features (circa end of 2022) such as:
Sound Monitoring Bar: A visual representation of audio activity in the room, aiding the PSM in identifying unusual sounds or noise levels.
Pre-recorded Messages: Enables the PSM to play predetermined messages in a patient’s room via TV or external speakers, available in five supported languages.
Blur Mode: Provides privacy to patients by obscuring their video feed during sensitive in-room procedures.
“Listen In” Feature: Allows the PSM to discreetly monitor audio from a room without initiating two-way communication with the patient.
These enhancements demonstrate the platform's adaptability and focus on both patient safety and user efficiency.
This image highlights the AI Scribe feature in action, designed to assist providers by transcribing and summarizing telehealth calls. When enabled, the system records and transcribes the audio of the call in real-time. At the end of the call, an LLM generates a concise summary, including key discussion points and action items, providing the provider with an easy-to-review summary.
In the top two panels, the provider can be seen dictating additional notes after the call, such as post-call action items, which are seamlessly added to the summary. The bottom panel displays the transcription and the summary output. This image, taken during development, reflects an early version of the feature with limited text formatting and a preliminary UI design.
This image shows the eSitter platform, a cutting-edge telehealth solution enabling Patient Safety Monitoring (PSM) professionals to remotely observe and manage multiple patients within a hospital setting. The interface features a dynamic grid capable of displaying the live video feeds of up to 16 patients - with 12 patients currently shown in this image. This robust system was developed with user-centered design principles and informed by extensive research, including interviews with dozens of PSM professionals, to ensure it addresses real-world needs and workflows.
On the left side, the interface shows detailed information about a selected patient. This includes both editable fields (e.g., specific notes or settings) and non-editable data for critical patient details. The eSitter interface also allows a PSM to interact with a suite of powerful tools: start “interactive mode” which activates a two-way audio channel to communicate directly with a patient, remotely select a patient-side camera if multiple are available, adjust the camera angles for better viewing of the patient, triggering alarms to alert on-site staff in emergency situations, and other advanced intervention capabilities.
The eSitter platform exemplifies innovation in patient safety and monitoring, seamlessly integrating functionality and usability to enhance healthcare outcomes.
This image highlights how the system supports multiple cameras connected to the patient-side hardware device. In this example, two cameras are available: a built-in camera with 4x zoom and an AVer Camera with 30x zoom. The AVer Camera, specifically designed for healthcare settings, offers remarkable capabilities, such as zooming in on a patient’s pupil to observe pupil movements and remotely assess potential head traumas. The image also shows part of the camera PTZ (pan-tilt-zoom) controls visible behind the camera selection popup. These controls enable the remote sitter to pan, tilt, and zoom the camera as needed, though they are automatically disabled if the selected camera does not support these functions.
This image on the top highlights the setup of the Smart Rails feature by a PSM professional. As the platform’s first AI-powered functionality, Smart Rails enabled eSitters to monitor more patients effectively. The feature allows PSMs to define boundaries, represented by a six-point polygon with adjustable vertices, around a patient’s bed or within specific areas of the room. The system tracks the patient’s skeletal structure and its movements, and generates alerts if the patient crosses the polygon’s boundaries.
The Smart Rails system supports two types of triggers:
Inside-Out Trigger: Alerts the PSM if the patient exits a designated safe area, such as their bed.
Outside-In Trigger: Generates an alert if the patient enters a restricted area, which is especially useful for monitoring patients at “flight risk”
The image also showcases the Alert Sensitivity Settings, which determine the system’s responsiveness:
High Sensitivity: Alerts triggered by any part of the patient’s body crossing the boundaries.
Medium Sensitivity: Excludes lower arms and legs from triggering alerts.
Low Sensitivity: Alerts only if the torso or head crosses the boundaries.
The images on the bottom show a later enhancement (introduced in early 2023) which added more AI capabilities. With this improvement, the system is capable to automatically recognize the bed or the patient’s location in the room, enabling the system to place boundaries automatically. This improvement significantly streamlined the setup process and enhanced usability for PSM professionals. In the bottom right image it’s possible to see how the system recognizes the bed and automatically places boundaries around it even when the camera placement is suboptimal and only part of the bed is visible. In the same image it’s also possible to see the night vision mode capabilities of the system in action and how the system works with the IR camera as well as the regular RGB sensor
Every aspect of the Smart Rails system is covered under an extensive patent application, protecting its innovative design, functionality and implementation.
Company: LG Electronics
Team: Emerging Technologies Department
Location: Santa Clara, CA
Year: 2021-2023
Demo of the Virtual Rounding call system in an interview with Atul Singh, General Manager, Digital Health at LG NOVA.
Full article here: https://www.healthcareittoday.com/2023/02/16/demo-of-the-lg-tv-integration-with-amwell/
This image features the LG SmartCam Pro, the patient-side hardware for the telehealth platform. It includes dual cameras (RGB and IR), essential ports for network and TV connectivity, onboard computational power, and an NPU for real-time AI applications on the edge.
More info in this LG SmartCam Pro brochure