Risk professionals are tasked with looking beyond the surface and finding potential pitfalls. For senior living organizations, it is essential to do the same. Blind adoption of a new product can turn what you had hoped would be a clear “win” in terms of staff workload and risk exposure into an entirely new minefield that even federal regulators are struggling to navigate.
So how can you thoughtfully vet these vendors and their products? It begins with asking the right questions. Reviewing vendor responses with your clinical risk team, including legal counsel, will instill confidence and assure all stakeholders that you have thoroughly weighed each product’s benefits and potential risks.
Data Collection & Access: What does your system actually touch?
AI-driven communication portals or apps linked to electronic medical records (EMR) can be a fantastic way to demonstrate proactive communication, customer service, and transparency to residents and their families. They can also help improve workflows in your organization, freeing up the clinical team’s time to do what they do best: provide medical and personal care to your clientele.
Many of these apps offer a range of features, including medication lists, test results, care plan updates, medical appointments, and more. Organizations who purchase these app systems can then choose which features to enable, thereby controlling how much or how little can be viewed by a particular family member or resident. Your team might say, “Good. We can control how much data is gathered by the app!” But this is not necessarily the case. The distinction your team needs to understand is the difference between what the app displays to families and what the vendor’s system collects and processes behind the scenes. These are almost always two different things.
When a family communication app integrates with an EMR, it typically establishes a broad data connection at the integration level rather than a feature-by-feature connection. EMR integration is usually configured once, granting the vendor’s system access to a defined set of data fields (medications, incidents, vitals, care notes, ADLs, etc.). By choosing to disable a feature, you may merely be disabling only the display layer, while the pipeline from the EMR to the vendor’s AI remains fully open, exposing the data to additional risk of misuse or breach.
Capture vs. Output: What does your system actually see and hear?
Many fall prevention products utilize sensors to detect movement in a resident’s living space. To promote privacy and dignity, vendors often depict individuals in that space as silhouettes or colored outlines, removing any identifying characteristics while gathering data on gait patterns, typical vs. atypical changes in plane, or attempts by residents requiring assistance to self-transfer.
However, as in the previous question, there are key differences between the output and what is initially collected by the vendor’s equipment. These fall detection “lenses” are essentially cameras or depth sensors (like infrared or LiDAR). The process works in layers:
- Capture: The sensor captures a full image or depth map of the room and the person in it, essentially the same raw input as a standard camera.
- Processing: An AI model running either on the device itself (“edge computing”) or on the vendor’s cloud servers analyzes that raw image.
- Output: The system renders a colored silhouette, skeleton outline, or blob, thereby stripping away identifying visual detail before displaying it to staff or families.
Legally, this is important for several reasons. A visual image of an identifiable resident is recognized as PHI under HIPAA. If the raw image is transmitted to the vendor’s cloud (however briefly) that transmission requires a BAA and appropriate safeguards. Many states are also moving toward requiring resident/family consent prior to deploying AI monitoring systems in private living spaces. The fact that the output is a silhouette does not eliminate the consent requirement if the image was fully captured. In the case of a vendor data breach, this could also potentially expose those raw images to bad actors. Be certain to carefully compare a vendor’s responses to data capture with HIPAA’s de-identification standards to ensure they meet the legal standard.
How the vendor’s AI works: Was it built for people like our residents?
It may surprise senior living leaders to learn that the answer to the above question is “probably not.” Most AI systems used in healthcare were trained on data from hospitals and acute care settings, with younger patients, shorter stays, higher staffing ratios, and very different movement and behavior patterns than what you see in a skilled nursing facility or memory care unit. The AI learned from a population that may look almost nothing like your residents. A resident with an atypical fall pattern (slow, sideways, against a wall) may not trigger an alert at all if the system was only trained on more dramatic, vertical falls.
Alternatively, the system may generate constant false alarms if a resident’s atypical gait looks like a fall to a system that was never trained on it. An AI that cries wolf too often is arguably more dangerous than no AI at all, because it erodes the human response the system depends on.
The Takeaway for Senior Living Leaders
AI-based communication and care enhancement products hold a great deal of promise in the senior living industry, and the three questions explored here are only a starting point. As you evaluate vendors, bring your clinical risk team and legal counsel into the conversation early and keep them there throughout the process. Asking vendors to put their answers in writing is a reasonable expectation, and a vendor who is reluctant to do so has, in a sense, already answered your question.
The regulatory landscape around AI in healthcare is moving quickly. Organizations that build thoughtful vendor vetting into their processes now will be better positioned as that landscape develops.
By asking these questions, you are making a sound investment in your organization and demonstrating the care and due diligence that your residents, your staff, and your reputation deserve. To learn how to minimize your risk around AI, contact our team today.
