How to improve EHR usability through testing

EHRs (Electronic Health Records) are intended to improve the quality, safety, and efficiency of the healthcare system. However, issues with their usability have posed harm to patients’ health. As noted, these challenges most often relate to the design and use of the tool.

Usability challenges frustrate physicians because they make simple tasks take more time and lead to workarounds. That’s why poor EHR usability is a key issue that requires urgent intervention.

In this article, healthcare software developers at Belitsoft address weaknesses of current testing practices and describe the ways to enhance EHR usability through testing.

Inadequate requirements of current standards

Many EHR vendors operate in limited test environments that are different from the work conditions used by clinicians. Developers mainly test their systems before they are tailored and used in specific healthcare facilities.

Pew’s (a non-profit organization) Project Director for Health Information Technology Ben Moscovitch identifies two key challenges. First, federal testing criteria are focused on vendors’ development and design. They do not address circumstances in which an EHR is customized as part of the implementation process or after the system goes live.

And secondly, Moscovitch sees the absence of requirements and guidance on how to test clinician interaction with the EHR for safety issues.

According to an August 2018 study from Pew, factors like variations in EHRs design, customization, and use may “lead to inefficiencies or workflow challenges and can fail to prevent – or even contribute to – patient harm.”

Consider the case of a 16-year-old Pablo Garcia who suffered a near-fatal seizure after receiving a 38,5-fold overdose of a common antibiotic via the hospital’s EHR system.

The system functioned as supposed and submitted an accurate prescription based on the input data. However, medical staff hadn’t realized that the EHR settings multiplied the requested dose by the patient’s weight.

The September research from Pew Charitable Trusts, MedStar Health’s National Center for Human Factors and American Medical Association highlights wide variations in how EHRs are put to work and assumes that providers could succeed if implementing performance standards to optimize usability.

Researchers used EHRs from two vendors (Epic and Cerner) across four clinic systems. They tracked how emergency physicians at each location hold six specific scenarios, collecting keystroke, mouse click, and video data.

“There was wide variability in task completion time, clicks and error rates,” described in the report. “For certain tasks, there was an average of a nine-fold difference in time and the eight-fold difference in clicks.”

Summing up, EHR optimization, in addition to vendor design and development, may have a positive effect on usability and patient safety. The challenge is that proper assessment and testing of how the systems are actually used in the wild can be hard to do.

Types of EHR usability testing to perform

Throughout the custom healthcare development, even the slightest changes can give rise to the insecure harmful environment. However, ONC requires only summative testing, which is limited to evaluation of EHR functionality.

Thus, for example, EHR vendors simply analyze whether a clinician can write prescriptions, rather than determine how likely it is for them to order a wrong medicine or dosage. As a result, the system fulfills ONC’s testing requirements but still has poor functionality.

To identify design problems, providers should conduct the following three types of usability testing with potential users:

• Formative testing

As a new EHR is being developed, vendors observe how medical staff use it and receive feedback on the system’s UI and UX. This stats is captured in the layout and future updates.

Practicing formative testing throughout the design process guarantees that usability is a central element of the project development. Plus, issues are detected early, when they can be fixed with little effort.

• Summative testing

Vendors can assess EHR usability by measuring how simply and successfully clinicians perform various tasks like prescribing. Besides, they can obtain formal users’ feedback once an EHR has gone through the design process.

A final test of a system’s usability may ensure that any problems are detected and fixed. The testing mainly includes off the shelf EHR functionality.

• Post-implementation testing

Medical staff often requests substantial modifications to the EHR before its installation and usage. Checking the system after it is implemented helps to find and fix problems that might arise when customizing.

EHR testing scenarios: best practices

In general, federal regulations do not oblige EHR developers to conduct robust testing to detect usability flaws. Absent tighter coordination, Pew, MedStar, and AMA offer model test cases to help vendors detect potentially dangerous usability-related risks throughout the product’s life cycle.

“The sample scenarios in the report provide thorough tests covering seven types of usability issues that clinicians may face, including unclear settings, data entry obstacles, and confusing system alerts,” said Moscovitch. “Each scenario was designed so that both health IT vendors and purchasers of EHR products can conduct realistic, safety-focused usability tests. The scenarios can be used directly by vendors or providers and can help build additional tests.”

According to the expert opinion, successful scenarios should:

• Be designed for intended participants (e.g. nurses, physicians, insurers) with different levels of computer expertise. If the audience is not defined then test cases may be used with the wrong focus group, which could invalidate the results.
• Represent a realistic clinical care process, address key socio-technical factors, such as how different members of the care team may interact with an EHR. The method helps to identify usability challenges that may occur when treating patients.
• Be shaped around a clinically oriented goal and contain clear, quantitative measures of success and failure. This method simplifies the ability to assess the use of EHRs in concrete ways, although goals and measures may vary from implementation to implementation.
• Include known risk areas, tasks that contribute to inefficiency, and challenging processes, e.g. drug dose tapering. Testing known risk areas helps to identify and resolve these challenges. Focusing on areas that could produce important tasks allows for implementing corrections that can address clinician concerns.

Moscovith hopes that more and more EHR vendors will adopt detailed test cases and use them not just in designing and development, but also in subsequent iterations as the technology matures.

Conclusion

As EHRs have become more widely adopted, vendors and healthcare providers have learned more about the challenges of using these systems. Thorough and ongoing testing of EHR systems is a way to guarantee their safety and effectiveness. The best practices and test case scenarios offer immediate steps that hospitals and healthcare software vendors can take.

About The Autohor: Dmitry Garbar

Dmitry Garbar is a Partner/Department Head at Belitsoft. He has been a part of the company since 2007, specializing in Healthcare applications.

(131)