Why is Human Factors important in Global Health?
Evaluating usability is essential in the development and implementation of medical devices in global health. Life-saving devices can be incredibly innovative or possess the potential to change technology landscapes. However, if users are unable to operate them as intended, they can cause more harm than good.
Medical errors that cause harm to patients are the most obvious adverse effects of difficult-to-use devices. Other impacts include resistance to using the device or the device breaking due to confusion around proper use. Unfortunately, for these reasons, many well-meaning medical device donations to low-resourced hospital settings have contributed to, what we call, a medical device equipment graveyard. This often looks like a storage room in a low-resource hospital full of unusable medical devices.
Often, the devices left in these device graveyards were not tested with users working in local settings, and confusion around the use of the devices led to the machines being unusable or unrepairable. When this occurs, the burden falls on the local hospital to dispose of the broken equipment, taking up time, valuable space, and effort.
Evaluating usability within local settings is a critical human factor undertaking in the process of designing and implementing medical devices appropriate for those settings. Usability testing should include local clinicians operating the equipment and collecting their insights and feedback.
I am passionate about ensuring medical devices are appropriately designed and tested for the settings where they are implemented. My human factors fellowship at Rice360 Institute of Global Health Technologies in partnership with NEST360 provides me an opportunity to contribute to appropriate design on a global scale through conducting usability studies and other human factors practices.
Human Factors at Rice360 Institute of Global Health Technologies
One goal of the NEST360 program is to deliver and sustain innovation that includes identifying newborn medical devices suitable for implementation within partner countries. This process involves testing devices to ensure they meet the requirements of the target settings, such as power adaptability and environmental conditions. An essential requirement is that these devices are usable in the local setting. At Rice360, in collaboration with our Houston and Malawi team members, I conduct summative usability studies to determine which devices are easy-to-use and best suited for implementation in NEST360 partner countries.
A summative usability study is meant to collect data that is typically used to make decisions about device selection or regulatory approval. We search the technology landscape to find and test commercially available devices we think have the potential for use in our target settings. Next, we conduct heuristic analysis, a human factors tool to assess potential usability errors, to filter out any device with significant usability challenges. We then take the remaining short-list of devices and design summative usability studies to test them.
Preparing for a usability study requires identifying the critical use tasks when the device is operated in a real setting. With the critical tasks identified, we design ways to simulate them at our study sites. Designing simulations is really fun! This part of the process allows me to express my creative self as it requires coming up with different ways to mimic clinical scenarios in a lab setting for various devices. Since these medical devices are not walk-up-and-use systems, we start by training users during the study, similar to what they would experience in a clinical setting.
Our study structure typically includes the following: firstly, participants are shown brief training videos to standardize the training experience across all participants; secondly, participants use the devices in a simulated setting; thirdly, they fill out surveys regarding their experience; and finally, they have the opportunity to elaborate on their experience with the device in a dedicated interview. Throughout these steps, we collect data on several metrics, categorized under three areas‑effectiveness, efficiency, and satisfaction of use. These metrics allow us to determine which devices are easy-to-use in our target settings, and we can ultimately implement them as NEST360 Qualified Technologies.
We also observe what, where, when, and how users are having trouble using medical devices, giving us detailed information on the nuances of use errors. My favorite part of my work is diving into these use nuances and working with participants to draw out user insights. It is a rewarding experience for me to understand what users want and need and to bring that invaluable feedback to other areas of our work, such as the design of NEST360 training materials for the devices we select to implement.
Conducting studies in our partner countries must be done conscientiously, considering the burden we introduce when we take time away from local clinicians and nurses. Therefore, we conduct an initial round of usability studies in Houston with proxy users to streamline our study process and further filter out devices with apparent usability setbacks. After a round of studies in Houston, we conduct studies in Malawi, which allows us to assess usability with our target users. We are also evaluating how reliable proxy users can be to supplement in-country usability studies, particularly when there might be high barriers to international studies, such as during the COVID-19 pandemic. In May 2022, I had the opportunity to travel to Malawi to evaluate a range of medical devices in partnership with our local research team. It was exciting to meet the colleagues I had only spoken to virtually via video call. It was wonderful to finally work with them in person during my two weeks in Malawi.
For areas in newborn health with a gap in the current technology landscape, Rice360 works to develop new technologies. We conduct formative usability studies with these devices during the product development process. Formative studies are meant to evaluate and improve device design. For these studies, we follow a similar format that we use for summative studies, but with a shift in focus. We determine what aspect of the design we want user feedback on and then conduct a study on that specific area. An interesting challenge in testing devices in development is designing ways to present realistic user interfaces even when that aspect of the design has not yet been created. Here, there is an added opportunity to contribute to design decisions based on user feedback. When these devices are ready for commercialization, we conduct summative usability studies again for regulatory submissions.
I’m incredibly grateful for the opportunity to do meaningful work in medical device development and implementation in sub-Saharan Africa and to work alongside expert international partners and collaborators. My work at Rice360 has empowered me to confidently carry on my research in the field of human factors and do what I love day in and day out.