This blog is the second half of a two-part series about Human Factors Testing and Medical devices. We’ll briefly recap before jumping in, but if you have not read the first entry, catch up on it here.
In the previous blog (Part 1), we discussed the difference between the two types of human factors testing for medical devices. One type focuses on how a user can inadvertently cause harm during a procedure, while the other is concerned with a user’s perception of the device. The FDA requires the first type of testing for premarket submissions and 510(k) premarket notifications but not the other. The blog also established why the second type, even if it’s “invisible” to the FDA and takes extra resources, is worthwhile to do because it helps predict user adoption.
In this blog, you will learn about the “when,” “what,” and “how” of Human Factors Testing, including specifics about test methods, timing, data analysis, and more.
Senior Industrial Designer
The first step toward evaluating your device is blocking off time to do it. But there are many timing details to consider.
To start with, at what point in the development process is a device ready to test? The short answer is that you can perform Human Factors Testing at all stages of development, even when all you have is an idea. Yes, Human Factors Testing can begin right after brainstorming, assuming you’ve captured your idea with a rough sketch or written description.
If that seems early, think about your own experiences coming up with ideas. You have probably verbally described a novel medical device to your peers or doodled one on the proverbial cocktail napkin. These device portrayals are rudimentary but work well enough to convey the concept to others. That doesn’t mean a tipsy conversation at an industry happy hour counts as preliminary evaluation. You need intention and data capture to gather insights that help the development process.
The FDA even encourages a prompt start to testing, recommending that preliminary evaluations happen “early in the design process.” These preliminary evaluations are what the FDA calls Human Factors Testing, which takes place before device verification and validation. Remember, these evaluations are risk-focused, but there is no reason you cannot test a device’s desirability at the same time you are testing its safety.
As for the frequency of testing, plan to test as often as you change the device design significantly. That way, the history of your device will be captured at every critical step. Referring back to this test data can be highly beneficial because it’s easy to forget why certain design decisions were made. When the design is complete, your test reports will tell a story of your device’s evolution.
Just as the FDA recommends certain timing for tests, it also addresses their format. Its guidance lists and categorizes about a dozen preliminary-stage Human Factors test methods.
The FDA does not expect you to use every test for every device, but we recommend the five baseline evaluations below because they almost always help. Your experience and the type of device can call for variations. Also, more methods exist than listed here or in the guidance, but they are too numerous to explain in this blog. Whichever techniques you choose, remember to use them for both kinds of Human Factors data: safety/efficacy and desirability.
FDA terminology is borrowed from “Applying Human Factors and Usability Engineering to Medical Devices” as much as possible in the list below.
This category includes what are technically “evaluations” instead of “tests” because they only involve research, not interacting with participants. They still require you to think from the perspective of users or analyze users’ experiences, though, which is why the FDA considers them forms of Human Factors Testing.
Task Analysis: This document lists every step a person performs to use your device. While it is often used for identifying high-risk actions, it also excels at revealing opportunities to make your user happier. These often relate to repetitive or unnecessary tasks that can be eliminated to save time and effort.
Researching Known Use-Related Problems: When an existing medical device in your space has problems, consider it an insight into how yours can be better. The FDA’s Manufacturer and User Facility Device Experience database (MAUDE) is one of the best places for free public information about adverse medical device events. Interviewing experts about the products they use can also reveal pain points.
While the above evaluations are analytical in nature because they consist of organizing and interpreting existing data, those below are empirical because they involve the generation of new data during experiments with participants.
Interviews: Interviews in this context focus not on your device idea but on your users’ needs. They are invaluable for understanding current procedure practices and identifying opportunities to improve them.
Concept/Idea Reviews: In these meetings, you show an expert, ideally a potential user, one or more concepts/ideas. The concepts can be sketches, written descriptions, or simple physical models.
Simulated Use/Usability Testing: When you have prototypes, you can give them to experts and ask them to use them in a realistic context. It works for partially- or fully-functioning prototypes, and the degree of realism of the setting can vary from an animal study to a simulated surgical suite in an office environment. It’s one of the only methods that the FDA recommends for Human Factors Validation Testing because of its efficacy in revealing use errors that can lead to hazardous situations.
Of all the advice in this blog, tailoring your investigations to device desirability is the most important if you want insights into user adoption.
The following suggestions will help you gather the most valuable information:
Ask About Looks: You may love the swooping lines and curves you designed into your prototype, but your users think they’re ridiculous. Polling them will ensure aesthetics are an asset, rather than a hindrance, to your device.
Align With Your Brand: This is much more than aesthetics, even though it is related to the above. If your device is part of a product line or you want it to be, are you able to repeat certain elements like knobs, buttons, etc. across your catalog? It can standardize how users interact with your products and make them more willing to adopt new additions by reducing fear of change.
Consider Home Health Users: If your device stays with the patient for more than a day and involves daily use, make sure you test with patients and their caretakers, whether professional or familial. These home health users often value aesthetics and features different from those of the hospital staff.
Consult a Medical Industrial Designer: If you’re stumped, Biotex’s design team can analyze your device and help construct your tests. Our designers specialize in achieving product desirability while also maintaining safety goals.
Get input from outside: Physicians and experts who have been integral to the invention of a device, either as inventors or long-term supporters, are in danger of wearing rose-colored glasses. Seek feedback from first-time users as often as possible.
Statistical Significance is Helpful: One user’s feedback is good, but ten users offer solid confidence in a design choice. Sample sizes for preliminary evaluations usually fall somewhere in the middle. Include as many users as possible.
Let Your Users Lead: This relates to both the questions you craft and your demeanor at demonstrations. Avoid leading by keeping neutral language, and instead of showing users exactly how to manipulate your prototype, ask them to try it themselves before you explain how.
Record, Record, Record: Even if you’re conducting an impromptu 20-minute concept review, capture feedback in a notebook, along with the user, date, and version of your device under discussion. Verbatim quotes are often best because they are less biased than your summaries. Better yet, ask to use your phone for an audio or video recording.
Once you have data, organize it before drawing conclusions. One great way to organize is to break your device down into features, such as “handle,” “plunger action,” “button 1,” etc., and use them as category headings. Then, move all feedback about one feature to one place. This way, you can see at a glance what multiple users think about each part of your device.
It can take as much time as setting up the evaluation in the first place, but the outcome is an easy-to-read map of where your device’s strong and weak points are. The weak points can be changed and re-tested iteratively until the majority of feedback is positive.
You should now be able to start planning when, what, and how to test your medical device’s usability. Remember to test early and often, to consider a wide range of methods before writing a test plan, and to use best practices on the big day.
Most companies can conduct at least some Human Factors testing on their own. However, there is no substitute for experience, so if you want a head start or run into a problem along the way, the human factors specialists at Biotex are always ready to help.
At Biotex, we’ve learned what makes a successful product through years of experience working with multiple clients and being part of FDA-cleared commercial releases. We're also looking for new and impactful technology. Guess what? Your medical device could be the next successful product we’d love to design.
Contact our team, and let’s collaborate!
Kim Najjar is a Senior Industrial Designer at Biotex, Inc., where she helps clients develop new medical devices by visualizing, prototyping, and testing their ideas. During device development, Kim consistently advocates for users' needs, whether they’re physicians, technicians, or patients. She is proud to be a part of Houston's thriving medical device innovation industry for over seven years.
