4 design lessons from the disastrous vaccine rollout
When Lisa deBettencourt helped her 75-year-old father register on New Hampshire’s COVID-19 vaccine website, the digital health design expert had low expectations. But even she was alarmed when, after informing the system her father was over 65 years old, it asked if he was currently pregnant. With no alternative, she carefully completed the rest of the process, or so she thought. She had no way of knowing that more frustrations were ahead.
deBettencourt’s experience attempting to register her father for the vaccine mirrors that of countless other people across the country, who have spent hours online, desperate for an appointment. But these frustrations are emblematic of a larger problem: government websites that are poorly designed and don’t actually reach the users they need to. Here are four lessons we can learn from the disastrous vaccine rollout.
Usability depends on good systems
Designing a registration website may seem simple, but like an iceberg, there’s often more beneath the surface than people realize. The Centers for Disease Control and Prevention (CDC) paid Deloitte $44 million to build the now notorious system called VAMS (Vaccine Administration Management System), to manage the entire vaccination process. The website deBettencourt used had to integrate with three other parts of the VAMS system (for healthcare providers, employers and regional governments). This project faced two major challenges. First, the American healthcare system has many parts that were built independently and aren’t standardized. It was struggling to function before the pandemic. Second, building things that are easy to use requires knowledge many organizations don’t have. In the software industry as a whole, it’s rare that user experience experts are hired on most projects, and few programmers and project managers are trained in basic methods for ensuring usable designs. (The CDC didn’t comment on the specific usability testing that was used for VAMS.)
deBettencourt faced a barrage of basic user interface mistakes like confusing instructions and hard-to-follow steps. When she thought she was finished, she had in reality only completed New Hampshire’s initial approval process. An email was sent to her father containing a code and a URL to log in to the actual VAMS system. Once there, she’d have to enter all of his information again, a step easily avoided if the two systems were compatible.
Identify who you’re designing for and target the challenging cases first
Helping parents register for the vaccine, as deBettencourt did, is common. Older generations, among the first to be approved, are the least technologically literate and often have older computers and phones. And yet many programmers only test their work with their own high-powered devices, leaving them unaware of how bad the actual experience of using their product will be. Building a tool is one thing, ensuring the right people can use it is another.
For example, the VAMS website uses tiny grey text, which even deBettencourt struggled to read. And instead of a simple list of checkboxes to pick from, some decisions required unnecessarily complex controls, difficult for old or arthritic hands, or anyone using a mobile device. Even with deBettencourt at the helm on her own computer, the process took far longer than it should have. She had to go back and forth with her father over text messages and phone calls to confirm details, increasing the chance of mistakes.
When content strategist Margot Bloomstein tried to register her parents through the New York State vaccine website, she faced other frustrations. On Twitter she shared that they live in a rural area on a slow bandwidth connection, another common scenario that wasn’t prioritized. Because of this oversight, the system crashed several times throughout the process. For anyone without good internet access in their home, or without a tech-savvy child or friend, accessing the system is much harder.
VAMS could have provided more robust web servers, a queuing system to prevent overloads or offered a low-bandwidth version of the website. At the very least, citizens should have been warned about the minimum requirements before they started.
Watch someone (try to) use an early design
Even the best designers know they won’t get everything right on their own. This means smart project teams invest in early usability testing. They watch a representative group of people use an early prototype and study the results carefully. This can be done quickly and affordably, but many projects never do it. When Hana Schank worked with the Transportation Security Administration (TSA) as director of product, she witnessed the throw-technology-at-the-problem approach common in both the government and the private sector. There’s faith that technology magically solves problems, but when technology isn’t guided by good design, it makes things worse. You can only know what a good design is by studying users, not the technology. Schank and her team introduced user research methods like usability studies to her TSA projects, saving money and time.
Many of the frustrations deBettencourt, Bloomstein and millions of others encountered were basic design mistakes that could have been easily found. This included non-standard date formats (instead of mm/dd/yyyy), confusing layouts, reliance on QR codes (which many Americans have never used), and even the rejection of correct ZIP codes that were only allowed after other fields were entered. Issues like slow response time and the system crashing are obviously design flaws too. The existence of these problems shows that studies were never done or were ignored. The CDC attributed at least one problem to user error, but a well-designed system doesn’t need to blame its users.
Make good design an organizational priority
Everyone claims good design is important, but few organizations make the needed investments to deliver on it. A project aimed at a crisis often means it’s too late to start making those investments. The U.S. government has 18F and USDS (United States Digital Service), two organizations that excel at good design, but they’re relatively small and underfunded. But perhaps the largest issue is the counterproductive rules for who is assigned projects. As Schank notes, the government’s procurement process is the root of many of its problems: Vendor choices are often limited to those who have already worked on government projects, which rewards familiarity more than competence. Companies like IBM, Oracle and Deloitte have a reputation for manipulating rules to maximize their profits on low-quality results.
VAMS has been a failure not just for citizens but for state governments too. Many have chosen to pay for alternatives, from their pandemic decimated budgets, even though VAMS was free. This means that the situation will vary widely from state to state. The hope for VAMS was to have one system that included all the necessary components for citizens, governments, and healthcare workers, so when they were updated every state would benefit, but that goal has been lost.
Some of the outrage Americans have felt using VAMS isn’t a surprise to those who have had to rely on government systems before. What’s changed is the pandemic has forced everyone, regardless of class, to engage with underfunded and under-designed systems for the first time. Let’s hope this realization underscores the importance of good design and results in significant improvements soon.
Scott Berkun is the author of How Design Makes The World, The Myths of Innovation, and other books.
(24)
