Prototyping & Usability Testing

© 2008 Josh Nesbit, Courtesy of PhotoshareAt this point, you have made the critical decisions necessary for technology design. You will either adopt and adapt an open source solution or build customized software. If you are handling the technology adaptation or development in house, you have the necessary expertise on staff. Or, if you are outsourcing your technology needs, you have chosen a technology partner. Now it is time to make your technology requirements a functioning reality.

The technology world is in a constant state of innovation. For example, for a smart phone or a computer to work as well as it should, its software and applications need to be regularly updated. Similarly, the technology that supports your mHealth program will change over time.

Further, technology development is an iterative process. Many technology groups use agile software development methodologies, including adaptive planning among a cross-functional team, incremental development, and overall flexibility to design and deliver an operational technology. Rather than starting with a binder full of specifications and building the entire solution, agile development breaks the specifications into many components in order to realize the end solution. Steps in an agile development process may look like this: 

  • At first, the technology might be built as a prototype—a low-cost, simple form of the technology—and tested with a small number of users for feedback.
  • Then the technology will be built to address that feedback and meet an initial set of functional requirements, and released in a “beta” form. Beta testing with users and administrators helps confirm usability, identify bugs, and improve overall functionality. (Beta Testing is covered in greater detail in the Planning for Implementation section of the Guide.)
  • When the technology is further adapted and ready for wide-scale deployment, the software components are packaged together and released, with the understanding that updates to the system will be needed in the future. 

Gathering user insights, preferences, and feedback at each stage of solution development is critical to the future success of the technology. Start as small and cheap as possible, and then continue to test, build, and iterate.

KEY CONSIDERATIONS

  • Does the solution function as intended? What does or does not work well?
  • What do end users like and dislike about the solution? What could be improved, and how?
  • Are end users interacting with the technology as expected?
  • Do end users understand the content? Do they have preferences for when and how often to be contacted?
  • Is the technology platform easy to use for project administrators? What do they like or dislike about the administrative user interface? What improvements can be made?
  • Is data collected as intended, in the correct format?
  • What are anticipated barriers to correct use based on observation and user testimonies? (For example, if the end user shares her phone with another family member, she may not receive the messages intended for her; or if a community health worker’s phone runs out of battery while he is in the field and he cannot collect data on his phone as planned.) What can be done to overcome these barriers?

THE EXPERTS SAY...

“[What works well in programs is] design research plus ethnographic practice: [we] wrote software with people using it; [we incorporated a] real-time feedback loop, which allowed mid-course corrections). [This is called] agile software development – you write a little [code] at a time, check with users, and then you write more iteratively (see what they actually do versus what the user tells you). This is core to mHealth development.” – Merrick Schaefer, World Bank (with UNICEF at the time of the interview), Programme Mwana, Zambia

“Usability testing was incorporated at several levels. First, we did a lot of internal testing for the content and platform. Then we did beta testing with a small group representing end users. Then, after the product’s release to general public, we monitored and incorporated feedback from users. Usability testing doesn’t stop after release of version 1—the solution should be continuously monitored and improved. Otherwise people stop using it.” – James BonTempo, CCP (with Jhpiego at the time of the interview)