The Jury is Still in: Determining the Impact of Digital Health Solutions

Nicole Ippoliti

FHI 360 | Technical Officer, Research Utilization

Over the past decade, the digital health field has sought to enhance opportunities to improve the delivery of, and access to, health services and information. At the third annual Global Digital Health Forum in Washington, D.C., programmers, researchers, tech providers, and investors convened for two days to share the latest innovations that are igniting the digital health arena in low- to middle-income countries. Presenters expanded on technologies ranging from a mobile app that offers reproductive health information to female refugees to highly sophisticated data dashboards that provide real-time feedback to community health workers. The vastness and endless potential of the field was an underpinning theme that excited and motivated participants and offered a glimpse into future horizons.

“Potential.” Let’s unpack this word, and its relation to the digital health field, for a moment. As we know, the field has evolved and grown for over a decade—we’ve seen a rapid explosion and diffusion of digital health solutions. To reflect on this, the conference session “Contributing to the Digital Health Evidence Base” discussed the impact of digital innovations on health outcomes and sought to answer the question, “How far have we really come?”

We know that digital health technologies are feasible and acceptable to users (whether clients or service providers), and that some digital health interventions have achieved knowledge, attitude, and behavior changes among target beneficiaries. Yet, throughout the conference, participants called for data to demonstrate the impact of digital technologies on health system processes and related patient outcomes. The kind of research strategies needed, however, was up for debate. The room was divided between those who called for rigorous evaluations (like randomized control trials), and those who advocated embracing less rigid and time-consuming evaluation strategies and standards of evidence (such as market research).

While evaluation strategies may still be up for debate, the following evidence gaps need to be addressed before we can attribute strong credibility to digital solutions as a high-impact practice:

  • Are digital approaches more efficient and cost-effective than non-digital applications?
  • What role should a digital health component play in the larger health ecosystem?
  • In an era of increasing resource limitations, what proportion of funding should be allocated to a digital component in combination with other programming elements?
  • Can a digital tech program achieve success as a standalone intervention?
  • What is the return on investment with respect to knowledge, behavior, and social norm changes?

As we work to improve the quality of evidence in digital health, there are a few resources available to help aid the process. The mHealth Evaluation, Reporting and Assessment (mERA) checklist, developed by the WHO mHealth Technical Evidence Review Group, is designed to standardize the reporting of mobile health interventions and outcomes. Similarly, the Principles for Digital Development offer guidance on identifying data sources and incorporating data into project design and decision-making.

As evidenced by the wide range of projects presented at the Global Digital Health Forum, we can see the ways in which the landscape of digital health innovation is rapidly growing. As we continue to innovate, let’s also be mindful of the critical work that is still needed in order to legitimize the digital health field.