Data Collection, Monitoring & Evaluation
As with all health interventions, mHealth implementers need to document the planning and implementation process, including key decisions that have been made. They also must demonstrate positive, measurable outcomes, including impact on health outcomes and health systems. Designing and incorporating a strong monitoring and evaluation (M&E) strategy for a project from the beginning, in conjunction with partners, will not only help ensure long-term support from stakeholders and funders, but will also help strengthen the evidence base for the field of mHealth.
To do this, those planning the M&E strategy need access to current information on the state of mHealth evidence, including where the information gaps lie and what will indicate success. This can be done via the mHealth Evidence Database, which houses the world’s peer-reviewed and grey literature from high-, middle-, and low-resource settings on mHealth effectiveness, cost-effectiveness and program efficiency.
The following pages of the Guide will:
- Examine unique challenges of monitoring and evaluating mHealth,
- Provide tips for designing M&E into your mHealth program, and
- Present an example of a high-quality mHealth research and evaluation strategy.
In addition, the mHealth Planning Tools offered in this Guide, which include a Planning Canvas, a Logic Model Template, and a Technology Decisions Worksheet, can help you track key decisions, planned activities, and anticipated outcomes of your mHealth solution and keep a record of the essential steps in your planning and implementation process.
- How will the mHealth planning and implementation process, including key decisions and the rationale behind them, be documented?
- How often will process documentation be analyzed in order to evaluate program implementation efforts?
- What indicators will be used to measure health outcomes? What indicators will be used to measure program success in other ways, such as cost savings?
- Do the project’s M&E indicators meet the evidence and reporting requirements of stakeholders and funders?
- What, if any, standardized mHealth indicators have been incorporated into the M&E plan?
- Can data generated by the technology platform be used for evaluation and reporting?
- Is the proposed evaluation design feasible and appropriate given the resources available?
- Has a system been designed to measure the financial cost of implementation and to analyze the cost-benefit ratio?
- How will the information and feedback generated by M&E be incorporated into program design and implementation on an ongoing basis?
- Do you have appropriate staff on the team to implement the M&E plan?
THE EXPERTS SAY...
“I [as the project manager] enrolled in the system. I knew that messages come in at about 2pm every day. One weekend, I noticed that I had not received a message by 8pm. I called my staff and our technology partner. They found that there was a group of 35 women who did not get the messages that day. So our technology partner fixed the problem by resending the messages. We realized that there has to be close monitoring involved to take care of such problems, and as a result, we added a data feature to generate a status report.” Priya Jha, Georgetown University's Institute for Reproductive Health, CycleTel, India
Sarriot E, Ricca J, Ryan L, Basnet J, Arscott-Mills S. Measuring sustainability as a programming tool for health sector investments: report from a pilot sustainability assessment in five Nepalese health districts. Int J Health Plann Mgmt. 2009;24:326-350.
Whittaker R, Merry S, Dorey E, Maddison R. A development and evaluation process for mHealth interventions: Examples from New Zealand. Journal of Health Communication: International Perspectives. 2012; 17 Suppl 1:11-21.