In recent years, online discussion forums have gained importance in sharing public health information among policy makers and practitioners in both developed and developing countries. Since 2005, the World Health Organization Department of Reproductive Health and Research (WHO/RHR) and the Knowledge for Health (K4Health) Project have supported more than 50 virtual discussion forums on the Implementing Best Practices Knowledge Gateway (IBP KG). 

K4Health has collected survey data on 26 forums hosted by WHO/RHR and JHU∙CCP. However, until recently, we had not systematically analyzed the impact of these forums on knowledge exchange and adaptation. “Six Years of Lessons Learned in Monitoring and Evaluating Online Discussion Forums”, published in the Knowledge Management & E-Learning: An International Journal in December 2011, offers one of the first looks at empirical data from virtual discussion forums in public health.

I am a co-author of the paper, along with Megan Avila, Catherine Richey and Lisa Mwaikambo. Our work was guided by the “Conceptual Framework for Monitoring and Evaluating Health Information Products,” published as part of the Guide to Monitoring and Evaluating Health Information Products and Services. Relatively few other monitoring and evaluation guidelines exist for virtual discussions in general. In public health specifically, little research has been conducted to determine links between online discussion and health outcomes.

On IBP KG, virtual discussion forums are moderated, time-bound discussions, which typically last from 1-6 weeks. Participants can choose to participate either online or by email. All contributions are collected; expert commentary is added, and the resulting digest is emailed to all registrants.

 We embarked on a study to determine value of how the exchange of evidence-based information and practices from the forums has informed the policy environment, enhanced programs and practices, and facilitated collaboration and capacity building. To do so, we evaluated quantitative data from 26 forums as well as synthesized qualitative results from in-depth interviews from two forums, using the parameters of “reach,” “usefulness,” and “use”

Evaluation findings include:

• On average, discussions had 109 participant postings. More than half (61%) of the postings came from forum participants working in less developed countries. The discussion with the most contributions was the two-week Patient Safety Virtual Global Discussion Forum (2010) with 437 postings.
• Most respondents (34%) worked at local and international non-governmental organizations/private volunteer organizations, 24% at academic/research institutions, and 19% worked at medical/health organizations. Other groups included government ministry (8%), reproductive health/family planning service provider (7%), private sector (5%), USAID (3%), United Nations agency (3%), and faith-based organizations (1%).
• The forums reached an average of 532 registrants per forum that joined from an average of 61 countries. 
• Just over half (51%) of respondents had forwarded forum postings to other people and had already downloaded resources shared during the discussion forum.  
• Over half (64%) of respondents were satisfied or very satisfied with the discussion content.  Most (54%) felt that the forums definitely met their specific goals, while 38% felt that they somewhat met their goals.

We wanted to look beyond the numbers, so in our next phase of evaluation we conducted 17 in-depth interviews from two forums. Participants commented on the usefulness of learning about experiences in other countries. One respondent from the 2009 “Access to Reproductive Health Essential Medicines and Contraceptives Forum: Why is it so Difficult to Achieve?” forum said, “As we continue to support the MOH [Ministry of Health] and highlight these issues both at local and national level, I now have other successful models which we could try to use tailoring it to the needs of this country.” However, respondents also reminded us that there are still barriers to access, including poor Internet connectivity as indicated by requests for paper and/or CD-ROM resources from past discussions. 

Online discussion forums contribute to global learning and collaboration among international public health professionals by addressing the “knowledge to practice gap.” The forums also provide a low-cost opportunity for geographically dispersed public health professionals to share knowledge that in turn, can inform program design and management, ultimately improving health outcomes.  

In conducting this evaluation, we learned that we had more success in measuring reach and usefulness, and have more work to do in capturing use. We hope that this study will spur additional research on evaluation of online discussion forums. The complete paper can be accessed as part of the December 2011 special issue on “Creating, Supporting, Sustaining and Evaluating Virtual Learning Communities” in the Knowledge Management & E-Learning: An International Journal (KM&EL).