The special case of NUMs and demand forecasting

Demand forecasting is particularly challenging for NUMs given the lack of historical data.  And, even if historical data exists, it may not be predictive of future demand. Thus, current forecasting tools that depend on historical inputs to forecast demand (see Section 7: Resources) may not be readily applicable for forecasting for NUMs.

Why is it important to address this challenge? Data from forecasts received by the USAID | DELIVER PROJECT show that NUMs have higher forecast error rates than other methods (male condoms, oral pills, and injectables). Forecast accuracy, or error, is defined as the absolute percentage difference between projected and actual quantities of a contraceptive distributed in a specific year for a client or program. Over-forecasting can be determined by subtracting the quantities forecasted over a specific time period from the quantities actually used during the same time period. Determination of under-forecasting is less precise, but it can be identified as an issue if all ordered stock has been distributed and demand for the product exceeds supply. Forecast errors for NUMs were more than 53 percentage points higher than other methods, as measured by the MAPE seen in the table below.[1]

Table 1. Median Absolute Percent Error in Forecasts by Method Type[2]





 

2008

2009

2010

MAPE

New and underused methods (NUMs)

50%

145%

 36%

77%

Other methods (male condoms, oral pills, injectables)

22%

17%

 32%

23.7%

Difference between NUMs and other methods

53.5%

Source: Data from 2006-2010 the USAID | DELIVER PROJECT Procurement Planning and Monitoring Report (PPMR) and/or PipeLine from 17 countries.  

The implication of higher forecast error rates is that NUMs have a greater likelihood of stock imbalances (stockouts, understocks, and overstocks). The data show that NUMs—and female condoms especially—have a higher incidence of over-forecasting. Note that stock imbalances are not solely correlated with forecast error; they could also reflect other supply chain-related issues, including financing, distribution, and reporting. 

Further, through key informant interviews, we found that most countries do not approach forecasting for NUMs differently than for other methods, which probably further exacerbates the forecast error rate, because programs are not necessarily trying to compensate for the absence of historical data when they forecast for NUMs. Without historical data, forecasters rely more on demographic data for assumption building, which often leads to over-estimations.

Refer to Appendix 2 for a discussion on the MAPE variability in Table 1. Refer to Appendix 3 for a compilation of data from the USAID | DELIVER PROJECT's PPMR and/or PipeLine from 17 countries, 2006-2010. 




[1] Note the USAID | DELIVER PROJECT uses a benchmark of 25% forecast error or less for contraceptives.

[2] A note about variability: Based on the data provided, we see a lot of variability in the overall median error rate for NUMs for the three years of data we have.  By contrast, non-NUMs show significantly less variability in forecast error. The quantities of NUMs evaluated were much smaller than for other methods, which may have led to higher error rates and fluctuations from one year to the next. Also, the values for NUMs represent absolute derived numbers rather than the median across all countries and products. For discussion about other possible causes of the variability, refer to Appendix 2.