Measurement methods

Many methods can be used to measure quality of care. Table 1 illustrates common methods of assessing quality, including advantages and disadvantages for specific methods. Individual methods have unique strengths and weaknesses, depending on the purpose and context of the measurement exercise. It is important to tailor the method to the specific need, including the specific quality dimension being measured. For example, a client interview may be the best method to assess client-centeredness of care, but it may not be a reliable method to measure adherence to treatment standards because the client may not be aware of what treatment should have been provided. Often a combination of methods can yield a fuller picture of quality than any one single method. Similarly, measurement methods feasible for use as part of a one-time assessment of quality of care (e.g., to evaluate a program intervention) may be impractical for use for routine measurement of care quality.

For facility-level improvement work, standardized individual patient records that capture patient-specific and clinical care data serve two important functions: (1) they support real-time clinical decision-making at the point of care; and (2) they permit data extraction for calculation, aggregation, and analysis of quality measures across different units of the system (e.g., provider-specific, facility-specific, district, national). Many health systems in low-resource settings do not have individual medical records, particularly at lower level facilities, and instead use registers to track patient-specific information. Such registers are often no more than columns drawn into a notebook and may contain varying amounts of patient-specific clinical data, depending on the register. Nevertheless, such registers can be manually adapted to capture simple routine best practices (e.g., addition of column to note counseling) while stronger patient records and more robust information systems are being developed.

Although measurement methods such as observation and client and provider questionnaires may be useful for one-off periodic assessments, such methods typically are not sustainable for routine measurement of quality in low-resource settings. Routine measurement of care quality may require a combination of measurement approaches, including adaptation of local records and registers, periodic client and provider interviews, and periodic observation of care. Even when primary data are available in local records, data are often inconsistent and of poor quality.

The challenges of documenting and tracking quality of care indicators as part of community QI efforts is particularly challenging. Basic information systems (registers, indicators, ongoing measurement processes) are often rudimentary to non-existent at the community level. Building community- and facility-based staff capacity to document, capture, and extract data to calculate quality measures is central to building capacity for continuous improvement at both community and facility levels in low-resource settings.

Table 1: Common Methods of Measuring Quality of Health Care

Measurement Method

Advantages

Disadvantages

Observation

  • Considered gold standard
  • Only method that measures performance of health service (as opposed to provider knowledge and competence, which may not correlate with provider performance)
  • May be best method for assessing the quality of complex tasks
  • Observer effect
  • Resource-intensive
  • Difficult to sustain in

    routine practice

Client Interview

(e.g., exit interview; household interview)

  • Client-centeredness of care
  • May be reliable for simple measures of whether a particular service was provided
  • Recall problems
  • Unequal knowledge between health workers and clients
  • Client reluctance to give honest feedback for fear of negative consequences

Death and Near-miss Audit

  • Targets adverse outcomes
  • May identify common quality deficits
  • Accountability
  • Retrospective (after the fact)
  • Limited evidence for association between routine audit and improved outcomes

Simulation

  • Next best method after observation for complex procedural tasks
  • Resource intensive
  • Unclear relationship between simulated competence and actual performance

Health Worker Questionnaire

  • Assesses health worker knowledge, self-reported practice, and attitudes
  • Does not assess provider competence or performance

Facility and Patient Records

Individual patient record

Registers

Other facility documents

  • Relatively sustainable and low-cost
  • May encourage better documentation and point-of-care use of data for decision-making
  • Records are often inadequate or absent altogether (e.g., no standardized individual patient record)
  • Providers and supervisors may falsely document data (intentionally or unintentionally)

Routine Information System

  • Efficient extraction of data
  • Most HMIS track few quality of care measures
  • HMIS may be rudimentary or non-existent at community level

Measuring quality of care is difficult in any setting; however, routine measurement of quality is especially challenging in low-resource settings, such as these factors:

  • Relative absence of quality of care (content) measures in many routine HMIS in low-resource settings
  • Absence of standardized individual patient records or registers in many facilities and community efforts
  • Lack of primary data to permit calculation of quality indicators (e.g., registers and individual records lack essential data; records may not be standardized; if standardized, records may not include essential information)
  • Multiple competing vertical registers often containing duplicative data (e.g., pregnancy register, iCCM)
  • Few routine indicators of performance of essential system functions (e.g., percentage of CHWs with stock of oral rehydration salts for community-based management of diarrheal illness

Inadequate data management skills among providers and managers; lack of literacy among community workers and clients