A randomised survey of the quality of antibiotics and other essential medicines in Indonesia, with volume-adjusted estimates of the prevalence of substandard medicines

Evidence before this study In 2017, the World Health Organization (WHO) estimated that 10.5% of medicines in all low- and middle-income countries were substandard (they did not meet the standards laid out in their market authorisation paperwork) or falsified (they deliberately misrepresented content, identity or source). More recent meta-analyses focusing on specific therapeutic groups report overall prevalences of poor quality antibiotic, antiretrovirals, cardiovascular and diabetes medicines in “low- and middle-income countries” of 17.4, 1.4, 15.4, and 10.8% respectively. The authors of these reviews all warn that these aggregate prevalences are not generalisable because of unrepresentative sampling and variations in medicines included; tests performed; reference standards and pharmacopeia used; and definitions used when translating multiple quality parameters into a single pass/fail measure.

No reviewed study randomised at the level of the medicine; indeed, many deliberately aimed to collect a variety of brands. None sampled from all the outlets from which patients buy medicines. All report raw prevalence only: the number of samples considered substandard, divided by the number tested. A single, more recent study in DRC Congo, which tested 239 samples of 3 medicines sampled from wholesalers, weighted results by sales volume of each brand; this weighting reduced prevalence estimates from 27.2% to 1.3%, suggesting that sales of lower-quality brands are limited. However, the authors acknowledge that the adjusted estimate likely under-represents true prevalence because degradation in the supply chain is not fully captured in samples collected from wholesalers.

Added value of this study This study collected medicines from a random selection of all the types of outlets where patients get medicines in seven socio-economically diverse geographic areas of Indonesia, the world’s fourth most populous country. Mystery shoppers bought five medicines from retail pharmacies, over-the-counter drug shops and the internet. We also sampled from public and private hospitals, primary health centres, doctors and nurses. We conducted full compendial testing on 1274 samples using internationally-recognised USP standards and monographs, and verified product data (batch numbers and expiry dates) with producers.

We weighted our raw estimates using brand-level sales data which covered both the retail sector and private and public health facilities, adding estimates for internet sales not captured in other data sources.

In this large, middle-income setting, weighting raw survey results by market size reduced the estimated prevalence of medicines classified as substandard by 47%, bringing our estimate considerably closer to that reported by the national medicine regulator, which randomised at the level of the medicine, and tested over 10 times as many samples. We also found that simple visual inspection identified a high proportion of samples confirmed as falsified; these samples were concentrated among branded products sold by unregulated internet vendors.

Implications for policy and practice Aggregation of data on medicine quality from studies which do not consider market size, especially across countries with widely different medicine markets, is likely to provide a misleading picture of the threat posed by substandard or falsified medicines. The adjusted data in our study in a large and diverse middle-income country suggests that current global estimates likely overstate the problem, but this should be confirmed in other settings, by routinely weighting survey or post-market surveillance data by market size of different brands and products. These data can be harvested from customs and excise records, production and distribution reports, public procurement systems and other sources.

Our study also highlights important differences between medicines, and points to concentration of falsified products in specific settings. A more nuanced understanding of the true distribution of substandard products and of fake medicines, achieved initially through weighted estimates, would allow regulators to target prevention and future detection efforts more effectively to reduce circulation of the products most likely to be harming patients in specific country settings.