A randomised survey of the quality of antibiotics and other essential medicines in Indonesia, with volume-adjusted estimates of the prevalence of substandard medicines ======================================================================================================================================================================= * Elizabeth Pisani * Ayu Rahmawati * Esti Mulatsari * Mawaddati Rahmi * William Nathanial * Yusi Anggriani * STARmeds Study Group ## Abstract **Background** The World Health Organization warns that substandard and falsified medicines threaten public health in low- and middle-income countries. However, medicine quality surveys are often small and unrepresentative of the market, and the true scale of the problem remains unknown. We investigated how weighting survey results by market volume altered estimates of medicine quality. **Methods** We collected 1274 samples of allopurinol, amlodipine, cefixime, amoxicillin and dexamethasone from the internet and a randomised sample of all outlet-types where medicines are sold or dispensed in seven districts across Indonesia, the world’s fourth most populous nation. We conducted compendial testing for identity, assay, dissolution and uniformity, using USP standards. Samples that failed any chemical test were considered substandard. We compared raw prevalence of substandard medicines with prevalence adjusted by the national sales volume of each brand, relative to its weight among survey samples. **Findings** The weighted prevalence of substandard products was 4.4%, 46.9% lower than the raw estimate (8.2%). More antibiotics failed testing than other medicines (weighted prevalence 8.5 vs 3.1; raw prevalence 13.6 vs 4.9, both p<0.000). We found no relationship between quality and price; branded status; public procurement status; or outlet type. **Interpretation** The weighted prevalence of substandard medicine more closely resembled the 4.0% reported by the Indonesian medicine in surveillance of 13,539 samples of a wider range of medicines. Weighting survey results by sales volume likely improves robustness of estimates of medicine quality measured in field surveys. **Funding** The UK National Institute for Health Research funded the study. **Evidence before this study** In 2017, the World Health Organization (WHO) estimated that 10.5% of medicines in all low- and middle-income countries were substandard (they did not meet the standards laid out in their market authorisation paperwork) or falsified (they deliberately misrepresented content, identity or source). More recent meta-analyses focusing on specific therapeutic groups report overall prevalences of poor quality antibiotic, antiretrovirals, cardiovascular and diabetes medicines in “low- and middle-income countries” of 17.4, 1.4, 15.4, and 10.8% respectively. The authors of these reviews all warn that these aggregate prevalences are not generalisable because of unrepresentative sampling and variations in medicines included; tests performed; reference standards and pharmacopeia used; and definitions used when translating multiple quality parameters into a single pass/fail measure. No reviewed study randomised at the level of the medicine; indeed, many deliberately aimed to collect a variety of brands. None sampled from all the outlets from which patients buy medicines. All report raw prevalence only: the number of samples considered substandard, divided by the number tested. A single, more recent study in DRC Congo, which tested 239 samples of 3 medicines sampled from wholesalers, weighted results by sales volume of each brand; this weighting reduced prevalence estimates from 27.2% to 1.3%, suggesting that sales of lower-quality brands are limited. However, the authors acknowledge that the adjusted estimate likely under-represents true prevalence because degradation in the supply chain is not fully captured in samples collected from wholesalers. **Added value of this study** This study collected medicines from a random selection of all the types of outlets where patients get medicines in seven socio-economically diverse geographic areas of Indonesia, the world’s fourth most populous country. Mystery shoppers bought five medicines from retail pharmacies, over-the-counter drug shops and the internet. We also sampled from public and private hospitals, primary health centres, doctors and nurses. We conducted full compendial testing on 1274 samples using internationally-recognised USP standards and monographs, and verified product data (batch numbers and expiry dates) with producers. We weighted our raw estimates using brand-level sales data which covered both the retail sector and private and public health facilities, adding estimates for internet sales not captured in other data sources. In this large, middle-income setting, weighting raw survey results by market size reduced the estimated prevalence of medicines classified as substandard by 47%, bringing our estimate considerably closer to that reported by the national medicine regulator, which randomised at the level of the medicine, and tested over 10 times as many samples. We also found that simple visual inspection identified a high proportion of samples confirmed as falsified; these samples were concentrated among branded products sold by unregulated internet vendors. **Implications for policy and practice** Aggregation of data on medicine quality from studies which do not consider market size, especially across countries with widely different medicine markets, is likely to provide a misleading picture of the threat posed by substandard or falsified medicines. The adjusted data in our study in a large and diverse middle-income country suggests that current global estimates likely overstate the problem, but this should be confirmed in other settings, by routinely weighting survey or post-market surveillance data by market size of different brands and products. These data can be harvested from customs and excise records, production and distribution reports, public procurement systems and other sources. Our study also highlights important differences between medicines, and points to concentration of falsified products in specific settings. A more nuanced understanding of the true distribution of substandard products and of fake medicines, achieved initially through weighted estimates, would allow regulators to target prevention and future detection efforts more effectively to reduce circulation of the products most likely to be harming patients in specific country settings. ## Introduction In 2017, the World Health Organization (WHO) estimated that 10.5% of medicines in all low- and middle-income countries were substandard (they did not meet the standards laid out in their market authorisation paperwork) or falsified (they deliberately misrepresented content, identity or source).1 If correct, this figure should be alarming to those striving to expand access to medicines for 7.3 billion citizens of these countries. The 2017 estimate was based on a review of 100 studies, mostly of malaria or TB medicines sampled from retail outlets in sub-Saharan Africa (see study archive for extracted data2). More recent reviews have found similar results, with overall aggregate prevalences of poor quality antibiotic, cardiovascular and diabetes medicines in low- and middle-income countries of 17.4, 15.4, and 10.8% respectively.3–6 All note that prevalence reported in published studies are unlikely to be generalisable because of small sample sizes, unrepresentative study designs and variations in medicines included; tests performed; and pharmacopeia and definitions used. No reviewed study considered market share of different brands when calculating results. Global health actors continue to call for survey data that would provide a more nuanced understanding of the actual prevalence and distribution of poor quality medicines circulating in specific markets.7,8 But the cost of testing medicines, combined with methodological challenges, mean truly representative sampling is rarely feasible.9,10 The STARmeds study, reported here, developed sampling and weighted estimation methods designed to increase the representativeness of medicine quality survey data in resource-limited settings. ## Methods All methods are described in greater detail in Supplementary Document 1, according to MEDQUARG guidelines.10 It provides details of secondary data sources, medicine and site selection, sample size calculation, sample frame construction, sample collection and handling, data entry and management, laboratory testing, ethics protections, and estimation methods. We summarise these briefly here. Additional data, including a MEDQUARG reporting checklist, estimation code, laboratory protocols, full sample-level data and codebook, and a free, downloadable toolkit for conducting medicine quality surveys are provided in the study archive.11 12 ### Study setting Middle-income Indonesia is home to one of the world’s largest and most generous public health insurance schemes; it also has a vibrant domestic pharmaceutical sector. National health insurance now covers 249 million people.13 Since inception of the system in 2014, national tenders pushed down the price of most medicines while increasing volumes consumed.14 Medicines are provided free to insured patients, but administrative procedures are burdensome and many patients buy medicines outside the public system. Public procurement accounted for just 14.4% of the estimated US$3.6 billion spent on medicines in Indonesia in 2022.15,16 Indonesian manufacturers warned falling prices may threaten medicine quality.17–19 Cases of falsified vaccines in 2017 and contaminated paediatric syrups in 2022 added to public mistrust of medicine quality in Indonesia.20,21 However, in an independent field survey of cardiovascular/diabetes medicines collected from regulated and unregulated outlets in one district in 2021, all 204 samples passed testing for assay (which measures the percent of labelled active ingredient identified in the sample) and dissolution (the percent dissolved within a specified timeframe).22 Indonesia’s medicine regulator (*Badan Pengawas Obat dan Makanan*, BPOM) conducts extensive proactive post-market surveillance, though only from regulated outlets. BPOM tested 10,980 medicines collected through randomised sampling in the regulated supply chain in 2021. 3.7% were substandard, meaning they failed at least one pharmacopeial test, were expired or damaged at the time of sampling, or were incorrectly registered or labelled. Another 2,559 samples were collected using risk-based sampling, focused on products of public health importance with high potential for quality defects; 5.2% failed on at least one dimension, for an overall estimate of 4.0% substandard products nationally.23,24 ### Study design STARmeds medicines were chosen based on public health importance, diversity of suppliers, risk of falsification and feasibility, in consultation with BPOM and other stakeholders. See Supplementary Document1 for details. Included products are shown in Table 1; all require prescriptions in Indonesia. View this table: [Table 1:](http://medrxiv.org/content/early/2024/06/06/2023.10.08.23296708/T1) Table 1: Products included in the STARmeds study Sampling locations were chosen purposively to reflect Indonesia’s geographic and economic diversity. We included all outlet types from which Indonesian patients commonly acquire medicines. Regulated outlets included pharmacies; public and private hospitals; primary health centres, and licensed phone apps. Unregulated outlets (technically forbidden from dispensing prescription medicines) included over-the-counter medicine shops, individual shops in bulk medicine markets; doctors; midwives and internet platforms. In each area, we listed and verified all outlets by type, selecting target outlets randomly. Table 2 shows locations, characteristics, sampling dates and type of outlet randomisation. View this table: [Table 2:](http://medrxiv.org/content/early/2024/06/06/2023.10.08.23296708/T2) Table 2: Sampling locations, characteristics and sampling dates ### Sample collection and handling Trained mystery shoppers bought medicines from selected retail outlets, using individualised sample frames which specified medicines, doses and a price point (cheaper or more expensive) but not a brand. Shoppers carried prescriptions, offering them if requested. In health facilities, study staff sampled overtly, buying a branded and an unbranded version of each study medicine if available. Each sample was stored in a separate, pre-barcoded bag. Field data (barcode, geolocation, price) were entered on smartphones, using KoboCollect software.25 Samples were delivered daily to local study hubs. They were inspected for anomalies in packaging or labelling. Data entry staff entered additional product details and took high-resolution photographs using tablets pre-loaded with the study software. Samples were stored with a temperature logger in an airconditioned room until transfer to the laboratory. ### Laboratory testing Samples were tested using United States Pharmacopeia (USP) reference standards, at PT Equilab International in Jakarta. For cefixime capsules, we followed Supplement 1 of *Farmakope Indonesia* 6th.26 All other medicines were tested according to USP 43, NF38 monographs.27 The acceptance criteria for each product are shown in Table 3. Identification, assay and uniformity of assay were tested using high performance liquid chromatography (HPLC-UV Waters, Alliance 2695 with UV Detector 2489). HPLC was also used in dissolution testing for amoxicillin tablets and dexamethasone. Dissolution of amoxicillin capsule, allopurinol, amlodipine and cefixime was analysed using Spectrophotometer-UV/VIS (Shimadzu UV-1800). No dissolution testing was performed on amoxicillin dry syrup formulation. Full laboratory protocols are in the study archive. We could not afford to test for impurities. View this table: [Table 3:](http://medrxiv.org/content/early/2024/06/06/2023.10.08.23296708/T3) Table 3: Acceptability criteria for pharmacopeial tests, USP 43 NF38 ### Product verification and falsification We provided all 79 market authorisation holders with per-sample data and high-definition photos of primary and (if available) secondary packaging, asking them to verify all sampled batch numbers, expiry dates and maximum retail prices against their production records. Samples with confirmed anomalies were considered falsified, as were any with no or incorrect active ingredient. ### Analysis and estimation We used Stata 17 and 18 for data cleaning, weighting and analysis.11,28 #### Product quality We defined a substandard sample as one that failed any pharmacopeial test, using the limits shown in Table 3. Raw prevalence was calculated by dividing the number of substandard samples by the number tested. Previously-suggested measures of “extreme deviation” do not take into account the variations in assay values permitted by most pharmacopeia.29,30 We calculated a single measure of deviation from permitted values as the maximum among the following: * percentage points by which assay exceeds or falls short of the limits of acceptability; * percentage points by which final dissolution value falls below the permitted threshold. To distinguish between marginal and more extreme deviation, we also report prevalence of samples that deviate from permitted levels by over 10%. #### Weighted estimates Outlets were chosen randomly, but mystery shoppers could not credibly request a list of individual medicines by volume for truly representative sampling. In our adjusted estimates, we thus weighted each sample in our study by the ratio of a brand, dose and formulation’s market share to its weight among our study samples, using the total market for all five medicines as the universe, as follows: ![Formula][1] For estimates of prevalence by medicine, we recalculated the relative volumes within the universe of each active ingredient. Details are provided in Supplementary Document1. #### Volume by product (active ingredient, formulation, brand and dose) To determine the volume for each product (medicine, formulation brand and dose), we used non-zero product-level sales volume data for calendar 2022 bought from pharmaceutical data aggregator IQVIA (n=467). For public procurement products (n=17), we added national public transaction volumes for calendar 2021 to IQVIA volumes, adjusting to avoid double counting in the hospital sector. We redistributed IQVIA’s unnamed “generic manufacturer” volumes across unbranded generics as described in Supplementary Document1, which also explains how we imputed volumes for 18 products collected in our survey but missing from IQVIA data. #### Price-related analyses When investigating the relationship between price and quality, we used the price paid for the specific sample, per smallest counting unit. Samples of public procurement medicines acquired at no cost were priced at the public procurement cost plus a government-permitted margin of 28% for tax and handling. ### Ethics and reporting The study design was widely discussed with BPOM and a multisectoral national working group on medicine quality. The study protocol was approved by institutional review boards at Universitas Indonesia (970/UN2.F1/ETIK/PPM.00.02/2020) and Imperial College London (21IC7265). We also obtained permissions from authorities in each sampling district. Dedicated study staff in the district research hubs provided full-time problem-solving support by phone to mystery shoppers. We immediately notified BPOM of any suspect product; we shared sample-level results for all study products once certificates of analysis were issued. ## Results We collected and tested 1,274 samples, as shown in Table 4; 82 were not of targeted doses. Four locally-made branded products were registered by multinational companies. The rest were registered by 75 different Indonesian pharmaceutical firms; medicines were made by 72 different manufacturers. We collected one originator product, 178 unique medicines sold under proprietary brand names (branded generics), and 101 unique unbranded products, differentiated by market authorisation holder. Table 4 shows the distribution of tested samples by sampling area and medicine, and the number of unique products. Table 5, meanwhile, shows the distribution of samples by location and type of outlet. Branded medicines dominated the online samples (72%); in physical outlets, 51% of samples were branded. View this table: [Table 4:](http://medrxiv.org/content/early/2024/06/06/2023.10.08.23296708/T4) Table 4: Number of samples tested by location of collection, medicines and dose, and number of unique products collected by medicine, dose and branded status View this table: [Table 5:](http://medrxiv.org/content/early/2024/06/06/2023.10.08.23296708/T5) Table 5: Distribution of samples and unique products by outlet type and location ### Raw quality estimates All samples contained the correct active ingredient. Table 6 shows further test results, and the number confirmed falsified among the 578 samples for which we received confirmation from market authorisation holders. View this table: [Table 6:](http://medrxiv.org/content/early/2024/06/06/2023.10.08.23296708/T6) Table 6: Pharmacopeial test results by medicine type Overall, 8.2% of samples failed at least one pharmacopeial test, with significant differences by medicine type. Samples expired at most recent testing date (n=47) were no more likely to fail testing than unexpired samples (8.5% vs 8.2%, p=0.95). The anti-hypertensive medicine amlodipine, the only chronic disease medicine in the study, had the lowest testing failure rate, at 1.6% (all failing in uniformity of content only). The antibiotics amoxicillin and cefixime had the highest testing failure rates (10.0% and 19.0% respectively, totalling 13.6%, compared with 4.9% for non antibiotics, p<0.000). Of 72 manufacturers, 30 (41.7%) made at least one of the substandard samples. Most of the failures were clustered close to permissible quality limits, (depicted schematically in Figure 1 as the zone between the dotted lines). Fifteen samples (1.2%) from 11 manufacturers, deviated from permissible limits on assay or dissolution by 10% or more; only 1 of these was confirmed falsified by its market authorisation holder. ![Figure 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/06/06/2023.10.08.23296708/F1.medium.gif) [Figure 1.](http://medrxiv.org/content/early/2024/06/06/2023.10.08.23296708/F1) Figure 1. For all samples failing assay or dissolution testing, maximum deviation from permitted limits, by active ingredient #### Falsified medicines No sample was deemed falsified based on test results. By December 2023, 45/79 market authorisation holders had checked sample details against manufacturing records, covering 578 samples; 21, from 12 companies, were confirmed falsified (3.6%). Seven of the falsified samples failed pharmacopeial tests (33.3%), compared with 9.4% of samples with confirmed correct production records (p <0.000). 20/21 falsified products were branded; six had fake batch numbers, while 15 had incorrect expiry dates. The majority of falsified samples (15/21) were purchased from unregulated internet sellers; only one was acquired from an outlet type permitted to sell prescription medicines. Study staff flagged 11/21 confirmed falsified products as suspicious at visual inspection. #### Source of medicines In regression analysis, after controlling for differences in medicines, the odds of testing substandard were 2.2 times higher for samples collected in rural areas, compared with those collected from physical locations in cities (p<0.001). There was no significant difference in the pharmacopeial quality of samples acquired from different types of outlets. 8.0% of online medicines failed testing, against 8.3% acquired from physical sources (p=0.83). However, among samples acquired online, those bought from individuals selling on general marketplaces or social media were more likely to fail any pharmacopeial test than those bought from licensed online vendors or verified online stores of physical pharmacies (11.3 vs 3.6%, p=0.01). Samples bought from the types of outlets included in BPOM’s routine post-market surveillance were no more likely to fail testing than excluded outlets (7.8% vs 8.9%, p=0.46). #### Branded status and price Branded and unbranded products were equally likely to fail testing (9.1% vs 7.2%, p=0.23) including after controlling for differences in medicine, district, or source. In unweighted analysis, medicines available free to patients in the public insurance system were significantly less likely to fail testing than medicines paid for out-of-pocket (3.8% vs 8.9%, p=0.031). Prices varied widely, both by brand and by outlet. In logistic regression, there was no relationship between price and pharmacopeial quality, including after controlling for differences in medicine, district, or source. The relationship between price and quality is reported in detail elsewhere.31 ### Weighted quality estimates We collected at least one sample of two thirds of the registered medicines and doses specifically targeted in our sampling frame (for example amlodipine 5mg), as well a third of products of other doses of the same formulations (e.g. amlodipine 10mg). In terms of market share, however, the samples we collected represented 97.1% of all targeted doses and 89.9% of other doses. Overall, we collected at least one sample of products representing 95.4% of the total Indonesian market for the five medicines and formulations included in STARmeds. We weighted our survey results using the ratio of a brands’ market share to its share of the total number of samples in our survey. Figure 2 plots the number of samples of each product tested (by pass/fail status) against its weight; Supplementary Table 1 lists the respective market shares and sample shares, and gives weights. ![Figure 2.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/06/06/2023.10.08.23296708/F2.medium.gif) [Figure 2.](http://medrxiv.org/content/early/2024/06/06/2023.10.08.23296708/F2) Figure 2. STARmeds testing outcomes per product, plotted against ratio of weight in market to study weight *Market share is calculated separately for the unregulated online market. Graph omits 4 samples with market share 16-76 times their share in the STARmeds sample; all passed testing Overall, 88% of the unique products in our sample (shown to the left of the dotted line in Figure 2) were over-represented, compared with their market share. This was partly because we deliberately sought unregulated online sellers who sell tiny volumes; the 14.6% of study samples from these sellers represented just 0.0022% of national market volume for study medicines. Additionally, we oversampled medicines with smaller distribution to allow for inter-medicine comparisons. For example, cefixime accounted for 15.3% of our sample, but just 3.7% of the market of our universe of 5 medicines. Failure rates were higher in products that were over-represented in our sample, compared with under-represented products (9.8% vs 3.3%, p<0.000). As Figure 3 shows, adjusted prevalence of substandard samples across products tested in the study was 46.9% lower than raw prevalence (4.4% vs 8.2%). When we weighted samples by their relative market share within the universe of a single medicine (also shown in Figure 3), the prevalence of poor quality amlodipine fell by two thirds, while amoxicillin dropped by just 13%. While in unadjusted analysis public procurement medicines appeared less likely to fail testing compared with other medicines, the relationship was reversed, and lost statistical significance, once adjusted for the high volumes of public procurement medicines. ![Figure 3.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/06/06/2023.10.08.23296708/F3.medium.gif) [Figure 3.](http://medrxiv.org/content/early/2024/06/06/2023.10.08.23296708/F3) Figure 3. Raw and adjusted prevalence for study medicines (Red value labels denote statistically significant differences in analysis of raw data; green labels note differences significant in the adjusted analysis) The weighted prevalence of medicines failing assay or dissolution by more than 10 percent of the permitted margin was 0.5%. ## Discussion We collected 1,274 samples of five medicines across four therapeutic groups from all sources from which Indonesian patients commonly acquire medicines, and tested them for identity, quantification and uniformity of active ingredient, and dissolution. The raw prevalence of substandard products was 8.2%. However, when we adjusted for market size of individual brands, the estimate fell by 46.9%, to 4.4%. This suggests that poor quality medicines do not, in general, achieve large market shares. Using the raw rather than the adjusted estimate, we risk overestimating the volume of substandard study medicines sold in a year in Indonesia by 75 million units. Our findings are consistent with the one other study we found which adjusted quality estimates for market size.32 In that study of an antimalarial, an antibiotic and an analgesic in the Democratic Republic of Congo, the raw estimate of poor quality medicines (27.2%) fell to 1.3% when adjusted for market size data provided by distributors. Since many academic studies of medicine quality included in recent metanalyses prioritise brand variety in their sampling,1,5,6,33 we believe other, unadjusted studies may similarly mis-represent (and possibly overstate) the true proportion of medicines that are of poor quality. Our adjusted estimate for five medicines was 10% higher than the 4.0% found by the Indonesian regulator BPOM in nation-wide post-market surveillance of over 10 times as many samples across all medicines in the months before the study.34 STARmeds sampling strategy and definitions of quality did not mirror BPOM’s exactly; the differences could affect the results in both directions. We included outlets not licensed to sell prescription medicines, and used mystery shoppers rather than uniformed regulators to sample in retail outlets. Both factors could *increase* our chance of collecting poor quality products, compared with BPOM’s strategy. In fact, we found no significant difference in quality between samples collected from outlet types sampled by BPOM and others. (Such similarity between regulated and informal markets was also recent reported in Nigeria.29) Other differences are more likely to *decrease* the relative chances of sampling poor quality products. We excluded sterile formulations at higher risk for production errors, and psychoactive medicines at greater risk for falsification, and did not include labelling errors in our definition of substandard products. Despite these differences in definitions and sampling, the weighted estimate in our study is very similar to that published by the national regulator for the Indonesian market as a whole, yet it was achieved with a sample that was less than one tenth of the size. A full consideration of the cost of surveillance will be published elsewhere, but we suggest that weighting raw estimates by market size is likely to be a resource-reducing option for arriving at a better understanding of the magnitude of the threat posed by substandard or falsified medicines in national markets. Sources of data on sales volumes vary by country. Data from commercial aggregators, where available, are expensive. Other potential sources include data from distributors, wholesalers, public procurers and public insurers, as well as tax and customs authorities. Even the adjusted figure implies a large number of substandard products in the Indonesian market -- close to 87 million tablets, capsules or 5ml doses across 5 medicines. However, most failed samples hovered close to Indonesia and the United States’ permitted limits. Using one of the other 60 pharmacopeia used by national regulators,35 some may have met required thresholds. Since no pharmacopeia provides an evidence base for its chosen thresholds, it is impossible to estimate the clinical impact of samples deemed to have failed by a small margin; for many patients, especially in populations with lower body weights than those in standard-setting countries, it may be limited. We do not question the need for clear and well-enforced quality standards, but believe that a consistent and clear rationale for chosen pharmacopeial thresholds would help authorities prioritise measures of greatest importance to public health. If quality is persistently low, the cumulative effect on a patient of even marginal deviations may be more severe, particularly in the case of therapy for chronic infections. Slightly lower than permitted dissolution of antibiotics may also place the product in the mutation selection window, favouring the development and spread of antibiotic-resistant bacteria.5,36–38 In our study, poor dissolution of cefixime is especially worrying, since WHO assigns it to the “Watch” category of antibiotics prone to resistance.39 Our findings were comparable with other studies in Indonesia, Cambodia, India and Togo in finding that quality did not vary by price.40–42 Researchers in Indonesia and other countries (including the United States, China and South Africa) have found that both patients and health-care providers distrust the quality of free medicines and unbranded generics; some suggest the distrust is deliberately sown to favour the financial interests of doctors, hospitals and pharmaceutical companies.43–47,20 Since the introduction of Indonesia’s national health insurance scheme, Indonesian media have also carried reports questioning the quality of its free medicines.48,49 We found no basis for selective distrust of free medicines or unbranded generics in Indonesia. We succeeded in verifying production records with 44/79 companies. Only one confirmed falsified product was found in a regulated outlet permitted to dispense prescription medicines, again mirroring a recent Nigerian study.29 Unexpectedly, the majority of confirmed falsified samples passed quality testing; most had expiry dates which did not match genuine batch numbers. This suggests criminals are extending dates on packaging of genuine products which continue to maintain serviceable quality beyond their authorised expiry dates. Simple visual inspection by researchers familiar with brand packaging identified more than half of the products that were confirmed falsified. A pilot programme to train health care workers in Indonesia and Tanzania to spot and report suspect medicines was deemed promising and could be expanded.50 However the signs would not be evident to most consumers, who do not have the same points of reference. ## Conclusion Although our study is one of the most comprehensive single-country academic surveys of medicine quality, we found that “headline” figure of 8.2% substandard medicines it yielded likely overestimated both prevalence of poor quality products and their threat to public health. When adjusted for brand volumes, just 0.5% deviated from permitted limits by more than 10%. In common with authors of recent systematic reviews, we believe the aggregation of results across dissimilar studies is misleading.3,5,6,33 We recommend that regulators and others conducting post-market surveillance or primary surveys of medicine quality consider weighting by market size to improve understanding of the volume and distribution of poor quality products, and to plan and resource effective responses. We note, however, that even estimates weighted by market volume are expensive and time-consuming to generate. Health authorities must assess the value of this investment relative to a case-finding approach with narrower focus on identifying and removing substandard and falsified products at highest risk of harming public health,51 perhaps considering market-wide surveys of prevalence every four or five years, with more targeted, risk-based surveillance in interim years. We suggest this strategy could deliver a resource-effective balance of data necessary to plan and calibrate regulator responses and minimise risk to patients of substandard and falsified medicines. ## Supporting information Supplementary Document 1 [[supplements/296708_file02.pdf]](pending:yes) Supplementary Table 1 [[supplements/296708_file03.pdf]](pending:yes) ## Data Availability Additional data are available in three locations, all within the STARmeds repository. Supplementary data for this specific paper (including the product volume data file used for adjusted estimates, the analysis code in Stata format for this paper, the supplementary methods description, supplementary figures and author contributions) are at: [https://doi.org/10.7910/DVN/QRKDWG](https://doi.org/10.7910/DVN/QRKDWG) Data and documentation related to STARmeds fieldwork more generally are in the study archive. This archive is easiest to use in Tree view. It contains the sample level data produced by the STARmeds field study, including raw laboratory data, in csv format. Also included are laboratory protocols and a more detailed description of methods. The archive can be accessed at: [https://doi.org/10.7910/DVN/RKYICP](https://doi.org/10.7910/DVN/RKYICP). Finally, we provide a free Toolkit to help researchers and regulators design and implement medicine quality field surveys using mystery shoppers. The toolkit contains downloadable and adaptable versions of data collection software, field control forms, field worker contracts and other potentially useful documentation. The Toolkit can be downloaded from: [https://doi.org/10.7910/DVN/OBIDHJ](https://doi.org/10.7910/DVN/OBIDHJ) [https://doi.org/10.7910/DVN/QRKDWG](https://doi.org/10.7910/DVN/QRKDWG) ## Conflict of interest statement Yusi Anggriani is a member of the Indonesian Ministry of Health’s advisory committee on medicine pricing, and a member of the World Health Organization Technical Advisory Group on Pricing Policies for Medicines. Elizabeth Pisani has worked as a consultant on research commissioned by the WHO Incidents and Substandard/Falsified medical products team. All other authors report no conflict of interest. ## Study Group members, and acknowledgements STARmeds was a collaboration between Universitas Pancasila, Imperial College London and Erasmus University Rotterdam. STARmeds study group members listed alphabetically by institution. Group member roles for this paper are provided in the paper-level supplementary materials. ### Universitas Pancasila Yusi Anggriani, Esti Mulatsari, William Nathanial, Yunita Nugrahani, Elizabeth Pisani, Jenny Pontoan, Ayu Rahmawati, Mawaddati Rahmi, Stanley Saputra, Hesty Utami. ### Imperial College London Adrian Gheorghe, Katharina Hauck, Sarah Njenga, Sara Valente de Almeida. Erasmus University Rotterdam ### Amalia Hasnida The team would like to thank Indonesia’s medicine regulator (Badan Pengawas Obat dan Makanan, BPOM) and the national statistics bureau (Badan Pusat Statistik, BPS) for active support in developing the methods described here. The majority of the data collectors were partners of BPS. We thank them for their hard work. We thank BPS statistical consultants Ardi Adji and Budi Santoso. We also thank members of a multisectoral working group on medicine quality estimation known as PEMO, which groups 12 Indonesian government institutions and 5 professional or industry associations, for advice provided over the course of the study, as well as the members of our Study Advisory Group (Michael Deats, Kharisma Nugroho, Yodi Mahendradhata, Raffaella Ravinetto, Selma Siahaan, Val Snewin, Virginia Wiseman and Firman Witoelar) for helpful advice. ## Funding source The study was funded by UK taxpayers through the UK Department of Health and Social Care and the National Institute for Health Research, under NIHR Global Health Policy and Systems Research Commissioned Awards ([https://www.nihr.ac.uk/](https://www.nihr.ac.uk/)), grant number NIHR131145. Thank you. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Thanks also to United States Pharmacopeia, who provided reference standards at discounted prices. ## Footnotes * ^ Membership of the STARmeds Study Group is provided in the acknowledgements * Includes updated product volume data, reducing the number of products with missing data, and increasing the accuracy of weighted estimates. Weighting methodology is revised to reflect the relative weight of a brand in our study and the wider market. * Received October 8, 2023. * Revision received June 6, 2024. * Accepted June 6, 2024. * © 2024, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/) ## References 1. 1.World Health Organization. A study on the public health and socioeconomic impact of substandard and falsified medical products [Internet]. Geneva: WHO; 2017 Nov [cited 2017 Nov 28]. Report No.: WHO/EMP/RHT/2017.02. Available from: [http://who.int/medicines/regulation/ssffc/publications/Layout-SEstudy-WEB.pdf](http://who.int/medicines/regulation/ssffc/publications/Layout-SEstudy-WEB.pdf) 2. 2.Pisani E, Rahmawati A. Replication Data for: A randomised survey of the quality of antibiotics and other essential medicines in Indonesia, with sales volume-adjusted estimates of the prevalence of substandard medicines. [Internet]. Harvard Dataverse; 2023 [cited 2023 Nov 13]. Available from: [https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/QRKDWG](https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/QRKDWG) 3. 3.Do NT, Bellingham K, Newton PN, Caillet C. The quality of medical products for cardiovascular diseases: a gap in global cardiac care. BMJ Global Health. 2021 Sep 1;6(9):e006523. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiYm1qZ2giO3M6NToicmVzaWQiO3M6MTE6IjYvOS9lMDA2NTIzIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMDYvMDYvMjAyMy4xMC4wOC4yMzI5NjcwOC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 4. 4.Ozawa S, Evans DR, Bessias S, Haynie DG, Yemeke TT, Laing SK, et al. Prevalence and estimated economic burden of substandard and falsified medicines in low- and middle-income countries: a systematic review and meta-analysis. JAMA Network Open. 2018 Aug 10;1(4):e181662. 5. 5.Zabala GA, Bellingham K, Vidhamaly V, Boupha P, Boutsamay K, Newton PN, et al. Substandard and falsified antibiotics: neglected drivers of antimicrobial resistance? BMJ Global Health. 2022 Aug 1;7(8):e008587. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiYm1qZ2giO3M6NToicmVzaWQiO3M6MTE6IjcvOC9lMDA4NTg3IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMDYvMDYvMjAyMy4xMC4wOC4yMzI5NjcwOC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 6. 6.Saraswati K, Sichanh C, Newton PN, Caillet C. Quality of medical products for diabetes management: a systematic review. BMJ global health. 2019;4(5):e001636–e001636. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiYm1qZ2giO3M6NToicmVzaWQiO3M6MTE6IjQvNS9lMDAxNjM2IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMDYvMDYvMjAyMy4xMC4wOC4yMzI5NjcwOC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 7. 7. Fleming Fund. Substandard and Falsified Medicines (WHO) [Internet]. Fleming Fund. Undated [cited 2023 Sep 14]. Available from: [https://www.flemingfund.org/grants/substandard-and-falsified-medicines-who/](https://www.flemingfund.org/grants/substandard-and-falsified-medicines-who/) 8. 8.Newton PN, Bond KC, Oxford Statement signatories. Global access to quality-assured medical products: the Oxford Statement and call to action. Lancet Glob Health. 2019;7(12):e1609–11. 9. 9.WHO Expert Committee on Specifications for Pharmaceutical Preparations. Guidelines on the Conduct of Surveys of the Quality of Medicines. WHO Technical Report Series, No. 996, 2016, Annex 7 [Internet]. World Health Organization; 2016 [cited 2017 Dec 7]. Available from: [https://digicollections.net/medicinedocs/#d/s22404en](https://digicollections.net/medicinedocs/#d/s22404en) 10. 10.Newton PN, Lee SJ, Goodman C, Fernández FM, Yeung S, Phanouvong S, et al. Guidelines for Field Surveys of the Quality of Medicines: A Proposal. PLoS Medicine. 2009;6(3):e52. 11. 11.STARmeds Study Team. STARmeds study repository: Methods, protocols tools and code [Internet]. Harvard Dataverse; 2023 [cited 2023 Aug 8]. Available from: [https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/RKYICP](https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/RKYICP) 12. 12.STARmeds Field Team. Conducting randomised field surveys of medicine quality using mystery shoppers: A practical guide and toolkit. [Internet]. Harvard Dataverse; 2023 [cited 2023 Aug 8]. Available from: [https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/OBIDHJ](https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/OBIDHJ) 13. 13.BPJS Kesehatan. Laporan pengelolaan program 2022 dan laporan keuangan 2022 (auditan) [Internet]. Jakarta, Indonesia: BPJS Kesehatan; 2023 [cited 2020 Jul 21]. Available from: [https://bpjs-kesehatan.go.id/#/informasi-publik-detail?slug=laporan-pengelolaan-program-dan-keuangan-bpjs-kesehatan-tahun-2022-auditan](https://bpjs-kesehatan.go.id/#/informasi-publik-detail?slug=laporan-pengelolaan-program-dan-keuangan-bpjs-kesehatan-tahun-2022-auditan) 14. 14.Anggriani Y, Ramadaniati HU, Sarnianto P, Pontoan J, Suryawati S. The impact of pharmaceutical policies on medicine procurement pricing in Indonesia under the implementation of Indonesia’s social health insurance system. Value Health Reg Issues. 2020;21:1–8. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.vhri.2019.05.005&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F06%2F06%2F2023.10.08.23296708.atom) 15. 15. Lembaga Kebijakan Pengadaan Barang/Jasa Pemerintah. Winners of Indonesian annual public procurement tender, per province, all medicines, 2018-2022. Provided to CEPHAS on request; 2023. 16. 16.Statista. Pharmaceuticals - Indonesia [Internet]. Statista. 2023 [cited 2023 Aug 31]. Available from: [https://www.statista.com/outlook/hmo/pharmaceuticals/indonesia](https://www.statista.com/outlook/hmo/pharmaceuticals/indonesia) 17. 17.Tirtokoesnadi. Industri farmasi perlu perhatian BPJSK, Kemenkes, BPOM, Kemenkeu untuk menghadapi saat Covid-19/virus Corona mewabah dan perkembangan industri farmasi kedepan. GP Farmasi; 2020. 18. 18.Ramadhan A. GP Farmasi usulkan alokasi pembayaran obat dalam anggaran BPJS Kesehatan. Antara News [Internet]. 2019 Jan 19 [cited 2019 Jan 21]; Available from: [https://www.antaranews.com/berita/789358/gp-farmasi-usulkan-alokasi-pembayaran-obat-dalam-anggaran-bpjs-kesehatan](https://www.antaranews.com/berita/789358/gp-farmasi-usulkan-alokasi-pembayaran-obat-dalam-anggaran-bpjs-kesehatan) 19. 19. GP Farmasi. Hutang jatuh tempo obat & alkes JKN belum dibayar mencapai Rp. 3,5 T per Juli 2018. 2018. 20. 20.Hasnida A, Kok MO, Pisani E. Challenges in maintaining medicine quality while aiming for universal health coverage: a qualitative analysis from Indonesia. BMJ Global Health. 2021 May 1;6(Suppl 3):e003663. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiYm1qZ2giO3M6NToicmVzaWQiO3M6MTc6IjYvU3VwcGxfMy9lMDAzNjYzIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMDYvMDYvMjAyMy4xMC4wOC4yMzI5NjcwOC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 21. 21.Reuters. Indonesia finds local trader forged ingredient label in probe of cough syrup deaths. Reuters [Internet]. 2023 Jan 30 [cited 2023 Feb 1]; Available from: [https://www.reuters.com/business/healthcare-pharmaceuticals/indonesia-finds-local-trader-forged-ingredient-label-probe-cough-syrup-deaths-2023-01-30/](https://www.reuters.com/business/healthcare-pharmaceuticals/indonesia-finds-local-trader-forged-ingredient-label-probe-cough-syrup-deaths-2023-01-30/) 22. 22.Dewi A, Patel A, Palagyi A, Praveen D, Ihsan BRP, Hariadini AL, et al. A study of the quality of cardiovascular and diabetes medicines in Malang District, Indonesia, using exposure-based sampling. BMJ Global Health. 2022 Nov 1;7(11):e009762. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiYm1qZ2giO3M6NToicmVzaWQiO3M6MTI6IjcvMTEvZTAwOTc2MiI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDI0LzA2LzA2LzIwMjMuMTAuMDguMjMyOTY3MDguYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 23. 23.Slamet L. Policy brief: pengawasan obat dan makanan, termasuk keamanan pangan. Kementrian PPN/Bappenas; 2019. 24. 24.Parliament of Indonesia SD. Legislator Nilai Peran BPOM Dalam Pengawasan Obat Bersifat Pasif [Internet]. 2022 [cited 2023 Sep 14]. Available from: [http://www.dpr.go.id/berita/detail/id/41503](http://www.dpr.go.id/berita/detail/id/41503) 25. 25.Kobo Inc. KoBoToolbox | Data Collection Tools for Challenging Environments [Internet]. KoBoToolbox. 2022 [cited 2022 Jun 29]. Available from: [https://www.kobotoolbox.org/](https://www.kobotoolbox.org/) 26. 26.Kementerian Kesehatan Republik Indonesia. Farmakope Indonesia Edisi VI [Internet]. 2020 [cited 2023 Aug 13]. Available from: [https://farmalkes.kemkes.go.id/2020/11/farmakope-indonesia-edisi-vi/](https://farmalkes.kemkes.go.id/2020/11/farmakope-indonesia-edisi-vi/) 27. 27.United States Pharmacopeia. United States Pharmacopeia National Formulary (43rd edition NF38). USP; 2021. 28. 28.Stata Corp. Stata. Texas, USA: Stata Corp; 2021. 29. 29.Gabel J, Lächele M, Sander K, Gnegel G, Sunny-Abarikwu N, Ohazulike RE, et al. Quality of Essential Medicines from Different Sources in Enugu and Anambra, Nigeria. The American Journal of Tropical Medicine and Hygiene [Internet]. 2024 May 14 [cited 2024 May 23];1(aop). Available from: [https://www.ajtmh.org/view/journals/tpmd/aop/article-10.4269-ajtmh.23-0837/article-10.4269-ajtmh.23-0837.xml](https://www.ajtmh.org/view/journals/tpmd/aop/article-10.4269-ajtmh.23-0837/article-10.4269-ajtmh.23-0837.xml) 30. 30.World Health Organisaton. Survey of the quality of selected antimalarial medicines circulating in six countries of sub-Saharan Africa [Internet]. Geneva: WHO; 2011 [cited 2016 Sep 4]. Available from: [http://www.who.int/medicines/publications/qamsareport/en/](http://www.who.int/medicines/publications/qamsareport/en/) 31. 31.Maria V, Tjandrawijaya WN, Rahmawati A, Anggriani Y, Sarnianto P, Pisani E. Are quality medicines affordable? Evidence from a large survey of medicine price and quality in Indonesia [Internet]. medRxiv; 2024 [cited 2024 Apr 5]. p. 2024.02.21.24303126. Available from: [https://www.medrxiv.org/content/10.1101/2024.02.21.24303126v1](https://www.medrxiv.org/content/10.1101/2024.02.21.24303126v1) 32. 32.Schiavetti B, Wynendaele E, Spiegeleer BD, Mbinze GJ, Kalenda N, Marini R, et al. The Quality of Medicines Used in Children and Supplied by Private Pharmaceutical Wholesalers in Kinshasa, Democratic Republic of Congo: A Prospective Survey. The American Journal of Tropical Medicine and Hygiene. 2018 Jan 8;98(3):894–903. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.4269/ajtmh.17-0732&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F06%2F06%2F2023.10.08.23296708.atom) 33. 33.Do NT, Boupha P, Newton PN, Caillet C. The quality of antiretroviral medicines: an uncertain problem. BMJ Glob Health. 2023 Mar;8(3):e011423. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiYm1qZ2giO3M6NToicmVzaWQiO3M6MTE6IjgvMy9lMDExNDIzIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMDYvMDYvMjAyMy4xMC4wOC4yMzI5NjcwOC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 34. 34.World Health Organization. 1 in 10 medical products in developing countries is substandard or falsified [Internet]. World Health Organization. 2017 [cited 2021 Dec 7]. Available from: [https://www.who.int/news/item/28-11-2017-1-in-10-medical-products-in-developing-countries-is-substandard-or-falsified](https://www.who.int/news/item/28-11-2017-1-in-10-medical-products-in-developing-countries-is-substandard-or-falsified) 35. 35.World Health Organizaton. Index of World Pharmacopoeias and Pharmacopoeial authorities; QAS/11.453/Rev.14 [Internet]. WHO; 2022 [cited 2022 Oct 13]. Available from: [https://www.who.int/publications/m/item/QAS-11.453-Rev.12](https://www.who.int/publications/m/item/QAS-11.453-Rev.12) 36. 36.Pisani E. Antimicrobial resistance: What does medicine quality have to do with it? [Internet]. London: Antimicrobial Review; 2015 Nov [cited 2016 Feb 11]. Available from: [http://amr-review.org/sites/default/files/ElizabethPisaniMedicinesQualitypaper.pdf](http://amr-review.org/sites/default/files/ElizabethPisaniMedicinesQualitypaper.pdf) 37. 37.Cavany S, Nanyonga S, Hauk C, Lim C, Tarning J, Sartorius B, et al. The uncertain role of substandard and falsified medicines in the emergence and spread of antimicrobial resistance. Nat Commun. 2023 Oct 3;14(1):6153. 38. 38.Murray CJ, Ikuta KS, Sharara F, Swetschinski L, Robles Aguilar G, Gray A, et al. Global burden of bacterial antimicrobial resistance in 2019: a systematic analysis. The Lancet. 2022 Feb;399(10325):629–55. 39. 39.2021 AWaRe classification [Internet]. Geneva: World Health Organization; 2021 Sep [cited 2022 Aug 8]. Report No.: WHO/HMP/HPS/EML/2021.04. Available from: [https://www.who.int/publications-detail-redirect/2021-aware-classification](https://www.who.int/publications-detail-redirect/2021-aware-classification) 40. 40.Singal GL, Nanda A, Kotwani A. A comparative evaluation of price and quality of some branded versus branded–generic medicines of the same manufacturer in India. Indian J Pharmacol. 2011 Apr;43(2):131–6. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.4103/0253-7613.77344&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21572645&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F06%2F06%2F2023.10.08.23296708.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000288608200006&link_type=ISI) 41. 41.Rahman MS, Yoshida N, Tsuboi H, Sokchamroeun U, Keila T, Sovannarith T, et al. A Cross-Sectional Investigation of the Quality of Selected Medicines for Noncommunicable Diseases in Private Community Drug Outlets in Cambodia during 2011–2013. The American Journal of Tropical Medicine and Hygiene. 2019 Nov 6;101(5):1018–26. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F06%2F06%2F2023.10.08.23296708.atom) 42. 42.Schäfermann S, Wemakor E, Hauk C, Heide L. Quality of medicines in southern Togo: investigation of antibiotics and of medicines for non-communicable diseases from pharmacies and informal vendors. PloS one. 2018;13(11):e0207911. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F06%2F06%2F2023.10.08.23296708.atom) 43. 43.Hassali MA, Wong ZY, Alrasheedy AA, Saleem F, Mohamad Yahaya AH, Aljadhey H. Perspectives of physicians practicing in low and middle income countries towards generic medicines: A narrative review. Health Policy. 2014 Sep 1;117(3):297–310. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.healthpol.2014.07.014&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25129135&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F06%2F06%2F2023.10.08.23296708.atom) 44. 44.Patel A, Gauld R, Norris P, Rades T. “This body does not want free medicines”: South African consumer perceptions of drug quality. Health Policy and Planning. 2010 Jan 1;25(1):61–9. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/heapol/czp039&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19726560&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F06%2F06%2F2023.10.08.23296708.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000273112200006&link_type=ISI) 45. 45.Riner B, Bussy A, Hélène-Pelage J, Moueza N, Lamy S, Carrère P. ‘No generics, Doctor!’ The perspective of general practitioners in two French regions. BMC Health Serv Res. 2017 Nov 9;17(1):707. 46. 46.Dunne SS, Dunne CP. What do people really think of generic medicines? A systematic review and critical appraisal of literature on stakeholder perceptions of generic drugs. BMC Medicine. 2015 Jul 29;13(1):173. 47. 47.Pisani E, Dewi A, Palagyi A, Praveen D, Ihsan BRP, Lawuningtyas Hariadini A, et al. Variation in Price of Cardiovascular and Diabetes Medicine in Indonesia, and Relationship with Quality: A Mixed Methods Study in East Java. Am J Trop Med Hyg. 2023 Jun 7;108(6):1287–99. 48. 48.Jayabuana NN. Sistem Lelang JKN Berisiko Gerus Kualitas Obat. 2017 Nov 9 [cited 2018 Aug 6]; Available from: [http://industri.bisnis.com/read/20171109/257/707814/sistem-lelang-jkn-berisiko-gerus-kualitas-obat](http://industri.bisnis.com/read/20171109/257/707814/sistem-lelang-jkn-berisiko-gerus-kualitas-obat) 49. 49.Sampurno. BPJS dan E-katalog Obat-obatan. Kompas [Internet]. 2018 Aug 7 [cited 2022 Oct 11]; Available from: [https://www.kompas.id/baca/lain-lain/2018/08/07/bpjs-dan-e-katalog-obat-obatan-2](https://www.kompas.id/baca/lain-lain/2018/08/07/bpjs-dan-e-katalog-obat-obatan-2) 50. 50.Wagnild JM, Lee D, Jayeola B, Lukito PK, Fimbo A, Hampshire K. Can a Smartphone Application Help Address Barriers to Reporting Substandard/Falsified Medical Products? A Pilot Study in Tanzania and Indonesia. Glob Health Sci Pract. 2023 Aug 3;ghsp;GHSP-D-23-00034v1. 51. 51.Pisani E, Hasnida A, Rahmi M, Kok MO, Harsono S, Anggriani Y. Substandard and Falsified Medicines: Proposed Methods for Case Finding and Sentinel Surveillance. JMIR Public Health and Surveillance. 2021 Aug 16;7(8):e29309. [1]: /embed/graphic-4.gif