The Clinimetric Profile of 11 Generic Quality Indicators for Appropriate Antibiotic Use in Hospitalized Patients in Indonesia ============================================================================================================================= * Rahajeng N. Tunjungputri * Khie Chen Lie * Adityo Susilo * Freddy C. Nainggolan * Chyntia O.M. Jasirwan * Rudy Hidayat * Dyah Purnamasari * Robert Sinto * Erni J. Nelwan ## ABSTRACT **Background** One of the efforts in antibiotic stewardship for limiting antibiotic resistance is the assessment of the quality of antibiotic use. To date, the quality and appropriateness of antibiotic use in Indonesia, a low-middle-income non-European country, have not yet been assessed using proven quality indicators (QIs). One recommended tool to measure appropriate empiric antibiotic use in hospitalized patients is the generic QIs reported by van den Bosch *et. al*., which demonstrated good validity and generalizability. **Objective** The aim of this study is to assess the clinimetric profile of 11 generic QIs based on van den Bosch *et. al*. in admitted adult patients receiving empiric antibiotics in Indonesian hospitals. **Methods** This is an observational study with a cross-sectional design conducted at two government referral hospitals in Jakarta, Indonesia, from August 1, 2022, to February 2, 2023. Adult inpatients who received empiric antibiotics due to suspected infection for ≥24 hours were included in the study. **Results** In 500 inpatients from 2 hospitals, all QIs demonstrated good measurability with less than 10% of missing data. Ten QIs showed good applicability of >10%. Four QIs show good performance, while six QIs have significant potential for improvement (two blood cultures are obtained before empiric antibiotic treatment; culture sample is obtained from location suspected of infection; antibiotic is switched from intravenous to oral when clinically possible; antibiotics are stopped when there is no evidence of infection; a guideline is available and renewed every three years; and a guideline is adjusted to the local resistance pattern) making them priority targets for interventions to enhance the quality of antibiotic use in Indonesia. All QIs display good inter-observer reliability, and no moderate or strong correlations are found between QIs. **Conclusion** The clinimetric assessment of QIs is imperative before their implementation in any setting different from their country of origin. Of the 11 generic QIs, 10 demonstrated good reliability and applicability at two hospitals in Indonesia. Future intervention studies can utilize these generic QIs to measure improvement in the appropriateness of antibiotic use. Keywords * Quality indicator * antibiotic * applicability * clinimetric profile * antibiotic stewardship ## Introduction The increasing antimicrobial resistance (AMR), accompanied by a decline in the discovery of new antibiotics, has become a global crisis.1 The emergence of multidrug-resistant organisms, including including extended spectrum beta-lactamases (ESBL) producing Enterobacterales, extensively drug-resistant (XDR) carbapenemase-producing *Klebsiella pneumoniae* and other Enterobacterales, as well as *Pseudomonas aeruginosa* and pan-drug-resistant (PDR) *Acinetobacter baumannii* has major impact on the mortality and morbidity of infections.2 Antibiotic stewardship is defined as coordinated interventions designed to improve and measure the appropriate use of antibiotics by promoting the selection of the optimal drug regimen including dosing, duration of therapy, and route of administration.3,4 Robust antibiotic stewardship has several benefits for clinicians, such as reducing antibiotic prescriptions, lowering the cost of managing infection cases, and reducing the number of infections caused by multidrug-resistant organisms (MDRO).3,5,6 One aspect of antibiotic stewardship is improving the quality of antibiotic use. A method to systematically assess the quality of antibiotics use is by using Quality Indicators (QIs) developed from various international guidelines and literature. QIs are often the result of a rigorous, multidisciplinary process.7–9 However, they cannot automatically be implemented in daily practice for all settings, such as in low- or middle-income countries. The validation of QIs in everyday clinical practice is crucial before using them to measure the quality and appropriateness of antibiotic use.10 Indonesia is a low-middle-income country in Southeast Asia with a population of 275 million and substantial inequalities across the country.11 In 2019, AMR accounts for the fourth largest cause of deaths just under cardiovascular diseases, malignancy and diabetes.12 The National Action Plan for AMR exists, with antimicrobial stewardship programs in place and national health scheme aiming to achieve universal health care.13,14 However, challenges remain with limited data on AMR epidemiology and inappropriate antibiotic use. Appropriate prescribing was estimated at 33.5% using the Gyssens method.14,15 The quality of antibiotic prescribing practices in the country are not yet assessed with the implementation of QIs.15 With many hospitals lacking the infrastructure to adequately measure process, outcome and structural indicators, such as insufficient training and infectious diseases and microbiology expertise, there is a need for QIs that are easily applicable and can soundly measure the quality of antibiotic use. Published sets of QIs often were particular to a specific clinical syndrome16–19 or patient population in the hospital.8,20 A recommended set of 11 generic QIs is one reported by van den Bosch et al.,21 which was reported to have high validity and generalizability for use in various hospital settings and infectious diseases. This study was aimed to assess the clinimetric profile (measurability, applicability, performance, improvement potential, inter-QI correlation, and Kappa coefficient) of the QIs based on van den Bosch *et. al.* for evaluating the appropriateness of antibiotic use in adult inpatients in 2 referral hospitals in Indonesia. ## Methods ### Setting, study population and data collection This was a multicenter observational study in which we measured the clinimetric profiles of 11 generic QIs by van den Bosch et. al.21 in 2 government referral hospitals, with Hospital 1 being an academic, national tertiary referral hospital and Hospital 2 being a nonacademic, secondary referral hospital. The medical records were electronic and paper-based in Hospital 1 and Hospital 2, respectively. Both hospitals had antibiotics stewardship programs at the time the study was conducted, and were on the national health scheme, which fully covered expenses for treatment as well as microbiology culture and sensitivity testing. The patients were included between July 2022-November 2022. To include a representative group of patients, we identified in each hospital on all departments, admitted adult patients who received empiric antibiotics for more than 24 hours because of suspected infection on the day of patient screening, using the point prevalence survey (PPS). For all admitted patients, those on antibiotics were screened and recruited in the study when meeting the study criteria. The hinclusion criteria were patients age ≥18 years old, had been started on systemic antibiotics at least 24 hours earlier, and having suspicion of bacterial infection. The exclusion criteria were patients on prophylactic antibiotics, and those who started antibiotics from a different hospital. For all included patients, clinical data and laboratory tests of the entire treatment course with antibiotics, from the start of antibiotic therapy until hospital discharge, were retrospectively extracted from medical and nursing records. To rule out prophylaxis, antibiotics had to be started at least 24 hours before inclusion. All data needed to calculate IQ were obtained by trained general practitioners. The data were extracted uniformly and entered into an electronic case report form (eCRF). This study is approved by the ethics committee of the Faculty of Medicine, Universitas Indonesia. ### Quality indicators and definitions We used the set of 11 generic QIs for appropriate antibiotic use developed by van den Bosch *et al.,* using a systematic modified Delphi method.21,22 The first 9 indicators are process indicators, while the last 2 indicators are structural indicators.21 All 11 QIs were tested for their clinimetric profiles in the two hospitals in our study. The list of the 11 generic QIs and their definitions are listed in **Table 1**. The criteria for safe switching within 72 h of treatment were as follows: (i) the patient was hemodynamically stable; (ii) no fever; (iii) normal white blood cell count; (iv) able to take oral medication; and (v) functioning gastrointestinal tract, without signs of malabsorption. A safe switch was not possible in cases of meningitis, intracranial abscesses, endocarditis, mediastinitis, *Legionella pneumonia*, exacerbations of cystic fibrosis, inadequately drained abscesses with empyema, severe soft tissue infections such as group A streptococcal infections, infections of foreign bodies, *Staphylococcus aureus* or *Pseudomonas aeruginosa* bacteremia, liver abscesses with empyema, osteomyelitis or arthritis, or chemotherapy-related neutropenia.23 View this table: [Table 1.](http://medrxiv.org/content/early/2023/11/25/2023.11.22.23298934/T1) Table 1. The list of the 11 generic QIs and their definitions. ### Analysis Collected data were entered into a database using the SPSS 25.0 for Windows® (SPSS Inc., Chicago, IL, USA). A descriptive analysis was performed for each indicator. Applicability represents the percentage of evaluated patients who had the necessary information to calculate the QIs. Performance was defined as the percentage of patients who adhered to the QIs. The potential for improvement for each IQ was calculated as 100% minus the performance score and was used to identify areas for improvement in the antibiotic prescriptions. Fifty samples were evaluated simultaneously by 2 investigators, to calculate inter-observer reliability for the performance scores of all QIs. The rate of agreement was calculated and expressed as kappa coefficients. Correlation between performance scores of the quality indicators were calculated using Spearman coefficient. ## Results The study included 500 subjects from Hospital 1 (n=317) and Hospital 2 (n=173). The screening of patients resulting in the included subjects are presented in **Figure 1**. The patient characteristics are presented in **Table 2**. The most common clinical syndrome was lower respiratory tract infection (n=281, 56.2%), with sepsis found in 14.4% of subjects. Other infections included brain and meningeal infections, acute gastroenteritis, exit site infections, osteoarticular infections, surgical wound infections, central line-associated bloodstream infections, mediastinitis, infected implants, upper respiratory tract infections, and lung abscesses. The distribution of wards in the two hospitals are presented in **Figure 2A**. Two or more infection diagnoses were present in 28,6% of subjects. All antibiotics (n=500, 100%) were initially administered intravenously. ![Figure 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2023/11/25/2023.11.22.23298934/F1.medium.gif) [Figure 1.](http://medrxiv.org/content/early/2023/11/25/2023.11.22.23298934/F1) Figure 1. Subject recruitment. Patients identified during the screening process. ![Figure 2.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2023/11/25/2023.11.22.23298934/F2.medium.gif) [Figure 2.](http://medrxiv.org/content/early/2023/11/25/2023.11.22.23298934/F2) Figure 2. The clinimetric profile of 11 generic quality indicators of appropriate antibiotic use for hospitalized patients. (A) Distribution of wards in 2 hospitals where subjects were included into the study. (B) Clinimetric profile heat map showing percentage clinimetric properties for all QIs. (C) The applicability of QIs in percentage. (D) The performance and improvement potentials of QIs in percentage. Heat map showing the Spearman correlation coefficient between all applicable QIs at the patient level. View this table: [Table 2.](http://medrxiv.org/content/early/2023/11/25/2023.11.22.23298934/T2) Table 2. Clinical characteristics. The clinimetric profile data are shown as heat map in **Figure 2B**, with data presented in **Table 3**. All QIs had good measureability in both hospitals, with missing data <10%. Ten QIs demonstrated good applicability (**Figure 2C**), except for QI 8 (therapeutic drug monitoring for aminoglycosides after >3 days and vancomycin after >5 days) with the applicability score of only 6,4%. Similar to the measurability scores, the applicability scores between the two hospitals in this study was also comparable. View this table: [Table 3.](http://medrxiv.org/content/early/2023/11/25/2023.11.22.23298934/T3) Table 3. Both hospitals exhibited variability in QI performance scores (**Figure 2D**). Hospital 1 had markedly higher performance scores than Hospital 2 for QI 1 (guideline concordance; 84.9% vs. 48.5%) and QI 3 (cultures obtained from suspected infection sites; 36.0% vs. 8.7%). In contrast, Hospital 2 had a higher performance score for QI 4 (documentation of antibiotic therapy plans at initiation; 97.3% vs. 73.5%). Among all applicable QIs, QI 2 on obtaining 2 sets of blood culture had the lowest performance score. Only 1,7% patients had two sets of blood culture taken, with 17% had one set of blood culture taken. Only 22 out of 72 subjects (30,5%) with sepsis had their blood culture taken. In this study, four patient-level QIs were found to have improvement potential scores above 30%: QI 2 (two blood cultures taken before antibiotic initiation, 98.6%), QI 3 (cultures obtained from suspected infection sites, 74%), QI 5 (switching from IV to oral antibiotics if clinically feasible, 48.1%), and QI 9 (discontinuation of empirical therapy if no clinical and/or microbiological evidence of infection, 93.7%). Additionally, QI 10 and QI 11 regarding hospital-level guidelines had a 50% potential for improvement. In this study, all QIs had a Kappa coefficient >0.6, and there was weak correlation between QIs, making it impossible to reduce the number of QIs for implementation. ## Discussion The need for appropriate antibiotic use to curb the growing antimicrobial resistance can be met with applying a set of QIs as part of an effective antimicrobial stewardship program. We herewith describe 10 generic QIs for appropriate antibiotic use in hospitalized patients with sound clinimetric properties applicable for daily use in Indonesia, with 6 QIs demonstrated improvement potential for prioritization of intervention. To the best of our knowledge, this is the first study identifying generic QIs with sound clinimetric profile for appropriate antibiotic use in all types of infections in Indonesia. Previously, performance properties of QIs for antimicrobial use have been assessed, but mostly only in European countries.7,9,17,18,21,24 Farida *et. al.*16 previously developed QIs for antibiotic use and measure their clinimetric properties in Indonesia, but only for pneumonia. Interestingly, 10 out of the 11 QIs in this study demonstrated sound applicability, in contrast to only 7 reported by van den Bosch *et. al.*21 in their validation study. Therapeutic drug monitoring (TDM) QI showed low applicability score due to the low use of vancomycin and aminoglycoside in our setting. This finding is also in line with that from van den Bosch *et. al.*21 and Arcenillas *et. al.*25 The QI was subsequently dismissed from our performance assessment. Indeed, TDM in Indonesia has not yet become part of routine clinical practice. This was largely due to the cost and expertise required,26 and the relative lack of need therein, as shown by this study. We would subsequently recommend this QI to also be dropped from being used elsewhere as 3 studies including ours21,25 have demonstrated its low applicability both in high- and middle-income countries. The improvement potential measures the sensitivity of a QI to detect variation in quality of care between hospitals and within a hospital. High-performance scores with low variation between hospitals render a QI insensitive and less useful in clinical practice because it has low potential for improvement. An improvement potential of >30% is considered sufficient for focusing intervention strategies on improving antibiotic use quality in a hospital. Four QIs at the patient level as well as two QIs at the hospital level were shown to have good improvement potential in this study. The prioritized QIs were: QI 2 (two blood cultures taken before antibiotic initiation), QI 3 (cultures obtained from suspected infection sites), QI 5 (switching from IV to oral antibiotics if clinically feasible), and QI 9 (discontinuation of empirical therapy if no clinical and/or microbiological evidence of infection) which focus on diagnostic and de-escalation practices. Process indicators are useful in determining the impact of antimicrobial stewardship. Beyond capturing the state of antibiotic use, the implementation of QIs may also aid in measuring how far an intervention impacts the quality antibiotic use.27,28 Measurement in the quality of antibiotic use has long been part of the routine practice in Indonesia. However, to date, the widely used audit method in Indonesia is the Gyssens flow chart, which was developed by Gyssens *et. al*.29 in 1992, both for clinical and research purposes.15,30 Compared to the Gyssens method, these 11 generic QIs by van den Bosch *et. al.* provided more insights into antibiotics prescribing practices, which includes assessments of: 1) processes prior to antibiotic prescribing (collection of 2 sets of blood culture samples and samples from suspected sites of infection), 2) documentation in the antiobiotic prescribing plan, 3) safe switch from iv to oral route; deescalation process which include 4) dose adjustment according to renal function (which captures the safety aspect of antibiotic use) and 5) stopping empiric antibiotic therapy when appropriate, as well as the availability of updated local antibiotics guidelines that are adapted to local antimicrobial resistace pattern at the hospital level. QI 2 (obtaining 2 sets of blood culture) performed extremely low in this study at 1,4%; whereas when 1 blood culture was obtained the adherence number increased to 17%. This finding was in line with that from Sinto *et. al.*31 which found that at least 13% of patients had their blood sample taken once for culture. The low number of blood culture utilization in our setting may be due to the lack of clear guidelines applied for identifying the indications of blood culture sampling, despite their availability.32–34 Furthermore, the QI with the biggest discrepancy in improvement potential among the two hospitals in this study is QI 1 (adherence to guideline). This may be affected by the fact that Hospital 2 with the lower performance score did not have their own antibiotics guideline, and without the national antibiotic guideline being used formally at the hospital. Therefore, decisions to start or choose which antibiotics were left for the clinicans, based on national/international guidelines, or to their own discretion. Indeed, barriers to antimicrobial stewardship in Indonesia still included the factors of antibiotic decision-making by clinicians. A study using questionnaires by Limato *et. al*.35 reported these factors include insufficient application of antimicrobial stewardship principles and microbiology services, the lack of confidence in prescribing decisions, and defensive prescribing due to diagnostics uncertainties, fear of patient deterioration, or because the patient insisted. A wide variation exists between hospitals, departments, work experience and medical hierarchy.11,15 The utilization of QIs could potentially overcome these issues by introducing them as actionable points. The 11 generic QIs in this study would provide clear guidance on appropriate antibiotic prescribing and empower clinical decision-making. Kallen *et. al.*8 developed a set of four indicators for appropriate antibiotic use in the ICU which were introduced as actionable QIs, together with an implementation toolbox which includes checklists and improvement strategies. Such strategies would render QIs not only useful for quality assessment, but also as a method of intervention. This study has several strengths. The selected 11 generic QIs were obtained through a multidisciplinary Delphi-RAND methodology,21,22 and were previously validated in a large group of patients,21,25 albeit exclusively in a European setting. Secondly, this study provides an insight into the clinimetric profiling of these QIs in a diverse, lower-middle income country, demonstrating its robustness. Thirdly, the study showed that assessment of these 11 QIs can be performed by a small number of people trained to collect this information from either an electronic or paper-based medical records, with good interobserver reliability, which supports its wide and ease of implementation. The potential limitations of this study are the involvement of two centers which are both located in Jakarta, the capital city of Indonesia. This would mean that the two hospitals have more similarities than its eastern or outer islands counterparts in Indonesia. Secondly, we had intended to include subjects in the ICU, as previous studies21,25 did not measure these QIs in the aforementioned population. However, we did not have a large number of patients originating from the ICU, and thus the specific applicability of these QIs in the ICU patients remains to be determined. In conclusion, the clinimetric assessment of QIs is imperative before their implementation in any setting different from their country of origin. Of the 11 QIs from van den Bosch *et. al.,* 10 demonstrated good reliability and applicability at the two hospitals in Indonesia, a low-middle-income non-European country. Interventions to improve the quality of antibiotic use can be focused on the six QIs with the greatest potential for improvement. Future intervention studies can utilize these generic QIs to measure improvement in the appropriateness of antibiotic use. ## Data Availability All data produced in the present study are available upon reasonable request to the authors. ## Footnotes * There is correction in the institutions of all authors. There is no correction in the content of the manuscript. * Received November 22, 2023. * Revision received November 23, 2023. * Accepted November 25, 2023. * © 2023, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/) ## References 1. 1.Giamarellou H, Galani L, Karavasilis T, Ioannidis K, Karaiskos I. Antimicrobial Stewardship in the Hospital Setting: A Narrative Review. Antibiotics [Internet]. 2023; 12(10). 2. 2.Cassini A, Högberg LD, Plachouras D, Quattrocchi A, Hoxha A, Simonsen GS, et al. Attributable deaths and disability-adjusted life-years caused by infections with antibiotic-resistant bacteria in the EU and the European Economic Area in 2015: a population-level modelling analysis. The Lancet infectious diseases. 2019;19(1):56–66. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S1473-099(18)30605-4&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F11%2F25%2F2023.11.22.23298934.atom) 3. 3.Barlam TF, Cosgrove SE, Abbo LM, MacDougall C, Schuetz AN, Septimus EJ, et al. Implementing an antibiotic stewardship program: guidelines by the Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America. Clin Infect Dis. 2016;62(10):e51–e77. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/cid/ciw118&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=27080992&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F11%2F25%2F2023.11.22.23298934.atom) 4. 4.Fishman N, Society for Healthcare Epidemiology of A, Infectious Diseases Society of A. Policy statement on antimicrobial stewardship by the society for healthcare epidemiology of America (SHEA), the infectious diseases society of America (IDSA), and the pediatric infectious diseases society (PIDS). Infect Control Hosp Epidemiol. 2012;33(4):322–7. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1086/665010&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22418625&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F11%2F25%2F2023.11.22.23298934.atom) 5. 5.Al-Omari A, Al Mutair A, Alhumaid S, Salih S, Alanazi A, Albarsan H, et al. The impact of antimicrobial stewardship program implementation at four tertiary private hospitals: results of a five-years pre-post analysis. Antimicrobial Resistance & Infection Control. 2020;9(1):1–9. 6. 6.D’Agata EMC, Tran D, Bautista J, Shemin D, Grima D. Clinical and economic benefits of antimicrobial stewardship programs in hemodialysis facilities: a decision analytic model. Clin J Am Soc Nephrol. 2018;13(9):1389. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiY2xpbmphc24iO3M6NToicmVzaWQiO3M6OToiMTMvOS8xMzg5IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjMvMTEvMjUvMjAyMy4xMS4yMi4yMzI5ODkzNC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 7. 7.Monnier AA, Schouten J, Le Maréchal M, Tebano G, Pulcini C, Stanić Benić M, et al. Quality indicators for responsible antibiotic use in the inpatient setting: a systematic review followed by an international multidisciplinary consensus procedure. J Antimicrob Chemother. 2018;73(suppl_6):vi30–vi9. 8. 8.Kallen MC, Prins JM. A systematic review of quality indicators for appropriate antibiotic use in hospitalized adult patients. Infect Dis Rep. 2017;9(1). 9. 9.Berrevoets MAH, Ten Oever J, Oerlemans AJM, Kullberg BJ, Hulscher ME, Schouten JA. Quality indicators for appropriate outpatient parenteral antimicrobial therapy in adults: a systematic review and RAND-modified Delphi procedure. Clin Infect Dis. 2020;70(6):1075–82. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/cid/ciz362&link_type=DOI) 10. 10.Stemkens R, Schouten JA, van Kessel SAM, Akkermans RP, Telgt DSC, Fleuren HWHA, et al. How to use quality indicators for antimicrobial stewardship in your hospital: a practical example on outpatient parenteral antimicrobial therapy. Clin Microbiol Infect. 2023;29(2):182–7. 11. 11.Mboi N, Syailendrawati R, Ostroff SM, Elyazar IRF, Glenn SD, Rachmawati T, et al. The state of health in Indonesia’s provinces, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019. The Lancet Global Health. 2022;10(11):e1632–e45. 12. 12.World Health Organization. TrACSS 2021-2022 2023 [Available from: [https://amrcountryprogress.org/download/AMR-self-assessment-surveyresponses-2020-2021.xlsx](https://amrcountryprogress.org/download/AMR-self-assessment-surveyresponses-2020-2021.xlsx). 13. 13.Wiseman V, Thabrany H, Asante A, Haemmerli M, Kosen S, Gilson L, et al. An evaluation of health systems equity in Indonesia: study protocol. International journal for equity in health. 2018;17(1):1–9. 14. 14.Parathon H, Kuntaman K, Widiastoety TH, Muliawan BT, Karuniawati A, Qibtiyah M, et al. Progress towards antimicrobial resistance containment and control in Indonesia. BMJ. 2017;358. 15. 15.Limato R, Lazarus G, Dernison P, Mudia M, Alamanda M, Nelwan EJ, et al. Optimizing antibiotic use in Indonesia: A systematic review and evidence synthesis to inform opportunities for intervention. The Lancet Regional Health-Southeast Asia. 2022. 16. 16.Farida H, Rondags A, Gasem MH, Leong K, Adityana A, van den Broek PJ, et al. Development of quality indicators to evaluate antibiotic treatment of patients with communityLacquired pneumonia in I ndonesia. Trop Med Int Health. 2015;20(4):501–9. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/tmi.12452&link_type=DOI) 17. 17.Schouten J, Hulscher M, Wollersheim H, Braspennning J, Kullberg B, Van der Meer J, et al. Quality of antibiotic use for lower respiratory tract infections at hospitals:(how) can we measure it? Clin Infect Dis. 2005;41(4):450–60. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1086/431983&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=16028151&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F11%2F25%2F2023.11.22.23298934.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000230611000005&link_type=ISI) 18. 18.Hermanides HS, Hulscher M, Schouten JA, Prins JM, Geerlings SE. Development of quality indicators for the antibiotic treatment of complicated urinary tract infections: a first step to measure and improve care. Clin Infect Dis. 2008;46(5):703–11. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1086/527384&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=18230045&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F11%2F25%2F2023.11.22.23298934.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000253453700010&link_type=ISI) 19. 19.Ten Oever J, Jansen JL, van der Vaart TW, Schouten JA, Hulscher MEJL, Verbon A. Development of quality indicators for the management of Staphylococcus aureus bacteraemia. J Antimicrob Chemother. 2019;74(11):3344–51. 20. 20.Schoffelen T, Schouten J, Hoogerwerf J, Quirós AM, May L, Ten Oever J, et al. Quality indicators for appropriate antimicrobial therapy in the emergency department: a pragmatic Delphi procedure. Clin Microbiol Infect. 2021;27(2):210–4. 21. 21.van den Bosch C, Hulscher M, Natsch S, Wille J, Prins J, Geerlings S. Applicability of generic quality indicators for appropriate antibiotic use in daily hospital practice: a cross-sectional point-prevalence multicenter study. Clin Microbiol Infect. 2016;22(10):888. e1-. e9. 22. 22.van den Bosch CM, Geerlings SE, Natsch S, Prins JM, Hulscher ME. Quality indicators to measure appropriate antibiotic use in hospitalized adults. Clin Infect Dis. 2015;60(2):281–91. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/cid/ciu747&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25266285&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F11%2F25%2F2023.11.22.23298934.atom) 23. 23.Sevinç F, Prins JM, Koopmans RP, Langendijk PN, Bossuyt PM, Dankert J, et al. Early switch from intravenous to oral antibiotics: guidelines and implementation in a large teaching hospital. J Antimicrob Chemother. 1999;43(4):601–6. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/jac/43.4.601&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=10350396&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F11%2F25%2F2023.11.22.23298934.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000080094200026&link_type=ISI) 24. 24.Pulcini C, Defres S, Aggarwal I, Nathwani D, Davey P. Design of a ’day 3 bundle’ to improve the reassessment of inpatient empirical antibiotic prescriptions. J Antimicrob Chemother. 2008;61(6):1384–8. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/jac/dkn113&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=18367462&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F11%2F25%2F2023.11.22.23298934.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000256172900034&link_type=ISI) 25. 25.Arcenillas P, Boix-Palop L, Gómez L, Xercavins M, March P, Martinez L, et al. Assessment of quality indicators for appropriate antibiotic use. Antimicrob Agents Chemother. 2018;62(12):10–1128. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1128/AAC.00523-18&link_type=DOI) 26. 26.Hou J, Marriott D, Cattaneo D, Stocker S, Stojanova J, Alffenaar J-W, et al. Therapeutic drug monitoring practices of anti-infectives: An Asia-wide cross-sectional survey. Front Pharmacol. 2022;13:992354. 27. 27.van Daalen FV, Lagerburg A, de Kort J, Rivas ES, Geerlings SE. Implementation of an antibiotic checklist increased appropriate antibiotic use in the hospital on Aruba. Int J Infect Dis. 2017;59:14–21. 28. 28.Marwick C, Watts E, Evans J, Davey P. Quality of care in sepsis management: development and testing of measures for improvement. J Antimicrob Chemother. 2007;60(3):694–7. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/jac/dkm234&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17595286&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F11%2F25%2F2023.11.22.23298934.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000249882200040&link_type=ISI) 29. 29.Gyssens IC, Van Den Broek PJ, Kullberg B-J, Hekster YA, Van Der Meer JW. Optimizing antimicrobial therapy. A method for antimicrobial drug me evaluation. J Antimicrob Chemother. 1992;30(5):724–7. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/jac/30.5.724&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=1493990&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F11%2F25%2F2023.11.22.23298934.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1992KC27000021&link_type=ISI) 30. 30.Nelwan EJ, Guterres H, Pasaribu AI, Shakinah S, Limato R, Widodo D. The Comparison of Point Prevalence Survey (PPS) and Gyssens Flowchart Approach on Antimicrobial Use Surveillance in Indonesian National Referral Hospital. Acta Med Indones. 2021;53(4):505–11. 31. 31.Sinto R, Lie KC, Setiati S, Suwarto S, Nelwan EJ, Djumaryo DH, et al. Blood culture utilization and epidemiology of antimicrobial-resistant bloodstream infections before and during the COVID-19 pandemic in the Indonesian national referral hospital. Antimicrobial Resistance & Infection Control. 2022;11(1):1–12. 32. 32.Lee A, Mirrett S, Reller LB, Weinstein MP. Detection of bloodstream infections in adults: how many blood cultures are needed? J Clin Microbiol. 2007;45(11):3546–8. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiamNtIjtzOjU6InJlc2lkIjtzOjEwOiI0NS8xMS8zNTQ2IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjMvMTEvMjUvMjAyMy4xMS4yMi4yMzI5ODkzNC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 33. 33.Miller JM, Binnicker MJ, Campbell S, Carroll KC, Chapin KC, Gilligan PH, et al. A guide to utilization of the microbiology laboratory for diagnosis of infectious diseases: 2018 update by the Infectious Diseases Society of America and the American Society for Microbiology. Clin Infect Dis. 2018;67(6):e1–e94. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/cid/ciy381&link_type=DOI) 34. 34.Neves L, Marra AR, Camargo TZS, dos Santos MC, Zulin F, da Silva PC, et al. Correlation between mass and volume of collected blood with positivity of blood cultures. BMC Res Notes. 2015;8:1–7. 35. 35.Limato R, Nelwan EJ, Mudia M, Alamanda M, Manurung ER, Mauleti IY, et al. Perceptions, views and practices regarding antibiotic prescribing and stewardship among hospital physicians in Jakarta, Indonesia: a questionnaire-based survey. BMJ open. 2022;12(5):e054768. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMjoiMTIvNS9lMDU0NzY4IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjMvMTEvMjUvMjAyMy4xMS4yMi4yMzI5ODkzNC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=)