Published benefits of ivermectin use in Itajaí, Brazil for COVID-19 infection, hospitalisation, and mortality are entirely explained by statistical artefacts ============================================================================================================================================================== * Robin Mills * Ana Carolina Peçanha Antonio * Greg Tucker-Kellogg ## Abstract **Background** Two recent publications by Kerr et al. (Cureus 14(1):e21272; Cureus 14(8): e28624) reported dramatic effects of prophylactic ivermectin use for both prevention of COVID-19 and reduction of COVID-19-related hospitalisation and mortality, including a dose-dependent effect of ivermectin prophylaxis. These papers have gained an unusually large public influence: they were incorporated into debates around COVID-19 policies and may have contributed to decreased trust in vaccine efficacy and public health authorities more broadly. Both studies were based on retrospective observational analysis of city-wide registry data from the city of Itajaí, Brazil from July-December 2020. **Methods** Starting with initially identified sources of error, we conducted a revised statistical analysis of available data, including data made available with the original papers and public data from the Brazil Ministry of Health. We identified additional uncorrected sources of bias and errors from the original analysis, including incorrect subject exclusion and missing subjects, an enrolment time bias, and multiple sources of immortal time bias. In models assuming no actual effect from ivermectin use, we conducted Monte Carlo simulations to estimate the contribution of these biases to any observed effect. **Results** Untreated statistical artefacts and methodological errors alone lead to dramatic apparent risk reduction associated with Ivermectin use in both studies. The magnitude of apparent risk reduction from these artefacts is comparable to the results reported by the studies themselves, including apparent protection from infection, hospitalisation, and death, and including the reported apparent dose-response relationship. **Conclusions** The inference of ivermectin efficacy reported in both papers is unsupported, as the observed effects are entirely explained by untreated statistical artefacts and methodological errors. Our re-analysis calls for caution in interpreting highly publicised observational studies and highlights the importance of common sources of bias in clinical research. ## Introduction The first half of 2020 was marked by both the beginning of the COVID-19 pandemic and frantic efforts around the globe to prevent and contain its spread. Most of those initial efforts relied on non-pharmaceutical interventions (e.g., social distancing, masks, travel restrictions, and regional lockdowns) to “flatten the curve” and reduce the strain on healthcare systems before effective treatments and vaccines became available [1, 2]. As the pandemic unfolded, healthcare systems worldwide rapidly became overburdened with an increasing number of severely ill patients and high COVID-19 mortality rates. In Brazil, there was immense interest in early COVID-19 treatment during the initial phase of the pandemic, including the potential use of the anti-parasitic drug ivermectin [3]. While there was no clinical evidence of ivermectin efficacy for COVID-19, initial *in vitro* studies at the time had shown potential antiviral activity of ivermectin in cell culture [4], which fuelled interest in its use. Starting in July 2020, the city of Itajaí (in the southern Brazil state of Santa Catarina) [5, 6] implemented a controversial city-wide program in July 2020 as a potential COVID-19 prophylaxis. Eligible residents were offered ivermectin pills with an intermittent dosing schedule of 0.2 mg/kg of body weight (up to a maximum 24 mg for those above 90 kg body weight) each day for two consecutive days, repeated every 15 days. In two closely related retrospective analysis studies of the Itajaí program, Kerr, Cadegiani et al. [7, hereafter KC22] and Kerr, Baldi et al. [8, hereafter KB22] made dramatic claims of ivermectin benefit. KC22 concluded that using ivermectin in the Itajaí program resulted in a 44% reduction in COVID-19 infections. Among all infected individuals, KC22 reported 37% reduction in hospitalisation and 43% reduction in mortality associated with ivermectin use. These already dramatic results were even larger after the application of propensity score matching (PSM) and adjustment for other covariates. Furthermore, KB22 presented an even more startling dose-dependent benefit among infected individuals: the so-called “strictly regular” ivermectin users experienced a 92% reduction in mortality compared to non-users and an 82% reduction compared to irregular users. These two papers gained significant public attention and contributed to ongoing public policy debates, both about COVID-19 treatment and prevention, and about trust in the medical and scientific establishment. Each paper has an Altmetric score in the top five per cent of *all* scientific research, has been the subject of news reporting, and has been shared on social media or viewed at the journal site hundreds of thousands of times. KC22 and KB22 have also attracted the attention of fact-checkers, but published critiques to date have mostly focused either on the limits of observational studies in general or on missing covariates and superficial peer review of these papers specifically [9–11]. In response, the Editor in Chief of Cureus has defended the peer review process and subsequent publication of these studies [12]. Given the widespread dissemination and discussion of these papers, it is crucial to provide an unbiased critique to foster a more informed public debate on scientific and medical research, ultimately helping to protect the public from scientific misinformation [13, 14]. This unbiased critique is particularly significant for influential papers addressing polarising topics of public interest, such as the use of ivermectin as a treatment for COVID-19 [15]. In this work, we take a direct approach by reanalysing the data available from KC22 and KB22 and combining it with public data from the Brazilian Health Ministry. We identify a variety of important statistical fallacies and other errors, and use simulations to estimate the consequences of leaving these issues untreated. Our analysis demonstrates that the seemingly dramatic benefits of prophylactic ivermectin for COVID-19, as reported in both KC22 and KB22, can be entirely attributed to unresolved statistical fallacies present in the original analyses. The code for all analyses presented in this manuscript can be found on GitHub, ensuring the reproducibility of our findings. ## Results ### The data from KC22 and KB22 The analyses of KC22 and KB22 compared events (infections, hospitalisations, and deaths) between participants in the program (who volunteered to take ivermectin as a *prophylactic* agent) and non-participants within a fixed study period (from July 7 through December 2, 2020). KC22 described the data set as *excluding* individuals who tested positive from the registry data prior to July 7, 2020. KC22 combined two data sets: one of the participants in the ivermectin prophylaxis program and the other from a citywide population registry to retrieve non-participants’ data. As we received no response about the availability of original data sets after contacting both the city authorities in Itajaí and the authors of KC22 and KB22, we restricted our analysis to KC22 supplementary data on OSF posted with KC22 by the corresponding author Flavio Cadegiani and official public data from the Brazil Ministry of Health. KC22 claimed virtually no missing values in the data because of mandatory reporting and indeed, most fields for each individual record were completely filled in. However, crucial information was missing for *all* participants in the data. For example, the amount of medicine for an individual was a function of weight (≤0.2 mg ivermectin/kg body weight/day for two days every 15 days), but neither body weight nor dosage category are reported. The KC22 data also failed to include any dates other than the date of birth; dates of program enrolment, medication collection, infection, hospitalisation, or death were all missing. Data entry errors in KC22 data were not uncommon. For example, while the maximum possible total ivermectin usage over the study period was 80 tablets, hundreds of users were recorded as having more than 80, and in some cases thousands, of tablets. ### KC22 mistakenly included prior infections and hospitalisations, primarily in the non-user group Dates of infection were absent in the KC22 uploaded data. Fortunately, official public data from the Brazilian Health Ministry’s Unified Health System (SUS) provides detailed information for all hospitalised COVID-19 patients in the national territory and includes related dates of initial symptoms. To confirm that infections before July 7 2020 were correctly excluded, we matched Itajaí residents from the SUS data with infected individuals from the KC22 data based on the date of birth, sex, and 2020 COVID-19 hospitalisation and death. Most hospitalised individuals in KC22 matched Brazilian Health Ministry data (138/185, 75%). To our surprise, COVID-19 symptom onset occurred before July 7, 2020 in 35 of the matched individuals, and the vast majority (29/35, 83%) were classified as non-users in the KC22 analysis (Table 1A, p < 0.001 for association between treatment group and mistaken inclusion). When we looked at subgroups by mortality for those hospitalised individuals, the bias for mistaken inclusion of early infections in the non-user group was even more severe for those who died (21 non-users, 2 user) than for those who survived (8 non-users, 4 users, Table 1B). View this table: [Table 1.](http://medrxiv.org/content/early/2023/08/15/2023.08.10.23293924/T1) Table 1. Mistaken inclusion of pre-study infections in the non-user group. KC22 study participants uniquely mapped with Brazil Health Ministry (SUS) individuals as described in the text. Parts (a) and (b) show overall and subgroup analysis, respectively. Because the SUS data set is focused on hospitalisations, we do not have any direct evidence that the biased inclusion of pre-study infections in the non-participant group extends to non-hospitalised subjects. However, the mistaken pre-study enrolment of non-users accounted for 41% of matched hospitalised non-users, and all infected non-users were included in the propensity score matching of KC22, so the impact of this mistake alone was dramatic. ### KC22 data was biased towards a subset of early infections KC22 and KB22 severely under-reported hospitalisations and deaths from COVID-19 in Itajaí. KC22 reported 185 total hospitalisations from a study population of 159,561 (0.12% hospitalised); official government data reports 2863 Itajaí (adult pop. 161,545, 1.8%) adult hospitalisations for COVID-19 in 2020, of which 1659 had symptom onset during the KC22 study period. Among those 1659 hospitalisations there were 499 reported deaths -a citywide post-hospitalisation COVID death rate of 30.1% during the study period. In contrast, KC22 reports 141 deaths and 185 hospitalisations during the same period, dramatically under-reporting both outcomes while simultaneously more than doubling the post-hospitalisation death rate to 76.2%. KC22 claimed that 71.3% of adult population of Itajaí (113,845) had participated on the ivermectin program. The under-reporting of hospitalisations and deaths cannot be attributed to the reported ivermectin effects claimed by KC22. To further understand this issue, we compared the dates of symptom onset for all adult Itajaí residents hospitalised for COVID-19 (obtained from the SUS records, Fig. 1A) to the onset dates of the 138 individuals uniquely mapped to hospitalisations in KC22 (Fig. 1B). Strikingly, the KC22 data not only mistakenly included hospitalised individuals with symptom onset *before* the study period, but almost all of the remaining matched hospitalised individuals experienced symptom onset *in the first half* of the study period. KC22 entirely neglected the second wave of COVID hospitalisations, which peaked at the end of study period. ![Figure 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2023/08/15/2023.08.10.23293924/F1.medium.gif) [Figure 1.](http://medrxiv.org/content/early/2023/08/15/2023.08.10.23293924/F1) Figure 1. Symptom onset and study group allocation. A. All official Brazilian Health Ministry entries for hospitalised adult residents of Itajaí. B. The 134 individuals mapped to the SUS data from KC22. C. The probability of an individual considered in the study being allocated to the non-user group changes dramatically over time during the allocation of participants in KC22. ### Enrolment to the ivermectin program continued during the study period Both KC22 and KB22 compared events between adult residents of Itajaí who did or did not participate in the ivermectin program over the entire study period. KC22 claimed “This strict interval avoids differences in terms of periods of exposure”. We definitely disagree: not all 113,845 participants received ivermectin exactly on July 7, 2020. Consider an individual who reported symptoms on July 8, but had not yet joined the ivermectin distribution program. That individual would be ineligible for inclusion in the KC22 study as an ivermectin user, but would instead be treated in the analysis of KC22 as an infected non-user. The biased allocation of new infections into the non-user group would continue for as long as the distribution program enrolled new participants (Fig. 1C). This alone is a classic case of immortal time bias [16], but because the most rapid enrolment occurred during the peak infection period in July 2020, it also coincides with substantial chronological bias [17]. To distinguish these two sources of bias during the enrolment period with other sources of bias described later, we refer them collectively as “enrolment bias”. ### Enrolment bias, incorrect inclusion of already infected participants, and biased sampling towards the beginning of the study led to large apparent protection against infection, hospitalisation, and death in KC22 We replicated the KC22 study using Monte Carlo simulations under the assumption of no ivermectin effect to assess the impact of the enrolment bias, the incorrect inclusion of subjects with symptomatic onset prior to the study, and the biased sampling towards the beginning of the study. We simulated symptom onset in individual patient data cohorts each the size of the data set in KC22. To simulate the dates of symptom onset, we used daily Itajaí notifications of infection from the Brazil Health Ministry. To simulate enrolment over time, we used contemporaneous local news reports of program enrolment for participants and assumed participants began taking ivermectin immediately. We developed multiple models to estimate the individual and cumulative effects of the identified biases and errors. Each model incorporated 7231 infections within a cohort of 159,560 individuals (infection rate of approximately 4.5%). Additionally, we simulated hospitalisations and deaths to match KC22 totals, using dates sampled from the Brazil Health Ministry records, as detailed in the Methods and Materials section. Results are reported based on 1000 simulations of each model and compared to reported values from KC22. Hospitalisations and deaths reported among infected individuals and compared to the infected individuals from KC22 prior to propensity score matching. The enrolment bias present in KC22 leads to an estimated fictitious risk reduction for infection of 18% for users of an ineffective medicine. This result stems from our first model (i-ENR), in which we randomly sampled infections from the distribution of onset dates between July 7 and December 2, 2020. In addition to the enrolment bias, the incorrect inclusion of subjects with symptomatic on-set prior to the study period leads to an additional 12% fictitious risk reduction for infection. In our second model (i-INF), we simulated incorrect inclusion by sampling from symptom onset dates where the notification date was between July 7 – December 2, 2020. The i-INF model thus encompassed both enrolment bias and the mistaken inclusion of early infections. Lastly, an additional 13% fictitious risk reduction for infection is caused by the biased sampling towards the beginning of study observed in KC22. In the third model family (i-KC22), we mimicked the observation of sampling bias in Fig. 1B by sampling infections strictly from infection dates of matched individuals, or sampling using a smoothed estimator (developed from the empirical data) of the chance of recording symptom in the study. Details of these models (which give similar results) are discussed in the methods. The i-KC22 model family most closely matched the errors and biases so far discussed in KC22. The results of the models, alongside the published results of KC22 and a trivial negative model (i-NEG, with neither ivermectin effect nor any biases) are shown in Table 2. Each source of bias reduced the apparent incident rates of infection, hospitalisation, and death of ivermectin users and, hence, increased the apparent risk reduction of the exposure. Biases in the design and execution of KC22 account for *all* of the reported protection attributed to ivermectin. View this table: [Table 2.](http://medrxiv.org/content/early/2023/08/15/2023.08.10.23293924/T2) Table 2. Apparent protection provided by biases and errors in KC22. Apparent protection against infection, hospitalisation, and death due to enrolment bias, mistaken inclusion of prior infections, and missing data. Comparisons are between users and non-users, as defined in KC22. Three different simulation strategies were used to simulate the isolated effects of biases identified in [7] along with a negative control (i-NEG). Hospitalisation and death statistics are reported among infected individuals. i-KC22 and the published KC22 data set include all three biases. Simulations are as described in the text. The estimated effect of enrolment bias in the i-ENR model is conservative, since we assumed optimistically that news reporting was accurate and that ivermectin use began immediately for all participants. While we have direct evidence for the temporal-biased sampling in the case of hospitalisation and death, the effect of temporal sampling bias for infections in i-KC22 is indirectly estimated based on assumed consistency with hospitalisations and deaths. Full details of the simulation methods are found in the Methods and in the GitHub repository accompanying this paper. ### KC22 hospitalisation and mortality results included attrition bias by design COVID-19 outcome events occur in sequence: hospitalisation usually precedes COVID-19 death; COVID-19 infection always precedes COVID-19 hospitalisation and COVID-19 death. KC22 stated they had included “all events” from July 7 – December 2, 2020, so hospitalisations and deaths *after* the study period following infections *during* the study period were ignored. This would not necessarily introduce additional bias if the allocation of ivermectin users and non-users had been balanced. However, the enrolment bias previously described leads not just to a fictitious benefit from treatment, but to a difference between exposure groups in the distribution of infections over time: non-user infections accumulate earlier in the study period, and infections among users accumulate later. Indeed, the median infection date for the non-user group over 1000 simulations was September 4, 2020, while that for the user group was October 2, almost a month later. The enrolment bias in recorded infections thus leads to additional bias due to attrition for later hospitalisations and deaths [18, 19]. As shown in Table 3, the KC22 study design under-counted hospitalisations and deaths among ivermectin users, because those individuals were enrolled over time while event counting was stopped at a fixed date. This became especially prominent when we added the temporal-biased sampling observed in Fig. 1B in the i-KC22 model, as shown in Table 3B. View this table: [Table 3.](http://medrxiv.org/content/early/2023/08/15/2023.08.10.23293924/T3) Table 3. Attrition bias provides apparent protection against hospitalisation and death. A. 1000 runs of i-ENR model, restricted to infected individuals. B. 1000 runs of i-KC22 model, restricted to infected individuals. C. Results as reported in KC22, table 6. Fig. 2 illustrates the basis of this finding in detail. In KC22, infections tended to occur earlier in the non-user group because of the delayed enrolment in the user group and the July 2020 infection peak (Fig. 2A). Death events after the follow-up period are lost to attrition as a rule in any given cohort, but this attrition is more likely among users than non-users in KC22, as shown for a single simulated cohort in Fig. 2B. A higher percentage of deaths than hospitalisations are lost to attrition in both groups (Fig. 2C) but the ivermectin user group is more likely to have uncounted hospitalisations and deaths than the non-user group. This translates to an increased risk of attrition in the ivermectin group, a relative risk that is higher for deaths than for hospitalisations (Fig. 2D). ![Figure 2.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2023/08/15/2023.08.10.23293924/F2.medium.gif) [Figure 2.](http://medrxiv.org/content/early/2023/08/15/2023.08.10.23293924/F2) Figure 2. Attrition bias in hospitalisations and deaths in KC22 stemmed from enrolment bias. A. Empirical distributions of simulated infection dates over 1000 runs of the i-ENR model. Note the delayed early peak of infections in the ivermectin user group. B. Example from one typical simulation of uncounted deaths among ivermectin users. Each line segment represents an individual in the simulation who was infected and later died, with infection and death dates at the end points. The study end date is marked with a vertical line. C. Hospitalisations and deaths are lost to attrition more frequently in the user group (1000 simulations of the i-ENR model). D. The relative risk of attrition in the ivermectin user group over 1000 simulations (the horizontal line shows a relative risk of 1). ![Figure 3.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2023/08/15/2023.08.10.23293924/F3.medium.gif) [Figure 3.](http://medrxiv.org/content/early/2023/08/15/2023.08.10.23293924/F3) Figure 3. KB22’s “regular” use group [8] created more immortal time bias. A. Simulated dates of hospitalisation in i-ENR for individuals grouped by exposure according to KB22-defined usage groups. The study end date is marked with a vertical line. B. Dates of death. C. Relative risk of hospitalisation and death for “regular” ivermectin users compared to “irregular” and non-users. Risk ratios are calculated and plotted for each of 1000 simulations. Dates of hospitalisation and death are shown for all individuals across 1000 simulations. Unattributed bias and sampling errors thus account for roughly all of the reported protection against hospitalisation and death among infected individuals in KC22. ### The “regular ivermectin user” distinction in KB22 created additional immortal time bias We now turn to the second paper from the Itajaí study (KB22), which considered the “regular use” of ivermectin. Because actual ivermectin use was not measured, KB22 treated the return to collect medication over time as a surrogate measure for the actual ivermectin intake. KB22 further subdivided ivermectin users into exposure groups based on the total amount of medication distributed: regular users (those who had received at least thirty 6 mg ivermectin tablets, or 180 mg total), irregular users (those who had received no more than 10 tablets), and non-users. According to KC22, newly diagnosed COVID-19 patients were recommended “not to use ivermectin” and that “The city did not provide or support any specific pharmacological outpatient treatment for subjects infected with COVID-19”. The citywide ivermectin program was, as KC22 and KB22 made clear, the use of ivermectin as a prophylactic for COVID-19, not as a treatment. The central claim of KB22 was a dose-response relationship between ivermectin use and protection from infection, hospitalisation, and death. The exposure groups in KB22, however, were assigned *retrospectively* based on the amount of medication distributed over time. Moreover, both the language of KC22 and contemporary news reports suggest that users would have stopped ivermectin use upon infection, but the analysis of KB22 assumes the opposite, treating ivermectin usage as an independent variable. How long would someone have to take ivermectin to be classified as a regular user in KB22? The maximum dosage in the Itajaí program was 4 tablets a day for two days, or 8 tablets every 15 days from day 2 of usage. This would require a minimum of 46 days after enrolment to be classified as a regular user. The more typical 3-tablet dosage would require 61 days. This entire time period is “immortal” time for regular users who stopped ivermectin use upon infection: infections during that time period would result in their allocation to other groups and an apparent reduction of the infection rate in regular users. We simulated the effects of these biases by randomly allocating intended usage among ivermectin users using the total tablet distribution from KC22 data, and truncating to simulated actual usage based on how long an individual participated in the study without infection. We assumed a body weight between 60-90 kg, so 3 tablets per day of use. Because actual changes in usage upon infection are unknown, we considered two distinct scenarios. In the first, deterministic, scenario, all infected users stopped ivermectin upon infection. In the second, probabilistic, scenario, we used the regularity groupings of KB22 as a proxy for commitment: regular users receiving ≥30 tablets would stop on infection with a probability of 5%, irregular users would stop with a probability of 30%. As there is no evidence that ivermectin was offered to COVID-19 inpatients at the time, all users would stop taking ivermectin on hospitalisation. In both scenarios, the immortal time bias leads to a strong fictitious dose-response relationship under the assumption of ineffective medicine. In the deterministic stopping scenario the dose-response relationship was even stronger than reported in KB22. In the probabilistic setting, which models a very small change in user behaviour upon infection, our simulations closely match the dose-response relationship found in KB22 (see Table 4). View this table: [Table 4.](http://medrxiv.org/content/early/2023/08/15/2023.08.10.23293924/T4) Table 4. Statistical biases and errors account for all effects of ivermectin for Covid-19 reported in KC22 and KB22. This is the iKC22 model with probabilistic stop on infection, including a stop probability of 0.3 (for irregular users) and 0.05 (for regular users). Details of the this and other models are described in the text. Results of all models are reported in the supplementary tables. 1Risk Ratio. 2Experiment artefacts cause a 100% risk reduction, which precludes any estimation of the effect of ivermectin in the KB22 and KC22 experimental setup for this outcome. 3Simulations showing 100% risk reduction preclude estimation of confidence intervals and p values after correction, but suggest that the artifacts account for at least all the observed benefit. ## Discussion ### Re-analysis of the data shows no benefit from ivermectin use on infections, hospitalisations, or death A comparison between the key findings in KC22 and KB22 and our re-analysis is found in Table 4. The apparent risk reduction from documented artefacts — including well-known biases and sampling errors — accounts for all of the reported benefits of ivermectin claimed in both KC22 and KB22. As the key findings from Table 4 underlie all subsequent analyses in KC22 and KB22, none of the results reported in either paper holds up to scrutiny. ### The errors are pervasive. Should they have been obvious? The biases and failings in KC22 and KB22 originate from a mix of sources. Some biases (such as the immortal time bias during enrolment and the KB22-specific immortal time bias from the definition of “regular users”) stem from the study design itself. These biases should be evident, immediately or after some thought, to any scientific or medical professional with training and experience in study design. The attrition bias in hospitalisations and deaths that is magnified by enrolment bias is perhaps less self-evident, but can still be understood from first principles. Understanding other sources of bias in KC22 and KB22 requires data. To appreciate the chronological bias during enrolment, for example, one must at least be aware of public data such as reported case rates over time. Still other issues, such as the pervasive sampling biases detailed above, require careful examination, cross-tabulation, and analysis of available data to identify the extent of particular issues. Another evident bias in KC22 and KB22, not addressed by our analysis, is their complete lack of attention to health inequity. This is hard to fathom given the likelihood of socioeconomic differences between participants and non-participants. Participation in the Itajaí program required individuals to take proactive steps: they needed to travel to distribution centres, sign up for the program, register their information, receive medication, and return periodically for medication refills. KC22 and KB22 included known *prognostic* factors in propensity score matching after infection. However, neither KC22 nor KB22 accounted for any socioeconomic factors as potential covariates affecting either infection risk or prognosis, even though impoverished and vulnerable populations are known to have higher risk and worse outcomes for many diseases and to be harder to reach and recruit for clinical studies [20, 21]. These well-known inequities were magnified during the early phases of COVID-19 because of stresses in public health systems [22–25]. Furthermore, health inequity — including during the COVID-19 pandemic — is an area of active study in Brazil, with greater risk of hospitalisation and death from social inequity and disadvantage [26–28]. While some of the issues we identified (such as those apparently arising from incomplete data and sampling errors) might escape even a rigorous review process, other issues (such as the multiple sources of immortal time bias) arise from the study design itself. In our view, these issues should have been recognised and addressed by the authors, and they should have been recognised and questioned by the reviewers. They were not. Instead, each paper was submitted, revised, and accepted in a matter of days, and immediately entered public discussion with a primary focus on the large size of the study population and the large magnitude of reported risk reduction, as if a large reported risk reduction must be true if the study itself is large. Unfortunately the opposite may hold if a study is built on a fallacious design. In such a case both the reported treatment effect and the apparent confidence (especially when reported as “statistical significance”) may increase with a larger study population [29–31], even if there is no actual effect of treatment. This appears to be the situation with KC22 and KB22. ### The growing consensus and persistent divide on use of ivermectin for Covid-19 Ivermectin has been known to have different mechanisms in vertebrates and nemotodes for over 30 years [32]. Ivermectin’s mechanism and safety as an antithelmintic stems from its potent targeting of glutamate-gated chloride channel receptors essential for nemotodes but not found in vertebrates [33]. When ivermectin was first suggested based on *in vitro* experiments as a potential anti-viral treatment for COVID-19 [4], it was not altogether unreasonable: some of the same researchers had reported a possible antiviral mechanism through nuclear α/β1 importin complex[34, 35]. However, those studies were undertaken in cell lines such as Vero E6 or Hela (both widely used for viral assays); when careful comparison studies were carried out in more relevant human bronchial epithelial cells, ivermectin had no effect on SARS-CoV-2 replication [36]. There were other reasons to be sceptical of the initial enthusiasm; the well-studied pharmacokinetics and pharmacodynamics of ivermectin suggested that it would be impossible to replicate in humans the concentrations required for *in vitro* activity [37, 38]. As in Itajaí, some doctors began prescribing ivermectin based on preliminary studies; most mainstream public health authorities encouraged clinical trials. As a result, hundreds of studies of ivermectin for COVID-19 have been published in the last three years, including dozens of clinical trials, numerous cell biology and biochemical studies, mechanistic speculation based on molecular docking, and competing reviews. Some of the earliest high-profile studies reporting large effects of ivermectin have been retracted or otherwise flagged for ethical concerns [39, 40]. Meta-analyses that included such flawed analysis have also been retracted [41] and reviews advocating for the immediate use of ivermectin [42] have been criticised as deeply problematic [43]. One possible lesson from both ivermectin meta-analysis and vaccine clinical trials is to require individual patient data, rather than summary data alone, for assessment of bias in meta-analysis [44, 45]. While rigorous randomised clinical trials have largely not found clinical benefit for ivermectin use in COVID-19 [46–49], and there is strong and growing scientific and clinical consensus against its clinical use for COVID-19 treatment or prophylaxis [50], this consensus is persistently rejected by ivermectin advocates. Regional, national, and international organisations1 have sprung up to advocate for ivermectin and other non-proven treatments for COVID-19, and strive to influence public opinion. Ivermectin-for-COVID advocacy groups maintain their position and influence through mechanisms including promoting “science by preprint”, exploiting perfunctory peer review, aggressively using social media, and cultivating socio-poltical alliances including the anti-vaccination movement. Observational studies are particular vulnerable to misinterpretation and use as misinformation. ### The use of simulation to solve statistical fallacies Statistical fallacies or ‘statistical lies’ affect our lives in many ways: we read them in newspapers, we hear them in conversation, we inadvertently make them ourselves, and they are unfortunately common in science. The fallacies often go hand-in-hand with cognitive heuristics that bias our perception of reality [51]. For instance, salience bias (the tendency to focus on remarkable events or prominent features) leads people to overestimate risk of rare events and make decisions that appear irrational and incur a cost to themselves and to society [52, 53]. Despite the review process, biases and statistical fallacies also arise in scientific literature [19, 54], and proliferation of these fallacies carries the potential for disaster: the incorrect conclusions of KC22 and KB22 for instance, were used to support arguments that ivermectin was at least as effective as vaccination against COVID-19 related death, potentially increasing vaccine hesitancy and thereby increasing the global death toll due to COVID-19. One of the most famous statistical fallacies is seen in the Monty Hall problem, named for the original host of the American television show “Let’s Make a Deal”, where a variant of this puzzle appeared in every episode. In this puzzle, a player is presented three closed doors and asked to choose one. Behind one of these doors is a prize which the player will win if they choose the correct door. Once the player has chosen a door, the host reveals which of the other two doors does not contain the prize, and subsequently asks the player if they would like to stick with their initial choice or switch to the other closed door. When presented for the first time, most people assume that switching their choice will not affect the likelihood of winning a prize The answer, however2 is that the likelihood of winning a prize after switching is two thirds, whereas the likelihood of winning is only one third when the participant doesn’t switch doors. Remarkably, even when people are shown explanations, simulations and mathematical proofs, many - including renown statisticians - still refuse to accept the answer of the puzzle [55, 56]. Studies using repeated simulations of the Monte Hall problem show a remarkable adoption of the correct answer. Herein participants play the game over and over on the computer, and get feedback on how often they won the prize. The Monte Carlo simulation that we use here is basically an automation of this process. Akin to the Monte Hall problem, researchers may falsely reason that it is correct to include participants that got infected by COVID-19 before registering/consuming ivermectin as non-users in a rolling registration context. But simulating the process unveils that an artificial efficacy emerges for the treatment group. We hope therefore that our work extends beyond a correction of these two papers, and helps researchers of observational trials in general not to repeat these fallacies. ## Methods and materials ### Starting data We used data made available by the authors of KC22 at DOI 10.17605/OSF.IO/UXHAF. This data was missing critical information, which we addressed as follows. City-wide monthly infections and deaths were taken from [43], which in turn obtained them from the Brazilian Health Ministry. We also used the Brazilian Health Ministry resource to obtain national reporting data for individuals including dates of symptom onset, hospitalisation, and death, though this was largely limited to hospitalised patients during the time of the study. We downloaded this data on 26 September 2022. While the study began enrolling participants on July 7, 2020, the KC22 data does not indicate when any individual patient joined the program and was provided ivermectin. We used an estimate of program enrolment over time by following news articles in the local Itajaí press (e.g. [57]). Sources for each estimate are in the GitHub repository. ### Analysis of overlap between KC22 and Brazilian Health Ministry data Data from KC22 was cross-tabulated with public data from the Brazilian Health Ministry. After identifying variables that were available in both data sets, we identified matches between infected individuals (both users and non-users) from KC22 and infected individuals from the health ministry data. Matches were counted only if they were identical for all of the variables considered (birth date, hospitalisation status, sex, and death outcome). A few entries in KC22 had more than one match to the health ministry data, which could be due to data errors or multiple infections. We included the match with the latest date of symptom onset in order to avoid over-counting pre-study inclusion in KC22. ### Allocation to exposure and outcome groups For each simulation experiment, repeated Bernoulli trials are conducted for each of the 159,560 individuals to assign them to exposure groups (users or non users) as well as outcomes (infections, hospitalisations, deaths). Hospitalisation was always preceded by infection, and death was always preceded by hospitalisation. All allocation probabilities were fixed based on the number of users, non-users, infections, hospitalisations, and deaths reported in KC22 (e.g. probability to be an ivermectin user is 113,844 / 159,560). ### Simulation of event times and ivermectin usage For each simulated individual that contracted COVID-19, was hospitalised, or died, the event date was sampled from the empirical distributions of official government records. In the case of missing data, we used complete cases to confirm published reports that time periods in the progression of COVID-19 were well-approximated by a Weibull distribution [58–60], and then created an objective function using the empirical mean and standard deviation of time delays to numerically optimise the Weibull scale and shape parameters to impute missing event dates. Intended ivermectin usage was truncated to actual usage as follows. In probabilistic truncation, infected users with high intended usage (≥30 pills) were less likely (p = 0.1) to stop usage on infection than those with lower intended usage (p = 0.5). In deterministic truncation, all infected users stopped ivermectin use upon infection. In all cases, infected users stopped upon hospitalisation. Ivermectin users where split into irregular (≤10 pills) and regular users (≥30 pills) based on simulated actual usage. ### Calculation of simulated estimates Risk ratios were calculated using unconditional maximum likelihood (Wald statistic), while confidence intervals and p values estimated from 10,000 bootstraps using the riskratio.boot function from the epitools package in R [61]. Summary statistics from KC22 and KB22 were reestimated the same way for consistency. For summary statistics after correction, we assumed any true effects of ivermectin were independent of apparent effects from artefacts and errors. Risk ratios, confidence intervals, and p values after correction were calculated by 10,000 bootstraps of simulated and reported data for each simulation. ### Data and code availability All data is publicly available at either DOI 10.17605/OSF.IO/UXHAF (posted by Kerr et al), the Brazil Ministry of Health ([https://www.gov.br/saude/pt-br](https://www.gov.br/saude/pt-br)), or Our World in Data ([https://ourworldindata.org/COVID-cases](https://ourworldindata.org/COVID-cases)). R code to reproduce the analysis is available at [https://github.com/gtuckerkellogg/itajai-reanalysis](https://github.com/gtuckerkellogg/itajai-reanalysis). ## Data Availability All data in the present work are available online at the URLs below. [https://osf.io/uxhaf/](https://osf.io/uxhaf/) [https://www.gov.br/saude/pt-br](https://www.gov.br/saude/pt-br) [https://github.com/gtuckerkellogg/itajai-reanalysis](https://github.com/gtuckerkellogg/itajai-reanalysis) ## Author contributions RM conducted the research required to uncover all fallacies mentioned in the manuscript (delayed registrations, missing data, incorrect inclusion of prior infections, all biases). GTK independently uncovered the immortal time bias, enrolment bias, and attrition bias. RM and GTK wrote the simulation and analysis code. GTK in particular, RM, and ACPA contributed to writing and reviewing the manuscript. ACPA acquired the data from the Brazilian Health Ministry. ACPA has been in frequent contact with Itajaí City Hall in an attempt to get access to missing raw data. RM has been in frequent contact with the KC22 and KB22 authors in an attempt to discuss the issues in their work before publishing this manuscript. ## Acknowledgements We thank Gideon Meyerowitz-Katz, Lisa Tucker-Kellogg, Andrew Gelman, Amichai Perlman, and Annelot Mills for their valuable comments and feedback. ## Appendix ### i-ENR model simulations View this table: [Table S1.](http://medrxiv.org/content/early/2023/08/15/2023.08.10.23293924/T5) Table S1. i-ENR model with probabilistic stop on infection. Stop probability: 0.30 (irregular), 0.05 (regular). Statistics for hospitalisations and deaths are limited to infected individuals. View this table: [Table S2.](http://medrxiv.org/content/early/2023/08/15/2023.08.10.23293924/T6) Table S2. i-ENR model with probabilistic stop on infection. Stop probability: 0.30 (irregular), 0.05 (regular). Statistics for hospitalisations and deaths are reported for all individuals in the cohort. View this table: [Table S3.](http://medrxiv.org/content/early/2023/08/15/2023.08.10.23293924/T7) Table S3. i-ENR model with uniform stop on infection. Stop probability: 1.0 (all individuals). Statistics for hospitalisations and deaths are limited to infected individuals. View this table: [Table S4.](http://medrxiv.org/content/early/2023/08/15/2023.08.10.23293924/T8) Table S4. i-ENR model with uniform stop on infection. Stop probability: 1.0 (all individuals). Statistics for hospitalisations and deaths are reported for all individuals in the cohort. ### i-INF model simulations View this table: [Table S5.](http://medrxiv.org/content/early/2023/08/15/2023.08.10.23293924/T9) Table S5. i-INF model with probabilistic stop on infection. Stop probability: 0.30 (irregular), 0.05 (regular). Statistics for hospitalisations and deaths are limited to infected individuals. View this table: [Table S6.](http://medrxiv.org/content/early/2023/08/15/2023.08.10.23293924/T10) Table S6. i-INF model with probabilistic stop on infection. Stop probability: 0.30 (irregular), 0.05 (regular). Statistics for hospitalisations and deaths are reported for all individuals in the cohort. View this table: [Table S7.](http://medrxiv.org/content/early/2023/08/15/2023.08.10.23293924/T11) Table S7. i-INF model with uniform stop on infection. Stop probability: 1.0 (all individuals). Statistics for hospitalisations and deaths are limited to infected individuals. View this table: [Table S8.](http://medrxiv.org/content/early/2023/08/15/2023.08.10.23293924/T12) Table S8. i-INF model with uniform stop on infection. Stop probability: 1.0 (all individuals). Statistics for hospitalisations and deaths are reported for all individuals in the cohort. ### i-KC22 model simulations View this table: [Table S9.](http://medrxiv.org/content/early/2023/08/15/2023.08.10.23293924/T13) Table S9. i-KC22 model with probabilistic stop on infection. Stop probability: 0.30 (irregular), 0.05 (regular). Statistics for hospitalisations and deaths are limited to infected individuals. View this table: [Table S10.](http://medrxiv.org/content/early/2023/08/15/2023.08.10.23293924/T14) Table S10. i-KC22 model with probabilistic stop on infection. Stop probability: 0.30 (irregular), 0.05 (regular). Statistics for hospitalisations and deaths are reported for all individuals in the cohort. View this table: [Table S11.](http://medrxiv.org/content/early/2023/08/15/2023.08.10.23293924/T15) Table S11. i-KC22 model with uniform stop on infection. Stop probability: 1.0 (all individuals). Statistics for hospitalisations and deaths are limited to infected individuals. ### i-KC22 model simulations View this table: [Table S12.](http://medrxiv.org/content/early/2023/08/15/2023.08.10.23293924/T16) Table S12. i-KC22 model with uniform stop on infection. Stop probability: 1.0 (all individuals). Statistics for hospitalisations and deaths are reported for all individuals in the cohort. ## Footnotes * **Data and code availability:** All data is publicly available from previous publications or public databases of the Brazil Ministry of Health. Links are provided at the end of the Methods section. Source code to reproduce the analyses here is available at [https://github.com/gtuckerkellogg/itajai-reanalysis](https://github.com/gtuckerkellogg/itajai-reanalysis). * **Funding:** No funding was received for this work. * **Competing interests:** GTK receives revenue from YouTube for content on scientific misinformation and received conference travel support from the Institute for Clinical Research (Malaysia) for a talk given at the 15th National Conference for Clinical Research (NCCR). ACPA and RM declare no competing interests. * **Ethics approval:** The Institutional Review Board of the National University of Singapore waived ethical approval of this work based on sole use of previously approved and publicly available subject data. Reference NUS-IRB-2023-474. * 1 These include the Front Line COVID-19 Critical Care Alliance (FLCCC), America’s Frontline Doctors, and the World Council for Health * 2 in the simplest case, assuming that the host always opens the wrong door and always gives the player a choice to switch * Received August 10, 2023. * Revision received August 10, 2023. * Accepted August 15, 2023. * © 2023, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution-NoDerivs 4.0 International), CC BY-ND 4.0, as described at [http://creativecommons.org/licenses/by-nd/4.0/](http://creativecommons.org/licenses/by-nd/4.0/) ## References 1. [1]. Seth Flaxman et al. “Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe”. In: Nature 584.7820 (Aug. 2020). Number: 7820 Publisher: Nature Publishing Group, pp. 257–261. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41586-020-2405-7&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F15%2F2023.08.10.23293924.atom) 2. [2]. Patrick G. T. Walker et al. “The impact of COVID-19 and strategies for mitigation and suppression in low- and middle-income countries”. In: Science 369.6502 (July 24, 2020). Publisher: American Association for the Advancement of Science, pp. 413–422. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6Mzoic2NpIjtzOjU6InJlc2lkIjtzOjEyOiIzNjkvNjUwMi80MTMiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMy8wOC8xNS8yMDIzLjA4LjEwLjIzMjkzOTI0LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 3. [3]. Leonardo Furlan and Bruno Caramelli. “The regrettable story of the “Covid Kit” and the “Early Treatment of Covid-19” in Brazil”. In: The Lancet Regional Health - Americas 4 (Dec. 2021), p. 100089. 4. [4]. Leon Caly et al. “The FDA-approved drug ivermectin inhibits the replication of SARS-CoV-2 in vitro”. In: Antiviral Research 178 (June 1, 2020), p. 104787. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F15%2F2023.08.10.23293924.atom) 5. [5].Decree 11940 2020 of Itajaí SC. Regulamenta a Distribuição do Medicamento Ivermectina no Município de Itajaí. July 7, 2020. URL: [https://web.archive.org/web/20221022092413/](https://web.archive.org/web/20221022092413/)[https://leismunicipais.com.br/a/sc/i/itajai/decreto/2020/1194/11940/decreto-n-11940-2020-regulamenta-a-distribuicao-do-medicamento-ivermectina-no-municipio-de-itajai](https://leismunicipais.com.br/a/sc/i/itajai/decreto/2020/1194/11940/decreto-n-11940-2020-regulamenta-a-distribuicao-do-medicamento-ivermectina-no-municipio-de-itajai) (visited on 10/22/2022). 6. [6].“The Brazilian city being turned into a coronavirus “lab experiment"”. In: Coda Story [https://www.codastory.com/waronscience/brazil-covid19-ivermectin/](https://www.codastory.com/waronscience/brazil-covid19-ivermectin/) (Aug. 27, 2020). 7. [7]. Lucy Kerr et al. “Ivermectin Prophylaxis Used for COVID-19: A Citywide, Prospective, Observational Study of 223,128 Subjects Using Propensity Score Matching”. In: Cureus 14.1 (Jan. 15, 2022). Publisher: Cureus. 8. [8]. Lucy Kerr et al. “Regular Use of Ivermectin as Prophylaxis for COVID-19 Led Up to a 92% Reduction in COVID-19 Mortality Rate in a Dose-Response Manner: Results of a Prospective Observational Study of a Strictly Controlled Population of 88,012 Subjects”. In: Cureus 14.8 (Aug. 31, 2022). Publisher: Cureus. 9. [9]. Catalina Jaramillo. “Clinical Trials Show Ivermectin Does Not Benefit COVID-19 Patients, Contrary to Social Media Claims”. In: FactCheck.org (Sept. 15, 2022). 10. [10].“Fact Check-Gaps in study claiming that ivermectin reduces risk of COVID-19 death by 92%”. In: Reuters (Sept. 20, 2022). 11. [11].“Study claiming that ivermectin reduces COVID-19 mortality by 92% has important methodological problems”. In: Health Feedback (Sept. 7, 2022). 12. [12].Author Adam Marcus. Journal says ivermectin study met standard for ‘credible science’. Retraction Watch. Sept. 19, 2022. URL: [https://retractionwatch.com/2022/09/19/journal-says-ivermectin-study-met-standard-for-credible-science/](https://retractionwatch.com/2022/09/19/journal-says-ivermectin-study-met-standard-for-credible-science/) (visited on 04/25/2023). 13. [13]. Sander van der Linden. “Misinformation: susceptibility, spread, and interventions to immunize the public”. In: Nature Medicine (Mar. 10, 2022). Publisher: Nature Publishing Group, pp. 1–8. 14. [14]. Gordon Pennycook et al. “Shifting attention to accuracy can reduce misinformation online”. In: Nature 592.7855 (Apr. 2021). Number: 7855 Publisher: Nature Publishing Group, pp. 590–595. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41586-021-03344-2&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33731933&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F15%2F2023.08.10.23293924.atom) 15. [15]. Kamila Janmohamed et al. “Interventions to Mitigate COVID-19 Misinformation: A Systematic Review and Meta-Analysis”. In: Journal of Health Communication 26.12 (Dec. 2, 2021). Publisher: Taylor & Francis _eprint: [https://doi.org/10.1080/10810730.2021.2021460](https://doi.org/10.1080/10810730.2021.2021460), pp. 846–857. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F15%2F2023.08.10.23293924.atom) 16. [16]. Catalog of Bias Collaboration, H Lee, and D Nunan. “Immortal time bias”. In: Catalog of Bias. May 16, 2020. 17. [17]. Catalog of Bias Collaboration, EA Spencer, and C Heneghan. “Chronological bias”. In: Catalog of Bias. July 18, 2017. 18. [18]. Catalog of Bias Collaboration et al. “Attrition bias”. In: Catalog of Bias. July 20, 2017. 19. [19]. D. L. Sackett. “Bias in analytic research”. In: Journal of Chronic Diseases 32.1 (1979), pp. 51–63. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/0021-9681(79)90012-2&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=447779&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F15%2F2023.08.10.23293924.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1979HC23400008&link_type=ISI) 20. [20]. Grace Keegan, Angelena Crown, and Kathie-Ann Joseph. “Diversity, Equity, and Inclusion in Clinical Trials”. In: Surgical Oncology Clinics of North America 32.1 (Jan. 2023), pp. 221–232. 21. [21]. Stacy J. UyBico, Shani Pavel, and Cary P. Gross. “Recruiting Vulnerable Populations into Research: A Systematic Review of Recruitment Interventions”. In: Journal of General Internal Medicine 22.6 (June 2007), pp. 852–863. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s11606-007-0126-3&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17375358&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F15%2F2023.08.10.23293924.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000246563900024&link_type=ISI) 22. [22]. Abigail LaPlante et al. “Enrollment, retention, and strategies for including disadvantaged populations in randomized controlled trials: a systematic review protocol”. In: Systematic Reviews 10.1 (Aug. 18, 2021), p. 233. 23. [23]. Alein Y. Haro-Ramos et al. “Frontline work and racial disparities in social and economic pandemic stressors during the first COVID-19 surge”. In: Health Services Research (Jan. 31, 2023). 24. [24]. Ricardo E. da Silva et al. “Factors Contributing to Exacerbating Vulnerabilities in Global Clinical Trials”. In: Frontiers in Pharmacology 8 (2017), p. 999. 25. [25]. Daniel T. Lackland et al. “Impact of COVID-19 on Clinical Research and Inclusion of Diverse Populations”. In: Ethnicity & Disease 30.3 (2020), pp. 429–432. 26. [26]. Paulo Ricardo Martins-Filho et al. “COVID-19 fatality rates related to social inequality in Northeast Brazil: a neighbourhood-level analysis”. In: Journal of Travel Medicine 27.7 (Nov. 9, 2020), taaa128. 27. [27]. Carlos Dornels Freire de Souza, Rodrigo Feliciano do Carmo, and Michael Ferreira Machado. “The burden of COVID-19 in Brazil is greater in areas with high social deprivation”. In: Journal of Travel Medicine 27.7 (Nov. 9, 2020), taaa145. 28. [28]. Gabriela Drummond Marques da Silva et al. “Influência da desigualdade socioeconômica na distribuição das internações e dos óbitos por covid-19 em municípios brasileiros, 2020: um estudo ecológico Influência da desigualdade socioeconômica na distribuição das internações e dos óbitos por covid-19 em municípios brasileiros, 2020: um estudo ecológico”. In: Epidemiologia e servicos de saude : revista do Sistema Unico de Saude do Brasil 32.1 (2023). Place: Brazil, e2022303. 29. [29]. Sander Greenland et al. “Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations”. In: European Journal of Epidemiology 31 (2016), pp. 337–350. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s10654-016-0149-3&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=27209009&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F15%2F2023.08.10.23293924.atom) 30. [30]. Jimmie Leppink, Kal Winston, and Patricia O’Sullivan. “Statistical significance does not imply a real effect”. In: Perspectives on Medical Education 5.2 (Apr. 1, 2016), pp. 122–124. 31. [31]. C. L. Bandt and J. R. Boen. “A prevalent misconception about sample size, statistical significance, and clinical importance”. In: Journal of Periodontology 43.3 (Mar. 1972), pp. 181–183. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=4501976&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F15%2F2023.08.10.23293924.atom) 32. [32].1. William C. Campbell. M. J. Turner and J. M. Schaeffer. “Mode of Action of Ivermectin”. In: Ivermectin and Abamectin. Ed. by William C. Campbell. New York, NY: Springer, 1989, pp. 73–88. 33. [33]. Mohammed Atif et al. “Effects of glutamate and ivermectin on single glutamate-gated chloride channels of the parasitic nematode H. contortus”. In: PLoS Pathogens 13.10 (Oct. 2017). Publisher: PLOS. 34. [34]. Kylie M. Wagstaff et al. “Ivermectin is a specific inhibitor of importin α/β-mediated nuclear import able to inhibit replication of HIV-1 and dengue virus”. In: Biochemical Journal 443.3 (Apr. 16, 2012), pp. 851–856. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBwYmlvY2hlbWoiO3M6NToicmVzaWQiO3M6OToiNDQzLzMvODUxIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjMvMDgvMTUvMjAyMy4wOC4xMC4yMzI5MzkyNC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 35. [35]. Sundy N. Y. Yang et al. “The broad spectrum antiviral ivermectin targets the host nuclear transport importin α/β1 heterodimer”. In: Antiviral Research 177 (May 1, 2020), p. 104760. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.antiviral.2020.104760&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F15%2F2023.08.10.23293924.atom) 36. [36]. Nilima Dinesh Kumar et al. “Moxidectin and Ivermectin Inhibit SARS-CoV-2 Replication in Vero E6 Cells but Not in Human Primary Bronchial Epithelial Cells”. In: Antimicrobial Agents and Chemotherapy 66.1 (Jan. 18, 2022). Publisher: American Society for Microbiology, e01543–21. 37. [37].Ricardo Peña-Silva et al. “Pharmacokinetic considerations on the repurposing of ivermectin for treatment of COVID-19”. In: British Journal of Clinical Pharmacology 87.3 (2021). _eprint: [https://onlinelibrary.wiley.com/doi/pdf/10.1111/bcp.14476](https://onlinelibrary.wiley.com/doi/pdf/10.1111/bcp.14476), pp. 1589–1590. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F15%2F2023.08.10.23293924.atom) 38. [38].Aránzazu González Canga et al. “The Pharmacokinetics and Interactions of Ivermectin in Humans—A Mini-review”. In: The AAPS Journal 10.1 (Jan. 25, 2008), pp. 42–46. 39. [39]. Sara Reardon. “Flawed ivermectin preprint highlights challenges of COVID drug studies”. In: Nature 596.7871 (Aug. 2, 2021). Bandiera\_abtest: a Cg\_type: News Number: 7871 Publisher: Nature Publishing Group Subject_term: Research data, SARS-CoV-2, Public health, Publishing, pp. 173–174. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/d41586-021-02081-w&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=34341573&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F15%2F2023.08.10.23293924.atom) 40. [40]. Gideon Meyerowitz-Katz et al. “Unethical studies of ivermectin for covid-19”. In: BMJ 377 (Apr. 14, 2022). Publisher: British Medical Journal Publishing Group Section: Editorial, o917. 41. [41]. Andrew Hill et al. “Retraction to: Meta-analysis of Randomized Trials of Ivermectin to Treat SARS-CoV-2 Infection”. In: Open Forum Infectious Diseases 9.3 (Mar. 2022), ofac056. 42. [42]. Pierre Kory et al. “Review of the Emerging Evidence Demonstrating the Efficacy of Ivermectin in the Prophylaxis and Treatment of COVID-19”. In: American Journal of Therapeutics 28.3 (June 2021), e299. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/MJT.0000000000001377&link_type=DOI) 43. [43]. Gabriela Buffet, Raul Mendoza-Sassi, and Marinos Fysekidis. “Inaccurate Real-World Data Does Not Provide Real-World Answers”. In: American Journal of Therapeutics 28.5 (June 29, 2021), e596–e598. 44. [44]. Jack M. Lawrence et al. “The lesson of ivermectin: meta-analyses based on summary data alone are inherently unreliable”. In: Nature Medicine 27.11 (Nov. 2021). Number: 11 Publisher: Nature Publishing Group, pp. 1853–1854. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F15%2F2023.08.10.23293924.atom) 45. [45]. Kyle A. Sheldrick, Gideon Meyerowitz-Katz, and Greg Tucker-Kellogg. “Plausibility of Claimed Covid-19 Vaccine Efficacies by Age: A Simulation Study”. In: American Journal of Therapeutics (2022). 46. [46]. Steven Chee Loon Lim et al. “Efficacy of Ivermectin Treatment on Disease Progression Among Adults With Mild to Moderate COVID-19 and Comorbidities: The I-TECH Randomized Clinical Trial”. In: JAMA Internal Medicine (Feb. 18, 2022). 47. [47]. Gilmar Reis et al. “Effect of Early Treatment with Ivermectin among Patients with Covid-19”. In: New England Journal of Medicine . (Mar. 30, 2022). Publisher: Massachusetts Medical Society _eprint: [https://doi.org/10.1056/NEJMoa2115869](https://doi.org/10.1056/NEJMoa2115869), null. 48. [48]. Susanna Naggie et al. “Effect of Ivermectin vs Placebo on Time to Sustained Recovery in Outpatients With Mild to Moderate COVID-19: A Randomized Clinical Trial”. In: JAMA 328.16 (Oct. 25, 2022), pp. 1595–1603. 49. [49]. Carolyn T. Bramante et al. “Randomized Trial of Metformin, Ivermectin, and Fluvoxamine for Covid-19”. In: The New England Journal of Medicine 387.7 (Aug. 18, 2022), pp. 599–610. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa2201662&link_type=DOI) 50. [50]. Maria Popp et al. “Ivermectin for preventing and treating COVID-19”. In: Cochrane Database of Systematic Reviews 6 (2022). Publisher: John Wiley & Sons, Ltd. 51. [51]. A. Tversky and D. Kahneman. “Judgment under uncertainty: Heuristics and biases”. In: Science 185.4157 (1974), pp. 1124–1131. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6Mzoic2NpIjtzOjU6InJlc2lkIjtzOjEzOiIxODUvNDE1Ny8xMTI0IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjMvMDgvMTUvMjAyMy4wOC4xMC4yMzI5MzkyNC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 52. [52]. Pedro Bordalo, Nicola Gennaioli, and Andrei Shleifer. “Salience Theory of Choice Under Risk”. In: The Quarterly Journal of Economics 127.3 (Aug. 1, 2012), pp. 1243–1285. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/qje/qjs018&link_type=DOI) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000307835000005&link_type=ISI) 53. [53]. Pedro Bordalo, Nicola Gennaioli, and Andrei Shleifer. “Salience and consumer choice”. In: Journal of Political Economy 121.5 (2013). Publisher: University of Chicago Press Chicago, IL, pp. 803–843. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1086/673885&link_type=DOI) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000330542600001&link_type=ISI) 54. [54]. Andrew Gelman and Eric Loken. “The statistical crisis in science”. In: American Scientist 102.6 (2014). Publisher: Sigma XI-The Scientific Research Society, p. 460. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1511/2014.111.460&link_type=DOI) 55. [55]. Peter Mueser and Donald Granberg. The Monty Hall Dilemma Revisited: Understanding the Interaction of Problem Definition and Decision Making. Experimental. University Library of Munich, Germany, 1999. 56. [56]. Lore Saenen et al. “Why Humans Fail in Solving the Monty Hall Dilemma: A Systematic Review”. In: Psychologica Belgica 58.1 (June 1, 2018). texids= saenenWhyHumansFail2018a, pp. 128–158. 57. [57].“TCE pede explicações à prefeitura de Itajaí sobre distribuição de ivermectina; médicos dividem opinião sobre uso contra Covid-19”. In: globo.com (July 31, 2020), https://g1.globo.com/sc/santa–catarina/noticia/2020/07/31/tce–pede–explicacoes–a–prefeitura–de–itajai–sobre–compra–e–distribuicao–de–antiparasitario–medicos–dividem–opiniao–sobre–uso–contra–covid–19.ghtml. 58. [58]. Ke Men et al. “Estimate the incubation period of coronavirus 2019 (COVID-19).” In: Computers in biology and medicine 158 (May 2023). Place: United States, p. 106794. 59. [59]. Long V. Bui et al. “Estimation of the incubation period of COVID-19 in Vietnam.” In: PloS one 15.12 (2020). Place: United States, e0243889. 60. [60]. Giacomo Grasselli et al. “Hospital-Acquired Infections in Critically Ill Patients With COVID-19.” In: Chest 160.2 (Aug. 2021). Place: United States, pp. 454–465. 61. [61]. Tomas J. Aragon. epitools: Epidemiology tools. manual. 2020.