Characterizing the effective reproduction number during the COVID-19 epidemic: Insights from Qatar’s experience ================================================================================================================= * Raghid Bsat * Hiam Chemaitelly * Peter Coyle * Patrick Tang * Mohammad R. Hasan * Zaina Al Kanaani * Einas Al Kuwari * Adeel A. Butt * Andrew Jeremijenko * Anvar Hassan Kaleeckal * Ali Nizar Latif * Riyazuddin Mohammad Shaik * Gheyath K. Nasrallah * Fatiha M. Benslimane * Hebah A. Al Khatib * Hadi M. Yassine * Mohamed G. Al Kuwari * Hamad Eid Al Romaihi * Mohamed H. Al-Thani * Abdullatif Al Khal * Roberto Bertollini * Laith J. Abu-Raddad * Houssein H. Ayoub ## Abstract **Background** The effective reproduction number, *R**t*, is a tool to track and understand epidemic dynamics. This investigation of *R**t* estimations was conducted to guide the national COVID-19 response in Qatar, from the onset of the epidemic until August 18, 2021. **Methods** Real-time “empirical” ![Graphic][1] was estimated using five methods, including the Robert Koch Institute, Cislaghi, Systrom-Bettencourt and Ribeiro, Wallinga and Teunis, and Cori et al. methods. *R* was also estimated using a transmission dynamics model ![Graphic][2]. Uncertainty and sensitivity analyses were conducted. Agreements between different *R**t* estimates were assessed by calculating correlation coefficients. **Results** ![Graphic][3] captured the evolution of the epidemic through three waves, public health response landmarks, effects of major social events, transient fluctuations coinciding with significant clusters of infection, and introduction and expansion of the B.1.1.7 variant. The various estimation methods produced consistent and overall comparable ![Graphic][4] estimates with generally large correlation coefficients. The Wallinga and Teunis method was the fastest at detecting changes in epidemic dynamics. ![Graphic][5] estimates were consistent whether using time series of symptomatic PCR-confirmed cases, all PCR-confirmed cases, acute-care hospital admissions, or ICU-care hospital admissions, to proxy trends in true infection incidence. ![Graphic][6] correlated strongly with ![Graphic][7] and provided an average ![Graphic][8]. **Conclusions** *R**t* estimations were robust and generated consistent results regardless of the data source or the method of estimation. Findings affirmed an influential role for *R**t* estimations in guiding national responses to the COVID-19 pandemic, even in resource-limited settings. Keywords * SARS-CoV-2 * COVID-19 * effective reproduction number * mathematical modeling * epidemiology ## INTRODUCTION The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic is the most serious global health challenge in recent history [1,2]. Coronavirus Disease 2019 (COVID-19) morbidity and mortality has imposed unparalleled burdens on healthcare systems worldwide, and necessitated unprecedented restrictions on mobility and on social and economic activities [3,4]. Tracking and monitoring each wave of infection have become essential to avoid the adverse consequences of infection transmission [5-8]. With such serious consequences to the healthcare system, economy, and society, decisions regarding the escalation or easing of restrictions have become a critical facet of policymaking since the discovery of the virus in December of 2019 [6,9,10]. The effective reproduction number (*R**t*), the average number of secondary infections each infection is generating at a given point in time [6,11-13], has proven to be one of the most influential tools in monitoring and tracking the epidemic, and informing the escalation and easing of restrictions [6,11-13]. In the present study, we report on the use of *R**t* in Qatar, to understand epidemic dynamics and to establish national policy decisions and public heath interventions, in what has become a successful country experience. Qatar is a peninsula in the Arabian Gulf with a diverse population of 2.8 million people [5,14] that has been affected by three SARS-CoV-2 epidemic waves [5,6,15-19]. The first wave started with the introduction of the virus in February of 2020 and peaked in late May 2020 [5,6]. The second wave started in mid-January, 2021, and was triggered by the introduction and expansion of the B.1.1.7 (Alpha [20]) variant [15-19,21]. This wave peaked in the first week of March, but was followed immediately by a third wave that was triggered by introduction and rapid expansion of the B.1.351 (Beta [20]) variant, which started in mid-March and peaked in mid-April, 2021 [15-19,21]. Two forms of *R**t* estimation have been used in Qatar to inform the national COVID-19 response, and each proved to have its own intrinsic public health value. The first is the real-time “empirical” estimation which is done by calculating *R**t* directly from diagnosed cases. Different methods were explored for estimating the empirical *R**t* (henceforth, ![Graphic][9]), and based on this exploration the Robert Koch Institute method [13,22] was used for feasibility, ease of use, and functionality. The second estimation method was model-based by calculating *R**t* using a population-level compartmental transmission dynamics model [6,23], hereafter designated as ![Graphic][10]. ## METHODS ### Data sources Analyses were conducted using the centralized, integrated, and standardized national SARS-CoV-2 databases compiled at Hamad Medical Corporation (HMC), the main public healthcare provider and the nationally designated provider for all COVID-19 healthcare needs. These databases have captured all SARS-CoV-2-related data since the start of the epidemic, including all records of polymerase chain reaction (PCR) testing, antibody testing, COVID-19 hospitalizations, vaccinations, infection severity classification per World Health Organization (WHO) guidelines [24], and COVID-19 deaths, also assessed per WHO guidelines [25]. Every PCR test conducted in Qatar, regardless of location (outpatient clinic, drive-thru, or hospital, etc.), is classified on the basis of symptoms and the reason for testing (clinical symptoms, contact tracing, random testing campaigns, individual requests, healthcare routine testing, pre-travel, and at port of entry). PCR-confirmed infections are classified as “symptomatic” if testing was done because of clinical suspicion due to symptoms compatible with a respiratory tract infection. Classification of infections by variant type was informed by weekly rounds of viral genome sequencing and multiplex, real-time reverse-transcription PCR (RT-qPCR) variant screening [26] of randomly collected clinical samples [15-19], as well as by results of deep sequencing of wastewater samples [17]. Based on existing evidence [27-29] and confirmation with viral genome sequencing [21], a B.1.1.7 case was defined as an S-gene “target failure” using the TaqPath COVID-19 Combo Kits (Thermo Fisher Scientific, USA) [30]. This accounted for >85% of PCR testing in Qatar, applying the criterion of a PCR cycle threshold (Ct) value ≤30 for both the N and ORF1ab genes, and a negative outcome for the S gene [29]. This definition was used to derive the B.1.1.7 case series data that were used subsequently to derive for only the B.1.1.7 variant. ### Empirical estimation methods Five methods [13,31] were used to calculate ![Graphic][11] from daily diagnosed cases. To minimize effects of bias due to variation in the PCR testing volume over time, ![Graphic][12] was calculated using only the time series of cases diagnosed due to presence of clinical symptoms. Cases diagnosed through testing conducted for other reasons were not used in these analyses, except in a sensitivity analysis. #### Robert Koch Institute method This method, which was chosen as the standard method for ![Graphic][13] estimation in Qatar, utilizes the generation time (*τ**G*), the time interval between the infection of an infector and an infectee in a transmission pair [13,22], to provide an estimate for ![Graphic][14]. ![Graphic][15]. is calculated as the sum of newly diagnosed cases during *τ**G* consecutive days over the sum of previously diagnosed cases during the *τ**G* preceding days [22]. *τ**G* was assumed to be seven days, as informed by empirical evidence [32,33]. To smooth the curve and to avoid strong daily variations due to noise, ![Graphic][16] was calculated as a three-day moving average. The range of uncertainty in the estimated ![Graphic][17] due to sampling variation was derived by applying the binomial sampling distribution to the number of positive PCR tests out of all tests, day by day, and repeating this process 1,000 times. Four sensitivity analyses on the estimated ![Graphic][18] were conducted. In the first sensitivity analysis, the time series of all diagnosed cases (regardless of reason for PCR testing) was used instead of the time series of only symptomatic cases. In the second and third sensitivity analyses, the time series of hospital admissions in acute-care beds and ICU-care beds was used to proxy the epidemic trend, instead of the time series of symptomatic cases. In the fourth sensitivity analysis, the generation time *τ**G* was assumed to be 5, 7, and 10 days, instead of the fixed value of seven days [32,33]. #### Cislaghi method This method utilizes the incubation time (*τ**I*), the time interval between infection and symptom onset in an infected individual [33], to generate an estimate for ![Graphic][19]. ![Graphic][20] is calculated as the number of newly diagnosed cases on day *s* over the number of newly diagnosed cases on day *s* − *τ**I* [34]. *τ**I* was assumed to be five days [32,33]. To smooth the curve and to avoid strong daily variations due to noise, ![Graphic][21] was calculated as a three-day moving average. #### Wallinga and Teunis method This method utilizes the serial interval (*τ**S*), the time interval between symptom onset of an infector and that of an infectee [33], to generate an estimate for ![Graphic][22]. A likelihood-based estimate for ![Graphic][23] is derived by using pairs of diagnosed cases and the probability distribution for *τ**S* [35]. *τ**S* was assumed to have a Weibull distribution with a mean of 5.19 days and a standard deviation of 1.39 days, as informed by a meta-analysis of available data for SARS-CoV-2 infection [36]. #### Systrom-Bettencourt and Ribeiro method This method utilizes an approximate relationship between ![Graphic][24] and the exponential growth of the epidemic, and assumes that ![Graphic][25] evolves as a Gausian process to provide a Bayesian ![Graphic][26] estimation [12,37-39]. A Gaussian filter was applied to account for daily variations (noise) in ![Graphic][27] using a variance that was estimated by maximizing the log-likelihood of observing newly diagnosed cases [12,37-39]. #### Cori et al. method This method utilizes the infectivity profile (*ω**S*) of an infected individual to generate an estimate for ![Graphic][28] [31]. The average ![Graphic][29] is estimated by the ratio of the number of newly diagnosed cases at time step *t*, to the sum of newly diagnosed cases up to time step *t* − 1, weighted by *ω**S*. The infectivity profile was approximated by the distribution of the serial interval [31]. Bayesian statistical inference based on a Poisson process was used to generate the posterior distribution of ![Graphic][30], after assuming a gamma prior distribution for ![Graphic][31] [31]. ### Model-based estimation method An age-structured deterministic mathematical model was developed to describe SARS-CoV-2 transmission dynamics in the population of Qatar [6,23]. The model was developed as informed by other models [6,23,40-42], and has been used, expanded, and continuously refined since the onset of the epidemic. This model has been the reference model for policy decision-making in Qatar, for providing forecasts, investigating epidemiology, and assessing the impact of public health interventions [6,23]. The model stratified the population into compartments according to age group (0-9, 10-19, 20-29, …, ≥80 years), infection status (infected, uninfected), infection type (asymptomatic/mild, severe, and critical), COVID-19 disease type (severe or critical disease), and vaccination status (vaccinated, unvaccinated) using sets of coupled, nonlinear differential equations (Figure S1 in the Online Supplementary Document). The model was parameterized using current data for SARS-CoV-2 natural history and epidemiology [6,23]. It was fitted to the national standardized, integrated, and centralized databases of SARS-CoV-2 diagnosed cases, SARS-CoV-2 PCR and antibody testing, COVID-19 hospitalizations, and COVID-19 mortality [6], as well as to data of a series of SARS-CoV-2 epidemiological studies in Qatar [5,21,43-48]. The size and demographic structure of the population of Qatar were based on data from Qatar’s Planning and Statistics Authority [5,14,49]. ![Graphic][32] was derived using this model and was expressed in terms of the social contact rate in the population, transmission probability of the infection per contact, duration of infectiousness, and proportion of the population that is still susceptible to the infection [6,23]. A detailed description of the model, its input data, and fitting are available in References [6,23]. The model was coded, fitted, and analyzed using MATLAB R2019a [50]. ### Agreements between *R**t* estimates Agreements between different *R**t* estimates were assessed by calculating both the Pearson correlation coefficient, to assess the existence of a linear relationship, and also by calculating the Spearman correlation coefficient, to assess the existence of a monotonic (rank) relationship. ### Ethical approvals The study was approved by the Hamad Medical Corporation and Weill Cornell Medicine-Qatar Institutional Review Boards with waiver of informed consent. ## RESULTS The ![Graphic][33] calculated using the Robert Koch Institute method captured effectively the evolution of the epidemic through its three waves, starting from the first wave (the wild-type variant wave) [5,6], the second (B.1.1.7) wave [15-19,21], and the third (B.1.135) wave [15-19,21] (Figure 1A). It also captured and correlated with key response landmarks, such as partial lockdowns during the three waves and subsequent easing of public health restrictions, and major social events that led to transient increases in the social contact rate in the population. It further captured transient fluctuations that were associated with significant clusters of infection, especially during low-incidence phases between August 1, 2020 and January 17, 2021, and between May 25, 2021 and August 18, 2021. ![Figure 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/10/07/2021.10.07.21264599/F1.medium.gif) [Figure 1.](http://medrxiv.org/content/early/2021/10/07/2021.10.07.21264599/F1) Figure 1. Effective reproduction numbers ![Graphic][34] and ![Graphic][35] in Qatar. A) Trend in ![Graphic][36] and ![Graphic][37], April 1, 2020 to August 18, 2021, and association with major events, response landmarks, and introduction and expansion of the B.1.1.7 and B.1.135 variants. B) Trend in ![Graphic][38] for only the B.1.1.7 variant cases, February 1, 2021 to April 1, 2021. ![Graphic][39] was estimated using the Robert Koch Institute method [22] applied to symptomatic case series data. The dashed green line represents the threshold of *R* = 1. The epidemic expansion of B.1.1.7 cases starting from January 18, 2021 was associated with a large and rapid increase in ![Graphic][40] (Figure 1A), suggesting the higher infectiousness of this variant. ![Graphic][41] calculated using only B.1.1.7 case series data is shown in Figure 1B and demonstrated higher values, confirming further the higher infectiousness of this variant. ![Graphic][42] for only the B.1.1.7 variant averaged 1.45 during the exponential growth phase of the second (B.1.1.7) wave (February 1-22, 2021). It was unstable during the first two weeks of this wave (January 18-31, 2021; not shown), as transmission appears to have been influenced by one or more superspreading events that were not representative of the average community transmission. It was also unstable after April 1, 2021, as the number of daily B.1.1.7 cases was too small. The first sensitivity analysis on the estimated ![Graphic][43], in which the time series of all diagnosed cases was used instead of only symptomatic cases, showed overall excellent agreement regardless of the input-data source used to calculate ![Graphic][44] (Figure 2A). The Pearson correlation coefficient was 0.914 (p-value<0.001) and the Spearman correlation coefficient was 0.913 (p-value<0.001), both confirming the excellent agreement. There were only few noticeable differences that occurred when the number of diagnosed cases was too small (periods when SARS-CoV-2 incidence was low); thus, ![Graphic][45] was more susceptible to transient variation in the number of diagnosed cases, such as due to sporadic, random PCR testing campaigns. Peaks in ![Graphic][46] were also slightly larger using only symptomatic cases versus all diagnosed cases. ![Figure 2:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/10/07/2021.10.07.21264599/F2.medium.gif) [Figure 2:](http://medrxiv.org/content/early/2021/10/07/2021.10.07.21264599/F2) Figure 2: Sensitivity analyses of estimated ![Graphic][47] using the Robert Koch Institute method. A) Sensitivity analysis using the time series of all diagnosed cases instead of only symptomatic cases in estimating ![Graphic][48]. B) Sensitivity analysis using the time series of hospital admissions in acute-care beds instead of symptomatic cases in estimating ![Graphic][49]. C) Sensitivity analysis using the time series of hospital admissions in ICU-care beds instead of symptomatic cases in estimating ![Graphic][50]. D) Sensitivity analysis using different values for the generation time in estimating ![Graphic][51]. The dashed green line represents the threshold of *R* = 1. The second sensitivity analysis, in which the time series of acute-care hospital admissions was used to proxy the epidemic trend, instead of the time series of symptomatic cases, showed rather strong correlation between ![Graphic][52] estimates (Figure 2B). The Pearson correlation coefficient was 0.512 (p-value<0.001) and the Spearman correlation coefficient was 0.716 (p-value<0.001), both confirming strong agreement. The third sensitivity analysis, in which the time series of ICU-care hospital admissions was used to proxy the epidemic trend, instead of the time series of symptomatic cases, also showed rather strong correlation between the ![Graphic][53] estimates, but also large fluctuations in ![Graphic][54] (Figure 2C). The Pearson correlation coefficient was 0.589 (p-value<0.001) and the Spearman correlation coefficient was 0.550 (p-value<0.001), both confirming strong agreement, but inferior to that for acute-care hospital admissions (Figure 2B versus Figure 2C). The fourth sensitivity analysis, in which different values for the generation time *τ**G* were used, showed also strong agreement between different ![Graphic][55] estimates (Figure 2D). The Pearson correlation coefficient was 0.901 (p-value<0.001) and the Spearman correlation coefficient was 0.900 (p-value<0.001), both confirming excellent agreement. The main differences between the estimates occurred in the timing and magnitude of peaks of the epidemic waves, as expected, since variation in generation time changes the rate of epidemic growth [51]. ![Graphic][56] estimated using the Robert Koch Institute method (Figure 3A), Cislaghi method (Figure 3B), Systrom-Bettencourt and Ribeiro method (Figure 3C), Wallinga and Teunis method (Figure 3D), and Cori et al. method (Figure 3E), all showed similar results and were able to capture the evolution of epidemic waves and transient variations due to national public-health response landmarks and major social events. There were also overall strong correlations between them (Table 1 and Figure 4). However, the Systrom-Bettencourt and Ribeiro method (Figure 3C) tended to provide something of an average ![Graphic][57] and was not as sensitive to transient changes in ![Graphic][58] (Figures 3 and 4). ![Figure 3:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/10/07/2021.10.07.21264599/F3.medium.gif) [Figure 3:](http://medrxiv.org/content/early/2021/10/07/2021.10.07.21264599/F3) Figure 3: Trend in in ![Graphic][59] Qatar, April 1, 2020 to August 18, 2021, using the A) Robert Koch Institute method [22], B) Cislaghi method [34], C) Systrom-Bettencourt and Ribeiro method [12,37-39], D) Wallinga and Teunis method [35], and E) Cori et al. method [31]. The figure includes the 95% uncertainty or credible interval, as applicable for each method. The dashed green line represents the threshold of *R* = 1. View this table: [Table 1.](http://medrxiv.org/content/early/2021/10/07/2021.10.07.21264599/T1) Table 1. Correlations between ![Graphic][60] and ![Graphic][61] using the A) Robert Koch Institute method [22], B) Cislaghi method [34], C) Systrom-Bettencourt and Ribeiro method [12,37-39], D) Wallinga and Teunis method [35], and E) Cori et al. method [31]. ![Figure 4:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/10/07/2021.10.07.21264599/F4.medium.gif) [Figure 4:](http://medrxiv.org/content/early/2021/10/07/2021.10.07.21264599/F4) Figure 4: Pairwise comparison between ![Graphic][62] estimated using the Robert Koch Institute method [22] and that estimated using the A) Cislaghi method [34], B) Systrom-Bettencourt and Ribeiro method [12,37-39], C) Wallinga and Teunis method [35], and D) Cori et al. method [31]. The dashed green line represents the threshold of *R* = 1. There were differences in how rapidly each method detected a change in epidemic dynamics (Figure 4). The Wallinga and Teunis method was the fastest at detecting a change, while the Robert Koch Institute method was the slowest, leading to weaker Pearson and Spearman correlation coefficients between them (Table 1). For instance, the surge in ![Graphic][63] during the first Eid al-Adha after epidemic onset (a festival that occurred between July 30 and August 6, 2020 and is associated with celebrations and social gatherings) was detected on August 1, August 7, August 8, August 11, and August 13 using the Wallinga and Teunis, Cislaghi, Systrom-Bettencourt and Ribeiro, Cori et al., and Robert Koch Institute methods, respectively. Uncertainty intervals around the ![Graphic][64] estimates of the various methods were narrow overall, except when the number of diagnosed symptomatic cases or the number of PCR tests was small, specifically during the low-incidence phases of the epidemic (Figure 3). Overall, the uncertainty in ![Graphic][65] estimates did not impact the interpretation of the ![Graphic][66] results (Figure 3). The only exception was for the Systrom-Bettencourt and Ribeiro method, as it showed rather wide uncertainty intervals compared to the point estimates for ![Graphic][67] (Figure 3C). The ![Graphic][68] correlated strongly with the ![Graphic][69] using different methods (Table 1), and provided somewhat of an average of the ![Graphic][70] (Figure 1). For example, ![Graphic][71] and ![Graphic][72] averaged 1.15 and 1.14 during the first wave, respectively. While it captured the three epidemic waves, it could not capture the transient fluctuations in ![Graphic][73], nor the effects of significant clusters during low-incidence phases. ## DISCUSSION Two forms of *R**t* estimation were used in Qatar to inform the national COVID-19 response, the “empirical” estimation, ![Graphic][74], and the model-based estimation, ![Graphic][75]. Both forms proved useful in real-time tracking of epidemic trends, understanding epidemic dynamics, and setting interventions to control transmission, such as application or easing of public health restrictions. Both forms were integral to the national public health response and to formulating evidence-based policy decisions to minimize the epidemic’s toll on health, society, and the economy throughout the phases of this epidemic. ![Graphic][76] effectively captured the evolution of the epidemic during its three waves, the effects of the response landmarks, such as the partial lockdowns and easing of public health restrictions, and the major social events that affected the social contact rate in the population. Even transient fluctuations in infection transmission that occurred because of significant infection clusters were captured by ![Graphic][77]. Strikingly, the introduction and expansion of the B.1.1.7 variant [21], that resulted in the second epidemic wave, was discovered immediately through ![Graphic][78] monitoring, as there was a sudden large, sustained increase in *R**t* that coincided precisely with a rapidly increasing number of S-gene “target failures” in PCR testing, even before viral genome sequencing was conducted to confirm the presence and expansion of this variant in the population [18]. While ![Graphic][79] provided an average *R**t* that closely tracked the average ![Graphic][80], it did not have the resolution to capture transient changes in ![Graphic][81], other than major changes associated with the three epidemic waves. Still, ![Graphic][82] was useful and influential, as it was, along with the model that generated it [6,23], the basis for forecasting and future planning, such as forecasting the epidemic time-course and epidemic potential, forecasting healthcare needs of acute-care and ICU-care bed hospitalizations, predicting the impact of social and physical distancing restrictions, planning for easing of restrictions, and forecasting the impact of different mass vaccination strategies [6,23]. Therefore, both forms of *R**t* complement each other and should be part of any effective COVID-19 national response. ![Graphic][83] estimation proved robust in sensitivity analyses conducted to assess its utility. Baseline estimation of ![Graphic][84] was based on the time series of symptomatic cases as a proxy of the actual incidence of SARS-CoV-2 infection in the population, which is unknown. Using the time series of all diagnosed cases instead of just symptomatic cases did not appreciably impact ![Graphic][85] estimation, even though PCR testing volume and strategies varied throughout the epidemic. Using the time series of acute-care hospital admissions instead of the time series of symptomatic cases also led to comparable estimates for ![Graphic][86]. This was also the case, but with weaker agreement, when the time series of ICU-care hospital admissions was used to proxy trends in infection incidence. This is not surprising as there is a long delay between onset of infection and ICU-care hospital admission, and the number of ICU-care admissions was relatively small with the low COVID-19 severity in Qatar’s predominantly young and working-age population [5,48]. Variations in the ![Graphic][87] assumed value for the generation time in the estimation did not heavily impact estimates. These findings support the robustness of the approach employed to estimate ![Graphic][88]. Examination of different methods to estimate ![Graphic][89] demonstrated consistency of the results, generally strong correlations between the estimates, and an acceptable level of uncertainty in them. The only exception was the Systrom-Bettencourt and Ribeiro method which tended to provide something of an average ![Graphic][90]. It was not as sensitive to transient changes in ![Graphic][91], and had wide uncertainty intervals compared to the point estimates. There were also differences in how rapidly each method detected a change in epidemic dynamics. The Wallinga and Teunis method was the fastest to detect a change, while the Robert Koch Institute method was the slowest. Yet, overall, these findings support the robustness of using these methods in ![Graphic][92] estimation and to guide COVID-19 national responses. This study has limitations. The estimated ![Graphic][93] and ![Graphic][94] are contingent on the validity and generalizability of input data. The uncertainty/credible intervals estimated here accounted for the uncertainty arising from sampling variation, or from our imperfect knowledge of specific epidemiological quantities, such as the serial interval, but did not account for other sources of uncertainty, such as our imperfect knowledge of the true incidence of infection in the population. To reduce bias due to variation in volume and strategies of PCR testing over time, ![Graphic][95] was calculated using the time series of symptomatic cases, but the distribution of the delay between onset of infection and onset of symptoms may bias these estimates. ![Graphic][96] was estimated using a deterministic compartmental model, but this type of model may not be representative of stochastic transmission dynamics, particularly when the number of infections is small. Despite these limitations, ![Graphic][97] and ![Graphic][98] were able to capture the evolution of the epidemic through its several waves, and to effectively inform the national response and policy decision-making. ## CONCLUSIONS *R**t* estimations played a critical and influential role in the COVID-19 national response in Qatar. Even though surveillance data of SARS-CoV-2 infection are imperfect and prone to bias, *R**t* estimations were robust and generated consistent results regardless of the data source used, or the method employed in generating estimates. These findings affirm the value and complementarity of using both ![Graphic][99] and ![Graphic][100] to track the epidemic in real-time and to inform public health decision making at a national level. This can also be done despite low-resource demands, as *R**t* estimation utilizes existing surveillance data. Moreover, application of some of the estimation methods is feasible even without established expertise in infectious disease modeling. Since the choice of estimation method does not impact the estimates, each country may decide on the best approach, method, and source of data to be used in the estimation, weighing feasibility, ease of use, and functionality, given its specific circumstances. ## Data Availability The dataset of this study is a property of the Qatar Ministry of Public Health that was provided to the researchers through a restricted-access agreement that prevents sharing the dataset with a third party or publicly. Future access to this dataset can be considered through a direct application for data access to Her Excellency the Minister of Public Health ([https://www.moph.gov.qa/english/Pages/default.aspx](https://www.moph.gov.qa/english/Pages/default.aspx)). ## Author contributions RB and HHA constructed, coded, and parameterized the mathematical models, and conducted the analyses. HC conducted statistical analyses. HHA and LJA. conceived and led the design of the study, construct and parameterization of the mathematical models, and wrote the first draft of the article. All authors contributed to discussion and interpretation of the results and to the writing of the manuscript. All authors have read and approved the final manuscript. ## Competing interests Dr. Butt has received institutional grant funding from Gilead Sciences unrelated to the work presented in this paper. Otherwise, we declare no competing interests. ## Supplementary Material ![Figure S1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/10/07/2021.10.07.21264599/F5.medium.gif) [Figure S1.](http://medrxiv.org/content/early/2021/10/07/2021.10.07.21264599/F5) Figure S1. A conceptual diagram illustrating the basic structure of the deterministic mathematical model developed to describe SARS-CoV-2 transmission dynamics in the population of Qatar. The detailed structure of this model and its description are found in [1,2]. In this figure, solid lines denote progression or forward movement from one population compartment to the next, while dashed lines denote backward movement from the present population compartment to the previous one. ## Acknowledgements HHA and RB acknowledge the joint support of Qatar University and Marubeni M-QJRC-2020-5. The authors are grateful for support provided by the Biomedical Research Program and the Biostatistics, Epidemiology, and Biomathematics Research Core, both at Weill Cornell Medicine-Qatar, as well as for support provided by the Ministry of Public Health and Hamad Medical Corporation. The developed mathematical models were made possible by NPRP grant number 9-040-3-008 (Principal investigator: LJA) and NPRP grant number 12S-0216-190094 (Principal investigator: LJA) from the Qatar National Research Fund (a member of Qatar Foundation; [https://www.qnrf.org](https://www.qnrf.org)). The authors are also grateful for the Qatar Genome Programme for institutional support for the reagents needed for the viral genome sequencing. The statements made herein are solely the responsibility of the authors. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. ## Footnotes * * Joint senior authors * Received October 7, 2021. * Revision received October 7, 2021. * Accepted October 7, 2021. * © 2021, Posted by Cold Spring Harbor Laboratory The copyright holder for this pre-print is the author. All rights reserved. The material may not be redistributed, re-used or adapted without the author's permission. ## References 1. COVID-19 Outbreak Live Update. Available from: [https://www.worldometers.info/coronavirus/](https://www.worldometers.info/coronavirus/). Accessed on March 27, 2020. 2020. 2. United Nations. Shared responsibility, global solidarity: Responding to the socio-economic impacts of COVID-19. Available from: [https://www.un.org/sites/un2.un.org/files/sg\_report\_socio-economic\_impact\_of\_covid19.pdf](https://www.un.org/sites/un2.un.org/files/sg\_report\_socio-economic_impact_of_covid19.pdf). Accessed on: April 16, 2020. 2020. 3. Kaplan J., Frias L., McFall-Johnsen M. A third of the global population is on coronavirus lockdown. Available from: [https://www.businessinsider.com.au/countries-on-lockdown-coronavirus-italy-2020-3](https://www.businessinsider.com.au/countries-on-lockdown-coronavirus-italy-2020-3) Accessd on: April 25, 2020. Business Insider Australia. 2020. 4. Gilardino RE. Does “flattening the curve” affect critical care services delivery for COVID-19? A global health perspective. International Journal of Health Policy and Management. 2020;9:503–7. 5. Abu-Raddad LJ, Chemaitelly H, Ayoub HH, Al Kanaani Z, Al Khal A, Al Kuwari E, et al. Characterizing the Qatar advanced-phase SARS-CoV-2 epidemic. Scientific Reports. 2021;11:6233. 6. Ayoub HH, Chemaitelly H, Seedat S, Makhoul M, Al Kanaani Z, Al Khal A, et al. Mathematical modeling of the SARS-CoV-2 epidemic in Qatar and its impact on the national response to COVID-19. J Glob Health. 2021;11:05005. 7. Kretzschmar ME, Rozhnova G, Bootsma MC, van Boven M, van de Wijgert JH, Bonten MJ. Impact of delays on effectiveness of contact tracing strategies for COVID-19: a modelling study. The Lancet Public Health. 2020;5:e452–e9. 8. Hozé N, Paireau J, Lapidus N, Tran Kiem C, Salje H, Severi G, et al. Monitoring the proportion of the population infected by SARS-CoV-2 using age-stratified hospitalisation and serological data: a modelling study. Lancet Public Health. 2021;6:e408–e15. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S2468-2667(21)00064-5&link_type=DOI) 9. Reiner RC, Barber RM, Collins JK, Zheng P, Adolph C, Albright J, et al. Modeling COVID-19 scenarios for the United States. Nature Medicine. 2021;27:94–105. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33097835&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F10%2F07%2F2021.10.07.21264599.atom) 10. Abueg M, Hinch R, Wu N, Liu L, Probert W, Wu A, et al. Modeling the effect of exposure notification and non-pharmaceutical interventions on COVID-19 transmission in Washington state. NPJ digital medicine. 2021;4:1–10. 11. Khailaie S, Mitra T, Bandyopadhyay A, Schips M, Mascheroni P, Vanella P, et al. Development of the reproduction number from coronavirus SARS-CoV-2 case data in Germany and implications for political measures. BMC Medicine. 2021;19:32. 12. Gostic KM, McGough L, Baskerville EB, Abbott S, Joshi K, Tedijanto C, et al. Practical considerations for measuring the effective reproductive number, Rt. PLoS Comput Biol. 2020;16:e1008409. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pcbi.1008409&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33301457&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F10%2F07%2F2021.10.07.21264599.atom) 13. Annunziato A, Asikainen T. Effective Reproduction Number Estimation from Data Series, EUR 30300 EN, Publications Office of the European Union, Luxembourg, 2020, ISBN 978-92-76-20749-8, doi:10.2760/036156, JRC121343. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2760/036156&link_type=DOI) 14. Planning and Statistics Authority-State of Qatar. Qatar Monthly Statistics. Available from: [https://www.psa.gov.qa/en/pages/default.aspx](https://www.psa.gov.qa/en/pages/default.aspx). Accessed on: May 26, 2020. 2020. 15. Abu-Raddad LJ, Chemaitelly H, Butt AA, National Study Group for Covid-19 Vaccination. Effectiveness of the BNT162b2 Covid-19 Vaccine against the B.1.1.7 and B.1.351 Variants. N Engl J Med. 2021. 16. Chemaitelly H, Yassine HM, Benslimane FM, Al Khatib HA, Tang P, Hasan MR, et al. mRNA-1273 COVID-19 vaccine effectiveness against the B.1.1.7 and B.1.351 variants and severe COVID-19 disease in Qatar. Nat Med. 2021. 17. National Project of Surveillance for Variants of Concern and Viral Genome Sequencing. Qatar viral genome sequencing data. Data on randomly collected samples. [https://www.gisaid.org/phylodynamics/global/nextstrain/](https://www.gisaid.org/phylodynamics/global/nextstrain/). 2021. Available: [https://www.gisaid.org/phylodynamics/global/nextstrain/](https://www.gisaid.org/phylodynamics/global/nextstrain/). Accessed. 18. Benslimane FM, Al Khatib HA, Al-Jamal O, Albatesh D, Boughattas S, Ahmed AA, et al. One year of SARS-CoV-2: Genomic characterization of COVID-19 outbreak in Qatar. medRxiv. 2021:2021.05.19.21257433. 19. Hasan MR, Kalikiri MKR, Mirza F, Sundararaju S, Sharma A, Lorenz S, et al. Real-Time SARS-CoV-2 Genotyping by High-Throughput Multiplex PCR Reveals the Epidemiology of the Variants of Concern in Qatar. medRxiv. 2021:2021.07.18.21260718. 20. World Health Organization. Tracking SARS-CoV-2 variants. Available from: [https://www.who.int/en/activities/tracking-SARS-CoV-2-variants/](https://www.who.int/en/activities/tracking-SARS-CoV-2-variants/). Accessed on: June 5, 2021. 2021. 21. Abu-Raddad LJ, Chemaitelly H, Ayoub HH, Coyle P, Malek JA, Ahmed AA, et al. Epidemiologic characterization of introduction and expansion of the SARS-CoV-2 B.1.1.7 variant and its reinfections in a national population. Under review. 2021. 22. Rober Koch Institut. Epidemiologisches Bulletin. 2020. Available at: [https://www.rki.de/DE/Content/Infekt/EpidBull/Archiv/2020/Ausgaben/17\_20.pdf?blob=publicationFile](https://www.rki.de/DE/Content/Infekt/EpidBull/Archiv/2020/Ausgaben/17_20.pdf?blob=publicationFile). Accessed 28 May 2021. 23. Ayoub HH, Chemaitelly H, Makhoul M, Al Kanaani Z, Al Kuwari E, Butt AA, et al. Epidemiological impact of prioritising SARS-CoV-2 vaccination by antibody status: mathematical modelling analyses. BMJ Innovations. 2021:bmjinnov-2021-000677. 24. World Health Organization. COVID-19 clinical management: living guidance. Available from: [https://www.who.int/publications/i/item/WHO-2019-nCoV-clinical-2021-1](https://www.who.int/publications/i/item/WHO-2019-nCoV-clinical-2021-1). Accessed on: May 15, 2021. 2021. 25. World Health Organization. International guidelines for certification and classification (coding) of COVID-19 as cause of death. Available from: [https://www.who.int/classifications/icd/Guidelines\_Cause\_of\_Death\_COVID-19-20200420-EN.pdf?ua=1](https://www.who.int/classifications/icd/Guidelines\_Cause_of_Death_COVID-19-20200420-EN.pdf?ua=1). Document Number: WHO/HQ/DDI/DNA/CAT. Accessed on May 15, 2021. 2020. 26. Vogels C, Fauver J, Grubaugh N. Multiplexed RT-qPCR to screen for SARS-COV-2 B.1.1.7, B.1.351, and P.1 variants of concern V.3. dx.doi.org/10.17504/protocols.io.br9vm966. 2021. Available: [https://www.protocols.io/view/multiplexed-rt-qpcr-to-screen-for-sars-cov-2-b-1-1-br9vm966](https://www.protocols.io/view/multiplexed-rt-qpcr-to-screen-for-sars-cov-2-b-1-1-br9vm966). Accessed. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.17504/protocols.io.br9vm966&link_type=DOI) 27. European Centre for Disease Prevention and Control. Rapid increase of a SARS-CoV-2 variant with multiple spike protein mutations observed in the United Kingdom. Available from: [https://www.ecdc.europa.eu/sites/default/files/documents/SARS-CoV-2-variant-multiple-spike-protein-mutations-United-Kingdom.pdf](https://www.ecdc.europa.eu/sites/default/files/documents/SARS-CoV-2-variant-multiple-spike-protein-mutations-United-Kingdom.pdf). Accessed on: February 10, 2021. 2020. Accessed. 28. Galloway SE, Paul P, MacCannell DR, Johansson MA, Brooks JT, MacNeil A, et al. Emergence of SARS-CoV-2 B.1.1.7 Lineage - United States, December 29, 2020-January 12, 2021. MMWR Morb Mortal Wkly Rep. 2021;70:95–9. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.15585/mmwr.mm7003e2&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33476315&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F10%2F07%2F2021.10.07.21264599.atom) 29. Challen R, Brooks-Pollock E, Read JM, Dyson L, Tsaneva-Atanasova K, Danon L. Risk of mortality in patients infected with SARS-CoV-2 variant of concern 202012/1: matched cohort study. BMJ. 2021;372:n579. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNzIvbWFyMDlfMTAvbjU3OSI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIxLzEwLzA3LzIwMjEuMTAuMDcuMjEyNjQ1OTkuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 30. Thermo Fisher Scientific. TaqPath™ COVID-19 CE-IVD RT-PCR Kit instructions for use. Available from: [https://assets.thermofisher.com/TFS-Assets/LSG/manuals/MAN0019215\_TaqPathCOVID-19\_CE-IVD\_RT-PCR%20Kit\_IFU.pdf](https://assets.thermofisher.com/TFS-Assets/LSG/manuals/MAN0019215\_TaqPathCOVID-19_CE-IVD_RT-PCR%20Kit_IFU.pdf). Accessed on December 02, 2020. 2020. 31. Cori A, Ferguson NM, Fraser C, Cauchemez S. A New Framework and Software to Estimate Time-Varying Reproduction Numbers During Epidemics. 2013;178:1505–12. 32. Deng Y, You C, Liu Y, Qin J, Zhou X-H. Estimation of incubation period and generation time based on observed length-biased epidemic cohort with censoring for COVID-19 outbreak in China. Biometrics. 2020;n/a. 33. Lehtinen S, Ashcroft P, Bonhoeffer S. On the relationship between serial interval, infectiousness profile and generation time. J R Soc Interface. 2021;18:20200756. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1098/rsif.2020.0756&link_type=DOI) 34. Cislaghi C. Un cruscotto per monitorare l’evoluzione dell’epidemia. Scienzainrete. 2020 April 9, 2020. 35. Wallinga J, Teunis P. Different Epidemic Curves for Severe Acute Respiratory Syndrome Reveal Similar Impacts of Control Measures. American Journal of Epidemiology. 2004;160:509–16. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/aje/kwh255&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=15353409&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F10%2F07%2F2021.10.07.21264599.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000223938000001&link_type=ISI) 36. Rai B, Shukla A, Dwivedi LK. Estimates of serial interval for COVID-19: A systematic review and meta-analysis. Clinical Epidemiology and Global Health. 2021;9:157–61. 37. Bettencourt LMA, Ribeiro RM. Real Time Bayesian Estimation of the Epidemic Potential of Emerging Infectious Diseases. PLoS ONE. 2008;3(5). 38. Systrom K. The Metric We Need to Manage COVID-19. Systrom - Data, Machine Learning and Technology. 2020 April 12, 2020. 39. Systrom K. Estimating COVID-19’s Rt in Real-Time. Realtime R0. United States of America: Github; 2020. 40. Makhoul M., Ayoub H.H., Chemaitelly H., Seedat S., Mumtaz G.R., Al-Omari S., et al. Epidemiological impact of SARS-CoV-2 vaccination: Mathematical modeling analyses. Vaccines. 2020;8. 41. Ayoub HH, Chemaitelly H, Mumtaz GR, Seedat S, Awad SF, Makhoul M, et al. Characterizing key attributes of the epidemiology of COVID-19 in China: Model-based estimations. Global Epidemiology. 2020;100042. 42. Ayoub HH, Chemaitelly H, Seedat S, Mumtaz GR, Makhoul M, Abu-Raddad LJ. Age could be driving variable SARS-CoV-2 epidemic trajectories worldwide. Plos One. 2020;15. 43. Al-Thani MH, Farag E, Bertollini R, Al Romaihi HE, Abdeen S, Abdelkarim A, et al. SARS-CoV-2 infection is at herd immunity in the majority segment of the population of Qatar. Open Forum Infectious Diseases. 2021. 44. Coyle PV, Chemaitelly H, Kacem M, Al Molawi NHA, El Kahlout RA, Gilliani I, et al. SARS-CoV-2 seroprevalence in the urban population of Qatar: An analysis of antibody testing on a sample of 112,941 individuals. iScience. 2021:102646. 45. Abu-Raddad LJ, Chemaitelly H, Malek JA, Ahmed AA, Mohamoud YA, Younuskunju S, et al. Assessment of the risk of SARS-CoV-2 reinfection in an intense re-exposure setting. Clin Infect Dis. 2020. 46. Abu-Raddad LJ, Chemaitelly H, Coyle P, Malek JA, Ahmed AA, Mohamoud YA, et al. SARS-CoV-2 antibody-positivity protects against reinfection for at least seven months with 95% efficacy. EClinicalMedicine. 2021;35:100861. 47. Jeremijenko A, Chemaitelly H, Ayoub HH, Alishaq M, Abou-Samra AB, Al Ajmi J, et al. Herd Immunity against Severe Acute Respiratory Syndrome Coronavirus 2 Infection in 10 Communities, Qatar. Emerg Infect Dis. 2021;27:1343–52. 48. Seedat S, Chemaitelly H, Ayoub H, Makhoul M, Mumtaz GR, Kanaani ZA, et al. SARS-CoV-2 infection hospitalization, severity, criticality, and fatality rates in Qatar. In Press. Scientific Reports. 2020. 49. Planning and Statistics Authority-State of Qatar. The Simplified Census of Population, Housing & Establishments. Available from: [https://www.psa.gov.qa/en/statistics/Statistical%20Releases/Population/Population/2018/Population\_social\_1\_2018\_AE.pdf](https://www.psa.gov.qa/en/statistics/Statistical%20Releases/Population/Population/2018/Population\_social_1_2018_AE.pdf) Accessed on: April 2, 2020. 2019. 50. MATLAB®. The Language of Technical Computing. The MathWorks, Inc. 2019. 51. Anderson RM, May RM. Infectious diseases of humans: dynamics and control: Oxford university press; 1992. ## References 1. Ayoub HH, Chemaitelly H, Makhoul M, Al Kanaani Z, Al Kuwari E, Butt AA, et al. Epidemiological impact of prioritising SARS-CoV-2 vaccination by antibody status: mathematical modelling analyses.BMJ Innovations. 2021:bmjinnov-2021-000677. 2. Ayoub HH, Chemaitelly H, Seedat S, Makhoul M, Al Kanaani Z, Al Khal A, et al. Mathematical modeling of the SARS-CoV-2 epidemic in Qatar and its impact on the national response to COVID-19. J Glob Health. 2021;11:05005. [1]: /embed/inline-graphic-1.gif [2]: /embed/inline-graphic-2.gif [3]: /embed/inline-graphic-3.gif [4]: /embed/inline-graphic-4.gif [5]: /embed/inline-graphic-5.gif [6]: /embed/inline-graphic-6.gif [7]: /embed/inline-graphic-7.gif [8]: /embed/inline-graphic-8.gif [9]: /embed/inline-graphic-9.gif [10]: /embed/inline-graphic-10.gif [11]: /embed/inline-graphic-11.gif [12]: /embed/inline-graphic-12.gif [13]: /embed/inline-graphic-13.gif [14]: /embed/inline-graphic-14.gif [15]: /embed/inline-graphic-15.gif [16]: /embed/inline-graphic-16.gif [17]: /embed/inline-graphic-17.gif [18]: /embed/inline-graphic-18.gif [19]: /embed/inline-graphic-19.gif [20]: /embed/inline-graphic-20.gif [21]: /embed/inline-graphic-21.gif [22]: /embed/inline-graphic-22.gif [23]: /embed/inline-graphic-23.gif [24]: /embed/inline-graphic-24.gif [25]: /embed/inline-graphic-25.gif [26]: /embed/inline-graphic-26.gif [27]: /embed/inline-graphic-27.gif [28]: /embed/inline-graphic-28.gif [29]: /embed/inline-graphic-29.gif [30]: /embed/inline-graphic-30.gif [31]: /embed/inline-graphic-31.gif [32]: /embed/inline-graphic-32.gif [33]: /embed/inline-graphic-33.gif [34]: F1/embed/inline-graphic-34.gif [35]: F1/embed/inline-graphic-35.gif [36]: F1/embed/inline-graphic-36.gif [37]: F1/embed/inline-graphic-37.gif [38]: F1/embed/inline-graphic-38.gif [39]: F1/embed/inline-graphic-39.gif [40]: /embed/inline-graphic-40.gif [41]: /embed/inline-graphic-41.gif [42]: /embed/inline-graphic-42.gif [43]: /embed/inline-graphic-43.gif [44]: /embed/inline-graphic-44.gif [45]: /embed/inline-graphic-45.gif [46]: /embed/inline-graphic-46.gif [47]: F2/embed/inline-graphic-47.gif [48]: F2/embed/inline-graphic-48.gif [49]: F2/embed/inline-graphic-49.gif [50]: F2/embed/inline-graphic-50.gif [51]: F2/embed/inline-graphic-51.gif [52]: /embed/inline-graphic-52.gif [53]: /embed/inline-graphic-53.gif [54]: /embed/inline-graphic-54.gif [55]: /embed/inline-graphic-55.gif [56]: /embed/inline-graphic-56.gif [57]: /embed/inline-graphic-57.gif [58]: /embed/inline-graphic-58.gif [59]: F3/embed/inline-graphic-59.gif [60]: T1/embed/inline-graphic-60.gif [61]: T1/embed/inline-graphic-61.gif [62]: F4/embed/inline-graphic-62.gif [63]: /embed/inline-graphic-63.gif [64]: /embed/inline-graphic-64.gif [65]: /embed/inline-graphic-65.gif [66]: /embed/inline-graphic-66.gif [67]: /embed/inline-graphic-67.gif [68]: /embed/inline-graphic-68.gif [69]: /embed/inline-graphic-69.gif [70]: /embed/inline-graphic-70.gif [71]: /embed/inline-graphic-71.gif [72]: /embed/inline-graphic-72.gif [73]: /embed/inline-graphic-73.gif [74]: /embed/inline-graphic-74.gif [75]: /embed/inline-graphic-75.gif [76]: /embed/inline-graphic-76.gif [77]: /embed/inline-graphic-77.gif [78]: /embed/inline-graphic-78.gif [79]: /embed/inline-graphic-79.gif [80]: /embed/inline-graphic-80.gif [81]: /embed/inline-graphic-81.gif [82]: /embed/inline-graphic-82.gif [83]: /embed/inline-graphic-83.gif [84]: /embed/inline-graphic-84.gif [85]: /embed/inline-graphic-85.gif [86]: /embed/inline-graphic-86.gif [87]: /embed/inline-graphic-87.gif [88]: /embed/inline-graphic-88.gif [89]: /embed/inline-graphic-89.gif [90]: /embed/inline-graphic-90.gif [91]: /embed/inline-graphic-91.gif [92]: /embed/inline-graphic-92.gif [93]: /embed/inline-graphic-93.gif [94]: /embed/inline-graphic-94.gif [95]: /embed/inline-graphic-95.gif [96]: /embed/inline-graphic-96.gif [97]: /embed/inline-graphic-97.gif [98]: /embed/inline-graphic-98.gif [99]: /embed/inline-graphic-99.gif [100]: /embed/inline-graphic-100.gif