Assessing Direct and Spillover Effects of Intervention Packages in Network-Randomized Studies ============================================================================================= * Ashley L. Buchanan * Raul Ulises Hernández-Ramírez * Judith Lok * Sten H. Vermund * Samuel R. Friedman * Donna Spiegelman ## ABSTRACT Intervention packages may result in a greater public health impact than single interventions. Understanding the separate impact of each component in the overall package effectiveness can improve intervention delivery. We propose an approach to evaluate the effects of a time-varying intervention package in a single study. In some network-randomized studies, only a subset of participants in exposed networks receive the intervention themselves. The spillover effect contrasts potential outcomes of persons not exposed themselves under intervention in the network versus no intervention in a control network. We estimated effects of components of the intervention package in HIV Prevention Trials Network 037, a Phase III network-randomized HIV prevention trial among people who inject drugs and their risk networks using Marginal Structural Models to adjust for time-varying confounding. The index participant in an intervention network received a peer education intervention initially at baseline, then boosters at 6 and 12 months. All participants were followed to ascertain HIV risk behaviors. There were 560 participants with at least one follow-up visit, 48% of whom randomized to the intervention, and 1,598 participant-visits were observed. The spillover effect of the boosters in the presence of initial peer education training was a 39% rate reduction (95% confidence interval (CI) = -57%, -13%). These methods will be useful for evaluation of intervention packages in studies with network features. KEY WORDS * Causal inference * Cluster-randomized trials * Spillover/Indirect effects * HIV/AIDS * Implementation Science * Interference * Package Interventions * Substance use ## Introduction Design and scale-up of intervention packages that effectively meet the needs of a target population offers a means for control of disease epidemics, such as HIV/AIDS [1–3]. Intervention packages are defined herein as a set of individual interventions (i.e., components) to prevent or treat a disease or condition through multiple pathways simultaneously. There are many interventions that confer partial protection against HIV transmission [1, 4] and offering these interventions in combination (e.g., HIV testing, treatment as prevention (TasP)) is an important strategy for curtailing the HIV epidemic. Several cluster-randomized trials of packages of HIV treatment and prevention interventions were conducted in Sub-Saharan Africa and demonstrated a range of effectiveness for TasP in combination with other known HIV prevention measures [5–12]. Further analysis of these studies is needed to better understand the effectiveness of each package component. Intervention packages are not exclusive to community-level HIV epidemic control. In substance use treatment and prevention, intervention packages are often tailored to specific subpopulations, such as people who inject drugs (PWID), in an effort to achieve a larger and more sustainable intervention impact in hard-to-reach populations [13–15]. In studies of intervention packages, treatment assignment may be randomly assigned to individuals and/or networks (i.e., social groupings, communities), or treatments can be self-selected by individuals within a network or other clustering feature. In a cluster, we assume that each individual’s exposure could influence any other cluster member’s outcome but no one’s outside the cluster, known as *partial interference* [16, 17]. In a network, information may be collected on each of the unique connections between individuals and assumptions about spillover (dissemination or interference) are based on the presence or absence of connections, known as *neighbor interference* [18, 19]. Previous work related to intervention packages illustrated causal inference approaches to account for implementation factors that were not randomized, such as enrollment in a program [20, 21], but did not consider possible spillover of the package components. Howe et al. [22] and Hernán et al. [23] developed and applied methodology to evaluate the joint causal effects of two non-randomized exposures; that is, their *interaction* in the presence of time-varying confounding in an observational study, employing joint Marginal Structural Models, but also did not consider spillover effects. He et al. [24] developed Marginal Structural Models for studies of a single intervention with spillover and fit the models using cluster-level propensity scores. Marginal Structural Models are a class of causal models that typically model the marginal mean of the counterfactual outcome and the parameters in the model correspond to average causal effects [25]. We developed methods for disentangling the effects of time-varying components of intervention packages in studies where spillover may be present using a Marginal Structural Model. We consider the setting of an HIV prevention network-randomized trial in which only one member of each network was exposed to the intervention package by the study investigators. The exposed members are here-after denoted as “index participants”. First, these individuals came forward to be the index member, then centered around each index, investigators ascertained their immediate HIV risk contacts, defined as all individuals sharing HIV risk behavior with this index (e.g., injection or sexual risk behavior). This is called an *egocentric network*. Then, the networks were randomized to intervention or control condition. For example, the indexes were randomly assigned to be a peer educators or not. After the initial training (and prior to the first outcome ascertainment), the index participants in intervention networks then educate their network members about HIV risk reduction behavior, further disseminating to their network members. This study design is frequently utilized in HIV prevention research among people who use drugs, as these populations can be challenging to reach [26–29]. The effects of both the complete intervention package and its individual components can be evaluated in a single study through application of appropriate statistical methods [2]. In network-randomized studies, there is randomization only at the network level. This does not preclude estimation of causal effects of self-selected exposures to individual package components among index participants and their network members due to non-adherence or ineligibility. Furthermore, an evaluation of the spillover of the intervention to other unexposed participants (i.e., network members) in intervention networks is also possible. Causal inference methods for observational studies can be employed to analyze data arising from a network-randomized trial setting lacking randomization at the individual level [30], mimicking a two-stage randomized trial. In this design, the intervention is delivered, the networks are randomized to an intervention allocation strategy, then individuals are randomized to the intervention according to the strategy assigned to their network [17, 31]. For example, in a vaccine trial, clusters (e.g., villages) are assigned first to an allocation strategy for vaccine assignment (e.g., 70% versus 30% randomized to vaccine), then based on the allocation strategy in the cluster, individuals are randomly assigned to vaccine or not [32]. Each component of the intervention package is considered a time-varying non-randomized exposure possibly subject to time-varying confounding. Effects can be estimated using inverse probability weights to fit Marginal Structural Models [30, 33]. This model can also be used to estimate spillover effects when an intervention package is implemented in HIV risk networks in which only one network member is exposed [34]. For example, immediate ART initiation may benefit the health outcomes of the index person in a household who is treated, for example, by improving that person’s immune system and preventing tuberculosis, and it may also benefit the health outcomes of their untreated family members by preventing HIV acquisition and tuberculosis, via contact with the exposed index person, known as a *spillover* (indirect or disseminated) effect [18, 32]. We propose estimators of direct and spillover package component effects. The *direct* effect compares the risk of the outcome when a participant is an index versus a network member under network exposure to a package component and evaluates the effect of being an index beyond being in a network with exposure to the package component. Illustrating with data from a network-randomized trial in the HIV Prevention Trials Network (HPTN), we employ inverse probability weighted (IPW) linear mixed effects models to estimate these effects adjusting for time-varying confounding. ## Methods ### Notation and Assumptions To guide our causal inference approach, the ideal experiment in this setting is conceptualized as a two-stage randomized design, where first networks are randomized to intervention or control, then individuals in each intervention network are randomized to be an index or not [35]. In egocentric networks, a single index would need to be randomly selected in each network. Typically, in the egocentric network-randomized designs, index assignment is not randomized, but rather self-selected by those individuals. Furthermore, exposure to the package component in a network was determined by both the randomization scheme and visit attendance by index participants. A network was exposed if its index member attended a visit when a package component was administered. Therefore, estimation of the effects of an intervention package component does not benefit from randomization and adjustment for confounding is necessary to identify causal effects. Let uppercase letters denote random variables and lowercase letters denote realizations of those random variables. Let *K* be the total number of networks (*k* = 1, …, *K*) and *i* = 1, …, *n**k* denote participants in network *k*. Let ∑*k* *n**k* = *N* be the total number of participants in the study, *j* = 0, …, *m**ki* denote the study visit for participant *i* in network *k*, where visit 0 corresponds to the baseline visit, *J**k* = ∑*i* *m**ki* denote the total number of visits across all participants in each network *k, M**k* = max*i* *m**ki*. Define *Y**kij* as the observed outcome for participant *i* in network *k* at visit *j*. Assume there is correlation between outcomes in a network and spillover can occur only within a network, but not between networks. Let *X**k* be the random intervention package assignment for network *k* and define an indicator *R**ki* for whether or not participant *i* in network *k* was an index participant. Let *A**hkj* denote exposure to the *h**th* intervention component for network *k* at visit *j*, with *H* intervention components, *h* = 1, …, *H*. Let **Ā***hkj* = (*A**hk*1, …, *A**Hkj*)*T* be a vector of size *j ×* 1 denoting the exposure histories for component *h* up to and including visit *j* for network *k*. Let **A***kj* = (*A*1*kj*, …, *A**Hkj*)*T* denote the vector of size *H ×* 1 denoting all package component exposures in network *k* at time *j* and let the history of the intervention package component exposures be denoted by the matrix ![Formula][1] Let **Z***ki* denote a *p ×* 1 vector of baseline covariates for participant *i* in network *k* and **Z***kij* denote a *q ×* 1 vector of time-varying covariates for participant *i* in network *k* at time *j*. Let ![Graphic][2] denote an individual’s covariate history up to and including visit *j*. We assume that there is only one index per network in this study design and let *i**k* denote the unique index member in each network *k*. Let ![Graphic][3] be a vector of size *pn**k* *×* 1 baseline covariates in network *k* and ![Graphic][4] be a vector of size *qn**k**j ×* 1 denoting covariate histories of all participants in network *k* up to and including time *j*. For ease of notation, let ![Graphic][5] be the vector of size *J**k* *×* 1 denoting outcomes across all participant-visits within network *k*. Assume **Y***k* *⊥* **Y***k**′*are independent for *k* ≠ *k**′*. Let ![Graphic][6] be a vector of size *n**k* *×* 1 denoting the index membership in network *k*. Let **U***ki* be a vector of size *s ×* 1 of unmeasured baseline covariates for participant *i* in network *k*. Let **U***k* denote a vector of the unmeasured baseline covariates of all participants in network *k*. Furthermore, at the network level, let ![Graphic][7] be a network-level aggregate function of the covariates, such as the mean of the history for each covariate, in network *k* up to and including time *j*. Let ![Graphic][8] denote the index member covariates at baseline and ![Graphic][9] denote the index member covariates in network *k* up to and including time *j*. There are *n**k* possible configurations of index status in an egocentric network. In this setting, each participant has potential outcomes **Y***ki,j*+1(***r, ā****j*), which correspond to the 2*H×j* *× n**k* vector of potential outcomes for participant *i* in network *k* at time *j* under the index status dicator vector **R***k* = ***r*** and package component history ***Ā****kj* = ***ā****j*. Without further refinement of the research questions of interest, it is not clear which potential outcomes to compare; therefore, additional identification assumptions are required. The sufficient conditions for valid estimation of causal effects have previously been described [36], including exchangeability, consistency and positivity. We assume partial interference, which allows for spillover between any members of an egocentric network, but not between networks. We also assume no contamination across study intervention arms [18]. For the time-varying package components **A***kj*, exchangeability may not hold because the network-level exposure is determined by both the randomization scheme and index visit attendance. However, we assume that exchangeability holds conditional on network-level time-varying covariates. At the individual level conditional on **Z***ki* and **U***ki*, we assume conditional *index member exchangeability*; that is, *Y**ki,j*+1(***r, ā****j*) *⊥* **R***ki*|**Z***ki*, **U***ki* [34]. For the time-varying package components in network *k*, we assume ![Graphic][10] for *j* = 0, …, *m**ki*. That is, we assume that exchangeability of potential outcomes corresponding to the package component exposures holds conditional on each network’s package component history and a network-level aggregate of covariate history, as well as its index member’s covariate history up to the prior visit *j*. The covariates required for exchangeability are ascertained from previous visits because the exposure is defined by index visit attendance, which can only be modeled across all indexes using information from previous visits. In this longitudinal analysis, we also assume that data from missed visits and drop out due to loss to follow-up are missing at random assuming the likelihood was correctly specified [37]. #### Estimands Let ***ā****j* denote a possible history of package components from baseline up to and including visit *j*. For example, the strategy “intended study intervention” is represented by ***ā****j* = (1, 1, 1, 0, …, 0) and the strategy “never exposed” is represented by ![Graphic][11]. Due to the assumptions above and only one index per network, the potential outcomes of interest are *Y**ki,j*+1(*r*, ***ā****j*). Assuming that the effects of interest do not depend on study visit, we define effects averaged over study visits after baseline. The *direct* (or individual) package effect is defined as the contrast in average potential outcomes when a participant is an index compared to when a participant is a network member under network exposure to the component history ***ā****j*; that is, on the ratio scale, *RR**D*(***ā****j*) = *E*[*Y**ki,j*+1(*r* = 1, ***ā****j*)]*/E*[*Y**ki,j*+1(*r* = 0, ***ā****j*)]. The *spillover* package effect compares average the potential outcomes of network members under network component history ***ā****j* versus no exposure to the component history ![Graphic][12] ; that is,![Graphic][13]. The *composite* package effect is ![Graphic][14]; that is, a comparison of the average potential outcomes of index members under network component history exposure ***ā****j* versus network members under no exposure to the component history ![Graphic][15]. The *overall* package effect compares average potential outcomes under network exposure to the component history ***ā****j* versus no network exposure, denoted as ![Graphic][16] [34]. Analogous effects can be defined on the risk difference scale and we note that other definitions of these effects are possible (e.g., spillover effect under initial exposure and boosters versus initial exposure only). ### Estimation and Inference We estimate the effect of each package component in a single model, while considering the presence of the remaining package components [34]. This outcome model is a generalized linear mixed effects model adjusted for individual-level confounding at baseline and inverse probability weighted to adjust for network level time-varying confounding. In Appendix A, we describe the models for the weights used to control for time-varying confounding in the Marginal Structural Model. Assuming that the effects of the covariates are not modified by index status (i.e., there are no ***ā****j* by **Z***ki* interactions) and the generalized linear model with the log link fits the data, the Marginal Structural Model is ![Formula][17] Recall that **U***ki* is a vector of unmeasured baseline covariates that may affect *R**ki* within levels of the baseline covariates **Z***ki*. Thus, even after conditioning on **Z***ki*, our estimators that involve *R**ki* may require further adjustment for unmeasured confounding **U***ki*. To adjust for differences between index and non-index members by accounting for unmeasured covariates **U***ki*, we consider the term ![Graphic][18] for the definition of these estimators, assuming (1) holds. If conditional exchangeability does not hold, ![Graphic][19] may be different from the null. Extending results in Buchanan et. al [34], suppressing the notation for the covariates for ease of notation, estimators for each of the parameters are ![Formula][20] On the relative risk scale, exp(*β*2***ā****j*) can be interpreted as the causal spillover effect of exposure to intervention package component history exposure ***ā****j* versus exposure to the history ![Graphic][21] representing no intervention package component from baseline up to and including visit *j*. In Appendix B, we consider a model for quantifying the effects of multiple package components that includes pairwise interactions between components in a single model. We used a robust empirical sandwich estimator of the variance from the generalized linear mixed model to construct 95% Wald-type confidence intervals. ### Illustrative Example #### HIV Prevention Trials Network Protocol 037 The HPTN Protocol 037 was a Phase III, network-randomized controlled HIV prevention trial with 696 participants who were PWID and their HIV risk networks in Philadelphia, PA, US [13]. A site in Chiang Mai, Thailand had too few PWID who HIV-seroconverted to be included in our analysis. This study evaluated the efficacy of a network-oriented peer education intervention package to promote HIV risk reduction among PWID and their HIV risk network. Index participants whose network was randomized to the intervention received an intervention package that consisted of an educational intervention at baseline and educational boosters at six and 12 months. Participants in both the intervention and control conditions received HIV counseling and testing at each study visit [13]. Table 1 presents the intervention package evaluated in HPTN 037. We are interested in the effect of the additional boosters among those who received the initial baseline intervention, as well as the direct and spillover effects of the intervention package components. Exposure to the booster was defined as a time-updated variable lagged by one visit accounting for the most recent exposure only. In this study, the initial and booster components were only received by the intervention network. View this table: [Table 1:](http://medrxiv.org/content/early/2022/03/28/2022.03.24.22272909/T1) Table 1: Interventions in HPTN 037 by study arm and network membership Direct and spillover effects of the package components were evaluated through comparisons of the incidence of any injection-related risk behavior and included report of any of the following: sharing injection equipment (needles, cookers, cotton, and rinse water), front and back loading (i.e., injecting drugs from one syringe to another), and injecting with people not well known or in shooting gallery in the past month. Following the original analysis of this study, these outcomes were assessed among participants who reported injection drug use in the six months prior to baseline. The longitudinal data were used to assess the effects of the intervention on the inter-visit incidence rates of any injection-related risk behavior using a multilevel generalized linear mixed effects model. These outcome models were also adjusted for baseline individual-level covariates that were known or suspected risk factors for the outcome, including race, ethnicity, report of any injection risk behavior, injected daily in the last month, alcohol use, and history of injection-related risk behavior. Time-varying confounding was adjusted for using stabilized inverse probability weights (IPW) (see Appendix A). The IPW generalized linear mixed effects model with a log link and binomial distribution was fit by weighting individual participants according to their estimated stabilized weights. In intervention networks only, the weight models were estimated in the network-level data with one record per network per visit, pooling across the visits where the network-level exposure status could change (e.g., 12 to 18 month visits). Ideally, we would include all covariates that were known or suspected risk factors; however, this resulted in overparameterization of the model given the size of the study, so we employed a variable selection procedure (see Appendix C). The denominator weight models included selected time-varying network-level aggregate covariates and index member covariates that were known or suspected risk factors for the outcome, selected pairwise interactions, which allowed for more flexibility in the model specification, and the package component exposure at the previous visit. The numerator weight model included the the package component exposure at the previous visit and selected baseline network-level covariates and index member covariates that were known or suspected risk factors for the outcome, which were also included in the outcome model. Both the models for numerator and denominator of the weights included a variable for time specified as study visit month. For some of the models, the log Binomial models did not converge and log Poisson models, which provide consistent but not fully efficient estimates of the relative risk, were used [38, 39]. In examination of the distribution of the weights, there was no evidence of model mis-specification or positivity violations because the mean of the stabilized weights distribution was approximately one (Table A1). All statistical tests performed were two sided and conducted at the significance level. The data analysis for this paper was generated using SAS software (Version 9.4) and we provide example SAS code in Appendix D. ## Results There were 696 participants, 651 reported injection drug use at baseline, and 560 participants had at least one follow-up visit with a total of 1,598 person visits. The size of the networks ranged from two to seven participants and network size was not associated with the outcome (P value = 0.30). Among the 560 participants, 270 (48%) were in intervention networks. Of the 232 indexes who reported injection drug behavior at baseline, 112 (48%) index participants received the initial peer education intervention. Table 2 displays the number of participants who received initial peer education and each of the booster sessions; 70 (44%) index participants received the 6-month booster and 59 (47%) index participants received the 12-month booster. View this table: [Table 2:](http://medrxiv.org/content/early/2022/03/28/2022.03.24.22272909/T2) Table 2: Package components received by index and network members during follow-up among HPTN 037 participants who reported injecting drugs at baseline and had at least one follow-up visit after baseline (n = 560) Table 3 displays the estimated direct, spillover, overall and composite rate ratios from several models with different approaches for covariate adjustment. Regardless of the adjustment approach, the estimated spillover effect for the initial peer education intervention was comparable. In the IPW models, there was an estimated 13% reduction in the spillover rate of report of any injection risk behavior (95% confidence interval (CI) = -31%, 10%). After adjustment with the IP weights, the estimated spillover rate reduction for the boosters was a 39% reduction (95% CI = -57%, -13%). We expect a rate of reporting any injection risk behavior 39% lower in network members within the intervention networks with exposure to at least one booster in addition to the initial training compared to within the control networks. The estimated direct effect for the initial peer education intervention was protective, but not significantly different from the null, regardless of the covariate adjustment approach; however, the estimated direct effect of the booster in addition to the initial intervention was null. In the IPW models, the estimated initial and booster composite effects were protective with a 24% (95% CI = -44%, 2%) and 37% rate reduction (95% CI = -57%, -8%), respectively. The estimated overall effect of the initial and booster were protective with a 17% (95% CI = -31%, 0%) and 38% (95% CI = -53%, -18%) rate reduction, respectively. We expect a 17% reduction in the overall rate of reporting injection risk behavior if the networks are exposed to the initial intervention compared to if there is no initial exposure. Furthermore, we expect a 38% decrease in the overall rate of reporting injection risk behavior if the intervention networks are exposed to at least one booster in addition to the initial training compared to if the networks are under the control condition. Based on the parameterization of the Marginal Structural Model, the rate ratios could be interpreted as the estimates from a trial in which participants are randomized to the booster at 6 months, 12 months, or no booster at either visit [40]. View this table: [Table 3:](http://medrxiv.org/content/early/2022/03/28/2022.03.24.22272909/T3) Table 3: Rate ratios (RR) for the direct, spillover, composite and overall effects of the HPTN 037 peer education package components on reducing report of any injection risk behavior (per person-visit) during follow-up with 95% confidence intervals (CI) among participants with at least one follow-up visit1 ## Discussion We proposed an approach to quantify direct and spillover effects of intervention packages for network-randomized trials. This setting could also arise in a network-randomized study with noncompliance or treatment ineligibility for some network members. We developed causal inference methods to evaluate time-varying components of intervention packages in studies where spillover may be present. The estimation of these effects provides information about the impact of individual package components and their influence on network member outcomes due to being in a network with an exposed index participant. These methods provide a more in-depth understanding of the effects of package components for both exposed index participants and those sharing networks with exposed participants. In HPTN 037, the estimated overall effect was stronger for the booster with initial intervention, as compared to the initial intervention alone, highlighting the importance of continued training for peer educators in this context. Interestingly, a protective effect was observed for network members for both the initial and booster interventions, without a corresponding benefit for those trained to be peer educators themselves. Because index members injected drugs more regularly at baseline than their network members, the index members were riskier actors and may have been less likely to respond to the intervention as compared to network members [13]. We found evidence that booster sessions strengthen the spillover of the intervention, which can be utilized when developing peer-led interventions [41]. This work is particularly timely as there are several HIV combination prevention cluster-randomized trials where interest remains to evaluate the effects of intervention packages, as well as the spillover effects within communities, to better understand the effects of TasP [3, 42]. Understanding the components that may be driving the observed effects of the intervention package could inform modifications to the existing public health strategies in these settings, by strengthening highly effective components and redeveloping those found to be partially effective or ineffective. Four large cluster-randomized HIV prevention trials in Sub-Saharan Africa have recently published their study findings [5–7, 9, 11, 12]. Evaluation of the universal “test and treat” interventions could include assessment of the intervention’s impact on the health outcomes among those in the communities assigned to immediate ART scale-up, but who did not receive immediate ART themselves. As in vaccine campaign design, this could inform the level of ART coverage needed to benefit the community and consistently achieve such targets as the UNAIDS 2025 [43]. The assumption of no unmeasured covariates associated with the network-level exposure and outcome or with the index status and outcome is untestable. Future work could develop methods to assess the sensitivity of these methods to unmeasured confounders [44]. The weight and outcome models were assumed to be correct (e.g., correct functional forms of the covariate). This is not guaranteed in practice and sensitivity analyses could be performed to evaluate the robustness of results to model specification. We also assumed that any data from missing visits were ignorable with respect to valid estimation of intervention effects. These models could be extended to include censoring weights to adjust for possibly informative loss to follow-up due to drop out [45]. In HPTN 037, the ascertainment of personal networks of index participants is likely only a partial ascertainment of each person’s network, which limits evaluation of spillover within the networks of recruited individuals. Our methods will be a useful tool to allow for evaluation of randomized and non-randomized intervention packages in a single study with network features. ## Supporting information Supplemental File [[supplements/272909_file02.pdf]](pending:yes) ## Data Availability Example SAS code used to fit the models presented in this paper can be found in the Supplement Appendix D. The HPTN 037 study data sets are publicly available and can be requested from the Statistical Center for HIV/AIDS Research and Prevention through the ATLAS Science Portal through [https://atlas.scharp.org/cpas/project/HPTN/](https://atlas.scharp.org/cpas/project/HPTN/). [https://atlas.scharp.org/cpas/project/HPTN/](https://atlas.scharp.org/cpas/project/HPTN/) ## Acknowledgements We thank Drs. Deborah Donnell and Carl Latkin for access to the HIV Prevention Trials Network (HPTN) 037 data and their valuable suggestions. Data from the HPTN were obtained with support from the National Institute on Drug Abuse (NIDA), National Institute of Allergies and Infectious Disease (NIAID), and National Institute of Mental Health (NIMH) under National Institutes of Health (NIH) grants UM1AI068×619 (HPTN Leadership and Operations Center), UM1AI068617 (HPTN Statistical and Data Management Center), and UM1AI068613 (HPTN Laboratory Center). ## Footnotes * CONFLICTS OF INTEREST: The authors have no conflicts of interest to declare. * FUNDING: ALB and SRF were supported by an Avenir Award DP2DA046856 from National Institute on Drug Abuse of the National Institutes of Health. RUH-R, ALB, and DS were supported by National Institutes of Health (NIH) grant R01AI112339. RUH-R, and DS were also supported by NIH grant P30MH062294-18. ALB and DS were also supported by NIH grant DP1ES025459 (DP1), while SHV was supported by NIH grant P30MH062294. JL was supported by the National Science Foundation (NSF) grant DMS 1854934. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the National Science Foundation. * DATA AND COMPUTING CODE AVAILABILITY: Example SAS code used to fit the models presented in this paper can be found in the Supplement Appendix D. The HPTN 037 study data sets are publicly available and can be requested from the Statistical Center for HIV/AIDS Research and Prevention through the ATLAS Science Portal through [https://atlas.scharp.org/cpas/project/HPTN/](https://atlas.scharp.org/cpas/project/HPTN/). ## ABBREVIATIONS (CI) : Confidence interval (GEEs) : Generalized estimating equations (HPTN) : HIV Prevention Trials Network (HIV) : Human Immunodeficiency Virus (PWID) : People who inject drugs (RR) : Risk/Rate ratio * Received March 24, 2022. * Revision received March 28, 2022. * Accepted March 28, 2022. * © 2022, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/) ## References 1. 1.Padian NS, McCoy SI, Manian S, Wilson D, Schwartländer B, Bertozzi SM. Evaluation of large-scale combination HIV prevention programs: essential issues. Journal of Acquired Immune Deficiency Syndromes. 2011; 58(2):e23–e28. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/QAI.0b013e318227af37&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21694607&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) 2. 2.DeGruttola V, Smith DM, Little SJ, Miller V. Developing and evaluating comprehensive HIV infection control strategies: Issues and challenges. Clinical Infectious Diseases. 2010; 50(3):S102– S107. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1086/651480&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=20397937&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000276765600006&link_type=ISI) 3. 3.Kurth AE, Celum C, Baeten JM, Vermund SH, Wasserheit JN. Combination HIV prevention: Significance, challenges, and opportunities. Current HIV/AIDS Reports. 2011; 8(1):62–72. 4. 4.Padian NS, Holmes CB, McCoy SI, Lyerla R, Bouey PD, Goosby EP. Implementation science for the US President’s Emergency Plan for AIDS Relief (PEPFAR). Journal of Acquired Immune Deficiency Syndromes. 2011; 56(3):199–203. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/QAI.0b013e31820bb448&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21239991&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) 5. 5.Tanser F, Bärnighausen T, Grapsa E, Zaidi J, Newell M-L. High coverage of ART associated with decline in risk of HIV acquisition in rural KwaZulu-Natal, South Africa. Science. 2013; 339(6122):966–971. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6Mzoic2NpIjtzOjU6InJlc2lkIjtzOjEyOiIzMzkvNjEyMi85NjYiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMi8wMy8yOC8yMDIyLjAzLjI0LjIyMjcyOTA5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 6. 6.Hayes R, Moulton L. Cluster Randomised Trials. London: Taylor & Francis, 2009. 7. 7.Hayes R, Ayles H, Beyers N, Sabapathy K, Floyd S, Shanaube K, et al. HPTN 071 (PopART): Rationale and design of a cluster-randomised trial of the population impact of an HIV combination prevention intervention including universal testing and treatment–a study protocol for a cluster randomised trial. Trials. 2014; 15(1):57. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/1745-6215-15-57&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24524229&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000333492300002&link_type=ISI) 8. 8.Hayes RJ, Donnell D, Floyd S, Mandla N, Bwalya J, Sabapathy K, et al. Effect of universal testing and treatment on HIV incidence – HPTN 071 (PopART). New England Journal of Medicine. 2019; 381(3):207–218. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa1814556&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) 9. 9.Iwuji CC, Orne-Gliemann J, Tanser F, Boyer S, Lessells RJ, Lert F, et al. Evaluation of the impact of immediate versus WHO recommendations-guided antiretroviral therapy initiation on HIV incidence: The ANRS 12249 TasP (Treatment as Prevention) trial in Hlabisa sub-district, KwaZulu-Natal, South Africa: Study protocol for a cluster randomised controlled trial. Trials. 2013; 14(1):230. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/1745-6215-14-230&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=23880306&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) 10. 10.Iwuji CC, Orne-Gliemann J, Larmarange J, Balestre E, Thiebaut R, Tanser F, et al. Universal test and treat and the HIV epidemic in rural South Africa: A phase 4, open-label, community cluster randomised trial. The Lancet HIV. 2018; 5(3):e116–e125. 11. 11.Chamie G, Kwarisiima D, Clark TD, Kabami J, Jain V, Geng E, et al. Leveraging rapid community-based HIV testing campaigns for non-communicable diseases in rural Uganda. PloS one. 2012; 7 (8):e43400. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0043400&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22916256&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) 12. 12.Chamie G, Kwarisiima D, Clark TD, Kabami J, Jain V, Geng E, et al. Uptake of community-based HIV testing during a multi-disease health campaign in rural Uganda. PloS one. 2014; 9(1):e84317. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0084317&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24392124&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) 13. 13.Latkin CA, Donnell D, Metzger D, Sherman S, Aramrattna A, Davis-Vogel A, et al. The efficacy of a network intervention to reduce HIV risk behaviors among drug users and risk partners in Chiang Mai, Thailand and Philadelphia, USA. Social Science and Medicine. 2009; 68(4):740– 748. 14. 14.Nikolopoulos GK, Pavlitina E, Muth SQ, Schneider J, Psichogiou M, Williams LD, et al. A network intervention that locates and intervenes with recently HIV-infected persons: The Transmission Reduction Intervention Project (TRIP). Scientific Reports. 2016; 6. 15. 15.Spoth R, Guyll M, Lillehoj CJ, Redmond C, Greenberg M. Prosper study of evidence-based intervention implementation quality by community–university partnerships. Journal of Community Psychology. 2007; 35(8):981–999. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/jcop.20207&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=20376336&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000250643100004&link_type=ISI) 16. 16.Sobel ME. What do randomized studies of housing mobility demonstrate? Causal inference in the face of interference. Journal of the American Statistical Association. 2006; 101(476):1398– 1407. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1198/016214506000000636&link_type=DOI) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000242869700010&link_type=ISI) 17. 17.Hudgens MG, Halloran ME. Toward causal inference with interference. Journal of the American Statistical Association. 2008; 103(482):832–842. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1198/016214508000000292&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19081744&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) 18. 18.Benjamin-Chung J, Arnold BF, Berger D, Luby SP, Miguel E, Colford Jr JM, et al. Spillover effects in epidemiology: parameters, study designs and methodological considerations. International Journal of Epidemiology. 2018; 47(1):332–347. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/ije/dyx201&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=29106568&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) 19. 19.Forastiere L, Airoldi EM, Mealli F. Identification and estimation of treatment and interference effects in observational studies on networks. Journal of the American Statistical Association. 2021; 116(534):901–918. 20. 20.Crowley DM, Coffman DL, Feinberg ME, Greenberg MT, Spoth RL. Evaluating the impact of implementation factors on family-based prevention programming: Methods for strengthening causal inference. Prevention Science. 2014; 15(2):246–255. 21. 21.Lippman SA, Shade SB, Hubbard AE. Inverse probability weighting in STI/HIV prevention research: methods for evaluating social and community interventions. Sexually Transmitted Diseases. 2010; 37(8):512–518. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=20375927&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) 22. 22.Howe CJ, Cole SR, Mehta SH, Kirk GD. Estimating the effects of multiple time-varying exposures using joint marginal structural models: alcohol consumption, injection drug use, and HIV acquisition. Epidemiology. 2012; 23(4):574–582. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/EDE.0b013e31824d1ccb&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22495473&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) 23. 23.Hernán MA, Brumback B, Robins JM. Marginal structural models to estimate the joint causal effect of nonrandomized treatments. Journal of the American Statistical Association. 2001; 96(454):440–448. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1198/016214501753168154&link_type=DOI) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000168986400008&link_type=ISI) 24. 24.He J, Stephens-Shields A, Joffe M. Marginal structural models to estimate the effects of timevarying treatments on clustered outcomes in the presence of interference. Statistical Methods in Medical Research. 2019; 28(2):613–625. 25. 25.Hernán MA, Robins JM. Causal Inference: What If. Boca Raton: Chapman & Hall/CRC, 2020. 26. 26.Friedman SR, Bolyard M, Khan M, Maslow C, Sandoval M, Mateu-Gelabert P, et al. Group sex events and HIV/STI risk in an urban network. Journal of Acquired Immune Deficiency Syndromes. 2008; 49(4):440–446. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/QAI.0b013e3181893f31&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19186355&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000260844300014&link_type=ISI) 27. 27.Khan MR, Epperson MW, Mateu-Gelabert P, Bolyard M, Sandoval M, Friedman SR. Incarceration, sex with an STI-or HIV-infected partner, and infection with an STI or HIV in Bushwick, Brooklyn, NY: A social network perspective. American Journal of Public Health. 2011; 101(6):1110–1117. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2105/AJPH.2009.184721&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21233443&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000290887000032&link_type=ISI) 28. 28.Friedman S, Bolyard M, Sandoval M, Mateu-Gelabert P, Maslow C, Zenilman J. Relative prevalence of different sexually transmitted infections in HIV-discordant sexual partnerships: Data from a risk network study in a high-risk New York neighbourhood. Sexually Transmitted Infections. 2008; 84(1):17–18. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoic2V4dHJhbnMiO3M6NToicmVzaWQiO3M6NzoiODQvMS8xNyI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIyLzAzLzI4LzIwMjIuMDMuMjQuMjIyNzI5MDkuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 29. 29.Tobin KE, Kuramoto SJ, Davey-Rothwell MA, Latkin CA. The STEP into Action study: A peerbased, personal risk network-focused HIV prevention intervention with injection drug users in Baltimore, Maryland. Addiction. 2011; 106(2):366–375. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1360-0443.2010.03146.x&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21054614&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000285922300024&link_type=ISI) 30. 30.Robins JM, Hernan MA, Brumback B. Marginal structural models and causal inference in epidemiology. Epidemiology. 2000; 11(5):550–560. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/00001648-200009000-00011&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=10955408&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000088854500011&link_type=ISI) 31. 31.Tchetgen Tchetgen EJ, VanderWeele TJ. On causal inference in the presence of interference. Statistical Methods in Medical Research. 2012; 21(1):55–75. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0962280210386779&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21068053&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) 32. 32.Halloran ME, Hudgens MG. Dependent happenings: A recent methodological review. Current Epidemiology Reports. 2016; 3(4):297–305. 33. 33.Hernán Má, Brumback B, Robins JM. Marginal structural models to estimate the causal effect of zidovudine on the survival of HIV-positive men. Epidemiology. 2000; 11(5):561–571. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/00001648-200009000-00012&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=10955409&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000088854500012&link_type=ISI) 34. 34.Buchanan A, Vermund S, Friedman S, Spiegelman D. Assessing Individual and Disseminated Effects in Network-Randomized Studies. American Journal of Epidemiology. 2018; 187(11):2449– 2459. 35. 35.Hernán MA, Robins JM. Using big data to emulate a target trial when a randomized trial is not available. American Journal of Epidemiology. 2016; 183(8):758–764. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/aje/kwv254&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=26994063&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) 36. 36.Ogburn EL, VanderWeele TJ. Causal diagrams for interference. Statistical Science. 2014; 29(4):559– 578. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1214/14-STS501&link_type=DOI) 37. 37.Fitzmaurice GM, Laird NM, Ware JH. Applied Longitudinal Analysis. Hoboken: John Wiley & Sons, 2012. 38. 38.Spiegelman D, Hertzmark E. Easy SAS calculations for risk or prevalence ratios and differences. American Journal of Epidemiology. 2005; 162(3):199–200. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/aje/kwi188&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=15987728&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000230724300001&link_type=ISI) 39. 39.Zou G. A modified poisson regression approach to prospective studies with binary data. American Journal of Epidemiology. 2004; 159(7):702–706. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/aje/kwh090&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=15033648&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000220484900009&link_type=ISI) 40. 40.Breskin A, Cole SR, Westreich D. Exploring the subtleties of inverse probability weighting and marginal structural models. Epidemiology. 2018; 29(3):352–355. 41. 41.Lancaster KE, Miller WC, Kiriazova T, Sarasvita R, Bui Q, Ha TV, et al. Designing an individually tailored multilevel intervention to increase engagement in HIV and substance use treatment among people who inject drugs with HIV: HPTN 074. AIDS Education and Prevention. 2019; 31(2):95–110. 42. 42.Perriat D, Balzer L, Hayes R, Lockman S, Walsh F, Ayles H, et al. Comparative assessment of five trials of universal HIV testing and treatment in sub-Saharan Africa. Journal of the International AIDS Society. 2018; 21(1):e25048. 43. 43.De Lay PR, Benzaken A, Karim QA, Aliyu S, Amole C, Ayala G, et al. Ending AIDS as a public health threat by 2030: Time to reset targets for 2025. PLoS Medicine. 2021; 18(6):e1003649. 44. 44.VanderWeele TJ, Tchetgen Tchetgen EJ, Halloran ME. Interference and sensitivity analysis. Statistical Science. 2014; 29(4):687–706. 45. 45.Robins JM, Finkelstein DM. Correcting for noncompliance and dependent censoring in an AIDS clinical trial with inverse probability of censoring weighted log-rank tests. Biometrics. 2000; 56(3):779–788. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.0006-341X.2000.00779.x&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=10985216&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000089069500018&link_type=ISI) 46. 46.Cole SR, Platt RW, Schisterman EF, Chu H, Westreich D, Richardson D, et al. Illustrating bias due to conditioning on a collider. International Journal of Epidemiology. 2010; 39(2):417–420. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/ije/dyp334&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19926667&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000276303800019&link_type=ISI) 47. 47.Cole SR, Hernán MA. Constructing inverse probability weights for marginal structural models. American Journal of Epidemiology. 2008; 168(6):656–664. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/aje/kwn164&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=18682488&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000258959200015&link_type=ISI) 48. 48.Bang H, Robins JM. Doubly robust estimation in missing data and causal inference models. Biometrics. 2005; 61(4):962–973. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1541-0420.2005.00377.x&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=16401269&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F03%2F28%2F2022.03.24.22272909.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000234062200013&link_type=ISI) [1]: /embed/graphic-1.gif [2]: /embed/inline-graphic-1.gif [3]: /embed/inline-graphic-2.gif [4]: /embed/inline-graphic-3.gif [5]: /embed/inline-graphic-4.gif [6]: /embed/inline-graphic-5.gif [7]: /embed/inline-graphic-6.gif [8]: /embed/inline-graphic-7.gif [9]: /embed/inline-graphic-8.gif [10]: /embed/inline-graphic-9.gif [11]: /embed/inline-graphic-10.gif [12]: /embed/inline-graphic-11.gif [13]: /embed/inline-graphic-12.gif [14]: /embed/inline-graphic-13.gif [15]: /embed/inline-graphic-14.gif [16]: /embed/inline-graphic-15.gif [17]: /embed/graphic-2.gif [18]: /embed/inline-graphic-16.gif [19]: /embed/inline-graphic-17.gif [20]: /embed/graphic-3.gif [21]: /embed/inline-graphic-18.gif