The international trial of nasal oxygen therapy after cardiac surgery (NOTACS) in patients at high risk of postoperative pulmonary complications: Economic evaluation protocol and analysis plan

a) Identification of resource use

Resource use relates to setting up and delivery of HFNT (including the HFNT device, level of use, associated oxygen consumption, training, and maintenance) or SOT (oxygen flow rate, delivery using nasal mask or prong) and its consequences i.e. use of hospital, primary, community and social care until 90 days after surgery for both trial arms. Resource use from the individual/family perspective include expenses for medical, non-medical and social care support services, and time spent by patients and caregivers following randomisation. Table 1 provides a summary of all resources and services identified and collected alongside the trial. As an incremental approach is used in costing, resources common to both arms and those related to conducting the research are excluded.

b) Measurement of resource use

Health and social care resources are collected using a variety of patient-specific case report forms (CRFs). Some are completed by research nurses abstracting data from patient-specific records (e.g. oxygen therapy, in-patient location, discharge details), some through research nurses interviewing patients face-to-face or using online, telephone or paper questions as preferred (e.g. quality of life, costs over hospital stay, service use and expenditure post-surgery). Participant location and medication diary following discharge is the only record directly filled in by the patients. Relevant resources used to provide the service but not limited to specific patients (for example, HFNT training, equipment and maintenance) are measured using research records if available. The source and level of resource use measurements taken are provided in Table 1.

The amount of oxygen used with HFNT and SOT forms a relevant differential resource between the trial arms and hence is costed. As the central oxygen supply is common to both HFNT and SOT, the analysis measures only the variation in amount of oxygen supplied either by HFNT or SOT at a patient-specific level. For SOT, the CRF records oxygen flow rate titration in L/min and duration of use to give total quantity of oxygen used. For HFNT, the CRF records FiO2 and the gas flow rate (L/min). Using an indicative chart, patient-specific data on FiO2 and gas flow rate will be used to determine amount of oxygen used by HFNT (24). Consumables (e.g., oxygen interface, tubing, nasal prong) will be quantified using patient-specific duration of oxygen therapy along with available duration of use guidance given by manufacturer or using clinical expert opinion (25).

The changes to the data collection instruments in AUS and NZ included contextual country-specific questions, responses and wording revisions for resource use data collection. This mainly included replacing country-specific questions on ethnicity, education, healthcare concession and out-of-pocket payments for ambulance travel, health professional visit and telephone consultations in AUS and NZ. Additionally, during the trial, amendments were made to data collection tools to improve data completion. Further details on data collection amendments are available in supplementary Table S2 (S1 Appendix).

c) Valuation of resource use

All health and social sector resources will be valued using national costs (for example, national cost collection in the UK) for the latest available year (26–32). Any unit cost not in the corresponding year will be adjusted using a health-or country-specific inflation index (for example, NHS cost inflation index published in Personal Social Services Research Unit (PSSRU) cost manual) (31,33–36). If national unit costs are unavailable, published literature or local cost sources will be used. Capital costs will be annualised using country-specific discounting recommendations (3.5% UK, 5% AUS, 3.5% NZ) or using the 3.5% UK recommendation for pooled results, further assessed in sensitivity analysis (20,21,37). For example, in base-case, the HFNT device will be annualised using an average lifespan of 5 years, 3.5% discount rate and an annualised maintenance cost using available sources. Patient reported expenses will be used to value patient perspective costs. Time ‘spent’ by patients and caregivers will be valued using the human capital approach based on gross national average wages (38–40). Unpaid work will be valued using wage earned for similar work on a paid basis or using national average earnings if unavailable.

d) Computation of total costs

Total cost will be computed at a patient level by summing the components contributing to the cost of intervention delivery, initial admission and up to 90 days post-surgery relevant to health/social care sector and patient/family perspectives. For example, in base-case, per patient cost of HFNT arm in UK will include NHS/PSS perspective costs for HFNT delivery, initial admission and service utilisation until end of follow-up respectively. Total costs will be obtained by multiplying the quantity of resource use by unit cost across each subcomponent and then summing across. Some costs (for example, HFNT device) are expected to be shared between patients. In such cases, annualization, life-span and apportioning rules (for example, number of patients using the device) will be used as appropriate.

a) Identification of patient reported outcomes

Days alive and at home after surgery (DAH90), a validated patient-reported metric, is the primary outcome endpoint for NOTACS trial (41). DAH90/DAH30 define home as a person’s usual abode at 90 (and 30) days post-surgery. Escalation of care or time away from usual abode during follow-up period will be counted as a day away from home as per defined protocol. DAH is sensitive to changes in surgical risk and is considered a superior measure of quality of surgery and care than standard complication and mortality rates (42,43). The CRF records changes in the participant’s overnight living location throughout follow-up. It includes details on whether stay was planned and if support for daily living activities was needed. The DAH measure will be used in cost-effectiveness analysis (CEA) as it was identified as the most important primary outcome by patient and public representatives during development of the proposal.

For cost-utility analysis (CUA), a standardised generic health related quality of life measure, the EQ-5D-5L, relevant to the UK, AUS, and NZ is used (20–22,44). The EQ-5D-5L consists of two parts; the first asks respondents to categorise their health today on five dimensions (mobility, self-care, usual activities, pain/discomfort, anxiety/depression) and five response levels (no problems, slight problems, moderate problems, severe problems, unable to) for each dimension, and the second part asks respondents to rate perception of their own overall health today on a scale of 0 (worst) to 100 (best) imaginable health.

b) Measurement and valuation of patient reported outcomes

DAH90 and DAH30 are collected using a mix of CRFs completed by patients and research nurses from date of index surgery to 90 days after surgery. Patients are given the location diary on discharge and have the option of completing follow-up questionnaires on paper, online or via telephone. Patients not completing follow-up CRFs at 30 and 90 days are contacted regularly up to 30+30 and 90+90 days as per trial protocol. Information may be gathered from the patient’s general practitioner (GP) surgery in case of non-response or to verify patient responses as per consent.

DAH90/DAH30 valuation accounts for any increase in care level from patient’s previous residence. For example, hospital readmission or additional care requirement at ‘home’ within 90 days of surgery is subtracted from the total number of days at home. Staying away from ‘home’ for social reasons without any additional care or support is considered as home. The protocol for what counts as days away from home is predefined and available in the update of the published statistical analysis plan (submitted and under review). The location is based on where a patient spends the night. Patient records reporting baseline or follow-up abode as ‘Other’ will be reviewed for whether the ‘other’ should be classed as ‘home’ and agreed on a case-by-case basis by at least two blinded members. An additional blinded member will be consulted if consensus is not reached. If a patient living at home is discharged from the hospital on day 6 after surgery but is subsequently readmitted for 4 days and then returns home until 90 days post-surgery, they would be assigned 80 DAH90.

The base case statistical analysis for the trial follows the approach used by Myles et al., to count days at home i.e. when a patient dies – a DAH value of 0 occurs for the whole period (18,41). For example, if a patient dies while still in the hospital or at home before 90 days, they are assigned 0 DAH90. This means no value is attributed whether a person lives for 1 or 89 days within the study period and no change can be seen if survival changes over time. The trial is not designed to capture change in survival and one reasoning to accept such a scoring is that it takes a more conservative view of benefit. However, for the economic evaluation there are two problems in using this approach. First there is evidence that costs and outcomes can be related and higher costs are particularly related to proximity to death (45–47). Through an increase in outliers, and potential miscasting of the relationship between costs and effects, assigning a zero value to all people dying could significantly bias regression results for cost-effectiveness. Second the cost-utility analysis, part of the suite of tools in economic evaluation used to support NICE’s decision-making, does account for all days lived. Therefore, the base-case cost-effectiveness analysis will be conducted using all recorded days at home and dubbed ‘DAH90(HE)’ to represent a difference with calculation of the main trial outcome DAH90, which will be used in sensitivity analysis, to allow comparison with the main trial analysis.

Responses recorded on the EQ-5D-5L tool will be converted into a health state utility score using UK, AUS and NZ specific value sets (48–50). As recommended in the UK, the EQ-5D-5L data will be mapped onto the 3L value set (48,51), with QALYs calculated using the area under curve method (52). The study design offers patients a tolerance window of 90 days for 90-day follow-up (and 30+30 days) to complete data collection. Because utility values may be expected to rise over time post-surgery, sensitivity analysis will explore the impact of using linear interpolations to obtain utility values at 90 days (53).

a) Handling missing data

Before commencing analysis, data quality checks will be conducted and any duplicates, missing or extreme values will be identified and documented. Any missing data will be cross-checked against the CRF and, where necessary, patient records reviewed for procedural error. Clear outliers, such as those exceeding 4 standard deviations, will be investigated further and may be excluded or treated as missing values. However, if a series of potential outliers appears within a skewed distribution, they may be retained. In such cases, the data may be transformed (e.g., logarithmic or square root transformation) if necessary and assessed in sensitivity analysis.

The pattern and amount of missing data at each time point will be summarised using frequency and percentage plots, with pattern of missingness for resource use and outcomes assessed using Pearson Chi2 goodness of fit and Wilcoxon rank-sum tests at each data collection point (54). If data are missing completely at random (MCAR) and the percentage of missing data is below 5%, a complete case analysis (CCA) will be undertaken. This is unlikely to be the case due to the need to aggregate resource use and costs across many variables. Data missing at random (MAR) will be addressed using multiple imputation (MI) methods. Missing cost and outcome data will be estimated using joint chained equations with predictive mean matching, stratified by treatment arm and country. We will explore censoring as a special case of missing data, particularly right censoring from loss to follow-up to judge the impact on estimating mean values for costs, DAH90 and QALYs using an inverse probability weighting approach (54). Imputation models will be determined during analysis as it is important that the model includes variables that are good predictors of missing variables. We will consider variables such as age, sex, baseline Barthel and/or EQ-5D-5L scores, baseline health service use, BMI, ethnicity, number of co-morbidities, whether surgery is elective, Euro Score II, ASA, and ARISCAT score. The number of imputed datasets required to achieve a stable imputation will be used (55). The distribution of imputed and observed values will be compared across both trial arms. Rubin’s rule will be used to generate a complete single data set for analysis, encompassing the variation and uncertainty preserved in the multiple imputed data sets (56). Sensitivity analysis will assess the impact of different missingness assumptions (56). Total costs, DAH90(HE), utility index score at day 90 and total QALYs with CCA, pre-imputed and imputed dataset will be presented for HFNT versus SOT comparison.

b) Statistical analysis of differences in resource use, costs, and outcomes for cost-effectiveness and cost-utility analyses

The intention-to-treat principle will be followed for base-case analyses. Resource use, costs and outcomes will be summarised by trial arm; a) at discharge, 30– and 90-days, and b) by country at 90-days using descriptive statistics (frequency, mean (standard deviation), median, minimum (including % of zeros), maximum, IQR, 95% confidence interval around the mean, histograms and box plots) along with tests of normality. Significant differences at 5% level, using raw and imputed data, will be tested using two-sample t-tests of equality for normally distributed variables, and the Mann-Whitney-Wilcoxon test and bootstrapped t-tests if skewed. This analysis will be repeated for a complete case analysis if feasible. Changes in the EQ-5D-5L health profile will be summarised using the Paretian Classification of Health Change and by reporting the ten most frequently observed health states at baseline and day 90 for both trial arms (57). A Health Profile Grid will be developed for visual representation (57).

Regression models will be fitted for mean costs and QALYs, with and without imputed values. Models will include intervention arm and variables to adjust for baseline patient characteristics (age, sex, Barthel, EQ-5D score, health service use, BMI, ethnicity, number of co-morbidities and whether surgery is elective, Euro Score II risk score, ASA and ARISCAT score). We will however be unable to control for baseline differences such as patient/family borne costs. We will examine data prior to selecting a regression model but will consider generalised linear or mixed models given their ability to handle large numbers of zeros and will test log-link and more flexible forms to examine goodness of fit. We will assess the relationship between baseline covariates for collinearity in the decision for covariate inclusion in the final model. We will also examine the impact of accounting for the correlation between costs and QALYs, by using seemingly unrelated regression analysis with and without correcting for confounding at baseline (58).

The difference in costs between the trial arms will be divided by the difference in outcomes between the trial arms, to give incremental cost-effectiveness/utility ratios. Incremental cost-utility ratios (ICURs) will be compared with country-specific costs-effectiveness thresholds (CET) used in the UK (£20,000/QALY), AUS (AU$ 28,033/QALY) and NZ (NZ$ 40,000/QALY) (20,45,46). Fully pooled results will be reported in the UK currency (country-specific unit cost applied to resource use followed by summing of all costs to UK currency using purchasing power parity (PPP)) and assessed against the CET value for UK. Country-specific incremental net monetary benefit (NMB) statistics, (INMB = incremental benefit * CET – incremental cost) will be presented with 95% CI limits. As no CETs are available for DAH outcome, the incremental cost-effectiveness ratios (ICERs) will be used to support wider decision making for any clearly dominant or dominated strategies. We will follow the recommended framework to present cost-effectiveness and cost-utility estimates for this multinational trial (61,62) and present following; a) country-specific ICUR estimates using country-specific data on cost and QALY respectively b) country-specific ICUR estimates using trial-wide resource use valued against country-specific unit costs and trial-wide EQ-5D responses valued against domain level EQ-5D value set for chosen country) and, depending on evidence of heterogeneity, c) pooled trial-wide ICUR estimate by applying country-specific unit costs to resource use followed by summing costs to the common UK currency using PPP and summing country-specific QALYs using country-specific value sets into a pooled QALY value. The Likelihood ratio test and range test of qualitative interaction will be used to examine if the direction of cost-effectiveness varies between the countries (63,64). The likelihood ratio and standardised range test statistic will be compared against respective critical values at a significance level of α = 0.05 for the three groups to indicate heterogeneity (63,65). We will explore the use of shrinkage estimates as an alternative to trial-wide pooling of cost-effectiveness results (66). Table 2 provides the methods and perspective considered in presenting the range of cost-effectiveness results.

c) Handling uncertainty

Parameter uncertainty will be handled using one-way sensitivity analysis (OWSA) and scenario analysis to assess the impact of following changes:

• i) Costs

  • HFNT intervention setup costs, including assumptions about length of life, device use and price.

  • Oxygen therapy cost, any variation in oxygen consumption calculation.

  • Any significant assumption in measurement or valuation methods for costs, including outliers (for example, methods for follow-up/intravenous medication data collection, discount rate, etc).

  • Cost at 30 days.

  • Perspective (impact of including patient/family costs)

• ii) Outcomes

  • DAH90 to assess the impact of assigning a DAH value of zero for those dying between zero to 90 days post-surgery.

  • Using linear interpolate and 90+90^th day value as a proxy for 90^th day utility value in QALY calculation.

  • Outcomes at 30 days.

Sampling uncertainty will be characterised using 95% CI limits for ICUR/ICER values based on non-parametric bootstrapping along with cost-effectiveness plane (CEP) and cost-effectiveness acceptability curves (CEAC) to inform decisions (20).

Heterogeneity will be assessed using sub-group analysis to improve robustness of our findings. The potential subgroups to be explored are:

• Patients requiring escalation of oxygen therapy.

• Time on treatment (at least 8 or 24 hours)