A rapid review of the effectiveness and cost effectiveness of interventions that make homes warmer and cheaper to heat for households in fuel poverty in rural and remote areas

1.1 Who is this review for?

This Rapid Review was conducted as part of the Health and Care Research Wales Evidence Centre Work Programme. The above question was suggested by Technical Advisory Cell (TAC), Welsh Government, and Taf Housing.

1.2 Background and purpose of this review

The cost-of-living across the UK has been on the increase since the start of 2021, reflecting the position globally. Road fuel prices and household energy bills have risen due to the effect of Russia’s invasion of Ukraine (Francis-Devine et al. 2022).

Fuel poverty is defined as “spending more than 10% of a household’s income on fuel for satisfactory heating and comfort and to sustain all energy services” (Charlier & Legendre 2021, p.121557). The main drivers of fuel poverty are energy prices, energy efficiency, low household income and how energy is used in the home (Scottish Government 2016). The consequences of fuel poverty and low incomes identified prior to the current cost of living crisis, for example living in a cold damp home, have a negative impact on both physical and mental wellbeing (Chakravorty 2022, Volkos & Symvoulakis 2021, Thomson et al. 2022, Roberts et al. 2022). It is envisaged that the number of excess winter deaths will increase as fuel poverty rates increase (Roberts et al. 2022).

In Wales, although the current cost-of living crisis effects all households, living in a rural community is often associated with additional daily living costs compared to those living outside of rural areas (Senedd Cymru 2022a, Roberts et al. 2022). In 2020, many domestic properties were not connected to the gas grid, the highest percentage being in rural areas including mid and west Wales (DBEIS 2021), it has been reported that nearly a third of rural households use oil as their main fuel for heating (Wales Audit Office 2019). Compared to networked energy sources, oil and liquid petroleum gas are more expensive and are not subjected to energy price caps (Senedd Cymru 2022a). A further concern is that rural housing is generally older, not as well insulated and less energy efficient than houses in urban locations (Senedd Cymru 2022a, Senedd Cymru 2022b). Higher petrol and diesel costs in rural filling stations, alongside a lack of available public transport leading to higher use of privately owned motor vehicles are also all contributing factors to rural fuel poverty (Senedd Cymru 2022a).

Programmes to improve how energy is used in the home (home energy advice), to improve the energy performance of homes, financial support and community level energy buying clubs and cooperative arrangements all have the potential to mitigate fuel poverty (Javornik & Mackie 2022, Powell et al. 2018) which in turn can result in health improvements and improved quality of life (Powell et al. 2018). Interventions can be large scale government subsidised activities focusing on improving the energy efficiency of housing stock and or household appliances or locally delivered projects through partnerships with local authorities, housing providers or third sector organisations. The aim of this rapid review was to determine the effectiveness of interventions that make homes warmer and cheaper to heat for households in fuel poverty in rural and remote areas.

2.1 Overview of the Evidence Base

The evidence base consists of primary research evaluations presented across 14 studies which consisted of one randomised control trial (with partial crossover) (Heyman et al. 2011), three quasi-experimental studies (Grey et al. 2017, Poortinga et al. 2017, Willand et al. 2019); two pre-test / post-test with a control group (Shortt & Rugraska 2007, Wade et al. 2019); two pre-test / post-test with no control group (Eadson & Leather 2017, Papada et al. 2021); one post-test with control group (Sharpe et al. 2020), one case study (McGinley et al. 2022); one case control study (Charlier et al. 2019) and three quantitative descriptive surveys (Miller et al. 2022, Sherriff et al. 2020, Welsh Government 2015). Five of these were part of wider mixed methods studies (Shortt & Rugkåsa 2007, Miller et al. 2022, Wade et al. 2019, Welsh Government 2015, Willand et al. 2019). Additional information (n=8) has been provided by UK grey literature sources including the evaluations and/or annual reports of the Welsh Government Warm Homes Program - Nest (Welsh Government 2022) and Arbed (Arbed am Byth 2022, Patterson 2012), Scottish schemes (Citizens Advice Scotland 2016, Citizens Advice Scotland 2017, Shelter Scotland 2017), UK schemes (DECC 2015) and English schemes (NEA 2019). Of these, four are catalogues of fuel poverty schemes (Citizens Advice Scotland 2016, DECC 2015, NEA 2019, Shelter Scotland 2017).

Only five of the evaluations focused entirely on rural properties and these were conducted in Scotland (Sherriff et al. 2020, Wade et al. 2019); Ireland (McGinley et al. 2022); Northern Ireland (Shortt & Rugkasa 2007) and Cornwall, England (Sharpe et al. 2020). A further evaluation was conducted in a remote region of Greece (Papada et al. 2021). The remainder of the evaluations (n=8) were conducted across both urban and rural areas throughout Wales (Grey et al. 2017, Miller et al. 2022, Pootinga et al. 2017; Welsh Government 2015) and England (Eadson & Leather 2017, Heyman et al. 2011); France (Charlier et al. 2019) and Australia (Willand et al. 2019). Most grey literature (n=7) covered both urban and rural areas across the UK, except the report by Shelter Scotland (2017) which focused mainly on rural areas.

The interventions that were evaluated were home energy advice, referral for support and/or insulation measures or home improvements (including Nest) (Eadson & Leather 2017, Sherriff et al. 2020, Wade et al. 2019, Welsh Government 2015), social energy subsidies and energy efficient housing (social housing) (Charlier et al. 2019); energy efficiency home improvements / retrofitting (Heyman et al. 2011, McGinley et al. 2022, Willand et al. 2019) (including Arbed)(Grey et al. 2017, Poortinga et al. 2017, Miller et al. 2022); central heating (Sharpe et al. 2020, Shortt & Rugkasa 2007); energy related living lab - advice with monitoring equipment (Papada et al. 2021). The grey literature evaluations and/or annual reports focused on energy efficiency home improvements and advice (n=1) (Welsh Government 2022), on energy efficiency home improvements / retrofitting (n=2) (Arbed am Byth 2022, Patterson 2012), advice (Citizens Advice Scotland 2017). The catalogues all described a variety of energy efficiency measures and other fuel poverty interventions (Citizens Advice Scotland 2016; DECC 2015, NEA 2019; Shelter Scotland 2017).

2.2 Effectiveness of interventions

The effectiveness of social energy subsidies (energy subsidy voucher of €48 to €227 a year, depending on the household income and composition) and energy efficient housing (social housing) to reduce fuel poverty was assessed by Charlier et al. (2019). The sample sizes of those in receipt of the social energy subsidy was small (n=81) compared to those in receipt social housing (n=422) and those in the control group (n=552), statistical analyses were poorly presented, there were inconsistencies in the descriptions of treatment and control groups. However, it was determined that providing social energy subsidies did not alleviate fuel poverty. Providing energy efficient housing (social housing) was found to be a more efficient means of alleviating fuel poverty.

Energy-related living lab which aims to provide low-cost methods to energy vulnerable households in order to tackle fuel poverty was the focus of the study by Papada et al. (2021). The living lab consisted of installing monitoring equipment in the homes, inspecting heating systems and providing energy specific advice. Households that had monitoring equipment installed, energy advisor visits and advice were significantly more likely to apply energy efficiency measures (p=0.001) compared to those who did not have energy monitoring equipment installed. Households also reported a positive change in their energy related behaviours and an improvement in their quality of life (improved thermal comfort, facing less moisture problems and reduced energy costs) compared to those who did not have energy monitoring equipment installed. Additionally, due to the maintenance of central heating systems in 12 of the households a significant increase in the burners’ energy-efficiency ratio was detected, resulting in considerable energy savings and a reduction of the households’ energy costs.

Heyman et al. (2011) conducted a randomised controlled trial (with partial crossover) to assess the personal, social and economic benefits of energy efficiency measures in the form of housing improvements (loft insulation, cavity wall insulation, draught exclusion, heating controls and central heating). Households that had received energy efficiency measures demonstrated improved Standard Assessment Procedure (SAP) ratings by 12 points compared to the control group (p<0.001). This translated into the households with energy efficiency measures generating increases in winter evenings living room temperatures of one to two degree Celsius compared to the control group (p=0.03). The greatest temperature increases were associated with the combination of heating system and insulation measures. Families did not respond to energy efficiency gains by reducing their heating expenditure. Because homes receiving energy efficiency measures were more fuel-efficient than those of the control group, the former could spend less on fuel than the latter in order to achieve the same room temperature. However, households with energy efficiency measures seemed to have increase temperatures, choosing increased warmth over reduced bills.

A post-intervention survey was used to assess the impact of an intervention to install a new first time central heating system in order to reduce fuel poverty (Sharpe et al. 2020). Responses were compared between a waiting list control and those who had central heating installed for the first time. Significantly (p<0.01) more households (50.75%) in the intervention group reported that there had been an improvement in their ability to pay their bills compared to the control (28.21%) Additionally, significantly (p<0.01) less households (58.75%) in the intervention group reported that they had to avoid heating their home due to costs compared to the control (83.54%).

A study conducted in rural areas in Northern Ireland (Shortt & Rugkasa 2007) evaluated the installation of central heating for households in fuel poverty. It was reported that there was relatively little change in average indoor temperature after installing central heating although this was only measured across 12 of the 54 households that were in receipt of heating installation and not compared with the control households. Average fuel costs of households in the intervention group significantly decreased from £1113 per year to £751.56 (p<0.001). Another objective of the study was to encourage people who were eligible for benefits, but not claiming them, to do so. However, this was inaccurately and poorly reported and no conclusions can be drawn.

The retrofitting of five detached properties in rural Ireland was investigated by McGinley et al. (2022). The authors reported that due to the differences in case study properties and measures installed (external insulation, heating systems, loft insulation) that it was difficult to make comparisons across the properties. However, each property experienced various benefits from retrofitting, including increased thermal comfort (ranging from 1% (case D) to 19% (case A) increase in indoor temperature), and reduced energy usage (ranging from 1% (case C) to 66% (case B) reduction in primary energy usage).

Willand et al. (2019) conducted a quasi-experimental study as part of a wider mixed methods investigation to quantify changes in indoor temperatures, energy consumption, energy costs and health due to building retrofits. The findings showed that the households who had received energy efficiency retrofit improvements intervention had improved their mean energy efficiency star rating from 0.8 stars to 3.5 stars. However, data was only available for a sub-sample of properties and no statistical analysis was performed to compare this to households in the control group. Valid indoor temperature data was available for 12 living rooms and 12 bedrooms. Taking only occupied days of the home into account, the retrofit intervention significantly reduced electricity consumption (p=0.17) but not gas consumption (p=0.742). These benefits were primarily attributed to the replacement of light bulbs with light-emitting diode (LED) lights and of portable electric heaters with new reverse cycle air conditioners (RC ACs). However, sample sizes were too small, as statistical analysis was only performed on sub samples of households (n=10 or less), and as such are underpowered.

3.1 Summary of the findings

The interplay of four factors including low income, high energy costs, insufficient energy performance, and high energy usage in the home environment can lead to fuel poverty (Scottish Government 2016). Thus, improving energy performance of houses is crucial in fighting fuel poverty, and government initiatives focusing on energy efficiency both in the UK and abroad have had some success. The findings of this rapid review indicate that energy efficiency interventions, such as external insulation, central heating installation, and heating system changes, can have a beneficial impact on householders’ ability to heat their homes and reduce energy costs. Furthermore, energy efficient social housing was found more effective in alleviating fuel poverty than social energy subsidies (Charlier et al. 2019). However, increases in fuel prices and insufficient income raises can influence the amount of benefit gained from interventions aiming to improve fuel poverty (Scottish Government 2016).

A number of schemes as reported in the catalogues across the UK incorporate providing energy advice to those in fuel poverty regarding topics, such as fuel debt, tariff and suppliers, energy behaviours, heating systems, and they can make referrals for energy efficiency measures schemes (Citizens Advice Scotland 2016, DECC 2015, NEA 2019, Shelter Scotland 2017). Where the outcomes of evaluations have been reported, households who act on the advice given are better able to heat their homes and report a reduction in average bill savings (Papada et al. 2021, Welsh Government 2015, Welsh Government 2022, Citizens Advice Scotland 2017).

This rapid review aimed to look for interventions in rural areas, although only three of the 11 included evaluations addressed rural fuel poverty specifically. Powell et al. (2018) in reporting an evidence review of interventions to address rural fuel poverty across OECD countries also identified very little specific evidence for rural areas and only tentative conclusions regarding the effectiveness of the interventions could be made. However, this mainly included grey literature reports with limited peer-reviewed evidence, in contrast with this rapid review.

Previous reviews have mainly described schemes that have the potential to address fuel poverty (Das et al. 2022) or that have investigated different types of interventions that encourage energy conservation across all households and not just those in fuel poverty (Das et al. 2022, Delmas et al. 2013, McAndrew et al. 2021). This rapid review is unique as it specifically looked at studies that investigated the impact of interventions on fuel poor populations.

3.2 Strengths and limitations of the available evidence

The included studies had several limitations based on the methodological assessment. Of the 14 included studies, one randomised controlled trial (Heyman et al. 2011) scored six out of a potential ten criteria on the critical appraisal checklist. Three questions were not applicable as the blinding of participants (Q4), or those delivering the intervention or assessing the outcome (Q5, Q6) was not feasible. The study was down scored as it was unclear if true randomisation was used in the assignment of participants (Q1) or if the allocation to treatment groups was concealed (Q2). Moreover, treatment groups were not similar at baseline (Q3).

Seven of the included studies were appraised by using the JBI checklist for quasi-experimental studies. Two of these, (Grey et al. 2017, Poortinga et al. 2018) met eight out of the potential nine critical appraisal criteria. However, the remaining four reports scored four (Eadson & Leather 2017, Papada et al. 2021, Shortt & Rugkasa 2007) and five (Wade et al. 2019; Willand et al. 2019). It was unclear if the participants included in any comparisons received similar treatment other than the intervention of interest (Q3). One of the studies (Papada et al. 2021) did not include multiple measurements of the outcome pre and post intervention (Q5). The other two studies either did not report or it was unclear whether outcomes were measured in the same way (Q7). Shortt & Rugkasa (2007) also had issues with the reliability of the measures used (Q8) and Wade et al. (2019) conducted statistical analysis on a sub sample of three households. Moreover, Eadson & Leather (2017) did not clarify whether the same participants or two different samples were compared descriptively, and no explanation was given regarding why inferential statistics were not conducted.

One study appraised with the JBI checklist for cohort studies (prospective) (Sharpe et al. 2020) scored only four out of a potential nine criteria. Although the groups were similar upon recruitment (Q1) and exposure was measured similarly to assign participants to groups (Q2) and in a valid and reliable way (Q3), any confounding factors or strategies to deal with confounding were not identified or stated (Q4, Q5). Furthermore, it was unclear if the participants were free of the outcomes at the start of the study (Q6) or if outcomes were measured in a valid and reliable way (Q7).

Of the remaining five included studies, one case report (McGinley et al. 2022) scored highly, meeting seven out of the eight critical appraisal criteria. Three studies were appraised using the JBI checklist for analytical cross-sectional studies (Sherriff et al. 2020, Miller et al. 2022, Welsh Government 2015) and they all score five out of a potential eight criteria. They did not report identifying confounding factors and strategies to deal with them (Q5, Q6). Finally, one case control study (Charlier et al. 2019) met five of the potential 10 criteria as it was unclear whether groups were comparable at the start of the study (Q1), and if the exposure was measured reliably and in the same way for all (Q5, Q6).

As evidenced, many of the included studies had quality issues in terms of being comparable at entry and any interventions and outcomes being measured reliably and consistently across groups. There was a lot of variation within the reports, in terms of the type of housing included in the sample (Papada et al. 2021), period of construction (Charlier et al. 2019), along with any measures installed, outcomes assessed, such as reduction in energy consumption, change in room temperature or thermal comfort. This made it difficult for studies to compare results across properties. It has been noted that this is also an issue across the wider evidence base in this field (Delmas et al. 2013, McAndrew et al. 2021). Additionally, occupancy and occupant behaviour, which can have a large impact on energy consumption were often not considered (Poortinga et al. 2017). Furthermore, the geographical scope of the studies reviewed varied, focusing on local authority, regional or national areas. At times this, together with the general lack of reporting, made it difficult to categorise studies as rural or urban.

Some studies presented no statistical analysis whilst others poorly presented the information with two studies only performing statistical analysis on a sub sample of households (Wade et al. 2019; Willand et al. 2019) and another only selectively reporting the key statistically significant findings (Grey et al. 2017). Sample size was an additional limiting factor for many of the included reports, with small samples restricting the reliability of the findings and the ability to draw conclusions regarding the effectiveness of the interventions for larger population groups. There was also an absence of any sample size calculations, highlighting a further need for caution when interpreting report findings.

There is a notable absence of economic evaluations of housing improvements within the included studies. This was also noted by Fenwick et al. (2013) who conducted a systematic review of economic analyses on the health impacts of housing improvements, and the majority of included studies (n=25/29) presented data on intervention and/or recipient costs only and, despite sufficient data, opportunities to conduct economic analysis have been missed. Furthermore, Thomson et al. (2009) mentioned in their systematic review whether included studies had reported some economic analysis. While some socioeconomic benefits were mentioned, Thomson et al. (2009) concluded that economic impact assessments should be planned alongside studies looking at the effect of housing improvements.

3.3 Implications for policy and practice

There is a need for high quality research, particularly for well-developed randomised controlled trials that investigate the effectiveness and cost-effectiveness of measures, such as external insulation, heating system changes, with careful consideration to confounding factors. The interventions will likely need to be multicomponent. However, identifying individual measures that are most effective, or redundant, is also important, which may require innovative adaptive trial design or analysis.

Future research should investigate which interventions are the most effective in what types of housing in rural areas to help the targeting of interventions better.

Policy makers and funding bodies need to make further investments into developing funding calls for investigating measures to alleviate fuel poverty, with particular focus on economic analysis.

3.4 Strengths and limitations of this Rapid Review

Strength of this rapid review is that systematic, protocol-driven procedures were followed in the searching and identification of key literature, supported by an experienced information specialist. Moreover, websites of relevant UK organisations and governments were searched to identify grey literature reports and evaluations. However, as this is a rapid review, some of the review processes are streamlined and modified to ensure timely production. Approximately Title and abstract screening of the citations retrieved from the database searches were conducted by one reviewer and approximately, 20% for consistency and accuracy by a second reviewer, therefore some potentially relevant studies might have been missed. All full text screening was conducted by one reviewer and checked by a second reviewer. Critical appraisal of all evidence was conducted by one reviewer and checked by a second reviewer.

As detailed above due to the quality of included studies, heterogeneity of interventions, outcomes and outcome measures used, it is not possible to provide strong implications for policy and practice on which interventions aiming to reduce fuel poverty in rural areas are the most beneficial. While, this review has its limitations, the included evidence consistently shows that energy efficiency measures might be beneficial to keep houses warm and alleviate fuel poverty, indicating potential for these interventions.