Greater use of randomised trials could help the uptake of energy efficiency by identifying which policy interventions work best.
More efficient use of energy is high on almost everyone’s list of good ways to reduce CO2 emissions. It can lead to large scale emissions reductions, is often cost-effective, and tends to be highly compatible with other policy goals such as energy security.
Efficiency standards for buildings, vehicles and appliances have played a critical role in improving energy efficiency, and will continue to do so. But standards are not the whole story. Rates of uptake of more efficient technology and processes and other changes in consumers’ behaviour can matter greatly.
However it is often impossible to know in advance how innovative policy interventions will affect behaviour. Consumers’ responses to novelty are unpredictable, and judging likely response is further complicated because consumers’ circumstances are often complex and varied. Even afterwards it may be difficult to judge whether an intervention has been effective because it’s impossible to say what would have happened otherwise.
Fortunately there are models from elsewhere that can help address these issues. A well proven means of judging the effectiveness of interventions is the use of randomised trials, in which one group is subject to an intervention and a similar control group is not. These trials look to avoid biases such as self-selection, for example where those most interested in something may participate disproportionately.
Double blind randomised control trials for new drugs form a benchmark for such tests. This approach requires two groups to be chosen differing only in whether they have a new drug or a placebo, with neither the patient nor the physician being aware who is getting which. Provided that all such studies of each new drug are published – a controversial area – there are two comparable groups, and valid statistical inferences can be drawn about whether the drug has been effective.
The double blind element of medical treatments is not always easy to reproduce in other fields, but the use of controlled trials is common in other areas. Technology companies often roll out two different versions of software online to subsets of users to see which gets the best response, as measured for example by click-through rates. This approach allows decisions to be data driven rather than based on judgement or experience. Tests on users may be ethically controversial, as Facebook found with experiments to its news feeds. And outcomes are not always desirable from the consumer’s point of view, for example when an option to turn down an offer is less visible on screen, even if few people want the offer. But effectiveness will likely have been demonstrated, at least for major websites.
Development organisations have used similar approaches in looking at uptake, for example testing different ways of increasing uptake of immunisation programmes [1].
Data driven decision making of this sort is often contrasted with traditional decision making based on the judgement of someone senior, which is sometimes referred to as HIPPO based decision making (Highest Paid Person’s Opinion). It also goes beyond a vague requirement for evidence based policy making, in that it requires a certain type of evidence to be gathered. This reduces the often-noted risk that evidence based policy-making turns into policy based evidence-making.
Controlled trials are now beginning to be used to test interventions designed to increase energy efficiency. In a trial in Norway [2] the labelling of appliances was changed to make it more meaningful to consumers. Labels in some stores showed lifetime electricity running costs and improved staff training while the control groups had labelling showing only annual kWh and no training. For fridge-freezers no significant effect was found. For tumble dryers the combined label and training reduced average energy use of tumble dryers sold by 4.9% while training alone led to a 3.4% reduction. The effect was strongest initially, but declined over time.
A similar change of labelling was undertaken in the UK in a joint study by John Lewis department stores working in collaboration with the Department of Energy and Climate Change (DECC) [3]. A statistically significant but small effect (0.7% increase in efficiency of appliances sold) was observed. Another, earlier, study on interventions for households with difficulties affording enough energy found no reduction in bills, but an increase in comfort [4].
It is encouraging to find such approaches beginning to be adopted. However they appear to remain very much the exception not the norm. There are many other areas where such trials could make a large contribution. Smart metering in particular could benefit from this. There are many options for both design and use of smart meters. It is far from clear which will work best. Trials are needed to find out. Although there have been a few such studies [5] many more are needed.
Such trials are not as cheap or easy as making a judgement about what will work and hoping for the best. And they represent a high hurdle for interventions to clear. But they are more robust as a result, and should lead to more effective (and cost-effective) outcomes. Controlled trials need to become more widespread if energy efficiency is to make a full contribution to reducing emissions.
Updated 18th April 2015
Notes:
[1] The use of controlled trials to look at poverty alleviation and development is described (among other topics) by Abhijit Vinayak Banerjee and Esther Duflo in their book Poor Economics and more concisely and relevantly by Duflo in the accompanying TED talk.
[2] Kallbekken et al. “Bridging the Energy Efficiency Gap: A Field Experiment on Lifetime Energy Costs and Household Appliances” Journal of Consumer Policy, 2013 http://link.springer.com/article/10.1007%2Fs10603-012-9211-z#page-1
[5] See for example:
http://www.ieadsm.org/Files/Content/Goette.pdf
Click to access energy-demand-research-project-final-analysis.pdf
https://ideas.repec.org/a/eee/eneeco/v45y2014icp234-243.html
On Climate Change Policy 