See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/261986659

The building energy performance gap: up close and personal

Conference Paper · April 2014

CITATIONS READS

15 2,090

2 authors:

Pieter de Wilde Rory Jones

University of Strathclyde University of Reading
196 PUBLICATIONS 4,258 CITATIONS 47 PUBLICATIONS 1,132 CITATIONS

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Climate Change Adaptation View project

The relationship between quality management and the thermal performance of buildings. View project

All content following this page was uploaded by Rory Jones on 07 January 2016.

The user has requested enhancement of the downloaded file.

 The building energy performance gap: up close and personal

Pieter de Wilde and Rory Jones

Building Performance Analysis Group, School of Architecture, Design and
Environment, Plymouth University
Roland Levinsky Building, Drake Circus, PL4 8AA, Devon, United Kingdom
[email protected], [email protected]
+44 (0)1752 586115, +44 (0)1752 585198

Abstract
Measurements of completed projects confirm significant gaps between the predicted
and actual energy performance of buildings. This is due to: actual occupant
behaviour; weather conditions; workmanship/installation errors; systems’ control
settings and modelling issues. Recent developments in automated meter reading
(AMR) and monitoring and targeting (M&T) make the performance gap visible to
owners/operators. Bridging the gap becomes even more important if the industry
intends to ‘occupant/climate change proof’ buildings. This paper reviews energy
performance gap literature, the findings of a workshop on the subject, and presents
ongoing work in this area. It concludes that the energy performance gap can only be
bridged through better definition and joint efforts across all actors involved in the
design, construction and operation of buildings/building (sub) systems.

Keywords energy performance prediction, energy use measurement, performance

gap

1. Introduction
With energy efficiency targets becoming more stringent and energy prices going up,
there is a growing interest in the building sector in the discrepancy between
predicted and measured energy use. It appears that this difference, between building
energy use as predicted at the building design stage and measured energy use once
a building is operational, is quite significant, in the order of a factor 1.5 to 2. For
evidence, see for instance the CarbonBuzz website (www.carbonbuzz.org) or Turner
and Frankel [1] . This difference between predicted and measured energy
performance is now commonly called the 'energy performance gap'. Various reports
and scientific papers have been published on the issue; see for instance Zero
Carbon Hub [2], Carbon Trust [3] and Menezes et al [4]. . While this might be
interesting for those operating in building science, this energy performance gap is a
serious problem for the industry: it underlines an issue with products of this industry
not meeting quantified ambitions, which is detrimental to customer confidence and
does not help the credibility of the building design and engineering disciplines.
Moreover, if a performance gap already exists for buildings that are designed to
function within today's occupancy schedules and climate conditions, the industry is
even less well-placed to develop buildings that are resilient and robust toward further
changes in use and climate conditions. Without bridging the performance gap the
industry cannot expect to move forward towards new business models such as
performance contracting, where a client pays for a specified indoor climate rather
than for hardware (building and subsystems) with unspecified operation conditions.

This paper addresses the energy performance gap through four main sections. The
first of these provides an overview of recent work on the energy performance gap,
discussing the literature on the subject. The second briefly presents a case study
conducted by the first author, investigating a probabilistic approach to the
performance gap. The third section reports on the findings of a workshop with
experts on the subject. The fourth and final section presents work that is currently
ongoing.

2. Review of present understanding of the energy performance gap

A review of literature on the energy performance gap indicates that there is a whole
range of contributing factors which span the building life cycle from project inception
to building operation.

During the design stage, miscommunication between the client and design team (or
between different actors within the team) about the future performance of the
building can be a root cause for the later performance gap issues [5]. The design
itself might constitute an initial issue, incorporating inefficient systems, wrong or
missing construction details, or lack simplicity and buildability. During design, a
significant contributing factor is that it is hard to predict the future occupancy and use
of the building and the control regimes that will be applied to key services [4]. The
drive towards energy efficient buildings also leads to a tendency to include Energy
Saving Technologies (EST) in many buildings; often these have teething problems
leading to a performance gap once the building is operational [6]). In many cases
EST do not meet the manufacturer’s performance specifications and are subject to
degradation over time.

Predictions made in the design stage fundamentally rely on analysts making sound
use of models, calculations and software tools to quantify future energy use.
Obviously this requires the use of appropriate tools and models and adequate
training of the analyst. However, any prediction inherently includes some degree of
uncertainty. Testing, validation and verification in the field of building energy
modelling are emerging areas that still need further development [7][8].

The actual construction process also contributes to the energy performance gap [9].
Achieving the required insulation and airtightness levels are sometimes challenging;
errors and defects might be hidden from view due to the fact that constructions are
typically layered. There are also direct impacts of change orders and value
engineering. Where change orders might appear to substitute equivalent products
these might in fact not be so from a detailed thermal point of view. Value engineering
might actually remove elements of the thermal system that are seen to be overly
expensive but which were critical in achieving a target performance. Building
commissioning and hand-over are also difficult processes that typically do not allow
for full performance testing due to budget and time constraints [10].

Once a building becomes operational, a key issue is that actual building use and real
weather conditions seldom match assumptions made during the design process.
Control settings of thermostats and within the Building Energy Management System
(BEMS) might not represent assumptions, or simply might not be programmed as
intended. Furthermore, one also needs to accept that metering itself comes with
issues and uncertainties [11]; this is especially true when it comes to capturing
contextual factors such as weather data and occupant behaviour. Measurement can
often have issues with accuracy, missing or incomplete data, as well as implausible
values, which lead to a ‘level’ of error in the results collected from metering. Post-
processing and cleaning of metering data is therefore essential, but can introduce
further threats to the validity of the results.

3. A probabilistic view of the performance gap

Most work on the performance gap is based on deterministic predictions and
measurements. Work at Plymouth University has piloted a probabilistic approach.
This was based on the premise that both predictions and measurements ought to
capture uncertainties and hence be represented by probability distributions or
histograms. If these could be established one might then study the equality between
prediction and measurement through a statistical test, such as the two-sample
Kolmogorov-Smirnov (K-S) test [12].

This concept was tested by means of a pilot study which focussed on the Roland
Levinsky Building at Plymouth University. An impression of the full complexity of this
building both in terms of geometry and services is provided by Figure 1. A detailed
EnergyPlus model of this building has been developed which will be used to obtain
'as simulated' data; this model includes 105 zones and the EnergyPlus "Input Data
File" (IDF) comprises 41,614 lines of text and has been used in previous studies [13].
Thise IDF model describes the geometry, construction materials, and control settings
of the building. Access was gained to two years of full automated meter readings
(electricity and gas) at 30 minute intervals for the building, thus providing the 'as
measured' component. This has been compared with simulation results from an
EnergyPlus model of the same building.
Figure 1: Roland Levinsky Building at Plymouth University, UK

In order to propagate design uncertainties in the EnergyPlus model, this was linked
to the GeorgiaTech Uncertainty and Risk Analysis Workbench GURA-W [14].

In studying the need for uncertainty propagation through GURA-W, it became

obvious that the comparison between energy efficiency prediction at the design
stage and measurements on a real building needs to cover a lot of different types of
uncertainties, such as intentional but unanticipated changes (due to value
engineering and change orders), errors during construction, variation in material
properties and dimensions, physical coefficients, occupant behaviour, control
settings, weather conditions, system performance degradation, impact of facility
management and others. For some of these, such as the uncertainty in material
properties, significant bodies of literature are available and uncertainty propagation is
feasible. For others, such as the impact of value engineering on the properties of the
final product, this is at present not possible due to lack of information. Results
obtained with GURA-W, propagating only a limited set of those parameters which
are at present quantifiable (still covering uncertainties in no less than 120 different
model parameters), are presented in Figure 2. For further background on this work,
see de Wilde et al [15].
Figure 2: Histograms for prediction of annual consumption of (left) gas for
heating, (middle) annual HVAC electricity use and (right) total annual
electricity use

These results were compared with the existing measurement data. This revealed
another problem: while it is relatively straightforward to develop a histogram for
simulated data, this is not the case for metered data. Two years of data, even when
captured at 30 minute intervals, still only give two data points at an annual level.
Increasing the resolution to monthly level does not solve this problem as it shifts
emphasis away from the quantity of interest. Figure 3 shows this effect, with actual
values represented in Table 1.

Figure 3: Comparison of histograms with actual measurements, which only

comes in single points. (Left) gas for heating, (middle) annual HVAC electricity
use and (right) total annual electricity use.

It is obvious from both figure and table that the predicted values of gas are far below
the actual energy consumption. For electricity, the HVAC values slightly over predict,
but this effect disappears when looking at the total electricity consumption.

Gas Elec HVAC Elec total

(kWh/annum) (kWh/annum) (kWh/annum)
Simulated 2.75 ± 0.50 x 105 3.80 ± 0.80 x 105 1.23 ± 0.08 x 106
2011 8.81 x 105 1.20 x 105 1.19 x 106
2012 6.68 x 105 1.29 x 105 1.21 x 106

Table 1: Comparison of simulated with metered data.

A similar effect, and similar discrepancies between predicted and measured results,
have been reported from other studies (Cherry Building in Atlanta, USA; PhD study
in progress). Better correspondence can be obtained by re-running the simulations
and correcting the design assumptions for weather and occupant behaviour with
actual observations, thus compensating for some of the external unknowns at design
stage. If all other uncertainties are left unchanged, the crude picture that emerges is
as follows:
 Energy use predictions at design stage, where the weather and occupant
behaviour must be guessed, can be as far away from measurement values as
a factor three.
 After re-running models with weather data and occupant behaviour that are in
line with the measurement context, models will be within roughly 40% of
metered values.
At present we do not have sufficient observational data to construct a detailed
distribution that allows the application of a meaningful statistical test. Also note that,
even with two points only, the resulting performance gap will be different from year to
year and thus varies over time.

4. Current status and opinions

The current views on the energy performance gap within the UK industry were
discussed at a workshop at Plymouth University on 25 October 2013; see Figure 4.
This workshop consisted of presentations by invited speakers as well as a forum
discussion. Access to the workshop was available for a small admission fee; with the
event having been announced widely through networks like IBPSA-England (the
regional affiliate of the International Building Performance Simulation Association)
and Constructing Excellence. The delegates were self-selected.

Figure 4: Delegates at the Plymouth University workshop, 25/10/2013. Image by

Jim Carfrae

An overview of the presentations on the day is presented in Table 2.

 Speaker Affiliation Title of presentation
P. de Plymouth University, UK Introduction and overview of the energy
Wilde performance gap
M. Colmer Technology Strategy Building Performance Evaluation: early
Board, UK programme findings
A. de AECOM, UK Predicting operational energy use at
Menezes design stage - can it be done?
C. University of Exeter, UK Automated meter reading, monitoring
Martinez- and targeting and machine learning -
Ortiz bridging the energy performance gap
D. University College Bridging the credibility gap: total
Mumovic London, UK performance of school buildings
R. Bunn Building Services Whose performance gap?
Research and
Information Association,
UK
R. Quincey Integrated Bridging the performance gap: IES and
Environmental VE Scan
Solutions, UK
D. Leeds Metropolitan Bridging the fabric performance gap
Johnston University, UK
D. Miles- Leeds Metropolitan Measuring the fabric performance gap
Shenton University, UK
W. Plokker VABI, the Netherlands The impact of air supply temperatures
on the performance gap
P. de Plymouth University, UK A probabilistic view of the performance
Wilde gap

Table 2: Speakers at Performance Gap Workshop, Plymouth, 25/10/2013

The presentations during the day and the forum discussion at the end of the
workshop highlighted the following issues and views on the performance gap:
 There are clearly different types of performance gaps that vary over time and with
context, and that depends on the point of view of those looking at building
performance. One might, for instance, define the energy gap as the difference
between the design at a conceptual stage and as measured once operational, but
just as well as the performance of the design at a stage where detailed
constructions have been prepared and the project goes out to tender and as
measured once operational; more often than not this is not fully defined. Also,
apart from the energy performance gap one might look at an air quality
performance gap, lighting performance gap, and others.
 There clearly is a strong tension between the energy performance certificates
(EPC) produced by Standard Assessment Procedures (SAP) and Simplified
Building Energy Model (SBEM) calculations, and display energy certificates
(DEC) based on measured energy use. It is rather unfortunate that both result in
very similar colour coded ratings with categories from A to G. This in spite of the
fact that EPC and DEC differ fundamentally in the energy usages that are being
covered. While the work by CIBSE TM54 goes some way towards bridging this
issue, the fundamentals are hard to convey to the general public.
 Various speakers raised the issue of communication and perception and whether
the energy performance gap is in fact a perception gap rather than one of
discrepancy between predicted and measured energy use.
 There is an issue to be addressed in terms of who can be made responsible for a
performance gap and bears the risk of future litigation. Aligned with this issue is
the question of who should take the initiative to bridge the performance gap. On
the one hand there appears a challenge for building designers and engineers to
make better predictions, or for building scientists to develop better prediction
tools. On the other hand the findings of the day suggest that the performance gap
is due to a multitude of underlying factors and that bridging the gap therefore will
require collaboration of all actors involved in the design, construction and
operation of buildings and building (sub)systems.
 It appears that most research into the energy performance gap focusses on non-
domestic buildings; dwellings seem to be overlooked in this discussion. This
might be due to the fact that domestic buildings are less likely to be subject of
transient building simulation and advanced metering/monitoring; however, it risks
missing out on a key sector of buildings.

5. Ongoing and future work

Ongoing work by the authors is concentrated in two key areas. Firstly, further work
continues on the Roland Levinsky case study building; secondly, a domestic line of
investigation is currently under development.

The work on the Roland Levinsky Building at Plymouth University is moving forward
in several ways. Further measurement data continues to be captured from the
building energy management system and energy metering. Additionally, work is
under way to capture actual use of the building in more detail. Analysis will compare
and contrast the current system control settings, timetabled activities within the
building, and surveys of actual building utilisation. Findings will be used to start a
campaign of calibration of the existing EnergyPlus model, while keeping track of all
calibration activities along the way using version control software as recommended
by Raftery et al [16]. Furthermore, there are initial contacts in place that will lead to a
review of the original energy calculations and simulations that were done at the
design stage of the building by the actual engineering team, which can be compared
with the parallel efforts of the academic research team.
In terms of the performance gap in the domestic part of the building sector, work has
started on the evaluation of homes on a new-build development in Torquay, Devon;
see Figure 5. In this case, monitoring equipment has been installed in a series of
identical properties as far as construction and building systems installed are
concerned. Data capture includes 5 minutely gas and electricity usage, indoor air
temperature and relative humidity in different rooms, occupancy of the dwellings and
window and door opening. Climate data is being captured for the whole site. The
monitoring will be undertaken for a minimum of three years and the data is being
exported off site in near real time via general packet radio service (GPRS). The
research will include a socio-technical household survey with the building occupants
to gather information about their socio-economic characteristics, comfort, ownership
of domestic appliances, environmental attitudes and behaviour. A detailed post-
occupancy evaluation will be completed on the monitored homes and researchers
will work with the occupants to improve understanding and operation of the energy
saving technologies installed, such as the mechanical ventilation and heat recovery
(MVHR) system. Thus this work will provide a data rich context to study how the gap
between predicted and measured energy consumption for these properties develops
over time.

Due to the fact that the monitoring relates to identical properties, this project also
provides a unique opportunity to develop an energy use distribution that reflects the
impact of design independent factors to the performance gap, such as occupant
behaviour, variation in plug in equipment, and others. On the design side, the team
has access to the SAP calculations done at the design stage. These will be
compared to transient EnergyPlus simulations of the same building design,
conducted by the researchers. This allows for model calibration.

The variations in energy demand between the identical monitored homes will be
correlated with occupant behavioural factors, such as the frequency and duration of
window and door opening, daily heating period, chosen heating set point
temperatures, proportion of home heated and operation of the MVHR system.

Figure 5: Monitored domestic development in Torquay, Devon. Image by Rory

Jones
6. Conclusions
This paper has summarized the existing literature on the energy performance gap,
discussed recent work at Plymouth University, as well as the findings from a
workshop held on the subject, and has presented some avenues of ongoing and
future work in this area. The key conclusions of this paper are:

 There are different types of energy performance gap that vary over time and
with context. One can also extend beyond the ‘energy performance gap’ and
look at issues of indoor environmental quality. A clear definition of the
performance gap needs to be devised to account for these emerging multi-
faceted concepts and interpretations. Without personalizing the 'performance
gap' and positioning it in a clearly defined context the concept is not
meaningful.
 In the UK, there clearly is a strong tension between the energy performance
certificates (EPCs, based on calculations) and display energy certificates
(DECs based on measured energy use). As EPCs and DECs are presented in
a very similar format but represent different figures, with EPCs not including
unregulated energy flows, this fuels the national debate on the energy
performance gap.
 An emerging debate relates to the communication and perception of the
energy performance gap. It is suggested by some that in fact the energy
performance gap is rather a perception gap rather than one of discrepancy
between predicted and measured energy use. However, other evidence
seems to indicate that there is a need to fundamentally review the way we
currently predict and measure performance. This view especially holds true in
terms of accounting for uncertainties.
 Although significant recent developments in automated meter reading and
building monitoring are providing actual energy consumption and
environmental data for buildings, the amount and types of data required to
clearly understand the root causes of the energy performance gap are still
severely lacking.
 At present, the responsibility for the performance gap has not been
apportioned to different actors in the design, construction and operation
stages of the building process and therefore it is unclear who should take the
initiative to bridge the performance gap.
 Most research into the energy performance gap focusses on non-domestic
buildings; dwellings seem to be overlooked in this discussion, therefore a
whole key sector of buildings is currently overlooked and the extent of the
performance gap in this sector remains unclear. The authors are currently
undertaking a domestic performance gap project in Torquay, Devon, to
address this need, with results due in Summer 2014.
References
1 Turner C and Frankel M, Energy performance of LEED for new construction
buildings (Final Report), New Buildings Institute, 2008.
2 Zero Carbon Hub, A review of the modelling tools and assumptions: Topic 4,
closing the gap between designed and built performance, Zero Carbon Hub, 2010.
3 Carbon Trust, Closing the gap: Lessons learned on realising the potential of low
carbon building design, Carbon Trust, 2011.
4 Menezes C, Cripps A, Bouchlaghem D and Buswell R, Predicted vs. actual energy
performance of non-domestic buildings: Using post-occupancy evaluation data to
reduce the performance gap. Applied Energy, 2012, 97: 355-364.
5 Newsham G, Mancini S and Birt B,, Do LEED-certified buildings save energy? Yes,
but... Energy and Buildings, 2009, 41: 897–905.
6 Newsham G, Birt B, Arsenault C, Thompson L, Veitch J, Mancini S, Galasiu A,
Macdonald I and Burns G, Do green buildings outperform conventional buildings?
Indoor Environment and energy performance in North American offices (Report RR-
329), Natural Research Council Canada, 2012.
7 Reddy T, Maor I and Panjapornporn C, Calibrating detailed building energy
simulation programs with measured data - Part I: general methodology, HVAC&R
Research, 2007, 13(2): 221-241.
8 Ryan E and Sanquist T, Validation of building energy modelling tools under
idealized and realistic conditions, Energy and Buildings, 2012, 47: 375-382.
9 Bell M, Wingfield J, Miles-Shenton D, Seavers J, Low carbon housing: lessons
from Elm Tree Mews, Joseph Rowntree Foundation, 2010.
10 Bunn R and Way M, Soft Landings, Building Services Research and Information
Association and Usable Building Trust, 2010.
11 National Measurement Network, The building performance gap - closing it
through better measurement, National Physical Laboratory, 2012.
12 Field A, Discovering statistics using SPSS (2nd ed.) Sage Publications Ltd., 2005,
London, UK.
13 Tian W and de Wilde P, Uncertainty and sensitivity analysis of the performance of
an air-conditioned campus building in the UK under probabilistic climate projections,
Automation in Construction, 2011, 20: 1096-1109.
14 Lee B, Sun Y, Augenbroe G and Paredic C, Towards better prediction of building
performance: A Workbench to propagate uncertainties through building simulation
models, Building Simulation '13, 13th International IBPSA Conference, Chambéry,
France, August 25-30, 2013.
15 de Wilde P, Y Sun and G Augenbroe, Quantifying the performance gap – a
probabilistic attempt, In Suter et al, eds, EG-ICE 2013, Conference on Intelligent
Computing in Engineering, Vienna, Austria, July 1-3 (USB) 2013.
16 Raftery P, Keane M and O'Donnell J, Calibrating whole building energy models:
An evidence-based methodology, Energy and Buildings, 2011, 43: 2356-2364.
 Acknowledgements
The work by described in this paper has been part-funded through a Royal Academy
of Engineering and Leverhulme Senior Research Fellowship, reference number
10226/48 (de Wilde) and the Engineering and Physical Sciences Research Council
(EPSRC) under the Transforming Energy Demand in Buildings through Digital
Innovation eViz project (grant reference EP/K002465/1).

View publication stats