aerospace

Article
Implementation of a Holistic MCDM-Based Approach to Assess
and Compare Aircraft, under the Prism of Sustainable Aviation
Dionysios N. Markatos * and Spiros G. Pantelakis

Laboratory of Technology & Strength of Materials, Department of Mechanical Engineering & Aeronautics,
University of Patras, 26500 Patras, Greece
* Correspondence: [email protected]

Abstract: Sustainability represents a key issue for the future of the aviation industry. The current
work aims to assess and compare aircraft, under the prism of sustainable aviation. In the proposed
approach, sustainability is understood as a trade-off between technological sustainability, economic
competitiveness/costs, and ecological sustainability, with the latter also including circular economy
aspects. To handle the trade-offs and lead to an effective decision, a multi-criteria decision-making
(MCDM) methodology is applied, combining the analytic hierarchy process (AHP) and an appropriate
weighted addition model. To demonstrate the proposed approach, a set of commercial aircraft
incorporating novel fuel/propulsion technologies are compared and ranked with regards to their
sustainability, using the metric of sustainability introduced. The dependency of the obtained ranking
on the significance attributed to each of the sustainability aspects considered was also performed and
discussed. To verify the reliability of the proposed approach, the obtained results are also compared
with those obtained from a popular ranking tool from the literature.

Keywords: sustainability; circularity; sustainable aviation; aircraft selection; MCDM; decision-

making support; holistic approach

Citation: Markatos, D.N.; Pantelakis, 1. Introduction

S.G. Implementation of a Holistic Sustainability represents one of the great challenges of our time. However, in the
MCDM-Based Approach to Assess literature, a variety of interpretations of sustainability may be found, e.g., in [1]. In addition,
and Compare Aircraft, under the sustainability is mostly understood as a qualitative term. Nevertheless, as the public’s
Prism of Sustainable Aviation.
desire for sustainable solutions grows stronger, so does the need to comprehensibly define
Aerospace 2023, 10, 240. https://
and accurately assess sustainability.
doi.org/10.3390/aerospace10030240
Sustainability is of major concern also for the aviation industry, while the Green Deal
Academic Editor: Konstantinos targets for decarbonization have put the sector under increasing pressure [2]. So far, sus-
Kontis tainability in aviation is mainly linked to the reduction of greenhouse gas emissions during
operations; therefore, the focus of research activities mainly lies on solutions associated with
Received: 5 January 2023
new propulsion and fuel technologies and the reduction of fuel consumption [2]. Although
Revised: 20 February 2023
Accepted: 27 February 2023
the latter is an essential part to achieve a carbon-neutral aviation, this approach covers
Published: 1 March 2023
only a part of the aircraft overall sustainability, as other material-related and fuel-related
aspects, such as production and manufacturing, end-of-life and circularity, maintenance,
and economic and socio-economic aspects, appear to be mainly neglected in the lifecycle
assessment of an aircraft [3,4].
Copyright: © 2023 by the authors. In the above context, aircraft selection emerges as an important issue towards achieving
Licensee MDPI, Basel, Switzerland. long-term sustainability goals in the aviation industry as the selection of an aircraft to
This article is an open access article complement the airline fleet represents a critical issue in transportation planning [5]. With
distributed under the terms and the aviation sector being significantly affected during the last years, mainly due to the
conditions of the Creative Commons
requirements for carbon-neutrality and a sustainable and circular aviation sector, the
Attribution (CC BY) license (https://
number of criteria under consideration when selecting an aircraft has increased; therefore,
creativecommons.org/licenses/by/
the determination of appropriate criteria in the aircraft selection process is of utmost
4.0/).

Aerospace 2023, 10, 240. https://doi.org/10.3390/aerospace10030240 https://www.mdpi.com/journal/aerospace

Aerospace 2023, 10, 240 2 of 13

importance [6]. However, the criteria considered in the implemented MCDM for the
evaluation, selection, or ranking of alternative aircraft, differ. Literature reviews on aircraft
selection, conducted in the frame of recent studies, suggested mostly technical, operational,
and economic criteria [5–12]. Moreover, it is worth noticing that several criteria considered
might be conflicting with regards to their impact on sustainability. As an example, the
increased use of composites to reduce weight and hence fuel consumption within the
mission increases the problems related to achieving the goal of circular aviation, due
to the challenges which are associated with their recycling. The problem becomes more
pronounced by accounting for the fact that the principal type of composites used in aviation
are thermoset-based. To tackle the multiple conflicting criteria involved when selecting
among several alternatives, simplify the assessment process, and support decision-making,
multi-criteria decision-making (MCDM) methods have emerged as the most employed
approach, inside and outside the aviation sector [5–18].
On the other hand, although the significance of environmental aspects has been mean-
while well-understood, less attention has been paid so far to implementing environmental
criteria when selecting an aircraft, while the studies addressing sustainability aspects when
aircraft selection is a matter of discussion appear to be scarce [10]. In the context of ensur-
ing environmental sustainability in the aviation sector, an increasing interest in emerging
aircraft technologies has been seen during the last years. Among these technologies, novel
fuel technologies, such as sustainable aviation fuels (SAFs), fully electric, hybrid-electric,
hydrogen propulsion systems, or hybrid-hydrogen systems (e.g., liquid hydrogen com-
bustion assisted by fuel cells), demonstrate the potential to enable a more efficient and
sustainable aviation [4,19,20]. The inherent differences and characteristics of such technolo-
gies generate new requirements for aircraft design, involving new lightweight materials
and structures, new propulsion architectures, and relevant aerodynamic technologies. Con-
sequently, the assessment of the impact of different integrated technologies at the aircraft
level is expected to result in several key performance indicators (KPIs) and lifecycle-related
metrics. As the above-mentioned KPIs refer to a wide spectrum ranging from costs to
emissions, performance/technological features, etc., their individual assessment may lead
to contradicting conclusions. Hence, the overall impact of a new technology needs to be
assessed in an interdisciplinary and holistic manner to derive reasonable conclusions for
the future aviation industry and its sustainability.
In the above frame, the novelty of the current work lies on the effort to assess and
compare aircraft, under the prism of sustainable aviation. In this context, the present work
is proposing a holistic interpretation of sustainability, where sustainability is understood
as being a trade-off between performance, economical, and ecological aspects, with the
latter also including circular economy considerations. To assess the sustainability of the
considered aircraft, a MCDM-based model/tool implemented by the authors at the com-
ponent level [21–23] is enhanced and adapted to the aircraft level. The MCDM-based
model combines the analytic hierarchy process (AHP) to derive the weights of potentially
contradicting criteria and aspects as well as an appropriate aggregation method, namely the
weighted sum model (WSM), to integrate relevant indicators into a single index, reflecting
a trade-off between the above-mentioned criteria. The advantage of integrating the WSM
into the proposed hybrid tool is that it offers a proportional linear transformation of the
raw data, namely the relative order of magnitude of the standardized scores remains equal.
The latter allows for a more effective and comprehensible interpretation of the final ranking
obtained as well as for distinguishing the impact of each term on the final ranking. The
introduced sustainability index aims to support the justification of a decision under the
prism of holistic sustainability, and the index can be understood as a metric of sustain-
ability and is used to obtain the ranking among the alternative aircraft. To illustrate the
proposed approach, a set of commercial passenger aircraft, of similar range, incorporating
novel aircraft fuel/propulsion technologies, were selected and ranked. The dependency
of the ranking obtained on the significance attributed to each of the sustainability aspects
considered was also assessed and discussed. Finally, the results are compared with those
Aerospace 2023, 10, 240 3 of 13

obtained by applying a popular ranking MCDM tool from the literature, namely TOPSIS,
in order to validate the reliability of the proposed approach.

2. Methodology
2.1. Definition of Sustainability
For the aviation industry to ensure that the sector is future proof, the development of
a new aircraft should be assessed on the basis of its impact towards a sustainable, circular,
and climate-neutral aviation. The latter should also apply when an aircraft technology is
the matter of discussion. Nevertheless, either for selecting a new aircraft among alternatives
or for assessing the impact of a new technology, a clear and comprehensive definition of the
term sustainability is required. Currently, sustainability in aviation is interpreted mainly
from the environmental point of view and is linked to the development and implementation
of novel energy carriers and fuels. However, this approach falls short of observing the
whole sustainability picture. To overcome this shortcoming, in the present work, the
following definition of sustainability at the product level is proposed: a product (or a
technology) can be considered as sustainable if it is competitive in terms of performance, as
compared to other similar products/technologies available in the market, and additionally
is sustainable from the economic and ecological viewpoint, with the latter including both the
environmental impact and circularity. In the proposed interpretation, sustainability emerges
as a matter of trade-off between potentially contradicting aspects, linked to performance,
economic, and ecological criteria. To this end, appropriate metrics related to the said
aspects will be selected and exploited for the assessment of the overall sustainability of
the investigated aircraft. Such metrics are often contradicting, and therefore, trade-offs
among them will be addressed. In this frame, to assess and handle such trade-offs, the
aircraft selection process has been viewed as a multi-criteria decision-making (MCDM)
problem. The latter allows for the calculation of an index and is understood as a metric of
sustainability. Said metric is calculated for each of the examined aircraft towards supporting
the user to select the most sustainable aircraft. The tool implemented for the calculation of
said sustainability metric is a hybrid one: it combines the analytic hierarchy process (AHP)
and a weighted sum model (WSM). AHP is implemented to derive the weights attributed
to the metrics considered, while WSM is used to make the aggregation of the metrics into a
single index. Output is a weighted sum of the normalized individual metrics. Details about
the AHP and WSM can be found in Section 2.3. The advantage of selecting the use of the
WSM into the proposed hybrid tool is that it offers a proportional linear transformation of
the raw data, namely it maintains the relative order of magnitude of the standardized scores.
The latter allows for a more effective and comprehensible interpretation of the final ranking
obtained as well as for distinguishing the impact of each of the considered metrics to the
final output. The involved calculations were performed via a spreadsheet (excel-based)
model. More details about the applied MCDM tool are presented in Section 2.3. To evaluate
the proposed approach, four different aircraft types have been compared. As reference
aircraft, A320 neo fueled by kerosene has been chosen. This has been compared with
three alternative aircraft, namely, a A320 neo aircraft fueled by SAF, and two conceptual
hydrogen-powered (LH2) aircrafts fueled either by blue hydrogen or green hydrogen.

2.2. Determination of Criteria and Metrics to Assess Aircraft Sustainability

When selecting an aircraft, the choice of the involved criteria and their respective met-
rics can have a significant effect on the ranking results among the aircraft types compared.
Based on the interpretation of sustainability as introduced in Section 2.1, the criteria of
technological performance, economic competitiveness/costs, environmental impact, and
circularity need to be considered as being the relevant criteria when aircraft sustainability
is assessed.
Aerospace 2023, 10, 240 4 of 13

2.2.1. Technological Performance Metrics

In the proposed approach, technological performance is represented through the
metrics of maximum payload and fuel intensity. Maximum payload capacity is defined as
the sum of the passenger, baggage, cargo, and postage loads, which constitute part of the
takeoff weight of a plane [11]. It can also be expressed as the total number of seats available
(seating capacity) for a given range and can be obtained from a payload-range diagram.
Seating capacity represents a very important selection criterion for the airlines, and it is not
uncommon that this criterion is highly prioritized over other selection criteria [24]. In the
current study, the maximum payload capacity is related to the number of seats available,
for which a median mission of 1500 km has been considered.
The fuel intensity of the aircraft is expressed as the energy required per RPK, where
RPK stands for revenue passenger kilometer. Fuel intensity is expressed as MJ/RPK and
varies with mission length and aircraft type. RPK is formulated as the total of revenue-
paying passengers times the distance travelled in kilometers and represents the most
acceptable normalizing metric in aviation. Since RPK measures the actual demand for air
transport, it is also referred to as ‘airline traffic’. Consequently, the fuel intensity or the fuel
consumption represents a critical criterion when an aircraft selection is considered.

2.2.2. Costs (Economic Competitiveness)

Costs considered to express the economic aspect of the sustainability are: (i) fuel price
(in EUR (€)/RPK), (ii) other direct operating costs (DOC), namely maintenance, labor, and
ownership costs, and (iii) aircraft purchase cost, which is the investment cost to purchase
an aircraft.

2.2.3. Environmental Impact Metrics

The metrics considered to express the environmental impact aspect of sustainability
are: (i) CO2 emissions (kgCO2 eq./RPK), (ii) NOx emissions (ppm/RPK), and (iii) contrails
(radiative forcing units).

2.2.4. Circularity Metrics

In the context of the transition towards a circular economy, the proper consideration
of the relationship between sustainability and circularity is mandatory. The term ‘circular
economy’ appeared for the first time in 1988 [25]. A variety of interpretations have been
proposed [26,27], and they lead to varying metrics and indicators in both form and con-
tent [25]. It is noteworthy that there are over 100 definitions of circular economy within the
literature, with majority of them focusing on materials’ preservation [28–30]. The prevailing
interpretation of circularity in aviation is the percentage of the components’ mass which can
be reused or recycled after end-of-life (EoL) of the aircraft. In certain cases, this percentage
could reach up to 85% [31,32]; however, the latter percentage regards previous-generation
aircraft, where the aircraft structure is mainly composed of metallic materials, mainly
aluminum. This interpretation has also been adopted in the present work. However, in
the above interpretation of circularity, the performance features of the recycled products,
which are an essential aspect in using a recycled product in aviation, are not accounted for.

2.3. The MCDM-Based Approach

The model combines two popular approaches to address evaluation problems, the
analytic hierarchy process (AHP) [33] and the weighted sum model (WSM) [34]. AHP
combined with other MCDM techniques is not uncommon in the literature, e.g., [8,13]. The
output of the proposed analysis is the summation of normalized and weighted individual
indicators. Proportionate normalization was used by dividing each value of the dataset by
the total sum of the dataset. Based on the above, the formula to assess the sustainability of
an aircraft takes the form:

S = K P × P + KC × C + KE × E + KCIRC × CIRC (1)

Aerospace 2023, 10, 240 5 of 13

where S is the quantitative sustainability index, and P, C, E, and CIRC are the normalized
and aggregated indices associated with technological performance, costs, environmental
impact, and circularity, respectively.
For the normalization of the relevant metrics, proportionate normalization was applied.
In proportionate normalization, each value of a dataset is divided by the total sum; in
this way, the normalized values maintain proportionality, reflecting the percentage of the
sum of the total indicator’s values. Following normalization, the normalized values were
multiplied by a weight factor referring to each of the considered criteria of technological
performance, costs, environmental impact, and circularity. The weight factors were derived
through the AHP, as will be described afterwards.
Importance factors (weights) KP , KC , KE , and KCIRC are attributed to the above metrics,
respectively. Said factors are subjective weight factors obtained from the AHP weighting
method, based on the user priorities. The weight factors reflect the importance of each
term to the overall index value. AHP is a widely used method based on the principle of
paired comparisons towards determining the relative priorities of the alternatives with
respect to several criteria [35]. The main steps of the AHP analysis are: (a) define the goal
and alternatives, (b) define the decision criteria, (c) assess the priority of each decision by
use of pairwise comparisons, (d) calculate the weights of the criteria, and (e) analyze the
consistency of the evaluation. The paired comparisons are used to compare the alternatives
regarding to the criteria defined and estimate the criteria weights, on a scale of 1 to 9, where
1 means that the criteria are of equal importance, while 9 means that the selected criterion
is extremely preferred over the criterion to which it is compared. The fundamental scale of
AHP, according to which the comparisons are made, is shown in Table 1.

Table 1. The AHP scale.

Scale Numerical Rating Reciprocal

Extremely preferred 9 1/9
Very strong to extremely 8 1/8
Very strongly preferred 7 1/7
Strongly to very strongly 6 1/6
Strongly preferred 5 1/5
Moderately to strongly 4 1/4
Moderately preferred 3 1/3
Equally to moderately 2 1/2
Equally preferred 1 1

The AHP hierarchy structure of the aircraft selection problem is depicted in Figure 1.
At the first level of the hierarchical analysis process, the overall goal is to select the airplane
type. At the second level, the criteria proposed to select an aircraft are depicted. At the
third level, the relevant sub-criteria linked to the main criteria are proposed. At the fourth
level, the alternative aircraft under comparison are shown. It should be noted that AHP
is implemented simply only for deriving the weights of the criteria considered (level 2
and level 3 of the flowchart in Figure 1), as the aggregation and subsequent ranking is
performed by applying the WSM. More specifically, considering pairwise comparison
matrices such as those that will be presented in Section 3.2, the steps to derive the weight
factors are the following: (a) The columns are normalized so that the sum of all column
values becomes 1. This is achieved by dividing each cell value by the sum of the column’s
cell values. (b) The weight factors are obtained by calculating the arithmetic mean of each
row of the normalized values.
Aerospace 2023, 10, x FOR PEER REVIEW 6 of 13

These factors are the ones used in Equation (1). They are multiplied by the normal-
Aerospace 2023, 10, 240 ized indicators associated with performance, environment, costs, and circularity, in order
6 of 13
to derive the final sustainability index.

Figure 1. The AHP hierarchy structure of the aircraft process.

Figure 1. The AHP hierarchy structure of the aircraft process.
These factors are the ones used in Equation (1). They are multiplied by the normalized
3. Implementation
indicators associated with performance, environment, costs, and circularity, in order to
3.1.derive the final sustainability index.
Case Study
To illustrate the proposed approach, a set of four commercial passenger aircraft were
3. Implementation
considered for comparison: a reference aircraft fueled either by Jet A fuel (kerosene) or
3.1. Case Study
SAF (e-kerosene), and a hydrogen-powered (LH2) aircraft fueled either by blue hydrogen
(producedTo illustrate the proposed
from natural approach, aby
gas and supported setcarbon
of fourcapture
commercial passenger
and storage) or aircraft
green hy-were
drogen (hydrogen produced through a renewable source) [36]. The chosen reference nar- or
considered for comparison: a reference aircraft fueled either by Jet A fuel (kerosene)
SAF (e-kerosene),
row-body turbofan isand a hydrogen-powered
the Airbus A320 neo, while(LH the2 )hydrogen-powered
aircraft fueled either by blue
aircraft hydro-
is a con-
gen (produced from natural gas and supported by carbon
ceptual design model which is a modification of the A320 neo, accommodating the LHcapture and storage) or green
2
hydrogen (hydrogen produced through a renewable source) [36].
storage system, and it is based on Airbus’s ZEROe program. The investigated aircraft and The chosen reference
thenarrow-body
relevant metrics turbofan is the
for each Airbusare
aircraft A320 neo, whileinthe
summarized hydrogen-powered
Table 2. The metrics have aircraft
beenis a
conceptual design model which is a modification of the A320 neo, accommodating the LH2
taken from a recent paper published by the International Council on Clean Transportation
storage system, and it is based on Airbus’s ZEROe program. The investigated aircraft and
(ICCT) [36], in which more details about the considered aircraft can be found.
the relevant metrics for each aircraft are summarized in Table 2. The metrics have been
taken
Table from a recent
2. Investigated paper
aircraft andpublished by the International
metrics considered Council on Clean
for aircraft sustainability Transportation
assessment.
(ICCT) [36], in which more details about the considered aircraft can be found.
LH2 (Narrow- LH2 (Narrow-Body
A320 Neo A320 Neo
Table
Type2. Investigated
of Aircraft aircraft and metrics considered for aircraft
Body sustainability assessment.
Turbofan) Turbofan)
(Kerosene) (SAF)
Blue Hydrogen Green Hydrogen
A320 Neo A320 Neo LH2 (Narrow-Body LH2 (Narrow-Body
Type of Aircraft Performance
Turbofan) Metrics Turbofan)
(Kerosene) (SAF)
Maximum payload/seats Blue Hydrogen Green Hydrogen
180 180
Performance Metrics
192 192
(number)
Maximum payload/seats (number) 180 180 192 192
Fuel intensity (MJ/RPK)Fuel intensity (MJ/RPK) 0.78
0.78 0.78 0.78 1.06 1.06 1.06
1.06
Environmental
Environmental Impact Metrics
Impact Metrics
Carbon intensity
Carbon intensity (kg CO2 eq./RPK) 0.07281 (kg 0.00036 0.03014 0.00045
Nitrogen oxides - 0.07281
- 0.00036 - 0.03014 0.00045
-
Water vapor CO 2eq./RPK)
- - - -
Nitrogen oxides - -
Economic Impact Metrics
- -
Fuel price (EUR (€)/RPK) Water 0.013
vapor 0.049- - 0.044 - 0.041-
DOC - - -
Economic Impact Metrics -
Aircraft purchase costs - - - -
Circularity Metrics
Recycled mass percentage 1 1 1 1
Aerospace 2023, 10, 240 7 of 13

3.2. Calculation of Sustainability Index Values

Based on Equation (1) and the described methodology, a sustainability index was
derived for each of the four considered aircraft and a ranking among them occurred. For
the implementation of the combined MCDM-based approach, a parametric spreadsheet
model was developed, combining the AHP and the linear aggregation (weighted sum)
algorithms. In this model, the user enters the data values associated with the criteria
and the sub-criteria considered for the comparison of the aircraft under study as well
as defines the relevant importance factors for each one of these criteria and sub-criteria.
Finally, a quantitative ranking among the aircraft is derived. It should be noted that
some metrics/sub-criteria, namely NOx emissions, contrails, and circularity, could not
be quantified as there is no sufficient data available. Regarding the circularity metric
considered, i.e., the mass percentage of the reused or recycled components, for the sake of
simplicity, this value has been approximated to 100% for all examined aircraft. In addition,
for the LH2 , the DOCs and aircraft purchase costs are not available. Therefore, to enable a
comparison without influencing its outcome, in the proposed equation of Section 2.3, the
above-mentioned metrics are considered as being equal for the aircraft under consideration.
To derive the weights through the AHP and assess the sensitivity of the ranking
obtained on the significance attributed to each of the sustainability aspects considered, five
different scenarios were considered by defining the different priorities through the AHP
scale of Table 1. The pairwise comparison matrices derived from the AHP analysis and the
resulting weights are shown in Table 3. Regarding the definition of the scenarios considered,
Scenario 1 assumes an equal importance among the considered criteria. Scenario 2 strongly
prioritizes the environmental impact over the other criteria, which are considered of equal
importance. In Scenario 3, costs are strongly prioritized over the other criteria, which are all
considered equally important. Scenario 4 assumes an equal importance between technolog-
ical performance and environmental impact, with both criteria being moderately preferred
over costs and circularity performance. Finally, in a more complex scenario (Scenario 5), en-
vironmental impact and costs are considered equally important, technological performance
and circularity are also considered of equal importance, environmental impact is strongly
preferred over circularity, while costs are strongly preferred over technological performance
and circularity. The spreadsheet model is also capable of accounting for importance weights’
variation among considered sub-criteria (e.g., between the maximum payload and the fuel
intensity), however, for the sake of simplicity, an equal importance among the considered
sub-criteria was considered in all scenarios. Moreover, the consistency ratio (CR) of the
AHP pairwise comparisons, i.e., a measure of how consistent the pairwise comparisons are,
was calculated. The consistency ratio was found to lie between zero and 0.07, where zero
indicates a perfect consistency among the judgments, while any consistency ratio below
the rule of thumb threshold of 0.1 is considered good [33].
Following the weights’ definition and normalization of the datasets, their aggregation
into a single index was performed. Finally, to verify the reliability of the proposed method,
the results have been compared with the ranking obtained from the TOPSIS ranking
tool [37], where the criteria weights have been derived from the AHP and coincide with
those utilized for the hybrid AHP-WSM model. The latter allows for a direct comparison
among the two MCDM tools.

3.3. Results and Discussion

The obtained indices and the ranking of the considered aircraft, as well as the com-
parison with the results obtained through the implementation of the TOPSIS model, are
shown in Table 4. The final rankings are also summarized in Figure 2.
Aerospace 2023, 10, 240 8 of 13

Table 3. Pairwise comparisons of the AHP model.

Scenario 1
Technological Performance Environmental Impact Costs Circularity Weight factor
Technological Performance 1 1 1 1 0.33
Environmental Impact 1 1 1 1 0.33
Costs 1 1 1 1 0.33
Circularity 1 1 1 1 0.33
CR: 0
Scenario 2
Technological Performance Environmental Impact Costs Circularity Weight factor
Technological Performance 1 1/5 1 1 0.625
Environmental Impact 5 1 5 5 0.125
Costs 1 1/5 1 1 0.125
Circularity 1 1/5 1 1 0.125
CR: 0
Scenario 3
Technological Performance Environmental Impact Costs Circularity Weight factor
Technological Performance 1 1 1/5 1 0.125
Environmental Impact 1 1 1/5 1 0.125
Costs 5 5 1 5 0.625
Circularity 1 1 1/5 1 0.125
CR: 0
Scenario 4
Technological Performance Environmental Impact Costs Circularity Weight factor
Technological Performance 1 1 3 3 0.375
Environmental Impact 1 1 3 3 0.375
Costs 1/3 1/3 1 1 0.125
Circularity 1/3 1/3 1 1 0.125
CR: 0
Scenario 5
Technological Performance Environmental Impact Costs Circularity Weight factor
Technological Performance 1 1 1/5 1 0.15
Environmental Impact 1 1 1 5 0.32
Costs 5 1 1 5 0.44
Circularity 1 1/5 1/5 1 0.09
CR: 0.07

The results obtained for the hybrid AHP-WSM model, for Scenario 1, suggested the
A320 neo using SAF as the most suitable option. It was followed closely by the LH2 aircraft
utilizing green hydrogen. The obtained ranking identified the A320 neo using SAF as the
most balanced option. This result is understandable as in this scenario, all criteria are
considered equally important and the obtained values for three out of the four criteria
considered for the ranking are superior as compared to the relevant values obtained from
the other aircraft.
When the environmental impact was prioritized over technological performance, costs,
and circularity (Scenario 2), the AHP-WSM tool again suggested the A320 neo using SAF
ranks as the most sustainable. It was followed very closely by the LH2 aircraft utilizing
green hydrogen. This result is also understandable since both aircraft are expected to
present a quite good environmental performance compared to the aircraft using kerosene
or blue hydrogen.
In Scenario 3, where costs were prioritized over technological performance, environ-
mental impact, and circularity, the highest sustainability index was for the A320 neo using
Aerospace 2023, 10, 240 9 of 13

kerosene. This result reflects the appreciably cheaper prices for kerosene fuel as compared
to both SAF and liquid hydrogen.
In Scenario 4, where technological performance and environmental impact were
prioritized over costs and circularity, the highest sustainability index was obtained once
again for the A320 using SAF. This result reflects the combination of a good technological
and environmental performance by involving this fuel, as compared to aircraft using
kerosene or liquid hydrogen.
In Scenario 5, where, among the criteria the complex comparisons listed in Table 3 were
performed, the LH2 aircraft utilizing green hydrogen appeared to be the most sustainable
option according to the AHP-WSM tool. It is worth noticing that in scenarios where
complex comparisons are made, a ranking on the basis of a sequence of logical reasoning
is much more difficult without involving a tool reducing subjectivity, such as the one
implemented in the present study.

Table 4. Comparison of AHP-WSM with AHP-TOPSIS.

Comparison of SWM with TOPSIS

SWM TOPSIS
Aircraft Type Index Value Ranking Order Index Value Ranking Order
Scenario 1
A320 neo (kerosene) 0.806 4 0.335 4
A320 neo (SAF) 0.844 1 0.668 2
LH2 (blue hydrogen) 0.812 3 0.497 3
LH2 (green hydrogen) 0.838 2 0.717 1
Scenario 2
A320 neo (kerosene) 0.786 4 0.3909 4
A320 neo (SAF) 0.921 1 0.681 1
LH2 (blue hydrogen) 0.858 3 0.454 3
LH2 (green hydrogen) 0.918 2 0.642 2
Scenario 3
A320 neo (kerosene) 0.888 1 0.713 1
A320 neo (SAF) 0.866 3 0.287 3
LH2 (blue hydrogen) 0.856 4 0.238 4
LH2 (green hydrogen) 0.872 2 0.371 2
Scenario 4
A320 neo (kerosene) 0.716 4 0.156 4
A320 neo (SAF) 0.793 1 0.857 2
LH2 (blue hydrogen) 0.744 3 0.571 3
LH2 (green hydrogen) 0.780 2 0.867 1
Scenario 5
A320 neo (kerosene) 0.835 4 0.407 4
A320 neo (SAF) 0.873 2 0.594 2
LH2 (blue hydrogen) 0.841 3 0.453 3
LH2 (green hydrogen) 0.875 1 0.654 1
Aerospace 2023, 10,
Aerospace 2023, 10, 240
x FOR PEER REVIEW 10 of 13
9 of

Figure 2.
Figure 2. Comparison of WSM
Comparison of WSM and
and TOPSIS
TOPSIS aircraft
aircraft ranking
ranking for
for the
the five
five scenarios
scenarios considered.
considered.

TableBased
4. Comparison of AHP-WSMofwith
on the comparison theAHP-TOPSIS.
AHP-WSM with those obtained from the applied
AHP-TOPSIS, for three out of five scenarios
Comparison (2, 3, with
of SWM and 5), the two tools suggested the
TOPSIS
same ranking. For Scenarios 1 and 4, a rank exchange was observed between the first and
SWM TOPSIS
second place. However, the first and second place for both models applied in said scenarios
Aircraft Type Index Value Ranking Order Index Value Ranking Order
are very close to each other, indicating that these two choices appear as almost equally
Scenario 1
sustainable choices.
A320 neo (kerosene) It needs 0.806
to be underlined that 4the aim of the present 0.335 4
work has been to demonstrate
A320 neo (SAF) 0.844 1 0.668
the approach of involving a quantitative holistic index for ranking aircraft of similar 2 range,
LH2 (blue hydrogen) with regards to 0.812 3
the holistic interpretation of sustainability0.497 3
introduced herein. However, the
LH2 (green hydrogen) rankings obtained
0.838 for the scenarios examined
2 entail a certain
0.717amount of uncertainty. 1 This is
owed to the fact that certain data for some of the sub-criteria
Scenario 2 involved are not yet available,
A320 neo (kerosene) and this mainly refers
0.786 to the LH 2 aircraft.
4 Hence, neglecting
0.3909 some of the sub-criteria
4 in the
performed analysis may appreciably overestimate or underestimate the sustainable index
A320 neo (SAF) 0.921 1 0.681 1
obtained. Such a case is, for example, the LH2 DOCs, which according to Clean Sky 2 [2]
LH2 (blue hydrogen) 0.858 3 0.454 3
are expected to rise considerably due to the larger airframe of the aircraft and the frequent
LH2 (green hydrogen) checks of the 0.918 2
LH2 tanks, especially during the first years0.642 2
of the LH2 aircraft introduction in
Scenario 3
the market. Moreover, increased costs associated with longer refueling times and increased
A320 neo (kerosene) turnaround times
0.888as well as increased1 personnel costs are 0.713expected when the LH 1 2 aircraft
A320 neo (SAF) 0.866 [2]. Considering
will be introduced 3 this in the analysis 0.287 3
would appreciably penalize the
LH2 (blue hydrogen) term cost, and hence
0.856 decrease the 4observed sustainability
0.238 index for these aircraft.
4 On
LH2 (green hydrogen) the other hand, for the LH2 aircraft,2 apart from the CO0.371
0.872 2 emissions, which have 2 the most
detrimental effect on climate change, NOx is also
Scenario 4 expected to be significantly reduced
as compared to aircraft using kerosene [38]. Considering this, the term expressing the
A320 neo (kerosene) 0.716 4 0.156 4
environmental impact would decrease and consequently result in an increase to the overall
A320 neo (SAF) 0.793 1 0.857 2
sustainability index for the LH2 aircraft. Finally, the contrail effect on climate impact, which
LH2 (blue hydrogen) has been found 0.744 3 0.571
0
to be comparable in magnitude to CO2 s impact [2], is expected to have a 3
LH2 (green hydrogen) 0.780 2 0.867 1
Aerospace 2023, 10, 240 11 of 13

considerable effect on the environmental impact term, and hence the overall sustainability
index. The latter remarks make evident the need to obtain relevant data and metrics to
enable a reliable sustainability assessment.

4. Conclusions
In the present work, a definition of sustainability was proposed, where sustainability
is understood as a matter of trade-off between potentially contradicting aspects, linked to
technological performance, economic, and ecological criteria. Sustainability was expressed
through a quantitative index. This index has been used to support selecting aircraft of a
similar range.
To this end, a MCDM-based approach to support decision-making when selecting
an aircraft was implemented. The MCDM-based model combined the analytic hierarchy
process (AHP) and a linear aggregation method to integrate appropriate indicators into a
single index. To illustrate the proposed approach, a set of commercial passenger aircraft of
a similar range were considered and compared. To assess the reliability of the proposed
approach and the sensitivity of our model to weights’ variation, several scenarios of
different complexity were considered. It must be noted that the proposed methodology led
to a ranking which was found to be sensitive to the weight variation, by accounting for a
variety of representative weighting scenarios. The ranking referring to the less complex
scenarios was found to be reasonable and consistent with the expected outcome. For more
complex scenarios, it has been highlighted that such a ranking cannot occur without the
use of tools such as the one implemented in the current work. The comparison of the
results with those obtained from the application of the TOPSIS method suggested the same
ranking for most of the scenarios considered. However, for two scenarios, a rank exchange
was observed between the first and the second place.
The proposed methodology can be exploited by the aviation industry in future ranking
and aircraft selection studies as a decision-making support tool, especially when contra-
dicting aspects are present, as well as to support monitoring of future aircraft impacts
on sustainability. However, it should be noted that the latter assessment is a preliminary
one, aiming to assess the potential of such technologies, based on the available data. The
maturity of the investigated technologies is currently at a low Technology Readiness Level
(TRL); however, essential efforts are being made (e.g., [2]) to mature them in the next years,
with the main aim to integrate them into new aircraft until 2035. The absence of data
for aircraft under development does not yet allow the full implementation of the model,
and hence data availability would considerably increase the confidence of the obtained
ranking. The latter limitation is owed to the fact that two out of four aircraft considered
are not yet existent and are being developed with the aim to respond to the current global
environmental challenges; in addition, such data were up to now only partially gathered
and shared by the aviation industry. It could be expected that with the ongoing efforts of
the aviation industry in the research of new technologies, provision and access to such data
will be facilitated, which will allow for a more precise assessment of the sustainability of
future aircraft. It is worth noticing that aircraft sustainability assessment can be further
enhanced with the introduction of additional criteria linked to the sustainability of the
aviation industry. Such criteria are, for example, the incorporation of social and circular
economy aspects and metrics. An investigation of the robustness of the proposed method
by including a thorough sensitivity analysis of the final output to the weights’ variation,
as well as to the data values’ variation, is a matter for a future study. Furthermore, a
comparison of the ranking obtained through alternative MCDM methods is a subject for
further research.

Author Contributions: Conceptualization, D.N.M. and S.G.P.; methodology, D.N.M. and S.G.P.;
software, D.N.M. and S.G.P.; validation, D.N.M. and S.G.P.; formal analysis, D.N.M. and S.G.P.;
investigation, D.N.M. and S.G.P.; resources, D.N.M. and S.G.P.; data curation, D.N.M. and S.G.P.;
writing—original draft preparation, D.N.M. and S.G.P.; writing—review and editing, D.N.M. and
Aerospace 2023, 10, 240 12 of 13

S.G.P. visualization, D.N.M. and S.G.P.; supervision, D.N.M. and S.G.P.; project administration, D.N.M.
and S.G.P. All authors have read and agreed to the published version of the manuscript.
Funding: This research received no external funding.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Not applicable.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Moore, J.E.; Mascarenhas, A.; Bain, J.; Straus, S.E. Developing a comprehensive definition of sustainability. Implement. Sci. 2017,
12, 110. [CrossRef] [PubMed]
2. Fuel Cells and Hydrogen 2 Joint Undertaking. Hydrogen-Powered Aviation: A Fact-Based Study of Hydrogen Technology, Economics,
and Climate Impact by 2050; Publications Offic of the Europen Union: Luxembourg, 2020.
3. Association of European Research Establishments in Aeronautics. Circular Aviation; A Research Programme of the Future Sky
Joint Research Initiative; EREA: Richmond, Australia, 2020.
4. Barke, A.; Bley, T.; Thies, C.; Weckenborg, C.; Spengler, T.S. Are Sustainable Aviation Fuels a Viable Option for Decarbonizing Air
Transport in Europe? An Environmental and Economic Sustainability Assessment. Appl. Sci. 2022, 12, 597. [CrossRef]
5. Rasaizadi, A.; Ardestani, A.; Nahavandi, N. Airplane Selection to Renovate Air Transportation System: A Multi-Criteria
Decision-Making Problem. Adv. Res. Civ. Eng. 2021, 3, 61–75.
6. Torğul, B.; Demiralay, E.; Paksoy, T. Training aircraft selection for department of flight training in fuzzy environment. Decis. Mak.
Appl. Manag. Eng. 2022, 5, 264–289. [CrossRef]
7. Ma, J. Training Aircraft Selection of the Vietnam People’s Air Force Using a Hybrid BWM-Fuzzy TOPSIS Method”. Period.
Polytech. Soc. Manag. Sci. 2022, 30, 141–157. [CrossRef]
8. Yilmaz, A.K.; Malagas, K.; Jawad, M.; Nikitakos, N. Aircraft selection process with Technique for Order Performance by Similarity
to Ideal Solution (TOPSIS) and AHP Integration. Int. J. Sustain. Aviat. 2020, 6, 220–235. [CrossRef]
9. Ardil, C. ’Aircraft Selection Using Multiple Criteria Decision Making Analysis Method with Different Data Normalization
Techniques’, World Academy of Science, Engineering and Technology, Open Science Index 156. Int. J. Ind. Syst. Eng. 2019, 13,
744–756.
10. Bakir, M.; Akan, S.; Özdemir, E. Regional aircraft selection with fuzzy piprecia and fuzzy marcos: A case study of the turkish
airline industry. Facta Univ. Ser. Mech. Eng. 2021, 19, 423–445. [CrossRef]
11. Kiraci, K.; Bakir, M. Application of commercial aircraft selection in aviation industry through multi-criteria decision making
methods. Celal Bayar Üniv. Sos. Bilim. Derg. 2018, 16, 307–332. [CrossRef]
12. Dožić, S.; Lutovac, T.; Kalić, M. Fuzzy AHP approach to passenger aircraft type selection. J. Air Transp. Manag. 2018, 68, 165–175.
[CrossRef]
13. Vrtagić, S.; Softić, E.; Subotić, M.; Stević, Ž.; Dordevic, M.; Ponjavic, M. Ranking Road Sections Based on MCDM Model: New
Improved Fuzzy SWARA (IMF SWARA). Axioms 2021, 10, 92. [CrossRef]
14. Stević, Ž.; Miškić, S.; Vojinović, D.; Huskanović, E.; Stanković, M.; Pamučar, D. Development of a Model for Evaluating the
Efficiency of Transport Companies: PCA–DEA–MCDM Model. Axioms 2022, 11, 140. [CrossRef]
15. Moslem, S.; Çelikbilek, Y. An integrated grey AHP-MOORA model for ameliorating public transport service quality. Eur. Transp.
Res. Rev. 2020, 12, 68. [CrossRef]
16. Rong, Y.; Yu, L.; Niu, W.; Liu, Y. Tapan Senapati, Arunodaya Raj Mishra, MARCOS approach based upon cubic Fermatean fuzzy set and
its application in evaluation and selecting cold chain logistics distribution center. Eng. Appl. Artif. Intell. 2022, 116, 105401. [CrossRef]
17. Senapati, T.; Chen, G.; Yager, R.R. Aczel–Alsina aggregation operators and their application to intuitionistic fuzzy multiple
attribute decision making. Int. J. Intell. Syst. 2022, 37, 1529–1551. [CrossRef]
18. Moslem, S.; Campisi, T.; Szmelter-Jarosz, A.; Duleba, S.; Nahiduzzaman, K.M.; Tesoriere, G. Best–Worst Method for Modelling
Mobility Choice after COVID-19: Evidence from Italy. Sustainability 2020, 12, 6824. [CrossRef]
19. Pinheiro Melo, S.; Barke, A.; Cerdas, F.; Thies, C.; Mennenga, M.; Spengler, T.S.; Herrmann, C. Sustainability assessment and
engineering of emerging aircraft technologies—Challenges, methods and tools. Sustainability 2020, 12, 5663. [CrossRef]
20. Thomson, R.; Sachdeva, N.; Healy, A.; Bailly, N.; Stern, C. Sustainable Aviation Fuels; Roland Berger Ltd.: London, UK, 2020.
21. Katsiropoulos, C.V.; Pantelakis, S.G. A novel holistic index for the optimization of composite components and manufacturing
processes with regard to quality, life cycle costs and environmental performance. Aerospace 2020, 7, 157. [CrossRef]
22. Markatos, D.N.; Katsiropoulos, C.; Tserpes, K.; Pantelakis, S. A holistic End-of-Life (EoL) Index for the quantitative impact
assessment of CFRP waste recycling techniques. Manuf. Rev. 2021, 8, 18.
23. Markatos, D.N.; Pantelakis, S.G. Assessment of the Impact of Material Selection on Aviation Sustainability, from a Circular
Economy Perspective. Aerospace 2022, 9, 52. [CrossRef]
Aerospace 2023, 10, 240 13 of 13

24. Dožić, S.; Kalić, M. Comparison of Two MCDM Methodologies in Aircraft Type Selection Problem. Transp. Res. Procedia 2015, 10,
910–919. [CrossRef]
25. Kneese, A.V. The Economics of Natural Resources. Popul. Dev. Rev. 1988, 14, 281–309. [CrossRef]
26. Saidani, M.; Yannou, B.; Leroy, Y.; Cluzel, F. How to Assess Product Performance in the Circular Economy? Proposed Requirements
for the Design of a Circularity Measurement Framework. Recycling 2017, 2, 6. [CrossRef]
27. Rigamonti, L.; Mancini, E. Life cycle assessment and circularity indicators. Int. J. Life Cycle Assess 2021, 26, 1937–1942. [CrossRef]
28. Corona, B.; Shen, L.i.; Reike, D.; Carreón, J.R.; Worrell, E. Towards sustainable development through the circular economy—A
review and critical assessment on current circularity metrics, Resources. Conserv. Recycl. 2019, 151, 104498. [CrossRef]
29. Kirchherr, J.; Reike, D.; Hekkert, M. Conceptualizing the circular economy: An analysis of 114 definitions. Resour. Conserv. Recycl.
2017, 127, 221–232. [CrossRef]
30. Moraga, G.; Huysveld, S.; Mathieux, F.; Blengini, G.A.; Alaerts, L.; Van Acker, K.; de Meester, S.; Dewulf, J. Circular economy
indicators: What do they measure? Resour. Conserv. Recycl. 2019, 146, 452–461. [CrossRef] [PubMed]
31. Zhao, D.; Guo, Z.; Xue, J. Research on Scrap Recycling of Retired Civil Aircraft IOP Conference Series: Earth and Environmental Science
2021; IOP Publishing: Bristol, UK, 2021; Volume 657, p. 012062s.
32. PAMELA Project. Process for Advanced Management of End of Life of Aircraft PROJECTS. Available online: https://trimis.ec.
europa.eu/project/process-advanced-management-end-life-aircraft (accessed on 1 December 2022).
33. Saaty, T.L. The Analytic Hierarchy Process: Planning, Priority Setting, Resource Allocation; McGraw-Hill: New York, NY, USA, 1980; p. 287.
34. Churchman, C.W.; Ackoff, R.L. An Approximate Measure of Value. J. Oper. Res. Soc. Am. 1954, 2, 172–187. [CrossRef]
35. Madzík, P.; Falát, L. State-of-the-art on analytic hierarchy process in the last 40 years: Literature review based on Latent Dirichlet
Allocation topic modelling. PLoS ONE 2022, 17, e0268777. [CrossRef]
36. Mukhopadhaya, J.; Rutherford, D. Performance analysis of evolutionary hydrogen-powered aircraft. In White Paper of the
International Council on Clean Transportation (ICCT); International Council on Clean Transportation: Washington, DC, USA, 2022.
37. Goswami, S.S.; Behera, D.K. An Analysis for Selecting Best Smartphone Model by AHP-TOPSIS Decision-Making Methodology.
Int. J. Serv. Sci. Manag. Eng. Technol. (IJSSMET) 2021, 12, 116–137. [CrossRef]
38. Choi, Y.; Lee, J. Estimation of Liquid Hydrogen Fuels in Aviation. Aerospace 2022, 9, 564. [CrossRef]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.