FEATURE: BLOCKCHAIN EVALUATION FRAMEWORK

The Blockchain several technologies have been intro-

duced, leading onto a third generation
of blockchains, more able to cope with

Trilemma the trend characteristics and nonfunc-

tional requirements. We divide these
such technological solutions into three
main categories: 1) Layer 1 solutions,
2) Layer 2 solutions, so-called rol-

An Evaluation Framework lups (divided into optimistic and zero-

knowledge approaches), and finally 3)
side-chains. In this article, we present
a framework for the evaluation and
Giovanni Quattrocchi and Filippo Scaramuzza ,
comparison of these approaches; our
Politecnico di Milano
comparison features both a quantita-
tive and qualitative lens of analysis
Damian A. Tamburri , Eindhoven University of Technology
rotating around the so-called block-
chain trilemma,2 namely, the con-
flicting relation binding together the
// We present a validated framework for the
three key properties that a blockchain
evaluation and comparison of three main strives to exhibit operationally. On the
third-generation blockchain categories, based one hand, the trilemma is per se not a
novel notion but, on the other hand, to
on the three main nonfunctional aspects the best of our knowledge we are the
that discriminate their use for the design first to approach the problem over the
technologies targeted by this article
and orchestration of complex blockchain- and at the architecture level, with the
oriented service applications, namely: end-goal of providing an early insight
into the tradeoffs entailed by the tri-
scalability, decentralization, and security. //
lemma itself, and the design principles
stemming from their evaluation. The
trilemma leads to conclude that it is
possible to have only two out of these
three properties regardless of the ar-
chitecture decisions made to approach
all three at once. Blockchains establish
trust through decentralized ledgers
and consensus mechanisms, initially
relying on energy-intensive PoW like
©SHUTTERSTOCK.COM/YURCHANKA SIARHEI Bitcoin and Ethereum, leading to scal-
ability and centralization concerns.
WITH THE INCREASE of network of blockchains, two main problems Innovations in blockchain architec-
traffic and load onto Bitcoin—which have arisen: scalability and interop- ture and consensus algorithms, such
by definition is a so-called proof-of- erability,1 namely, the interaction of as proof-of-stake (PoS), have emerged
work (PoW) network and is highly cen- multiple blockchain-oriented archi- to address these issues, exemplified
tralized—and Ethereum, respectively, tectures within the same larger-scale by Ethereum’s transition to PoS in
the first and the second generations system. To solve these two issues, Ethereum 2.0, enhancing efficiency
and security. To improve blockchain
Digital Object Identifier 10.1109/MS.2024.3417341 scalability and efficiency, new solu-
Date of publication 26 June 2024; date of current version 10 October 2024.
tions are being developed. This work
This work is licensed under a Creative Commons
Attribution 4.0 License. For more information,
see https://creativecommons.org/licenses/by/4.0/ NOVEMBER /DECEMBER 2024 | I E E E S O F T WA R E 101
 FEATURE: BLOCKCHAIN EVALUATION FRAMEWORK

focuses on five recently developed plat- with a simple card-sorting exercise3 the scope of the same exercise, Layer
forms that were elicited in an informal with over 40 practitioners of 3+ years 2 (L2) solutions emerged as competi-
panel held with over 50+ blockchain of blockchain-oriented design experi- tive assets, with features including
architects as part of the NWO-MVI ence, was enacted the result of which rollups and sidechains like Arbitrum,
CHAIN project (https://chainresearch. led us to a few select Layer 1 (L1) so- zkSync, and Polygon, operate on top
eu /events/online-meeting-platform lutions, such as Cardano and Solana, of base layer blockchains to offload
-for-responsible-innovation/). which introduce alternative consen- computation and enhance transac-
Within the scope of the aforemen- sus algorithms and sharding to pro- tion throughput, maintaining security
tioned project, a pilot study to identify cess transactions more effectively. In while increasing performance.

Evaluation Framework
Definition
To find a way to evaluate the main
Select Technology Scalability Evaluation
to Evaluate properties of these scaling solu-
tions, we took inspiration from the
problem statement that shaped and
inspired these solutions, i.e., the
Search for History of Blockchain Trilemma.
Search for
TPS and Select the
Theoretical TPS The scalability trilemma states
Highest
that there are three properties that a
blockchain tries to have and that it is
Identify the Scaling
possible to have only two out of these
Solution Category
Decentralization and Find the three, with an architecture tradeoff
Level Decentralization existing across all three architecture
Bottleneck variability points. The three proper-
ties are as follows:

L1 L2 • Scalability: The chain can pro-

Sidechains
(Miners/ (Validators/ cess more transactions than a
Aggregators) (Validators)
Stake Pool) single regular node can verify,
e.g., a consumer laptop.
• Decentralization: The chain
can run without any trust de-
pendencies on a small group of
Find the Nakamoto Security
Coefficient Evaluation large, centralized actors. This
is typically interpreted to mean
that there should not be any
trust (or even honest-majority
Find Public assumption) in a set of nodes
Find Consensus
Vulnerabilities, Bugs, that you cannot join with just
Model
Attack Reports a consumer laptop.
• Security: The chain can resist a large
percentage of participating nodes
Quantitative trying to attack it (ideally 50%).
Evaluation Based
End
On Nakamoto
The intended process model is
Coefficient
shown in Figure 1 while the replication
package of our analysis is available at
FIGURE 1. Process model diagram. https://zenodo.org/records/10251476.

102 I E E E S O F T WA R E | W W W. C O M P U T E R . O R G / S O F T W A R E | @ I E E E S O F T WA R E
Scalability Evaluation much centralization.” Specifically, we with our intuitive notion of a “more
There is no actual research work on can think of a nonuniform distribu- centralized” system, the fact that
the definition of a metric for quanti- tion of wealth as highly unequal and the Gini coefficient is restricted to a
fying scaling in blockchain technolo- a nonuniform distribution of power 0–1 scale means that it does not di-
gies. The first idea, and what in the as highly centralized. rectly measure the number of in-
end we ended up using, is the adop- The Gini coefficient is calculated dividuals or entities required to
tion of the so-called transactions using the Lorenz curve as depicted in compromise a system. An alterna-
per second (TPS), i.e., the number of Figure 2. We computed the cumula- tive approach is to define a similar
transactions successfully added to a tive distribution of held power starting metric based on the Lorenz curve
block and validated in one second. To from lowest values to highest, such as from which the Gini coefficient is
be as objective as possible, we used staked tokens, relative to the number calculated, which is called the Naka-
two main types of TPS: The first is of entities possessing that power (e.g., moto coefficient.6 Given a subsystem
the one claimed to be the theoretical validators). This process generates a s with K entities, let p 1 2 g 2 p K be
one, and the second one is the maxi- curve, i.e., the Lorenz curve, that vi- the proportions of the subsystem con-
mum value that has been measured sually encapsulates the cumulative trolled by each of the K participants
over the history of transaction count distribution of decentralization in the such that R Ki p i = 1. Then we define
per day. We excluded from the analy- analyzed platform. A Lorenz curve the Nakamoto coefficient as
sis a real-time TPS computation since that closely aligns to linear function
it relies too much on the current load (y = x) indicates a more even distribu-
N s|= min ( k ! [1,f, K]: | p i $ threshold 2 .
k

of the network and on other factors tion of power, thus high decentraliza- i =1

that are time-dependent. tion, whereas a significant divergence

from this line suggests inequality and In other words, the Nakamoto co-
Decentralization greater centralization. efficient (the higher the better) of a
Evaluation The equation for the Gini coef- subsystem Ns is the minimum num-
Despite the widely acknowledged ficient can be calculated from the ber of entities that holds a specific
importance of this property, most area B under the Lorenz curve and threshold of control. In the case of
discussion on the topic lacks quan- the area A under the so-called line of PoW networks, such a threshold is
tification. Sarada Prasad Gochhayat equality as shown as follows: set to 51%, while for PoS networks
et al.4 proposed a work that aims to is set to 33%.7 This metric is the one
A
measure decentralization using the Gini coefficient = A + B . we have chosen to conduct the re-
Gini coefficient. There are strik- search, where the individuals or en-
ing similarities between the concepts However, the Gini coefficient has tities are taken into account differ
of “too much inequality– and “too an issue: While a high value tracks based on the underlying technology:

1. Perfect Equality 2. Unequal 3. More Unequal 4. Total Inequality

100%
% Participation
Cumulative

G1 = 0

G2 > 0
G3 > G2 G4 = 1

0%
0% Cumulative 100% 0% Cumulative 100% 0% Cumulative 100% 0% Cumulative 100%
% Population % Population % Population % Population

FIGURE 2. The Lorenz curve is shown in red. As the cumulative distribution diverges from a straight line, the Gini coefficient (G)
increases from 0 to 1.5

NOVEMBER /DECEMBER 2024 | I E E E S O F T WA R E 103

 FEATURE: BLOCKCHAIN EVALUATION FRAMEWORK

• L1 solutions and sidechains: the TPS metric is the block’s body size. Polygon
individuals or entities are the Some other efficiency improvements The theoretical value of TPS is 7200,
blocks validator nodes might be present in the protocol’s as stated by the Polygon team. This
• L2 solutions: the individuals or implementation. Nevertheless, such value has been obtained in a Polygon
entities are the aggregator nodes. considerations are beyond the scope local testnet. Anyway, the maximum
of this conceptual work. value of transactions that have been
Security Evaluation reached in the mainnet is 9,177,310
The best way to measure security Solana (https://polygonscan.com/chart/tx).
quantitatively should be to count the The theoretical value for TPS in So- Hence Polygon reached ~101.97 TPS.
number of vulnerabilities and exploits/ lana, as stated in its resident imple-
attacks. Since this is practically im- mentation documentation, can be Decentralization
possible, we have tried to mention the computed as follows. On a 1-Gb/s Evaluation
well-known vulnerabilities in the con- network connection, the maximum
sensus model adopted and to highlight number of transactions possible is Cardano
some special efforts spent by the devel- 1 gigabit per second/176 bytes = During each epoch, rewards are dis-
opers to solve specific security issues. 710k TPS. From the Solana explorer tributed among all stakeholders, who
Where possible, it would be appro- (https://analytics.solscan.io/public/ have delegated to a stake pool, either
priate to add a discussion about the dashboard/) it is possible to see the their stake pool or another pool. These
known vulnerability and the reports max count of transactions per day, rewards are autogenerated by the pro-
of attacks or bugs. counting only the meaningful ones, tocol itself and are not managed by the
i.e., payments and instructions (this stake pools. From PoolTool (https://
Scalability Evaluation tool also takes into account spe- pooltool.io), we got the list of stake
cial transactions, like votes, that the pools and the amount of ADA staked.
Cardano Solana protocol needs to operate). As shown in Figure 3, the Nakamoto
There is no actual claim about a the- This count reaches its maximum at coefficient is 119.
oretical number of TPS from the 152,330,000 transactions per day,
Cardano Team. So, to find this met- i.e., 1763 TPS. Solana
ric, we crafted a method based on Solana adopts the Delegated Proof of
the protocol parameters: Since the Arbitrum Stake consensus algorithm, where vali-
maximum size of a block (its body in The theoretical value of TPS we dating transactions (blocks) are nodes
bytes) is equal to maxBlockBodySize: 65536 found is 40,000. This value was cal- called Validators. We got the list of
bytes. We then took the transaction culated if the maximum capacity of validators from Validators.app (https://
average size using Cardano GraphQL Ethereum was filled by the Arbitrum www.validators.app/?locale=en&
(https://github.com/input-output-hk/ rollup batches and with no other L1 network=mainnet&order=stake) and
cardano-graphql), obtaining a value transactions. From the Arbitrum ex- the relative amount of SOL staked.
of 613 b. This means that, on aver- plorer (https://arbiscan.io/chart/tx) As shown in Figure 3, the Nakamoto
age, the number of transactions per we got the highest number of trans- coefficient is 18.
block is equal to 107. Therefore, since actions per day, which is 267,608,
a block is validated on average ev- i.e., 3.09 TPS. Arbitrum
ery 20 s, the theoretical TPS we ob- Arbitrum, as an optimistic rollup,
tained is 5.35. As for the maximum zkSync batch inner transactions and sends
TPS Cardano has ever reached, from The theoretical value of TPS as stated them, grouped as one, to the Ethe-
Messari,8 we can see it got to 495,825 in the documentation is ~2000.9 This reum L1. To do so, Arbitrum takes
transactions in a day. Hence, the max- value was evaluated with the current advantage of three types of batcher
imum throughput ever got is 5.74. block gas limit of 12.5 M. From EthTPS nodes: forwarders, aggregators, and
The theoretical TPS we computed is (https://ethtps.info/Network?name= sequencers. Users can send their L2
slightly less than the maximum ever ZKSync), we can get the maximum transactions to any of these three
reached. This is due to our assump- value for TPS in the last two years, nodes. Forwarder nodes forward any
tion that the only factor affecting the which is equal to 0.521. L2 transactions to a designated address

104 I E E E S O F T WA R E | W W W. C O M P U T E R . O R G / S O F T W A R E | @ I E E E S O F T WA R E
 Cardano Nakamoto Coefficient: 119 Solana Nakamoto Coefficient: 18

1 Line of Ideal Decentralization 1 Line of Ideal Decentralization

Lorenz Curve Above 66.6% Lorenz Curve Above 66.6%
Lorenz Curve Below 66.6% Lorenz Curve Below 66.6%
0.8 0.8

Cumulative SOL Staked

Cumulative ADA Staked

0.6 0.6

0.4 0.4

0.2 0.2

0 0

0 500 1,000 1,500 2,000 2,500 3,000 0 250 500 750 1,000 1,250 1,500 1,750
Stake Pools Validators
Arbitrum Nakamoto Coefficient: 2,515 zkSync Nakamoto Coefficient: 1
1 Line of Ideal Decentralization 1 Line of Ideal Decentralization
Lorenz Curve Above 51% Lorenz Curve Above 51%
Cumulative Transactions Batches Sent

Cumulative Number of Batches Sent

Lorenz Curve Below 51% Lorenz Curve Below 51%

0.8 0.8

0.6 0.6

0.4 0.4

0.2 0.2

0 0

0 2000 4000 6000 0 200 400 600 800

Aggregators Aggregators

Polygon CardanNakamoto Coefficient: 2

1
Line of Ideal Decentralization
Lorenz Curve Above 66.6%
Lorenz Curve Below 66.6%
0.8
Cumulative MATIC Staked

0.6

0.4

0.2

0
0 20 40 60 80 100
Validators

FIGURE 3. Nakamoto coefficients for decentralization in Cardano (119), Solana (18), Arbitrum (2,515), zkSync (1), and Polygon (2).

NOVEMBER /DECEMBER 2024 | I E E E S O F T WA R E 105

 FEATURE: BLOCKCHAIN EVALUATION FRAMEWORK

of another node. The designated node of the entire network. This analy- sequencers, we chose to present the
can be either a sequencer or an aggre- sis has been conducted over 10,000 most decentralized setup, resulting in
gator and is referred to as the aggrega- different transactions thanks to the a Nakamoto coefficient of 2,515.
tor address. Aggregator nodes will take Etherscan application programming In contrast, zkSync does not offer
a group of incoming L2 transactions interfaces. As shown in Figure 3, the such alternatives, leading to a Na-
and batch them into a single message Nakamoto coefficient is 1. kamoto coefficient of 1, as a single
to the delayed inbox (a smart contract aggregator predominantly handles
on Ethereum). The sequencer node Polygon most L2-to-L1 transactions. Finally,
will also take a group of incoming Polygon basically acts as a PoS Polygon, operating as a sidechain
L2 transactions and batch them into blockchain with a set of validators. solution, exhibits a Nakamoto coef-
a single message, but it will send the The set of validators was taken from ficient of 2, indicating a significant
batch message to the sequencer in- the Polygon website (https://wallet. level of centralization.
box instead. If the sequencer node polygon.technology/staking/validator). Overall, third-generation block-
stops adding transactions to the se- The analysis has been conducted by chains do not appear to excel in
quencer inbox, anyone can force the taking the number of validators and decentralization, with the notable ex-
sequencer inbox to include transac- the respective amount of MATIC ception of Arbitrum, which achieves
tions from the delayed inbox via a staked. As shown in Figure 3 the Na- this when relying on user-operated
smart contract function call. This al- kamoto coefficient is 2. aggregators.
lows the Arbitrum network to always
be available and resistant to a mali- Discussion Security Evaluation
cious sequencer.10 In result, as shown L1 solutions like Cardano and Solana
in Figure 3 the ­Nakamoto coefficient exhibit similar trends, with Naka- Cardano
is 2,515. moto coefficients of 119 and 18, re- Cardano has a special implementa-
In Figure 3, we present the Lo- spectively. However, these figures are tion of the Proof of Stake consen-
renz curves concerning the aggrega- modest when compared to the total sus algorithm, named Ouroboros.11
tor nodes. Since an important role is number of validators and stake pools, Ouroboros processes transaction
played by the Arbitrum sequencer, which surpass 3,000 and 1,750, re- blocks by dividing chains into epochs,
too, we calculated, in the same spectively. This indicates that control which are further divided into time
way, the number of transactions in of the network can be achieved by slots. A slot leader is elected for each
the same period, which is equal to gaining influence over 4% for Car- time slot and is responsible for add-
2,756. To have a comparison, we dano and 1% for Solana of selected ing a block to the chain. To protect
calculated the number of transac- selected peers. Regarding L2 solu- against adversarial attempts to sub-
tions of the top 2,515 aggregators, tions like Arbitrum and zkSync, a vert the protocol, each new slot
which is equal to 4,999. similar methodology was employed to leader is required to consider the
calculate the Nakamoto coefficient, last few blocks of the received chain
zkSync leveraging the number of aggregators as transient: Only the chain that pre-
zkSync, as a ZK rollup, has a single (instead of stakers or validators) and cedes a prespecified number of tran-
smart contract called delayed in- corresponding transaction volumes sient blocks is considered settled.
box, to which aggregator nodes send (instead of tokens staked). This is also referred to as the settle-
batches of transactions. In this eval- Arbitrum provides two evaluation ment delay and is the mechanism
uation, our focus on the decentral- perspectives: One considers the single through which the ledger is securely
ization bottleneck rotates around default sequencer managed by the plat- passed between participants. An-
the number of aggregators and the form creators, while the other evaluates other interesting point in the secu-
transaction batches sent to the de- user-operated sequencers as an alterna- rity of Cardano is that it’s written in
layed inbox. This point is crucial as tive trust model Considering the single Haskell, a functional programming
more influential and widely adopted sequencer, Arbitrum would exhibit ex- language that encourages building
aggregators can manipulate transac- treme centralization with a Nakamoto a system using pure functions. This
tions’ order or introduce delays, po- coefficient of 1. However, given the leads to a design where components
tentially undermining the integrity option for users to create their own are conveniently testable in isolation.

106 I E E E S O F T WA R E | W W W. C O M P U T E R . O R G / S O F T W A R E | @ I E E E S O F T WA R E
Solana instead, it relies on Ethereum’s one. 3. In case the validators fail to
In PoW consensus, block times need Anyway, Arbitrum has several fea- process the requests, the system
to be very large (~10 min) to minimize tures, such as a Rollup, to avoid the enters exodus mode and every
the odds of multiple validators pro- so-called censorships attacks: Once user can immediately exit all
ducing a new valid block at the same a forking attack occurs, it will inevi- of their assets by making a
time. There’s no such constraint in PoS tably be discovered by a certain chal- direct transaction on the
consensus, but without reliable time- lenger. Therefore since the attack will Ethereum mainnet.
stamps, a validator cannot determine
the order of incoming blocks. The pop-
ular workaround is to tag each block
with a timestamp. Because of clock
drift and variance in network latencies, This means every single transaction
the timestamp is only accurate within
an hour or two. To “workaround the
is verified by a smart contract on the
workaround,” these systems lengthen Ethereum mainnet utilizing verifying
block times to provide reasonable cer- the proof of the validity of the block.
tainty that the median timestamp on
each block is always increasing. So-
lana takes a very different approach,
which it calls proof of history or PoH.
Leader nodes “timestamp” blocks fail as long as there is a virtuous chal- Polygon
with cryptographic proofs that some lenger, the attacker must jam all pos- Rewards are distributed to all stak-
duration of time has passed since the sible challengers. If there are many ers proportional to their stake at
last proof. All data hashed into the such challengers, the attack will al- every checkpoint with an exception
proof most certainly have occurred ready be difficult to accomplish. being the proposer getting an ad-
before the proof was generated. The ditional bonus. User reward bal-
node then shares the new block with zkSync ance gets updated in the contract
validator nodes, which can verify zkSync is built on zkRollup architec- which is referred to while claiming
those proofs. The blocks can arrive at tures.13 This means every single trans- rewards. Of course, validating does
validators in any order or even could action is verified by a smart contract have some risks. Stakes are at risk of
be replayed years later. With such on the Ethereum mainnet utilizing getting slashed in case the validator
reliable synchronization guarantees, verifying the proof of the validity of node commits a malicious act like
Solana can break blocks into smaller the block. Thus, no validator can ever double signing and validator down-
batches of transactions called entries. move the system into an incorrect state time which also affects the linked
Entries are streamed to validators in or take users’ funds. In the ultimate delegators at that checkpoint. As
real time before any notion of block emergency case of all validators being long as two-thirds of the weighted
consensus.12 Another valuable point in shut down or becoming unresponsive, stake of the validators is honest, the
Solana security is that smart contracts the emergency exit mechanism ensures chain will progress accurately. Vali-
are written in Rust. This language puts that users will keep control of their as- dators stake their MATIC tokens as
its entire attention to detail and pattern sets. It works as follows: collateral to work for the security
recognition. Solidity surely has more of the network and, in exchange for
well-known vulnerable patterns mak- 1. If transactions of a user are be- their service, earn rewards.
ing it easier to identify weak code and ing ignored, for any reason, by
fix (or exploit). the validators, an exit request Quantitative Comparison
can be submitted directly on the Herein, we present an initial quan-
Arbitrum mainnet into the priority queue. titative assessment of the selected
It’s worth noting that Arbitrum, be- 2. Validators are obliged to process blockchain technologies within the
ing an Ethereum rollup, doesn’t im- priority queue requests within a trilemma. The results, summarized
plement its consensus model, but short time window (. 1 week) in Table 1, facilitate a comparative

NOVEMBER /DECEMBER 2024 | I E E E S O F T WA R E 107

 FEATURE: BLOCKCHAIN EVALUATION FRAMEWORK

evaluation of each technology’s per- on Ethereum for final transaction coefficient of 2. This illustrates
formance in these key areas. settlement, we used Ethereum’s PoS how each platform navigates the
Scalability, indicated by TPS, security data as a proxy. This de- scalability-decentralization-security
varies significantly among the tech- cision stems from the operational trilemma, optimizing certain aspects
nologies, with Solana demonstrating characteristics of rollups, where the while compromising others to vary-
high throughput, wherefore the only security, especially in a decentralized ing degrees.
previous work that approached such scenario with multiple independent It is crucial to acknowledge the
a proposition is the review by Sanka batcher nodes, is closely aligned with influence of technology popularity
et al.14 Decentralization, reflected by that of Ethereum’s PoS model. on the maximum TPS in the scalabil-
the Nakamoto coefficient, highlights Obtained results highlight dis- ity evaluation. A notable limitation
Arbitrum’s high score, suggesting tinct tradeoffs among scalability, of the TPS metric is its sensitivity
a robust decentralized network. Se- decentralization, and security. For to the adoption and popularity of a
curity, while difficult to quantify example, Solana exhibits an impres- particular blockchain technology.
directly, has been computed as the sive scalability with a TPS of 1,763, For instance, a blockchain’s wide-
cost in U.S. dollars to obtain control suggesting its capability to handle spread use and acceptance can im-
of the blockchain. In particular, we high transaction volumes efficiently. pact its maximum TPS, as more
used the following metric: However, its Nakamoto coefficient participants and nodes contribute to
n is relatively low at 18, indicating the network, potentially leading to
cost|= c ) | s i a lesser degree of decentralization more transactions being processed.
i
compared to other platforms such This phenomenon underscores the
where n is the total amount of valida- as Arbitrum, which boasts a Naka- need to consider other approaches,
tors, c is the token cost in U.S. dol- moto coefficient of 2,515, reflecting like the theoretical TPS we included
lars, and s i is the amount of tokens a high level of network distribution in the study.
staked by validator i. The higher this and fault tolerance. This study underscores the com-
cost is the more difficult it is to take In terms of security, both zkSync plexity of optimizing blockchain
control of the blockchain. and Arbitrum exhibit high values, technologies across scalability, de-
For the security cost metric with a cost of US$20.6 billion since centralization, and security. The suit-
computation in PoS blockchains, they rely on the underlying secure ability of a blockchain for specific
we utilized available staking data Ethereum L1 net work. Cardano applications depends on the particu-
specific to each blockchain. How- and Polygon offer a balance between lar requirements and tradeoffs consid-
ever, for rollups like Arbitrum and these metrics, with Cardano achiev- ered acceptable for the use case. This
zkSync, the absence of a native stak- ing a TPS of 5.74 and a Nakamoto analysis provides a foundation for un-
ing mechanism necessitated a dif- coefficient of 119, and Polygon reach- derstanding these tradeoffs, aiding in
ferent method. Given their reliance ing a TPS of 101.97 with a Nakamoto the informed selection of blockchain
technologies for third-generation
applications.
TABLE 1. Comparison of blockchain approaches.

T
Scalability Decentralization Security he academic literature cur-
(cost in billions of U.S. dollars to
rently lacks explicit analy-
Blockchain (TPS) (Nakamoto coefficient) obtain control of the blockchain) sis of strategies to evaluate
and compare third-generation block-
Cardano 5.74 119 0.528
chains due to the emerging nature of
Solana 1763 18 9.11 the research field. This work pres-
ents a framework to evaluate third-
Polygon 101.97 2 0.846
generation blockchains based on the
Arbitrum 3.09 2515 20.6 Blockchain Trilemma.
In conclusion, further research
zkSync 0.521 1 20.6
is needed to properly ascertain the

108 I E E E S O F T WA R E | W W W. C O M P U T E R . O R G / S O F T W A R E | @ I E E E S O F T WA R E
 the program “Responsible Innovation.
ABOUT THE AUTHORS

Designing for Public Values in a Digital

GIOVANNI QUATTROCCHI is a junior assistant professor at World” and by project 3A-Italy Circular
Politecnico di Milano, 20133 Milan, Italy. His research interests and Sustainable Made in Italy—MICS
include self-adaptive systems, software architectures, edge (3A-ITALY) CUP D43C22003120001
computing, and blockchain-based systems. Quattrocchi (Grant PE00000004).
received his Ph.D. in computer engineering from Politecnico di
Milano. Contact him at [email protected]. References
1. U. W. Chohan, The Limits to Block-
chain? Scaling vs. Decentralization.
FILIPPO SCARAMUZZA is a research assistant at Politec- Rochester, NY, USA: SSRN, Jan.
nico di Milano, 20133 Milan, Italy. His research interests 2019, doi: 10.2139/ssrn.3338560.
include blockchain-oriented designs and their nonfunctional 2. S. He, Y. Ning, H. Chen, C. Xing, and
aspects. Scaramuzza received his M.Sc. in computer science L.-J. Zhang, “Layered consensus mecha-
and engineering from Politechnico di Milano. Contact him at nism in consortium blockchain for en-
[email protected]. terprise services,” in Blockchain—ICBC
(Lecture Notes in Computer Science).
DAMIAN A. TAMBURRI is an associate professor at the J. Joshi, S. Nepal, Q. Zhang, and L.-J.
Jheronimus Academy of Data Science, Eindhoven University Zhang, Eds. Cham, Switzerland: Springer-
of Technology, 5211 s’ Hertogenbosch, The Netherlands. His Verlag, 2019, vol. 11521, pp. 49–64.
research interests include data-intensive services DevOps/ 3. T. Zimmermann, “Card-sorting,” in
DataOps, social software engineering, and AI software Perspectives on Data Science for Soft-
engineering. Tamburri received his Ph.D. in information and ware Engineering, T. Menzies, L. A.
software engineering from VU University Amsterdam. Contact Williams, and T. Zimmermann, Eds.,
him at [email protected]. New York, NY, USA: Academic Press,
2016, pp. 137–141.
4. S. P. Gochhayat, S. Shetty, R. Muk-
kamala, P. Foytik, G. A. Kamhoua,
and L. Njilla, “Measuring decentral-
ity in blockchain based systems,”
nature and nurture of the blockchain of Figure 3 induces a partial order IEEE Access, vol. 8, pp. 178,372–
trilemma, especially in sight of between the select target technolo- 178,390, 2020, doi: 10.1109/
multimixes of third-generation gies; such order is to be used when ACCESS.2020.3026577.
blockchain-oriented computing. Fur- designing multichain solutions striv- 5. J. Hanson. “Can the Gini coefficient
thermore, other paradigms, such ing to achieve specific properties be negative?” Accessed: May 04,
as the blockchain quadrilemma,15 from the trilemma (e.g., decentral- 2022. [Online]. Available: https://
merit exploration to propose novel ization). Third, finally, our approach www.quora.com/Can-the-Gini
insights, while also integrating the itself serves as a general methodol- -coefficient-be-negative
findings presented in this study. At ogy to evaluate blockchain technol- 6. S. Chu and S. Wang, “The curses of
the same time several key findings ogy against the trilemma wherefore blockchain decentralization,” 2018,
of our study act as design principles the design needs dictate it. arXiv:1810.02937.
for blockchain-oriented design of the 7. “Ethereum proof-of-stake attack and
future. First, the assets on Table 1 Acknowledgment defense.” Ethereum.Org. Accessed:
provide valuable starting points to This research was supported in Jul. 1, 2024. [Online]. Available:
analysis blockchain-oriented designs part by the Dutch Research Coun- https://ethereum.org/en/developers/
featuring those specific technolo- cil (NWO) as part of the CHAIN docs/consensus-mechanisms/pos/
gies, for example, in the scope of project (https://chainresearch.eu/ attack-and-defense/
ad-hoc architecture tradeoff analy- event s /on l i ne -meet i ng-plat for m 8. “Cardano (ADA) transaction count.”
sis exercises. Second, the contents -for-responsible-innovation/) within Messari. Accessed: May 05, 2022.

NOVEMBER /DECEMBER 2024 | I E E E S O F T WA R E 109

 FEATURE: BLOCKCHAIN EVALUATION FRAMEWORK

[Online]. Available: https://messari. provably secure proof-of-stake block- 14. A. I. Sanka and R. C. C. Cheung,
io/charts/cardano/txn-cnt chain protocol,” in Advances in Cryptol- “A systematic review of blockchain
9. “zkEVM FAQ.” zkSync. Accessed: ogy – CRYPTO 2017, J. Katz and H. scalability: Issues, solutions, analysis
May 08, 2022. [Online]. Available: Shacham, Eds., Cham, Switzerland: and future research,” J. Netw. Com-
https://docs.zksync.io/zkevm/ Springer-Verlag, 2017, pp. 357–388. put. Appl., vol. 195, Dec. 2021, Art.
#how-scalable-is-a-zk-rollup 12. Anatoly Yakovenko, Solana Foun- no. 103232, doi: 10.1016/j.jnca.
10. Kyle Charbonnet. “A technical intro- dation. “Solana: A new architecture 2021.103232.
duction to Arbitrum’s optimistic rollup.” for a high performance blockchain.” 15. F. Mogavero, I. Visconti, A.
Medium. Accessed: May 09, 2022. Accessed: Jul. 1, 2024. [Online]. Vitaletti, and M. Zecchini,
[Online]. Available: https://medium. Available: https://solana.com/ “The blockchain quadrilemma:
com/privacy-scaling-explorations/ solana-whitepaper.pdf. v0.8.13 When also computational effec-
a-technical-introduction-to-arbitrums 13. Matter Labs. zkSync documentation. tiveness matters,” in Proc. IEEE
-optimistic-rollup-860955ea5fec zkSync. Accessed: May 04, 2022. Symp. Comput. Commun. (ISCC),
11. A. Kiayias, A. Russell, B. David, [Online]. Available: https://docs. 2021, pp. 1–6, doi: 10.1109/
and R. Oliynykov, “Ouroboros: A zksync.io/userdocs/ ISCC53001.2021.9631511.

Open Access funding provided by ‘Politecnico di Milano’ within the CRUI CARE Agreement

C a l l l e s
A r t i c
for putin
g
e C o m
Pe r vasiv h e la t
es t
IEEE ul p a p e r s on t
, u s ef ib l e ve , si
a c ce s s per va
seek s pme n t s in
d eve l o o pic s
ev iewe d u ting. T
peer-r co m p
itous
a n d u biqu of t wa
re
e , ogy, s
mobil c h n o l
a r e te
hard w g an d
in cl u d e
o r l d s e n s in
l-w
e , re a c tion,
ra s t r uc tur t e r i ntera
inf u
e li n e
s: - comp
guid n , h u man d in g
Au t hor / intera
ct i o
n s , inclu
rg /mc era ti o y.
p uter.o o n sid privac
www
.c o m
tm s y s t em s c u r i t y, an d
thor.h an d sec
ive /au bilit y,
pe r v a s
ls: e n t , s c ala
a i ym
e r det d e p lo
Furth .org ive
p u te r
sive @ co m
g/p ervas
p e r va ter.or
.c ompu
www
Digital Object Identifier 10.1109/MS.2024.3459148

110 I E E E S O F T WA R E | W W W. C O M P U T E R . O R G / S O F T W A R E | @ I E E E S O F T WA R E