Ethereum Devs Issue Final Report on Finality SNAFU

Loss of finality a week ago led to ‘the lowest participation of any epoch in Mainnet ever,’ developers say

article-image

alfernec/Shutterstock, modified by Blockworks

share

One week ago, the Ethereum network encountered a finality glitch that caused significant angst within the project’s developer community.

The problem stemmed from two consensus clients, including one of the leading clients, Prysm, spending too much computing resources on processing “mostly worthless” block attestations —  a part of the proof-of-stake consensus mechanism.

In a report issued by the team behind Prysm, the failure to finalize — while largely unnoticed by users — was worse the second time around on Friday. In one epoch, client participation fell to 30.7%, which the Prysm team says was “the lowest participation of any epoch in [Ethereum] Mainnet ever.”

The Prysm client is run on 37.6% of Ethereum validating nodes, according to clientdiversity.org. It is developed by Prysmatic Labs, which was acquired by Offchain Labs — the developers of Arbitrum — in October 2022.

An Ethereum epoch is roughly 6.4 minutes and comprises 32 “slots” — read blocks in the blockchain — of 12 seconds each. The network failed to finalize on Friday for nine epochs (nearly an hour). This was long enough to trigger the first-ever “inactivity leak” mode, which penalizes validators that have stopped contributing to consensus. 

But the damage was inconsequential: about $0.27 per validator.

“In total, we estimate that 28 ETH of penalties were applied and validators missed 55 ETH or more of potential revenue,” the Prysm team said. “This is less than 0.00015 ETH per validator.”

What went wrong

Prysm’s client began to struggle to handle incoming attestations with target checkpoints two epochs behind the then-current one. These valid attestations required clients to recalculate previous states of Ethereum’s Beacon chain and caused Prysm clients to exhaust their resources, resulting in a failure to meet validator clients’ requests promptly.

Ironically, contributing to the client’s woes was the huge success of Ethereum staking post-Shapella, when withdrawals of staked ether became possible for the first time.

A sharp increase in new validators requesting to join the Beacon chain since the start of May has pushed ETH staked net of withdrawals up by about 1.3 million, to a total 20.5 million — a value in dollar terms of more than $37 billion, on-chain data shows.

That has lengthened the queue to join the network by nearly 60,000 validators, meaning a new validator requesting to join today will not be active until some time in late June.

Beacon chain clients were stressed by all these incoming ether staking deposits, the Offchain Labs report explained.

“When the beacon state was smaller, Prysm would be able to handle these attestations and recover appropriately. However, with the large spike in deposits and the growing validator registry size, Prysm was unable to recover this time,” the team said.

Bug squashed

The Prysm client now ignores attestations for old blocks, which the report says are generally of no value to the network. Minority client Teku implemented a similar fix last weekend.

The current majority consensus client, Lighthouse, already was programmed to drop the problematic attestations and suffered no loss of liveness. The mere existence of client diversity has been largely credited with allowing the network to recover on its own without any manual intervention.

Aside from the technical stress on the software and hardware, the incident also took a toll on some members of the Ethereum community.

Loading Tweet..

Ethereum’s “community health consultant,” known as Superphiz, for one, declared he was taking some time off from Twitter, to “regain some physical fitness routines and mental clarity.”


Get the news in your inbox. Explore Blockworks newsletters:

  • Blockworks Daily: The newsletter that helps thousands of investors understand crypto and the markets, by Byron Gilliam.
  • Empire: Start your morning with the top news and analysis to inform your day in crypto.
  • Forward Guidance: Reporting and analysis on the growing intersection of crypto and macroeconomics, policy and finance.
  • 0xResearch: Alpha directly in your inbox. Market highlights, data, degen trade ideas, governance updates, token performance and more.
  • Lightspeed: Built for Solana investors, developers and community members. The latest from one of crypto’s hottest networks.
  • The Drop: For crypto collectors and traders, covering apps, games, memes and more.
  • Supply Shock: Tracking Bitcoin’s rise from internet plaything worth less than a penny to global phenomenon disrupting money as we know it.
Tags

Upcoming Events

Industry City | Brooklyn, NY

TUES - THURS, JUNE 24 - 26, 2025

Permissionless IV serves as the definitive gathering for crypto’s technical founders, developers, and builders to come together and create the future.If you’re ready to shape the future of crypto, Permissionless IV is where it happens.

Old Billingsgate

Mon - Wed, October 13 - 15, 2025

Blockworks’ Digital Asset Summit (DAS) will feature conversations between the builders, allocators, and legislators who will shape the trajectory of the digital asset ecosystem in the US and abroad.

recent research

Research Report Templates.jpg

Research

Bluefin possibly stands at an inflection point. The token is near an all-time low yet the protocol’s spot volume market share and derivatives exchange usage have been increasing month over month since its November launch. Given its current market position and the upcoming upgrades (for both Bluefin and SUI), there may be upside potential before the increased supply growth in December. However, strong opposition from existing competitors (like Cetus and Suilend), as well as new entrants (like Aftermath), pose key challenges to Bluefin’s medium-term success.

article-image

Top Committee Democrat Sen. Elizabeth Warren in her opening statement accused Atkins of “helping billionaire CEOs like Sam Bankman-Fried”

article-image

Introducing garbled circuits for enhanced privacy and regulatory compliance

article-image

Ross Ulbricht was a freedom maximalist building freedom tech, powered by Bitcoin

article-image

Solana validators can reap benefits including payments, votes and community clout

article-image

Sponsored

WalletConnect is cementing itself as the essential connectivity layer, ensuring wallets remain the entry point for billions of users

article-image

According to a legal filing, Galaxy Digital helped boost the price of LUNA while quietly selling its tokens