Ethereum’s Shapella Upgrade Went Through Despite ‘Ridiculous Bug’

The latest All Core Developer call highlighted an issue with Ethereum’s leading consensus client, Prysm


sdecoret/Shutterstock modified by Blockworks


Ethereum core developers met for their first group call since the activation of the Shanghai-Capella fork on Thursday to prepare for the next phase of the network’s evolution.

Tim Beiko, who runs protocol meetings for Ethereum, briefly congratulated the assembled devs before getting down to business.

“Great work everyone on Shapella — it worked,” Beiko said.

But there was one issue of consequence: A bug in the main execution client Prysm, related to MEV-Boost, caused validators to fail to propose blocks — a crucial part of the normal functioning of the blockchain.

The Prysm team issued a post-mortem analysis on Friday, identifying the root cause as a missing field during the “unblinding” of the “builder blinded block.”

In the post-Merge Ethereum blockchain, the “builder blinded block” is a type of block that has been processed by a tool called the “builder” to hide certain information. “Unblinding” this block refers to the process of revealing the hidden information using a cryptographic technique. The missing field — BlsToExecutionChanges — was supposed to be populated during this process, but it was not, which caused the issue.

“It’s safe to say that most of the missed blocks were due to this bug,” pseudonymous Prysm developer Potuz told his colleagues on the Thursday call.

This bug was difficult to detect when Shapella was activated on testnets, like Goerli.

Luckily, a “circuit breaker” triggered early on, and relayer teams reacted quickly to temporarily disable Prysm support from most relayers, preventing the issue from having a wider impact on network operation. 

However, the incident highlighted several lessons learned, including the need for more comprehensive testing, and closer collaboration with other teams, such as Flashbots.

“We should set up a way that all of our coding is oriented towards testing the builder,” Potuz said. “This is a ridiculous bug that should not have happened at all. There’s many places where this should have been tested, and all of them failed at the same time. This is impressive.”

A retrospective analysis of the Goerli testnet discovered evidence of the bug there, but the effect was not significant enough in the test environment to attract developer attention, the Prysm team explained on the call.

Around 40% of the Ethereum network nodes run Prysm software, but that percentage has been gradually shrinking as the Ethereum community recognized the need for client diversity. Other clients, Lighthouse, Teku, Nimbus and Lodestar picked up the slack to keep the chain running.

With Shapella safely in the rear view mirror, developers can now turn their full attention to the next major upgrade, known as Cancun-Deneb, or “Dencun” for short. Cancun, Mexico, was the site of the Devcon 3 conference, in 2017, and Deneb is a star in constellation Cygnus, the swan and a vertex of the Summer Triangle.

Don’t miss the next big story – join our free daily newsletter.


Upcoming Events

Hilton Metropole | 225 Edgware Rd, London

Mon - Wed, March 18 - 20, 2024

Crypto’s premier institutional conference returns to London in March 2024. The DAS: London Experience: Attend expert-led panel discussions and fireside chats Hear the latest developments regarding the crypto and digital asset regulatory environment directly from policymakers and experts.

Salt Lake City, UT

WED - FRI, OCTOBER 9 - 11, 2024

Pack your bags, anon — we’re heading west! Join us in the beautiful Salt Lake City for the third installment of Permissionless. Come for the alpha, stay for the fresh air. Permissionless III promises unforgettable panels, killer networking opportunities, and mountains […]

recent research

Research report - cover graphics (1).jpg


In this report, we dive into crypto private market data to gather insights on where the future of the industry is headed. Despite a notable downturn in private raises, capital continues to infuse promising projects that aim to transform payments, banking, consumer experiences, community, and more, with 2023 being the fourth-largest year for crypto venture capital.


Revolut said that the standalone crypto exchange is currently “invite only”


The stock price jump comes after Coinbase reported ending its seven-quarter run of net losses during the fourth quarter


BUZZ holds shares of Coinbase, Robinhood and MicroStrategy


Opinion: Even though I didn’t pay for my “Diamond Hands” burger with BTC, don’t let that fool you into thinking that crypto’s development is futile


The results mark “a major positive inflection point,” one analyst says, as the exchange carries net income momentum into a crypto rally


While the slate of 10 US spot bitcoin funds have tallied $4.6 billion of net inflows thus far, half of the field is lagging the leaders