Byron Gilliam

This is a segment from The Breakdown newsletter. To read full editions, <a href="https://blockworks.co/brand/the-breakdown" target="_blank" rel="noreferrer noopener">subscribe</a>.



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
“You have to enjoy failure.” — James Dyson
</blockquote>



In a 2003 research paper, Google engineers <a href="https://static.googleusercontent.com/media/research.google.com/en//archive/googlecluster-ieee.pdf">shared</a> their new approach of building “reliable computing infrastructure from clusters of unreliable commodity PCs.”



This was a radical change. Instead of paying up for super high-end Sun servers engineered for near-perfect reliability, Google was embracing cheap, commodity hardware running free Linux software. They then developed additional software to work around the hardware&#8217;s frequent failures.



Redundancy through volume, they found, is cheaper than precision through engineering.



This is the model for modern AI data centers, too: You can&#8217;t have a three-month training run crash just because one GPU dies out of 20,000.



It’s not, however, the model for manufacturing the GPUs themselves.



Instead, semiconductor fabrication plants are built to produce GPUs with an ultra-low level of defects, which demands near-perfection at every step of an incredibly complex process — the opposite of Google’s recommendation for data centers. But this is not engineering perfectionism. It’s an economic necessity.&nbsp;



“Why do we need to have 99.9999% reliability at every step of the process?” Ben Thompson <a href="https://sharptech.fm/member/episode/an-open-ai-reminder-netflixs-expanding-appetite-q-a-on-remote-work-taco-bell-and-data-centers-in-space">asks</a>. “It’s all the cost of capital equipment.”



Mostly that’s because the $200 million ASML machines every fabrication plant uses to make cutting-edge semiconductors “are the most overengineered things on earth,” Thompson explains.&nbsp;



This creates a feedback loop of spiraling cost and perfection: Fabrication plants have to be near-perfect to be profitable because the ASML machines are so expensive — and the ASML machines are so expensive because they have to be perfect.



As a result, it now costs as much as $20 billion to build a single semiconductor fab.



Thompson compares the process of making semiconductors with hyper-complex ASML machines with the process of making rockets at SpaceX, where Elon Musk chooses to “blow up a bunch of Starships because you don’t want to be wasting anything on complexity that actually isn’t necessary.”



Repeatedly blowing up rockets helps SpaceX find the “exact line where you want to be,” as Thompson puts it — the line between complexity and usefulness that allows them to avoid the astronomical upfront costs required by perfection.&nbsp;



In other words, failure can be a feature, not a bug, of complex systems.



<h2 class="wp-block-heading">It’s about the journey…</h2>



Failure is a core feature of crypto systems, too: Every blockchain is designed around the assumption that some percentage of its nodes will be malicious or faulty.



This makes blockchains resilient — like Google&#8217;s data centers, they continue to produce valid blocks despite individual failures. But that fault tolerance doesn&#8217;t extend to the code running on the blockchain.



Smart contract code, written by fallible humans, will inevitably have bugs or design flaws. This can be costly, because unlike traditional finance, there&#8217;s no reversing a crypto transaction once a flawed contract or protocol is exploited.&nbsp;



Code eliminates intermediaries, but at the cost of zero fault tolerance.



This makes code-is-law crypto a system that philosophically embraces decentralized fault tolerance but architecturally demands perfection.



There is no way to design your way around that.



In traditional finance, where systems are permissioned and transactions are reversible, banks can aim for near-perfection by implementing ever-stricter KYC/AML rules and hiring ever more compliance officers.&nbsp;&nbsp;



In crypto finance, where systems are permissionless and transactions are irreversible, we can only aim to learn from our mistakes.&nbsp;



There have been a lot of them: from The DAO exploit that nearly sank Ethereum in 2016 to the record-setting $1.5 billion Bybit hack in 2025.



With luck, though, these could come to look in hindsight like Elon Musk’s exploding rockets. Each exploit reveals an attack vector — reentrancy, oracle manipulation, protocol design flaws — that can be fixed in future iterations.&nbsp;



Like Starship explosions, the failures show what actually needs hardening vs. what was unnecessary caution — each an opportunity to find the right balance of complexity and usefulness in a new kind of financial system.&nbsp;



Crypto is often judged by its failures, which is fair enough, given their frequency and size.&nbsp;



But the failures aren&#8217;t bugs in the process of building a new financial system. They are the process.



To enjoy crypto, then, you’ll have to enjoy failure, too.



Happy new year.



<hr class="wp-block-separator has-alpha-channel-opacity"/>



Get the news in your inbox. Explore Blockworks newsletters:



<ul class="wp-block-list">
<li><a href="https://blockworks.co/newsletter/the-breakdown">The Breakdown</a>: Decoding crypto and the markets. Daily.</li>



<li><a href="https://blockworks.co/newsletter/0xresearch">0xResearch</a>: Alpha in your inbox. Think like an analyst. </li>
</ul>

Crypto

Failure is an option in crypto finance

Liz Coyne

On this episode of Blockworks&#8217; podcast &#8220;<a href="https://www.youtube.com/watch?v=G6HESVcM4Ic" target="_blank" rel="noreferrer noopener">On the Margin</a>,&#8221; Michael Ippolito, co-founder, sits down with Demetri Kofinas, a media entrepreneur, financial analyst and host of the renowned <a href="https://hiddenforces.io" target="_blank" rel="noreferrer noopener">Hidden Forces</a> podcast.



Mike and Demetri discuss a wide range of topics from political volatility, the fourth turning and what prospects decentralization can offer to society.



They also covered: 



<ul class="wp-block-list"><li>Financial nihilism</li><li>US and China relations</li><li>How a bitcoin standard would differ from a gold standard</li></ul>



<figure class="wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio"><div class="wp-block-embed__wrapper">
<iframe loading="lazy" title="Financial nihilism and political volatility | Demetri Kofinas" width="1024" height="576" src="https://www.youtube.com/embed/G6HESVcM4Ic?feature=oembed" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
</div></figure>



&#8220;On the Margin&#8221; is a two-part show.&nbsp;Once a week, Mike interviews industry experts, and then at the end of the week he is joined by co-host <a href="https://cms.blockworks.co/newsletter/" target="_blank" rel="noreferrer noopener">Tyler Neville</a> where they break down the top crypto and macro stories. Episodes go live every Wednesday and Saturday morning.



Subscribe to &#8220;On the Margin&#8221; today on <a href="https://podcasts.apple.com/us/podcast/lyn-alden-explains-connection-between-inflation-bitcoin/id1558223079?i=1000513411286" target="_blank" rel="noreferrer noopener">Apple</a> or <a href="https://spoti.fi/3bTa9uw" target="_blank" rel="noreferrer noopener">Spotify</a>, or watch episodes on <a href="https://youtube.com/playlist?list=PLvER9nHSRN3xVs_p-dl7L7K_ZgL6lSudu" target="_blank" rel="noreferrer noopener">YouTube</a>.

China

Podcast: When Global Superpowers Collide

On this episode of Blockworks&#8217; podcast &#8220;<a href="https://www.youtube.com/watch?v=N4j2op6uIo4" target="_blank" rel="noreferrer noopener">On the Margin</a>,&#8221; Michael Ippolito, co-founder, sits down with former presidential advisor and economist Dr. Pippa Malmgren. Dr. Malmgren shares insights on the effects of decentralization, creative destruction, debt restructuring, and how crypto fits into her outlook.



They covered: 



<ul class="wp-block-list"><li>Decentralized Finance (DeFi)</li><li>Centralization vs. Decentralization</li><li>Innovation&#8217;s effect on the Future</li></ul>



<figure class="wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio"><div class="wp-block-embed__wrapper">
<iframe loading="lazy" title="Globalization, Decentralization &amp; Innovation post COVID | Pippa Malmgren" width="1024" height="576" src="https://www.youtube.com/embed/N4j2op6uIo4?start=182&#038;feature=oembed" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
</div></figure>



&#8220;On the Margin&#8221; is a two-part show.&nbsp;Once a week, Mike interviews industry experts, and then at the end of the week he is joined by co-host <a href="https://cms.blockworks.co/newsletter/" target="_blank" rel="noreferrer noopener">Tyler Neville</a> where they break down the top crypto and macro stories. Episodes go live every Wednesday and Saturday morning.



Subscribe to &#8220;On the Margin&#8221; today on <a href="https://podcasts.apple.com/us/podcast/lyn-alden-explains-connection-between-inflation-bitcoin/id1558223079?i=1000513411286" target="_blank" rel="noreferrer noopener">Apple</a> or <a href="https://spoti.fi/3bTa9uw" target="_blank" rel="noreferrer noopener">Spotify</a>, or watch episodes on <a href="https://youtube.com/playlist?list=PLvER9nHSRN3xVs_p-dl7L7K_ZgL6lSudu" target="_blank" rel="noreferrer noopener">YouTube</a>.

innovation

Newsletter

Blockworks Research