Stop calling them audits

One of the things you learn when you join a professional services firm like EY — which has its history rooted in financial statement audits and is still very much present in this business — is that everyone seems to use the word audit without actually understanding what it is or the rules that go around it.

I’m not an auditor, either by training or current profession, and I’m not here to write about how audits work. I’ll leave this to my audit colleagues. What I do want to do is take a moment to share all the many things in the world of blockchain and crypto that are not audits — even though they get called that all the time. 

For blockchain and crypto to thrive, we need a whole range of verification tools that go far beyond a traditional financial statement audit. Equally important, we need to be more specific in the language we use to describe them. 

Instead of calling everything an audit (and implying there’s some kind of magical guarantee that comes with that word), let’s start looking at the full arsenal of tools with names that aren’t so misleading.

How to avoid being mislead

Crypto loves to refer to other kinds of financial tools as audits. However, even though some of these tools literally have “audit” in the name, they are not actually audits.

Attestation reports

While financial statement audits are done on an annual basis, attestation reports are reports performed under similar professional standards that can be done as needed and for a specific activity or area. Attestation reports typically have to be signed-off by a certified public accountant. There are quite a few different kinds of these reports, and none of them carry the same weight as a full audit.

One area that I think will become routine for attestation reports are stablecoins. Banks, or stablecoin firms, will want to have professional standards for their reserves reports, specifically around how they support any one digital asset off-chain, and how this aligns with the on-chain liabilities.

It’s important to note here that “attestation” reports provided by non-audit firms do not carry the same weight or professional requirements as those that are done by certified public accountants.

Agreed-upon procedures (AUPs)

AUP reports are when an auditor performs specific, agreed-upon procedures and generates a report: They’re all about testing some specific facts. 

For blockchain, a fact that might be tested is “did this token get transferred within the agreed upon dates.” AUPs, unlike audits, do not have an opinion associated with them, but they do provide an analysis of a specific part of an organization’s activities and responsibilities conducted by a third party. 

The AUP report on a token transfer would not, for example, make any assessment as to whether or not the price paid for that token is a fair market value. AUPs are often done to analyze a specific element of the business. However, because the scope is “agreed upon” by the engaging party(ies), they can often be unsuitable for third-party users.

Systems and organization controls reports (SOC)

In many cases, it is not possible for auditors to feasibly keep track of all information or verify it at all times. Additionally, there are often few choices for data available, so there is no alternative — you have but to trust a single data provider. 

Though this is not yet widely the case, it is easy to see a future where smart contracts are triggered by external data that is only available from a single supplier. For example, if I have a smart contract and payment is triggered by delivery of the goods, that means I am dependent on the logistics company to report delivery of those goods.

SOC reports provide information on third parties’ control over data, as well as the evidence of how these controls function. In this case, auditors cannot verify every shipment, and it is not feasible to apply crowdsourcing to delivery data on millions of packages. 

However, auditors can understand and test the process and controls that a company has in place for reporting out this data. Third-party service providers can have an SOC engagement performed, and share that report with whomever they provide services for (and their auditors).

SOC reports come in three major “flavors.” SOC 1 reports usually concern financial controls while SOC 2 and 3 reports cover security, availability, processing integrity, and privacy business controls. 

Over time, I expect to see many organizations obtain SOC reports for key parts of their blockchain-related business process.

Smart contract audits (aren’t)

One of the most common services offered in the world of blockchain software are smart contract audits. These are generally designed to see if a smart contract functions as expected and is free from major technical errors, bugs or security weaknesses. In theory, a thoroughly audited smart contract would be much lower risk than one that has not been carefully reviewed and tested.

Unfortunately, smart contract audits (despite the name) are not audits and should not be treated as such. 

Indeed, while EY offers services that are very similar to what other companies call “audits,” ours are called “smart contract reviews” for a very good reason: We’re not offering any assurance or guarantee that the software is free from bugs. There isn’t any known practical way to make such a guarantee. Additionally, there really aren’t any globally regulated and widely adopted standards for software verification against which one could objectively compare a particular smart contract.

That is not to say I don’t strongly recommend smart contract reviews. I do. And because there isn’t any perfect standard for securing smart contracts, the more effort you put into testing and bug hunting, the better. You should get more than one if you’re talking about serious money being deployed into these applications or sensitive data. But just understand it’s not an audit, and it doesn’t come with the kind of rigorous expectations you would have with a globally regulated and standardized audit approach.

Proof-of-reserves (also not an audit)

More recently, companies and projects have turned to proof-of-reserves (PoR) as evidence of their solvency. The idea is that individual users should be able to match their account balance with the on-chain data — if an exchange shows their PoR, their on-chain balance should be equal to how much money their users have deposited. 

In practice, this doesn’t work well: Firms that have offered this service have generally stopped doing so. 

There are two problems that often come up with proof-of-reserves. First, it’s technically and mathematically difficult to follow the on-chain and off-chain data verification process. 

I had some EY R&D staff look at cases where they had accounts that offered “proof of reserves.” In both cases, while they were able to make the numbers “match up,” they all found it challenging, time-consuming and not particularly comforting. And these are people who have full-time jobs in blockchain and doctorate-level math skills and programming skills. If it’s this hard for them, it’s basically impossible for the rest of us.

Secondly, proof-of-reserves doesn’t take into account the rest of the organization or balance sheet. Yes, the firm might have all that money, but if they have terrible business controls and someone else is borrowing against that money, it’s not necessarily all there for the depositor.  

Proof-of-reserves presents only one slice of a total financial statement picture, and it’s much too easy to imagine ways to manipulate that data to make the numbers add up.

Trust but verify

The idea that we can “trust the math” and not worry about the people involved is enchanting, but it has led nowhere. 

Complex technology systems have unpredictable behaviors, so even if every blockchain investor and user was mathematically sophisticated and technically proficient, we would still face risk. On top of that, we’ve seen time and again that this is a business with lots of bad actors. In this environment, both for your business partners and your own operations, it’s hard to overstate how important multiple layers of external verification and transparency are worth.

Because no oversight process is perfect, the more transparent and deeply entwined into a regulatory environment a firm is, the better. Being audited is good, but being audited and also regulated by, say the FDIC, the Federal Reserve or the Securities and Exchange Commission is probably even better. 

And never ever believe a firm that says their systems are too sophisticated or too complex for an auditor to understand.

Start your day with top crypto insights from David Canellis and Katherine Ross. Subscribe to the Empire newsletter.