CTA: In these threads, we attempt to further the discussion of a key problem in this category and evolve our understanding of the domain space where research work has not yet answered the specific problem or question being considered. These posts are living documents, and it is our hope that the community will continue to contribute to their structure and content.
This post discusses the basic approach to a security audit, using Ethereum smart contracts as an example.
How should one perform code audits?
- An audit is an independent assessment of a project, seen through the lens of a particular concern. Examples of these concerns include security, compliance, economics, etc.
- In the crypto space, audits have largely focused on security, with their primary goal being to assess correctness (with respect to a specification), as well as to identify potential vulnerabilities in the implemented code. These are known as code audits.
- A code audit assesses the security quality of a target code, typically as a means to convey trust to the community at large.
- As in any other business-to-business relationship, an audit has an agreed timeline and scope, and its findings point to specific improvements in the code, ranked according to a severity scale.
- NOTE: If you are interested in improving the understanding of what is an audit, please visit the “What is an audit” thread. Most of the definitions herein presented stem from that post.
Essentially, a code audit relies on two things: automation and manual review of the target code.
Automation concerns the use of push-button scanners that automatically attempt to discover pre-defined bug types. The latter includes arithmetic overflow/underflow, reentrancy cases, assertion failures, transaction order dependencies, etc. For auditing of Ethereum smart contracts, two main tools are widely used: Slither and Mythril. These tools simulate the execution of the target smart contract, attempting to find execution paths matching specific issues. In the realm of formal verification and testing, the underlying mechanism used in such simulations is known as symbolic execution.
Slither and Mythril (and symbolic execution tools alike) can only identify a set of generic issues that are independent of the contract logic and business rules. Most of these tools also report a significant number of false positives (which get easier and faster to spot the more experienced an auditor becomes). Hence, findings are limited in scope. Human reviews generally aim to inspect the code in a line-by-line fashion, attempting to find issues dependent on the contract implementation and underlying business requirements. The quality of such reviews directly reflects the experience brought by the audit team.
Unfortunately, the road to learning how to do a quality manual code audit is not straight. This is due to the fact that a generic and deterministic audit script (proprietary or not) does not exist; manual code inspection and the creation of an underlying mental model of the code and its related issues depends on human abstraction, which in turn depends on subjectivity and training. Hence, outputs vary from auditor to auditor.
The best approach to becoming an auditor is to learn by doing. For training purposes, I recommend the following resources:
- Ethernaut a game in which one hacks smart contracts, proceeding through phases
- Hack solidity: an online and hands-on list of videos showing on how to execute specific contract exploits
- Smart contract security best practices: a curated list of security related-issues in smart contract development.
From those, an aspiring auditor should be able to memorize the major issues in smart contract development.
As a general yet non-deterministic guideline, an auditor should at the very least be able to perform two steps when manually auditing a smart contract:
Given a specification (e.g., white paper, internal docs, external docs), an auditor should verify that the code is doing what the specification states. The goal is to check whether the code is in line with the business requirements. This is an intensive and tedious task. Essentially one reads the code, understands it, and makes sure it meets the given specification.
From the previous step, the auditor should have been able to glean a mental model of the target code. The next step requires more insight. It involves investigating how a user might put the pieces of a contract together in a way that the original developers never intended. Here an auditor needs to have (or develop) an attacker’s mindset. This comes with experience. A good way to bootstrap that mindset is to read post mortems and replay the attack itself. Having built a mental model of the target code, one starts investigating potential threats, by means of asking questions like:
- Is it possible for any actor to put the contract in an unusable state?
- Is there a sequence of transactions that could cause the smart contract to misbehave?
- Are funds secure?
- Are there underlying assumptions made by the contract that may not hold?
- Are there interactions with external contracts? If so, are they trusted? If not, how could they exploit the current code?
While there is no single prescriptive approach, there are a few basic strategies one can take to start a smart contract audit. Additionally, in the context of having general strategies, it becomes unlikely that any two auditors will find the same set of issues.
It will be useful to continue to add to the list of auditing tools and strategies to ensure that auditors build upon each others’ experiences.