Research Summary: Smart Contract Security: A Practitioner’s Perspective (2021)

Auditing and Security
, ,
1

TLDR; Smart contract engineers are required to be more aware of security issues than regular software engineers. They use multiple tactics to address security issues and continually address them throughout the lifecycle of smart contract development.

  • Smart Contract Engineers keep up to date with the latest security trends and hacks via Q&A websites (47%), blockchain forums (60%), and research papers (53%) [p. 1416].
  • Seventy-two percent of smart contract engineers surveyed used more than one security strategy in smart contract development, including code review, code style checking, reuse of code from reliable sources, and integration of fuzzing into the development lifecycle.
  • The most commonly used tools for smart contract security were Mythril (19%), Oyente (12%), SmartCheck (14%), and Slither (12%).
  • Future work may seek to do three things:
    • (1) systematize smart contract security approaches for the industry;
    • (2) identify common smart contract libraries for reuse of snippets of validated and debugged code;
    • (3) investigate various blockchains and coding tools through comparative analysis lenses for platform-level vulnerabilities.

Core Research Question

For smart contract engineers, what security practices are common throughout the lifecycle of smart contract development?

Citation

Z. Wan, X. Xia, D. Lo, J. Chen, X. Luo, and X. Yang, “Smart Contract Security: A Practitioners’ Perspective,” 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), 2021, pp. 1410-1422, doi: 10.1109/ICSE43902.2021.00127.

Background

  • The intended audience of this survey was smart contract auditors who are curious about general tactics. It was not intended to be a rigorous deep dive into methods and software development lifecycles.
  • Smart contracts are an appealing decentralized application technology for enabling computation on top of blockchains in response to off-chain events.
  • Smart contracts consist of executable code that enforces the terms of an agreement between untrusted parties on a blockchain.
  • Smart contracts are plagued with security issues. For example, The DAO was hacked for USD $34 million worth of Ether. (For more information on the latest hacks, please visit the rekt.news site.). Researchers have proposed language-based security, static analysis, and runtime verification to address security issues.
  • Academic research, before this article, did not cover practitioners’ tactics for addressing those security concerns.
  • Using a mixed-methods approach, the authors conducted 13 qualitative interviews with smart contract engineers and deployed a quantitative survey of 156 smart contract engineers to gather intel on their security practices during development.

Summary

Smart contracts are executable code built on top of blockchains. They are extremely useful for executing agreements between untrusted parties, and promise to be a key enabler of social automation in the 21st century. However, smart contracts remain rife with security bugs that can result in very costly exploits. Smart contract engineers care about protecting their users as well as their companies’ reputation for secure code. The security issue mitigation tactics used most commonly are code reviews, code reuse from reliable sources, and fuzzing tools incorporated into engineers’ continuous integration/continuous development process.

Keeping up to date with security issues is a challenge. The most common sources of information for the latest security news according to the paper are blockchain forums, question and answer websites, and research papers.

Practitioners are using more than one tactic for securing their smart contracts, but are in need of a way to systematize an industry standard approach to security within their development process, as none currently exists. Future work may seek to do three things: (1) systematize smart contract security approaches for the industry, (2) provide common smart contract libraries reused and snippets of helpful code, and (3) investigate the different blockchains as well as different tools through comparative analysis lenses.

Methods

There were two methods used: qualitative interviews (16 participants) and a survey (156 participants) deployed.

  • Qualitative Interview Methodology:
    • Three categories of questions were asked:
    • (1) Asked demographic questions
    • (2) Asked open-ended questions about security awareness and practices of smart contract development.
    • (3) Asked interviewees to discuss their sources for security-related knowledge, strategies, and tools for smart contract security management.
    • Qualitative interviews used to open card sort and determine a list of statements and potential answers for the developing survey questions.
  • Survey Methodology:
    • Includes multiple-choice and free-text answer questions. Multiple-choice questions derived from qualitative interviews.
      • (1) Asked demographic questions
        • Do you have experience with smart contracts?
        • What best describes the primary blockchain platform you currently work on? Potential answers were: (1) public blockchain (2) consortium blockchain, (3) private blockchain, and (4) other.
      • Used 5-point Likert Scale ( strongly disagree, disagree, neutral, agree, strongly agree) to rank the importance of each questions in examining the following categories:
        • the importance of security for smart contracts
        • security motivators and deterrents
        • efforts towards security of smart contracts
        • experience of security problems with smart contracts
        • sources of security knowledge related to smart contracts
        • strategies for addressing smart contract security
        • tools used to detect security vulnerabilities in smart contracts

Results

  • Who took the survey?
    • Most respondents were based in China (61 participants) and the United States (16 participants).
    • They averaged two years experience with smart contracts per respondent.
    • Most respondents to the survey worked on public blockchains (80) such as Ethereum.
  • Results are divided into 3 categories:
    • (1) Perceptions of Smart Contract Security,
    • (2) Security Practices in Smart Contract Development,
    • (3) Effect of Blockchain Platforms on Security Perceptions and Practices

These following are the results gained divided into the above 3 categories.

  • RQ1: Perceptions of Smart Contract Security:
    • Importance of Security
      • 85% rated security very important
      • 69% rated security extremely important
    • Security Motivators
      • “I care about smart contract security because…”
        • protect users
        • protect the reputation of their companies
    • Security Deterrents
      • 40% responded they had had at least one out of three potential security problems:
        • vulnerabilities in unshipped code
        • vulnerabilities in shipped code
        • security breaches
      • Security problems occurred when?
        • Before code shipped (22%)
        • After code shipped (19%)
        • Security breach after code shipped (10%)
    • Sources of Security Knowledge
      • blockchain forums (60%)
      • research papers (53%)
      • question and answer website (47%)
      • support from professional security experts (47%)
  • RQ2: Security Practices in Smart Contract Development
    • Efforts toward Security
      • 29% of time overall developing smart contracts
      • 14 respondents indicated that no time was spent on security.
    • Strategies for Handling Smart Contract Security
      strategies for handling smart contract security
      strategies for handling smart contract security894×534 202 KB
      • 72% of respondents use more than one tactic for security mitigation
      • 72% rely on code review often or very often
      • 61% do code style checking
      • 58% reuse code from reliable sources
      • 28% incorporate fuzzing into the development lifecycle
      • Free text answers included:
        • Security By Design
        • Programming Languages
        • Dependency Management
        • Learning from Past Experience
        • Seeking Support from Experts
    • Tools to Address Smart Contract Security
      • 54% adopt smart contract security tools in their development
      • 45% rely on security plugins in IDEs
      • 19% use Mythril
      • 12% use Oyente
      • 14% use SmartCheck
      • 12% use Slither
  • RQ3: Effect of Blockchain Platforms on Security Perceptions and Practices
    • Security Motivators
      • practitioners of public blockchains are more motivated to address smart contract security concerns than those of consortium or private blockchains.
    • Security Deterrents
      • practitioners of public blockchains are more likely to take responsibility for addressing smart contract security
    • Security Efforts across Stages
      • Practitioners of public blockchains spend more effort towards security throughout the six stages of the development lifecycle
    • Security Strategies
      • public blockchain practitioners tend to perform code reviews more frequently than consortium or private blockchains

Discussion and Key Takeaways

  • Smart contracts are more prone to attacks than regular software.
  • More frequent code reuse specifically exposes more security risks than in regular software development.
  • Most smart contract engineers use more than one method of securing their contracts throughout the lifecycle of their development.

Implications and Follow-Up

  • Future work may be standardization and operationalizing the process of building security into smart contracts for practitioners.
  • Future studies may include code snippets and smart contract libraries with helpful examples and tools to facilitate library updates.
  • Future studies could investigate the differences between blockchains, as well as examine and detail the differences between different tools used for smart contract security.

Applicability

  • As a smart contract engineer, more than one method of security smart contracts throughout the lifecycle of development should be used.
  • Incorporate code reuse, code review, and fuzzing tools into your development cycle for security best practices, according to this study.
  • Security tools most commonly used by smart contract engineers are Mythril, Slither, Oyente, and SmartCheck.
  • Be mindful of blockchain platform limitations, language design limitations, and smart contract specific potential attacks as outlined in codified references such as the SWC Registry or the DASP registry.
9 Likes
2

Hi, Max. Thank you for contributing.

I am curious why you might think the division among smart contract engineers regarding security tools is so varied. I say this because it seems that naturally 1-2 tools stand out in general but not in this case (aside from the IDE plugins).

  • When you refer to standardizing/operationalizing security building, do you have any examples of this already taking direction in the status quo?
  • If not, what are the industry factors that are preventing systems from doing so?
3 Likes
3

I’m interested in your thoughts about the claims versus the sample sizes for the survey and interviews. The cautious researcher in me would be putting caveats with the findings and suggesting some follow-up steps of larger sample sizes and maybe longitudinal surveys…

2 Likes
4

@maxgrok it’s a real pleasure having this in the forum, and thank you so much for contributing this interesting summary. A quick question to kick things off: can you provide us with some context for what the crypto auditing and security landscape looks like at the moment? Are there any institutions devoted to ensuring standards or trade organizations?

1 Like
5

@maxgrok, many thanks for submitting this summary to the forum. Obviously, the subject is extremely important. For me, this statement jumped off the page:

Because smart contracts promise widespread social automation, I would think that code security in this domain would be at least as important as that of, for example, autonomous vehicles. Competition and different approaches exist in that field, too, but the idea that an adversary might take over the steering or accelerator of multiple cars is a sobering prospect indeed. No one would say that “an industry standard approach to security” doesn’t exist for autonomous vehicles.

So how do you account for this situation in the field of smart contract coding? Is the field itself (or this study) simply too immature to base any real-world conclusions on?

6 Likes
6

My guess is that the field immaturity hypothesis is a strong one, @rlombreglia

A few months back, there was a great discussion that sprang up in response to a post by @lnrdpss that might also enhance our understanding of this research summary.

3 Likes
7

I agree. I think that the sample size could have been bigger. For what it is, it’s a good entry-level understanding of the smart contract security space, but would have liked it to have gone deeper and longer.

For example, instead of just one interview and a survey at mass, it would be nice to have followed their practices specifically asking questions about how they conduct manual reviews and how frequently they change /update their tooling.

However, the claims are pretty generally accurate. There is nothing that I came across that seemed outlandish.

8

There is the Enterprise Ethereum Alliance that is building auditing standards, as well as the SCRF compiling common best practices, but there is not a general agreed upon standard. Each smart contract security firm has their own style preferences and standards for auditing right now. What is preventing it from happening is that these firms are not incentivized to share their knowledge properly with the larger community. They profit from knowledge working and staying on the cutting edge.

I believe in a more collaborative approach and am looking forward to seeing the standards evolve and grow into something more standard.

1 Like
9

Right now, as for the industry, it’s pretty ad hoc and word of mouth who is skilled at what. You really need to network and be meeting people in person at conferences or getting referrals to auditors. The known and skilled auditors are booked up 6 months in advance to even take a look at one’s codebase. There is a huge gap between supply and demand. Supply being small and demand being sky-high.

There is an auditing DAO I am co-founding called Spearbit (along with Jake Lang and Spencer MacDonald, as well as advisors and other team members) that seeks to ameliorate this by scaling and upskilling more junior auditors, pairing them with senior auditors, and takes a collaborative, knowledge sharing approach to smart contract security auditing.

As for your second question, the EEA is dedicated to creating standards for auditors right now as one of their projects and the SCRF is attempting something similar last I checked. There are no standards or trade organizations that I know of that are enforcing standards that are reliable and up to date like accounting standards has GAAP in the US. It’s much more fluid than that and new attacks are happening all the time, so the lead time on standard creation vs. standard enforcement is rather high. i.e. by the time standards may form, they are out of date.

2 Likes
10

There are a lot of people who have opinions about standards, but no agreed upon standards by the large, well-known auditors or auditing firms. This would require cross-company/DAO collabs. As of right now, each has its’ own knowledge base, knowledge work, and they protect it.

I’m not surprised at the shock of the lack of standards. I’m surprised too, but it’s only been 6 years since Ethereum was founded. It’s still wild, wild west territory and ,unlike ‘autonomous vehicles’, no one is literally dying directly as a result of the code. It’s mostly arbitrage financial attacks and code exploits, which, while they can be financially devastating, are not in and of themselves lethal. There is, for this reason, less lead time for launching code to production and more of a focus on smart contract security as ‘optional’ in some projects’ opinions.

i.e. the field is relatively new and there is currently a coordination happening to create those standards, but all we have are opinions right now. Secureum is the closest one can come to the body of knowledge required to be an auditor, but it just started in October and the results are up in the air.

That said, there are common vulnerabilities that are known such as the SWC registry that is commonly referred to or Solidity best practices, but they do not cover comprehensive protection.

I hope that sufficiently answers your questions!

2 Likes
11

In addition to community based efforts, one should also take into account platforms imposing their own security standards. For instance, centralized exchanges have strict guidelines that new tokens must comply to; same for IEO platforms (e.g., Coinlist). So, either you comply or you are out.

2 Likes
12

@maxgrok One thing that caught my attention was the fact that 28% of respondents claimed to use fuzzing, but no fuzzing tool was listed amongst the tools used by developers that participated in the survey. Do you happen to know what fuzzing tool were respondents referring to?

1 Like
13

There is also DeFiSafety, which is a platform that ranks DeFi projects based on specific quality metrics - see their Process Quality Review Process (PQRs). A publicly available list of scores can be found here.

This sort of effort is a great way to push projects towards better security; if a project gets a low score, it is just bad PR. In a way, the incentives are there :slight_smile:. CertiK also has a score system, but it is less clear to me how they achieve scores.

While the PQRs from DeFiSafety are by no means a bullet proof standard (and in all honesty, no standard will ever meet that level), it has great benefits:

  • it comes from a neutral entity that has no financial incentive to increase/decrease scores;
  • scores are verifiable, as items have clear guidelines on how to be evaluated;
  • reports are publicly available.

Standards, IMHO, will organically start to happen from initiatives such as DeFiSafety; others will likely to be imposed by central platforms (e.g, token security in exchanges); some will be pushed by auditing firms, as projects comply with practices set by auditing firms as a pre-requisite for acceptable security levels. Following a bottom-up approach, eventually, the community will converge on what is acceptable from a security stand point and what is not.

4 Likes