Key Problems in Auditing and Security

CTA: In these threads we attempt to inventory key areas of exploration that may exist in a particular discipline. These posts are living documents and it is our hope that the community will contribute to the list.

CTA: We need help identifying Key Problems in the Auditing and Security space. Please help us! Here’s what we think a Key Problem looks like:

  • Provides direction for individual research efforts and projects
  • Is a broadly applicable question or problem statement
  • Requires many coordinated research efforts to answer or solve

What makes a good audit?

Audit thoroughness varies. As auditing companies compete with one another, some are likely to dominate the market and set the standard by simply eliminating competition. Ideally, however, setting the standard for a good audit should be a collaborative effort. From a crypto community stand-point, a good audit requires different companies to talk to one another, as well as to software engineers in the crypto space. Discussions that could drive such an effort include detailed insights on auditing practices such as internal methodologies, threat models, checklists, etc. Researchers, in turn, could help evaluate existing practices within the audit space.

Here are some avenues that the research community could begin exploring:

  • What audit methodologies produce better findings, and under what circumstances?
  • How comprehensive are the threat models in crypto?
  • What is the breadth of existing audit checklists?
  • How do methodologies differ across the board, and how do we evaluate them?

Last, but not least, researchers could propose their own auditing frameworks and seek their validation within the crypto community.

What security KPIs (Key Performance Indicators) should crypto projects use?

Currently, projects rely on audit reports as a go-pass for launch. These audits report auditors’ findings when they read a target code line-by-line. Essentially, an audit relies on a single KPI: the number of code issues found, weighted by how severe they are. However, other security indicators do exist and could be used to assess the security in crypto projects during their entire lifecycle (e.g., number of individuals with privileged roles within a contract, the project’s truck factor, amount of unaudited code, number of external dependencies, test suite quality, etc). However, evidence on what KPIs effectively work in assessing how secure crypto projects are is an open question.

Here are some avenues that the research community could begin exploring:

  • What KPIs should one use when assessing the security of crypto projects? Do they have any predictive power?
  • What threshold values should one rely on when accepting or rejecting a given KPI?
  • Which KPIs do different stakeholders seek? Do they change per domain application?
4 Likes

Great to see this topic here. Was just talking to someone about it last week. IMHO some of the key-problems listed here could be broken down into stand-alone topics, e.g. the topic of testing. I created a new topic for testing (which could be seen as a sub-topic of this topic) under: What constitutes a good test suite?

2 Likes

@banescusebi Welcome to Smart Contract Research Forum — thanks for contributing!

Instead of breaking key problems into smaller sub-categories, perhaps a better approach is let their specialization to occur organically. One possibility, for instance, is to have a discussion post (e.g., as you did with testing), and then tag the post with the key problems label.

1 Like

Thanks Leo. I could swear I added that label to the testing post. Not sure what happened and why it disappeared.

2 Likes

Oh, sorry to hear that. I can put it back to the post. Maybe it was a mistake on our end. Apologies for that.

1 Like