Post Idea: Found a post-mortem on Zapper Polygon bridge attack, that authors have conducted together with Zapper.fi from their side: My Questions

Research Body:

"Here is a small story of one vulnerability that could lead to a disaster but luckily didn’t.

It turned out that over 30 contracts of Zapper.fi had a critical bug since November 2020. All of them used similar logic of external calls to exchanges like 0x. As a result, anyone who gave an allowance to spend ERC20 tokens to any of those contracts were in danger of losing tokens from their wallet at any time.

Over $100M of user funds exposed to that attack. Every user that has used Yearn zap or Zapper native functions once in the past 7 months could wake up one morning without anything in their wallet. Luckily, all of the vulnerable contracts were fixed in 4 iterations of fixes that happened during the past 3 months.

We independently found that issue reported it and exploited the last remaining contract with the bug to secure user funds.

Story

Recently, we’ve been developing a new project, and we were looking for a solution to zap user funds. During this research, we independently found a significant security breach in one of the Zapper.fi contracts on June 8th 2021. It turned out that this issue exists in the other 35 deployed contracts by Zapper from November 2020. However, most of the contracts were paused in mid-April and the end of May.

On June 10th 2021, white-hat hacker lukash_dev reported the same issue in the newly deployed contracts that still had been there. Zapper.fi had paused the contracts and took a few days to release a post-mortem. Pausing the contracts was a solution, but sadly not the solution. Later, Zapper.fi released a post-mortem to disclose the vulnerability, so we decided to act.

The vulnerability in those 35 contracts potentially lets anyone call any method of any contract on behalf of Zapper.fi contracts. Therefore all wallets that had any ERC20 approvals for that contract turn out to be vulnerable. And any exploiter could transfer funds from user wallets to their own directly via specially engineered calls to Zapper.fi contracts.

Although Zapper team had paused most of the vulnerable contracts, one contract had been still there. Luckily, it was under the radar for most users. The one (due to an engineering error) lacked a “pausing” modifier for its Zap function. So, in contrast to all other Zapper contracts, this particular contract is unable to be paused.

The contract itself is an outdated Polygon bridge zapper

0x1f0d1927498fbd4f9e8558704ce5b658929527ec

It has the modifier stopInEmergency , however, it’s not used for ZapBridge.

And here is a line contains the vulnerability that is quite similar to all of the rest 35 contracts.

(bool success, ) = swapTarget.call.value(valueToSend)(swapCallData);

It allows anyone to call any contract with any data on behalf of Zapper’s contract. It was intended to be used to integrate with external exchanges like 0x or 1inch.

The problem there is that it is possible to set the address of swapTarget to any ERC20 token and encode a call to transferFrom(userWithAllowance, exploiterAddress, amount) method into swapCallData and it will go through. As a result, funds will be transferred to exploiterAddress from the userWithAllowance wallet, which gave an allowance to the vulnerable contract any time in past.

It was a huge risk to users’ funds, so we’ve decided to step in. The biggest problem with this broken contract is that there is no “right” solution to the vulnerability. Only performing your own attack.

By the time of the lukash_dev disclosure, we had already developed two smart contracts for the exploit; we were also checking if a similar issue exists in other big projects so that our disclosure won’t potentially damage anyone else. (Luckily, most of the other contracts had been implemented safer, except one minor contract that we’ve reported to another team).

The first smart contract: 0x7284E5CD49F47Da42d355BcB5ab64Fbb45D7eff6 exploited the logic; took all funds from users with non-zero allowance for the vulnerable contract; and deposited it into the second contract: storage.

The second contract 0x63cd1f35063e1bdd01355fb2bee4ecee05d94b84, the safe storage, allows to deposit user’s tokens, but it doesn’t allow to claim them until the user removes the ERC20 allowance to the vulnerable Zapper contract.

After development, we’ve reached Zapper, collaborated with their dev team, deployed contracts, and performed an attack. In the end, we’ve transferred ownership 0x6a3eedcd970b3ba2c2d24942aa81e46ab07479be02e95e709308e82592615fca of the safe storage contract to the Zapper team.

We hadn’t used any private pool, and as a result, part of our transactions had been frontrunned by some bots. We apologize for that.

Luckily, the bot owners have contacted the Zapper developer team, and most of the funds appear to be restored. The rest will be covered by Zapper. No user will lose funds.

Overall ~$2.5mln worth of tokens were affected by the attack. Around $600k of them were frontrunned by a white-hat bot and should be returned soon.

We really want to thank Zapper for their rapid response and competence.

Next steps

All users who interacted with this particular contract 0x1f0d1927498FBD4f9E8558704Ce5B658929527Ec should remove all their approvals to that contract as soon as possible.

Users that got their funds taken should remove approvals and claim their funds from storage at any time.

It is available via Zapper front-end.

It’s recommended for users to remove all approvals from all old Zapper contracts. Being those contracts accidentally unpaused, users are risking losing everything. (check the additional remark)

All currently working Zapper contracts are safe to use and vulnerable free in our opinion.

Additional remark

Right now, the other 34 Zapper contracts still have the same issue but have been paused. (Here’s the list of them)

It means that all users who have approved their tokens to Zapper might lose their funds if contracts become unpaused. Tens of millions of dollars worth of user tokens are at risk and can be taken in 3 minutes, in case Zapper deployer wallet become compromised, or someone unpauses vulnerable contracts by mistake 0xfF350eDc2242Ca4d7252A64746aec4A5487a852B. We urge the Zapper team to renounce the ownership of the vulnerable contracts, thus guaranteeing that those contracts will never be unpaused again.

The bug was there for over 7 months. During that time period, anyone could have take tokens from all users that interacted with those vulnerable contracts. Over $100M worth of tokens were at risk. Yearn.fi had integrated Zapper in March, and most people who used yearn zap function between March and April were susceptible to this exploit.

The bug was partially fixed in April, partially fixed in May and finally resolved in June. However, the time frame for the potential exploit was alarmingly wide. This shows how fragile DeFi is.

We urge all other developer teams that using similar logic (integration with external services like 0x, 1inch, paraswap) to pay close attention to their code and re-review it for potential problems.

Morale

The next big DeFi exploit may come from the fact that we, as users, carelessly leave many approvals for every contract that we have ever used. This time, the ecosystem managed to dodge the bullet. As a user, manage your approvals with more accuracy. Unlimited approvals have been rarely abused in the past. However, they might damage the ecosystem really hard in the future.

Stay tuned for some amazing news onwards.

AndreiKei , VV."


I have the following questions:

  1. Are other projects vulnerable to this kind of issue?

  2. The authors state that they "hadn’t used any private pool, and as a result, part of their transactions had been frontrunned by some bots. ", Do you think they are telling the truth? Was there really an attack?

  3. Well, and the third question, as practice has shown frontrun bots can protect the project from the withdrawal of money under “certain” conditions, whether their use is justified ethically? Looks like they might own this bots.

11 Likes

This would definitely make a great subject for a full post or discussion topic.

  1. Considering the general approach to copying code base, there are undoubtedly going to be projects that copied the problematic smart contracts/code base as the source for their defi projects and will be subject to the same vulnerabilities.

  2. While we have to take their word at a certain point, the analysis of the smart contract would show the activity and will confirm or deny their claims concerning the movement of funds if the attacks were not in fact processed in private pools. Considering how the attack played out, their narrative about a white-hat frontrunning transactions is believable. The Flashbots team would likely have engaged in this manner and not kept the funds recovered from the attack, so I would not be surprised if other white hats acted in the same manner.

  3. This is a great question in that you’re right that frontrunning can help projects from having funds removed when they are vulnerable by doing it first with the bots being deployed by those wanting to protect the network instead of attack it. They don’t necessarily own the bots, but it is likely that the bot owner does not actually want to harm the networks or users within the networks and is behaving in line with a white hat. Whether it’s “ethical” or not is debatable, but the problem is that in a MEV controlled environment, it’s inevitable. In that context, it seems relevant to front-run “for good” to prevent nefarious attackers from exploiting vulnerabilities.

7 Likes

Thank you! This is very intriguing information to think about, especially the third point. I will try to make an analysis of such methods.Front run for good could become popular - I’ve seen this kind of thing a couple of times :slightly_smiling_face:

5 Likes

Definitely read Flashbots Flashbots: Frontrunning the MEV crisis | Flashbots (medium.com) if you haven’t yet!

5 Likes

Welcome to the forum @OffcierCia. Way to kick it off strong!

There is definitely a discussion post (or two, or three, or…) here. Collaboration between you and @Larry_Bates?

6 Likes

Thank you! I want to first make a kind of sketch of the picture with statistics and then present it to the public discussion, working on various works now

6 Likes

This is a fantastic thread. Interesting to see how this all worked. I think a bit more color on the statistics of the post-mortem would make for a compelling read.

6 Likes

Firstly I’d like to congratulate you on finding the issue and successfully saving the funds. It’s unsettling that this has been an issue lingering in the open for 7 months before anyone discovered it. Are there any new best practices that came out of this that project teams could deploy to identify vulnerabilities?

It’s unlikely integration with other protocols and services is going away, composability is a DeFi property users love, it’s only likely to see growth in numbers. Are there any “rules of thumb” that teams should keep in mind when building and deploying integrations?

If I’m understanding correctly this was not an issue specific to bridging chains but rather an issue that just happened to occur on bridge contract?

6 Likes

Thank you from all neon finance whine hats! I talked to the direct author of the test attack, he told me that it was a problem in the bridge and that they were surprised by the strange reaction of Zap: “https://twitter.com/zapper_fi/status/1404429179794362369”… Cause they found 100M$ under this issue. Well, I ve collected most of security check lists here: GitHub - OffcierCia/ultimate-defi-research-base: Here we collect and discuss the best DeFI & Blockchain researches and tools. Feel free to DM me on Twitter or open pool request., (security section)

the best tactic is to fork ETH and repeat all known attacks, anyway IMHO.

5 Likes

Also, the author of the attack confirmed that the frontrun bots did not belong to them, and did not get in touch with them, probably the project or community own bots. The original plan was to return the funds publicly, but using your own pool is a risk, so Autho decided to hope there were no bots. Which of course did not happen.

In summary, we can say we see an example of a well-planned attack and a very interesting and apparently not fully understood by the team method of protection.

4 Likes

There is one problem that bothers me.
Why the normal running contract 0xe34B087Bf3C99E664316A15B01E5295eB3512760 is v1.1
The contract with the vulnerability 0x1f0d1927498FBD4f9E8558704Ce5B658929527Ec is v1.2

However,Zapper Twitter called the contract with vuln is “Old Version”? :thinking:

4 Likes

Also find it strange. Neon white hats said that Zap even ignored them

1 Like

First few days after a public release of a vulnerability always have unreliable information exchanges that later get clarified. I wouldn’t put too much weight into the language being used as being intentionally misleading or anything like that.

Also, concerning Zap ignoring them: not making excuses but a lot of developers don’t have the greatest social intelligence. If the Zap team already knew about it, they might have chosen to ignore a prompt instead of acknowledging it. A “thanks” or some sort of reply was probably warranted, but if the Zap team already knew about it the Neon people wouldn’t necessarily have been able to add anything further.

I am not suggesting the Zap response has been the greatest, but I can understand not being able to respond to every message in a timely manner in the middle of trying to patch a vulnerability.

5 Likes

that’s a good point :heart:

2 Likes

Oh!!! I got it
Adversary didn’t theft the token from Zapper Contracts but the user of Zapper who approve Zapper to TransferFrom him.
That’s the answer why none token exists in bridge contracts~
1615393851323

great, so the next question is

Why Zapper Twitter claim that they exploited the vulnerability so that they rescued the fund.
How they did that?
by Transfer all allowance from user to Zapper’s vault before adversary?

3 Likes

Zapper :zap: on Twitter: “@Frankieboy93 Our new Polygon Bridge contract went live about 20 days ago.” / Twitter

I think that’s exactly what they did. Sounds like the timing is such that the vulnerability was only discovered after everything had largely been migrated and in that the devs were able to secure any unmigrated funds before an attacker could move them.

4 Likes

There’s another point ~
The Zapper team told me that it was a slip of the pen on Twitter and that the Bridge latest version is v1.3


Seems like a typo of contracts name.

4 Likes

I think this is a perfect example of what it is like for an organization to be handling public relations while trying to patch a vulnerability in real-time. Typos will be made, and misunderstandings will occur. The issue is whether an organization deserves the benefit of the doubt and what they have done to deserve it (or not). In this case, the Zapper team seems to have been completely transparent whenever they were able to and do not seem to have violated the community’s trust at any point.

Vulnerabilities happen. The real issue is how organizations respond to patching those vulnerabilities. In this case, it would seem that the Zapper team and community have averted disaster through transparent communication. It is often difficult for organizations to know how to handle vulnerability disclosures, but in this case it seems to have been the “best-case scenario” with a couple misunderstandings in the PR/

6 Likes

This line puts in perspective/paints a picture of the risks of interacting with smart contracts. Just one bug has given a bad actor multiple attack vectors to drain funds.

No matter how much I love smart contracts and their excellent utility, the typical risks involved are a big disadvantage. That said, I’m glad protocols and users are being enlightened with post-mortems like this. Armed with such timely information, they can act fast to secure their assets.

As someone with a huge interest in smart contract security, I’m curious as to the tools/methodology used to identify the bug.

Contract approval has always seemed like a safe activity. But, of course, approving a transaction/interaction with a bugged contract could open the user to being phished or hacked.

I’m also wondering how many contract approvals have been exploited in the past year with details about the scale of the attacks.

Kudos on finding the vulnerability and saving user funds. :clap: :clap:

Not all heroes wear capes.