Research Summary - Chainlink Off-Chain Reporting Protocol

Oracles and Data
, ,
1

Citation

  • Breidenbach, L., Cachin, C., Coventry, A., Juels, A., & Miller, A. (2021, February 24). Chainlink Off-Chain Reporting Protocol. Retrieved from https://chain.link/ocrpaper

Link

Core Research Question

  • How does the Chainlink Off-Chain Reporting protocol work, what are its design goals, and what are the algorithms used for its implementation in the Chainlink Network?

Background

  • Oracles are off-chain agents that connect on-chain smart contracts to external resources such as data APIs that reside outside of a blockchain network. Such data is often required in the execution of automated smart contract applications.
  • Price Feeds are on-chain reference contracts updated by oracles and provide smart contracts access to financial market data regarding various assets, enabling the creation of decentralized finance (DeFi) applications.
  • Gas is the unit that measures the amount of computational effort required to execute and validate transactions on the Ethereum blockchain. More complex transactions performing many operations consume more units of gas.
  • Gas price is the amount of ETH that is required to be paid per unit of gas to miners on the Ethereum network. Gas prices are denoted in Gwei, with each unit Gwei being equal to 10-9 ETH. Times of higher network congestion result in a higher gas price and more expensive transactions.
  • On-chain Aggregation is an oracle network model where each node fetches data from an external data source and posts it on-chain within separate transactions. Each transaction consumes gas and thus each node must pay an individual transaction fee determined by the current gas price.
  • Off-chain Aggregation is an oracle network model where each node fetches data from an external data source and then collectively aggregates their responses off-chain into a single transaction containing a sorted list of values. Only a single node submits the transaction on-chain, reducing the total amount of gas consumed per update.
  • Oracle Report is a collection of responses from nodes within an oracle network during one update period. A report includes each node’s individual observation and their associated signature.

Summary

  • The Off-Chain Reporting (OCR) Protocol is a scalability upgrade of the decentralized oracle network Chainlink which decreases the on-chain gas costs of generating updates for the Price Reference Feeds by moving the data aggregation process off-chain using a distributed peer-to-peer network. OCR reduces the number of on-chain transactions required per update in an oracle network with n nodes from O(n) to O(1).

  • The OCR protocol is described as being developed with four primary goals:

    • Resilience: The protocol should be resilient to different kinds of failures involving Byzantine nodes and infrastructure crashes. An honest OCR node that is temporarily disrupted should be able to recover and rejoin the protocol quickly and without manual intervention.
    • Simplicity: In order to quickly meet market demand for oracle scalability, the protocol is designed to favor a straightforward implementation.
    • Low Transaction Fees: On-chain interactions should be minimized to lower the amount of fees nodes are required to pay. Ethereum transactions can carry a significant fee, particularly during periods of network congestion. OCR favors off-chain communication and computation wherever possible.
    • Low Latency: The time between when an update is initiated off-chain and when the data is included on-chain should be minimized as DeFi smart contracts require fresh data. The protocol should be able to produce an off-chain report within a few seconds and have it confirmed on the blockchain as soon as possible.

Method

  • The paper’s authors first formalized a model of the OCR protocol’s liveness and safety thresholds when there is at least n > 3 nodes:

    • Any f < n/3 oracles may exhibit Byzantine faults and behave arbitrarily as if controlled by an adversarial actor.
    • It is expected the OCR protocol operates with n = 3f + 1 oracles as this gives optimal resilience (Byzantine nodes are less than â…“ of network).
    • If f oracles are Byzantine-faulty (malicious node) and c oracles are benign-faulty (unresponsive honest node) with f < n/3 but f + c ≥ n/3, then the protocol may lose liveness but always satisfies the safety properties.
    • Using the median value from at least λ = 2f + 1 observations (responses from more than â…” of the network) ensure the final report is plausible in the sense that faulty oracles cannot move the median outside the range submitted by correct oracles.

  • The OCR protocol is described as being structured into three primary protocols run by each node in the network:

    • Pacemaker:

      • The pacemaker protocol drives the report generation process, which is structured into epochs.
      • Each epoch has a different leader node who coordinates the creation of a predetermined number of reports with observations provided from the follower nodes.
      • The pacemaker protocol runs continuously and periodically initiates a new epoch and pseudo-randomly selects a new leader.
      • Each follower node monitors the performance of the leader and if not enough progress is made within a specific timeframe, a new epoch is initiated and a new leader node is selected.

    • Report generation:

      • The report generation protocol divides each epoch into numerous rounds, with each round corresponding to the creation of a new report.
      • In each round, observations from each oracle node are collected and an aggregated report is generated that is signed by a threshold of oracles.
      • To prevent unnecessary on-chain transactions, a report is only created and validated by oracles if the previous on-chain update has deviated beyond a specific threshold against an off-chain data source (e.g. 0.5%) or a specific time interval has passed (e.g. 1 hour).

    • Transmission:

      • The transmission protocol encapsulates how the report is submitted on-chain and does not require communication between nodes.
      • The transmission protocol delays each oracle’s submission on-chain pseudo-randomly to ensure a staged sending process, ensuring not too many redundant copies are submitted on-chain.
      • Once a report has been validated by miners and added to the blockchain ledger, the transmission protocol process ends for the current round.

  • The specific steps taken for the creation of a single report within a round follow the below procedures:

    • A new round starts and the leader of the current epoch requests an observation from all follower nodes in the network.
    • Each follower node fetches data from a predefined data source API, signs the data using their private key, and sends the result back to the leader node.
    • The leader node waits for at least 2f + 1 follower nodes to respond, plus a grace period, then sorts the responses by value, generates a report, and sends it to all follower nodes.
    • Each follower node validates the report by checking the values are sorted, contains observations from at least 2f + 1 follower nodes, all signatures are valid, and that the median value exceeds the deviation threshold of the previous on-chain update or a time-based heartbeat condition has occurred.
    • If all conditions are met, each follower node generates a compressed report with just the node observations and oracle identities, signs it, and sends it back to the leader node.
    • Once the leader node obtains signed reports from more than f follower nodes, the leader assembles a final report from the followers’ signed reports, which is then broadcasted to all follower nodes.
    • When each follower node receives the final report for the first time, they rebroadcast it to every node, ensuring all nodes have received the report.
    • When a follower node receives broadcast from more than f nodes, the transmission protocol is started.
    • The report generation protocol for a round is now complete and the leader node waits a predefined amount of time until starting a new round. If a new epoch occurs or the leader does not make progress in a specified amount of time, then a new epoch is created and a new round initiates.
    • In the transmission protocol, to prevent unnecessary gas costs, the report is put through a filter and passes if 1) there is no backlog of reports or 2) the median value in the new report deviates beyond the median value of the report in the backlog by a sufficient threshold.
    • If the report passes the filter, a staging process begins where one or multiple nodes are pseudorandomly chosen to create a transaction to submit the report on-chain.
    • If a transaction is not confirmed on-chain within a specific time delay, a round robin approach is started where additional nodes begin to make an on-chain transaction in a time-staggered manner.
    • Once the report is added to the blockchain, an on-chain smart contract validates the signatures, stores the median value, pays nodes who contributed an observation and compensates the transmitter for its transaction gas costs.
    • The transmission protocol for a round ends when the corresponding report has been accepted by the contract.

854Ă—1334 79.4 KB

Results

  • Through the creation of a technical specification for the Off-Chain Reporting Protocol and its sub-protocols, the authors of this paper have defined how the decentralized oracle network Chainlink reduced the on-chain gas costs of Price Feed updates by up to 90% through batching node observations into a single on-chain transaction.

  • The authors also modeled the security assumptions of the Off-Chain Reporting Protocol including the proportion of Byzantine nodes for optimal resilience and the security measures implemented to protect against malicious leaders, ensuring progress cannot be halted beyond a predetermined amount of time.

Follow-up and Future Work

  • Multiple avenues are described in how the Chainlink Off-Chain Reporting Protocol could improve in the future.
    • Currently, reports are generated on a static interval. It may be desirable to produce more frequent reports upon observing changes within the data feed itself, such as increased volatility. However, such a design could generate reports at a faster rate than the frequency with which Ethereum produces blocks.
    • The OCR protocol can adopt an aggregated threshold signature scheme instead of the separate ECDSA currently used. The constant on-chain gas costs of verifying a single threshold signature would allow for larger oracle sets and further lower on-chain gas costs.
    • A new oracle list could be signed by the current quorum of oracles to change the off-chain oracle set without requiring the owner to intervene. This would allow for dynamic changing of the oracles used within an OCR-based oracle network.

Applicability

  • The current application of the Chainlink Off-Chain Reporting Protocol is primarily focused on improving the Price Reference Feeds, which provide a secure source of high-quality financial market data for the Decentralized Finance (DeFi) ecosystem. By reducing the gas costs of updates, deviation thresholds can be lowered, more price feeds can be launched, and a greater number of nodes can be added to each network due to the more efficient use of Ethereum’s transactional bandwidth.

  • The Off-Chain Reporting Protocol can also be used for the creation of any type of on-chain reference feed required by smart contract applications, such as the current temperature of a specific real-world location, the amount of off-chain US dollars in a bank account backing an on-chain stablecoin, and numerous other datasets. Additionally, Chainlink OCR networks can be natively launched on other blockchains beyond Ethereum to extend the Chainlink Network’s cost-effective data computation model to the growing ecosystem of interacting chains.

Discussion Topics

  • Will the gas improvements from OCR lead to a reduction in blockspace demand from oracles or will more oracle networks be launched to fill the gap?
  • What is the lowest deviation threshold at which an OCR-based price feed could realistically update, particularly during times of extreme volatility and network congestion?
  • How can the OCR Protocol be used for other oracle network models beyond on-chain reference feeds?
6 Likes
4

I wonder, from the perspective of DeFi protocols or anyone making price feed queries from chainlink nodes, besides the increased frequency of updates possible, is there any other draw to this less tested and more novel approach (via OCR) rather than the standard onchain feeds, which will run in parallel and not be removed?

2 Likes
5

Increased frequency (fresher data) and reduced absolute cost for the use of an individual feed are a couple of the immediate benefits. Both of these factors can potentially enable additional usecases for smart contract based platforms that rely on extremely fresh data (a number of trading usecases where very small changes in the price of an asset have significant consequences, locational data, etc) and absolute number of users, as the cost of using an individual feed is driven down.

Ultimately OCR represents an additional version of Chainlink’s core client, which will run in parallel with the FluxMonitor client.

3 Likes
6

What is the process for adding my node to an oracle set / feed ?

7

This is a question better served for a Chainlink specific channel, like their Discord. This forum is intended for technical discussion of academic research relating to computer science, blockchain, cryptography, etc.

1 Like
8

You can’t measure the security of the system without describing how a node is added to the validator set.

Is is permissioned? Is it permissionless? What guarantees does the listing process provide?

All very relevant research questions.

9

Hi @thegostep, Chainlink nodes are permissionless to operate, while the users (data requesters) themselves choose which nodes are composed into their oracle network. The selection of nodes can be based upon different metrics such as a node’s off-chain reputation, historical uptime, average response time, selection of data sources offered, etc. Operators can list their node on third party marketplaces and reputation services like https://market.link and https://reputation.link for discoverability.

However, I would like to note that the discussion topic within this thread is specifically regarding the Off-Chain Reporting paper and the new method of aggregating data it presents.

6 Likes
10

I can see how this type of system might be the actual type of infrastructure that gets implemented into a grocery store or warehouse as a means of tracking transactions with an aggregated chain of events. The issue becomes the simultaneous needs to track events, inventory, and sales transactions on the same item whether it is a banana or even an NFT to put it in a crypto context. A grocery store needs a point of sale system in order to accurately track all those elements of a single item going through the system. If an item is tracked directly on a blockchain, putting each event on chain will unquestionably result in more fees accrued on whatever respective chain the data is being stored.

Taking events and aggregating them into a consolidated transaction off-chain to be input onto a chain later in the process not only reduces fees, but for an actual business organization would be more in line with daily operations and how inventory flows in from a warehouse to be tracked, placed on the shelf, and eventually sold to a customer. While it is highly unlikely a grocery store will use Chainlink’s OCRP directly, this type of protocol may in fact be the exact type of infrastructure that a point of sale system provider would implement.

If we reframe it, would it even make sense for a business to want to put every single transaction " directly on-chain" when aggregating transactions cuts operational costs?

8 Likes
11

Dovetailing off of your example above @Larry_Bates I view the idea of being able to aggregate data off-chain and then upload it all at once in a single transaction through one leader node may be highly beneficial in developing economies, especially agriculturally centered ones where market prices may fluctuate drastically.

For example, we’ve seen how the introduction of cell phones to agricultural markets in Niger and India helped decrease price dispersion and bring the markets closer to the Law of One Price. This allows for fairer market practices, lowered inequality, and higher bargaining power for small-scale producers. Before cell phone integration to markets, there was high price disparity allowing for large firms to take advantage of smaller producers.

Oracles, like Chainlink, have the ability to take this one step further. If buyers peg their purchasing price to global/state-level prices obtained via a network similar to that of the existing Price Reference Feeds (but for goods not cryptocurrencies) then inequality regarding market prices may approach zero. However, one of the biggest challenges of implementing this system would be gas costs/fees.

The implementation of OCR would allow for this data to be aggregated in developing economies for a fraction of the price currently required. Personally, I do not think the idea of using oracles in developing economies, especially agriculture-centered ones, was feasible due to the data upload and cost requirements for on-chain aggregation. Now with the potential of allowing a single node to submit a transaction of aggregated data on-chain this theoretical implementation may be feasible due to the substantial reduction of costs.

Back to your original question and idea of supermarkets using this type of data, I’m not sold that is the best use case. In time it may very well be, and I hope to see blockchain, in general, being adopted more and more by supermarkets. I view agricultural producers and industrial manufacturers acting in the B2B space, as well logistic companies, to be some of the sectors outside of the current DeFi ecosystem, that could benefit from OCR in the long run.

5 Likes
12

I realize that my allusion to “point of sale” systems might need some more context. “Batch-processing” has been a staple of computer science theory to reduce cost of processing transactions and increase efficiency in processing large volumes of data for decades. Batch processing is a standard in large-volume data processing, and is one of the main methods of consolidating data for global networks including the Internet of Things.

OCRP effectively reproducing batch processing in a decentralized environment represents a leap in applying computer science theory to decentralized computing frameworks, in that distributed networks have already been using batched processing for a significant amount of time with proven results in cost reduction and processing efficiency improvement.

5 Likes
13

Like @Larry_Bates has stated I don’t think the real-world benefits of OCR have been fully realized. Right now, I would argue most people simply think of price oracles when they think of Chainlink, but the implementation of OCR enabling what @Larry_Bates states above:

OCRP effectively reproducing batch processing in a decentralized environment represents a leap in applying computer science theory to decentralized computing frameworks

implies that any organization that processes large batches of data can now do so on blockchain at a fraction of the price they could have before. This has large-reaching implications for all sorts of industries that we’ve yet to discuss here. I can see this being used by governments for things like batches of FOIA requests, insurance companies for large batches of claims after natural disasters, or even an application for voting where we could local voting locations all uploading results via OCR which would reduce the number of locations that would have to independently upload data.

Also, I want to point out, a SCRF member @patrickalphac created a very easy-to-follow video explaining OCR and what it means in basic terms here for anyone interested in taking a look.

6 Likes
14

Thanks for sharing the video, @tjd233. And obviously, thanks to @patrickalphac for making it.

I’ve enjoyed reading this discussion so far, but it is leaving a lingering question in my mind. Since OCR is so seemingly overwhelmingly beneficial, why isn’t it being more readily and rapidly implemented everywhere? Before that question becomes a rabbit hole, I know there are a bunch of “people are the problem” elements here. Certainly awareness of blockchain technology and/or trust in blockchain are likely contributors (as has been somewhat noted by others, e. g. Craggs & Rashid, 2019). I don’t want to move the conversation away from the technical capacities here, so the question I’m ultimately asking is are there technological downsides to OCR that would need to be overcome in order to facilitate more widespread adoption?

6 Likes
15

Thanks all! Sorry I’ve not been as active here.

Basically the question to your answer is “it is”. Price Feeds are slowly being rolled over to OCR networks. These are the networks used by Aave, Synthetix, Set Protocol, etc.

On the technical side, there isn’t much for downsides. Obviously, the upsides are that it drastically cuts down on gas due to the 1 transaction being sent. The upsides come from a much more sophisticated implementation of the network.

One thing to mention is that due to this, the networks are reaching a quorum off-chain. The off-chain quorum you may have noticed is actually higher. You need 2/3rds majority (similar to PoS vs PoW) to reach a quorum. This is why adding more nodes is not only important but necessary for the networks. Each node needs to connect to the peer-to-peer network of other nodes, and therefore has to add some security features since they are now talking with multiple systems.

Most nodes additionally are adding a cache like redis. OCR pings off-chain data providers much more often, and multiple jobs could be hitting the same API multiple times. To not hit API thresholds, a cache will actually ping the API just once, update all values, and then the node will just pull from the cache. So this is a technical barrier that node operators need to add and be aware of. If they don’t understand how to setup a cache, this of course could be a barrier. They could still have an OCR node without a cache, however they then will probably have increased data costs by hitting APIs more often.

Hope that helps.

7 Likes
16

@Eric after reading through this again, and taking a look at some other discussions online I have two questions that are related to how data is aggregated using OCR and then how that impacts the overall security of the system. I’m not 100% familiar with how some of these work on the Chainlink network so if they come across as basic questions my apologies.

  1. Before the implementation of OCR, not all nodes needed to report all data being reported to the network correct? And with the implementation of OCR, I’m assuming this is the same, where not all nodes/groups need to verify all types of data being reported?

Basically, if we take @Larry_Bates example above of batch transactions, and let’s say the insurance industry decides to implement OCR for batch payment reporting, then only certain nodes within the OCR network would need to verify these transactions before the singular node reports the data correct? It seems like once new industries enter the ecosystem, and more use cases for Chainlink/OCR are implemented, we would start seeing fragmented groups forming to aggregate transactions/data within their industry specialization. Is this idea correct?

  1. If my theoretical implementation above is accurate, wouldn’t this potential fragmentation of who is aggregating data off-chain and then uploading it lead to easier attacks and data manipulation through off-line coordination?

Again, if we take the insurance example above. If a conglomerate of insurance companies decided they wanted to manipulate data for their companys’ gain couldn’t they coordinate off-line, aggregate falsified data, and upload that data as one transaction? While I understand the cost-saving benefits of OCR it does seem to allow for malicious actors to upload falsified data easier/cheaper as well. How does this implementation impact the security of the data being reported? I believe @patrickalphac touches on this in his previous post a little but I was hoping to expand the conversation around this a little more.

7 Likes
17

Hi @tjd233, I can provide some insight on this topic. You are correct on the first point, Chainlink isn’t a monolithic network that requires global consensus like a blockchain, but is more like a framework for constructing independent and heterogenous oracle networks. This means nodes do not deliver data to every network, but only to networks that they explicitly have been added to. This ensures each node does not need to purchase API subscriptions to every data source that is needed by users (becomes expensive as the network scales), but nodes can instead specialize in the oracle services they provide. This enables support for permissioned data sources that not all nodes may have access to (e.g. enterprise backends, premium APIs). Oracle networks following a monolithic model can usually only support data sources that are accessible to all nodes like lower quality free open APIs. Chainlink has a blogpost on this topic that dives more into the network’s architecture topic more here.

For the second point, there is a few considerations in regards to how Chainlink oracle networks are secured. The cryptoeconomic incentives of being an oracle node operator is derived from revenue and reputation. This includes the opportunity cost of losing all future revenue/profit in the oracle ecosystem if they’re malicious (requesters would remove that node from any network they serve due to the destruction of trust), a reduction in the node operator’s reputation (which could be significant for enterprise run nodes), the loss of revenue from external services a node operators provides (e.g. telecommunications, PoS validation pool services, blockchain DevOps services, etc), as well as the implicit stake node operators have in the network by getting paid in LINK tokens (whose value is derived from the health of the network and its security).

These collective incentives raise the cost of attack, making it so it’s more profitable for node to be honest. It’s similar to the incentives that ensure Bitcoin miners are honest (they want to uphold their future revenue and the value of their mined BTC, while ASIC manufactures and exchanges want to uphold their reputation). In the future, security can be bolstered through the addition of explicit staking where nodes deposit LINK tokens as stake in service agreements, which can be slashed by the network for malicious or non-performance. These economic incentives combined with decentralization makes it harder from both a game theory and social coordination perspective to successfully pull off a collusion attack.

OCR networks can also be considered as having a higher cost of attack compared the previous Chainlink oracle network version because while the OCR upgrade improve efficiency, the additional scalability allows for more decentralized networks to be created and larger profit margin for nodes, thus future a greater revenue. All of these factors together contribute to the security of oracle networks, but you do bring up a good point that security fragmentation should be considered when using an oracle. If there already exists a secure oracle network for the data you need, it’s likely better to contribute and join that network rather than trying to launch your own network.

5 Likes
18

@Zach thank you for this detailed response and the added medium article. After reading both I think I have the answer I was looking for regarding my previous question.

One main thing I took away from your post is how important the implementation of staking will be in the future for the security of the network. Your statement:

In the future, security can be bolstered through the addition of explicit staking where nodes deposit LINK tokens as stake in service agreements, which can be slashed by the network for malicious or non-performance.

seems critical to me to help ensure the validity of the independent/heterogenous oracle networks, especially given the implementation of OCR.

5 Likes
19

Yes agreed, I see explicit staking as a key aspect of scaling up network security in the medium to long term, particularly when the underlying smart contracts consuming this data begin to secure increasing amounts of value from traditional markets, which often involve trillions of dollars (compared to the billions in DeFi today). Future revenue and reputation are the initial foundation of oracle node security, but the staking of collateral significantly strengthens these guarantees, as well as further quantifies security for users (x amount of collateral gets you y amount of network security for a contract holding z amount of value).

4 Likes
20

Great analysis and discussion.

Why is OCR 2 used in cross chain communication to generate the payload of the message?

For example, if the contract on one chain wants to bridge to another chain both an asset and an instruction set for the destination chain, OCR 2 generates that message after receiving the original payload.

There’s no computation off chain since the payload goes to the bridge contract. It’s not a queryable api so multiple nodes asking thwarts byzantine.

Is the benefit reliability that the transmission will be executed?

2 Likes