Ethereum-like logs on NEAR

Development
1

Background

Currently NEAR supports Ethereum-like events in two ways:

  • Through env::log.
    • Pros:
      • Used by contract that were already deployed, like fungible tokens, lockup contracts, staking pools, etc;
      • Merkelized and merkle root is included in header, which makes it provable;
    • Cons:
      • Only accepts UTF8 strings, which is enforced by the protocol;
  • Treat part of the function call output as event. E.g. make function calls output always return (Vec<Event>, Result<T>). This is currently used by the Ethereum bridge;
    • Pros:
      • Technically exactly the same as emitting binary logs;
      • Merkelized, since function call outputs are merkelized;
      • Binary, which allows having Ethereum-like ABI;
    • Cons:
      • Deserializing one event requires deserializing all events that happened during contract execution. In Ethereum logs are serialized and merkelized independently;

Both approaches have a common disadvantage that they currently do not support Bloom filter similar to Ethereum events, which allows faster verification that an event has occurred on the chain.

Motivation

Ethereum has a very well-thought DevX with events: support of ABI, easy provability, Bloom filters. We would like to have the same DevX for NEAR events.

Proposed solution

I propose we achieve it in two, potentially separate steps:

  • Add log_bin host function in addition to log_utf8 and log_utf16 (link). This function would accept any binary. We then will switch env::log in Rust SDK to use it instead;
  • Add Bloom filter into block header;

Additionally, the first step would require some updates in near-api-js and explorer to support displaying binary logs.

Alternatives

Another proposed alternative is to add a new type of logs, together with introduction of a separate merkelization mechanism.
The disadvantages of this approach is the following:

  • Currently deployed contracts will not support it, so any kind of solution built on top of NEAR (e.g. the Graph) will not support contracts that were already deployed. Unfortunately, there are many important contracts that were already deployed that we need to continue supporting;
  • It adds an almost duplicated new functionality, since merkelization mechanism is going to be exactly the same. This will unnecessarily complicate protocol spec and node implementation. Any kind of aggregation solution built on top of NEAR will also become more complex;
  • It might confuse users developing contracts on NEAR, since it might not be clear why these logs are treated differently.

Next steps

After a discussion in this thread, we are going to create a NEP proposal following the format described here.

4 Likes
2

For now, there is a limitation of 100 logs in a whole tx, no matter how many cross-contract calls it includes. And using log as a way to contain events may seem a little casual. I prefer to leave log to its original purpose, that is to show important contract execution msg, and extend function call output to enable events.

3

It feels it might be useful to spell-out the intended use-cases for log-like functions. To me, it seems that there might be two different problems here:

  • Emitting “debug logs”: here we want strings to make display easy, but don’t particularly care about querying specific entries, as they are unstructured.
  • Emitting “events”, which are logically part of the result of the contract execution. That is, the goal is to explicitly let other parties to watch and react to events.
1 Like
4

It is a matter of naming. We can rename “log” to “event” everywhere in SDK and developers wouldn’t notice a difference if both of them are binary and support ABI. The limitation of 100 logs per contract execution is easy to change.

That’s right. In Ethereum both are achieved through the same functionality, called “events”.

5

@nearmax Curious about the downstream consumers, what are you thinking the api would feel like to use? Are you thinking logs would then have topics or would consumers, filter solely by the originating contract and type information?

I think maybe @marco-sundsk and @matklad have a point that perhaps event is slightly different than log in meaning. Dropping this here for clarification.

Event in ethereum maybe has Topic, coupled with types for each of those kind of event names

event Deposit(address indexed _from, bytes32 indexed _id, uint _value);
emit Deposit(msg.sender, _id, msg.value);

consumption from the non contract universe is then scoped by the event signature if I recall (event name, types)

What are you thinking we would do to support this or would we not?

6

Thanks for the comment @zcstarr . I don’t see a reason why we couldn’t fully replicate event DevX from Ethereum using NEAR current “logs”, including subscription to topics. AFAIU behind the scenes once Ethereum is serialized using ABI it is just a regular binary blob (Bloom filter are just there as a performance optimization for checking whether a particular event was emitted or not). If NEAR env::log accepts emitting structured events like this (someone would need to translate it to Rust and AssemblyScript syntax):

event Deposit(address indexed _from, bytes32 indexed _id, uint _value);
emit Deposit(msg.sender, _id, msg.value);

Then NEAR RPC can provide the same kind of subscription mechanism as the one available for Ethereum.

CC @austinabell , @tachyon , @mikedotexe , @josh.quintal , @jim from DevPlatform , @volovyk-s who owns near-api-js and would need to add events support into it, and @frol and @janewang who would need to extend node RPC to replicate Ethereum event-related endpoints.

7

Depends on how you define “fully replicate” because, without some mechanism to filter out events that do not match the criteria, it forces all logs within a range to be re-computed if not being persisted, and either forcing all logs to be persisted and processed or being re-computed is not a scalable solution. Ethereum bloom filters can only indicate if a log has not happened within a block/tx and do not indicate whether a specific event did happen, but this drastically reduces the amount of unnecessary replaying and/or processing of old state and receipts.

There is another issue I see with grouping these events directly with the ones that exist, and that’s because there can be collisions of event identifiers and the first bytes of a traditional log, meaning you could emit that an event happened but it was just a utf8 log string with matching initial bytes.

8

I am not aware whether the filtering mechanism is implemented on protocol or data-indexing level by Ethereum nodes. @marcelo and @eatmore do you know, by any chance? If it is implemented on protocol level then we should study it, and potentially add it incrementally later. If it is a data-indexing level, i.e. the node internally does heavy job of indexing data in such way that its RPC API allows quick filtering, then we can implement it too regardless of how we support events on the protocol level.

That’s a good point, we should be aware it, and reduce the chance of this happening by accident through SDK, but we cannot prohibit it from happening on the protocol level. It is a good idea to keep protocol agnostic to specifics or SDK, contract language, and data indexing solutions. For example even right now AssemblyScript contracts can write logs in UTF16 and Rust contracts write it in UTF8. If user does not know (e.g. from metadata/ABI) what format contract is using for the logs then already today they might erroneously attempt to deserialize using wrong encoding. To my understanding, similarly, on Ethereum someone can emit a manually created blob as an event and trick users who use this contract to think that it is some other type of event.

9

It is on the protocol level, based on the eth yellowpaper, a bloom filter is included in the block format as well as transaction receipts. The block bloom filter is just an OR of all transaction receipt bloom filters iirc.

Both of these are part of the consensus. Usually, eth nodes will store more data associated with receipts in a DB to cache more data to avoid recomputing, but the logs bloom is directly part of the protocol as you can see from the rlp encoding of receipts in geth for example.

1 Like
10

I miscommunicated, my argument is not about Bloom filter. Bloom filter as you said can only indicated whether logs as happened within a block/tx, as oppose to whether it happened in the range. If Bloom filter has proven to be useful for Ethereum DevX (this would need to be verified by @josh.quintal , since not all Ethereum features turned out to be useful, and some of them are not used) then we can incrementally add Bloom filter to NEAR headers later, regardless on what type of log we decide to support on the protocol level. I assumed you were advocating for another mechanism that would allow to check for log inclusion in a block range.

1 Like
11

Good thought here. I wrote up a post related to this, particularly “events” here:

12

As an update, it does seem that we’re needing to move on the idea of logs. In my opinion, it feels like adding bloom filters and topics might, as Max suggested, be added incrementally.

For now it feels like we want to utilize NEAR logs as events and iterate upon that. We have partners that are needing bulk/batch transfer and mint for fungible and non-fungible tokens. I’ll link this comment that seems to capture where we’re headed: