[PROPOSAL] Gas weights to fight instability to due to undercharging

Development
,
1

Dear friends of NEAR Protocol,

Stability of our blockchain is crucial. The protocol team at Pagoda, which I am a part of, is constantly working on improvements to keep up with the ecosystem growth and new use cases, without compromising stability. Virtually all this work is done in public.

However, it happened in the past that we surfaced corner cases for which nearcore is too slow to keep up with the regular block production time. This means we are undercharging certain transactions, or in other words, the protocol gas parameters are too low. These types of issues must be treated confidentially until fixed, or we risk exploitation.

I propose a solution that translates undercharging issues from stability risks to economical inefficiencies. Please read the proposal below if you find the time and let me know what you think about this strategy.

Gas Weight Proposal

Today, gas has two somewhat orthogonal roles.

  1. Gas is money. It is used to avoid spam by charging users.

  2. Gas is CPU time. It defines how many transactions fit in a chunk so that validators can apply it within a second.

The idea is to decouple these two by introducing gas weights. Each gas parameter still has a gas cost that determines what users have to pay. But when filling a chunk with transactions, each parameter is multiplied with its gas weight to compute the expected CPU time.

The gas weight can be thought of as an undercharging factor. If a parameter is 1.5 times too low to guarantee stability, the gas weight is 1.5. A chunk will be full 1.5 times faster when gas for this parameter is burned. This deterministically throttles the throughput to match what validators can actually handle.

Ideally, all weights should always be 1. But when we discover undercharging issues, we can set a larger weight. (This would require a protocol upgrade.) The stability concern is then resolved as soon as the weight is active.

Questions that Nobody Asked Yet

Why not just increase the gas cost?
Because that would break contracts that rely on current gas costs. It can also steeply increase operating costs for the most active users of the blockchain. I believe the protocol must serve its users and not the other way around. Therefore, increasing gas costs without prior consent by the affected parties can only be considered in absolute emergency scenarios.

Isn’t it problematic to increase gas weights?
Gas weights above 1 would only be a temporary solution. Whenever we have a gas weight > 1, we as the community can discuss this publicly and find a solution to the specific problem together.

In the best case, we find technical optimizations that allow us to reset the gas weight to 1 without increasing costs.

In other cases, the only solution is to increase the gas parameter. But the dApp developers who are affected by this change should have a chance to voice their opinion, suggest alternatives, and implement necessary changes before the gas cost is increased.

What are past examples of undercharging?
We used to undercharge contract deployment. The responsible team has implemented a number of optimizations but a gas increase was still necessary. Meta-pool and Sputnik-DAO were affected by this change, among others. Finding all affected parties and reaching out to them before implementing the change took us more effort than implementing the gas weight proposal would take.

Future possibilities

We can also think about gas weights smaller than 1. For example, if we want to charge gas instead of token balance for extra storage bytes, it would make sense to set the gas weight = 0 for the part that covers on-chain storage. Otherwise, the throughput would be throttled unnecessarily.

A further option would be to change gas weights dynamically without a protocol upgrade when block production has become too slow. This would be a catch-all, self-healing solution that requires zero intervention from anyone. The network would simply throttle throughput when block time remains too high for long enough.

Call for Action

Please let us know if you are for or against this change. This proposal is not tied to any specific feature or project, yet. If nobody cares about this, we will simply drop it and focus on other issues. (There is always plenty of high-priority work.)
But if enough people want this, I can try to get this prioritized within Pagoda and actually work on it.

3 Likes
2

+1 - I especially like the part about avoiding the breakage of existing contracts.

3

We recently discussed this within Pagoda and came to the conclusion that we need this not only for transparency, but also to have “emergency buttons” for when undercharging situations are discovered. So we are now working on this.

One question came to me a few days ago: How will gas weights affect gas price? (Price is how much NEAR / gas it costs to purchase gas.)

To recap, if gas volume for a block goes above 50%, the gas price is increased for the next block. The question is, if we want to calculate the 50% with or without weights.

  1. If gas weight is not included, then we can go above 50% total capacity without increasing the gas price
  2. If we do include gas weights, less gas has to be burnt to drive up the price.

In both cases 1% of block capacity becomes cheaper if we have gas weights >1. Which means DoS attacks become cheaper. The difference is in case 2 the DoS attack is increasing gas cost quicker, which is good to drive up the cost of DoS dynamically but bad for legitimate users that suddenly pay much more tokens for the same amount of gas.

I think for robustness against DoS, we almost have to go with number 2.

4

Great post!

I would consider different name, because we already have “gas weights” Callbacks | NEAR Documentation defining assignment of attached gas to cross-contract calls. Also “gas weight” can confuse devs into thinking they actually have to pay more. Maybe “parameter weight” or “operation weight”?

1 Like
5

I think this is a good idea! Especially the future possibility part, which sounds very exciting :slight_smile: I was also thinking about the same question you raised regarding gas price change and I agree that 2 may be a better option.

Another question is how these weights will be set and updated. Ideally we should have the ability to dynamically adjust the weights while the chain is running without having to do a new release :slight_smile:

1 Like
6

I think a first version that requires a protocol upgrade for every weight change would be a good start. This is easy enough and works just like changing parameters.

But I agree with your goal. I suggest we make it a separate NEP, but of course we can think about it already. :slight_smile:

My idea is that validators can add new weight definitions manually after reaching a conclusion off-chain. (Pagoda may or may not be involved.) Or maybe even automatically increase all weights by a fixed factor when block processing time gets slow. How ever nodes come up with new weights, they will then propose the new gas weight. This could be a field latest_weights_hash in the block header similar to latest_protocol_version.

Once a 2/3 majority of validators proposed the same set of new weights, we could switch to that. I think theoretically we could even switch mid-epoch, as we don’t need to update the binary, we only need a new weight definition which can be shared through the network like state sync.

But the details are complicated. latest_weights_hash should probably be a hash of (weights + signature of 2/3 validators + valid_after_height) in every block to ensure no node accidentally uses the wrong weights. And valid_after_height would have to be monotonic to not reuse old proposals to revert a change. But interplay with finality / forks could become tricky. And who knows how many other details I completely forgot so far.

7

@jakmeier question: we already have gas limit in each chunk that could be dynamically adjusted (the logic is not implemented but nothing prevents us from doing so). If the goal is to prevent undercharging, it seems that we can just implement some way to adjust the gas limit through consensus. Put it another way, how do you think we should implement the gas weight idea?

8

Yes, I want to implement it exactly like you say, by changing how the limit per chunk is calculated. But not dynamically in this first proposal. (I’ve found very little support for the idea of changing it dynamically in my discussions within the protocol team so far. And it’s also a bit too complicated to include in the first version.)

The key feature here is that the limit will be respecting the weighted gas parameters, so that we can throttle the usage of specific parts of the runtime at the gas parameter granularity.

9

@jakmeier There is existing mechanism that Bowen refers to, that allows to adjust overall gas limit of the chunk to prevent exactly possible attacks by flooding chunks with expensive operations.

The current implementation in nearcore always returns the same limit, but changing it to return reduced number if the chunk application took longer than target time and chunk is full would address overall concern of undercharging leading to block time problems. Note that this would not require protocol change as this is already part of the protocol.

On the gas weight idea - it seems like indeed a very hacky way to address a problem that will require proper operations to mange.

So before adopting it I would want to make sure that general throttling that is currently in place (gas price increase and chunk limit voting) are not addressing ongoing concerns.

1 Like
10

Oh I see, thanks for clarifying what Bowen meant here, I didn’t quite catch it. That is, I assume you are talking about the chunk header field "gas_limit" which currently is fixed at the corresponding genesis config option, but the protocol allows chunk producers to adjust the value by 0.1%, right? I never noticed that we could adjust this value dynamically by 0.1% per chunk without a protocol change. Indeed this solves some problems more easily than I assumed.

I have spelled out the idea, as I understand it, in details on Github: Dynamic Chunk Gas Limit (only in discussion) · Issue #8280 · near/nearcore · GitHub

We can discuss this idea in isolation on the Github issue. It addresses similar problems as gas weights but is orthogonal in its implementation.

I believe some ongoing concerns that are tracked in https://github.com/near/nearcore-private/issues/27 are not covered by gas price + chunk limits. But we are currently not confident we can discuss those issues publicly.

11

Thanks everyone for discussion so far!

To close the loop on this, we will be proceeding with a formal NEP for this proposal under a new name “Parameter Compute Costs” that could be found here: https://github.com/near/NEPs/pull/455.

If you’re interested, please take a look and comment on the NEP and engage in the review process (to be scheduled soon, stay tuned on the PR).

1 Like