Block and chunk producer selection algorithm in simple nightshade

@alexatnear @Bowen

I’ve updated my model to include delegation. The full write up is here.

The final result is:

where f and x are as before, S_COP is the total chunk-only producer stake (including delgation), S_BP is the total block producer stake (including delegation), D_COP is the total stake delegated to chunk-only producers (note: D_COP <= S_COP), and \phi_COP is the delegation fee charged by chunk-only producers.

Notice the upper bound is now a linear combination of two terms. In the limit of no delegation (D_COP = 0) we recover the result of the original model (a good sanity check). We also recover the original result in the limit \phi_COP -> 1, which is also reasonable (if the fee is 100% then there is no reason to delegate so we should recover the original result).

In the limit where the total stake is dominated by delegation (S_COP ~= D_COP), then we have an upper bound given by ((1-f)/f)(1 + (x-1)\phi_COP). I haven’t fully thought through the implications of this bound yet, but I don’t like the idea that the stake ratio can be constrained based on the whims of the chunk-only producers.

I’ll think about the implications of this result some more. But let me know if you have any thoughts on it as well.

Could you explain why the upper bound is asymmetric, i.e, why does it only rely on chunk only producers?

This is just the way the algebra shakes out. The upper bound comes from the following condition: in equilibrium, it will not be more profitable for any chunk-only producer to switch to being a block producer. The lower bound comes from the symmetric condition: in equilibrium, it will not be more profitable for any block producer to switch to being a chunk-only producer.

In the lower bound case, the terms involving BP quantities (like D_BP / S_BP) cancel out which results in a cleaner expression. In the upper bound case they do not cancel out, so there are chunk-only producer quantities which remain.

The reason things cancel out in one case but not the other is because there is an inherent asymmetry in the setup in the cost term. In both cases the cost term takes the form (x-1)C, but it appears on different sides of the inequality in the different cases. It is clear that x - 1 >= 0 and C >= 0, so the lower bound on (x-1)C is also 0. When this cost term is on the smaller side of the inequality we can simplify the expression by replacing it with its lower bound (ie 0). When the cost term is on the larger side of the inequality we replace it with its upper bound, which comes from the condition that chunk-only producers are profitable (we could instead use the condition that block producers are profitable, but this does not help in simplifying the expression). When replacing the cost term with its upper bound we pick up the extra terms ultimately stick around until the final result.

There isn’t much here at the “whim” of the chunk producers. Both D_{cop} and S_{cop} have some other equilibrium constraints not captures in this formula. E.g. people will not choose not to delegate to chunk producers at all (and make the coefficient on the right skyrocket to x) just to “mess with the system”, because not delegating to chunk producers at all is bad for them.

Also, S_{cop} is present on both sides of the inequality, so we are expressing the upper bound for a ratio in terms of its nominator, which is not ideal.

We also fix φ, which in itself is a subject of reaching an equilibrium (e.g. block producers might want to lower their φ to attract more delegated stake to their side?).

The real givens we have are: f, C, x, and also the total stake of the pools, and the total delegated stake.

Separately, while fixing some precise upper and lower bound is helpful, are there objections to the model in which the reward is split between block producers and chunk producers in some proportion (e.g. equally) with the information we have so far?

I suggest we proceed with building it, because intuitively, assuming high percentage of delegations, it converges to a split of stake close to the ratio of how the reward is split between the BPs and CPs, and the noise in how people actually make decisions will outweigh the all the fine-tuning we can make.

I agree it’s not ideal, but I do like that the upper bound is expressed in terms of quantities which have some easy interpretation, so we can apply any intuition we have about them. E.g. your intuition is that D_COP / S_COP will be close to 1. I could work harder to try and untalgle everything until it is all in terms of external variables.

I agree that based on the information we have so far it looks like this incentive mechanism should be fine. I’m writing a simulation of it now. This will give empirical data in addition to the analytical work I’ve already done. I think this is an important step before we move forward with a production implementation because getting the incentive mechanism wrong could be very bad for the system, so it is worth the additional time investment to have confidence it will work as intended.

I agree with Alex that we should not spend too much time optimizing the model based on unverified (and maybe unverifiable) assumptions. For example, it is hard to estimate the percentage of delegations chunk producers will get. It could be that most of their stake come from delegations, or the other way around — most people use their own stake and do not accept delegations.

Sure, my goal is to not optimize the any model parameters (e.g. f). I just want to make sure the incentive mechanism is not exploitable or creates an undesirable equilibrium. And we can test what happens (or try to analytically calculate bounds) for a wide range of possibilities, including when there is more or less delegation happening.

1 Like

I’ve done some work with my simulation of the incentive mechanism and found some interesting results. The code for the simulation is on my GitHub: GitHub - birchmd/near-bp-sim

The key results are:

  1. I still think the reward fraction needs to be in favour of the block producers (in the plots shown below I have f = 0.55. And it should still be a protocol parameter so that it can be adjusted in the future.
  2. There is a sort of phase-transition that occurs as the chunk producers increase their delegation fee. Counter-intuitively, chunk producers pull the equilibrium stake fraction towards their side by increasing their delegation fee to a relatively high number (>50%). This is illustrated in the plots below. I think the reason this happens is as follows: there will always be some delegators on both BP and COP sides, even when COP fees are higher because the reward is finite, and if all delegated stake is on the side with lower fees it is split among many participants. Whereas, if some switch to the higher fee side then they earn more despite paying a higher fee because the reward on that side is split among fewer participants. Moreover, because the delegation fee is higher on the COP side, and the hardware costs are lower, it attracts more node operators as well. The net effect is more total stake on the COP side. It is also true that the BPs cannot re-balance the stake fraction by increasing their own delegation fee. This is probably a good thing since it means it does not create a situation where it is a race to the top.
  3. In the parameter ranges we expect in reality (2 <= x <= 4, \phi_BP = 0.15, \phi_COP <= 0.4), there are more block producers than chunk producers. And even outside that range there is never more than twice as much stake on the chunk-only producer side. I.e. we can establish practical bounds 0.5 <= S_COP/S_BP <= 2, and this seems like an acceptable range.

My conclusion is that this incentive mechanism seems like it will work well for what we want to accomplish.


2 Likes

I will be the first to admit that I don’t understand everything above, but I do have a couple thoughts.

  1. I saw a few comments about the cost of hardware for Block Producers vs Chunk Producers. This seems irrelevant when considering the current seat cost and the rewards lost when a validator is kicked. Even if this was reduced by 10x it still doesn’t look like a material cost.
  2. If Simple Nightshade is an intermediate solution, doesn’t that inevitably hinder a final Nightshade solution?
1 Like

I listened to Alex’s latest chain link video and this whole thing appears to be redesigning the consensuses mechanism on the fly. I’m not wanting to rail on anyone but this is all completely different than what was sold in the white paper and absolutely is attempting to swap out a nuclear reactor in a submarine in the middle of the pacific. You are dealing with game theory consensus mechanism changes on the fly.

The change to nightshade lite has been pretty opaque and I would have missed it if I hadn’t dropped in for latest videos.

I’m a little confused about the status of sharding with the guildnet and if that’s the version going to the main net first, then implementing the chunk producers, then implementing whatever Alex is discussing which I’ve heard designs similar to DOT or Cosmos etc

Also will re-iterate that hardware being 2% as expensive as the amount of money staked is completely appropriate. Developer time to set up an integrate a validator is going to cost far more than hardware cost.

Simple nightshade does not prevent us from going to full nightshade. Once everything is ready, we can remove the restriction that block producers need to track all shards

1 Like

I don’t think that is an accurate analogy. For large protocol changes it takes a very long time from ideation to implementation and will not be done in a hurry precisely to make sure that we don’t miss anything in the process. The discussions we had are about new ideas and it does not imply that we will necessarily implement them. As for the current status, what is deployed on guildnet is usable sharding, but it is flawed from a security point of view – slashing is not built and that is why we decided to go with simple nightshade first.

2 Likes

Does “tracking all shards” and “validating on all shards” mean the same thing?

Could you explain how guildnet is “flawed from a security point of view”?
Are there two options: Build/Enable slashing or Simple Nightshade?

Yes.

Yes. There are other alternatives (sync sharding for example) that may require larger changes.