We want ideas on protocol changes to be both iterated on rapidly and be adhering to the highest standards of the quality control. This is a difficult balance to achieve but this proposal is trying to do exactly that. Please read how this proposal addresses quality control goals and feel free to suggest adjustments or alternatives to the solutions.
We want to iterate on protocol changes rapidly because we believe in blockchain to be an ever-evolving technology. No whitepaper can capture all the current and future ideas, so we accept protocol design to be a living organism – once it stops adapting it dies and users will gradually move to other blockchains that capture ideas that the old blockchain did not.
We want to adhere to the highest standards with protocol changes for two reasons:
- We don’t want to compromise our technological values: safety, user-friendliness, developer-friendliness, performance, scalability, decentralization, not needing to trust anybody specifically. In certain cases trade-offs can be made, but this needs to be explicitly discussed and not swept under the rug;
- We don’t want to engineer ourselves into a corner, i.e. we don’t want to make a protocol change that will either constrain our further development impeding our velocity, or it will introduce undesirable things that we would want to remove but it will be either extremely costly/laborious or impossible to remove.
The ability of the blockchain to evolve rapidly while not compromising on quality determines its survivability in a long-run.
This means that any protocol change proposal should have the following qualities:
- It should be meticulous and exhaustive to make sure there are no uncertainties, double-interpretations, and gaps;
- It should be focused both on immediate and distant effects. Effects can be distant in two ways:
- Time – things that happen way ahead in the future. In turn, they can be of two sub-types:
- Delayed effects – effects that happen because enough time has passed. E.g. a blockchain has accumulated an absurdly large state and is now hitting hardware constraints;
- New context – effects that happen because the context changed or a new context emerged;
- Dependencies – things that happen through a number of transitive dependencies. E.g. a blockchain changes how it charges for the state usage → contract developers change their contracts → … → users have transactions failing all the time, because they do not use the contracts correctly;
- Time – things that happen way ahead in the future. In turn, they can be of two sub-types:
Working with the Risks
It is however, not possible to have a perfect foresight, but we can have a system while trying to have one.
There are four types of risks:
- Known knowns. We know about an existence of a specific risk, and we have full understanding of it. E.g. we know that state staking can “lock” the contract if there is not enough balance to perform state changes, but we also know that anyone can send tokens to it to unlock it, so most likely such locking of a contract does not create surface of attack for malicious actors, since it is trivial to undo;
- Known unknowns. We know about the existence of the specific risk, but it is not well-understood and all side-effects are hard to account for. E.g. we know that state staking would require attaching tokens to function calls to offset the difference in state size, however we don’t have full understanding on how much of inconvenience to the users it is going to be and how much more complex this will make the contracts;
- Unknown unknowns. Risks that exist but we did not think of;
- Unknown knowns. We know about the risk but we refuse to acknowledge it, e.g. because we like the proposal and have a personal attachment to it, thus we avoid criticising it. Since it is often hard to criticize our own brain-child an explicit process needs to be in place to surface all potential unknown knowns. E.g. author knows that the proposal will open certain hypothetical angles of attack or degrade the experience but does not emphasize and justify it because they think it either dilutes the message or is a side-track;
Many design discussions start with all four kinds of risks, but for protocol proposal to be accepted we want to be left only with known knowns. There will be processes to achieve it:
- Known unknowns → known knowns. Once we noticed that there is a risk whose extend we don’t fully understand, the author of the proposal and its proponents will be required to work on clarifying it until it becomes a known known;
-
Unknown unknowns & unknown knowns → known unknowns. Since these risks are not written down anywhere explicitly, the only way to uncover them is to “fish” for them (similarly to how a red team would try to uncover security risks by playing an opposing force). There will be two methods to do it:
- Willing participants of the discussion will play an opposing force, whether voluntarily or they will be nominated and even funded by stakeholders like NEAR Foundation to construct scenarios in which the proposed change can compromise our technological values or velocity, in the process uncovering unknown risks;
- There will be standard scenarios that will be agreed ahead of time by the community. Each proposal would have to include a walk-through for each standard scenario before it is submitted. E.g. give a detailed example of a cross-contract transaction that contains merged promises, callbacks, and a contract factory. Explain how the proposed change will affect each receipt, including refunds in a high detail. How will values of each field in each receipt change with the proposed change?
Roles, Ownership, and Accountability
We want the process to be democratic and inclusive, but we also want to have accountability. This means the proposal needs to have owners – people who carry the burden of proving “beyond reasonable doubt” that the proposed design does what it claims to do and that it does not negatively affect our technological values or impede our future velocity. The owners of the proposal are the author and its potential proponents. It is their responsibility to prove that the proposal works, and it is not responsibility of others to prove that it does not work. The default decision for any protocol change should be a No.
We also need admins who will decide when the proposal is accepted or rejected. In a long run, this will be a council elected by the stakeholders, which most likely will be only the validators and people that validators consider as an authority or experts, since validators have an ultimate control of deciding which protocol version to run on their machines. However, at the beginning as a way to bootstrap this process admins will be a preselected group of people. Admins will also be moderating the discussion once the proposal is submitted to make sure the discussion is civil and focused.
As explained above, there will be opponents and people nominated to play the opposing force and try to do the following:
- Find gaps, double-interpretations, inconsistencies;
- Suggest better alternatives and ask author and proponents to justify why alternative is not better;
- Find angles of attack and other kinds of abuse;
- If the proposed design makes trade-offs with our technological values and velocity, then argue why these trade-offs are not justifiable;
Additionally, there are people who work on different parts of the protocol and have deep understanding of it. It makes sense to invite them into the discussion and when judgement call is needed (e.g. when the trade-offs are present) ask them to weigh-in.
To summarize, the roles are:
-
The author.
- Election process: Self-elected by submitting a proposal;
- Responsibilities: Submit the proposal following the format, address the comments from the community until proposal is accepted;
-
The admins.
- Election process: Initially pre-selected, but eventually elected by the validators. They are the same for all proposals;
- Responsibilities: Moderate the discussion, make the final accept/reject decision, enforce the timeline, recognize/invite subject matter experts;
-
The incentivized auditors.
- Election process: Funded by stakeholders, e.g. NEAR Foundation;
- Responsibilities: scrutinize the proposal, the incentive model might be paying per finding;
-
The subject matter experts (SMEs).
- Election process: recognized or nominated by admins either for all proposals or for specific proposals;
- Responsibilities: Present their take on the specific proposal;
-
The affected projects.
- Election process: The author of the proposal needs to list all major projects that will be affected by the change in the proposal. Admins then invite representatives from major projects for the discussion;
- Responsibilities: Explain in detail how they will be affected if the proposal is implemented;
-
The community members.
- Election process: none, anyone can join the discussion as proponent, opponent, or non-incentivized auditor;
- Responsibilities: Add clarity to the discussion, surface issues, present alternative points of view;
-
The implementers.
- Election process: self-elected by being one of the owners of a client implementation for our protocol;
- Responsibilities: implement the change, if accepted. Very likely, they will be in sync with the validators and other stakeholders so they will have an interest in implementing the change;
Note, some roles might overlap, e.g. the implementers will also likely be the SMEs. However, certain roles should be separate to avoid conflicts of interest, e.g. the author should be separate from the admins and the auditors. If the admins want to submit a proposal themselves then they need to recuse themselves from the process.
The Proposal Process
We don’t want proposals to be stuck in a limbo state of always being in progress, therefore we commit to always either accept or reject the proposal.
It can be rejected for the following reasons:
- No-op proposal. It does not improve any of our values or velocity;
- Unjustifiable trade-offs. It makes the trade-offs in technological values or velocity that are not justifiable. Note, that if proposal is too difficult to implement or it adds too much architectural complexity it can be rejected as impeding future velocity too much;
- Not ready. The proposal does not satisfy the required format, e.g. it does not explain all standard scenarios. Also, we don’t want the proposal discussion the be the place where the process is created, and so the authors and the proponents should come with a sufficiently ready proposal and do not expect to iterate on it with the community. They can however iterate on the proposal with the community in other open forums outside the official proposal process. Therefore, if proposal is submitted but community keeps finding gaps in the proposal, e.g. like reasonable alternatives, adversarial angles or gaps in the design, then even if author and proponents continue explaining them, after some time proposal runs out of time and is rejected;
- Duplicate. A similar proposal has been already submitted and the author did not explain the difference sufficiently;
- Spam. Someone is trying to get their proposal through by spamming;
The time-line of the proposal is the following:
- Initial phase (1 week at most)
1.1. The process is initiated by an author submitting it;
1.2. The proposal is examined by the admins. If it deviates from the format the proposal is rejected and allowed to be resubmitted in 1 month;
1.3. Admins invite SMEs (some SMEs recognized by admins might join voluntarily) and representatives from the affected projects;
1.4. (Optional) The stakeholders hire the auditors; - Discussion phase (2 months at most)
2.1. SMEs and affected projects weigh-in;
2.2. Auditors respond with their findings;
2.3. Other community members join the discussion;
2.4. Once the proposal is at the point where it is exhaustive, non-ambiguous, highly-detailed, and all findings are addressed and incorporated into the proposal, the admins make a decision on accepting or rejecting it;
2.5. If proposal runs out of time it is rejected; - Implementation phase
3.1. Admins invite representatives from the affected projects and the implementers to discuss the timeline of rolling out the feature. Once the timeline is agreed on, the remaining process is delegated to the implementers.
The Format
Below is the format that every proposal needs to follow.
Background
Any background that readers who are not SMEs need to know to understand the proposal, including the motivation, and the alternatives.
Motivation
Explanation on why the proposed change is needed. It needs to address our technological values or improve velocity. If the proposal is doing trade-offs they need to be explicitly outlined and justified.
Affected major projects
List of affected major projects and the ways in which they will be affected.
Design
A thorough design, which should include:
- Any new data-structures or modifications to existing ones;
- Any new mathematical formulas and algorithms which should be unambiguously formalized;
- Any API changes;
- Thorough explanations of the migration schemes and how exactly they should be performed;
Standard scenarios
Author should take each of the standard scenarios and describe unambiguously, in high-detail how the proposed change will affect it. This section should be treated as seriously as the design section, because it allows to uncover all the complex corner cases and give a complete picture to the readers.
Side-effects
All side-effects that this change will create for our technological values and velocity, including: developer-experience, safety, decentralization, etc.
Alternatives
A list of all alternatives that author is aware of with unambiguous explanation on why the proposed design is better.
Pre-mortem
The author should imagine the most likely scenarios in which this design can fail once it is accepted and implemented. The author then should elaborate on how exactly it will happen and what will be the consequences.
(Optional) Implementation details
Any hints on how this change can be implemented.
Discussion Place
It is important that there is a place where free-form public ideation can happen without any constraints. @mikedotexe and @Bowen will discuss what we should recommend as a place for ideation and will ink it in https://github.com/near/NEPs/blob/master/README.md.
Once it is ready to be submitted, the author will create a PR in https://github.com/near/NEPs that creates a file in proposals
folder that follows the above Format. PR should also include the necessary modification to the protocol specification in specs
folder.
Once the proposal is approved the admins will approve and merge the PR.
This achieves the following goals:
- The history of the discussion is permanently preserved through git comments;
- If proposal is accepted, it is automatically applied to the documentation;
- If documentation is lacking in this area, we are creating it by making the author of the proposal write it;
- Github projects can be used to track the progress of the proposal in the Kanban board.
Adjustments to the Process
In some cases critical bugs need to be fixed quickly without being blocked by the process, in that case the proposal needs to be written retroactively for archival reasons. Note, that this should only be done in very rare cases, since even the bug fixes themselves can introduce bugs and the entire point of the process is to guard from these kind of bugs.
Also, in some cases bugs need to be fixed discreetly. In that case the proposal process needs to be done privately and the proposal should be made public only once the change has propagated through the network.
The decision to allow certain proposals to be retroactive or private should be made and coordinated by the admins.
Appendix A. Initial Standard Scenarios
A. If the proposal touches anything related to the receipts, including their structure, how fields are interpreted, their behavior, then the proposal needs to include detailed explanation on how the following scenario will be affected:
- Alice signs a transaction that calls contract Bob;
- Bob calls contract Carol and Dave, joins their promises and attaches a callback to contract Erin;
- Erin then deploys a contract to account Frank;
- Describe also how all refunds and data receipts will be changed with your proposal;
B. If the proposal touches anything related to state, including introducing new forms of state. Then the project needs to explain the following:
- How the dynamic resharding and resharding in general will be affected;
- How the creation of state witness will be affected. Will there be new state witnesses? What will be the difficulties with distributing, creating and validating them? Can someone fake a state-witness to slash a validator?
- Will it be possible to manipulate the structure of the state structure? Please explain exactly how?
- Please explain if anything will change with regards to the light client and validation of the state;
- Anything will change in terms of how much data we store on the disk
C. If the proposal touches anything related to the host functions, including modifying, adding new, or changing the meaning of the host functions.
- Provide a full-specification of the function independent on the implementation;
- Explain how we can guarantee this host function to be sand-boxed;
- Explain how we can have a precise deterministic fee estimation mechanism for it;
- Explain how the fees are going to differ on different platforms;
- Will this function be eventually unused? If yes, then what is the rationale for including it?
Append B. Examples of the Proposals
We will take a couple of most recent WIP proposals and polish them until they adhere to the format. Once it is done, we will link them here.