There have been some questions about our focus with regards to protocol development. This post will hopefully provide a clearer picture on what we plan to work on in the next 1-2 years. I’ve made a gantt graph (attached below) to show the roadmap for the major protocol features and improvements. Please note that for those that are planned for next year and beyond, the estimates are likely to be somewhat off. [0001] Some highlights of what we plan to deliver this year: Simple nightshade with 8 shards with Aurora in its own shard. This will improve the total tps by 5-8x and potentially reduce the storage cost further. Increase the number of validators on mainnet Improve single-shard performance. This includes the following: Moving contract compilation from runtime to deployment time, which will reduce function call base cost from 4.6Tgas to ~0.2Tgas. This means that for a lot of function calls, it will mean 20% - 50% gas cost reduction. Upgrade to wasmer1. This will reduce gas costs across the board and most notably, will reduce regular ops gas cost by ~50%. Zooming out from the short term focus to look at the longer term picture, we will likely work on and deliver the following in the next couple of years: Gas price auction. The exact details haven’t been fully decided yet, but the idea is that we would like to allow users to specify the maximal gas price they are willing to pay for a transaction and take transactions in the decreasing order of their gas prices. This will be a big change to how transactions are charged currently and we decide to make this change to better handle congestion. Nightshade. This means most of what is described in the original white paper except hidden validators, which we likely won’t need for a while since there are entities who are incentivized to run fishermen in practice. After we deliver simple nightshade, what is left consists of mostly two things: challenges and dynamic resharding. Both require quite some effort to implement due to their technical complexity. Improvements for contract development. There are quite a few things that we plan to work on to both reduce the cost of contract calls and make the developer experience better: Reduce storage IO cost. Currently storage reads and writes are relatively expensive, which both leads to lower throughput and also higher transaction fees for smart contracts. We plan to rewrite the storage to make reads and writes faster. A proposal can be found here. Rethink how gas and storage costs work. Developers today have to think about how much gas to attach to transactions and cross-contract calls. They also have to take storage cost into consideration when writing methods that modify the contract state, which makes the contract development more cumbersome. One of the possible options to address this issue is to unify gas and storage cost and make transaction attach NEAR explicitly. Some form of synchronous execution environment. In the original nightshade design, all executions are asynchronous and there was no consideration for synchronous execution. However, from the feedback of developers it is quite clear that financial applications could benefit immensely from synchronous execution. To address this issue, we have considered several options, including introducing special synchronous shard(s) and building synchronous vaults on top of async shards. The crypto landscape has changed dramatically since when the nightshade paper was written. While we are still working in the direction prescribed by the white paper, we also happily embrace new ideas to make the protocol better. After all, NEAR stands for iteration, and we will keep evolving to serve our developers, to help grow our community, and to stay true to our mission of empowering all people to control their own money, data, and power of governance

@Bowen, thanks for publishing this. What about the idea of introducing high-end HW shards? We expect that Aurora use cases may hit the block size (or a half of block size) quite fast. As a simple solution to it, Aurora shard can run on advanced hardware, which is able to do more operations per second, which would lead to the ability to increase the block size. Obviously, this means lots of specific tests on new target hardware. This is the most simple way to scale one-sharded version of Aurora, and it would be good, if we can work in that direction. CC: @Arto

That means we need to have the ability to adjust gas limit individually for each shard and introduce special validators who won’t be able to rotate with validators from other shards. This is similar to the idea of introducing special synchronous shards.

There are several issues with that: It is possible to run a higher-end SSD or CPUs with more performant cores, but it is unlikely that the performance will increase beyond 2x, and if it does we should probably make it a requirement for all of our shards anyway; If Aurora runs on a different shard that has more performant hardware then we would need a different set of gas fees calibrated for that platform. Calibrating and maintaining gas fees is extremely labor-intense and we should avoid adding new gas fees as much as possible, since mistakes in gas fees can open us to vulnerabilities; EVM does not execute operations concurrently, so adding more cores will not contribute to TPS. If we want to leverage more cores we would either need to host several EVMs on one shard (which is not different than having multiple shards with one EVM in each) or create a complex scheduling mechanism which would be very invasive to EVM code that would allow a single EVM to execute non-interfering transactions concurrently. And even in that case, it won’t increase the true TPS, because a single contract will still be bound by a single core.

[image] nearmax: If Aurora runs on a different shard that has more performant hardware then we would need a different set of gas fees calibrated for that platform. Calibrating and maintaining gas fees is extremely labor-intense and we should avoid adding new gas fees as much as possible, since mistakes in gas fees can open us to vulnerabilities; I think Alex meant that instead of having separate gas fees we can have a different gas limit for that shard. This would make things a bit easier, but yeah I agree with your concerns

Gas fees will not scale proportionally, unfortunately.

Yes I just updated Airtable - Gantt last week

Protocol Development Roadmap

Development

erik.near August 30, 2021, 7:22am 7

Hi, is there a refreshed / updated or dynamic version of this we can refer to for the latest?

Engineering Weekly Update -- 2021-06-21

Topic		Replies	Views
The simple nightshade launch and what to expect next Near Community and nodes	0	1516	October 24, 2021
Increase the gas limit of a single function call Proposals proposals	5	562	May 20, 2022
HSP-027: Remove the NEAR Developer Gas Rebate Proposals	20	794	September 17, 2026
2021-01-18 Engineering Update Engineering Updates weekly-updates	2	1181	January 19, 2021
2021 Q2 Engineering OKRs Engineering Updates weekly-updates	0	799	April 28, 2021

Protocol Development Roadmap

Related topics