[PROPOSAL] Account Extensions: Contract Namespaces

Account extensions

This proposal is the first in a series of upgrades to the NEAR Protocol known as “Account Extensions.”

The goals of Account Extensions are:

  1. Lowering the financial barrier to entry for new NEAR users.
  2. Making it easier for non-developers to deploy smart contracts on their accounts (e.g. multisig wallets).
  3. Making it cheaper for developers to deploy the same contract multiple times (e.g. factory contracts).

This proposal is the first step towards those goals.

Problem statement

Currently, it is possible to deploy a single smart contract to a NEAR account using a DEPLOY_CONTRACT action. Subsequent DEPLOY_CONTRACT actions completely replace existing code with new code, while leaving all storage intact. Contracts deployed on locked accounts (those with no full access keys deployed) therefore must implement an upgrade mechanism or remain static forever.

If a user wishes to create a smart contract that implements an NEP standard, the user must either write an implementation of the standard, or hope that one already exists that the user can use. When multiple standards are involved (e.g. NEP-141, NEP-145, NEP-148, NEP-197, NEP-330) in the same contract, plus some custom behaviors, this can mean that the user must effectively take on the role of developer to integrate everything together properly.

Composable contracts proposal

“Account Namespaces,” or, alternatively, “Composable Contracts,” attempts to solve this problem by implementing contract namespaces on each account. Whereas accounts currently have a 1:0…1 relationship with contracts, namespaces would expand that relationship to 1:*.

For example, this would allow alice.near to deploy a multisig smart contract to alice.near:multisig, and later deploy a dead man switch contract to alice.near:deadmanswitch. Both contracts would be available for execution at the same time. Their storage would be separate.

This allows for developers to release “composable contracts” or “contract modules”: smart contracts that can be easily deployed to a namespace on an account without necessarily affecting the functionality of other smart contracts already deployed on that account. Contract modules would be attractive to non-developer users who wish to add more advanced features to their accounts without writing code.

This also allows for smart contracts to self-mutate one piece at a time, rather than all at once.

Terms & Notation

We will use the notation alice.near:multisig to mean “the multisig namespace on the account alice.near.”

If a contract is not deployed to a specific namespace, we say it has been deployed to the “empty” or “default” namespace. This is the current status of all smart contracts on mainnet.

We will use the term “namespaced contract” to refer to contracts that are deployed to namespaces other than the default namespace.

“Legacy contracts” shall refer to contracts that are unaware of namespaces. This is the current status of all smart contracts on mainnet.

Implementation details

The DEPLOY_CONTRACT action will be modified to include an optional namespace field. The code attached to the action will be deployed to the specified namespace, replacing any existing code. Storage would remain untouched. If the namespace field is missing or is the empty string ””, the contract is deployed to the default namespace.

This feature faces a few important questions before it can be implemented:

  1. How do namespaced contracts interact with other contracts?
  2. How do other actors (users or contracts) interact with namespaced contracts?

The answers to these questions dictate the specific design decisions that will affect this feature.

The team at Pagoda has been discussing this proposal for some time, and has come up with two different routes for implementation:

Explicit namespaces

This route involves modifying the FUNCTION_CALL action to include a new, optional namespace field, which, if specified, will direct the call to the contract stored under that namespace on the receiving account. If the namespace field is missing, the function call will fall back to the default namespace.

(Instead of introducing a new field, the namespace could equivalently be added to the receiver account ID field, however, this would probably break all existing contracts that use near_sdk::AccountId, since the account ID / namespace separator character must be one that is currently disallowed in account IDs. The namespace could equivalently be added to the method name field (e.g. multisig->sign instead of just sign), but this would break existing contract standards. However, if the introduction of namespaces is accompanied by new contract standards, this might not be an issue.)

Contracts receiving function calls from namespaced contracts would still see the unadulterated account ID of the containing account in near_sdk::env::predecessor_account_id(). However, near_sdk::env::predecessor_namespace() would also be available, returning the namespace that the predecessor contract occupies.

Furthermore, future NEPs would be able to specify a “well-known namespace” for contract standards, which would identify a particular namespace (e.g. nep500) under which standard-compliant namespaced contracts are expected to live.

Strengths & weaknesses

This route is generally “cleaner,” leading to a well-defined separation between namespaced and non-namespaced interactions.

However, it offers subpar backwards compatibility with existing contracts: a namespaced contract could potentially send interactions to a legacy contract which saves only the account ID, and fails to save the namespace. The legacy contract would then be unable to properly initiate interactions with the namespaced contract, possibly resulting in unexpected behavior.

Impact on stakeholders

Five different stakeholders have been identified. The impact on each is described below:

  • End-users: low. Namespaces would be part of the transactions generated by dapps and smart contracts, etc. likely requiring little to no additional input from the average end-user.
  • Legacy smart contracts: high. Legacy contracts are unaware of namespaces and will flat-out not work with new, namespaced contracts. This could lead to a fractured contract ecosystem, especially if the legacy contracts in question do not have a means of performing upgrades.
  • Smart contract developer: medium. Cross-contract interactions now have an additional (albeit small) field to worry about.
  • Protocol maintainers: medium. Storage and interactions are slightly more complex.
  • Node operators: medium. A database migration is required.

Routing table

This route involves setting up a protocol-level routing table for dispatching incoming function calls. Routing tables map incoming method names to a namespace, method name pair. This eliminates namespaces from being a public construct. The routing table would establish a single, unified interface for all of the namespaced contracts to the public blockchain. Method names not found in the routing table fall back to the default namespace.

For example, let’s say alice.near has deployed an NEP-141 contract to alice.near:ft and an NEP-171 contract to alice.near:nft. The routing table might look like this:

  • ft_transfer{ namespace: “ft”, method_name: “ft_transfer” }
  • ft_total_supply{ namespace: “ft”, method_name: “ft_total_supply” }
  • ft_new{ namespace: “ft”, method_name: “new” }
  • nft_transfer{ namespace: “nft”, method_name: “nft_transfer” }
  • nft_new{ namespace: “nft”, method_name: “new” }

The routing table would be modified through DEPLOY_CONTRACT actions, via a new routing_table field, which contains entries which will be merged with the existing routing table, overwriting duplicates.

Strengths & weaknesses

The routing table approach completely hides namespaces from other contracts on the blockchain, allowing for perfect backward compatibility with legacy contracts, since this approach would not involve changing the receipts posted to the blockchain.

However, this comes at the expense of higher complexity for developers and users. Method name collisions are non-trivial to resolve. Although it is fairly easy to automatically generate a functional routing table, developers still must now include routing table design in the development process. Additionally, if a non-developer user tries to deploy a contract module whose routing table conflicts with a contract already deployed on that account, then the non-developer user is confronted with the task of designing a routing table: a task they probably really don’t want to do.

Impact on stakeholders

Five different stakeholders have been identified. The impact on each is described below:

  • End-users: high. Most of the time, routing tables can be generated automatically without fear of collisions. If there is a collision, it could be the case that the user is trying to, for example, deploy a second fungible token smart contract on the same account, in which case the reason (and potential resolution) of the issue should be fairly clear to the user. However, if it is the case that there is a partial collision between two contracts (like two new methods), the resolution may not be immediately apparent to the end-user. Worse, if it is up to the end-user to resolve these conflicts, the user could very easily ruin their existing routing table, bricking the contracts (and potentially their account).
  • Legacy smart contracts: low. Routing tables are completely compatible with existing receipts.
  • Smart contract developer: medium. It should be possible to automatically generate routing tables that are suitable for a large fraction of cases, requiring little extra work on the part of the developer.
  • Protocol maintainers: medium. Accounts and interactions are slightly more complex.
  • Node operators: medium. A database migration is required.

Benefits of namespaces: TL;DR

  • Easier contract composability, e.g. just deploy the NEP-141 and NEP-330 modules and you’re set. Good for developers (esp. w.r.t. contract factories) and non-developer users (one-click deploy composable modules).
  • Safer and easier contract upgrades, e.g. have an upgrade_controller namespace that manages upgrading the rest of the contract’s components, so a bad upgrade doesn’t brick the account.

In the future, contract namespaces could also integrate nicely with:

  • Contract module subscriptions: A “contract subscription” allows an account to deploy just the hash of a smart contract instead of the full binary. This, in conjunction with namespaces, would allow a user to pick and choose a set of account features to deploy with negligible gas and storage costs.
  • Permissions: It could be possible to set permissions on a particular namespace. For example, restricting the namespace dao_multisig to interactions with dao.near only.
  • Synchronous execution. Namespaced contracts on the same account should be able to communicate with each other synchronously instead of asynchronously.

Feedback

The team at Pagoda is seeking community feedback on this proposal and the two proposed implementation approaches: explicit namespaces vs. routing tables.

10 Likes

Hmm… are the contracts going to share any state e.g. account balance, etc.? Not clear how this would work in those cases.

Are you able to provide some concrete evidence of how difficult this is in practice please?

2 Likes

Namespaced contracts operate under the same account state (balance, account ID, etc.), but the contract state tries are separate. Under this proposal (i.e. not including the “sync execution” extension) namespaced contracts deployed on the same account would have to communicate using regular receipts (i.e. reflexive function calls).

“Issue an NFT” UIs don’t usually provide much, if any, flexibility in terms of the functionality of the contract deployed. High degrees of customization, particularly composing multiple unrelated standards into one contract (like an NFT contract which also has multisig features), usually require some manual coding.

As for concrete evidence that this is difficult in practice, I could point to the stereotypically steep learning curve for Rust programming, the different security surfaces that a programmer must be familiar with to develop a secure multisig interface, etc.

However, this point is moreso trying to say that users may want to add features to their accounts (e.g. multisig, dead man switch) in custom configurations without needing to learn anything about coding, and not that said coding would be difficult, even if it would be.


Here is an idea of how to leverage namespaces that might clarify how useful this could be to the average end-user:

NEAR Wallet contains an “Add account extensions” section containing the following list:

  • Multisig
  • Dead man switch
  • Neth controller
  • Lockup contract
  • NFT
  • FT
  • Multi-token

Each option has a checkbox beside it. Users may select any combination of features, input minimal configuration details (e.g. list of account IDs and minimum approvals for multisig) and click “Save” to deploy the correct set of namespaced contracts on their account, without ever needing to touch any code or leave the wallet interface.

3 Likes

Thanks for the explanation. This makes sense to me.

1 Like

My first question would be, is there any reason we actually need this as a protocol extension? The way I see it, this proposal only covers contracts that want to pop from the NEAR balance. So we could “just” deploy to alice.near a contract that accepts any transfer request from sub-accounts (thereafter the “multi-contract contract”), and then deploy multisig.alice.near that’d figure out its own ID, and call alice.near’s “multi-contract contract” when it needs any funds. This should work fine, as AFAIR only alice.near can create multisig.alice.near (but that’d need proper confirmation)

If this alternative solution idea were to not work, I wonder how much of the “explicit namespaces” backwards compatibility troubles is an issue in practice. Do we have any significant contract that cannot be updated, sends a callback to a contract that called it, and would reasonably be called by an extension contract?

PS: I’d also suggest using a syntax like multisig@alice.near rather than alice.near:multisig, just so that the hierarchy stays readily apparent. Or using alice.near/multisig by analogy with HTTP maybe, if the other order is preferred?

4 Likes

When we call env::storage_usage(), currently, both account state and contract state will be calculated. If using namespace contract, what is the behavior? Will all namespaces storage cost be calculated?
Besides, multiple contracts in one account means every contract own the control of the account, it will increase security risk, and it will break some Account Market Dapps such as NomenMarket, NameSky, etc. which use unique contract to control the account.

2 Likes

Is WASM in WASM another approach to implement Account Extensions, which requires almost no change at protocol level? Or maybe we can just use JSVM here, if JS contracts are implemented properly?

2 Likes

Thanks for the feedback.

While it is true that each namespaced contract (as the proposal currently stands) would have access to the entire NEAR balance of the account, some more salient differences between the “multi-contract contract” model and namespaced contract model are:

  • Namespaced contracts can act directly on behalf of the main account, i.e. their cross-contract interactions have near_sdk::env::predecessor_account_id() == "alice.near".
  • The multi-contract contract model requires the deployment of a non-trivial proxy contract to the main account. One of the goals of the Account Extensions initiative is to reduce storage costs.
  • Since subaccounts are just regular accounts, interactions between the sub- and main accounts require normal, asynchronous receipts. Synchronous execution (at least between namespaced contracts on the same account) is one of the eventual goals.
  • View calls are not allowed to submit receipts, so an NEP-171 nft_token proxied view call to a multi-contract contract would fail.
  • (Other items mentioned under the “Benefits of namespaces: TL;DR” heading above.)

Although it might not always be preached from the hills, it is recommended to “lock” (delete all full-access keys from) production-ready smart contracts on mainnet. Contract upgrades can be a source of trustedness, so upgrade functionality may not always be included. On principle, I do not think it reflects well on NEAR Protocol to flip the script on contract developers who have heretofore followed “good practice,” and require them to upgrade or deprecate newly-christened “legacy” contracts. However, that’s just my opinion, and not necessarily empirically-backed.

The alice.near:multisig syntax actually is inspired by HTTP URLs, which use a similar syntax for port numbers, which identify applications running on a host. @ usually symbolizes “user,” and / “file path.”

I am inclined to believe that it would calculate the sum total consumption of all contracts deployed on the account. This proposal is not for namespaced contracts to act like independent entities simply occupying space on someone else’s account, but for them to be unified as a single, cohesive (yet multifaceted), actor.

However, the exact behavior is up for discussion.

It may be reasonable to introduce a mechanism by which existing namespaces can be enumerated so they can be deleted. However, you are correct in that it sounds like even with this additional mechanism those existing smart contracts would probably break.

1 Like

Yes, it would solve some of the immediate issues presented, and enable some of the same features as well. I’m not intimately familiar with the WASM-in-WASM proposal, but based on the linked discussion it looks like it would aggravate some of the very issues we’re hoping to alleviate with Account Extensions, namely financial barriers and fees.

I am not sure if the WASM-in-WASM proposal also includes a “fallback”-style feature, wherein a smart contract can still be invoked with a method name not explicitly exported from the WASM blob, which would be necessary for parity with the namespace proposal.

Storage layout for WASM-in-WASM is not clear based on my understanding of the linked discussion.

2 Likes

Hmm but this is actually a problem, that the routing table is trying to solve… is it not?

As far as I can tell, this is mostly the case for the part about eg. global contracts, more than for contract namespaces themselves. And global contracts would naturally reduce the cost of the non-trivial proxy contract to 0, as literally every multi-contract account would be deploying it.

In this case, I don’t think it would make sense to accept this proposal as-is without the synchronous execution being integrated too? As things stand, I (someone who has not been involved in the design of account extensions, and thus do not know of the broader context it is placed in) find it hard to see benefits of this proposal as outweighing the costs.

Maybe with synchronous execution the benefits could clearly outweigh the costs! In which case we could accept the full NEP (including synchronous execution), and then we start by implementing the contract namespaces as stated here first, because it’d be a natural first implementation step.

I’m totally with you on this! I just wonder how many smart contracts actually ever:

  • call back to the caller in a way that would be incompatible with explicit extensions,
  • would have a chance to be called by an extension, as opposed to the main account, and
  • cannot be just (updated or) replaced by a smart contract on another account that does take explicit extensions into account.

It feels to me like a niche use case, but even if we have one it might be enough to say we cannot go the “explicit extensions” road!

I actually had the same thought when writing my comment, and then decided to include it anyway because most end-users of computers actually no longer know about the : syntax for port numbers. Anyway, no strong opinions here.

I believe wasm in wasm is indeed another approach to a similar problem, similarly to single-contract wasm extensions. However, it is much less fleshed out than this proposal to the best of my knowledge.

At last nearcon when we (Contract Runtime) chatted with Aurora folks, who seemed like the biggest proponents of an extension model at the time, it sounded like an acceptable solution for them.

However, things may have changed quite a lot in the meantime! And contract namespaces solve a slightly different problem, as even wasm in wasm would need some way of figuring out which sub-contract to call (which could end up being contract namespaces’ routing table too).

2 Likes

It sounds like the main concerns being presented here are that the features contract namespaces enable on their own don’t outweigh the protocol work they require to warrant their introduction, which is a sentiment I personally would tend to agree with. However, contract namespaces are a first step in the Account Extensions upgrade, the epic for which has just been posted here, and which Illia spoke about at length on Twitter last year.

I think this is not uncommon. For example, a (legacy) escrow contract escrow.near might have the following workflow:

  1. alice.near deploys a fungible token to alice.near:ft.
  2. alice.near calls alice.near:ft->ft_transfer_call({"receiver_id": "escrow.near", "amount": 1, "msg": {"beneficiary_id": "bob.near"}}), where the msg field tells escrow.near the recipient of the funds after the conditions of the escrow (specified elsewhere) are met.
  3. The ft_transfer_call call creates a promise chain that invokes escrow.near->ft_on_transfer. escrow.near, unaware of namespaces, only saves alice.near as the address of the fungible token.
  4. The conditions of the escrow are fulfilled. escrow.near calls alice.near->ft_transfer({"receiver_id": "bob.near", "amount": 1}), which fails, because escrow.near failed to specify the correct namespace.

A similar problem could be encountered by DEX contracts, albeit in a more convoluted way.

One way to sidestep this problem is to say that NEP-141-compliant contracts must be deployed on the default (empty) namespace, and we will introduce new fungible token standards for namespaced contracts.

1 Like

Thank you for your answer! I now understand that sync execution and synchronous cross-contract calls are the same thing, I still had them separated in my brain ^^’

As far as I can tell, the “multi-contract contract” proposal and the “explicit namespaces” are currently (ie. notwithstanding future changes like synchronous execution) basically the same. However, this all changes as soon as sync execution comes up, as we probably don’t want to just allow any account to say “all my child accounts are now on the same shard”, as child accounts can have different access keys and that’d allow for shards to become very imbalanced.

Though TBH I kinda like the idea of one contract being able to say “all contracts below me are now on the same shard and can interact synchronously”. Sure it’d lose some of the nice sharding properties we currently have (no incentive to being placed in the same shard), but it’d replace that with a market-based shard placement (we could imagine one contract selling places underneath it to other contracts that want to interact synchronously while staying a small enough contract group that it doesn’t just get its shard overloaded). Is this an idea that has been explored yet?

Also I guess your message means that there’s no choice but go for the “routing table” approach, either routing namespaces or as a general solution to allow one contract to route method calls to another contract without needing to change the deployed contract.

Thanks Jacob for writing down the problem and possible solutions in detail! I have a suggestion that will make the design more complex but possibly easier to handle the backward compatibility issue:

  • There is a notion of the default namespace which will be preserved even after this change. When a function call only specifies the name of a contract, it will be executed on the default namespace.
  • For legacy contracts, they should be able to still interact with the contracts that they are able to interact with today after the change, because there is no change to the default namespace. They will not be able to interact with namespaced contracts, but I don’t really see a use case for that. Please correct me if I missed something, but it is likely end-users who will deploy namespaced contracts and because it is not a thing today, I don’t think app contracts would attempt to call a contract deployed on a user account. Usually the interaction with a user account happens in the form of token transfers, which is not affected by this change. Now for applications that decide to adopt this new feature and upgrade their smart contracts to use namespaces, likely they should think about backward compatibility anyway. Put it different, it is always possible to break a chain of calls if one of the contracts in the call chain upgrades itself and breaks its existing API. That problem exists regardless of whether we have namespaced contracts.
2 Likes

I have a few thoughts on the two alternatives proposed in comments.

TLDR:

  • Proxy accounts add latency and still require a routing table.
  • WASM in WASM is too much (complex) work to do in one go.
  • Overall, the original proposal by Jacob seems the most straight-forward first step but some more clarification is needed.

Proxy accounts

(Specifically the flavour where we deploy extensions to subaccounts and use the parent account to proxy actions.)

Yes, we could equally optimize their storage cost. But it would increase gas cost and, more importantly, latency.

Also, as others mentioned, the routing. Again, we either need something like a routing table or we fragment the ecosystem into legacy contracts and new contracts that can’t interact between each other.

To me, adding a routing table that still requires a proxy contract and the cross-contract call overhead seems like a weird feature to add. Hence, I’d rather implement “proper” namespacing as proposed by Jacob.

OTOH, if we don’t care about backwards compatibility for applications like legacy marketplaces and what not, then I feel like this proxy-contract idea might work to get account extensions started and we could optimize for latency later if we have confirmation that users like extensions but don’t like waiting the extra latency due to the proxy cross contract call.

WASM in WASM

I think it could work but requires complex smart contract code and complex protocol changes.

First, at least the way aurora would use it, this wouldn’t deploy subcomponents or expose any functions the the outside world. They just want to send some WASM code as a Vec<u8> argument to the function call and then execute that in a host function call, sort of like JS eval() evalues a string as JS code.

Of course, we can go beyond what the aurora team proposed so far. We could design WASM components exactly as we see fit, giving each component natively isolated storage and allow synchronous calls to other components with a routing table in place for internal and external calls.

Or if we really want to avoid adding too much to the protocol, I believe a sufficiently complex OS-like smart contract standard should be able to do provide storage separation and some form of routing without protocol support, as long as wasm_eval() works. But I don’t quite see who would tackle the task of designing this.

Regardless which way we go with WASM in WASM, the sum of work seems truely immense. But putting that aside, I think some form of WASM components might just be the holy grail of on-chain composability. :slight_smile:

However, I also feel that starting with namespaces and extending it to synchronous calls will bring us to about the same place as if we start with WASM in WASM and extend it with storage isolation and method routing.

My conclusion so far

  1. It seems we have to first answer the question how much we care about compatibility of existing applications that can’t or don’t want to update. But we probably don’t even understand the full extent of things that can break, yet. (As new problems were identified quickly even in previous comments.)
  2. Permissions seem like an important topic which hasn’t been discussed much. There is a wide gap between “any extension has full access to everything” and the “install app in sandbox” experience we are so used to from our mobile phones.
  3. Ideally, we pick a first step towards a solution that can not only reduce storage cost but also facilitates sync execution. The original proposal seems like a legit way to do that first step.
1 Like

Hi Bowen, thanks for the response. I think this is a good addition, thank you! Although I didn’t detail it in the post, the proof-of-concept implementation supports this kind of fallback.

Regardless of which is implemented first, I think that WASM-in-WASM could serve as a good complement to namespaces as well.


Overall, it seems like the community response to this proposal is lukewarm-positive. We’ll do some more work and discussions and try to hammer out a concrete NEP soon. Thanks everyone for the contributions!

The NEP draft has been created: NEP-480: Account Namespaces by encody · Pull Request #480 · near/NEPs · GitHub

We will use the notation alice.near:multisig to mean “the multisig namespace on the account alice.near.”

I would like to bring up that this notation might lead to issues for the Rainbow Bridge since it uses a <near_account_id>:<msg> notation to pass messages (ref, ref). To which extend probably depends on whether a namespace could ultimately end up being involved in token transfers.

This notation (so far) is just for ease of readability, and is not currently part of the (in-progress) proof-of-concept implementation, as account ID and namespace are always separate fields. However, this is a good thing to keep in mind as the NEP progresses.