There have been recurrent suggestions to execute certain contracts (like EVM) with LLVM backend for Wasm, since LLVM is known to be faster than Singlepass backend that we are currently using. There are several interpretations to this idea:
We keep the gas counter;
We remove the gas counter but then introduce EVM-specific gas fees to make it not free;
We execute well-defined parts of EVM with LLVM without gas counter, and everything execute everything else in EVM the old way;
Keep the gas counter
This idea has the following three issues:
It does not actually reduce the contract gas cost/speed. Gas counter is injected into Wasm code before Wasm code is passed to Wasm VM for execution. This is the same approach that Parity is using (in fact, we are using their instrumentation tools to do it, see pwasm-utils). This means that independently on the VM frontend (Wasmer or Wasmtime) or the VM backend (Singlepass, LLVM, Cranelift, Wasmi, Lightbeam) contract will consume exactly the same amount of gas during execution. In fact, if total gas consumption by the contract depended on the VM backend then the specifics of the backend would’ve been a part of the protocol – this is because the nodes would need to agree on specific total gas consumption by the contract for each transaction. Making specifics of the VM backend part of the protocol is not possible, because Wasm compilers, even simple ones are indescribably complex and are platform-dependent;
LLVM is unacceptably complex. NEAR is currently struggling with a large number of bugs in Wasmer’s Singlepass VM backend, and just to address all these bugs we need to expand the Contract Runtime team. LLVM is orders of magnitude more complex than Singlpass backend, we would need to expand Contract Runtime team to unpractical size just to maintain it and very likely we would not be able to guarantee its safety anyway;
Even if we introduce separate fee for LLVM execution it will likely be the same as the fee for the current Singlepass execution. The difference between LLVM and Singlepass is that the former is an optimizing complier, while the latter is a non-optimizing single-pass compiler. This means that LLVM performs significantly better on average than Singlepass. However, runtime fees are dictated by the worst case scenario. Even if we introduce llvm_op_cost fee alongside regular_op_cost they will likely be very close to each other, since llvm_op_cost will be dictated by cost of Wasm operations when LLVM optimizations do not apply. It would be unsafe to compute llvm_op_cost based on the average cost of LLVM execution, since someone could technically trick EVM into execution paths that are not well-optimized by LLVM, thus slowing down block production on the Mainnet.;
Gas meter accounts for majority of the execution time. According to some benchmarks, most of the CPU instructions and correspondingly time is spend not on executing the Wasm code itself but on invoking the gas meter for every Wasm block. This is because gas meter is implemented through usage of host functions (there is no known safe way of implementing it differently) and host functions are crossing guest-host boundary which is inherently an expensive operation for Wasm, in general. Therefore, speeding up Wasm code execution itself will not speed up Wasm contracts drastically.
Any one of the above reasons is sufficient to conclude that this idea is not feasiblel.
Remove the gas counter but then introduce EVM-specific gas fees to make it not free
Currently EVM Wasm contract is a complex machinery that knows how to process Ethereum transactions, including contract deployments and cross-contract calls. We could whitelist EVM contract and allow it to be executed with LLVM and without injected gas metering. This will greatly reduce the number of CPU instructions needed to execute EVM. However, operations on the blockchain cannot be free and so we need to charge EVM contract somehow anyway. The only idea I am familiar with is listing all input parameters of EVM and trying to derive fees depending on the input parameters of each EVM transaction. Unfortunately, this will very likely be an unwieldy task. Even for simple operations like storage read/writes or cryptographic primitives it turned out to be a multi-person effort to pin-point fee cost within 15% accuracy, and we still keep finding bugs and fixing them. This is due to a large number of factors that require their own large post to describe. Given that EVM is much more complex than a single crypto function or a storage/read write it is unlikely we will be able to properly assign fees to all EVM operations within 15% accuracy.
Execute well-defined parts of EVM with LLVM without gas counter, and everything execute everything else in EVM the old way
The only solution that I think is feasible for speeding up EVM, is to extract CPU-intense simple EVM subroutines into host functions supported by NEAR runtime. In addition to extracting Math API we should extract EVM bytecode interpreter that would be called multiple times from a single EVM transaction. Assuming the bytecode interpreter is simple and hermetic (does not try to access the state on its own) it will be feasible to have a manageable correct gas estimation for it.
There are two approaches to it:
Bytecode interpreter is a Rust host function;
Bytecode interpreter is a precompiled Wasm host function executed with LLVM.
The second approach has the following pros and cons compared to the first one.
Pros (in decreasing importance):
Extracting part of Runtime logic into Wasm has many advantages described here and here. The biggest one is that we prevent littering runtime code with conditional compilation if-else branches that need to be maintained permanently;
It is easier to manage dependencies like access to storage, blockchain context, etc through the imports of the Wasm module;
It makes protocol spec simpler. Instead of describing the entire EVM bytecode in the spec, we can now link the Wasm file itself;
Encapsulated Wasm code is easier to optimize with existing Wasm tools, like wasm-opt, etc;
It is portable for if we ever decide to reimplement NEAR node in a different language;
We can potentially avoid guest-host boundary communication. Metered and non-metered parts of EVM can potentially be linked together avoiding costly interoperability. This will require both of them to run either on LLVM or Singlepass backend. The latter would still be fine, sine most of the performance boost will come from avoiding gas metering anyway.
Cons:
LLVM is slightly slower than native execution.
It currently seems to me like we have not choice but to follow the second approach, and we should be ready to spend majority of the engineering resources on ensuring correct gas estimation as compared to the engineering resources needed to adding the precompile itself.
I believe that’s not true – metering can be implemented safely in the VM layer. That is, rather than instrumenting wasm itself, the wasm runtime can add instrumentation when compiling wasm to native code. That instrumentation can bypass host functions, and be significantly cheaper.
Let me give some numbers here. Running this code for n = 100_000 takes
native code 0.9ms
wasmer1 singlepass 0.9ms
near contract with gas wasmer 0.17 122ms
near contract with gas wasmer 1 15ms
near contract no gas wasmer 0.17 1ms
near contract no gas wasmer 1 0.9ms
native code is just cargo run --release the benchmark directly. wasmer1 singlepass uses wasmer1 to directly run the code. near contract runs the code as a contract, with or without gas metering, using wasmer0 or wasmer1.
Note that the snippet is in, some sense, worst case. I don’t know how big of an issue gas metering is for EVM contract.
AFAIK There are only two approaches to gas metering right now:
Instrumenting Wasm to have a host function call on each Wasm block to deduct gas;
Keeping gas counter inside the memory of the Wasm contract and deducting from it without performing host function call. AFAIU this is what Wasmer is doing – their middleware modifies Wasm code to contain this kind of counter. Unfortunately, I don’t think there is a proof that this kind of gas counter is not abusable by the smart contract itself. E.g. hypothetically it is possible to construct a contract that can overwrite the value of such counter allowing it to execute for free. (I learnt about their gas counter from their presentation in 2019 ).
It is also possible to have a direct gas counting if executing Wasm through interpreter, but I am not aware of any other method except the above two for compiled contracts.
This would be ideal. However, there are difficulties with it:
Metering should be VM-independent, since metering defines the protocol. Injecting gas meter call on each Wasm block is a VM-independent approach. Wasmer metering should be specified to the extend where it is possible to re-implement it on a different VM;
If contract is compiled then it is not clear how it would signal to the host when Wasm operation/block was executed without instrumentation;
In my mind, having an efficient Wasm gas counter is a fundamental problem. We do need some instrumentation or otherwise host wouldn’t know when compiled Wasm contract executed an operation/block. We cannot have stateless instrumentation (like having counter inside Wasm memory) to avoid abuse by the contract. So it all boils down to injecting dumb host function calls into the Wasm code.
Fascinating! I did more research here, and here’s what I’ve found:
wasmer indeed uses wasm-level instrumentation, and not vm-level instrumentation, as I originally thought. That is, they inject a wasm global, and instrument the code to decrement this global in every basic block.
However, I believe that wasm spec guarantees that this instrumentation approach is safe and not abusable. The only way to wasm global variable from within wasm code is via global.set|get instructions, which take the index of a global as a compile-time constant. That the constant denotes existing global of the correct type is checked at module validation time. Instrumentation adds a new global, after the module has been validated. Unlike functions, there’s no equivalent to call_indirect instruction for accessing a global, determined at runtime.
I haven’t found an official super-direct statement claiming that this is safe, but:
wasm security document mentions that locals and globals are accessible only via indexes
spec mentions that globals are accessible only via global.get|set instructions
ctrl+f in the instructions document doesn’t reveal any additional ways to access the global
So, this instrumentation is safe in principle, although I haven’t verified that its bug free.
The only hazard I see here (pointed out by @olonho) is that some future version of wasm will add instructions for indirect access to global. I don’t think this is a real problem: the fact that locals & globals are managed is a part of security model; indirect access is not necessary a problem – indirect acesses go via table, and it’s possible to verify that a specific entity is not escaped via a table; in the hypothetical case where instruction is added to lookup arbitrary global by raw index, we can refuse to run contacts with this particular insturction.
I think it’d be hard to run evm naively. I’d expect at most 3x speed up from switch from singlepass wasm to native LLVM. One thing to realize is that wasm contracts are already optimized with LLVM (when they were compiled from rust to wasm). What we comparing here between singlepass and LLVM is basically just register allocator and peephole optimizer. There’s some benefits in using LLVM, but its less than an order-of-magnitude. As an example, for the first (synthetic) benchmark, the preformance of singlepass wasm matches native LLVM.
The problem with it is that if common Rust libraries upon compilation to Wasm will start utilizing this functionality then our developers won’t be able to import them, which would impede development of large contracts, similarly to how AS developers do not have a large choice of libraries to import these days.
Plus, to re-iterate again, the fees are determined by the worst case scenario when LLVM optimisations fail. So if for some programs LLVM is not faster than Singlepass then for all contracts LLVM will not produce lower fees/higher execution speed.
)