what kind of data do you need to show that AssemblyScript is valuable.
There are standard techniques on doing market research and A/B testing, I would start with them. I am however not familiar with market research, but off the top of my head we should be taking care of the following:
- Identifying the segment that we are addressing and making sure the selection method is not biased. Are web developers the only segment that we care about? If not are we are reaching out only to web developers when asking about contract language preference? Can @AlexTinsman or @sashahudzilin help us identify the user profile that we are targeting and ratios between front-end/back-end/mobile/blockchain developers that we are surveying? Given, the most recent changes in org strategy by @illia the survey should be targeting more existing blockchain developers as oppose to front-end developers;
- Identifying if there is a bias in how we are sourcing the data. GH matches are biased because AssemblyScript is set to default in create-near-app, third-parties default to AssemblyScript, etc;
- Identifying the null hypothesis. What are we comparing the survey results to? Are we comparing AssemblyScript preference to Rust, other general purpose languages, or to blockchain-specific languages? Did we run survey with exactly the same questions for these other languages?
- Identifying minimum number of people that need to be surveyed. What number is high enough to make results distinguishable from a statistical fluke? Consider this survey.
p=0.419 chose Rust over JS-like languages.
n=248, which means 2-sigma is
0.06. If we had less than 100 people replying to this survey instead, it wouldn’t have been statistically significant;
- We need to actually have quantitative data;
- deep UX research from @corwinharrell and @amgando showing users prefer AssemblyScript and don’t have good understanding why would they use Rust instead
This research is not actionable, because it does not present any quantitative data. We had similar researches done by me and @frol during hackathon that showed strong preference of Rust over AssemblyScript. We even had collected numbers and extensive feedback.
having 200m locked in smart contract would be good example, maybe we need to make it more clear
- about 5x-10x smaller .wasm sizes on practice currently in prod. Which impacts both storage cost and execution cost because of .wasm compilation.
To clarify, this explains your personal preference, but it is not a market research.
So I don’t think we are biased towards AS.
We have bias towards AS because:
It’s probably true though never was supported by strong data (e.g. nearcore had serious security issues despite using Rust) but by trust into maturity of language.
Again, issue in language compiler or STD is not the same as issue in user code. Rust is more safe than AS, because Rust has stable Rust->Wasm compiler, while AS->Wasm is not stable. Plus there is a whole bunch of large organizations like Google and Mozilla that put their money or Rust being safe by using it in their core products. Let me re-post the criteria for the language compiler+STD to be considered stable:
Social approval. Large organizations put their money on it by using it in their core products;
Self-approval. The compiler+STD is classified as stable by their own core developers;
So far, the above is satisfied for Rust but is not satisfied for AssemblyScript.
Unfortunately C++ generally has similar issues as Rust: slow compilation speed
We still don’t have any quantitative data to support the argument that compilation time is an issue.
Unfortunately C++ generally has similar issues as Rust: not something most webdevs would want to use
I am not sure we have agreed that web developers is our primary market in short or medium term.
@luciotato I agree there are only advantages in making AS more compatible with TS. You are also far not alone with having frustrating experience with trying to write TS only to find out TS != AS.
To clarify one point in my position: I’m against removing AssemblyScript from our docs and tooling.
Is there a reason why we don’t have a switch in http://docs.near.org/ that allows users to switch all documentation to Rust or AssemblyScript view?
I think you may have gotten the wrong impression about this. We were not referring to AssemblyScript as TypeScript during the research nor have we made that mistake since we agreed as a team almost a year ago to be explicit about AssemblyScript.
The question in the research states:
Given the option of writing smart contracts in TypeScript/AssemblyScript or Rust, which would you prefer currently, and why?
It conflates TypeScript and AssemblyScript as if it is one entity.
@dcode AssemblyScript lacks the variety of libraries available for other languages which is one of the main reasons our users choose Rust over AssemblyScript. It would be very helpful if you could write a TypeScript->AssemblyScript transpiler that would do its best to remove manual effort of converting TypeScript libraries into AssemblyScript. Would you supervise this project if it was funded?
- IDE support using TS, including inline documentation and go to declaration support (even on Github!)
I am not sure what you are referring to.
At this point continued support isn’t as costly as starting fresh with another language and reaching the same level of DevX.
Our decision should not be influenced by the Sunk Cost Fallacy.
As I explain below to @erik.near here are non-financial costs to supporting multiple languages. Also, as I explained above we haven’t decided whether we want to target web developers as our primary market in the next 6 months.
Roadmap for next six months:
To clarify, we don’t have any evidence that fuzzing tools would produce significant value to people writing contracts. However, we have expert opinions of it being overvalued in this specific scenario of using it for AssemblyScript contracts. Fuzz testing is similar formal verification – everyone talks about it, it sounds nice in theory, it makes us feel safe, but it is not as efficient as improving test coverage by writing tests manually.
With some initial planning our part this roadmap is achievable in the next six months.
What is the expected impact of this initiative? How much closer this will bring us to stable AssemblyScript compiler?
- If devs want to deal with millions of dollars in their contracts and have done the work of learning Rust, why wouldn’t they (grudgingly) go through some additional difficulty in the build/test/deploy process? Seems like the “building super secure contracts” persona will not favor dev UX over that, so maybe the marginal value of investing more effort in rust, especially in our default flows, isn’t worth it. [this assumes the marginal future Rust dev effort is in UX-related stuff, but maybe it’s actually in deeper tech stuff and I didn’t catch that]
I partially agree, the marginal value of improving Rust DevX now is lower compared to when we were improving Rust DevX in the early days.
On the flip side, the highly iterative experimental dev archetype seems to value UX over security/precision. If the path to success in our community is about low barriers to entry and high iteration speed, this seems like the optimal place to put additional resources.
That’s true, what this archetype does not realize is that they can only enjoy good DevX if safety is guaranteed. I am not sure people would be able to enjoy MacOS same as much if it was riddled with viruses and exploits.
- Speaking of resources, what opens up if we use abundance thinking? If there are things we can do in parallel (or with increasing speed) by applying additional financial resources or bringing in other teams, etc, that’s what grants are for. Obviously, we have to focus the docs and particularly the time of core team members, but if there’s anything we can expand/extend/improve outside of that, please do so!
Reposting my answer to @erik.near here:
It sounds that you are describing Option 3. We can even do it for other languages and not just AssemblyScript and Rust. The reason why we need to have a discussion is because supporting language comes at other costs that are not financial:
- Distractions to the horizontal teams, like: Berrypickers, DevRel and whoever writes documentation, Partner Relations, etc;
- Potential reputation hit if we support something that later turns out buggy our with security vulnerabilities.
Also, stable compiler requires a scale of support that only large organization like Mozilla or Google can offer.