RFC: Investigation around AssemblyScript

[valstu]

This PR contains inital investigation around AssemblyScript support. So far this repo adds following findings:

• Changes to next.config.js file so we can actually use AssemblyScript compiler on next.js project
• Changes to compilation logic, when user clicks compile we check out if the file is TS (aka AssemblyScript) file and branch out to different compilation logic using asc.
• At the moment this supports compiling single files, haven't tested if this works when there are other files imported on a single file Should support now also imports inside files
• Updated React to version 18 which caused some regression on few component typings, fixed those
• This also uses latest Next.js version now
• Monaco editor syntax highlighting is still a mess
• Doesn't have any sort of language server support
• Better error handling is needed for compilation step, atm. logic is quite barebones

Hopefully this helps testing and investigation around this topic. You can now add .ts files and write some example hooks in AssemblyScript and compile them. Deploy should work, at least it sends the SetHook request in correct way but haven't been able successfully deploy a hook yet. We're getting this error:

OpenLedger:TRC HookSet[rsVQvKqonS4cBmdiWGXRsh11Ax7J1tu9GP-B504F8082003DFCC377CFDE52A1EA4613480695CD725FA6CABC69F97D87E619D]: Malformed transaction. hook and cbak function definition must have exactly one int64_t return type.

[vercel]

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/equilibriumco/xrpl-hooks-ide/4mwCRoJYBMsRM9Ur2CJD84ujUud1
✅ Preview: https://xrpl-hooks-ide-git-feat-asc-compiler-equilibriumco.vercel.app

[vbar]

So, could it also include the Language Server ( https://github.com/typescript-language-server/typescript-language-server )? That still doesn't have quite the right syntax, but I think tsserver could be modified, at least not to show errors on common type names...

[valstu]

Added import support to files

[WietseWind]

Here's something ugly I made some time back with AssemblyScript that compiles to a working Hook:

https://pf2aqwq.dlvr.cloud/#IwAhAAoAaQBtAHAAbwByAHQAIAAnAC4ALwBYAHIAcABsAEgAbwBvAGsAcwAnAAoACgBlAHgAcABvAHIAdAAgAGYAdQBuAGMAdABpAG8AbgAgAGMAYgBhAGsAKAByAGUAcwBlAHIAdgBlAGQAOgAgAGkANgA0ACkAOgAgAGkANgA0ACAAewAKACAAIAByAGUAdAB1AHIAbgAgADAACgB9AAoACgBlAHgAcABvAHIAdAAgAGYAdQBuAGMAdABpAG8AbgAgAGgAbwBvAGsAKAByAGUAcwBlAHIAdgBlAGQAOgAgAGkANgA0ACkAOgAgAGkANgA0ACAAewAKACAAIABHAFUAQQBSAEQAKAAxACwAIAAxACkACgAKACAAIABmAG8AcgAgACgAbABlAHQAIABpACAAPQAgADAAOwAgAEcAVQBBAFIARAAoADEAMAAsACAAMQAwACkALAAgAGkAIAA8ACAAMwA7ACAAaQArACsAKQAgAHsACgAgACAAIAAgAEwATwBHACgAJwA92A7eJwApAAoAIAAgAH0ACgAKACAAIABPAEsAKAAnADzYid8nACkACgAKACAAIAByAGUAdAB1AHIAbgAgADAACgB9AAoAIwAhAGgAdABtAGwACgA8AHMAYwByAGkAcAB0ACAAcwByAGMAPQAiAGgAdAB0AHAAcwA6AC8ALwBiAHUAbgBkAGwAZQAuAHIAdQBuAC8AYgB1AGYAZgBlAHIAQAA2AC4AMAAuADMALwBpAG4AZABlAHgALgBqAHMAIgA+ADwALwBzAGMAcgBpAHAAdAA+AAoACgA8AGgANQA+AE0AZQB0AGgAbwBkAHMAOgA8AC8AaAA1AD4ACgA8AHQAZQB4AHQAYQByAGUAYQAgAGkAZAA9ACIAbwB1AHQAcAB1AHQAIgAgAHMAdAB5AGwAZQA9ACIAaABlAGkAZwBoAHQAOgAgADIAMAAwAHAAeAA7ACAAdwBpAGQAdABoADoAIAA3ADUAJQAiACAAcgBlAGEAZABvAG4AbAB5AD4APAAvAHQAZQB4AHQAYQByAGUAYQA+AAoACgA8AGgANQA+AEgAbwBvAGsAIABXAEEAUwBNACAASABlAHgAOgA8AC8AaAA1AD4ACgA8AHAAcgBlACAAaQBkAD0AIgBoAGUAeAAiAD4ARQBuAGMAbwBkAGkAbgBnAC4ALgAuADwALwBwAHIAZQA+AAoACgA8AHMAYwByAGkAcAB0AD4ACgAgACAAbABvAGEAZABlAHIALgBpAG4AcwB0AGEAbgB0AGkAYQB0AGUAKABtAG8AZAB1AGwAZQBfAHcAYQBzAG0ALAAgAHsACgAgACAAIAAgAGUAbgB2ADoAIAB7AAoAIAAgACAAIAAgACAAXwBnADoAIAAoACkAIAA9AD4AIAB7AH0ALAAKACAAIAAgACAAIAAgAGEAYwBjAGUAcAB0ADoAIAAoACkAIAA9AD4AIAB7AH0ALAAKACAAIAAgACAAfQAKACAAIAB9ACkACgAgACAALgB0AGgAZQBuACgAKAB7ACAAZQB4AHAAbwByAHQAcwAgAH0AKQAgAD0APgAgAHsACgAgACAAIAAgAGMAbwBuAHMAdAAgAG8AdQB0AHAAdQB0ACAAPQAgAGQAbwBjAHUAbQBlAG4AdAAuAGcAZQB0AEUAbABlAG0AZQBuAHQAQgB5AEkAZAAoACcAbwB1AHQAcAB1AHQAJwApAAoAIAAgACAAIABjAG8AbgBzAHQAIABtAGUAdABoAG8AZABzACAAPQAgAE8AYgBqAGUAYwB0AC4AawBlAHkAcwAoAGUAeABwAG8AcgB0AHMAKQAKACAAIAAgACAAIAAgAC4AZgBpAGwAdABlAHIAKABhACAAPQA+ACAAYQAuAHMAbABpAGMAZQAoADAALAAgADIAKQAgACEAPQA9ACAAJwBfAF8AJwApAAoAIAAgACAAIAAgACAALgBmAGkAbAB0AGUAcgAoAGEAIAA9AD4AIABbACcAbQBlAG0AbwByAHkAJwAsACAAJwB0AGEAYgBsAGUAJwBdAC4AaQBuAGQAZQB4AE8AZgAoAGEAKQAgADwAIAAwACkACgAgACAAIAAgAG8AdQB0AHAAdQB0AC4AdgBhAGwAdQBlACAAPQAgAEoAUwBPAE4ALgBzAHQAcgBpAG4AZwBpAGYAeQAoAG0AZQB0AGgAbwBkAHMALAAgAG4AdQBsAGwALAAgADIAKQAKACAAIAB9ACkACgA8AC8AcwBjAHIAaQBwAHQAPgAKAAoAPABzAGMAcgBpAHAAdAA+AAoAIAAgAGMAbwBuAHMAdAAgAGgAbwBvAGsASABlAHgAIAA9ACAAYgB1AGYAZgBlAHIALgBCAHUAZgBmAGUAcgAuAGYAcgBvAG0AKABtAG8AZAB1AGwAZQBfAHcAYQBzAG0AKQAuAHQAbwBTAHQAcgBpAG4AZwAoACcAaABlAHgAJwApAAoAIAAgAGQAbwBjAHUAbQBlAG4AdAAuAGcAZQB0AEUAbABlAG0AZQBuAHQAQgB5AEkAZAAoACcAaABlAHgAJwApAC4AaQBuAG4AZQByAFQAZQB4AHQAIAA9ACAAaABvAG8AawBIAGUAeAAKADwALwBzAGMAcgBpAHAAdAA+AA==

[vbar]

Here's something ugly I made some time back with AssemblyScript that compiles to a working Hook:

Hmm, which version of asc is it using? Mine (0.20.4) doesn't have --binaryFile option (and doesn't compile even with --outFile)...

[valstu]

Thanks @WietseWind. I also found/got this gist made by you https://gist.github.com/WietseWind/0a0d07e7df8e44efa9c8befa55d97b7d. Seems like that with both examples, at least the WAT output is similar when using same compile settings, which is nice! In our investigation we're using the latest AssemblyScript compiler.

Have you been able to deploy any of the hooks built with AssemblyScript? We bumped into this error: Malformed transaction. hook and cbak function definition must have exactly one int64_t return type. Error itself is quite clear but the function definitions have i64 return type which I guess shoud be correct. 🤔

[valstu]

Here's something ugly I made some time back with AssemblyScript that compiles to a working Hook:

Hmm, which version of asc is it using? Mine (0.20.4) doesn't have --binaryFile option (and doesn't compile even with --outFile)...

Seems like Wietse's example is using this version AssemblyScript v0.19.23 (Binaryen v102.0.0-nightly.20211028, Long v5.2.0) at least that's printed on the browser console :)

Version 0.20.0 (24 days ago) introduced quite a bit of changes, for example they started to use ESM modules and while doing that they removed the old browser SDK, also a lots of other changes were done (check release notes for v0.20.0). That release (and issues posted around it) kind of triggered me to investigate the changes in the first place and find out how it might affect us if we want to proceed with the AssemblyScript support.

So I guess this is the release where they changed the cli options as well. Weird that there isn't any big mention about it.

[vbar]

BTW the hook binary from the example above does deploy on hooks v1 testnet - so the hook and cbak function definition must have exactly one int64_t return type problem is apparently something changed for v2...

[valstu]

BTW the hook binary from the example above does deploy on hooks v1 testnet - so the hook and cbak function definition must have exactly one int64_t return type problem is apparently something changed for v2...

Aah, good point, I tested only against v2

[WietseWind]

I figured it out. Got it working on the latest version of AssemblyScript (v0.20.4).

The problem occurs if there are const exports instead of functions.

Works:

export function OK (text: string = 'OK'): void {
  ACCEPT(text, text.length * 2, 0)
}

Does not work:

export const OK = (text: string = 'OK'): void => {
  ACCEPT(text, text.length * 2, 0)
}

Super simple working example:
module.ts:

import {GUARD, LOG, OK} from './HooksLib'

export function cbak(reserved: i32 = 0): i64 {
  return 0
}

export function hook(reserved: i32 = 0): i64 {
  GUARD(1, 1)

  for (let i = 0; GUARD(10, 10), i < 3; i++) {
    LOG('-- :D --')
  }

  OK('-- :) --')

  return 0
}

HooksLib.ts:

@external("env", "_g")
export declare function GUARD(id: i32, maxiter: i32): i64

@external("env", "accept")
export declare function ACCEPT(read_ptr: string, read_len: i32, err: i64): i64

@external("env", "trace")
export declare function TRACE(mread_ptr: string, mread_len: i32, dread_ptr: string, dread_len: i32, as_hex: i32): i64

////////////////////////

export function LOG (text: string): void {
  TRACE(text, text.length * 2, text, text.length * 2, 0)
}

export function OK (text: string = 'OK'): void {
  ACCEPT(text, text.length * 2, 0)
}

Compile:

asc -O3 --noAssert --runtime minimal \
  --textFile ./module.wat \
  --outFile ./module.wasm \
  ./src/module.ts

🎉
https://hooks-testnet-v2-explorer.xrpl-labs.com/tx/F275FF6AC9F3365EEBAE03BA44A6A189FDB9BA3CA6D8F853D9C62A0CB3A34EA2

[WietseWind]

Nice one @ asc compiler:

  --lib ./src/lib

Adds one or multiple paths to custom library components and exports of all top-level files at this path as globals.

So no more 'import {...} from 'our-hooks-lib' 🎉

--

I'm only focussing at V2 now, it's working on V1 but V2 shows a couple of issues. This sample above:

1. Compiles
2. Allows on Hook Set

It's actually not working after that:

1. It only works if the Guard is defined as return type i64 in AssemblyScript (when defined as i32 as it should be, the SetHook returns Malformed transaction, debug output: Malformed transaction. hook and cbak function definition must have exactly one int64_t return type.
2. When Guard is declared as return type i64, the Hook can be set, but when triggered:

11:06:54 View:TRC HookInfo[[Carol]()-[Alice]()]: creating wasm instance 
11:06:54 View:WRN HookError[[Carol]()-[Alice]()]: WASM VM error incompatible import type 
11:06:54 View:TRC HookInfo[[Carol]()-[Alice]()]: ROLLBACK RS: '' RC: -1

@RichardAH is looking into this :)

[valstu]

Oh nice! 🥳 @WietseWind your update to your latest comment was the solution, we actually used only function statements while investigating this topic, so our problem wasn't that we used const exports. It was the return type on guard that was the issue all along.

[WietseWind]

@valstu Great. Meanwhile @RichardAH found a bug in the Hooks source so: getting there :)

[WietseWind]

@valstu @vbar Bug fixed, deployed to Testnet V2 and confirmed it's working.

So: lib: Guard returns i32:
https://gist.github.com/WietseWind/ec1bcbd42c159d45a206a4e29f5135c7

🎉

[valstu]

Oh wow! And we were only supposed to investigate the topic but it kind of works already 😅 Well there is of course the language server stuff which is completely separate topic but cool that we can actually create working hooks with AssemblyScript 🎉

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated
xrpl-hooks-ide	✅ Ready (Inspect)	Visit Preview	Aug 24, 2022 at 2:13PM (UTC)

[socket-security]

Socket Security Report

👍 No new dependency issues detected in pull request

Socket.dev scan summary

Issue	Status
Did you mean?	✅ no new possible package typos
Install scripts	✅ no new install scripts
Telemetry	✅ no new telemetry
Troll package	✅ no new troll packages
Malware	✅ no new malware
Native code	✅ no new native modules

Powered by socket.dev

[valstu]

This is now up to date against main branch