[Help Wanted/Design] Improve TypeScript integration

[matthid]

What?

Basically, I want to improve the interop with TypeScript and asking for help and directions.

This means extending the fable-compiler in two areas:

1. Emit TypeScript bindings (we currently write them by hand)
2. Consume/Import TypeScript (we currently use ts2fable for this)

Please feel free to correct me throughout the post, suggest alternative implementations or even point out why this is a bad idea. Any help regarding the design, implementation or else is welcome. Feel free to connect privately via Twitter or Slack (or Mail).

Why?

The linked related issues (see the end of this post) should give a couple of reasonable use cases. But in particular:

• After "1." it is easier to publish fable-based libraries on npm without any additional manual work.
• "1." should allow import "./MyFile.fs" in TypeScript/Javascript with proper IDE support. (It is not exactly clear what additional thing we would need to do here for the IDE to pick up the typings)
• "2." Would make it a lot easier for newcomers to use packages in the npm ecosystem
• "2." Might allow us to reduce our work of maintaining and keeping the typings updated.
• (Future discussions, just thinking our loud) After 1 + 2 we could potentially publish packages on npm. For F# tooling we might still need to include a reference assembly (or just the .net assembly, considering WebAssembly...). There are however a lot of other issues to be solved. Just putting this here to put it up for discussion if this is what we want eventually.

How?

I'd like to talk about 1 only for now (and extend the post later)

Emit TypeScript bindings

Looking at the available APIs and the related discussions I feel like the best bet is to use the TypeScript API as there are online tools which make is easy to understand and work with.

I have played a bit with the code base and noticed the following:

• We could just use the TypeScript-API in fable-compiler-js, we just need a couple of bindings or use unsafe calls.
• However, in regular .NET based fable we cannot use the TypeScript API, so we need a intermediate serializable datastructure for type declarations built from Fable.AST. It will look similar to Fable.AST but only type-specific stuff (all Expression stuff will be removed)
• I haven't figured out what the best way is to inject TypeScript definitions into webpack via fable-loader, but as a first step I would just output the .d.ts files somewhere (alongside the .fs file for example).

Yes it probably is a bit of work but all it all it sounds doable to me. In practice:

• We could start with a quite minimal implementation only supporting simple interfaces (for example) and using any for everything else
• We probably want to make this opt-in until we are more confident
• We can extend this feature for feature (ie typings will become better over time)

Consume/Import TypeScript

I will expand this section later or throughout this discussion. But my current ideas are:

• Create a "build-in" type-provider type MyTypings = TscImport("typings.d.ts")
• Write .fs files, for example based on attributes -> add to project file via globbing

Related issues

This issue is a continuation of:

• Fable2Typescript backend #104
• TypeScript bindings #232
• TypeScript integration #805
• Suggestion: fable-loader should output type information #1159

[Luiz-Monad]

The other day I was working on making ts2fable work better translating types. But I stopped after I started hitting problems with the need for type tagged unions. fsharp/fslang-suggestions#538
Also I didn't find a good way to represent the Pick row polymorphism with F# column polymorphism. They don't quite fit together for a 1:1 translation of types.

[Luiz-Monad]

Perhaps we could do the opposite, convert all the TypeScript AST to F# AST. [insert evil mad computing scientist laugh]

[alfonsogarciacaro]

Thanks a lot for this detailed issue @matthid. As we've talked some types, it's true I'm a bit skeptical about the Typescript integration because as @Luiz-Monad says, they type systems of both languages have differed quite a bit. But I'd love to be proven wrong and see how we can take advantage of .d.ts declarations.

I can see you've done your homework and listed the older issues around the topic so I don't need to repeat the info... or actually look for it myself because I've already forgotten it :) But I'll still try to add some comments as I remember things. For now two quick notes:

• There's already work in progress by @ncave to bring back the annotations in the Babel AST Adding Babel types #1615. Babel can now parse Typescript, but I don't remember if it works the other way around (annotations in Babel AST used to emit Typescript).
• About importing F# files into Typescript, I've noticed the problem is Typescript doesn't let you declare a non-js module with a relative path. There's a note about this now in the docs.

[matthid]

Thanks!

But I stopped after I started hitting problems with the need for type tagged unions.
Also I didn't find a good way to represent the Pick row polymorphism with F# column polymorphism

As we've talked some types, it's true I'm a bit skeptical about the Typescript integration because as @Luiz-Monad says, they type systems of both languages have differed quite a bit.

Yes, I think this is indeed a problem, but my current thinking is this is only really a problem for "2." not for "1." (or do you know any examples where this is a problem for "1."?). In general it feels like the TypeScript type system is more powerfull than the F# one. This most likely a general problem in the ecosystem as TypeScript just tries to "type" existing javascript. So this basically means we have problems mapping existing code to our type-system.

To solve this for "2." we are not lost either:

• Long term: Openeing suggestions like you did, but we also need to lobby the discussions to stay useful for fable (as you can see in Erased type-tagged anonymous union types fsharp/fslang-suggestions#538)
• Short term: We can map unknown types to "obj" or a common base-type of the union.
• Mid term: We are a compiler so we could do code generation and emit accessors similar to how we would write them manually by hand. But we need to think about compatibility for this.

There's already work in progress by @ncave to bring back the annotations in the Babel AST #1615. Babel can now parse Typescript, but I don't remember if it works the other way around (annotations in Babel AST used to emit Typescript).

I did some research with google but I cannot figure out what the benefit of this is. If babel cannot write .d.ts files out of this what is it good for? Maybe you or @ncave can explain?

About importing F# files into Typescript, I've noticed the problem is Typescript doesn't let you declare a non-js module with a relative path. There's a note about this now in the docs.

Yes I think we need a bit of fiddling around to make that import work flawlessly but in worst case scenario we can also generate a .ts file which imports the declaration and proper .fs import and then we can tell people to import .ts instead of .fs?

[MangelMaxime]

but I don't remember if it works the other way around (annotations in Babel AST used to emit Typescript)

If I remember correctly, Babel only supports compiling TypeScript into JavaScript. It doesn't generate TypeScript.

At first Fable Conf, at least it was explained that they wanted to support TypeScropt -> JavaScript transpilation and do a "better job" than TypeScript compiler because of all the Babel ecosystem, configuration and optimisation.

[alfonsogarciacaro]

In the first attempts to generate .d.ts files I used babel-dts-generator that was capable of extracting the annotations from Babel AST to create the declarations. But it seems the plugin is not supported now :/ Another option would be to output the annotations as JSDoc comments, as Typescript can also use them to provide intellisense.

If I'm not mistaken, the purpose of @ncave was to make Babel output the type annotations as comments, then do another pass to remove them and finally see if the result was compatible with one these Typescripts subsets targeting WebAssembly like AssemblyScript. Yes, black magic as usual 😉 But if in both cases the purpose is to include type annotations in the output it will be a chance to take two birds with one stone.

[matthid]

Another option would be to output the annotations as JSDoc comments,

I have seen that suggestion, but I doubt it would we easier than writing TypeScript definitions via API. On the other hand, if we get them 'for free' from babel that would be another story and probably the easier approach.

Question is how this fits libraries? I assume we would suggest to include comments in the bundle?

[ncave]

@alfonsogarciacaro

• We can perhaps merge Adding Babel types #1615 as is, it's behind a compiler option so it's non-intrusive. It can be nice to let people play with it on the REPL, if the option is exposed in the UI.
• Adding Babel types #1615 hasn't changed much in the last year (just rebased), but it already gets you pretty far in outputting Type Annotations (and more can be added easily).
• The first step after that would be to make sure the fable-library compiles to something the TypeScript compiler can accept.
• One of the first type annotations that needs fixing is the function parameters, need to be uncurried to match the uncurrying at the call site.

Side Note:

• I was planning to use that PR to generate types for compiling with AssemblyScript. Unfortunately that project hasn't progressed as much as I hoped in the last year. It got reference counting GC, but still lacks support for some basic features like closures and iterators (perhaps because webassembly itself is slow in adding the required features to support that out of the box).

I hope this changes in the future, but in the mean time targetting TypeScript should be much easier.

[matthid]

The first step after that would be to make sure the fable-library compiles to something the TypeScript compiler can accept.

I hope this changes in the future, but in the meantime targetting TypeScript should be much easier.

Not sure what these mean. Does this mean the way forward is to forward the JSDoc-formatted output into the TypeScript compiler, which will itself write the .d.ts files? Or do you mean writing .d.ts files from fable itself? Can you please clarify @ncave ?

Or put differently: I'm open to invest some time into this problem, however after all these discussions I'm still not sure what our best bet is at this time?

[ncave]

@MattiD Adding type annotations in the Babel AST generates types in the Babel output directly. It's just a matter of completeness (output the correct type annotations for all corner cases). For example:

let rec factorial n =
    if n = 0 then 1
    else n * factorial (n-1)

let iterate action (array: 'T[]) =
    for i = 0 to array.Length - 1 do
        action array.[i]

let rec sum xs =
    match xs with
    | []    -> 0
    | y::ys -> y + sum ys

compiles to (with type annotations turned on):

export function factorial(n: number): number {
  if (n === 0) {
    return 1;
  } else {
    return n * factorial(n - 1) | 0;
  }
}
export function iterate<T>(action: (arg0: T) => void, array: Array<T>): void {
  for (let i = 0; i <= array.length - 1; i++) {
    action(array[i]);
  }
}
export function sum(xs: any): number {
  if (xs.tail != null) {
    const ys = xs.tail;
    const y = xs.head | 0;
    return y + sum(ys) | 0;
  } else {
    return 0;
  }
}

As you can see, it's not perfect and there is some work to do (quite a few types are just stubbed as any for now), but IMO you can get quite far with that.

[matthid]

@ncave And I guess TypeScript can consume annotated JavaScript ootb? Or do we need to process this further?

[ncave]

@matthid The above is valid TypeScript produced by Babel, just rename .js to .ts.
But as I said, the PR needs some love to add the proper types, it's hasn't really changed much since @alfonsogarciacaro first added type annotations long, long time ago in Fable 1.

[matthid]

Will take a closer look at the weekend, thanks for clarifying

[alfonsogarciacaro]

I'll soon create a next branch for the next major release to merge #1839. We could also use it to merge #1615 and experiment with it :)