Create the API generator tool

[MangelMaxime]

Here is what API documentation could looks like:

For DU:

For record:

[MangelMaxime]

It seems like I should be able to generate a really rough API documentation generator.

I am able to generate this for all the records in Nacare.Core/Type.fs

This is generated from the typed AST provided by FCS which means that I can also have access to TypeInference to documentation function, methods, etc. in the future.

The first step is to be able to generate the documentation as shown above for the DUs and Record coming from Nacara.Core/Type.fs. This will serve as a POC and also will allow me to release Nacara in v1 with a the core API documented.

[MangelMaxime]

• Add the attribute list
• Add member support for the records and DUs
• Add support for function alias like:
  type LayoutRenderFunc = RendererContext -> PageContext -> JS.Promise<ReactElement>

[TheAngryByrd]

Hey 👋

I ended up taking the tool for a spin and here are my thoughts:

Tutorial

I know, I know, it's a beta and such, but can always be better. I think it would be good to include a tutorial from install to setup/integration with a new nacara site. I found myself hitting edge cases with how menu.json/nacara.js.config, the base-url of the tool interact, and where to exactly output the files.

API tool needs to know each project

While it makes sense that you need an entry point to start generating docs from, if you have multiple libraries, it might be nicer to be able to pass in a glob or a list and let the tool generate from them all.

API tool should have a watch mode (and maybe handle the publish step for you)

Going from XML comment -> to website involves a change to docs, a publish of that library, then a run of the api tool. It would be a smoother experience if the tool handled taking in the projects it needs to rebuild, publishing them on changes, and generating the markdown files from those changes. See here for an example of that flow. If nacara doesn't want to run dotnet publish, it should at least watch a directory where the dlls/xml get dropped.

Api Tool assumes third party dependencies to be on microsoft docs referencable.

For example: https://docs.microsoft.com/en-gb/dotnet/api/newtonsoft.json.linq.jtoken. This is probably impossible to figure out where the actual docs live for any given third party library but it might be worth either checking for 404s of the site or possibly having a lookup for known packages to known good sites.

Certain tags in the XML aren't displayed

• example
• exception
• see

Aren't rendered on the webpage.

API Generation doesn't integrate nicely with the side-nav-bar

While regular documentation has a nice sidebar experience, the API does not. Fake does have a sidebar experience and would be nice to replicate it here since we already have this UX pattern.

Knowing about certain attributes

It would be nice to know about RequireQualifiedAccess and Obsolete (possibly others) and render these somehow on the page so you know about them.

References

• Repo I was using to test with

[MangelMaxime]

Hey wave

I ended up taking the tool for a spin and here are my thoughts:

Hello thank you for taking the time of looking into Nacara API gen even if it is in really rough state.

Tutorial

I know, I know, it's a beta and such, but can always be better. I think it would be good to include a tutorial from install to setup/integration with a new nacara site. I found myself hitting edge cases with how menu.json/nacara.js.config, the base-url of the tool interact, and where to exactly output the files.

Hum, depending on what the edge cases about menu.json or nacara.config.js are they should be covered in the documentation.

About the generated structure of the files indeed this is something to improve. As I mentioned the API gen is really in early stage, I published it only because I needed to generate the Nacara API instead of writing it manually.

If you think something is unclear in the Nacara documentation don't hesitate to tell me or send a PR.

API tool needs to know each project

While it makes sense that you need an entry point to start generating docs from, if you have multiple libraries, it might be nicer to be able to pass in a glob or a list and let the tool generate from them all.

This is complex to do because F# Formatting doesn't provide the real raw XML comment... That's why for now, I really on dotnet publish to have access to the dlls and the raw xml file so I can parse it.

I need to experiment more to see if it is possible to start a single process to watch all the libraries or if I have to spawn one instance per library.

API tool should have a watch mode (and maybe handle the publish step for you)

Going from XML comment -> to website involves a change to docs, a publish of that library, then a run of the api tool. It would be a smoother experience if the tool handled taking in the projects it needs to rebuild, publishing them on changes, and generating the markdown files from those changes. See here for an example of that flow. If nacara doesn't want to run dotnet publish, it should at least watch a directory where the dlls/xml get dropped.

I agree that it should have a watch mode, it should be improved in future version with a better integration into Nacara config file too.

I would love to be able to kind of generate the documentation of all the APIs of the project at once even for the dependencies to avoid invalid links etc.

But for that, I need to find a way to get the raw comments. Either by finding the XML files or by forking/improving F# formatting to give me access to them.

Api Tool assumes third party dependencies to be on microsoft docs referencable.

For example: https://docs.microsoft.com/en-gb/dotnet/api/newtonsoft.json.linq.jtoken. This is probably impossible to figure out where the actual docs live for any given third party library but it might be worth either checking for 404s of the site or possibly having a lookup for known packages to known good sites.

I was going to write that this a limitation of F# formatting which is true because that part of the documentation if generated by it right now.

But, while re-reading my comment: fsprojects/FSharp.Formatting#708 (comment)

It seems like I had found a solution to get around it. I will have a look at it.

Certain tags in the XML aren't displayed

* [example](https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/xmldoc/recommended-tags#example)

* [exception](https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/xmldoc/recommended-tags#exception)

* [see](https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/xmldoc/recommended-tags#see)

Aren't rendered on the webpage.

The example tag needs to be between remark or summary tag and perhaps the returns too.

This is because, I based myself on the C# XML recommendation examples.

The see tag has the safe limitation currently, even if the C# XML recommendation is strange. This is the only tag documented as This tag allows a link to be specified within text. where it is not inside another tag element in their example...

The current parser force the order of the elements like that

1. Summary
2. Remarks (not done yet)
3. Parameters list
4. Returns
5. Exception (not done yet)
6. TypesParams (not done yet) + resolved generics types
7. See also (not done yet)

The difficulty, is when something is not put inside of these top levels tags as recommanded by the C# XML "specification" right now it get skipped. One way, to solve it could be to make a second pass which removed all the the tags I listed above and if it found something, it happen it to the remarks section. This can still cause strange result if the user write his doc comments "poorly" but at least it will not strip content.

API Generation doesn't integrate nicely with the side-nav-bar

While regular documentation has a nice sidebar experience, the API does not. Fake does have a sidebar experience and would be nice to replicate it here since we already have this UX pattern.

I do agree, this is something that will be hard to do correctly.

The menu is kind of strongly related to which renderer is used for Nacara. For example,
if I have a different style for the menu rendered than on the others pages it will looks strange.

Also in case, you missed it there are some anchor generated on the API page

Knowing about certain attributes

It would be nice to know about RequireQualifiedAccess and Obsolete (possibly others) and render these somehow on the page so you know about them.

I agree with you, this is something I left out of the initial really but I need to add it.

References

* [Repo I was using to test with](https://github.com/TheAngryByrd/nacara-tests)

[MangelMaxime]

API Generation doesn't integrate nicely with the side-nav-bar

While regular documentation has a nice sidebar experience, the API does not. Fake does have a sidebar experience and would be nice to replicate it here since we already have this UX pattern.

I do agree, this is something that will be hard to do correctly.

The menu is kind of strongly related to which renderer is used for Nacara. For example, if I have a different style for the menu rendered than on the others pages it will looks strange.

Also in case, you missed it there are some anchor generated on the API page

Actually, the menu are auto generated by Nacara layout so in theory, if I generate h2 elements it should be possible to hook the menu "generator" to it.

[dsyme]

Hey, I'd love to help ensure that the ApiDocsModel coming out of FSharp.Formatting is sufficient - I know there are issues like fsprojects/FSharp.Formatting#708 and I'd like to understand what you need.

The visuals of your API Docs are also great and maybe it's feasible to fold that back into FSharp.Formatting too once mature, as an optional replacement for GenerateHtml.fs

[MangelMaxime]

Hello,

thank you for looking at the work done with Nacara :)

I am still experimenting so I am unsure if my list will be exhaustive but what I need right now is:

1. Access the to Raw XML comments: How can we access the Raw doc comments? fsprojects/FSharp.Formatting#710

   This is needed to apply the comment formatter included in Nacara which is an improvement over the one from Ionide

   Right now, I am hacking my way into the XML file and using regex to try find the XML comments.

2. Access to the type information to generate the code section at the top of the files.

This code, is not coming from the F# source but reconstructed from the type information coming from FCS from objects like FSharpType or Symbol etc. This allow me to generates links between the types and also apply syntax highlighting.

The code is not pretty but I think this can gives you an idea of how I am generating the snippets.

Nacara/src/Nacara.ApiGen/Source/Render.fs

Lines 223 to 370 in 33dc6c6

	let rec renderParameterType (isTopLevel : bool) (typ : FSharpType) : TextNode =
	// This correspond to a generic paramter like: 'T
	if typ.IsGenericParameter then
	TextNode.Node [
	TextNode.Tick
	TextNode.Text typ.GenericParameter.DisplayName
	]
	// Not a generic type we can display it as it is
	// Example:
	// - string
	// - int
	// - MyObject
	else if typ.GenericArguments.Count = 0 then
	TextNode.Text typ.TypeDefinition.DisplayName
	// This is a generic type we need more logic
	else
	// This is a function, we need to generate something like:
	// - 'T -> string
	// - 'T -> 'T option
	if typ.IsFunctionType then
	let separator =
	TextNode.Node [
	TextNode.Space
	TextNode.Arrow
	TextNode.Space
	]
	let result =
	[
	for index in 0 .. typ.GenericArguments.Count - 1 do
	let arg = typ.GenericArguments.[index]
	// Add the separator if this is not the first argument
	if index <> 0 then
	separator
	// This correspond to a generic paramter like: 'T
	if arg.IsGenericParameter then
	TextNode.Tick
	TextNode.Text arg.GenericParameter.DisplayName
	// This is a type definition like: 'T option or Choice<'T1, 'T2>
	else if arg.HasTypeDefinition then
	// For some generic types definition we don't add the generic arguments
	if arg.TypeDefinition.DisplayName = "exn"
	|| arg.TypeDefinition.DisplayName = "unit" then
	TextNode.Text arg.TypeDefinition.DisplayName
	else
	// This is the name of the type definition
	// In Choice<'T1, 'T2> this correspond to Choice
	TextNode.Text arg.TypeDefinition.DisplayName
	TextNode.LessThan
	// Render the generic parameters list in the form of 'T1, 'T2
	renderGenericParameters arg.TypeDefinition.GenericParameters
	TextNode.GreaterThan
	else if arg.IsFunctionType then
	let res =
	[
	for index in 0 .. arg.GenericArguments.Count - 1 do
	let arg = arg.GenericArguments.[index]
	if index <> 0 then
	TextNode.Space
	TextNode.Arrow
	TextNode.Space
	renderParameterType false arg
	]
	// Try to detect curried case
	// Like in:
	// let create (f: ('T -> unit) -> (exn -> unit) -> unit): JS.Promise<'T> = jsNative
	// FCS gives back an equivalent of :
	// let create (f: ('T -> unit) -> ((exn -> unit) -> unit)): JS.Promise<'T> = jsNative
	// So we try to detect it to avoid the extract Parents
	match res with
	| (TextNode.Node (TextNode.LeftParent :: _ ) :: _ ) ->
	TextNode.Node res
	| _ ->
	TextNode.Node [
	TextNode.LeftParent
	yield! res
	TextNode.RightParent
	]
	else
	let i = 0
	TextNode.Text "Unkown syntax please open an issue"
	]
	// If this is a top level function we don't neeed to add the parenthesis
	TextNode.Node [
	if not isTopLevel then
	TextNode.LeftParent
	TextNode.Node result
	if not isTopLevel then
	TextNode.RightParent
	]
	else
	let separator =
	TextNode.Node [
	TextNode.Space
	TextNode.Comma
	]
	let result =
	[
	for index in 0 .. typ.GenericArguments.Count - 1 do
	let arg = typ.GenericArguments.[index]
	// Add the separator if this is not the first argument
	if index <> 0 then
	separator
	if arg.IsGenericParameter then
	TextNode.Tick
	TextNode.Text arg.GenericParameter.DisplayName
	else
	let url =
	arg.TypeDefinition.FullName
	|> String.toLower
	|> String.replace "." "-"
	|> String.append ".html"
	let subType =
	renderParameterType false arg
	TextNode.Anchor (url, arg.TypeDefinition.DisplayName)
	TextNode.LessThan
	subType
	TextNode.GreaterThan
	]
	TextNode.Node result

3. Bonus: Have F# formatting works "out of the box"

What I mean by that, is that currently I need to run dotnet publish and then run F# Formatting against the generated folder. This is the easiest way I found to have the all DLLs information.

I understand that doing the DLLs/dependencies resolution is a complex thing but the IDEs are already doing it. So it would be nice if F# formatting could do the same.

Doing dotnet publish right now also have the benefit of providing me the XML files because of this issue fsprojects/FSharp.Formatting#710

---

The visuals of your API Docs are also great and maybe it's feasible to fold that back into FSharp.Formatting too once mature, as an optional replacement for GenerateHtml.fs

Thank you :)

It would be nice indeed, it would be nice especially to benefit from features like watch mode instead of doing a full compilation of the library at each changes which takes some times to do.

The only thing is that Nacara needs to have a Markdown file to process with header at the top.

Example:

---
layout: api
---

• The header is need for Nacara to know which layout to use for the HTML generation.
• The markdown file is needed in order to have syntax highlight for the code snippets in the doc comments

When I have a stable layout and customisation system for the API layout. I would be happy to ping you to discuss about adding it back in F# Formatting if that still makes sense.

PS: I know that you are working on fixing some of the issues I reported at the same time I write this message.