RFC: Syntax for "declaration references" (hyperlinks)

[octogonz]

Certain tags such as {@inheritdoc} contain references to other API items. AEDoc supports two notations for API item references:

• For internal references, we use: exportName.memberName

  Example: {@inheritdoc BaseWidget.draw}

  The ".memberName" is optional and included e.g. if the API item is a member of a class or interface.

• For external references, we use: @scopeName/packageName:exportName.memberName

  Example: {@inheritdoc @microsoft/widget-lib:BaseWidget.draw}

  The "@scopeName/" and ".memberName" are optional. The "@scopeName/" is used for scoped NPM packages.

This might be a reasonable starting point. However, it doesn't consider nesting members such as namespaces (which has turned out to be surprisingly common for legacy SDKs, despite all the downsides and attempts to deprecate them).

Also, AEDoc today overlooks an interesting edge case:

export class A {
    public b: string = 'hello';
}

export namespace A {
    export let b: number = 123;
}

let a = new A();

// a.b is a string
// A.b is a number

In this example, an expression such as {@link A.b} would be ambiguous. Which declaration is it referring to? How to handle this?

[DovydasNavickas]

Me and @MartynasZilinskas have a solution to this. We'll post more info a bit later today / tomorrow. We also have it working in ts-extractor's rewritten version according to what we talked about privately with you (ids, aforementioned ambiguity for same name, different types and not requiring JSON output). We also seem to have solved "declarations vs symbols" thingy and already extract both and also deal with them ids-wise.

P.S. You called hyperlinks what we call ids, but I'd rather we call them ids or identifiers, because we can use them in various situations, not only hyperlinks and they have to be unique and point to something specific, which is identifier.

[aciccarello]

How should internal references be looked up? TypeDoc isn't limited to a single API export so you could have multiple BaseWidgets that could match. Imports could give you a hint but I don't think we want the documentation parser to have to read those and it wouldn't work in all cases.

[octogonz]

@DovydasNavickas I've retitled this from "hyperlinks" to "API item references." This clarifies that we're not just talking about clickable hyperlinks. The name "ids" seemed a little too vague.

How should internal references be looked up? TypeDoc isn't limited to a single API export
so you could have multiple BaseWidgets that could match.

Option 1: Require the names to be exported

Today API Extractor only allows references to exported items, using the official exported name. Today it doesn't support multiple entry points, but it would be fairly straightforward.

For example, if an API consumer has to write this:

import { Button } from '@scope/my-library/ui/controls';

Then the {@link} tag for an external consumer could look like this:

/**
 * {@link @scope/my-library/ui/controls:Button.render | the render() method}
 */

So for a local reference inside of @scope/my-library, we might expect it to disambiguate the entry point like this:

/**
 * {@link /ui/controls:Button.render | the render() method}
 */

Note that the TSDoc layer would not be directly involved in this. The @microsoft/tsdoc library would simply provide a standard notation and parser, which might return an AST node like this:

// {@link @scope/my-library/ui/controls:Button.render | the render() method}

{
  "nodeKind": "linkNode",
  "apiItemReference": {
    "scope": "@scope",
    "library": "my-library",
    "modulePath": "/ui/controls",
    "itemPath": "Button.render"
  },
  "linkText": {
    "nodeKind": "textNode",
    "content": "the render() method"
  }
}

Option 2: Rely on the compiler symbols

Another option would be to rely on the compiler symbols. This is fairly straightforward with today's compiler API, but only if the symbol is actually imported. Example:

import { Button as RenamedForSomeReason } from '../../ui/controls';

/**
 * {@link RenamedForSomeReason.render | the render() method}
 */

[MartynasZilinskas]

// index.ts file in `example-package`

// example-package/index:Foo#class
class Foo {
    // example-package/index:Foo#class.a#property
    public a: string;
}

// example-package/index:Foo#namespace
namespace Foo {
    // example-package/index:Foo#class.a#variable
    const a: number;
}

We ugpraded the format for specifying api item.
{package-name}/{path-to-file}:{selector}#{kind}

I am thinking of adding an additional ApiItem symbol. It will respresent all declarations behind that symbol.
For example:

"example-package/index:Foo": {
    "members": [
        "example-package/index:Foo#class", 
        "example-package/index:Foo#namespace"
    ]
}

When resolving id without specific kind part and that symbol has only single declaration, than Symbol ApiItem will act as alias to that single declaration.

// `example-package` at index.ts
namespace Foo {
    const a: number;
}

class Foo {
    /**
     * Property `a`.
     */
    public a: string;
}

// `other-package`
class Bar extends Bar{
    /**
     * {@inheritdoc example-package/index:Foo#class.a}
     */
    public a: string;
}

Specifying kind for Foo is mandatory because we have symbol that has two declarations.

The only problem I see is overloads, because they have same name and kinds.

P.S. Working delimiters !@#%^&*)+?;,..

[octogonz]

I like this idea of using # to disambiguate merged declarations. I was originally considering proposing something like :: vs . from C++ to distinguish static versus non-static members. But it would still leave open the question of how to refer to Foo as a namespace versus Foo as a class, an API reference site would probably want separate manual pages for these manifestations.

But I don't think you would ever need the #variable part. The a is unambiguous in example-package/index:Foo#class.a.

Property getters/setters are another example of multiple declarations for the same symbol, but I feel fairly strongly they should be treated as a single API item.

Overloads are an interesting case. C# /// comments would use a notation like {@link Widget.doSomething(string, string)} but that requires a full type system to resolve, and could be very verbose in TypeScript.

However, TypeScript requires all of the overloaded signatures to be right next to each other. They cannot be merged from different files. Also the ordering matters (since it determines matching precedence). So for this case, perhaps we could simply number them?

class Widget {

  // This is {@link Widget.doSomething#1}
  doSomething(name: string, age: number): void;

  // This is {@link Widget.doSomething#2}
  doSomething(options: { name: string, age: number }): void;

  // This is {@link Widget.doSomething#3}
  doSomething(): void;

  // This is the private implementation
  doSomething(whatever?: any, age?: number): void {
  }

}

It's not super elegant... but then neither are overloads. We don't use overloads all that much since the reflection check has a perf overhead, and the behavior gets weird with subclassing and such. It is a half-baked feature compared to languages such as Java or C++. API Extractor doesn't even give them separate manual pages; we treat overloads as a single function with a complicated type signature.

For people who really care about this, a more elaborate solution might be to provide a way to create named monikers. Something like this:

class Widget {

  /** @overload nameAndAge */
  doSomething(name: string, age: number): void;
  /** @overload options */
  doSomething(options: { name: string, age: number }): void;
  /** @overload none */
  doSomething(): void;

  doSomething(junk?: any, age?: number): void {
  }

}

/** The {@link Widget.doSomething#nameAndAge} is the first overload.

Any better ideas?

[MartynasZilinskas]

I upgraded ts-extractor to use this format of ids for ast items.

{package-name}/{path-to-file}:{selector}#{kind}&{itemCounter}

Code sample:

export function pickCard(x: { suit: string; card: number }[]): number;
export function pickCard(x: number): { suit: string; card: number };
export function pickCard(x: any): number | { suit: string; card: number } | undefined {
    const suits = ["hearts", "spades", "clubs", "diamonds"];
    // Check to see if we're working with an object/array
    // if so, they gave us the deck and we'll pick the card
    if (typeof x == "object") {
        let pickedCard = Math.floor(Math.random() * x.length);
        return pickedCard;
    } else if (typeof x == "number") {
        // Otherwise just let them pick the card
        let pickedSuit = Math.floor(x / 13);
        return { suit: suits[pickedSuit], card: x % 13 };
    }
}

This how ast item ids look like under the hood.

@simplrjs/package-name/index.ts // SourceFile
@simplrjs/package-name/index.ts:pickCard // AstSymbol
@simplrjs/package-name/index.ts:pickCard#function&1 // AstFunction
@simplrjs/package-name/index.ts:pickCard#function&1.x // AstSymbol
@simplrjs/package-name/index.ts:pickCard#function&1.x#parameter&1 // AstParameter
@simplrjs/package-name/index.ts:pickCard#function&2 // AstFunction
@simplrjs/package-name/index.ts:pickCard#function&2.x // AstSymbol
@simplrjs/package-name/index.ts:pickCard#function&2.x#parameter&1 // AstParameter
@simplrjs/package-name/index.ts:pickCard#function&3 // AstFunction
@simplrjs/package-name/index.ts:pickCard#function&3.x // AstSymbol
@simplrjs/package-name/index.ts:pickCard#function&3.x#parameter&1 // AstParameter

We are using a flat Map for items registry, so items ids need to be unique.

But I don't think you would ever need the #variable part. The a is unambiguous in example-package/index:Foo#class.a.

If id referencing to a symbol and it only has 1 declaration, then you don't need to specify #{kind} part.

Simple example

/**
 * Says Hello world!
 */
export const a: string = "Hello World!";

/**
 * {@inheritdoc @simplrjs/package-name/index.ts:a}
 */
export const b: string = "-";

Example

export namespace A.B.C.D.E {
    /**
     * Says Hello world!
     */
    export const a: string = "Hello World!";
}

/**
 * {@inheritdoc @simplrjs/package-name/index.ts:A.B.C.D.E.a}
 */
export const b: string = "-";

However, TypeScript requires all of the overloaded signatures to be right next to each other. They cannot be merged from different files. Also the ordering matters (since it determines matching precedence). So for this case, perhaps we could simply number them?

If you need to inherit documentation from a specific overload, numbering is the way to go.

[DovydasNavickas]

@pgonzal said:

But I don't think you would ever need the #variable part. The a is unambiguous in example-package/index:Foo#class.a.

@MartynasZilinskas said:

If id is referencing a symbol and it only has 1 declaration, then you don't need to specify the #{kind} part.

I think we have situation, where we have two different goals:

• Registry ids have to be unique, therefore they have to be as verbose as they can get.

But:

• API item references should be more loose for convenience and could omit some of the parts, if one is sure that they are optional.

What I'm a bit more careful about is having those optional parts omitted and reference becoming ambiguous later on in the development, when say another declaration is added to the symbol (e.g. inherit doc or a link gets ambiguous).

On the other hand, that might even become a perk.
If the reference becomes ambiguous later on, you might need to update the reference to a more specific one either way, so this could be a good signal that you really need to take a look into that older part and specify what exactly are you referring to.

Thoughts?

[yume-chan]

For the overload part, I strongly disagree using magic numbers to label them. If the performance is a concern, the doc generator may produce extra metadata to help distinguish them (by index, for example) for other tools, but the developer must have the ability to reference overloads by their signatures.

The #{type} syntax looks strange, how about just (class)Foo.(property)a? I think a pair of parentheses may not cause syntactic ambiguities. Or maybe <class>Foo.<property>a like a TypeScript cast operator.

[MartynasZilinskas]

@yume-chan The concern is not with performance, but how verbose can it become, like @pgonzal pointed out. The order of overload signatures matters, so numbering them makes sense.

Consider this example:

class Widget {
    doSomething(name: string, age: number): void; // Widget.doSomething(string, number)
    doSomething(options: { name: string; age: number }): void; // Widget.doSomething({ name: string; age: number })    
    doSomething(): void; // Widget.doSomething()
}

How reference will look like for this overload?

doSomething(options: { name: string; age: number }): void;

---

I see the real problem when you have whole bunch of overloads (example). In this case, labeling would be convenient.

Usage:

@label <name>

Example:

class Widget {
    /**
     * Do something with name and age.
     * @label nameAndAge
     */
    doSomething(name: string, age: number): void; // Widget.doSomething#nameAndAge or Widget.doSomething&1
    doSomething(options: { name: string; age: number }): void; // Widget.doSomething&2
    /**
     * No arguments needed to do something.
     */
    doSomething(): void; // Widget.doSomething&3
}

class CustomWidget {
    /**
     * {@inheritdoc Widget.doSomething#nameAndAge}
     */
    doSomething(name: string, age: number): void;
    /**
     * {@inheritdoc Widget.doSomething&3}
     */
    doSomething(): void {}
}

All possible naming variants so far:

• @label
• @overload-label
• @overload

What do you think about @label @pgonzal? I think it represents better what it means than @overload.

---

The #{type} syntax looks strange, how about just (class)Foo.(property)a? I think a pair of parentheses may not cause syntactic ambiguities. Or maybe <class>Foo.<property>a like a TypeScript cast operator.

The #{kind} you're referring to is not a type. The idea here is to specify which syntax kind it represents when more than one declaration is available under the same symbol name. Plus, in ts-extractor we want to use these references as unique identifiers in AST items registry.

[yume-chan]

How reference will look like for this overload?

I now understand the problem is the complex type system in TypeScript. I came from C# and I have used to using signature to reference overloads.

labeling would be convenient.

I think label is a good idea.

The #{kind} you're referring to is not a type.

I know, I just used the word "type". But class is a function and namespace is an object, they are still "types", aren't them? Anyway, I suggest a new syntax to distinguish different node kinds by a TypeScript style type conversion operator. It's more familiar with TypeScript programmers and because the kind is in the front of name, it's more easily to read and understand.

[DovydasNavickas]

But class is a function and namespace is an object, they are still "types", aren't them?

Well, in TypeScript's realm they are declarations, not types. And their common symbol is what binds them.

I suggest a new syntax to distinguish different node kinds by a TypeScript style type conversion operator.

I think this might be a confusing analogy. Also, if we write (class)Foo and (namespace)Foo, it is in the wrong order semantically, because a symbol Foo might have two declarations: class and namespace.
And therefore this disambiguation should be specified later in the reference:

Foo(class)
Foo(namespace)

But then it looks weird for me as it resembles a function call.
And # is something that specifies things in urls and is more likely to be understood more naturally:

Foo#class
Foo#namespace

And if you define reference for property myProp in a class Foo, it would look:

Foo#class.myProp

Which goes as Foo#class and dot-notation into its members. If we look at Foo#class as a single unit and for the sake of argument name it as FooClass, this reference would become:

FooClass.myProp

And this notation is more than familiar for anyone having done programming.

Also, if we adopt the @label tag, we can forbid reserved keywords usage for labels, such as class and namespace, but let people use other strings without spaces, e.g. magical-overload. Then we could use the same # notation with labels for code:

class Foo {
   public myProp: string;
    /**
     * @label magical-overload
     */
   public myMethod(a: string): void;
   public myMethod(): void;
}

This would be the references:

Foo#class.myProp                    // <-- Property
Foo#class.myMethod#magical-overload // <-- Specific overload with a label `magical-overload`