feat: Shell API autocomplete type definitions MONGOSH-2031 MONGOSH-2032 MONGOSH-2173 #2434

lerouxb · 2025-04-17T14:58:03Z

This is a continuation of Anna's work here.

Usage:

npm run api-generate

Implements:

Changes:

merged in main, resolving the conflicts
added generics to the rs, sh and sp objects
added a separate api.ts entrypoint, generating api-processed.d.ts (although this needs work - trying to figure out how to reference it in package.json)
bundled more deps so that we don't import those types in the api-processed.d.ts file. The node.js packages are still bing imported, though. Will have to test if that works with the language server
fixed the tests (at least the ones I've noticed were broken so far)
added an api-globals.d.ts which declares the ShellAPI methods and the ShellBson functions as globals. This then gets appended to the end of api-processed.d.ts.

Notes:

We'll have to do https://jira.mongodb.org/browse/MONGOSH-2170 as a follow-up once feat(autocomplete): add type generator from the php yaml definitions devtools-shared#521 is in

```sh cd packages/shell-api && \ npm run compile && \ npx api-extractor run ; \ npx ts-node bin/api-postprocess.ts ; \ cat lib/api-processed.d.ts ```

lerouxb · 2025-04-17T15:05:46Z

packages/autocomplete/src/index.spec.ts

@@ -294,7 +294,7 @@ describe('completer.completer', function () {

    it('returns all suggestions', async function () {
      const i = 'db.';
-      const attr = shellSignatures.Database.attributes as any;
+      const attr = shellSignatures.DatabaseImpl.attributes as any;


The fact that Database and Collection are now types and the classes got replaced with DatabaseImpl and CollectionImpl is having a lot of knock-on effects and kinda ended up being responsible for most of this diff.

I'm wondering if we should keep them as Database and Collection but then rather name the new types to be something like DatabaseWithSchema and CollectionWithSchema. Then maybe the change will be smaller?

Not sure which way around would work out better ergonomically overall.

lerouxb · 2025-04-17T15:08:12Z

packages/shell-api/package.json

+      "types": "./lib/index.d.ts"
+    },
+    "./api": {
+      "types": "./lib/api-processed.d.ts"


I'm not sure what would be the best way to export/publish this and then reference it from the outside.

I think import type ShellApi from '@monogdb-js/shell-api/api' works with this and I tried it manually from a file, but I'll see shortly when I plug this into mongodb-ts-autocomplete

In the end I have to fs.readFile() to get the file as text so we can bundle it as a giant string, so this is kinda moot.

In the end I have to fs.readFile() to get the file as text so we can bundle it as a giant string, so this is kinda moot.

Yeah, exactly. Do we do this as part of this PR? Put it in a .ts file that only exports that giant string?

Right now I generate that giant string in mongodb-ts-autocomplete (see npm run extract-types in mongodb-js/devtools-shared#520), but I suppose we can also export it here. I don't know which way is best. I don't want to accidentally bundle a giant string..

packages/shell-api/src/api.ts

lerouxb · 2025-04-17T15:09:01Z

packages/shell-api/src/collection.spec.ts

    it('calls help function', async function () {
      expect((await toShellResult(apiClass.help())).type).to.equal('Help');
      expect((await toShellResult(apiClass.help)).type).to.equal('Help');
    });
  });
  describe('signatures', function () {
    it('type', function () {
-      expect(signatures.Collection.type).to.equal('Collection');
+      // TODO: do we want the signatures to be CollectionImpl or Collection?


Just a reminder for myself to remove this TODO comment before merging and after bringing it up with the team.

packages/shell-api/src/api.ts

…tside

lerouxb · 2025-04-22T10:47:25Z

packages/build/src/packaging/package/helpers.ts

-    'Running "' + joinedCommand + '" in ' + args[2]?.cwd ?? process.cwd()
+    // eslint-disable-next-line @typescript-eslint/ban-ts-comment
+    // @ts-ignore TS2869 Right operand of ?? is unreachable because the left operand is never nullish.
+    `Running "${joinedCommand}" in ${args[2]?.cwd ?? process.cwd()}`


drive-by. For some reason I got a compile error on this line in CI.

lerouxb · 2025-04-23T15:01:08Z

packages/shell-api/api-extractor.json

+    "@mongosh/service-provider-core",
+    "@mongosh/types",
+    "mongodb",
+    "bson"


We have bson set up in mongodb-ts-autocomplete, so I might remove it from here. We'd then just have to find/replace the line where we import from 'bson' to import from '/bson.ts' so that it will be loaded from the typescript service at runtime. (I already got that working for my stubbed mql and shell api implementations, so we can just reuse it.)

lerouxb · 2025-04-24T17:06:21Z

packages/shell-api/src/api-globals.d.ts

@@ -0,0 +1,44 @@
+declare global {


This file is basically a snippet that gets appended at the end of api-globals.d.ts after we api-extract that. See bin/api-postprocess.ts.

I first tried to write a babel plugin to walk the ShellAPI class's methods and the ShellBson interface's methods, but that turned out to be really complicated.

There is one benefit to having them all mapped here explicitly: We don't end up accidentally leaking everything on ShellAPI into the global scope whether we want to or not. And it is arguably handy seeing them all enumerated here.

The only other variables added as globals would be db, sh, sp and rs which all get specified dynamically by mongodb-ts-autocomplete so they reference the server/database/collection schema generics.

lerouxb · 2025-04-28T09:05:00Z

packages/shell-api/bin/api-postprocess.ts

+    browserslistConfigFile: false,
+    compact: false,
+    sourceType: 'module',
+    plugins: [applyAsyncRewriterChanges()],


We end up with these errors:

~/mongo/mongosh/packages/shell-api % npx tsc --lib es2019 --target es2018 --moduleResolution node --types node lib/api-processed.d.ts lib/api-processed.d.ts:4412:3 - error TS2416: Property 'hasNext' in type 'Cursor' is not assignable to the same property in base type 'AggregateOrFindCursor<ServiceProviderFindCursor<Document_2>>'. Type '() => boolean' is not assignable to type '() => Promise<boolean>'. Type 'boolean' is not assignable to type 'Promise<boolean>'. 4412 hasNext(): boolean; ~~~~~~~ lib/api-processed.d.ts:4418:3 - error TS2416: Property 'next' in type 'Cursor' is not assignable to the same property in base type 'AggregateOrFindCursor<ServiceProviderFindCursor<Document_2>>'. Type '() => Document_2' is not assignable to type '() => Promise<Document_2>'. Type 'Document_2' is missing the following properties from type 'Promise<Document_2>': then, catch, finally, [Symbol.toStringTag] 4418 next(): Document_2 | null; ~~~~ Found 2 errors in the same file, starting at: lib/api-processed.d.ts:4412

Weirdly these seem to be the only two cases - if I revert those two return types manually then the file is all green.

The ts language service works around it as far as I can tell, so it doesn't impact autocomplete. If we did want to make the generated .ts valid we'll have to do something like modify the base type's methods' return types or somehow flatten the return type so that it doesn't inherit anymore?

Maybe there are other ideas.

Wild idea: we can just add the comment

// @ts-expect-error The type of this function is not compatible with the type of the function in the base class

as part of the babel processing...

The reason I ultimately care about this is because it would be nice to have some smoke tests where we import this file and use it.

If this is still an issue, would it be possible to adjust the base methods as well? That feels like the cleanest (and most correct) approach

I filed https://jira.mongodb.org/browse/MONGOSH-2209

Had a quick look and decided to split that into its own ticket rather.

lerouxb · 2025-05-01T13:58:59Z

packages/shell-api/bin/api-postprocess.ts

+
+function getHelpClassName(className: string) {
+  switch (className) {
+    case 'CollectionImpl':


I pointed this out below as well, but this is an example of the kind of fallout from renaming Collection to CollectionImpl and reusing the name for a type. And doing the same for Database.

Maybe this is OK but we could also consider leaving Collection and Database unchanged and then just having the types named something else like CollectionWithSchema and DatabaseWithSchema. Or whatever might be a better name.

packages/shell-api/bin/api-postprocess.ts

addaleax · 2025-05-08T11:34:40Z

packages/shell-api/bin/api-postprocess.ts

+    browserslistConfigFile: false,
+    compact: false,
+    sourceType: 'module',
+    plugins: [applyAsyncRewriterChanges()],


If this is still an issue, would it be possible to adjust the base methods as well? That feels like the cleanest (and most correct) approach

addaleax · 2025-05-08T11:35:38Z

packages/shell-api/package.json

+      "types": "./lib/index.d.ts"
+    },
+    "./api": {
+      "types": "./lib/api-processed.d.ts"


In the end I have to fs.readFile() to get the file as text so we can bundle it as a giant string, so this is kinda moot.

Yeah, exactly. Do we do this as part of this PR? Put it in a .ts file that only exports that giant string?

packages/shell-api/src/api.ts

addaleax and others added 9 commits January 24, 2024 17:59

[Do Not Merge] PoC for shell-api autocomplete type definitions

cfbafad

```sh cd packages/shell-api && \ npm run compile && \ npx api-extractor run ; \ npx ts-node bin/api-postprocess.ts ; \ cat lib/api-processed.d.ts ```

fixup: add index signature support

0e17bc2

fixup

4d72278

more bundled packages

256fc60

Merge branch 'main' into shell-api-ac-type-definitions

d81ff54

add rs, sh, sp generics

198fb06

fix tests

59c04f3

hopefully fix the api export

3545fd8

something went wrong with the package-lock.json file

f34e213

lerouxb commented Apr 17, 2025

View reviewed changes

packages/shell-api/src/api.ts Outdated Show resolved Hide resolved

lerouxb commented Apr 17, 2025

View reviewed changes

packages/shell-api/src/api.ts Outdated Show resolved Hide resolved

lerouxb added 2 commits April 22, 2025 11:36

don't include the REPLACEME boilerplate which will be added on the ou…

51946ea

…tside

workaround possible false positive error

41a4560

lerouxb commented Apr 22, 2025

View reviewed changes

lerouxb added 2 commits April 22, 2025 11:49

no idea why it added that

33ee0b5

we should probably run api-generate automatically

ae48313

This was referenced Apr 22, 2025

feat: add mongodb-ts-autocomplete package MONGOSH-2034 mongodb-js/devtools-shared#520

Open

feat: initial feature-flagged, draft new autocompleter MONGOSH-2036 #2424

Draft

add missing deps

880135f

lerouxb marked this pull request as ready for review April 22, 2025 12:54

lerouxb commented Apr 23, 2025

View reviewed changes

add the ShellApi and ShellBson globals to the shell api

25240df

lerouxb changed the title ~~feat: Shell API autocomplete type definitions MONGOSH-2032~~ feat: Shell API autocomplete type definitions MONGOSH-2032 MONGOSH-2173 Apr 24, 2025

lerouxb changed the title ~~feat: Shell API autocomplete type definitions MONGOSH-2032 MONGOSH-2173~~ feat: Shell API autocomplete type definitions MONGOSH-2031 MONGOSH-2032 MONGOSH-2173 Apr 24, 2025

remove comment

5511582

lerouxb commented Apr 24, 2025

View reviewed changes

linebreak

c6d8219

lerouxb commented Apr 28, 2025

View reviewed changes

add help comments to the extracted/generated shell-api

8f11992

lerouxb commented May 1, 2025

View reviewed changes

lerouxb mentioned this pull request May 2, 2025

[WIP DO NOT MERGE] Autocomplete2 demo #2445

Closed

addaleax reviewed May 8, 2025

View reviewed changes

lerouxb added 3 commits May 8, 2025 14:04

strip down api.ts

4ad7978

remove commented code

adaf032

no idea

109ce54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Shell API autocomplete type definitions MONGOSH-2031 MONGOSH-2032 MONGOSH-2173 #2434

feat: Shell API autocomplete type definitions MONGOSH-2031 MONGOSH-2032 MONGOSH-2173 #2434

lerouxb commented Apr 17, 2025 •

edited

Loading

lerouxb Apr 17, 2025

lerouxb Apr 17, 2025

lerouxb May 4, 2025

addaleax May 8, 2025

lerouxb May 8, 2025 •

edited

Loading

lerouxb Apr 17, 2025

lerouxb Apr 22, 2025

lerouxb Apr 23, 2025 •

edited

Loading

lerouxb Apr 24, 2025

lerouxb Apr 28, 2025

lerouxb Apr 28, 2025

lerouxb Apr 28, 2025

addaleax May 8, 2025

lerouxb May 8, 2025

lerouxb May 1, 2025 •

edited

Loading

addaleax May 8, 2025

addaleax May 8, 2025

feat: Shell API autocomplete type definitions MONGOSH-2031 MONGOSH-2032 MONGOSH-2173 #2434

Are you sure you want to change the base?

feat: Shell API autocomplete type definitions MONGOSH-2031 MONGOSH-2032 MONGOSH-2173 #2434

Conversation

lerouxb commented Apr 17, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lerouxb May 8, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lerouxb Apr 23, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lerouxb May 1, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lerouxb commented Apr 17, 2025 •

edited

Loading

lerouxb May 8, 2025 •

edited

Loading

lerouxb Apr 23, 2025 •

edited

Loading

lerouxb May 1, 2025 •

edited

Loading