Ensure refs in tsconfig files are synced with internal deps (#8384)

### Problem

We use project references to tell TypeScript and other tools about the
structure of the monorepo. Both the root- and package-level tsconfig
files have a `references` field which is used to link packages together.
All packages that we want to build and publish need to be present in
`references` within the root `tsconfig.json` and `tsconfig.build.json`
files, and the `references` field in both root-level and package-level
tsconfig files need to list all internal dependencies.

However, it's a pain to keep references up to date, especially as as
packages are added or dependencies are updated. There are cases in the
past, in fact, where packages were not published or type errors occurred
because the references were not correctly kept in sync.

### Solution

We don't need to do this manual work. We can infer root references by
scanning workspaces, and we can infer references for an individual
package by scanning that package's `dependencies` for other workspace
packages.

This commit adds a script that can be run on both the root workspace and
child workspaces to check and/or regenerate the `references` field for
each tsconfig file so that it is kept in sync with dependencies
throughout the monorepo. There are also new package scripts for use:

- Root level
- **`lint:tsconfigs`:** Validates the root `tsconfig.json` and
`tsconfig.build.json`, erroring if any are out of sync.
- **`lint:tsconfigs:all`:** Validates all `tsconfig.json` and
`tsconfig.build.json` across the monorepo, erroring if any are out of
sync.
- **`lint:tsconfigs:fix`:** Regenerates the root `tsconfig.json` and
`tsconfig.build.json`.
- **`lint:tsconfigs:fix:all`:** Regenerates `tsconfig.json` and
`tsconfig.build.json` files across the monorepo.
- Package level
- **`lint:tsconfigs`:** Validates `tsconfig.json` and
`tsconfig.build.json` in the package, erroring if any are out of sync.
- **`lint:tsconfigs:fix`:** Regenerates `tsconfig.json` and
`tsconfig.build.json` in the package.

As the names indicate, tsconfig file validation is also a part of the
lint pipeline, so CI will now fail if any files are out of date. To make
sure this doesn't happen in the future, this commit also corrects
tsconfig files across the board to add missing references or remove
extra references.

### Why hand-roll a solution?

There are some existing solutions for syncing references within the
TypeScript community, some of which are documented in [this issue][1].
However, I reviewed them and rejected them for various reasons:

- **[`typescript-monorepo-toolkit`][2]:** Doesn't update references for
individual packages based on `dependencies`
- **[`update-ts-references`][3]:** Works, but re-sorts all references,
and drops the leading `./` from references in root tsconfig files; also,
generated files fail Prettier validation
-
**[`@monorepo-utils/workspaces-to-typescript-project-references`][4]:**
Works, but re-sorts all references, and drops the leading `./` from
references in root tsconfig files; also generated files fail Prettier
validation

My thought is that by building our own script, we get to control the
exact changes that are made to tsconfig files, and we get to run all
files through Oxfmt so that engineers do not have to reformat them
manually.

[1]: microsoft/TypeScript#25376
[2]: https://github.com/Bnaya/typescript-monorepo-toolkit
[3]: https://github.com/eBayClassifiedsGroup/update-ts-references
[4]:
https://github.com/azu/monorepo-utils/tree/master/packages/@monorepo-utils/workspaces-to-typescript-project-references

## References

<!--
Are there any issues that this pull request is tied to?
Are there other links that reviewers should consult to understand these
changes better?
Are there client or consumer pull requests to adopt any breaking
changes?

For example:

* Fixes #12345
* Related to #67890
-->

Closes #966.

Also closes https://consensyssoftware.atlassian.net/browse/WPC-649.

## Checklist

- [ ] I've updated the test suite for new or updated code as appropriate
- [ ] I've updated documentation (JSDoc, Markdown, etc.) for new or
updated code as appropriate
- [ ] I've communicated my changes to consumers by [updating changelogs
for packages I've
changed](https://github.com/MetaMask/core/tree/main/docs/processes/updating-changelogs.md)
- [ ] I've introduced [breaking
changes](https://github.com/MetaMask/core/tree/main/docs/processes/breaking-changes.md)
in this PR and have prepared draft pull requests for clients and
consumer packages to resolve them

---------

Co-authored-by: Maarten Zuidhoorn <maarten@zuidhoorn.com>

Author: mcmire

.github/workflows/lint-build-test.yml

@@ -42,6 +42,7 @@ jobs:
           - lint:teams
           - messenger-action-types:check
           - readme-content:check
+          - lint:tsconfigs:all
     steps:
       - name: Checkout and setup environment
         uses: MetaMask/action-checkout-and-setup@v3

README.md

@@ -378,6 +378,7 @@ linkStyle default opacity:0.5
   earn_controller --> network_controller;
   earn_controller --> transaction_controller;
   eip_5792_middleware --> messenger;
+  eip_5792_middleware --> preferences_controller;
   eip_5792_middleware --> transaction_controller;
   eip_5792_middleware --> keyring_controller;
   eip_7702_internal_rpc_middleware --> controller_utils;
@@ -412,6 +413,7 @@ linkStyle default opacity:0.5
   json_rpc_engine --> messenger;
   json_rpc_middleware_stream --> json_rpc_engine;
   keyring_controller --> base_controller;
+  keyring_controller --> controller_utils;
   keyring_controller --> messenger;
   logging_controller --> base_controller;
   logging_controller --> controller_utils;

eslint.config.mjs

@@ -116,10 +116,10 @@ const config = createConfig([
       '**/docusaurus.config.ts',
       '**/test/**/*.{js,ts}',
       '**/tests/**/*.{js,ts}',
-      'scripts/*.ts',
-      'scripts/create-package/**/*.ts',
+      'scripts/**/*.{ts,mts}',
       'packages/platform-api-docs/**/*.ts',
     ],
+    ignores: ['scripts/create-package/package-template/**/*.ts'],
     extends: [nodejs],
   },
   {
@@ -130,7 +130,7 @@ const config = createConfig([
     },
   },
   {
-    files: ['**/*.ts'],
+    files: ['**/*.ts', '**/*.mts'],
     extends: [typescript],
     languageOptions: {
       parserOptions: {

package.json

@@ -25,14 +25,18 @@
     "docs:platform-api:build": "yarn workspace @metamask/platform-api-docs cli ../.. --build --project-label Core",
     "docs:platform-api:dev": "yarn workspace @metamask/platform-api-docs cli ../.. --dev --project-label Core",
     "docs:platform-api:serve": "yarn workspace @metamask/platform-api-docs cli ../.. --serve --project-label Core",
-    "lint": "yarn lint:eslint && echo && yarn lint:misc --check && yarn constraints && yarn lint:dependencies && yarn lint:teams && yarn messenger-action-types:check && yarn readme-content:check",
+    "lint": "yarn lint:eslint && echo && yarn lint:misc --check && yarn constraints && yarn lint:dependencies && yarn lint:teams && yarn messenger-action-types:check && yarn readme-content:check && yarn lint:tsconfigs:all",
     "lint:dependencies": "knip --dependencies && yarn dedupe --check",
     "lint:dependencies:fix": "knip --dependencies && yarn dedupe",
     "lint:eslint": "yarn build:only-clean && NODE_OPTIONS='--max-old-space-size=8192' yarn eslint",
     "lint:fix": "yarn lint:eslint --fix --prune-suppressions && echo && yarn lint:misc --write && yarn constraints --fix && yarn lint:dependencies:fix && yarn messenger-action-types:generate && yarn readme-content:update",
     "lint:misc": "oxfmt --ignore-path .gitignore",
     "lint:misc:check": "yarn lint:misc --check",
     "lint:teams": "tsx scripts/lint-teams-json.ts",
+    "lint:tsconfigs": "tsx scripts/lint-tsconfigs/lint-tsconfigs.mts",
+    "lint:tsconfigs:all": "yarn workspaces foreach --all --parallel --interlaced --verbose run lint:tsconfigs",
+    "lint:tsconfigs:fix": "tsx scripts/lint-tsconfigs/lint-tsconfigs.mts --fix",
+    "lint:tsconfigs:fix:all": "yarn workspaces foreach --all --parallel --interlaced --verbose run lint:tsconfigs:fix",
     "messenger-action-types:check": "yarn workspaces foreach --all --no-private --parallel --interlaced --verbose run messenger-action-types:check",
     "messenger-action-types:generate": "yarn workspaces foreach --all --no-private --parallel --interlaced --verbose run messenger-action-types:generate",
     "postinstall": "yarn skills:postinstall",

packages/account-tree-controller/package.json

@@ -44,6 +44,8 @@
     "build:docs": "typedoc",
     "changelog:update": "../../scripts/update-changelog.sh @metamask/account-tree-controller",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/account-tree-controller",
+    "lint:tsconfigs": "tsx ../../scripts/lint-tsconfigs/lint-tsconfigs.mts",
+    "lint:tsconfigs:fix": "tsx ../../scripts/lint-tsconfigs/lint-tsconfigs.mts --fix",
     "messenger-action-types:check": "tsx ../../packages/messenger-cli/src/cli.ts --formatter oxfmt --check",
     "messenger-action-types:generate": "tsx ../../packages/messenger-cli/src/cli.ts --formatter oxfmt --generate",
     "since-latest-release": "../../scripts/since-latest-release.sh",

packages/accounts-controller/package.json

@@ -44,6 +44,8 @@
     "build:docs": "typedoc",
     "changelog:update": "../../scripts/update-changelog.sh @metamask/accounts-controller",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/accounts-controller",
+    "lint:tsconfigs": "tsx ../../scripts/lint-tsconfigs/lint-tsconfigs.mts",
+    "lint:tsconfigs:fix": "tsx ../../scripts/lint-tsconfigs/lint-tsconfigs.mts --fix",
     "messenger-action-types:check": "tsx ../../packages/messenger-cli/src/cli.ts --formatter oxfmt --check",
     "messenger-action-types:generate": "tsx ../../packages/messenger-cli/src/cli.ts --formatter oxfmt --generate",
     "since-latest-release": "../../scripts/since-latest-release.sh",

packages/accounts-controller/tsconfig.build.json

@@ -10,9 +10,18 @@
     {
       "path": "../base-controller/tsconfig.build.json"
     },
-    { "path": "../keyring-controller/tsconfig.build.json" },
-    { "path": "../network-controller/tsconfig.build.json" },
-    { "path": "../messenger/tsconfig.build.json" }
+    {
+      "path": "../keyring-controller/tsconfig.build.json"
+    },
+    {
+      "path": "../network-controller/tsconfig.build.json"
+    },
+    {
+      "path": "../messenger/tsconfig.build.json"
+    },
+    {
+      "path": "../controller-utils/tsconfig.build.json"
+    }
   ],
   "include": ["../../types", "./src"]
 }

packages/accounts-controller/tsconfig.json

@@ -10,8 +10,15 @@
     {
       "path": "../keyring-controller"
     },
-    { "path": "../network-controller" },
-    { "path": "../messenger" }
+    {
+      "path": "../network-controller"
+    },
+    {
+      "path": "../messenger"
+    },
+    {
+      "path": "../controller-utils"
+    }
   ],
   "include": ["../../types", "./src", "./tests"]
 }

packages/address-book-controller/package.json

@@ -44,6 +44,8 @@
     "build:docs": "typedoc",
     "changelog:update": "../../scripts/update-changelog.sh @metamask/address-book-controller",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/address-book-controller",
+    "lint:tsconfigs": "tsx ../../scripts/lint-tsconfigs/lint-tsconfigs.mts",
+    "lint:tsconfigs:fix": "tsx ../../scripts/lint-tsconfigs/lint-tsconfigs.mts --fix",
     "messenger-action-types:check": "tsx ../../packages/messenger-cli/src/cli.ts --formatter oxfmt --check",
     "messenger-action-types:generate": "tsx ../../packages/messenger-cli/src/cli.ts --formatter oxfmt --generate",
     "since-latest-release": "../../scripts/since-latest-release.sh",

packages/ai-controllers/package.json

@@ -44,6 +44,8 @@
     "build:docs": "typedoc",
     "changelog:update": "../../scripts/update-changelog.sh @metamask/ai-controllers",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/ai-controllers",
+    "lint:tsconfigs": "tsx ../../scripts/lint-tsconfigs/lint-tsconfigs.mts",
+    "lint:tsconfigs:fix": "tsx ../../scripts/lint-tsconfigs/lint-tsconfigs.mts --fix",
     "messenger-action-types:check": "tsx ../../packages/messenger-cli/src/cli.ts --formatter oxfmt --check",
     "messenger-action-types:generate": "tsx ../../packages/messenger-cli/src/cli.ts --formatter oxfmt --generate",
     "since-latest-release": "../../scripts/since-latest-release.sh",