Skip to content

Commit dd80942

Browse files
JoshuaKGoldbergJoshuaKGoldberg
committed
docs: reorganize README.md with comparisons/ directory
1 parent 28a1970 commit dd80942

6 files changed

+204
-205
lines changed

‎README.md

Lines changed: 10 additions & 205 deletions
Original file line numberDiff line numberDiff line change
@@ -40,25 +40,29 @@ The `caseEntries` values in `src/data.ts` can be modified to test:
4040

4141
## Results
4242

43-
Right now, `parserOptions.project` _with_ single-run inference outperforms `parserOptions.projectService`.
43+
Right now, `parserOptions.project` with single-run inference outperforms `parserOptions.projectService`.
4444
This is a performance issue and we are investigating it as a bug.
4545

4646
```plaintext
4747
┌───────┬───────────────────────┬───────────────────────┐
4848
│ files │ project (even layout) │ service (even layout) │
49-
┼───────┼───────────────────────┼───────────────────────┤
50-
│ 1024 │ '1.750 s ± 0.008 s' │ '2.473 s ± 0.011 s' │
51-
┴───────┴───────────────────────┴───────────────────────┘
49+
├───────┼───────────────────────┼───────────────────────┤
50+
│ 1024 │ '2.371 s ± 0.029 s' │ '2.724 s ± 0.049 s' │
51+
└───────┴───────────────────────┴───────────────────────┘
5252
```
5353

5454
See [typescript-eslint/typescript-eslint#9571 Performance: parserOptions.projectService no longer outperforms parserOptions.project](https://github.com/typescript-eslint/typescript-eslint/issues/9571) in typescript-eslint.
55-
Also see the 📌 pinned issues later in this file.
5655

5756
### Result Measurement Notes
5857

59-
- Example measurements taken on an M1 Max Mac Studio with Node.js 22.4.1
58+
- Example measurements taken on an M1 Max Mac Studio with Node.js 22.12.0
6059
- These results are similar across TypeScript versions: 5.0.4, 5.4.5, and 5.5.3
6160

61+
## Comparisons
62+
63+
The `comparisons/` directory contains details on more specific comparisons.
64+
See each `comparisons/*.md` file for details on what's being measured.
65+
6266
## Traces
6367

6468
The `traces/` directory contains more specific traces for investigations.
@@ -67,205 +71,6 @@ The `traces/` directory contains more specific traces for investigations.
6771
6872
All comparisons were run on a common shape of linting: 1024 files with the "even" (triangle-shaped) imports layout.
6973

70-
### Comparison: Globals in Scopes
71-
72-
> 📌 Filed on typescript-eslint as [⚡ Performance: Overhead of populateGlobalsFromLib in scope-manager](https://github.com/typescript-eslint/typescript-eslint/issues/9575).
73-
74-
This trace shows the impact of `@typescript-eslint/scope-manager`'s `populateGlobalsFromLib`.
75-
76-
See `traces/globals-scope-manager/`:
77-
78-
- `baseline.cpuprofile`: Baseline measurement with no changes
79-
- `skipping.cpuprofile`: Commenting out the contents of `populateGlobalsFromLib`
80-
81-
They were generated with:
82-
83-
```shell
84-
cd files-1024-layout-even-singlerun-true-types-service
85-
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=baseline.cpuprofile ../../node_modules/eslint/bin/eslint.js
86-
# clear ../../node_modules/@typescript-eslint/scope-manager/dist/referencer/Referencer.js > populateGlobalsFromLib
87-
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=skipping.cpuprofile ../../node_modules/eslint/bin/eslint.js
88-
```
89-
90-
Hyperfine measurements show a ~20% improvement in lint time:
91-
92-
| Variant | Measurement | User Time |
93-
| -------- | ----------------- | --------- |
94-
| Baseline | 3.137 s ± 0.024 s | 4.417 s |
95-
| Skipping | 2.477 s ± 0.014 s | 3.501 s |
96-
97-
### Comparison: Project and Project Service
98-
99-
This is a preliminary trace to start debugging their differences.
100-
101-
See `traces/Project 1 - Service 2.cpuprofile`.
102-
103-
- Trace #1: `parserOptions.project`
104-
- Trace #2: `parserOptions.projectService`
105-
106-
It was generated with:
107-
108-
```shell
109-
cd cases/files-1024-layout-even-singlerun-true-types-project
110-
node --inspect-brk ../../node_modules/eslint/bin/eslint.js
111-
cd ../files-1024-layout-even-singlerun-true-types-service
112-
node --inspect-brk ../../node_modules/eslint/bin/eslint.js
113-
```
114-
115-
Comparing equivalent code paths:
116-
117-
| Code Path | Project | Service |
118-
| ----------------- | ------- | ------- |
119-
| All `verifyText`s | 2040ms | 2859ms |
120-
| `parseForESLint` | 993ms | 1090ms |
121-
122-
### Comparison: Project Service Client File Cleanups
123-
124-
> 📌 Filed on TypeScript as [⚡ Performance: Project service spends excess time cleaning client files when called synchronously](https://github.com/microsoft/TypeScript/issues/59335).
125-
126-
This comparison shows the cost of the TypeScript project service calling `cleanupProjectsAndScriptInfos`.
127-
128-
See `traces/service-file-cleanup/`:
129-
130-
- `baseline.cpuprofile`: Baseline measurement with no changes
131-
- `skipping.cpuprofile`: Commenting out the contents of TypeScript's `cleanupProjectsAndScriptInfos`
132-
133-
They were generated with:
134-
135-
```shell
136-
cd files-1024-layout-even-singlerun-true-types-service
137-
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=baseline.cpuprofile ../../node_modules/eslint/bin/eslint.js
138-
# clear ../../node_modules/typescript/lib/typescript.js > cleanupProjectsAndScriptInfos
139-
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=skipping.cpuprofile ../../node_modules/eslint/bin/eslint.js
140-
```
141-
142-
Hyperfine measurements show a ~15-20% improvement in lint time:
143-
144-
| Variant | Measurement | User Time |
145-
| -------- | ----------------- | --------- |
146-
| Baseline | 3.215 s ± 0.041 s | 4.483 s |
147-
| Skipping | 2.501 s ± 0.017 s | 3.758 s |
148-
149-
### Comparison: Project Service Uncached File System Stats
150-
151-
> 📌 Filed on TypeScript as [⚡ Performance: Project service doesn't cache all fs.statSync](https://github.com/microsoft/TypeScript/issues/59338).
152-
153-
This comparison shows the cost uncached `fs.statSync` calls inside the project service.
154-
155-
See `traces/service-uncached-stats/`:
156-
157-
- `baseline.cpuprofile`: Baseline measurement with no changes
158-
- `caching.cpuprofile`: Adding a caching `Map` to TypeScript's `statSync`
159-
160-
They were generated with:
161-
162-
```shell
163-
cd files-1024-layout-even-singlerun-true-types-service
164-
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=baseline.cpuprofile ../../node_modules/eslint/bin/eslint.js
165-
# edit ../../node_modules/typescript/lib/typescript.js > statSync (see diff below)
166-
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=caching.cpuprofile ../../node_modules/eslint/bin/eslint.js
167-
```
168-
169-
<details>
170-
<summary><code>diff</code> patch to switch to the <em>Caching</em> variant...</summary>
171-
172-
```diff
173-
diff --git a/node_modules/typescript/lib/typescript.js b/node_modules/typescript/lib/typescript.js
174-
index 4baad59..44639d5 100644
175-
--- a/node_modules/typescript/lib/typescript.js
176-
+++ b/node_modules/typescript/lib/typescript.js
177-
@@ -8546,9 +8546,15 @@ var sys = (() => {
178-
}
179-
}
180-
};
181-
+ const statCache = new Map();
182-
return nodeSystem;
183-
function statSync(path) {
184-
- return _fs.statSync(path, { throwIfNoEntry: false });
185-
+ if (statCache.has(path)) {
186-
+ return statCache.get(path);
187-
+ }
188-
+ const result = _fs.statSync(path, { throwIfNoEntry: false });
189-
+ statCache.set(path, result);
190-
+ return result;
191-
}
192-
function enableCPUProfiler(path, cb) {
193-
if (activeSession) {
194-
```
195-
196-
</details>
197-
198-
Hyperfine measurements show a ~7-12% improvement in lint time:
199-
200-
| Variant | Measurement | User Time |
201-
| -------- | ----------------- | --------- |
202-
| Baseline | 3.112 s ± 0.033 s | 4.382 |
203-
| Caching | 2.740 s ± 0.030 s | 4.032 |
204-
205-
### Comparison: Project Service Uncached File System Path Reads
206-
207-
> 📌 Filed on TypeScript as [⚡ Performance: Project service doesn't cache all fs.realpath](https://github.com/microsoft/TypeScript/issues/59342).
208-
209-
This comparison shows the cost uncached `fs.realpath` calls inside the project service.
210-
211-
See `traces/service-uncached-realpaths/`:
212-
213-
- `baseline.cpuprofile`: Baseline measurement with no changes
214-
- `caching.cpuprofile`: Adding a caching `Map` to TypeScript's `realpath`
215-
216-
They were generated with:
217-
218-
```shell
219-
cd files-1024-layout-even-singlerun-true-types-service
220-
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=baseline.cpuprofile ../../node_modules/eslint/bin/eslint.js
221-
# edit ../../node_modules/typescript/lib/typescript.js > realpath (see diff below)
222-
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=caching.cpuprofile ../../node_modules/eslint/bin/eslint.js
223-
```
224-
225-
<details>
226-
<summary><code>diff</code> patch to switch to the <em>Caching</em> variant...</summary>
227-
228-
```diff
229-
diff --git a/node_modules/typescript/lib/typescript.js b/node_modules/typescript/lib/typescript.js
230-
index 4baad59..e53476d 100644
231-
--- a/node_modules/typescript/lib/typescript.js
232-
+++ b/node_modules/typescript/lib/typescript.js
233-
@@ -13,6 +13,8 @@ See the Apache Version 2.0 License for specific language governing permissions
234-
and limitations under the License.
235-
***************************************************************************** */
236-
237-
+var realpathCache = new Map();
238-
+
239-
var ts = {}; ((module) => {
240-
"use strict";
241-
var __defProp = Object.defineProperty;
242-
@@ -8798,6 +8800,15 @@ var sys = (() => {
243-
return path.length < 260 ? _fs.realpathSync.native(path) : _fs.realpathSync(path);
244-
}
245-
function realpath(path) {
246-
+ const cached = realpathCache.get(path);
247-
+ if (cached) {
248-
+ return cached;
249-
+ }
250-
+ const result = realpathWorker(path);
251-
+ realpathCache.set(path, result);
252-
+ return result;
253-
+ }
254-
+ function realpathWorker(path) {
255-
try {
256-
return fsRealpath(path);
257-
} catch {
258-
```
259-
260-
</details>
261-
262-
Hyperfine measurements with `--runs 50` show a ~0.5-2.5% improvement in lint time:
263-
264-
| Variant | Measurement | User Time |
265-
| -------- | ----------------- | --------- |
266-
| Baseline | 3.153 s ± 0.039 s | 4.403 s |
267-
| Caching | 3.073 s ± 0.048 s | 4.377 s |
268-
26974
## Contributors
27075

27176
<!-- spellchecker: disable -->

‎comparisons/globals-in-scopes.md

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# Comparison: Globals in Scopes
2+
3+
> 📌 Filed on typescript-eslint as [⚡ Performance: Overhead of populateGlobalsFromLib in scope-manager](https://github.com/typescript-eslint/typescript-eslint/issues/9575).
4+
5+
This trace shows the impact of `@typescript-eslint/scope-manager`'s `populateGlobalsFromLib`.
6+
7+
See `traces/globals-scope-manager/`:
8+
9+
- `baseline.cpuprofile`: Baseline measurement with no changes
10+
- `skipping.cpuprofile`: Commenting out the contents of `populateGlobalsFromLib`
11+
12+
They were generated with:
13+
14+
```shell
15+
cd files-1024-layout-even-singlerun-true-types-service
16+
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=baseline.cpuprofile ../../node_modules/eslint/bin/eslint.js
17+
# clear ../../node_modules/@typescript-eslint/scope-manager/dist/referencer/Referencer.js > populateGlobalsFromLib
18+
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=skipping.cpuprofile ../../node_modules/eslint/bin/eslint.js
19+
```
20+
21+
Hyperfine measurements show a ~20% improvement in lint time:
22+
23+
| Variant | Measurement | User Time |
24+
| -------- | ----------------- | --------- |
25+
| Baseline | 3.137 s ± 0.024 s | 4.417 s |
26+
| Skipping | 2.477 s ± 0.014 s | 3.501 s |

‎comparisons/project-and-project-service.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Comparison: Project and Project Service
2+
3+
This is a preliminary trace to start debugging their differences.
4+
5+
See `traces/Project 1 - Service 2.cpuprofile`.
6+
7+
- Trace #1: `parserOptions.project`
8+
- Trace #2: `parserOptions.projectService`
9+
10+
It was generated with:
11+
12+
```shell
13+
cd cases/files-1024-layout-even-singlerun-true-types-project
14+
node --inspect-brk ../../node_modules/eslint/bin/eslint.js
15+
cd ../files-1024-layout-even-singlerun-true-types-service
16+
node --inspect-brk ../../node_modules/eslint/bin/eslint.js
17+
```
18+
19+
Comparing equivalent code paths:
20+
21+
| Code Path | Project | Service |
22+
| ----------------- | ------- | ------- |
23+
| All `verifyText`s | 2040ms | 2859ms |
24+
| `parseForESLint` | 993ms | 1090ms |

‎comparisons/project-service-client-file-cleanups.md

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# Comparison: Project Service Client File Cleanups
2+
3+
> 📌 Filed on TypeScript as [⚡ Performance: Project service spends excess time cleaning client files when called synchronously](https://github.com/microsoft/TypeScript/issues/59335).
4+
5+
This comparison shows the cost of the TypeScript project service calling `cleanupProjectsAndScriptInfos`.
6+
7+
See `traces/service-file-cleanup/`:
8+
9+
- `baseline.cpuprofile`: Baseline measurement with no changes
10+
- `skipping.cpuprofile`: Commenting out the contents of TypeScript's `cleanupProjectsAndScriptInfos`
11+
12+
They were generated with:
13+
14+
```shell
15+
cd files-1024-layout-even-singlerun-true-types-service
16+
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=baseline.cpuprofile ../../node_modules/eslint/bin/eslint.js
17+
# clear ../../node_modules/typescript/lib/typescript.js > cleanupProjectsAndScriptInfos
18+
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=skipping.cpuprofile ../../node_modules/eslint/bin/eslint.js
19+
```
20+
21+
Hyperfine measurements show a ~15-20% improvement in lint time:
22+
23+
| Variant | Measurement | User Time |
24+
| -------- | ----------------- | --------- |
25+
| Baseline | 3.215 s ± 0.041 s | 4.483 s |
26+
| Skipping | 2.501 s ± 0.017 s | 3.758 s |

‎comparisons/project-service-uncached-file-system-path-reads.md

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# Comparison: Project Service Uncached File System Path Reads
2+
3+
> 📌 Filed on TypeScript as [⚡ Performance: Project service doesn't cache all fs.realpath](https://github.com/microsoft/TypeScript/issues/59342).
4+
5+
This comparison shows the cost uncached `fs.realpath` calls inside the project service.
6+
7+
See `traces/service-uncached-realpaths/`:
8+
9+
- `baseline.cpuprofile`: Baseline measurement with no changes
10+
- `caching.cpuprofile`: Adding a caching `Map` to TypeScript's `realpath`
11+
12+
They were generated with:
13+
14+
```shell
15+
cd files-1024-layout-even-singlerun-true-types-service
16+
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=baseline.cpuprofile ../../node_modules/eslint/bin/eslint.js
17+
# edit ../../node_modules/typescript/lib/typescript.js > realpath (see diff below)
18+
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=caching.cpuprofile ../../node_modules/eslint/bin/eslint.js
19+
```
20+
21+
<details>
22+
<summary><code>diff</code> patch to switch to the <em>Caching</em> variant...</summary>
23+
24+
```diff
25+
diff --git a/node_modules/typescript/lib/typescript.js b/node_modules/typescript/lib/typescript.js
26+
index 4baad59..e53476d 100644
27+
--- a/node_modules/typescript/lib/typescript.js
28+
+++ b/node_modules/typescript/lib/typescript.js
29+
@@ -13,6 +13,8 @@ See the Apache Version 2.0 License for specific language governing permissions
30+
and limitations under the License.
31+
***************************************************************************** */
32+
33+
+var realpathCache = new Map();
34+
+
35+
var ts = {}; ((module) => {
36+
"use strict";
37+
var __defProp = Object.defineProperty;
38+
@@ -8798,6 +8800,15 @@ var sys = (() => {
39+
return path.length < 260 ? _fs.realpathSync.native(path) : _fs.realpathSync(path);
40+
}
41+
function realpath(path) {
42+
+ const cached = realpathCache.get(path);
43+
+ if (cached) {
44+
+ return cached;
45+
+ }
46+
+ const result = realpathWorker(path);
47+
+ realpathCache.set(path, result);
48+
+ return result;
49+
+ }
50+
+ function realpathWorker(path) {
51+
try {
52+
return fsRealpath(path);
53+
} catch {
54+
```
55+
56+
</details>
57+
58+
Hyperfine measurements with `--runs 50` show a ~0.5-2.5% improvement in lint time:
59+
60+
| Variant | Measurement | User Time |
61+
| -------- | ----------------- | --------- |
62+
| Baseline | 3.153 s ± 0.039 s | 4.403 s |
63+
| Caching | 3.073 s ± 0.048 s | 4.377 s |

0 commit comments

Comments
 (0)