Symlinks cause parserOptions.project errors

[nikparo]

• I have tried restarting my IDE and the issue persists.
• I have updated to the latest version of the packages.
• I have read the FAQ and my problem is not listed.

Repro

https://github.com/nikparo/ts-eslint-symlinks

Expected Result

The targeted directories and files should be linted, whether or not there are symlinks elsewhere pointing at them.

Actual Result

If there is a symlink somewhere pointing at a directory, then linting it will throw Parsing error: "parserOptions.project" has been set for @typescript-eslint/parser. errors. At least if the symlink alphabetically comes before the directory.

> [email protected] lint /Users/nik/Projects/tmp/ts-eslint-symlinks
> eslint libs/symlinked-lib/**/*.ts

  typescript-eslint:typescript-estree:parser parserOptions.project (excluding ignored) matched projects: Set { '/users/nik/projects/tmp/ts-eslint-symlinks/tsconfig.base.json' } +0ms
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: /Users/nik/Projects/tmp/ts-eslint-symlinks/libs/symlinked-lib/src/index.ts +0ms
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. /users/nik/projects/tmp/ts-eslint-symlinks/libs/symlinked-lib/src/index.ts +0ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /users/nik/projects/tmp/ts-eslint-symlinks/tsconfig.base.json. +0ms

/Users/nik/Projects/tmp/ts-eslint-symlinks/libs/symlinked-lib/src/index.ts
  0:0  error  Parsing error: "parserOptions.project" has been set for @typescript-eslint/parser.
The file does not match your project config: libs/symlinked-lib/src/index.ts.
The file must be included in at least one of the projects provided

By deleting the symlink apps/legacy-app/libs (which points at libs) the errors go away. Whether the symlink is ignored or not using ignorePatterns does not seem to matter.

Additional Info

I believe I have tracked this down to the use of updatedProgram.getRootFileNames() in createWatchProgram.js. That method call seems to only get the first instance of a symlinked file. I.e. apps/legacy-app/libs/symlinked-lib/src/index.ts will be in the returned fileList, but libs/symlinked-lib/src/index.ts will not.

Versions

package	version
@typescript-eslint/typescript-estree	4.14.1
TypeScript	4.1.3
node	v12.20.0

[bradzacher]

We build on top of TypeScript and eslint, and we delegate all file reading to those packages.

I haven't looked into it, but I don't know it there's much we can do without reading from disk and attempting to pre-empt what TS is doing to normalise the file paths.

Attempting to encode knowledge about how TS/ESLint handle symlinks is also a recipe for disaster when TS releases a new version or ESLint releases a major.

Additionally disk reads aren't cheap so IDK if it's a great idea to just build into the regular flow.

Symlinks are also a pretty rare usecase in the community (hence this is the first time someone has reported this), so I'm not sure how much effort is worth putting into this.
If you want to look into this, I'm happy to find out the results of a deeper investigation! I can answer any questions you might have.

I'd be looking for a champion to work on this, as I don't have the time to look into this niche of an issue (similar to vue support - vuejs/eslint-plugin-vue#1296)

[nikparo]

Unfortunately I don't really know any of the technologies very well, otherwise I may have dug into it a bit further. I'm currently more inclined to try and structure my project a bit differently instead to work around the issue.

I did notice though that TS seems to have some symlink functionality built in, as updatedProgram also has a getSymlinkCache function. It didn't provide anything of immediate value though.

[armano2]

i'm going to look into this

[armano2]

i have few news about this issue, and this seem to be caused by our normalization of paths <- that was a fix introduced to fix issues with editors

i can confirm that i can reproduce this issue on linux

[larshp]

any workarounds possible?

[nikparo]

I discovered this because we have a monorepo (@nrwl/nx) containing a legacy app among other things. We decided to make our libraries "buildable" and import transpiled code instead. This got us around the issue even though we still use symlinks, since we have set up eslint to ignore the transpiled code.

[D-Pow]

I was experiencing this as well, and finally came across a solution. Basically, we need to compare pwd (which preserves symlinks) with the repo's root directory (which doesn't preserve symlinks in most cases), strip out any nested paths from pwd if present, and then export the resulting pwd - nestedPath string as a constant for all other files/configs to use.

The gist is:

/***   ./config/utils/Files.js   ***/
const fs = require('fs');
const path = require('path');
const childProcess = require('child_process');


/*
 * Fix paths to use the path of the current process' directory if it's within a symlinked path.
 * Otherwise, IDEs may have trouble with automatic resolutions for ESLint, file paths, etc.
 * 
 * Short summary:
 * - process.cwd() == `npm prefix` - Doesn't preserve symlinks
 * - process.env.PWD == `pwd` - Preserves symlinks
 * - fs.realpathSync(process.env.PWD) == `realpath $(pwd)` - Doesn't preserve symlinks
 */

// Root directory of repository - absolute path without preserving symlinks
// You could use your own logic here, this was my choice so the root dir was always the same regardless
// of where a script/process was run
const realPath = path.resolve(
    childProcess
        .execSync('npm prefix')
        .toString()
        .replace(/\n/g, '')
);
// Root dir preserving symlinks + current process' path (e.g. if in a nested directory from the root)
const currentProcessPath = process.env.PWD;
// Only the nested directory, used for diffing any (non-)symlink-preserving path (nested or not) with
// the repo root in order to convert PWD to the root dir
const currentProcessNestedPath = fs.realpathSync(currentProcessPath).replace(realPath, '');
// Root dir preserving symlinks without the nested directory
const rootDirMaintainingSymlinks = currentProcessPath.replace(currentProcessNestedPath, '');


// Final constant for use in other files
export const rootDir = rootDirMaintainingSymlinks;

// Now to resolve the tsconfig.json
export const tsconfigPath = path.resolve(rootDir, 'tsconfig.json');



/***   ./.eslintrc.js   ***/
const {
    rootDir,
    tsconfigPath,
} = require('./config/utils/Files');


module.exports = {
    parser: '@typescript-eslint/parser',
    parserOptions: {
        // Set the root directory of the repo - now preserving symlinks
        tsconfigRootDir: rootDir,
        // Path of tsconfig.json - now preserving symlinks
        project: tsconfigPath,
        // ...
    },
    // ...
};

Tested on Linux Mint Uma 20.2.

[gutenye]

Following this discussion, I think this will fix the issue, Change

typescript-eslint/packages/typescript-estree/src/create-program/createProjectProgram.ts

Line 26 in da48527

getProgramsForProjects(code, extra.filePath, extra),

to

const realPath = fs.realpathSync(extra.filePath)
getProgramsForProjects(code, realPath, extra), 

[ThayalanGR]

Isn't this fix (@gutenye) not yet the part of current release?

[mateo-m]

Any news on this issue?
I need to use symbolic links for my projects but ESLint seems to break because of that.

[ThayalanGR]

Not only Eslint, Many of the built-ins (eg: Git changes feature will not work as expected on symlinked dirs - try checking it out) and custom plugins of vscode fails to work along with symlinked dirs, so I suggest you to find a way to work without sym-linking, I wasted plethora of time looking for a fix, as mentioned in this thread, if I write fix for one plugin (like Eslint) the same issue will arise in other plugins as well.. since this is never ending loop and all the plugins were relatively referenced this causes error while resolving symbolic links, changing our dev flow is the only fix right now, since we can't write patch like this for every plugin which has this issue.

[mateo-m]

Not only Eslint, Many of the built-ins (eg: Git changes feature will not work as expected on symlinked dirs - try checking it out) and custom plugins of vscode fails to work along with symlinked dirs, so I suggest you to find a way to work without sym-linking, I wasted plethora of time looking for a fix, as mentioned in this thread, if I write fix for one plugin (like Eslint) the same issue will arise in other plugins as well.. since this is never ending loop and all the plugins were relatively referenced this causes error while resolving symbolic links, changing our dev flow is the only fix right now, since we can't write patch like this for every plugin which has this issue.

Oh, so issue stems from VS Code? Not ESLint itself? @ThayalanGR

Edit:
Seems like a work-around "fix" would be to use bind mounts instead of symbolic links.
I'll test this potential band-aid when I have some time then tell you guys if it solved my issue.

[ThayalanGR]

Not only Eslint, Many of the built-ins (eg: Git changes feature will not work as expected on symlinked dirs - try checking it out) and custom plugins of vscode fails to work along with symlinked dirs, so I suggest you to find a way to work without sym-linking, I wasted plethora of time looking for a fix, as mentioned in this thread, if I write fix for one plugin (like Eslint) the same issue will arise in other plugins as well.. since this is never ending loop and all the plugins were relatively referenced this causes error while resolving symbolic links, changing our dev flow is the only fix right now, since we can't write patch like this for every plugin which has this issue.

Oh, so issue stems from VS Code? Not ESLint itself? @ThayalanGR

Edit:

Seems like a work-around "fix" would be to use bind mounts instead of symbolic links.

I'll test this potential band-aid when I have some time then tell you guys if it solved my issue.

No, it is not stems from vscode, both having this symlink issue in common.

[alex-kinokon]

I’m also having this problem using ESLint from a symlink location resulting in the error project was set to `true` but couldn't find any tsconfig.json relative to .... The problem seems to be

typescript-eslint/packages/typescript-estree/src/parseSettings/getProjectConfigFiles.ts

Line 36 in 810fc8c

let directory = path.dirname(parseSettings.filePath);

typescript-eslint/packages/typescript-estree/src/parseSettings/getProjectConfigFiles.ts

Lines 55 to 58 in 810fc8c

	} while (
	directory.length > 1 &&
	directory.length >= parseSettings.tsconfigRootDir.length
	);

from PR #6084. directory is the path before symlink resolution and parseSettings.tsconfigRootDir is the path after symlink resolution, causing the length comparison to fail. Perhaps we can wrap

typescript-eslint/packages/typescript-estree/src/parseSettings/createParseSettings.ts

Lines 68 to 73 in 810fc8c

	filePath: ensureAbsolutePath(
	typeof options.filePath === 'string' && options.filePath !== '<input>'
	? options.filePath
	: getFileName(options.jsx),
	tsconfigRootDir,
	),

with fs.realpathSync here?

[JoshuaKGoldberg]

@alex-kinokon feel free to send a PR, that'd be lovely! I'm not 100% sure that the project: true issue is the same as the original one in the issue, but either way if there's a demonstrable problem we'd happily take in a fix.

I'll extend that invitation to everybody seeing this. We the maintainers haven't had time to send a PR ourselves but I see at least 2-3 suggestions of how to fix the issue in the thread. Wonderful opportunity for someone to send a code contribution in. 😄

[ulfgebhardt]

@alex-kinokon Can you please provide your insight as PR ❤️

[birgersp]

(noob here)

I see that calling fs.realpathSync is a suggested solution, but can we use functions from the fs module in this project? Wouldn't that require an import of @types/node?

[JoshuaKGoldberg]

Some code already calls to fs functions. Search for 'fs' imports and you'll come across files like useProvidedPrograms.ts.

[birgersp]

Some code already calls to fs functions. Search for 'fs' imports and you'll come across files like useProvidedPrograms.ts.

Thanks. Tried applying the propsed fixed but tests went bananas.

Hoping that someone who knows this project will pick this up 🤞🙏

[jakebailey]

Drive-by, but is this "fixed" by just using the project service? After all, surely this code works with tsserver in VS Code, so theoretically should behave no differently with the project service enabled.

[JoshuaKGoldberg]

That's a workaround, yeah.

[BigSamu]

Any final solution on this? To avoid using a work around.

[JoshuaKGoldberg]

If there was an update, it'd be posted here. There is not. It's still status: accepting prs. So if this is impacting you, we'd welcome you sending a PR to fix it! 🙂

[bradzacher]

With any issue opened in this project — it either has visible progress in the form of an attached PR, or it has no progress.

We are a community-run project. The volunteer maintainers spend most of their time triaging issues and reviewing PRs. This means that most issues will not progress unless a member of the community steps up and champions it.

If this issue is important to you — consider being that champion.
If not — please just subscribe to the issue and wait patiently.

[BigSamu]

@JoshuaKGoldberg, @bradzacher,

Sorry. My previous message was not intended to be pushy. I will wait patiently for this.