Skip to content

esm: non file baseurls and fix cache key collision #42392

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

esm: non file baseurls and fix cache key collision #42392

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
0cdd803
esm: fix baseURL for non-files
bmeck Mar 18, 2022
cfdd468
fixup: make import.meta properly init
bmeck Mar 18, 2022
ff4b7c0
fixup: oopsie
bmeck Mar 18, 2022
d80ab5b
fixup: oopsie
bmeck Mar 18, 2022
1513e3c
fixup: lint
bmeck Mar 18, 2022
b6b2a91
fixup: Update lib/internal/modules/esm/loader.js
bmeck Mar 21, 2022
3c518c2
fixup: review nits
bmeck Mar 21, 2022
933139e
fixup: lint
bmeck Mar 21, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion lib/internal/main/eval_stdin.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ readStdin((code) => {

const print = getOptionValue('--print');
if (getOptionValue('--input-type') === 'module')
evalModule(code, print);
evalModule(code, print, 'node:stdin');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CJS is using [stdin] instead of node:stdin for this, should we chose node:[stdin] instead? I don't feel strongly about this at all, but if there are no drawbacks maybe we should aim for consistency.

On another note, I've noticed that this PR makes the following work:

$ echo 'export {test} from "node:stdin"' > test.mjs
$ out/Release/node --input-type=module <<EOF
export let test = 4;
import { test as test2 } from './test.mjs';
console.log({test,test2})
test++;
console.log({test,test2});
EOF
{ test: 4, test2: 4 }
{ test: 5, test2: 5 }

While it doesn't seem to work at all on master:

$ echo 'export {test} from "./[eval1]"' > test.mjs
$ node --input-type=module <<EOF
export let test = 4;
import { test as test2 } from './test.mjs';
console.log({test,test2})
test++;
console.log({test,test2});
EOF
node:internal/errors:465
    ErrorCaptureStackTrace(err);
    ^

Error [ERR_MODULE_NOT_FOUND]: Cannot find module '…/[eval1]' imported from …/test.mjs
    at new NodeError (node:internal/errors:372:5)
    at finalizeResolution (node:internal/modules/esm/resolve:404:11)
    at moduleResolve (node:internal/modules/esm/resolve:963:10)
    at defaultResolve (node:internal/modules/esm/resolve:1172:11)
    at ESMLoader.resolve (node:internal/modules/esm/loader:580:30)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:294:18)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:80:40)
    at link (node:internal/modules/esm/module_job:78:36) {
  code: 'ERR_MODULE_NOT_FOUND'
}

Node.js v17.7.2

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrapping both in [] seems good for consistency, then we could also use [eval] like CJS.

I'm fine removing the relative importing for these modules as long as it gives a little more consistency with CJS where you can use require() to get relative files from [stdin].

else
evalScript('[stdin]',
code,
Expand Down
2 changes: 1 addition & 1 deletion lib/internal/main/eval_string.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ markBootstrapComplete();
const source = getOptionValue('--eval');
const print = getOptionValue('--print');
if (getOptionValue('--input-type') === 'module')
evalModule(source, print);
evalModule(source, print, 'node:execArgv');
else
evalScript('[eval]',
source,
Expand Down
4 changes: 1 addition & 3 deletions lib/internal/modules/esm/initialize_import_meta.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,15 +26,13 @@ function createImportMetaResolve(defaultParentUrl) {
* @param {{url: string}} context
*/
function initializeImportMeta(meta, context) {
let url = context.url;
const url = asyncESM.esmLoader.getBaseURL(context.url);

// Alphabetical
if (experimentalImportMetaResolve) {
meta.resolve = createImportMetaResolve(url);
}

url = asyncESM.esmLoader.getBaseURL(url);

meta.url = url;
}

Expand Down
30 changes: 21 additions & 9 deletions lib/internal/modules/esm/loader.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ const {
PromiseAll,
RegExpPrototypeExec,
SafeArrayIterator,
SafeMap,
SafeWeakMap,
StringPrototypeStartsWith,
globalThis,
Expand All @@ -30,7 +31,7 @@ const {
ERR_INVALID_RETURN_VALUE,
ERR_UNKNOWN_MODULE_FORMAT
} = require('internal/errors').codes;
const { pathToFileURL, isURLInstance, URL } = require('internal/url');
const { isURLInstance, URL } = require('internal/url');
const {
isAnyArrayBuffer,
isArrayBufferView,
Expand Down Expand Up @@ -92,10 +93,6 @@ class ESMLoader {
*/
cjsCache = new SafeWeakMap();

/**
* The index for assigning unique URLs to anonymous module evaluation
*/
evalIndex = 0;

/**
* Registry of loaded modules, akin to `require.cache`
Expand Down Expand Up @@ -207,8 +204,10 @@ class ESMLoader {

async eval(
source,
url = pathToFileURL(`${process.cwd()}/[eval${++this.evalIndex}]`).href
url,
baseURL
) {
this.baseURLs.set(url, baseURL);
const evalInstance = (url) => {
const { ModuleWrap, callbackMap } = internalBinding('module_wrap');
const module = new ModuleWrap(url, undefined, source, 0, 0);
Expand All @@ -217,6 +216,9 @@ class ESMLoader {
return this.import(specifier,
this.getBaseURL(url),
importAssertions);
},
initializeImportMeta: (meta, { url }) => {
this.importMetaInitialize(meta, { url });
}
});

Expand All @@ -232,6 +234,7 @@ class ESMLoader {
};
}

baseURLs = new SafeMap();
/**
* Returns the url to use for the resolution of a given cache key url
* These are not guaranteed to be the same.
Expand All @@ -253,20 +256,29 @@ class ESMLoader {
* @returns {string}
*/
getBaseURL(url) {
let baseURL = this.baseURLs.get(url);
// This will only be a string if it exists in the Map otherwise, it
// doesn't have a baseURL or needs to compute the baseURL
if (typeof baseURL === 'string') {
return baseURL;
}
if (
StringPrototypeStartsWith(url, 'http:') ||
StringPrototypeStartsWith(url, 'https:')
) {
// The request & response have already settled, so they are in
// fetchModule's cache, in which case, fetchModule returns
// immediately and synchronously
url = fetchModule(new URL(url), { parentURL: url }).resolvedHREF;
baseURL = fetchModule(new URL(url), { parentURL: url }).resolvedHREF;
// This should only occur if the module hasn't been fetched yet
if (typeof url !== 'string') {
if (typeof baseURL !== 'string') {
throw new ERR_INTERNAL_ASSERTION(`Base url for module ${url} not loaded.`);
}
this.baseURLs.set(url, baseURL);
Copy link

@Mifrill Mifrill Dec 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why we are mutating baseURLs on the call getter (getBaseURL)?

} else {
baseURL = url;
}
return url;
return baseURL;
}

/**
Expand Down
13 changes: 11 additions & 2 deletions lib/internal/process/execution.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ const {
emitAfter,
popAsyncContext,
} = require('internal/async_hooks');
const { pathToFileURL } = require('internal/url');

// shouldAbortOnUncaughtToggle is a typed array for faster
// communication with JS.
Expand All @@ -39,13 +40,21 @@ function tryGetCwd() {
}
}

function evalModule(source, print) {
/**
* The index for assigning unique URLs to anonymous module evaluation
*/
let evalIndex = 0;
function evalModule(source,
print,
cacheKey = pathToFileURL(`node:eval/[eval${++evalIndex}]`),
baseURL = pathToFileURL(process.cwd() + '/').href,
) {
if (print) {
throw new ERR_EVAL_ESM_CANNOT_PRINT();
}
const { loadESM } = require('internal/process/esm_loader');
const { handleMainPromise } = require('internal/modules/run_main');
return handleMainPromise(loadESM((loader) => loader.eval(source)));
return handleMainPromise(loadESM((loader) => loader.eval(source, cacheKey, baseURL)));
}

function evalScript(name, body, breakFirstLine, print) {
Expand Down