Skip to content

fs: Performance Enhancement: Concurrent Execution of Async Tasks with Promise.all ✈ #51215

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 16 commits into
base: main
Choose a base branch
from
Open

fs: Performance Enhancement: Concurrent Execution of Async Tasks with Promise.all ✈ #51215

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
5ffc1c9
perf:Running async task in concurrent
sanjaiyan-dev Dec 19, 2023
7e98301
Update cp.js
sanjaiyan-dev Dec 19, 2023
4e5897c
Update recursive_watch.js
sanjaiyan-dev Dec 19, 2023
f31f573
Update watch_mode.js
sanjaiyan-dev Dec 19, 2023
321ba33
Update lib/internal/fs/cp/cp.js
sanjaiyan-dev Dec 20, 2023
be6d2d2
Update lib/internal/fs/cp/cp.js
sanjaiyan-dev Dec 20, 2023
2c5cf26
Update lib/internal/fs/recursive_watch.js
sanjaiyan-dev Dec 20, 2023
4db71a0
Update lib/internal/fs/recursive_watch.js
sanjaiyan-dev Dec 20, 2023
77bafb3
Update lib/internal/main/watch_mode.js
sanjaiyan-dev Dec 20, 2023
a7d7f5f
Update lib/internal/main/watch_mode.js
sanjaiyan-dev Dec 20, 2023
d7c20b9
Update lib/internal/fs/cp/cp.js
sanjaiyan-dev Dec 20, 2023
4de13db
Update lib/internal/fs/recursive_watch.js
sanjaiyan-dev Dec 20, 2023
306208e
Update lib/internal/main/watch_mode.js
sanjaiyan-dev Dec 20, 2023
a8b8a45
fix: Fix some minor bugs
sanjaiyan-dev Dec 20, 2023
1ab227e
fixes: Fix memory and lint issues
sanjaiyan-dev Dec 24, 2023
a77ecb4
fix: remove unwanted array
sanjaiyan-dev Dec 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion lib/internal/fs/cp/cp.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,12 @@
const {
ArrayPrototypeEvery,
ArrayPrototypeFilter,
ArrayPrototypePush,
Boolean,
PromisePrototypeThen,
PromiseReject,
SafePromiseAll,
SafePromiseAllReturnVoid,
StringPrototypeSplit,
} = primordials;
const {
Expand Down Expand Up @@ -313,13 +315,18 @@ async function mkDirAndCopy(srcMode, src, dest, opts) {

async function copyDir(src, dest, opts) {
const dir = await opendir(src);
const promises = [];

for await (const { name } of dir) {
const srcItem = join(src, name);
const destItem = join(dest, name);
const { destStat, skipped } = await checkPaths(srcItem, destItem, opts);
if (!skipped) await getStatsForCopy(destStat, srcItem, destItem, opts);
if (!skipped) {
ArrayPrototypePush(promises, getStatsForCopy(destStat, srcItem, destItem, opts));
}
}

await SafePromiseAllReturnVoid(promises);
}

async function onLink(destStat, src, dest, opts) {
Expand Down
4 changes: 3 additions & 1 deletion lib/internal/fs/recursive_watch.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,7 @@ class FSWatcher extends EventEmitter {

try {
const files = await opendir(folder);
const promises = [];

for await (const file of files) {
if (this.#closed) {
Expand All @@ -159,10 +160,11 @@ class FSWatcher extends EventEmitter {
if (file.isFile()) {
this.#watchFile(f);
} else if (file.isDirectory() && !file.isSymbolicLink()) {
await this.#watchFolder(f);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving to Promise.all, this code will not probably throw EMFILE, too many open files?

ArrayPrototypePush(promises, this.#watchFolder(f));
}
}
}
await SafePromiseAllReturnVoid(promises);
} catch (error) {
this.emit('error', error);
}
Expand Down