Stat fetch default

[jace-roell]

Proposed changes

Modifies the behavior of FileSystem calls in regards to fetching and caching values, aswell as caching similar in-flight requests to avoid redundant calls.

Feature Flags

• FeatureFlag.ts class: new class that allows feature flags to be set in the feature-flags.json file.
  Currently is used for the fetchByDefault feature flag, this flag will be removed prior to merging of this PR, but it is kept in order to allow for toggling of behavior when testing

fetchByDefault true vs false

• false: maintains the logic on main branch
  fetch=false (default): will only look in the fileSystem cache (this.root) for entries
  fetch=true: will do a remote lookup, regardless of a value in the cache
• true: new logic (proposed new behavior)
  fetch=false (default): will do a lookup in the fileSystem cache but will fallback to a remote lookup in the case where one is not found
  fetch=true: will ignore values in the FileSystem cache, does a remote lookup regardless

How to toggle a feature flag:

1. Open the command palette and run Developer: Open Extensions Folder
2. Navigate to the Zowe Explorer directory
3. The fetchByDefault feature flag may be modified in the feature-flags.json file

Request caching

• When a FileSystem request is made, a key is generated for the request. The key and request are stored in a map. If a request with the same key is made, the subsequent request will return the promise of the original to avoid redundant requests. The map entry is cleared upon the resolution of of the promise.
• The key includes the function, URI, and any truthy query parameters.

Enhanced Dataset Caching

• When making a FileSystem call on a Dataset member, the request will return all of the members of the parent PDS and find the corresponding member. This logic has been enhanced to cache all of the returned members instead of the just the desired member.

Catching invalid DS URIs

• Catches more vscode stat() calls with URIs that could not exist as data sets

Trailing slash fix

• Fixes a bug where URIs with trailing slashes would throw an error

Example Script:

    const profile = "<profile>";
    const dsn1 = "<DSN>/<MEM1>";
    const dsn2 = "<DSN>/<MEM2>";

        setTimeout(async () => {
            try {
                console.info("Stat1 DS");

                const statResult = await vscode.workspace.fs.stat(vscode.Uri.parse(`zowe-ds:/${profile}/${dsn1}`));
                console.info(`Stat1 success ${JSON.stringify(statResult)}`);
            } catch (err) {
                console.error(`Stat1 ds error ${JSON.stringify(err)}`);
            }
        }, 3000);

        setTimeout(async () => {
            try {
                console.info("Stat2");
                const statResult = await vscode.workspace.fs.stat(vscode.Uri.parse(`zowe-ds:/${profile}/${dsn2}`));
                console.info(`Stat2 success ${JSON.stringify(statResult)}`);
            } catch (err) {
                console.error(`Stat2 error ${JSON.stringify(err)}`);
            }
        }, 3000);

Release Notes

Milestone:

Changelog:

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• Enhancement (non-breaking change which adds or improves functionality)
• Breaking change (a change that would cause existing functionality to not work as expected)
• Documentation (Markdown, README updates)
• Other (please specify above in "Proposed changes" section)

Checklist

General

• I have read the CONTRIBUTOR GUIDANCE wiki
• All PR dependencies have been merged and published (if applicable)
• A GIF or screenshot is included in the PR for visual changes
• The pre-publish command has been executed:
  • v2 and below: yarn workspace vscode-extension-for-zowe vscode:prepublish
  • v3: pnpm --filter vscode-extension-for-zowe vscode:prepublish
• New ZE APIs are tested with extender types that haven't adopted yet to determine breaking changes. Can use Zowe zFTP marketplace extension.

Code coverage

• There is coverage for the code that I have added
• I have added new test cases and they are passing
• I have manually tested the changes

Deployment

• I have tested new functionality with the FTP extension and profile verifying no extender profile type breakages introduced
• I have added developer documentation (if applicable)
• Documentation should be added to Zowe Docs
  • If you're an outside contributor, please post in the #zowe-doc Slack channel to coordinate documentation.
  • Otherwise, please check with the rest of the squad about any needed documentation before merging.
• These changes may need ported to the appropriate branches (list here):

Further comments

[github-actions]

Build Details

Name	Information
PR Commit	79b6c3d
Merge Commit	54e6cac
Size	3.75 MB
Last Updated By	jace-roell
Last Updated	Jan 28, 26, 3:16:46 PM UTC
Expires At	Apr 28, 26, 3:15:04 PM UTC

[codecov]

Codecov Report

❌ Patch coverage is 95.98394% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.27%. Comparing base (ea8c248) to head (79b6c3d).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...we-explorer/src/trees/dataset/DatasetFSProvider.ts	93.33%	6 Missing ⚠️
...ckages/zowe-explorer-api/src/utils/FeatureFlags.ts	95.16%	3 Missing ⚠️
...kages/zowe-explorer/src/trees/shared/SharedInit.ts	66.66%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3963      +/-   ##
==========================================
+ Coverage   94.24%   94.27%   +0.02%     
==========================================
  Files         129      130       +1     
  Lines       16169    16372     +203     
  Branches     3723     3815      +92     
==========================================
+ Hits        15239    15435     +196     
- Misses        928      935       +7     
  Partials        2        2              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[slavek-kucera]

With the fetch by default is enabled, is there still a missed opportunity to share the request when two different callers issue stat (and other calls) for the same resource - one is with fetch=true while the other does not provider the fetch parameter?

[jace-roell]

As noted in the PR description, the behavior for fetch=true remains unchanged. The fetchByDefault logic is an optimization of the default behavior (fetch=false); therefore, to take advantage of this new lookup logic, the fetch=true query parameter must be removed from extender calls.

Due to this, the request reuse case would be when fetch=false and no fetch parameter is present, which is currently implemented.

[slavek-kucera]

The scenario is

• fetchByDefault enabled
• cache is empty

two requests are issued by (possibly different extenders):

• fetch=true (because the caller 1 wants up-to-date results)
• without fetch parameter (caller 2 just wants to know the info and does not require the most up-to-date version)

The result will be two concurrent calls to the backend for the same information.

[jace-roell]

I see

the current implementation does not have the granularity to separate the remoteLookup and lookup call since they are both within the wrapped promise. It would be a large refactor to separate the calls such that the fetch=false fallback remoteLookup call and fetch=true remoteLookup call would be reused.

[t1m0thyj]

Implementation LGTM, thanks @jace-roell! Will approve once debug statements are removed.

[zFernand0]

Changes MSTM! 😋

Left a small comment asking for some tsdoc, but everything LGTM! 🙏

Will do some testing before I approve 🥳