Skip to content

Relese GCSFuse v3.8.0 in CSI driver#1264

Merged
uriel-guzman merged 1 commit intoGoogleCloudPlatform:mainfrom
uriel-guzman:deprecate-type-cache
Mar 19, 2026
Merged

Relese GCSFuse v3.8.0 in CSI driver#1264
uriel-guzman merged 1 commit intoGoogleCloudPlatform:mainfrom
uriel-guzman:deprecate-type-cache

Conversation

@uriel-guzman
Copy link
Copy Markdown
Collaborator

@uriel-guzman uriel-guzman commented Mar 16, 2026

What type of PR is this?
/kind feature

What this PR does / why we need it:
Release GCSFuse v3.8.0 in the CSI driver.

GCSFuse deprecated type cache and merge it to stat cache in v3.8.0, so we should stop calculating this recommendation in profiles.

Flat buckets increase their upper bound for average stat cache per file from 1500 to 1700 bytes (because it merged with type cache), so a new HNS annotation has been added in order to make this differentiation.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Manual verification

Flat bucket

PV scanned successfully with correct annotations:

...
Annotations:     gke-gcsfuse/bucket-scan-hns-enabled: false
                 gke-gcsfuse/bucket-scan-last-updated-time: 2026-03-19T16:35:26Z
                 gke-gcsfuse/bucket-scan-location-type: region
                 gke-gcsfuse/bucket-scan-num-objects: 16
                 gke-gcsfuse/bucket-scan-status: completed
                 gke-gcsfuse/bucket-scan-total-size-bytes: 16011557081
                 pv.kubernetes.io/bound-by-controller: yes
...
  Normal  ScanOperationStartSucceeded  3s    gke-gcsfuse-scanner  Started bucket scan for PV "my-pv", bucket "uriel-bucket", directory "", with timeout 5m0s
  Normal  ScanOperationSucceeded       2s    gke-gcsfuse-scanner  Bucket scan completed successfully for bucket "uriel-bucket", directory "" (0s): 16 objects, 16011557081 bytes

Recommendation:

    "inputSignals": {
      "bucketLocationType": "region",
      "nodeAllocatableMemoryBytes": 29775454208,
      "fuseBudgetEphemeralStorageBytes": 40001060756,
      "sidecarLimitMemoryBytes": 0,
      "fuseBudgetMemoryBytes": 20842790745,
      "nodeHasEphemeralStorageLSSD": false,
      "nodeAllocatableEphemeralStorageBytes": 47060071478,
      "requiredMetadataStatCacheBytes": 27200, # 16 * 1700
      "bucketTotalDataSizeBytes": 16011557081,
      "sidecarLimitEphemeralStorageBytes": 0,
      "bucketHNSEnabled": false, # Flat bucket
      "requiredFileCacheBytes": 16011557081,
      "bucketTotalObjects": 16,
      "nodeType": "general_purpose"
      # No type cache
    },
    "decision": {
      "fileCacheMedium": "ram",
      "fileCacheBytes": 16011557081,
     "metadataStatCacheBytes": 27200 # 16 * 1700 
     # No type cache
    }

HNS bucket

PV scan successful

Name:            my-pv
Labels:          <none>
Annotations:     gke-gcsfuse/bucket-scan-hns-enabled: true
                 gke-gcsfuse/bucket-scan-last-updated-time: 2026-03-19T16:49:43Z
                 gke-gcsfuse/bucket-scan-location-type: region
                 gke-gcsfuse/bucket-scan-num-objects: 1
                 gke-gcsfuse/bucket-scan-status: completed
                 gke-gcsfuse/bucket-scan-total-size-bytes: 10737418240
                 pv.kubernetes.io/bound-by-controller: yes
...
  Normal  ScanOperationStartSucceeded  5s    gke-gcsfuse-scanner  Started bucket scan for PV "my-pv", bucket "gargnitin-sptest-hns-usc1", directory "", with timeout 5m0s
  Normal  ScanOperationSucceeded       5s    gke-gcsfuse-scanner  Bucket scan completed successfully for bucket "gargnitin-sptest-hns-usc1", directory "" (0s): 1 objects, 10737418240 bytes

Recommendation log:

    "decision": {
      "metadataStatCacheBytes": 1500, # 1 * 1500
      "fileCacheMedium": "ram",
      "fileCacheBytes": 10737418240
    },
    "inputSignals": {
      "requiredFileCacheBytes": 10737418240,
      "fuseBudgetEphemeralStorageBytes": 40001060756,
      "requiredMetadataStatCacheBytes": 1500, # 1 * 1500
      "bucketHNSEnabled": true, # HNS detected
      "nodeHasEphemeralStorageLSSD": false,
      "bucketTotalObjects": 1,
      "fuseBudgetMemoryBytes": 20842816445,
      "nodeAllocatableMemoryBytes": 29775454208,
      "sidecarLimitMemoryBytes": 0,
      "bucketLocationType": "region",
      "nodeAllocatableEphemeralStorageBytes": 47060071478,
      "nodeType": "general_purpose",
      "sidecarLimitEphemeralStorageBytes": 0,
      "bucketTotalDataSizeBytes": 10737418240
    },

Does this PR introduce a user-facing change?:

Release GCSFuse v3.8.0 in CSI driver and type cache from GKE GCSFuse Profiles

@google-oss-prow
Copy link
Copy Markdown

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@gemini-code-assist
Copy link
Copy Markdown

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on simplifying the cache recommendation logic by completely deprecating and removing the metadata type cache. This change streamlines the codebase, reduces complexity, and ensures that the system only manages relevant caching mechanisms. Additionally, Go workspace files have been introduced, which will facilitate better module management and development practices.

Highlights

  • Deprecation of Metadata Type Cache: The metadataTypeCacheBytes and all associated constants, fields, and calculation logic have been completely removed from the cache recommendation system. This simplifies the codebase by eliminating an unused or deprecated caching mechanism.
  • Refactoring of Cache Recommendation Logic: The core cache recommendation function recommendMetadataCacheSize was renamed to recommendMetadataStatCacheSize and streamlined to focus exclusively on the metadata stat cache. The overall recommendCacheConfigs function was also simplified by removing type cache-specific override handling.
  • Test Suite Updates: Corresponding test cases and constants in recommender_test.go and profiles.go were updated to remove all references and assertions related to the metadata type cache, ensuring the test suite aligns with the updated functionality.
  • Go Workspace Introduction: New go.work and go.work.sum files were added, indicating the adoption of Go workspaces for managing modules within the repository, likely to improve development and testing workflows.
Changelog
  • go.work
    • Added a Go workspace file to define modules used in the project.
  • go.work.sum
    • Added a Go workspace sum file to record cryptographic checksums of module dependencies.
  • pkg/profiles/recommender.go
    • Removed metadataTypeCacheBytesPerObject constant.
    • Removed typeCacheConfigFileKey and typeCacheCLIKey constants.
    • Removed metadataTypeCacheBytes field from cacheRequirements and cacheOverrides structs.
    • Removed overridableCalc struct and related logic.
    • Removed metadataTypeCacheBytes field from recommendation struct.
    • Removed logging and calculation logic related to metadataTypeCacheBytes in logRecommendation.
    • Removed RequiredMetadataTypeCacheBytes from logEntry.InputSignals.
    • Removed MetadataTypeCacheBytes from logEntry.Decision.
    • Removed metadata-cache:type-cache-max-size-mb from comments and mount option processing.
    • Simplified recommendCacheConfigs by removing type cache specific calculation and override handling.
    • Renamed recommendMetadataCacheSize to recommendMetadataStatCacheSize and updated its parameters and internal logging messages to be specific to stat cache.
    • Removed type cache specific logic from parseCacheOverrides and mergeMountOptionsOnMissingKeys.
  • pkg/profiles/recommender_test.go
    • Removed metadataTypeCacheCapacity from defaultSCParams and allSCParams.
    • Updated TestBuildProfileConfig and TestBuildSCDetails test cases to remove metadataTypeCacheCapacity.
    • Updated TestBuildCacheRequirements test cases to remove metadataTypeCacheBytes from expected cacheRequirements.
    • Renamed TestRecommendMetadataCacheSize to TestRecommendMetadataStatCacheSize and updated calls to recommendMetadataStatCacheSize.
    • Removed objPerType and reqType constants.
    • Updated TestRecommendCacheConfigs test cases to remove metadataTypeCacheBytes from expected recommendation and adjust memoryBytes calculations.
    • Removed test cases specifically for type cache.
    • Updated TestMergeRecommendedMountOptionsOnMissingKeys test cases to remove metadata-cache:type-cache-max-size-mb from expected options and adjust memoryBytes calculations.
    • Removed test cases related to metadata type cache max size mb from TestMergeMountOptionsOnMissingKeys.
  • test/e2e/testsuites/profiles.go
    • Removed metadataTypeCacheMaxSizeMiBMountOptionKey constant.
    • Removed metadataTypeCacheMaxSizeMiBMountOptionKey from VerifyMountOptionsArePassedWithConfigFormat and expectedSubstrings in test cases.
    • Removed metadataTypeCacheMaxSizeMiBMountOptionKey from modifyPVForProfiles.
Activity
  • No significant activity has been recorded on this pull request yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@uriel-guzman uriel-guzman force-pushed the deprecate-type-cache branch from 13180cd to ce6f48e Compare March 16, 2026 22:29
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request effectively deprecates and removes the 'type cache' feature from the GCS FUSE recommender. The changes are comprehensive, touching the core logic, tests, and E2E tests to remove all traces of the type cache. The removal of the overridableCalc struct and related logic in recommendCacheConfigs simplifies the code for cache size calculation significantly, which is a great improvement for maintainability. The test suite has been thoroughly updated to reflect these changes. Overall, the changes are solid, with one minor suggestion for improving code clarity.

@uriel-guzman uriel-guzman force-pushed the deprecate-type-cache branch from ce6f48e to ab23c0c Compare March 17, 2026 15:36
@uriel-guzman uriel-guzman changed the title Deprecate type cache Deprecate type cache from GKE GCSFuse Profiles Mar 17, 2026
@uriel-guzman
Copy link
Copy Markdown
Collaborator Author

/gemini review

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request effectively deprecates and removes the 'type cache' from the GKE GCSFuse Profiles feature, aligning with its upcoming deprecation in GCSFuse v3.8.0. The changes are consistently applied across the recommender logic, unit tests, and E2E tests. The removal of the type cache logic has also led to a welcome simplification of the codebase, particularly in how cache size recommendations and overrides are handled. The changes are correct and improve the maintainability of the code. I have one minor suggestion to improve a test's error message for consistency.

@uriel-guzman uriel-guzman force-pushed the deprecate-type-cache branch 3 times, most recently from 36367ad to a5b4ceb Compare March 19, 2026 16:22
@uriel-guzman uriel-guzman changed the title Deprecate type cache from GKE GCSFuse Profiles Relese GCSFuse v3.8.0 in CSI driver Mar 19, 2026
@uriel-guzman uriel-guzman force-pushed the deprecate-type-cache branch from a5b4ceb to 6bb7f6d Compare March 19, 2026 16:52
@google-oss-prow
Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: amacaskill, uriel-guzman

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@uriel-guzman uriel-guzman marked this pull request as ready for review March 19, 2026 21:26
@uriel-guzman uriel-guzman force-pushed the deprecate-type-cache branch from 6bb7f6d to 5cc063b Compare March 19, 2026 21:29
@google-oss-prow
Copy link
Copy Markdown

New changes are detected. LGTM label has been removed.

@google-oss-prow google-oss-prow bot removed the lgtm label Mar 19, 2026
@uriel-guzman uriel-guzman merged commit 154b98e into GoogleCloudPlatform:main Mar 19, 2026
7 of 9 checks passed
uriel-guzman added a commit that referenced this pull request Mar 20, 2026
…1264-upstream-release-1.22

Automated cherry pick of #1264: Relese GCSFuse v3.8.0 in CSI driver
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants