Request for MongoDB Data Purge Strategy and Version Upgrade

[kabilesh13]

Hi Team,

As we are deploying MongoDB within our Kubernetes cluster, I understand that all chaos workflow and results data are stored in this database, which will result in continuous data growth over time. Do we currently have a purging mechanism in place? Additionally, do we know which data can be safely deleted and the appropriate procedures or locations for performing these deletions?

I’ve also noticed that the latest MongoDB version is 8, while we are still operating on version 6. Is there a specific reason for this, or are there plans to upgrade?

Thank you.

[Debjyoti2004]

Hi @PriteshKiri, I’m interested in picking this up.

I have checked the migration path from MongoDB 6 to 8 and have a plan for the data purge strategy using TTL indexes. Could you please assign this issue to me so I can start the implementation?

[PriteshKiri]

Sure @Debjyoti2004

[PriteshKiri]

Looping @ispeakc0de and @Jonsy13 to share their thoughts.

[Debjyoti2004]

Hi @ispeakc0de @Jonsy13 @PriteshKiri ,

I have completed a local compatibility audit for the MongoDB 6.0 → 8.0 upgrade. Here are my findings and the proposed implementation plan:

1. Backend Compatibility (Verified)

I spun up a local 3-node mongo:8.0 replica set and ran the Authentication Server against it.

• Connection: The Go driver successfully connected to the 8.0 cluster.
• Initialization: Core collections (users, project, auth-config) were created automatically without errors.
• Runtime: The HTTP and gRPC servers started correctly on ports 3000/3030.

Conclusion: The current application logic and driver are safe for the upgrade.

2. Breaking Change: Tooling & Scripts

The mongo:8.0 image has removed the legacy mongo shell binary.

• Impact: Any CI/CD pipelines, local setup scripts, or documentation relying on the mongo command will fail (Error: executable file not found in $PATH).
• Fix: I verified that migrating these commands to mongosh works correctly for replica set initialization.

Implementation Plan

I am ready to open a PR to address this in three parts:

1. Version Bump: Update Docker manifests and Helm charts to use mongo:8.0.
2. Script Migration: Replace all instances of mongo with mongosh in the setup scripts/docs to fix the breaking change.
3. Data Purge (TTL Index): Implement the TTL Index strategy on the workflow results collection (configurable via a DATA_RETENTION_DAYS env var) to handle the automated data cleanup.

Please let me know if this plan aligns with your expectations, and I will proceed with the PR!