Support N MongoDB shards #2219

ghost · 2025-05-06T11:04:51Z

Here, we only support deployment of multiple shards. Upgrades is managed by upper layers for now.

bert-e · 2025-05-06T11:04:55Z

Hello williamlardier,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
`/after_pull_request`	Wait for the given pull request id to be merged before continuing with the current one.
`/bypass_author_approval`	Bypass the pull request author's approval	⭐
`/bypass_build_status`	Bypass the build and test status	⭐
`/bypass_commit_size`	Bypass the check on the size of the changeset `TBA`	⭐
`/bypass_incompatible_branch`	Bypass the check on the source branch prefix	⭐
`/bypass_jira_check`	Bypass the Jira issue check	⭐
`/bypass_peer_approval`	Bypass the pull request peers' approval	⭐
`/bypass_leader_approval`	Bypass the pull request leaders' approval	⭐
`/approve`	Instruct Bert-E that the author has approved the pull request.		✍️
`/create_pull_requests`	Allow the creation of integration pull requests.
`/create_integration_branches`	Allow the creation of integration branches.
`/no_octopus`	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
`/unanimity`	Change review acceptance criteria from `one reviewer at least` to `all reviewers`
`/wait`	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
`/help`	Print Bert-E's manual in the pull request.
`/status`	Print Bert-E's current status in the pull request `TBA`
`/clear`	Remove all comments from Bert-E from the history `TBA`
`/retry`	Re-start a fresh build `TBA`
`/build`	Re-start a fresh build `TBA`
`/force_reset`	Delete integration branches & pull requests, and restart merge process from the beginning.
`/reset`	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

bert-e · 2025-05-06T11:05:29Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
2 peers

.github/scripts/end2end/configs/zenko.yaml

.github/scripts/end2end/generate-kustomization.sh

.github/workflows/end2end.yaml

monitoring/mongodb/alerts.yaml

francoisferrand · 2025-05-21T15:34:43Z

solution-base/mongodb/charts/mongodb-sharded/templates/shard/shard-data-statefulset.yaml

+            {{- if gt $.Values.shards 1 }}
+            shard: {{ $i | quote }}
+            {{- end }}
+{{ toYaml $.Values.shardsvr.persistence.selector.matchLabels | indent 12 }}


why change this line, we do not use this variable?

instead of patching the chart, could we not set this shardsvr.persistence.selector variable to the appropriate value to get the same result?

The variable is used, for instance we have this in the sts after deployment:

selector: matchLabels: app.kubernetes.io/name: mongodb app.kubernetes.io/part-of: zenko shard: "1"

But it's actually set from the build.sh file:

--set 'shardsvr.persistence.selector.matchLabels.app\.kubernetes\.io/part-of=zenko' \

This means, with the change, that we need to access the .matchLabels and not the parent variable. This is also something we use as a label selector in the artesca installer, so we cannot just remove it.

So in short:

we pass this info when rendering the yamls with helm

the patch is necessary because we need to add these labels for artesca

now that we also have the shard, we cannot just pass a static value, we must compute it in the yaml dynamically

this change forces us to access the .matchLabels

i'll check if we can add this to the values.yaml file

we don't use the values.yaml file (the one we have the default, part of the chart), but specify the values in argument to helm call

My comment was updated in between, indeed, but we must still specify the shard number, so we need to edit this part, and also, we want to keep the selectors we had already. So unless you think this is wrong, to me the change is fine?

solution-base/mongodb/charts/mongodb-sharded/templates/shard/shard-data-podmonitor.yaml

solution-base/build.sh

ghost · 2025-05-26T15:17:15Z

.github/scripts/end2end/install-kind-dependencies.sh

    sed -i "s/MONGODB_SHARDSERVER_RAM_LIMIT/${MONGODB_SHARDSERVER_RAM_LIMIT}/g" $DIR/_build/root/deploy/*
    sed -i "s/MONGODB_SHARDSERVER_RAM_REQUEST/${MONGODB_SHARDSERVER_RAM_REQUEST}/g" $DIR/_build/root/deploy/*
    sed -i "s/MONGODB_MONGOS_RAM_REQUEST/${MONGODB_MONGOS_RAM_REQUEST}/g" $DIR/_build/root/deploy/*
-


Note for reviewers: this was removed because we had this logic twice in the function, maybe a rebase issue at some point. See above.

.github/scripts/end2end/deploy-zenko.sh

.github/scripts/end2end/install-kind-dependencies.sh

ghost · 2025-06-05T07:32:25Z

Added a commit to remove the matchLabel for the shard0, so that we do not require any downtime during shard expansion.

VERSION

bert-e · 2025-07-01T10:08:57Z

Incorrect fix version

The Fix Version/s in issue ZENKO-4641 contains:

2.12.1

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

2.12.2

Please check the Fix Version/s of ZENKO-4641, or the target
branch of this pull request.

We add support for multiple shards. The Kustomization file is thus removed, and we instead generate it based on the current configuration. Issue: ZENKO-4641

Issue: ZENKO-4641

Added to Zenko as an annotation, but now compatible with multiple shards. Issue: ZENKO-4641

This implementation tries to use a single MongoDB mongod process per instance so that we maximize RAM usage and performances. Issue: ZENKO-4641

Selectors should now be updated to consider the current shard Issue: ZENKO-4641

Issue: ZENKO-4641

- Mutualize http tests in a single CI run. - All deployments must now be sharded, hence testing separately http endpoints is not enough. - We also mutualize the runner to enable HTTPs after the initial deployments. This reduces costs, and ensure basic set of tests are executed when using 2 shards. Issue: ZENKO-4641

We useed to re-run Vault functional tests in Zenko. This is not needed, as covered by CTST test suite now. Issue: ZENKO-4641

- We want to support one P-S-S topology ever 3 servers. - With 6+ nodes support added, we now need to ensure the replicas of configsvr & shardsvr are not exceeding 3. Issue: ZENKO-4641

Will be useful for CI testing only Issue: ZENKO-4641

Issue: ZENKO-4641

The previous alert was not properly accounting for multiple shards, leading to alerts with multi shards even if we have a nominal state. Issue: ZENKO-4641

Issue: ZENKO-4641

To avoid having to delete the STS for existing deployments, we must avoid changing the matchSelectors for the shard 0. Removing only the shard selector is not enough, as we risk selecting volumes from other shards. Instead, we choose to modify the app name to include, only for new shards, the new shard ID. Shard 0 won't be updated, and the new shards will have their own labels. Issue: ZENKO-4641

ghost · 2025-07-01T13:10:14Z

/approve

bert-e · 2025-07-01T14:58:34Z

I have successfully merged the changeset of this pull request
into targetted development branches:

✔️ development/2.12

The following branches have NOT changed:

development/2.10
development/2.11
development/2.5
development/2.6
development/2.7
development/2.8
development/2.9

Please check the status of the associated issue ZENKO-4641.

Goodbye williamlardier.

The following options are set: approve

ghost changed the base branch from development/2.11 to development/2.12 May 6, 2025 11:05

ghost deleted a comment from bert-e May 6, 2025

ghost force-pushed the feature/ZENKO-4641-v2 branch 5 times, most recently from 365e3d4 to b1d1cdc Compare May 19, 2025 10:08

ghost requested review from benzekrimaha, francoisferrand and welansari May 20, 2025 08:42

ghost mentioned this pull request May 21, 2025

Feature/zenko 4641 #2183

Closed

francoisferrand reviewed May 21, 2025

View reviewed changes

ghost force-pushed the feature/ZENKO-4641-v2 branch 9 times, most recently from d3be7fc to d3f8a9d Compare May 26, 2025 14:25

ghost commented May 26, 2025

View reviewed changes

francoisferrand reviewed May 27, 2025

View reviewed changes

.github/scripts/end2end/deploy-zenko.sh Outdated Show resolved Hide resolved

francoisferrand reviewed May 27, 2025

View reviewed changes

.github/scripts/end2end/install-kind-dependencies.sh Show resolved Hide resolved

francoisferrand reviewed May 27, 2025

View reviewed changes

.github/scripts/end2end/install-kind-dependencies.sh Show resolved Hide resolved

francoisferrand approved these changes May 27, 2025

View reviewed changes

ghost force-pushed the feature/ZENKO-4641-v2 branch from f1098b9 to fa66847 Compare May 27, 2025 09:21

ghost force-pushed the feature/ZENKO-4641-v2 branch from 9062ee2 to 440c8b5 Compare June 5, 2025 07:21

ghost force-pushed the feature/ZENKO-4641-v2 branch 2 times, most recently from 8944786 to ed4be0c Compare June 5, 2025 09:41

welansari approved these changes Jul 1, 2025

View reviewed changes

VERSION Outdated Show resolved Hide resolved

Seragonia added 18 commits July 1, 2025 15:07

Generate Kustomization dynamically

4d98c45

We add support for multiple shards. The Kustomization file is thus removed, and we instead generate it based on the current configuration. Issue: ZENKO-4641

Generate kustomization based on MONGODB_SHARD_COUNT

4bee0f7

Issue: ZENKO-4641

Generate and deploy N shards

7685c67

Issue: ZENKO-4641

Use the generated list of backbeat endpoints

62d9013

Added to Zenko as an annotation, but now compatible with multiple shards. Issue: ZENKO-4641

Ensure MongoDB pods are not in the same servers

62ec982

This implementation tries to use a single MongoDB mongod process per instance so that we maximize RAM usage and performances. Issue: ZENKO-4641

Update PV selectors

56ac511

Selectors should now be updated to consider the current shard Issue: ZENKO-4641

Add a 2-shard deployment test suite in the CI

b57e0ef

Issue: ZENKO-4641

Update patches

cd457e2

Issue: ZENKO-4641

Remove old vault test suite

db4da9e

We useed to re-run Vault functional tests in Zenko. This is not needed, as covered by CTST test suite now. Issue: ZENKO-4641

Ensure we do not deploy more than 3 replicas per sts

f56df2e

- We want to support one P-S-S topology ever 3 servers. - With 6+ nodes support added, we now need to ensure the replicas of configsvr & shardsvr are not exceeding 3. Issue: ZENKO-4641

Support 2 shards in 3 servers mode

eb8024d

Will be useful for CI testing only Issue: ZENKO-4641

Use latest bitnami chart repo

7189334

Issue: ZENKO-4641

Generate MongoDB configurations for 12 nodes

b4055f9

Issue: ZENKO-4641

Update MongoDBNotSynced alert

2710957

The previous alert was not properly accounting for multiple shards, leading to alerts with multi shards even if we have a nominal state. Issue: ZENKO-4641

Code improvements

82ffc1a

Issue: ZENKO-4641

Use safer yq to update mongodb images

b178c26

Issue: ZENKO-4641

ghost force-pushed the feature/ZENKO-4641-v2 branch from ed4be0c to 1f703a1 Compare July 1, 2025 13:08

ghost deleted a comment from bert-e Jul 1, 2025

bert-e merged commit 1f703a1 into development/2.12 Jul 1, 2025
14 checks passed

bert-e deleted the feature/ZENKO-4641-v2 branch July 1, 2025 14:58

Support N MongoDB shards #2219

Support N MongoDB shards #2219

Uh oh!

Conversation

ghost commented May 6, 2025

Uh oh!

bert-e commented May 6, 2025

Hello williamlardier,

Uh oh!

bert-e commented May 6, 2025

Waiting for approval

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

francoisferrand May 21, 2025

Choose a reason for hiding this comment

Uh oh!

ghost May 21, 2025 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

francoisferrand May 26, 2025

Choose a reason for hiding this comment

Uh oh!

ghost May 26, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ghost May 26, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ghost commented Jun 5, 2025

Uh oh!

Uh oh!

bert-e commented Jul 1, 2025

Incorrect fix version

Uh oh!

ghost commented Jul 1, 2025

Uh oh!

bert-e commented Jul 1, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ghost May 21, 2025 •

edited by ghost

Loading