KUBE-176 - KUBE-177 : Worktree percluster operator docs#1391
Draft
Julien-Ben wants to merge 9 commits into
Draft
KUBE-176 - KUBE-177 : Worktree percluster operator docs#1391Julien-Ben wants to merge 9 commits into
Julien-Ben wants to merge 9 commits into
Conversation
Contributor
MCK 1.10.0 Release NotesNew Features
Bug Fixes
|
The kind tooling pre-installs istio 1.16, which cannot serve the mesh DNS prerequisite the docs assume (injected pods resolving Services that exist in only one cluster) on the current node images, and the e2e tests never notice because the operator mirrors its services into every cluster. Provide an installer that replaces it with a current release on the member clusters, same flat-network multi-primary topology, scoped to the ephemeral clusters of the task that runs it.
… kind Runs the docs/search/12 runbook end to end on the multi-cluster kind environment: the ra-* prerequisite suites (ra-01 and ra-03 come with the kind environment; the wrapper reinstalls a current istio and replicates the ra-03 namespace labels), then the scenario snippets verbatim. Manual patch trigger, public images flavor. Hardening baked in from the first green iterations: the variant depends on the kubectl-mongodb plugin build, ra-04 and ra-05 are retried while the young mesh and the cert-manager webhook come up (resetting the snippet run log so retries re-execute), and the task exec timeout is raised to 3h for the OM install.
Julien-Ben
force-pushed
the
worktree-percluster-operator-docs
branch
from
July 16, 2026 17:06
e3fc245 to
319a0d5
Compare
Julien-Ben
added
the
skip-changelog
Julien-Ben
changed the title
The Evergreen task hosts cannot resolve mesh DNS names for Services existing in only one cluster, with istio 1.16 and 1.30 alike and across kind node images, while the same setup passes on other hosts and on GKE. ra-04 checks exactly that prerequisite, so it is unsatisfiable on this substrate; skip it with the reasoning documented in the wrapper (the GKE snippets variant still runs it verbatim). Cover the one name that needs remote resolution, om-svc during ra-06, by mirroring it into the other member clusters as a selectorless Service plus EndpointSlice pointing at cluster 0's clusterIP, the pattern validated in the manual kind run.
The steps referenced snippets by path inside bash fences, which GitHub does not link. Add a link line after each fence so readers can jump to the exact script; content still lives only in the snippet files.
Review findings on the runbook snippets: 12_0400 reported success even when its patch matched no automation config process and never waited for the agents to converge, so a naming mismatch only surfaced two steps later as empty query results. Port the naming check and the goal version wait from 13_0330, and make that wait fail on timeout in both. Guard both scripts against an unassigned om-svc-ext LoadBalancer IP, which produced a misleading could-not-resolve-project-id error. Skip index creation in 12_0530 when the index already exists so the step can be re-run after a partial failure.
The repo convention reserves internal in a snippet filename for CI-only assertion scripts that no documentation references. These three are fully documented steps in the README, and their scenario 12 counterparts carry no internal marker, so align the names with the convention.
A three-way review (pedagogy, walkthrough, accuracy) against the CE/TSE audience found the guides mechanically correct but missing the mental model in places. Define the load-bearing concepts (reconcile loop, Automation Config, the proxy/Envoy tier, mongotHost vs searchIndexManagementHostAndPort) in scenario 12's How It Works and move that section above the recognition table it feeds. Add the missing failure modes to both troubleshooting tables (cross-cluster DNS, pending LoadBalancer IP, the CA-ConfigMap presence-check quirk as its own row), success criteria to the steps that lacked them, the exact env sourcing order, and glossary entries. Fix stale code citations (the mongodbsearch_types.go line numbers drifted) and stop overstating scenario 13's e2e parity: the e2e parks the sharded source on a third operator-free cluster, this doc co-locates it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
FOR REVIEW:
The easiest way to see the guide rendered is to open it in the branch URL (the diff will be heard to read):
https://github.com/mongodb/mongodb-kubernetes/blob/worktree-percluster-operator-docs/docs/search/12-search-percluster-operator-rs/README.md
MongoDB Search can run with an independent Search operator in every member cluster while the central operator keeps managing the source deployment. Field teams have nothing on this topology today: how to recognize it at a customer, deploy it, or troubleshoot it. This PR adds two internal runbooks made of executable snippets, plus a manual-trigger Evergreen variant that verifies them:
docs/search/12-search-percluster-operator-rs/: per-cluster Search operators and a unified MongoDBSearch CR over a multi-cluster replica set.docs/search/13-search-percluster-operator-sharded/: the same topology over a sharded source.public_kind_multi_cluster_code_snippets: Evergreen variant (patch-only) that builds the ra-02..ra-07 prerequisites and runs scenario 12 end to end on multi-cluster kind.How to read: open
docs/search/12-search-percluster-operator-rs/on GitHub and the README renders as the guide: a recognition table ("is my customer running this?"), a How It Works section with a topology diagram and resource-name decoding, then numbered steps that each link to the exact snippet they run, ending with troubleshooting and a glossary. The snippet files are the single source of truth; the README references them by link, so the two cannot drift.Proof of Work
$searchand$vectorSearchquery snippets.Checklist
skip-changeloglabel if not needed