diff --git a/docs/content/.pages b/docs/content/.pages new file mode 100644 index 0000000..830682c --- /dev/null +++ b/docs/content/.pages @@ -0,0 +1,7 @@ +nav: + - Home: README.md + - Getting Started: getting-started.md + - Publishing Resources: publish-resources + - Consuming Services: consuming-services.md + - FAQ: faq.md + - Contributing: contributing diff --git a/docs/content/README.md b/docs/content/README.md index fa6d63a..14d3faf 100644 --- a/docs/content/README.md +++ b/docs/content/README.md @@ -109,7 +109,7 @@ able to delete and recreate objects to follow this GVK change, which is a level simply do not want to deal with at this point in time. Also, `APIResourceSchemas` are immutable themselves. -More information is available in the [Publishing Resources](./publish-resources.md) guide. +More information is available in the [Publishing Resources](./publish-resources/index.md) guide. ### APIExports diff --git a/docs/content/releasing.md b/docs/content/contributing/releasing.md similarity index 100% rename from docs/content/releasing.md rename to docs/content/contributing/releasing.md diff --git a/docs/content/getting-started.md b/docs/content/getting-started.md index 515f93f..2e78883 100644 --- a/docs/content/getting-started.md +++ b/docs/content/getting-started.md @@ -39,6 +39,32 @@ $ kubectl create --filename apiexport.yaml apiexport/test.example.com created ``` +### Optional: Create Initial APIBinding + +To save resources, kcp doesn't start API endpoints for `APIExports` that are not in use (i.e. that don't +have an active `APIBinding`). To avoid cryptic errors in the Sync Agent logs about resources not being found, +you can create an initial `APIBinding` in the same (or another) workspace as your `APIExport`. + +It could look like this: + +```yaml +apiVersion: apis.kcp.io/v1alpha1 +kind: APIBinding +metadata: + name: test.example.com +spec: + reference: + export: + name: test.example.com +``` + +While still being in your `:workspace:you:want:to:create:it` workspace, you could create the `APIBinding` like this: + +```sh +$ kubectl create --filename apibinding.yaml +apibinding/test.example.com created +``` + ## Sync Agent Installation The Sync Agent can be installed into any namespace, but in our example we are going with `kcp-system`. @@ -47,7 +73,7 @@ to, but that is the common setup. Ultimately the Sync Agent synchronizes data be endpoints. Now that the `APIExport` is created, switch to the Kubernetes cluster from which you wish to -[publish resources](publish-resources.md). You will need to ensure that a kubeconfig with access to +[publish resources](./publish-resources/index.md). You will need to ensure that a kubeconfig with access to the kcp workspace that the `APIExport` has been created in is stored as a `Secret` on this cluster. Make sure that the kubeconfig points to the right workspace (not necessarily the `root` workspace). @@ -237,7 +263,7 @@ subjects: ## Publish Resources Once the Sync Agent Pods are up and running, you should be able to follow the -[Publishing Resources](publish-resources.md) guide. +[Publishing Resources](./publish-resources/index.md) guide. ## Consume Service diff --git a/docs/content/publish-resources/.pages b/docs/content/publish-resources/.pages new file mode 100644 index 0000000..3caec12 --- /dev/null +++ b/docs/content/publish-resources/.pages @@ -0,0 +1,4 @@ +nav: + - index.md + - api-lifecycle.md + - technical-details.md diff --git a/docs/content/api-lifecycle.md b/docs/content/publish-resources/api-lifecycle.md similarity index 100% rename from docs/content/api-lifecycle.md rename to docs/content/publish-resources/api-lifecycle.md diff --git a/docs/content/publish-resources.md b/docs/content/publish-resources/index.md similarity index 77% rename from docs/content/publish-resources.md rename to docs/content/publish-resources/index.md index 971b904..edb89fa 100644 --- a/docs/content/publish-resources.md +++ b/docs/content/publish-resources/index.md @@ -491,124 +491,3 @@ spec: # replacement: '...' ``` -## Technical Details - -The following sections go into more details of the behind the scenes magic. - -### Synchronization - -Even though the whole configuration is written from the standpoint of the service owner, the actual -synchronization logic considers the kcp side as the canonical source of truth. The Sync Agent -continuously tries to make the local objects look like the ones in kcp, while pushing status updates -back into kcp (if the given `PublishedResource` (i.e. CRD) has a `status` subresource enabled). - -### Local <-> Remote Connection - -The Sync Agent tries to keep sync-related metadata on the service cluster, away from the consumers. -This is both to prevent vandalism and to hide implementation details. - -To ensure stability against future changes, once the Sync Agent has determined how a local object -should be named, it will remember this decision in the object's metadata. This is so that on future -reconciliations, the (potentially costly, but probably not) renaming logic does not need to be -applied again. This allows the Sync Agent to change defaults and also allows the service owner to make -changes to the naming rules without breaking existing objects. - -Since we do not want to store metadata on the kcp side, we instead rely on label selectors on -the local objects. Each object on the service cluster has a label for the remote cluster name, -namespace and object name, and when trying to find the matching local object, the Sync Agent simply -does a label-based search. - -There is currently no sync-related metadata available on source objects (in kcp workspaces), as this -would either be annotations (untyped strings...) or require schema changes to allow additional -fields in basically random CRDs. - -Note that fields like `generation` or `resourceVersion` are not relevant for any of the sync logic. - -### Reconcile Loop - -The sync loop can be divided into 5 parts: - -1. find the local object -2. handle deletion -3. ensure the destination object exists -4. ensure the destination object's content matches the source object -5. synchronize related resources the same way (repeat 1-4 for each related resource) - -#### Phase 1: Find the Local Object - -For this, as mentioned in the connection chapter above, the Sync Agent tries to follow label selectors -on the service cluster. This helps prevent cluttering with consumer workspaces with sync metadata. -If no object is found to match the labels, that's fine and the loop will continue with phase 2, -in which a possible Conflict error (if labels broke) is handled gracefully. - -The remote object in the workspace becomes the `source object` and its local equivalent on the -service cluster is called the `destination object`. - -#### Phase 2: Handle Deletion - -A finalizer is used in the kcp workspaces to prevent orphans in the service cluster side. This -is the only real evidence in the kcp side that the Sync Agent is even doing things. When a remote -(source) object is deleted, the corresponding local object is deleted as well. Once the local object -is gone, the finalizer is removed from the source object. - -#### Phase 3: Ensure Object Existence - -We have a source object and now need to create the destination. This chart shows what's happening. - -```mermaid -graph TB - A(source object):::state --> B([cleanup if in deletion]):::step - B --> C([ensure finalizer on source object]):::step - C --> D{exists local object?} - - D -- yes --> I("continue with next phase…"):::state - D -- no --> E([apply projection]):::step - - subgraph "ensure dest object exists" - E --> G([ensure resulting namespace exists]):::step - G --> H([create local object]):::step - H --> H_err{Errors?} - H_err -- Conflict --> J([attempt to adopt existing object]):::step - end - - H_err -- success --> I - J --> I - - classDef step color:#77F - classDef state color:#F77 -``` - -After we followed through with these steps, both the source and destination objects exists and we -can continue with phase 4. - -Resource adoption happens when creation of the initial local object fails. This can happen when labels -get mangled. If such a conflict happens, the Sync Agent will "adopt" the existing local object by -adding / fixing the labels on it, so that for the next reconciliation it will be found and updated. - -#### Phase 4: Content Synchronization - -Content synchronization is rather simple, really. - -First the source "spec" is used to patch the local object. Note that this step is called "spec", but -should actually be called "all top-level elements besides `apiVersion`, `kind`, `status` and -`metadata`, but still including some labels and annotations"; so if you were to publish RBAC objects, -the syncer would include `roleRef` field, for example). - -To allow proper patch generation, the last known state is kept on the local object, similar to how -`kubectl` creates an annotation for it. This is required for the Sync Agent to properly detect changes -made by mutation webhooks on the service cluster. - -If the published resource (CRD) has a `status` subresource enabled (not just a `status` field in its -scheme, it must be a real subresource), then the Sync Agent will copy the status from the local object -back up to the remote (source) object. - -#### Phase 5: Sync Related Resources - -The same logic for synchronizing the main published resource applies to their related resources as -well. The only difference is that the source side can be either remote (workspace) or local -(service cluster). - -Since the Sync Agent tries its best to keep sync-related data out of kcp workspaces, the last known -state for related resources is _not_ kept together with the destination object in the kcp workspaces. -Instead all known states (from the main object and all related resources) is kept in a single Secret -on the service cluster side. diff --git a/docs/content/publish-resources/technical-details.md b/docs/content/publish-resources/technical-details.md new file mode 100644 index 0000000..4c0a2e1 --- /dev/null +++ b/docs/content/publish-resources/technical-details.md @@ -0,0 +1,121 @@ +# Technical Details + +The following sections go into more details of the behind the scenes magic. + +## Synchronization + +Even though the whole configuration is written from the standpoint of the service owner, the actual +synchronization logic considers the kcp side as the canonical source of truth. The Sync Agent +continuously tries to make the local objects look like the ones in kcp, while pushing status updates +back into kcp (if the given `PublishedResource` (i.e. CRD) has a `status` subresource enabled). + +## Local <-> Remote Connection + +The Sync Agent tries to keep sync-related metadata on the service cluster, away from the consumers. +This is both to prevent vandalism and to hide implementation details. + +To ensure stability against future changes, once the Sync Agent has determined how a local object +should be named, it will remember this decision in the object's metadata. This is so that on future +reconciliations, the (potentially costly, but probably not) renaming logic does not need to be +applied again. This allows the Sync Agent to change defaults and also allows the service owner to make +changes to the naming rules without breaking existing objects. + +Since we do not want to store metadata on the kcp side, we instead rely on label selectors on +the local objects. Each object on the service cluster has a label for the remote cluster name, +namespace and object name, and when trying to find the matching local object, the Sync Agent simply +does a label-based search. + +There is currently no sync-related metadata available on source objects (in kcp workspaces), as this +would either be annotations (untyped strings...) or require schema changes to allow additional +fields in basically random CRDs. + +Note that fields like `generation` or `resourceVersion` are not relevant for any of the sync logic. + +## Reconcile Loop + +The sync loop can be divided into 5 parts: + +1. find the local object +2. handle deletion +3. ensure the destination object exists +4. ensure the destination object's content matches the source object +5. synchronize related resources the same way (repeat 1-4 for each related resource) + +### Phase 1: Find the Local Object + +For this, as mentioned in the connection chapter above, the Sync Agent tries to follow label selectors +on the service cluster. This helps prevent cluttering with consumer workspaces with sync metadata. +If no object is found to match the labels, that's fine and the loop will continue with phase 2, +in which a possible Conflict error (if labels broke) is handled gracefully. + +The remote object in the workspace becomes the `source object` and its local equivalent on the +service cluster is called the `destination object`. + +### Phase 2: Handle Deletion + +A finalizer is used in the kcp workspaces to prevent orphans in the service cluster side. This +is the only real evidence in the kcp side that the Sync Agent is even doing things. When a remote +(source) object is deleted, the corresponding local object is deleted as well. Once the local object +is gone, the finalizer is removed from the source object. + +### Phase 3: Ensure Object Existence + +We have a source object and now need to create the destination. This chart shows what's happening. + +```mermaid +graph TB + A(source object):::state --> B([cleanup if in deletion]):::step + B --> C([ensure finalizer on source object]):::step + C --> D{exists local object?} + + D -- yes --> I("continue with next phase…"):::state + D -- no --> E([apply projection]):::step + + subgraph "ensure dest object exists" + E --> G([ensure resulting namespace exists]):::step + G --> H([create local object]):::step + H --> H_err{Errors?} + H_err -- Conflict --> J([attempt to adopt existing object]):::step + end + + H_err -- success --> I + J --> I + + classDef step color:#77F + classDef state color:#F77 +``` + +After we followed through with these steps, both the source and destination objects exists and we +can continue with phase 4. + +Resource adoption happens when creation of the initial local object fails. This can happen when labels +get mangled. If such a conflict happens, the Sync Agent will "adopt" the existing local object by +adding / fixing the labels on it, so that for the next reconciliation it will be found and updated. + +### Phase 4: Content Synchronization + +Content synchronization is rather simple, really. + +First the source "spec" is used to patch the local object. Note that this step is called "spec", but +should actually be called "all top-level elements besides `apiVersion`, `kind`, `status` and +`metadata`, but still including some labels and annotations"; so if you were to publish RBAC objects, +the syncer would include `roleRef` field, for example). + +To allow proper patch generation, the last known state is kept on the local object, similar to how +`kubectl` creates an annotation for it. This is required for the Sync Agent to properly detect changes +made by mutation webhooks on the service cluster. + +If the published resource (CRD) has a `status` subresource enabled (not just a `status` field in its +scheme, it must be a real subresource), then the Sync Agent will copy the status from the local object +back up to the remote (source) object. + +### Phase 5: Sync Related Resources + +The same logic for synchronizing the main published resource applies to their related resources as +well. The only difference is that the source side can be either remote (workspace) or local +(service cluster). + +Since the Sync Agent tries its best to keep sync-related data out of kcp workspaces, the last known +state for related resources is _not_ kept together with the destination object in the kcp workspaces. +Instead all known states (from the main object and all related resources) is kept in a single Secret +on the service cluster side. diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 8ad481d..3a279a1 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -17,16 +17,6 @@ repo_url: https://github.com/kcp-dev/api-syncagent repo_name: kcp-dev/api-syncagent site_url: https://docs.kcp.io/api-syncagent/ -# Site navigation -nav: - - Home: README.md - - Getting Started: getting-started.md - - Publishing Resources: publish-resources.md - - Consuming Services: consuming-services.md - - API Lifecycle: api-lifecycle.md - - FAQ: faq.md - - Release Process: releasing.md - # Site content docs_dir: 'content' # Where to generate