Skip to content

Draft PR of deployment controller skeleton #611

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 24 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
20604cf
deployment controller skeleton init
Catoverflow Apr 14, 2025
4ea440d
add dc controller doc
Catoverflow Apr 16, 2025
a495a9a
add note on controlled rs
Catoverflow Apr 16, 2025
29e27f1
complete skeleton, model to be fulfulled
Catoverflow Apr 16, 2025
ce7ca93
add todo
Catoverflow Apr 16, 2025
cbce0d2
add corresponding model
Catoverflow Apr 17, 2025
e0823ad
evolve over the spec for dc
Catoverflow Apr 17, 2025
6a5da41
model for vdeployment controller complete, proceed to executable part
Catoverflow Apr 18, 2025
98f36dc
add exec part for vdeployment controller
Catoverflow Apr 18, 2025
28ebed7
update state machine model
Catoverflow Apr 21, 2025
a987730
hack vd dependency, bugs fix, update reconciler to match new state ma…
Catoverflow Apr 21, 2025
9359756
add clone trait for vrs, bugs fix for vd
Catoverflow Apr 21, 2025
0299447
bugs fix; mask Option<replicas>, reconcile_core postcondition for the…
Catoverflow Apr 22, 2025
58aaaea
bugs fix, unmask problems
Catoverflow Apr 22, 2025
228933b
changes made during the meeting
Catoverflow Apr 22, 2025
95fa36a
add Default for VRSSpec, move seq_filter_implies_contains to vstd_ext…
Catoverflow Apr 22, 2025
2319ab9
prove objects_to_vrs_list by reusing the proof for objects_to_pods
Catoverflow Apr 22, 2025
6a70c3c
get rid of old lemma
Catoverflow Apr 23, 2025
8be32a4
fed up with silly proofs
Catoverflow Apr 23, 2025
f0eed55
followup with the spec change
Catoverflow Apr 23, 2025
64affec
rewrite filter_old_and_new_vrs to follow the spec pattern, also try t…
Catoverflow Apr 23, 2025
3124431
no progress on make_replica_set
Catoverflow Apr 23, 2025
f89a421
fix filter_vrs_list and Clone trait for VRS
codyjrivera Apr 24, 2025
b4a1da4
bug fix of inconsistency between - and _
Catoverflow Apr 25, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions discussion/kubernetes-model/deployment_controller.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Deployment Controller (DC)

## Reconciliation Login

> In `pkg/controller/deployment/deployment_controller.go`

Reconciliation is performed by `syncDeployment`, which can be modeled as state machine:

0. Orphaning, adoption. This one is not considered in the model

1. [List all replicasets](https://github.com/kubernetes/kubernetes/blob/cdc807a9e849b651fb48c962cc18e25d39ec5edf/pkg/controller/deployment/deployment_controller.go#L629) (RS) owned by this DC from API Server

2. [Get all pods](https://github.com/kubernetes/kubernetes/blob/cdc807a9e849b651fb48c962cc18e25d39ec5edf/pkg/controller/deployment/deployment_controller.go#L638) managed by controlled RS and create a RS-pod map from API Server

3. Make decision on how to manage controlled RS:
x. Rollback, not considered in the model
Compare managed RS template and DC template,
**A**. if the number of replica is different, this is a [scaling event](https://github.com/kubernetes/kubernetes/blob/cdc807a9e849b651fb48c962cc18e25d39ec5edf/pkg/controller/deployment/deployment_controller.go#L665), just scale up/down the single newest controlled RS
**B**. The application managed* by DC should be updated (`spec.template.container`). Based on the configured update policy**:

​ I. [rollout](https://github.com/kubernetes/kubernetes/blob/cdc807a9e849b651fb48c962cc18e25d39ec5edf/pkg/controller/deployment/rolling.go#L31): (Create if non-existing and) scale up new RS, then scale down old RS
​ II. [recreate](https://github.com/kubernetes/kubernetes/blob/cdc807a9e849b651fb48c962cc18e25d39ec5edf/pkg/controller/deployment/recreate.go#L29): Scale down old RS, then (create if non-existing and) scale up the new RS.

---

*: The deployment controller will only control at most one new replica set, and multiple old replica sets

> In k8s implementation it's [possible](https://github.com/kubernetes/kubernetes/blob/cdc807a9e849b651fb48c962cc18e25d39ec5edf/pkg/controller/deployment/util/deployment_util.go#L633-L634) to have multiple new replica sets in rare cases

**: The scale up/down process should satisfy `maxSurge` and `maxUnavailable` properties of DC, which means DC may not scale the controlled new & old RS to desired replicas in one step. Instead DC will scale up & down gradually in turn. Currently we do not model this feature.

## Reconciliation State Machine

**Rolling Update**

```mermaid
graph LR
1((init))-->|list rs|2((list rs))
2-->|ε|3((roll rs))
3-->|scale new rs<br>scale old rs|3
3-->4((done))
```

**Rollout Update**

> we need to figure out how to add a "barrier" of waiting till all old pods stop running, then add new rs

```mermaid
graph LR
1((init))-->|list rs|2((after list rs))
2-->|list pod|3((after list pod))
3-->|ε|4((roll rs))
4-->|scale new rs<br>stop old rs|4
4-->|any old pods running<br>/all set|5((done))
```

2 changes: 1 addition & 1 deletion src/deps_hack/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -272,7 +272,7 @@ pub struct FluentBitConfigSpec {
}

#[derive(
kube::CustomResource, Debug, Clone, serde::Deserialize, serde::Serialize, schemars::JsonSchema,
kube::CustomResource, Default, Debug, Clone, serde::Deserialize, serde::Serialize, schemars::JsonSchema,
)]
#[kube(group = "anvil.dev", version = "v1", kind = "VReplicaSet")]
#[kube(shortname = "vrs", namespaced)]
Expand Down
1 change: 1 addition & 0 deletions src/kubernetes_api_objects/exec/label_selector.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ pub struct LabelSelector {
}

impl LabelSelector {

pub spec fn view(&self) -> LabelSelectorView;

#[verifier(external_body)]
Expand Down
7 changes: 7 additions & 0 deletions src/kubernetes_api_objects/exec/pod_template_spec.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,13 @@ pub struct PodTemplateSpec {
impl PodTemplateSpec {
pub spec fn view(&self) -> PodTemplateSpecView;

#[verifier(external_body)]
pub fn eq(&self, other: &Self) -> (b: bool)
ensures b == (self.view() == other.view())
{
self.inner == other.inner
}

#[verifier(external_body)]
pub fn default() -> (pod_template_spec: PodTemplateSpec)
ensures pod_template_spec@ == PodTemplateSpecView::default(),
Expand Down
1 change: 1 addition & 0 deletions src/v2/controllers/vdeployment_controller/exec/mod.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
pub mod reconciler;
Loading
Loading