|
| 1 | +# kube-exec-controller |
| 2 | +[](https://opensource.box.com/badges) |
| 3 | +[](https://app.travis-ci.com/box/kube-exec-controller) |
| 4 | +[](https://goreportcard.com/report/github.com/box/kube-exec-controller) |
| 5 | + |
| 6 | +kube-exec-controller is an [admission controller](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/) for handling container drift (caused by kubectl `exec`, `attach`, `cp`, or other interactive requests) inside a Kubernetes cluster. It runs as a Deployment and can be referred in a `ValidatingWebhookConfiguration` (see the provided [demo/](demo/) as an example) to detect and evict interacted Pods after a pre-defined interval. This project also includes a [kubectl plugin](https://kubernetes.io/docs/tasks/extend-kubectl/kubectl-plugins/), named `kubectl-pi` (*pod-interaction*), for checking such interacted Pods or extending their eviction time. |
| 7 | + |
| 8 | +Here is an overview of running a `kubectl exec` command in a K8s cluster with this admission controller service enabled: |
| 9 | + |
| 10 | + |
| 11 | + |
| 12 | +## Install |
| 13 | +#### Prerequisite |
| 14 | +- [Docker (17.05+)](https://www.docker.com/get-started) |
| 15 | +- [Kubernetes (1.16+)](https://kubernetes.io/) |
| 16 | +- [Kind (for local development only)](https://kind.sigs.k8s.io/) |
| 17 | + |
| 18 | +If you have a local K8s cluster up running, you can deploy kube-exec-controller and apply its validating admission webhooks simply by: |
| 19 | +``` |
| 20 | +$ git clone [email protected]:box/kube-exec-controller.git |
| 21 | +$ cd kube-exec-controller |
| 22 | +$ make deploy |
| 23 | +``` |
| 24 | + |
| 25 | +You should get a demo app and its admission webhooks deployed after the above `make deploy` command completes: |
| 26 | +``` |
| 27 | +$ kubectl get pod,service -n kube-exec-controller |
| 28 | +NAME READY STATUS RESTARTS AGE |
| 29 | +pod/demo-deploy-5d5cd95f94-jwf5b 1/1 Running 0 9s |
| 30 | +
|
| 31 | +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE |
| 32 | +service/demo-service ClusterIP 10.96.211.63 <none> 443/TCP 9s |
| 33 | +
|
| 34 | +$ kubectl get ValidatingWebhookConfiguration |
| 35 | +NAME WEBHOOKS AGE |
| 36 | +demo-validating-webhook-config 2 24s |
| 37 | +``` |
| 38 | + |
| 39 | +To see how kube-exec-controller works, let's create a test Pod in your local cluster and send a `kubectl exec` request to it: |
| 40 | +``` |
| 41 | +$ kubectl run test --image=nginx |
| 42 | +pod/test created |
| 43 | +
|
| 44 | +$ kubectl exec test -- touch new-file |
| 45 | +``` |
| 46 | + |
| 47 | +You will see the test Pod has some labels attached and receives corresponding K8s events from our controller app: |
| 48 | +``` |
| 49 | +$ kubectl get pod --show-labels |
| 50 | +NAME READY STATUS RESTARTS AGE LABELS |
| 51 | +test 1/1 Running 0 2s box.com/podInitialInteractionTimestamp=1634408037,box.com/podInteractorUsername=kubernetes-admin,box.com/podTTLDuration=2m0s,run=test |
| 52 | +
|
| 53 | +$ kubectl describe pod test |
| 54 | +... |
| 55 | +Warning PodInteraction 20s kube-exec-controller Pod was interacted with 'kubectl exec/attach' command by a user 'kubernetes-admin' initially at time 2021-10-16 18:04:44.5257517 +0000 UTC m=+27.185038701 |
| 56 | +Warning PodInteraction 21s kube-exec-controller Pod will be evicted at time 2021-10-16 18:06:44 +0000 UTC (in about 1m59s) |
| 57 | +``` |
| 58 | + |
| 59 | +You can also utilize the `kubectl pi` plugin to get more detailed info or request an extension to the test Pod's eviction time: |
| 60 | +``` |
| 61 | +$ kubectl pi get |
| 62 | +POD_NAME INTERACTOR POD_TTL EXTENSION EXTENSION_REQUESTER EVICTION_TIME |
| 63 | +test kubernetes-admin 2m0s 2021-10-16 18:06:44 +0000 UTC |
| 64 | +
|
| 65 | +$ kubectl pi extend --duration=1m |
| 66 | +Successfully extended the termination time of pod/test with a duration=1m |
| 67 | +
|
| 68 | +$ kubectl pi get |
| 69 | +POD_NAME INTERACTOR POD_TTL EXTENSION EXTENSION_REQUESTER EVICTION_TIME |
| 70 | +test kubernetes-admin 2m0s 1m kubernetes-admin 2021-10-16 18:07:44 +0000 UTC |
| 71 | +
|
| 72 | +$ kubectl describe pod test |
| 73 | +... |
| 74 | +Warning PodInteraction 30s kube-exec-controller Pod eviction time has been extended by '1m', as requested from user 'kubernetes-admin'. New eviction time: 2021-10-16 18:07:44 +0000 UTC |
| 75 | +Warning PodInteraction 30s kube-exec-controller Pod will be evicted at time 2021-10-16 18:07:44 +0000 UTC (in about 2m21s) |
| 76 | +``` |
| 77 | + |
| 78 | +## Usage |
| 79 | +#### kube-exec-controller |
| 80 | +``` |
| 81 | +$ kube-exec-controller --help |
| 82 | +Usage of kube-exec-controller: |
| 83 | + -api-server string |
| 84 | + URL to K8s api-server, required if kube-proxy is not set up |
| 85 | + -cert-path string |
| 86 | + Path to the PEM-encoded TLS certificate |
| 87 | + -extend-chan-size int |
| 88 | + Buffer size of the channel for handling Pod extension (default 500) |
| 89 | + -interact-chan-size int |
| 90 | + Buffer size of the channel for handling Pod interaction (default 500) |
| 91 | + -key-path string |
| 92 | + Path to the un-encrypted TLS key |
| 93 | + -log-level debug |
| 94 | + Log level. debug, `info`, `warn`, `error` are currently supported (default "info") |
| 95 | + -namespace-allowlist string |
| 96 | + Comma separated list of namespaces that allow interaction without evicting their Pods |
| 97 | + -port int |
| 98 | + Port for the app to listen on (default 8443) |
| 99 | + -ttl-seconds int |
| 100 | + TTL (time-to-live) of interacted Pods before getting evicted by the controller (default 600) |
| 101 | +``` |
| 102 | + |
| 103 | +#### kubectl-pi |
| 104 | +``` |
| 105 | +$ kubectl pi --help |
| 106 | +Get pod interaction info or request an extension of its termination time |
| 107 | +
|
| 108 | +Usage: |
| 109 | + kubectl pi [command] [flags] |
| 110 | +
|
| 111 | +Examples: |
| 112 | +
|
| 113 | + # get interaction info of specified pod(s) |
| 114 | + kubectl pi get <pod-name-1> <pod-name-2> <...> -n POD_NAMESPACE |
| 115 | +
|
| 116 | + # get interaction info of all pods under the given namespace |
| 117 | + kubectl pi get -n <pod-namespace> --all |
| 118 | +
|
| 119 | + # extend termination time of interacted pod(s) |
| 120 | + kubectl pi extend -d <duration> <pod-name-1> <pod-name-2> <...> -n POD_NAMESPACE |
| 121 | +
|
| 122 | + # extend termination time of all interacted pods under the given namespace |
| 123 | + kubectl pi extend -d <duration> -n <pod-namespace> --all |
| 124 | +
|
| 125 | +Flags: |
| 126 | + -a, --all if present, select all pods under specified namespace (and ignore any given pod podName) |
| 127 | + --cluster string The name of the kubeconfig cluster to use |
| 128 | + --context string The name of the kubeconfig context to use |
| 129 | + -d, --duration string a relative duration such as 5s, 2m, or 3h, default to 30m (default "30m") |
| 130 | + -h, --help help for kubectl |
| 131 | + -n, --namespace string If present, the namespace scope for this CLI request |
| 132 | + ... |
| 133 | +``` |
| 134 | + |
| 135 | +## Contribution |
| 136 | +Refer to [CONTRIBUTING.md](CONTRIBUTING.md) |
| 137 | + |
1 | 138 | ## Copyright and License
|
2 |
| - |
3 |
| -Copyright 2015 Box, Inc. All rights reserved. |
4 |
| - |
| 139 | +Copyright 2021 Box, Inc. All rights reserved. |
| 140 | + |
5 | 141 | Licensed under the Apache License, Version 2.0 (the "License");
|
6 | 142 | you may not use this file except in compliance with the License.
|
7 | 143 | You may obtain a copy of the License at
|
8 |
| - |
| 144 | + |
9 | 145 | http://www.apache.org/licenses/LICENSE-2.0
|
10 |
| - |
| 146 | + |
11 | 147 | Unless required by applicable law or agreed to in writing, software
|
12 | 148 | distributed under the License is distributed on an "AS IS" BASIS,
|
13 | 149 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
0 commit comments