Collect pod specific process information such as ps output, memory etc

[mrajagopal]

Although we have pod level detail within k8s, there are times it may be necessary to understanding the process level utilisation of CPU and memory within the pod/container to assess inefficiencies.
At present, one would need to instruct the customer to access the pod to run various commands which can be prone to error.

Describe the solution you'd like
The objective of this issue is to explore collecting pod specific process information such as ps output, memory etc.

[cpuengr949]

Some of the commands that have been asked to run are below, it would be nice to have all of this in the support package if needed.
cat /sys/fs/cgroup/memory.stat from within a Ingress Controller pod
for p in $(pidof nginx nginx-ingress); do sudo pmap ${p} -X; done from the k8s node running the Ingress Controller pods
for p in $(pidof nginx nginx-ingress); do cat /proc/${p}/status; done since azure was unable to run the command above
Collect the following information from the API endpoint from the pods:

/api/8/processes
/api/8/connections
/api/8/slabs
/api/8/http/requests
/api/8/http/server_zones
/api/8/http/location_zones
/api/8/http/caches
/api/8/http/upstreams
/api/8/http/keyvals

[mrajagopal]

Our plan is to deploy a custom container as a sidecar in the pod to fetch the necessary data.

[mrajagopal]

• As a temporary measure, we can attached a container to the nginx-ingress container and collect the data needed:

pod=$(kb get po -n kic-0 -o name); kb -n kic-0 debug -it $pod --image=alpine:latest --target=nginx-ingress -c f5-debugger

• Once the alpine container is created, you should get a shell prompt to that container where you will need to add curl and jq (optional):

apk add curl jq

• As we have attached this f5-debugger container to nginx-ingress container, we can now get the API output using curl and jq :

for i in /api/8/processes /api/8/connections /api/8/slabs /api/8/http/requests /api/8/http/server_zones /api/8/http/location_zones /api/8/http/caches /api/8/http/upstreams /api/8/http/keyvals; do echo "**** $i ****" ; curl "127.0.0.1:8080/$i" | jq .; echo ""; done