Nodes performance

[jonesa99]

Hi, I'm starting to look into kwok and from a blog I read that "Kwok doesn’t accurately reflect the performance of real nodes because it approximates behaviors using simple formulas". Is it possible to find these formulas somewhere? And how can a fake node simulate a job? Do you need to also run some real nodes? Thanks for the help

[wzshiming]

Refer to https://kwok.sigs.k8s.io/docs/user/stages-configuration/

[jonesa99]

So I don't get what kwok is simulating. I can create different nodes and pods, assign a pipeline to them and get metrics like time and cpu. How are these metrics generated? And what are they useful for? In my mind it seemed to be useful to simulate the run of a pipeline even if my resources are not enough due to big data volume

[wzshiming]

I don't know how Argo collects this data, but there is always a way to simulate it, give Argo the data it needs, kwok doesn't care what it is.

https://kwok.sigs.k8s.io/docs/design/introduction/#user-stories

In fact, the main goal is unit testing and large-scale testing.

[wzshiming]

How are these metrics generated? And what are they useful for?

I can't answer this question at the moment, because I'm not sure how Argo works.

[jshin48]

Yes, that will be included in the future, but right now it's mostly about nodes, pods, and metrics.

Hello, is this feature applied in the current version?
I am also wondering how to put the execution time of each pod for the simulation.
Does it have to be specified within a yaml file when pod creation request is submitted?
If so, which field? Is there any example yaml file that contains this information?

[wzshiming]

Yes, that will be included in the future, but right now it's mostly about nodes, pods, and metrics.

Hello, is this feature applied in the current version? I am also wondering how to put the execution time of each pod for the simulation. Does it have to be specified within a yaml file when pod creation request is submitted? If so, which field? Is there any example yaml file that contains this information?

It feels like your question is out of context with this discussion...

If you want to change the execution time of a pod, you can to changing the default stages configuration. Here is a basic principle and 2 basic cases, the main action is to configure the delay property of the pod stage. More examples can be found in the github search on how to used by other user

[jonesa99]

okay so in general it can be used to simulate cluster in large scale, but also in this case how could you simulate to have huge amount of nodes? how do the fake resources behave?

[jonesa99]

found it

[jonesa99]

sorry for not updating you, I made some progress and kwok seems useful for my case. I just have a quick question; in order to scale a pipeline using Argo I should expect the CPU and memory to increase when moving from 1 to 2 pods, while the execution time should decrease. Is it correct? And should I see this behavior also in kwok?

[wzshiming]

Yes, Simulation of CPU/Memory usage is expected to be supported in the next release.

[jonesa99]

okay thanks. so atm in kwok I can only see the difference in time execution when changing the number of pods, right?

[wzshiming]

I think the time you can get it directly from the Pod

[jonesa99]

yes I can do it, I am just wondering what are the possible metrics to collect

[wzshiming]

The metric also provides the execution time, but I don't know which one argo uses, the one from the Pod or the one from the metric.