Add a test plan for the pulp-worker scalability test. #494
Conversation
decko
force-pushed
the
worker_testplan
branch
2 times, most recently
from
49aa8be to
2ebe84d
Compare
|
|
||
| For database, we're using the AWS RDS PostgreSQL 16, on a db.m7g.2xlarge instance class. | ||
| In theory, the maximum number of connections is defined by `LEAST({DBInstanceClassMemory/9531392}, 5000)` | ||
| as written [here](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Limits.html#RDS_Limits.MaxConnections) |
There was a problem hiding this comment.
Maybe, it would be helpful to interpret this with a claim of the total count.
|
|
||
| ## Test Plan | ||
|
|
||
| 1. Warn the pulp stakeholders (@pulp-service-stakeholders) that a test gonna happen in stage. All tasks must be stopped. |
There was a problem hiding this comment.
Realistically we can't ask folks to stop the tasks. I think warning them we're scaling down the memory footprint and their tasks may fail due to memory errors is what they need to know. Also I think they need to know how long the test will run for (how long can they expect failures).
images/assets/patches/0023-Add-the-task-name-to-the-task-log.patch
Outdated
Show resolved
Hide resolved
| 2. Create a MR reducing the resource for each worker, and increasing the number of workers. [here](https://gitlab.cee.redhat.com/service/app-interface/-/blob/master/data/services/pulp/deploy.yml#L75). | ||
| It can be approved by anyone from the team. Just need to wait for the app-sre bot to merge it. | ||
| 3. After it got merged, check for the database metrics and application logs. The results from the run will be down in this document. | ||
| 4. We need to check if there's any new worker being started. For that, we need to use the `WorkersAPI` and check a set with the names from time to time. |
There was a problem hiding this comment.
This step needs more detail. To me, it's the whole metric of interest. I'd avoid writing real rigorous code here, and maybe verbally describe the set-name difference "check" and maybe in pseudo-code describe it for clarity for the reader.
There was a problem hiding this comment.
I've added an example of a script that can do the job.
decko
force-pushed
the
worker_testplan
branch
2 times, most recently
from
090b7bd to
a4d8f39
Compare
No description provided.