Skip to content

Migration loadtest from 4.89.0 > 4.90.0 is causing extremely high Server & DB CPU load #50228

Description

@AndreyKizimenko

Fleet versions

  • Discovered: 4.90
  • Reproduced: 4.90

Web browser and operating system:


💥  Actual behavior

Image

Fleet Server CPU avg = 96.62
RDS Writer CPU avg = 89.75
RDS Writer Threads Running (vCPUs=16) = 31.78 (expected < 16)

🛠️ Expected behavior

TODO

🧑‍💻  Steps to reproduce

These steps:

  • Have been confirmed to consistently lead to reproduction in multiple Fleet instances.
  • Describe the workflow that led to the error, but have not yet been reproduced in multiple Fleet instances.
  1. Spin up a loadtest environment with 100k hosts and 150k hosts infra on a 4.89.0 branch
  2. Migrate to 4.90.0
  3. Track DB and Server CPU load

🕯️ More info (optional)

N/A

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working as documented~release blockerThis story is currently blocking the next release of Fleet from going out.

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions