Skip to content

Explore: Adjust the number of workers based on generation #22

Open
@casperisfine

Description

This remain to be seen whether we'll actually need this, but would be worth exploring.

The assumption is that the first few generation will invalidate shared pages rather quickly, as such the peak memory usage at first might get quite high.

A possible solution to this would be to start with a lower number of workers to compensate for the lower CoW effectiveness, and then increase the number of workers of processes with each new generation.

This would mean pitchfork needs a bit of a warm up before achieving peak throughput.

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions