Skip to content

Conversation

@elfkuzco
Copy link
Collaborator

@elfkuzco elfkuzco commented Nov 1, 2025

Rationale

This PR adds support for measuring resources used by the scraper. For the CPU stats, it uses the Exponentially Weighted Moving Average to measure the percentage of CPU usage.

Changes

This closes #1423

@elfkuzco
Copy link
Collaborator Author

elfkuzco commented Nov 1, 2025

Implementation of disk usage is yet to be added as it would rely on the approval of docker/docker-py#3370

@elfkuzco elfkuzco self-assigned this Nov 2, 2025
@benoit74
Copy link
Collaborator

benoit74 commented Nov 3, 2025

As discussed on Slack, I propose we wait few days for https://github.com/docker/docker-py/ maintainers to give us an answer.

If they don't reply soon enough, I propose two possible plans:

  • consider only mounts size in disk usage stat for the time being, since this is in general the "core" of disk usage, at least for big ZIMs which are primary concern
  • reimplement our own very-limited Docker SDK for only the operations we need ; could make since because Python Docker SDK seems to receive little attention from Docker, plus we only use few methods and could plug directly to the REST API just like the SDK does for these few methods we are using

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Measure and report all tasks resources usage

3 participants