Rate limiting headers such as Retry-After or X-Ratelimit-Reset that provide an amount of waiting time needed before retrying the request are currently handled by blocking wait on urllib level. This makes the Celery tasks that hit the limits block for the specified time, congesting workers and potentially getting timeout-killed.
When a task is rate-limited, the waiting time should be propagated and the task scheduled for retry after that time on Celery level.