Appender does not flush documents which received 429 if bulk indexer is never active

[carsonip]

Appender does not handle this edge case well. e.g. Appender receives event E1. Bulk Indexer BI1 handles E1, but gets a document level 429. This document is then rewritten to BI1's buffer. But consider a scenario where BI1 is never scheduled to be active, then E1 will sit inside BI1's buffer forever, past flush timeout, until Appender.Close is called.

[marclop]

@carsonip I did consider this while reviewing the 429 work. However, I deemed it highly unlikely since 429s are bound to happen when many events go through the system, since it's a symptom that Elasticsearch is overwhelmed.

If as little as one event is regularly going through the client, all BulkIndexers should eventually be used.

There is an argument to be made for other clients (unrelated to the client that uses go-docappender) being the culprit of the 429s, in which case this could be worse.