ES single index reaches 9T, causing skywalking read/write failure

[foxyriver]

We use ES for data storage of Skywalking, which creates indexes in ES on a daily basis. Currently, the single index in ES reaches 9T, leading to frequent read and write failures. Does Skywalking have plans to support splitting indexes with finer intensity, or are there any other solutions？

[wu-sheng]

Why don't you create more shards for daily index? That is how the elastic scaling out.

[foxyriver]

We have created more shards for the index, but when the single index reaches 10T, even if the shard settings are reasonable, this index is difficult to maintain

[wu-sheng]

Shards and multiple indices are no different, AFAIK. You wouldn't see miracle. If you have to run the search to a huge scale of data, that is the cost.