Allow separate buckets for chunks and index (TSDB/Single Store) to support WORM

[vitas-pm]

Is your feature request related to a problem? Please describe.
Hey. I'm hitting a wall trying to set up Loki with strict WORM (Write Once, Read Many) compliance on S3.

I have a requirement where the log bucket must have Object Lock enabled so nothing can be deleted. This works great for chunks since they are immutable. However, this breaks the Compactor when using the TSDB schema (Single Store).

Because Loki currently forces chunks and indices into the same object_store, the Compactor tries to merge and delete old index files, gets a 403 Forbidden from S3, and fails. I can't grant delete permissions because the bucket policy forbids it entirely.

Describe the solution you'd like
I'd like to be able to configure a separate bucket for indices, distinct from the chunks bucket.

We can already separate ruler and admin buckets in the config. It would be great to extend this to the index, allowing a setup like this:

storage:
  bucketNames:
    chunks: my-worm-bucket   # Immutable / Object Lock enabled
    index: my-index-bucket   # Mutable / Compactor enabled
    ruler: my-ruler-bucket

This way, I can keep my logs compliant in a locked bucket but let the Compactor do its cleanup job in a separate, standard bucket.

Describe alternatives you've considered

1. Disabling the Compactor: This works to stop the errors, but it results in index fragmentation and hurts query performance/costs over time.
2. Prefix-based WORM: I looked into applying Object Lock only to the chunks/ prefix, but my cloud provider and many others only support Object Lock at the bucket level.

Additional context
I'm using the tsdb schema (v13). This seems like a necessary feature for anyone running Loki in highly regulated environments (banking/enterprise) where WORM is mandatory.

[jkroepke]

Why do you need to run compactor, if chunks should not be deleted?

[vitas-pm]

It's not about retention (I have that disabled). It's about Index De-duplication.

With the tsdb schema, ingesters write thousands of tiny index files per day. If I understand it correctly, the Compactor’s job is to merge those into larger, efficient files and delete the old fragments.

Because my bucket is WORM, the Compactor crashes when it tries to delete the old index fragments. If I disable the Compactor completely to avoid the crash, I end up with massive index fragmentation, which kills query performance and drives up S3 API costs.

Hope that clarifies it.