Rewrite index names on ingest

[nablaone]

This PR adds options to rewrite index names on ingest. It's beneficial if we want all daily indices into single index.

It is configured as a list of rules. These rules are applied in order (sorted names).

• from contains a matching regexp
• to part contains a match group name

Rewriting is done at a very early stage of processing.

Here is a example config:

  - name: my-ingest-processor
    type: quesma-v1-processor-ingest
    config:
      indexNameRewriteRules:
        0:
          from: (.*?)(-\d{4}\.\d{2}\.\d{2})$
          to: "$1"
        1:
          from: (.*?)(-\d{4}\.\d{2})$
          to: "$1"
        3:
          from: (.*?)(.\d{4}-\d{2})$
          to: "$1"
        4: 
          from: (.*?)(.\d{4}-\d{2}-\d{2})$
          to: "$1"  

It will rewrite logs-2023.01 into logs.

[Copilot]

Pull Request Overview

This PR introduces a configurable index name rewriting feature for the ingest processor, allowing users to apply ordered regex-based rewrite rules before routing data.

• Injects a new IndexNameRewriter into IngestProcessor and wires it through bulk write logic
• Implements IndexNameRewriter with no‐op and regex‐based rewriters
• Updates V2 config translation and validation to support ordered rewrite rules, and adds both unit and integration tests

Reviewed Changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
platform/ingest/processor.go	Added IndexNameRewriter field, getter, and constructor injection
platform/ingest/index_name_rewriter.go	New regex‐based and no‐op rewriter implementations
platform/functionality/bulk/bulk.go	Extended SplitBulk and Write to accept and apply rewriter
platform/functionality/bulk/bulk_test.go	Updated tests to pass a NoOpIndexNameRewriter
platform/config/config.go	Added legacy config slice for rewrite rules
platform/config/config_v2.go	Added map field, validation for rewrite rules
platform/config/config_v2_util.go	Translated and ordered V2 rewrite rules into legacy form
platform/config/config_v2_test.go	Added unit test for V2 config translation of rules
platform/config/test_configs/index_name_rewrite_rules.yaml	New test YAML for rewrite rules
ci/it/configs/quesma-index-name-rewrite.yml.template	Integration test config template with rewrite rules
ci/it/testcases/test_index_name_rewrite.go	New IT testcase verifying rewrite behavior
ci/it/integration_test.go	Hooked up the new rewrite integration test

Comments suppressed due to low confidence (3)

platform/ingest/index_name_rewriter.go:12

• [nitpick] Add a Go doc comment above the IndexNameRewriter interface to explain its purpose, e.g. // IndexNameRewriter allows rewriting of Elasticsearch index names on ingest.

type IndexNameRewriter interface {

platform/ingest/index_name_rewriter.go:67

• There are no direct unit tests for indexNameRegexpRewriter.RewriteIndex. Adding table‐driven tests for matching and non‐matching patterns would ensure the logic is exercised.

func (i *indexNameRegexpRewriter) RewriteIndex(indexName string) string {

platform/functionality/bulk/bulk.go:121

• The ingest package isn’t imported here, leading to an undefined ingest.IndexNameRewriter type. Add "github.com/QuesmaOrg/quesma/platform/ingest" to the import block.

func SplitBulk(ctx context.Context, defaultIndex *string, bulk types.NDJSON, maxBulkSize int, tableResolver table_resolver.TableResolver, rewriter ingest.IndexNameRewriter) ([]BulkItem, map[string][]BulkRequestEntry, []byte, []BulkRequestEntry, error) {