Skip to content

[Feature Request] Storage Reduction for id fields #14831

Open
@mgodwan

Description

@mgodwan

Is your feature request related to a problem? Please describe

Today, _id field is:

  1. Indexed using the FST data structure
  2. Stored using stored field mapper

For time-series data, the generated _id is not used very often as a query term. The _id field generated is optimized for query but by generating it in a way that we have longer common prefix, we may be able to reduce the storage size it takes.

* These are essentially flake ids but we use 6 (not 8) bytes for timestamp, and use 3 (not 2) bytes for sequence number. We also reorder
* bytes in a way that does not make ids sort in order anymore, but is more friendly to the way that the Lucene terms dictionary is
* structured.

Describe the solution you'd like

A new implementation of UUID generator which can reduce the storage size for the field

Related component

Indexing:Performance

Describe alternatives you've considered

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

Indexing:PerformanceenhancementEnhancement or improvement to existing feature or requestv3.0.0Issues and PRs related to version 3.0.0

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions