Skip to content

Fix FIO OpenSearch field-limit explosion#20

Open
sravan27 wants to merge 1 commit into
redhat-performance:mainfrom
sravan27:sravan/fio-field-limit
Open

Fix FIO OpenSearch field-limit explosion#20
sravan27 wants to merge 1 commit into
redhat-performance:mainfrom
sravan27:sravan/fio-field-limit

Conversation

@sravan27

@sravan27 sravan27 commented Jun 5, 2026

Copy link
Copy Markdown

Fixes #19.

Summary

  • Process FIO archives as one summary document per workload via process_multiple(), matching the existing multi-document orchestration pattern.
  • Remove nested per-job jobs arrays from FIO summary metrics and time-series point metrics.
  • Preserve per-job time-series detail as scalar job_N_* point metrics in the existing zathras-timeseries export path.
  • Add schema validation that rejects nested time-series metric values before they can create unbounded OpenSearch mappings.

Validation

  • python -m pytest tests/test_processor_parsing.py tests/test_schema.py -q
  • python -m pytest -q
  • python -m py_compile src/chronicler/processors/fio_processor.py src/chronicler/schema.py tests/test_processor_parsing.py
  • git diff --check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FIO results exceed OpenSearch 5000 field limit (91,746 fields)

1 participant