`DocumentParser` class refactoring

[salvatore-campagna]

Description

The DocumentParser class has undergone significant changes to enhance synthetic source support, which has introduced added complexity and difficult-to-understand logic into the code.

To address these issues, a more effective design could involve creating a base class that consolidates shared generic code, along with two specific implementations: one for parsing documents when the source is stored and another for parsing documents when the source is synthetic. This approach would not only simplify the overall document parsing logic but also allow for specialization in the parsing behavior, making it easier to manage and understand.

By separating the parsing logic based on the type of source, we can achieve a cleaner and more maintainable codebase, ultimately improving the functionality of the DocumentParser.

[elasticsearchmachine]

Pinging @elastic/es-storage-engine (Team:StorageEngine)

[henriquepaes1]

Hey @salvatore-campagna, may I work on this issue?

[lkts]

@henriquepaes1 thank you for your interest in this. I think this is not the best issue unfortunately because it requires right communication with the team and design work.

[lkts]

One follow-up idea from #116049:

So we know that we can precompute which fields need special handling for synthetic source (in most cases). If that is the case why don't we do the following. We perform document parsing normally. Then for each field that needs special handling:

• Create an XContentParser for source bytes that has a filter applied for the field name.
• Do a parsing pass that should yield only values of this specific field.
• Write values to _ignored_source
  This would remove some/all synthetic source logic from actual document parser, allows to see all values and do optimizations based on that knowledge, eliminates complex structure of DocumentParserListener. Performance impact if of course TBD.

[lkts]

Reintroducing the previous refactoring attempt involves:
re-introducing #116049
merging #119369 on top (or incorporating it in one above)