feat: enhance exporter architecture with chunked vs full export and context#24
Open
disstudio wants to merge 2 commits intoSylius:mainfrom
Open
feat: enhance exporter architecture with chunked vs full export and context#24disstudio wants to merge 2 commits intoSylius:mainfrom
disstudio wants to merge 2 commits intoSylius:mainfrom
Conversation
Merge main from head repo
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changelog
feat: Pass context into exporter
Add context array with
resourceAlias,batchIndex,exportFilenamefields, so exporter implementations can perform more complex logic (such as headers, export by template, individual for each resource etc).feat: Add ability to choose two strategies for exporter
If
ExporterInterface::supportsBatchedExport()istrue,export()is called multiple times (once for each data chunk). It is suitable for formats like CSV, each export operation can append new data to the file. This strategy has limitations - it requires that the export queue has only one worker to avoid concurrent write to file.If it is
false, all exported data is merged to one array andexport()is called once with all collected data. This strategy will consume more memory for large datasets.feat: CsvExporter can write headers
fix: Possibility to add custom exporters outside the bundle
Removed
BatchedExportDataManager::FILE_BASED_FORMATSdue to its limitation to internal exporters only.fix: Unable to export more than one batch
ExportCompletedHandler:
iterator_to_array(...$batchedData);replaced byarray_merge(...iterator_to_array($batchedData));