Skip to content

feat: enhance exporter architecture with chunked vs full export and context#24

Open
disstudio wants to merge 2 commits intoSylius:mainfrom
disstudio:feature/batched-export
Open

feat: enhance exporter architecture with chunked vs full export and context#24
disstudio wants to merge 2 commits intoSylius:mainfrom
disstudio:feature/batched-export

Conversation

@disstudio
Copy link
Contributor

Changelog

feat: Pass context into exporter

Add context array with resourceAlias, batchIndex, exportFilename fields, so exporter implementations can perform more complex logic (such as headers, export by template, individual for each resource etc).

feat: Add ability to choose two strategies for exporter

If ExporterInterface::supportsBatchedExport() is true, export() is called multiple times (once for each data chunk). It is suitable for formats like CSV, each export operation can append new data to the file. This strategy has limitations - it requires that the export queue has only one worker to avoid concurrent write to file.
If it is false, all exported data is merged to one array and export() is called once with all collected data. This strategy will consume more memory for large datasets.

feat: CsvExporter can write headers

fix: Possibility to add custom exporters outside the bundle

Removed BatchedExportDataManager::FILE_BASED_FORMATS due to its limitation to internal exporters only.

fix: Unable to export more than one batch

ExportCompletedHandler: iterator_to_array(...$batchedData); replaced by array_merge(...iterator_to_array($batchedData));

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant