Add chunking to file streaming for improved performance

Author: alxbridge

CHANGELOG.md

@@ -5,11 +5,15 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## Unreleased
+
+- Improve performance of streamed file downloads changed in v0.8 ([#44](https://github.com/torchbox/wagtail-bynder/pull/44/) @alxbridge
+
 ## [0.8] - 2025-11-12
 
 ### Fixed
 
-- Improved handling for unexpected server error responses when downloading Bynder assets ([#43](hhttps://github.com/torchbox/wagtail-bynder/pull/43/)) @alxbridge
+- Improved handling for unexpected server error responses when downloading Bynder assets ([#43](https://github.com/torchbox/wagtail-bynder/pull/43/)) @alxbridge
 
 ## [0.7] - 2025-01-15
 

src/wagtail_bynder/utils.py

@@ -36,9 +36,8 @@ def download_file(
     file = BytesIO()
     # Stream the file to memory. We use iter_content() instead of the default iterator for requests.Response,
     # as the latter uses iter_lines() which isn't suitable for streaming binary data.
-    for chunk in response.iter_content():
-        if not chunk:
-            continue
+    # Get data in largish 8KB chunks, for more performant streaming while staying within CPU cache limits
+    for chunk in response.iter_content(chunk_size=8192):
         file.write(chunk)
         if file.tell() > max_filesize:
             file.truncate(0)
@@ -47,6 +46,11 @@ def download_file(
             )
 
     size = file.tell()
+    # Catch empty case where iter_content wouldn't have iterated
+    if size == 0:
+        raise BynderAssetDownloadError(
+            f"Downloaded file '{name}' from Bynder is empty."
+        )
     file.seek(0)
 
     content_type, charset = mimetypes.guess_type(name)