-
-
Notifications
You must be signed in to change notification settings - Fork 41
Open
Description
I am trying to stream chunks of my multipart upload to disk. (Previously I used field.bytes() which works fine but can produce OOM on my raspberry pi).
my code using multer v3.1.0 (through axum) looks like:
pub async fn write_all_chunks_of_field(path: &Path, mut field: Field<'_>) -> Result<(), Error> {
info!(
"Trying to write to {} - (content_type = {:?})",
path.display(),
field.content_type()
);
let mut file = File::create(path).await?;
let mut c = 0;
loop {
match field.chunk().await {
Err(e) => {
error!("Error while chunking: {:?}", e);
return Err(map_to_io_error(e));
}
Ok(option) => match option {
None => {
info!("File written to {}", path.display());
break;
}
Some(b) => {
info!("{}: size = {}", c, b.len());
c += 1;
file.write_all(&*b).await?;
}
},
}
}
Ok(())
}
Logs produced look like:
2025-02-13T19:22:45.628219Z TRACE axum::serve: connection 127.0.0.1:55554 accepted
2025-02-13T19:22:45.629305Z INFO server::handler: Trying to write to "./data/upload_in_progress\\./67fd780
f-1c6f-421d-be72-d96ec240cdc5_BLKK7336.mp4" - (content_type = Some("video/mp4"))
2025-02-13T19:22:45.630431Z INFO server::handler: 0: size = 31919
2025-02-13T19:22:45.630891Z INFO server::handler: 1: size = 32768
2025-02-13T19:22:45.631552Z INFO server::handler: 2: size = 65536
2025-02-13T19:22:45.632944Z INFO server::handler: 3: size = 131072
2025-02-13T19:22:45.635134Z INFO server::handler: 4: size = 262144
2025-02-13T19:22:45.638365Z INFO server::handler: 5: size = 507904
2025-02-13T19:22:45.641431Z INFO server::handler: 6: size = 507904
2025-02-13T19:22:45.644492Z INFO server::handler: 7: size = 507904
2025-02-13T19:22:45.645066Z ERROR server::handler: Error while chunking: Error parsing `multipart/form-data
` request
I have tried curl, postman and my own tokio client app. It seems to happen always after 6-7 chunks, regardless of file being uploaded and content_type.
Anything that's smaller than that seems to run through without issues.
I am new to async programming in rust, so I am sorry if I made some obvious mistake. Any help welcome that would aid me in finding my issue would be highly appreciated
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels