Increase or make configurable the DefaultBodyLimit (Axum) for PDF uploads in baml-cli serv mode

[partizanosBY]

Product

BAML

Describe the bug

When running baml-cli serv (mode BAML-over-HTTP API server that exposes your BAML functions via HTTP endpoint), there is a limitation on the size of the files that can be uploaded for processing (such as PDFs). Project code uses Axum - web application framework https://github.com/BoundaryML/baml/blob/0.218.0/engine/baml-runtime/src/cli/serve/mod.rs#L299
By default, Axum uses with DefaultBodyLimit , which restricts the request body to 2 MB (see the documentation: https://docs.rs/axum/latest/axum/extract/struct.DefaultBodyLimit.html).

This limitation makes it impossible to upload or process PDF files larger than 2 MB in baml-cli serve mode (see documentation: https://docs.boundaryml.com/ref/baml-cli/serve). When using models such as Gemini, which accepts files up to 50 MB, the current default greatly limits the ability to work with files more 2 MB.

There is currently no way to configure or disable this limit when running baml-cli serve mode, which prevents use cases that require uploading larger documents.

Suggestion:

• Make the DefaultBodyLimit value configurable via an environment variable or CLI flag, so that users can set an explicit body size limit according to their needs.
• Allow an option to disable the DefaultBodyLimit entirely when necessary.

References:

• Axum DefaultBodyLimit Documentation
• Source Code
• baml-cli documentation

Reproduction Steps

1. Run BAML in serv mode
   baml-cli serve
2. Attempt to upload a PDF file larger than 2 MB for processing.
3. Observe that the request fails due to body size limitation imposed by Axum DefaultBodyLimit with error:
   Failed to buffer the request body: length limit exceeded

BAML Version

0.218.0

Language/Framework

No response

LLM Provider

Google

LLM Model

No response

Operating System

Linux

Browser

Chrome

Code Editor

VS Code

[linear]

BAML-577

[partizanosBY]

Thanks, @sxlijin , for the quick response. Should we expect this feature in the next release?

[sxlijin]

Yes, this will go out in the next release! I'll leave this open until the release goes out.

[partizanosBY]

I checked and see that the changes in 0.218.1 didn't solve the issue with processing large PDFs.
A global fix was needed, but only one specific path was fixed.

I suggest setting a global limit in

engine/baml-runtime/src/cli/serve/mod.rs via tower_http::limit::RequestBodyLimitLayer

use tower_http::limit::RequestBodyLimitLayer;

let app = Router::new()
.route("/call/:method", post(handler))
.layer(RequestBodyLimitLayer::new(50 * 1024 * 1024)); // 50 MB

[partizanosBY]

UP !