Skip to content

inets: Remove max_client_body_chunk support #11476

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
garazdawi wants to merge 1 commit into
base: master
Choose a base branch
from
Open

inets: Remove max_client_body_chunk support #11476

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions lib/inets/doc/guides/hardening.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -250,11 +250,6 @@ requests that can exhaust memory. Set explicit limits:
exceeds this value with a 413 response, before reading the body. Default:
`100_000_000` (100 MB).

- **[`max_client_body_chunk`](`m:httpd#max_client_body_chunk`)** - When handling large PUT or POST bodies via
`mod_esi`, setting this option enforces chunked delivery to the ESI
callback. This prevents the server from buffering the entire request body
in memory, which could be exploited to cause memory exhaustion.

> #### Note {: .info }
>
> `max_body_size` and `max_content_length` serve different purposes.
Expand Down
24 changes: 6 additions & 18 deletions lib/inets/src/http_server/httpd.erl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -195,12 +195,6 @@ property list.
the server closes the connection. The server closes it even if there are
queued request. Default is no limit.

- [](){: #max_client_body_chunk } **`{max_client_body_chunk, integer()}`**
Enforces chunking of a HTTP PUT or POST body data to be delivered to the
mod_esi callback. Note this is not supported for mod_cgi. Default is no limit
e.i the whole body is delivered as one entity, which could be very memory
consuming. `m:mod_esi`.

[](){: #props_admin }

### Administrative Properties
Expand Down Expand Up @@ -1021,8 +1015,7 @@ reload_config(ConfigFile, Mode) ->
| {max_header_size, integer()}
| {max_content_length, integer()}
| {max_uri_size, integer()}
| {max_keep_alive_request, integer()}
| {max_client_body_chunk, integer()},
| {max_keep_alive_request, integer()},
AdminOption :: {mime_types, [{MimeType :: string(), Extension :: string()}] | Path}
| {mime_type, string()}
| {server_admin, string()}
Expand Down Expand Up @@ -1073,8 +1066,7 @@ server.
| {max_header_size, integer()}
| {max_content_length, integer()}
| {max_uri_size, integer()}
| {max_keep_alive_request, integer()}
| {max_client_body_chunk, integer()},
| {max_keep_alive_request, integer()},
AdminOption :: {mime_types, [{MimeType :: string(), Extension :: string()}] | Path}
| {mime_type, string()}
| {server_admin, string()}
Expand Down Expand Up @@ -1111,8 +1103,7 @@ server.
| {max_header_size, integer()}
| {max_content_length, integer()}
| {max_uri_size, integer()}
| {max_keep_alive_request, integer()}
| {max_client_body_chunk, integer()},
| {max_keep_alive_request, integer()},
AdminOption :: {mime_types, [{MimeType :: string(), Extension :: string()}] | Path}
| {mime_type, string()}
| {server_admin, string()}
Expand Down Expand Up @@ -1166,8 +1157,7 @@ info(Address, Port) when is_integer(Port) ->
| {max_header_size, integer()}
| {max_content_length, integer()}
| {max_uri_size, integer()}
| {max_keep_alive_request, integer()}
| {max_client_body_chunk, integer()},
| {max_keep_alive_request, integer()},
AdminOption :: {mime_types, [{MimeType :: string(), Extension :: string()}] | Path}
| {mime_type, string()}
| {server_admin, string()}
Expand Down Expand Up @@ -1205,8 +1195,7 @@ info(Address, Port) when is_integer(Port) ->
| {max_header_size, integer()}
| {max_content_length, integer()}
| {max_uri_size, integer()}
| {max_keep_alive_request, integer()}
| {max_client_body_chunk, integer()},
| {max_keep_alive_request, integer()},
AdminOption :: {mime_types, [{MimeType :: string(), Extension :: string()}] | Path}
| {mime_type, string()}
| {server_admin, string()}
Expand Down Expand Up @@ -1263,8 +1252,7 @@ options of the server.
| {max_header_size, integer()}
| {max_content_length, integer()}
| {max_uri_size, integer()}
| {max_keep_alive_request, integer()}
| {max_client_body_chunk, integer()},
| {max_keep_alive_request, integer()},
AdminOption :: {mime_types, [{MimeType :: string(), Extension :: string()}] | Path}
| {mime_type, string()}
| {server_admin, string()}
Expand Down
48 changes: 10 additions & 38 deletions lib/inets/src/http_server/httpd_request_handler.erl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -236,9 +236,8 @@ handle_cast(Msg, #state{mod = #mod{config_db = Db} = ModData} = State) ->
%%--------------------------------------------------------------------
handle_info({Proto, Socket, Data},
#state{mfa = {Module, Function, Args},
chunk = {ChunkState, _},
mod = #mod{socket_type = SockType,
socket = Socket} = ModData} = State)
socket = Socket} = ModData} = State)
when (((Proto =:= tcp) orelse
(Proto =:= ssl) orelse
(Proto =:= dummy)) andalso is_binary(Data)) ->
Expand Down Expand Up @@ -266,7 +265,7 @@ handle_info({Proto, Socket, Data},
{stop, normal, State#state{response_sent = true,
mod = NewModData}};
{error, {version_error, ErrCode, ErrStr}, Version} ->
NewModData = ModData#mod{http_version = Version},
NewModData = ModData#mod{http_version = Version},
httpd_response:send_status(NewModData, ErrCode, ErrStr),
{stop, normal, State#state{response_sent = true,
mod = NewModData}};
Expand All @@ -275,18 +274,15 @@ handle_info({Proto, Socket, Data},
httpd_response:send_status(NewModData, ErrCode, ErrStr),
{stop, normal, State#state{response_sent = true,
mod = NewModData}};
NewMFA ->
setopts(Socket, SockType, [{active, once}]),
case NewDataSize of
undefined ->
{noreply, State#state{mfa = NewMFA}};
_ ->
{noreply, State#state{mfa = NewMFA, data = NewDataSize}}
end

{http_chunk = Module, Function, Args} when ChunkState =/= undefined ->
NewState = handle_chunk(Module, Function, Args, State),
{noreply, NewState};
NewMFA ->
setopts(Socket, SockType, [{active, once}]),
case NewDataSize of
undefined ->
{noreply, State#state{mfa = NewMFA}};
_ ->
{noreply, State#state{mfa = NewMFA, data = NewDataSize}}
end
end;

%% Error cases
Expand Down Expand Up @@ -535,7 +531,6 @@ handle_body(#state{headers = Headers, body = Body,
{stop, normal, State#state{response_sent = true}};
_ ->
Length = list_to_integer(Headers#http_request_h.'content-length'),
MaxChunk = max_client_body_chunk(ConfigDB),
case Length =< MaxBodySize orelse MaxBodySize == nolimit of
true ->
case httpd_request:body_chunk_first(Body, Length, MaxChunk) of
Expand Down Expand Up @@ -617,29 +612,6 @@ expect(Headers, _, ConfigDB) ->
end
end.

handle_chunk(http_chunk = Module, decode_data = Function,
[ChunkSize, TotalChunk, {MaxBodySize, BodySoFar, _AccLength, MaxHeaderSize}],
#state{chunk = {_, CbState},
mod = #mod{socket_type = SockType,
socket = Socket} = ModData} = State) ->
{continue, NewCbState} = httpd_response:handle_continuation(ModData#mod{entity_body =
{continue, BodySoFar, CbState}}),
setopts(Socket, SockType, [{active, once}]),
State#state{chunk = {continue, NewCbState}, mfa = {Module, Function, [ChunkSize, TotalChunk, {MaxBodySize, <<>>, 0, MaxHeaderSize}]}};

handle_chunk(http_chunk = Module, decode_size = Function,
[Data, HexList, _AccSize, {MaxBodySize, BodySoFar, _AccLength, MaxHeaderSize}],
#state{chunk = {_, CbState},
mod = #mod{socket_type = SockType,
socket = Socket} = ModData} = State) ->
{continue, NewCbState} = httpd_response:handle_continuation(ModData#mod{entity_body = {continue, BodySoFar, CbState}}),
setopts(Socket, SockType, [{active, once}]),
State#state{chunk = {continue, NewCbState}, mfa = {Module, Function, [Data, HexList, 0, {MaxBodySize, <<>>, 0, MaxHeaderSize}]}};
handle_chunk(Module, Function, Args, #state{mod = #mod{socket_type = SockType,
socket = Socket}} = State) ->
setopts(Socket, SockType, [{active, once}]),
State#state{mfa = {Module, Function, Args}}.

handle_internal_chunk(#state{chunk = {ChunkState, CbState}, body = Chunk,
mod = #mod{socket_type = SockType,
socket = Socket} = ModData} = State, Module, Function, Args)->
Expand Down
34 changes: 2 additions & 32 deletions lib/inets/src/http_server/mod_esi.erl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,27 +138,6 @@ that no HTTP header fields will be generated. This behaviour depends on the

- `Input`: query data of a GET request or the body of a PUT or POST request.

The default behavior (legacy reasons) for delivering the body, is that the
whole body is gathered and converted to a string. But if the httpd config
parameter [`max_client_body_chunk`](`m:httpd#max_client_body_chunk`) is set,
the body will be delivered as binary chunks instead. The maximum size of the
chunks is either [`max_client_body_chunk`](`m:httpd#max_client_body_chunk`) or
decided by the client if it uses HTTP chunked encoding to send the body.

When using the chunking mechanism, this callback must return `{continue,
State::term()}` for all calls where `Input` is `{first, Data::binary()}` or
`{continue, Data::binary(), State::term()}`. When `Input` is `{last,
Data::binary(), State::term()}` the return value will be ignored.

The input `State` is the last returned `State`, in it the callback can include
any data that it needs to keep track of when handling the chunks.

> #### Note {: .info }
>
> Note that if the body is small all data may be delivered in only one chunk and
> then the callback will be called with `{last, Data::binary(), undefined}`
> without getting called with `{first, Data::binary()}`.

## Setting a response status

To set the response status code, the special `status` response header can be
Expand All @@ -170,20 +149,11 @@ JSON response body, one could pass the following:
```
""".
-doc(#{group => <<"ESI Callback Functions">>}).
-callback 'Function'(SessionID, Env, Input) -> {continue, State} | _
-callback 'Function'(SessionID, Env, Input) -> _
when
SessionID :: session_id(),
Env :: [env()],
Input :: string() | ChunkedData,
ChunkedData ::
{first, Data :: binary()} |
{continue,
Data :: binary(),
State :: term()} |
{last,
Data :: binary(),
State :: term()},
State :: term().
Input :: string().

-optional_callbacks(['Function'/3]).

Expand Down
Loading