Skip to content

Commit 695b628

Browse files
Tabriziantanmayv25
authored andcommitted
Add documentation for forwarding headers (#5662)
* Add documentation for forwarding headers * Add an example
1 parent a9f46de commit 695b628

File tree

1 file changed

+15
-0
lines changed

1 file changed

+15
-0
lines changed

docs/protocol/extension_parameters.md

+15
Original file line numberDiff line numberDiff line change
@@ -85,3 +85,18 @@ Content-Length: <xx>
8585
The `parameters` field in the
8686
ModelInferRequest message can be used to send custom parameters.
8787

88+
## Forwarding HTTP/GRPC Headers as Parameters
89+
90+
Triton can forward HTTP/GRPC headers as inference request parameters. By
91+
specifying a regular expression in `--http-header-forward-pattern` and
92+
`--grpc-header-forward-pattern`,
93+
Triton will add the headers that match with the regular experession as request
94+
parameters. All the forwarded headers will be added as a parameter with string
95+
value. For example to forward all the headers that start with 'PREFIX_' from
96+
both HTTP and GRPC, you should add `--http-header-forward-pattern PREFIX_.*
97+
--grpc-header-forward-pattern PREFIX_.*` to your `tritonserver` command.
98+
99+
The forwarded headers can be accessed using the
100+
[Python](https://github.com/triton-inference-server/python_backend#inference-request-parameters)
101+
or C Backend APIs as inference request parameters.
102+

0 commit comments

Comments
 (0)