File tree 1 file changed +15
-0
lines changed
1 file changed +15
-0
lines changed Original file line number Diff line number Diff line change @@ -85,3 +85,18 @@ Content-Length: <xx>
85
85
The ` parameters ` field in the
86
86
ModelInferRequest message can be used to send custom parameters.
87
87
88
+ ## Forwarding HTTP/GRPC Headers as Parameters
89
+
90
+ Triton can forward HTTP/GRPC headers as inference request parameters. By
91
+ specifying a regular expression in ` --http-header-forward-pattern ` and
92
+ ` --grpc-header-forward-pattern ` ,
93
+ Triton will add the headers that match with the regular experession as request
94
+ parameters. All the forwarded headers will be added as a parameter with string
95
+ value. For example to forward all the headers that start with 'PREFIX_ ' from
96
+ both HTTP and GRPC, you should add `--http-header-forward-pattern PREFIX_ .*
97
+ --grpc-header-forward-pattern PREFIX_ .* ` to your ` tritonserver` command.
98
+
99
+ The forwarded headers can be accessed using the
100
+ [ Python] ( https://github.com/triton-inference-server/python_backend#inference-request-parameters )
101
+ or C Backend APIs as inference request parameters.
102
+
You can’t perform that action at this time.
0 commit comments