@@ -41,19 +41,25 @@ server.
41
41
What's New In 0.10.0 Beta
42
42
-------------------------
43
43
44
+ * `Custom backend support
45
+ <https://docs.nvidia.com/deeplearning/sdk/tensorrt-inference-server-guide/docs/model_repository.html#custom-backends> `_.
46
+ TRTIS allows individual models to be implemented with custom
47
+ backends instead of by a deep-learning framework. With a custom
48
+ backend a model can implement any logic desired, while still
49
+ benefiting from the GPU support, concurrent execution, dynamic
50
+ batching and other features provided by TRTIS.
51
+
52
+ Features
53
+ --------
54
+
55
+ The inference server provides the following features.
56
+
44
57
* `Multiple framework support
45
58
<https://docs.nvidia.com/deeplearning/sdk/tensorrt-inference-server-guide/docs/model_repository.html#framework-model-definition> `_. The
46
59
server can manage any number and mix of models (limited by system
47
60
disk and memory resources). Supports TensorRT, TensorFlow GraphDef,
48
61
TensorFlow SavedModel and Caffe2 NetDef model formats. Also supports
49
62
TensorFlow-TensorRT integrated models.
50
- * `Custom backend support
51
- <https://docs.nvidia.com/deeplearning/sdk/tensorrt-inference-server-guide/docs/model_repository.html#custom-backends> `_. TRTIS
52
- allows individual models to be implemented with custom backends
53
- instead of by a deep-learning framework. With a custom backend a
54
- model can implement any logic desired, while still benefiting from
55
- the GPU support, concurrent execution, dynamic batching and other
56
- features provided by TRTIS.
57
63
* TRTIS `monitors the model repository
58
64
<https://docs.nvidia.com/deeplearning/sdk/tensorrt-inference-server-guide/docs/model_repository.html#modifying-the-model-repository> `_
59
65
for any change and dynamically reloads the model(s) when necessary,
0 commit comments