diff --git a/Conceptual_Guide/Part_2-improving_resource_utilization/README.md b/Conceptual_Guide/Part_2-improving_resource_utilization/README.md index a8edf319..b8e68fd8 100644 --- a/Conceptual_Guide/Part_2-improving_resource_utilization/README.md +++ b/Conceptual_Guide/Part_2-improving_resource_utilization/README.md @@ -79,7 +79,7 @@ Let's take the previous example and discuss the effect of adding multiple models For a "no dynamic batching" case, as there are two models to execute, the queries are distributed equally. Users can also add [priorities](https://github.com/triton-inference-server/server/blob/main/docs/user_guide/model_configuration.md#priority) to prioritize or de-prioritize any specific instance group. -When considering the case of multiple instances with dynamic batches enabled, the following happens. Owing to the availability of another instance, query `B` which arrives with some delay can be executed using the second instance. With some delay allocated, instance 1 gets filled and launched by time `T = X/2` and since queries `D` and `E` stack up to fill up to the maximum batch size, the second model can start inference without any delay. +When considering the case of multiple instances with dynamic batches enabled, the following happens. Owing to the availability of another instance, query `B` which arrives with some delay can be executed using the second instance. With some delay allocated, instance 1 gets filled and launched by time `T = X/3` and since queries `D` and `E` stack up to fill up to the maximum batch size, the second model can start inference without any delay. The key takeaway from the above examples is that the Triton Inference Server provides flexibility with respect to policies related to creating more efficient batching, thus enabling better resource utilization, resulting in reduced latency and increased throughput. diff --git a/Conceptual_Guide/Part_2-improving_resource_utilization/img/dynamic_batching.PNG b/Conceptual_Guide/Part_2-improving_resource_utilization/img/dynamic_batching.PNG index f720bbf2..867bb208 100644 Binary files a/Conceptual_Guide/Part_2-improving_resource_utilization/img/dynamic_batching.PNG and b/Conceptual_Guide/Part_2-improving_resource_utilization/img/dynamic_batching.PNG differ diff --git a/Conceptual_Guide/Part_2-improving_resource_utilization/img/multi_instance.PNG b/Conceptual_Guide/Part_2-improving_resource_utilization/img/multi_instance.PNG index 26983904..aeb3b4e8 100644 Binary files a/Conceptual_Guide/Part_2-improving_resource_utilization/img/multi_instance.PNG and b/Conceptual_Guide/Part_2-improving_resource_utilization/img/multi_instance.PNG differ