You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/architecture/load_planner.md
+3Lines changed: 3 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,6 +2,9 @@
2
2
3
3
This document covers load-based planner in `examples/llm/components/planner.py`.
4
4
5
+
> [!WARNING]
6
+
> Bare metal deployment with local connector is deprecated. The only option to deploy load-based planner is via k8s. We will update the examples in this document soon.
Copy file name to clipboardExpand all lines: docs/architecture/sla_planner.md
+3Lines changed: 3 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,6 +7,9 @@ The SLA (Service Level Agreement)-based planner is an intelligent autoscaling sy
7
7
> [!NOTE]
8
8
> Currently, SLA-based planner only supports disaggregated setup.
9
9
10
+
> [!WARNING]
11
+
> Bare metal deployment with local connector is deprecated. The only option to deploy SLA-based planner is via k8s. We will update the examples in this document soon.
12
+
10
13
## Features
11
14
12
15
***SLA-driven scaling**: Automatically scales prefill/decode workers to meet TTFT and ITL targets
Copy file name to clipboardExpand all lines: docs/guides/planner_benchmark/README.md
+3Lines changed: 3 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,6 +19,9 @@ limitations under the License.
19
19
20
20
This guide shows an example of benchmarking `LocalPlanner` performance with synthetic data. In this example, we focus on 8x H100 SXM GPU and `deepseek-ai/DeepSeek-R1-Distill-Llama-8B` model with TP1 prefill and decode engine.
21
21
22
+
> [!WARNING]
23
+
> Bare metal deployment with local connector is deprecated. The only option to deploy planner is via k8s. We will update the examples in this document soon.
24
+
22
25
## Synthetic Data Generation
23
26
24
27
We first generate synthetic data with varying request rate from 0.75 to 3 using the provided `generate_synthetic_data.py` script.
0 commit comments