[llm-d-legacy] Import the TOPSAIL legacy LLM-D project for more advance testing#42
Merged
kpouget merged 4 commits intoopenshift-psap:mainfrom Apr 27, 2026
Merged
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Warning Rate limit exceeded
To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (68)
📝 WalkthroughWalkthroughThis PR introduces comprehensive LLM-D testing infrastructure, including CI orchestration, cluster preparation, Kubernetes inference service deployments, Grafana monitoring dashboards, benchmark running with GuideLLM, visualization pipelines, and matrix benchmarking support. It also updates CI error-handling logic in core and legacy libraries. Changes
Sequence Diagram(s)sequenceDiagram
participant CLI as Test CLI
participant Prepare as Cluster Prepare
participant Deploy as ISVC Deploy
participant Bench as GuideLLM Bench
participant Capture as State Capture
participant Visualize as Visualization
CLI->>Prepare: prepare_ci()
Prepare->>Prepare: Setup operators/namespace
Prepare->>Prepare: Deploy Grafana/monitoring
Prepare->>Prepare: Update pull secrets
Prepare->>Prepare: Download models to PVC
Prepare->>Prepare: Preload container images
CLI->>Deploy: test_ci() per flavor
Deploy->>Deploy: Parse & reshape ISVC YAML
Deploy->>Deploy: Apply EPP routing config
Deploy->>Deploy: Deploy LLMInferenceService
Deploy->>Deploy: Wait for readiness
Deploy->>Bench: run_guidellm_benchmark()
Bench->>Bench: Create Job + PVC
Bench->>Bench: Run GuideLLM benchmark
Bench->>Bench: Extract results.json
CLI->>Capture: capture_llm_inference_service_state()
Capture->>Capture: Dump ISVC/pods/logs
Capture->>Capture: Capture Prometheus metrics
CLI->>Visualize: generate_visualization()
Visualize->>Visualize: Parse results → LTS payload
Visualize->>Visualize: Generate plots/reports
Estimated code review effort🎯 4 (Complex) | ⏱️ ~75 minutes Possibly related PRs
Poem
✨ Finishing Touches🧪 Generate unit tests (beta)
|
Contributor
Author
|
/test fournos llm_d_legacy psap_h200 intelligentrouting-flavors |
|
🔴 Test of 'llm_d_legacy test' failed after 00 hours 00 minutes 00 seconds 🔴 • Link to the test results. • No reports index generated... • No test configuration ( • Failure indicator: Empty. |
|
🔴 Test of 'fournos_launcher submit' failed after 00 hours 00 minutes 31 seconds 🔴 • Link to the test results. • No reports index generated... Test configuration: |
Contributor
Author
|
/test fournos llm_d_legacy psap_h200 intelligentrouting-flavors |
|
🔴 Test of 'llm_d_legacy test' failed after 00 hours 00 minutes 00 seconds 🔴 • Link to the test results. • No reports index generated... • No test configuration ( • Failure indicator: Empty. |
|
🔴 Test of 'fournos_launcher submit' failed after 00 hours 00 minutes 37 seconds 🔴 • Link to the test results. • No reports index generated... Test configuration: |
Contributor
Author
|
/test fournos llm_d_legacy psap_h200 intelligentrouting-flavors |
|
🔴 Test of 'llm_d_legacy test' failed after 00 hours 00 minutes 00 seconds 🔴 • Link to the test results. • No reports index generated... • No test configuration ( • Failure indicator: Empty. |
|
🔴 Test of 'fournos_launcher submit' failed after 00 hours 00 minutes 31 seconds 🔴 • Link to the test results. • No reports index generated... Test configuration: |
kpouget
force-pushed
the
llm_d_legacy
branch
2 times, most recently
from
c5124a7 to
770b830
Compare
Contributor
Author
|
/test fournos llm_d_legacy psap_h200 intelligentrouting-flavors |
|
🔴 Test of 'llm_d_legacy test' failed after 00 hours 00 minutes 00 seconds 🔴 • Link to the test results. • No reports index generated... • No test configuration ( • Failure indicator: Empty. |
|
🔴 Test of 'fournos_launcher submit' failed after 00 hours 00 minutes 32 seconds 🔴 • Link to the test results. • No reports index generated... Test configuration: |
Contributor
Author
|
/test fournos llm_d_legacy psap_h200 intelligentrouting-flavors |
|
🔴 Test of 'llm_d_legacy test' failed after 00 hours 00 minutes 00 seconds 🔴 • Link to the test results. • No reports index generated... • No test configuration ( • Failure indicator: Empty. |
|
🔴 Test of 'fournos_launcher submit' failed after 00 hours 00 minutes 31 seconds 🔴 • Link to the test results. • No reports index generated... Test configuration: |
Contributor
Author
|
/test fournos llm_d_legacy psap_h200 intelligentrouting-flavors |
|
🔴 Test of 'llm_d_legacy test' failed after 00 hours 00 minutes 00 seconds 🔴 • Link to the test results. • No reports index generated... • No test configuration ( |
|
🔴 Test of 'fournos_launcher submit' failed after 00 hours 00 minutes 31 seconds 🔴 • Link to the test results. • No reports index generated... Test configuration: |
Contributor
Author
|
/test fournos llm_d_legacy psap_h200 intelligentrouting-flavors |
|
🔴 Test of 'llm_d_legacy test' failed after 00 hours 00 minutes 00 seconds 🔴 • Link to the test results. • No reports index generated... • No test configuration ( |
|
🔴 Test of 'fournos_launcher submit' failed after 00 hours 07 minutes 32 seconds 🔴 • Link to the test results. • No reports index generated... Test configuration: |
Contributor
Author
|
/test fournos llm_d_legacy psap_h200 intelligentrouting-flavors |
1 similar comment
Contributor
Author
|
/test fournos llm_d_legacy psap_h200 intelligentrouting-flavors |
|
🔴 Test of 'llm_d_legacy test' failed after 00 hours 07 minutes 56 seconds 🔴 • Link to the test results. • No reports index generated... • No test configuration ( |
|
🔴 Test of 'fournos_launcher submit' failed after 00 hours 08 minutes 26 seconds 🔴 • Link to the test results. • No reports index generated... Test configuration: |
Contributor
Author
|
/test fournos llm_d_legacy psap_h200 intelligentrouting-flavors |
|
🔴 Test of 'llm_d_legacy test' failed after 00 hours 07 minutes 10 seconds 🔴 • Link to the test results. • No reports index generated... • No test configuration ( |
|
🔴 Test of 'fournos_launcher submit' failed after 00 hours 07 minutes 45 seconds 🔴 • Link to the test results. • No reports index generated... Test configuration: |
kpouget
force-pushed
the
llm_d_legacy
branch
3 times, most recently
from
fbc2a21 to
e4793a5
Compare
Contributor
Author
|
/test fournos llm_d_legacy psap_h200 intelligentrouting-flavors |
|
🔴 Test of 'llm_d_legacy test' failed after 00 hours 07 minutes 10 seconds 🔴 • Link to the test results. • No reports index generated... • No test configuration ( |
|
🔴 Test of 'fournos_launcher submit' failed after 00 hours 09 minutes 01 seconds 🔴 • Link to the test results. • No reports index generated... Test configuration: |
Contributor
Author
|
/test fournos llm_d_legacy psap_h200 intelligentrouting-flavors |
|
🟢 Test of 'llm_d_legacy test' succeeded after 00 hours 07 minutes 12 seconds 🟢 • Link to the test results. • No reports index generated... • No test configuration ( |
|
🟢 Test of 'fournos_launcher submit' succeeded after 00 hours 07 minutes 44 seconds 🟢 • Link to the test results. • No reports index generated... Test configuration: |
Contributor
Author
|
I merge this to keep progressing with the framework testing. Will be further tested in upcoming PRs. The ruff testing failures are against the legacy code, not the new one. Not fixing it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary by CodeRabbit
Release Notes