Skip to content

Add setup scripts for Benchmark VM and GPU VM#2338

Merged
cb-github-robot merged 2 commits intocloud-barista:mainfrom
leehyeoklee:gpu-telemetry
Mar 3, 2026
Merged

Add setup scripts for Benchmark VM and GPU VM#2338
cb-github-robot merged 2 commits intocloud-barista:mainfrom
leehyeoklee:gpu-telemetry

Conversation

@leehyeoklee
Copy link
Copy Markdown
Contributor

Add Streamlined LLM Serving and Benchmarking Setup Scripts

This PR introduces two new setup scripts that simplify the process of deploying LLM serving infrastructure and benchmarking environments with real-time metrics monitoring.

New Scripts

1. setupGpuVM.sh

A comprehensive setup script for GPU VMs that automates the entire LLM serving stack deployment in a single command.

Features:

  • Installs vLLM framework
  • Deploys and serves specified LLM model
  • Configures GPU telemetry for metrics collection

2. setupBenchmarkVM.sh

Sets up a dedicated benchmark VM with monitoring infrastructure and testing tools.

Features:

  • Deploys Prometheus + Grafana monitoring stack for multi-GPU VM monitoring
  • Downloads export_metrics.sh for CSV-based metrics export
  • Downloads run_guidellm.sh for benchmark execution

Usage Examples

GPU VM Remote Command Execution

Step 1: Install GPU driver (requires reboot - separate step)

# Use existing driver installation remote command
curl -fsSL https://raw.githubusercontent.com/cloud-barista/cb-tumblebug/main/scripts/usecases/llm/installGpuDriver.sh | bash

Step 2: Deploy vLLM, serve model, and configure telemetry

curl -fsSL https://raw.githubusercontent.com/cloud-barista/cb-tumblebug/main/scripts/usecases/llm/setupGpuVMForLLM.sh | bash -s -- Qwen/Qwen2.5-1.5B-Instruct

Benchmark VM Remote Command Execution

Step 1: Setup monitoring stack and download benchmark tools

GPU_VM_IPS='<GPU_VM_IP1> [GPU_VM_IP2] ...'
curl -fsSL https://raw.githubusercontent.com/cloud-barista/cb-tumblebug/main/scripts/usecases/llm/telemetry/setupBenchmarkVM.sh | bash -s -- $GPU_VM_IPS

@seokho-son
Copy link
Copy Markdown
Member

Hi @leehyeoklee
Thanks for this contribution! I think these scripts will be very useful for repeated experiments.

I have a minor comment. The name of each script file is not very intuitive to me. Could you consider improving the naming to make it clearer?

@leehyeoklee
Copy link
Copy Markdown
Contributor Author

@seokho-son

setupGpuVm.sh => setupvLLMTelegraf.sh
setupBenchmarkVm.sh => setupBenchmarkMonitoring.sh

how about it?🤔

@seokho-son
Copy link
Copy Markdown
Member

@leehyeoklee

How about

setupGpuVm.sh => setupBenchmarkTarget.sh
setupBenchmarkVm.sh => setupBenchmarkManager.sh

according to our objective. :)

@leehyeoklee
Copy link
Copy Markdown
Contributor Author

@seokho-son
Thanks for the feedback!
I agree that these names are more intuitive and better represent the actual functions.

I've updated the script names. Thanks for the help! 😊

@seokho-son
Copy link
Copy Markdown
Member

/approve

@github-actions github-actions bot added the approved This PR is approved and will be merged soon. label Mar 3, 2026
@cb-github-robot cb-github-robot merged commit fbb7c5f into cloud-barista:main Mar 3, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved This PR is approved and will be merged soon.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants