CI Performance Baselines - SGLang Documentation

Sequential Request Checks

Server cases can exercise request-to-request state without restarting the server:

DiffusionTestCase(
    id="image_t2i",
    server_args=server_args,
    sampling_params=sampling_params,
    perf_repeat_requests=2,
)

Every request is checked, including the first one. Performance checks include load/runtime peak memory; consistency checks include audio when expect_audio_output=True. A failure in either request fails the case, even if the other succeeds. Reports retain the request index instead of overwriting the first result with the second. Keep workloads small, but regenerate baselines and GT whenever sampling parameters change. MiniMax-H3 T2VA and ref2va use different checkpoint partitions, so they remain separate cases, each issuing two requests to its own server. The T2VA case includes VAE layerwise offload to exercise weight restoration between requests; ref2va retains the resident VAE path. Use SGLANG_GEN_BASELINE=1 pytest -s ... -k <case_id> to collect baselines in the same sequential order, retaining the worst observed value of each metric. SGLANG_GEN_GT=1 with SGLANG_GT_OUTPUT_DIR saves GT; repeated requests share one GT.

Perf Baseline Generation Script

python/sglang/multimodal_gen/test/scripts/gen_perf_baselines.py starts a local diffusion server, issues requests for selected test cases, aggregates stage/denoise-step/E2E timings from the perf log, and writes the results back to the scenarios section of perf_baselines.json.

Usage

Update a single case:

python python/sglang/multimodal_gen/test/scripts/gen_perf_baselines.py --case qwen_image_t2i

Select by regex:

python python/sglang/multimodal_gen/test/scripts/gen_perf_baselines.py --match 'qwen_image_.*'

Run all keys from the baseline file scenarios:

python python/sglang/multimodal_gen/test/scripts/gen_perf_baselines.py --all-from-baseline

Specify input/output paths and timeout:

python python/sglang/multimodal_gen/test/scripts/gen_perf_baselines.py --baseline python/sglang/multimodal_gen/test/server/perf_baselines.json --out /tmp/perf_baselines.json --timeout 600

Nightly Comparison Telemetry

The nightly comparison reports the median of three client-side measurements. SGLang requests also write server perf dumps for stage-level timing. An unreadable dump does not invalidate a completed client measurement, but the dashboard reports server sample coverage (for example, 2/3) and computes the server total and stage medians only from readable dumps. Treat incomplete coverage as a telemetry warning rather than a performance regression by itself.

​Sequential Request Checks

​Perf Baseline Generation Script

​Usage

​Nightly Comparison Telemetry

Sequential Request Checks

Perf Baseline Generation Script

Usage

Nightly Comparison Telemetry