Skip to content

vLLM CPU Performance Evaluation

Comprehensive performance evaluation framework for vLLM on CPU platforms.

This site provides testing methodology, automation tools, and platform configurations for evaluating vLLM inference performance on CPU-based systems.

Getting Started

Install with ./cpueval install, then run your first benchmark

Quick Start Guide →

Test Suites

All supported benchmarks, cpueval commands, and suite documentation

Test Suites Overview →

cpueval CLI

Install, doctor, and run benchmark suites from one command

CLI Guide →

Models

Supported LLM, embedding, and audio models

Model Catalog →

Results

Analyze results with dashboards and experiment tracking

Dashboards Guide →

Methodology

Testing approach, metrics, and 3-phase strategy

Testing Methodology →

Test Suites at a Glance

Suite Status Focus
Concurrent Load Validated P95 latency under concurrent requests
Offline Batch Validated Bulk document processing
Embedding Models Validated Embedding throughput and latency
Audio Models Validated Whisper ASR performance
Scalability WIP Maximum throughput and sweep curves
Resource Contention Planned Multi-tenant scenarios

See the Test Suites Overview for cpueval commands, suite selection guidance, and links to detailed documentation.

Key Features

  • cpueval CLI — Matrix-first benchmarking with 8 test suites
  • 3-Phase Testing — Baseline, realistic, and production methodology
  • Ansible automation — Reproducible, distributed test execution
  • MTEB integration — Embedding quality evaluation
  • Streamlit dashboards — Interactive results analysis
  • MLflow tracking — Experiment comparison and history

Browse the full Documentation index or use the sidebar navigation to explore guides by topic.