Documentation

Central index for the vLLM CPU Performance Evaluation framework.

Start Here

I want to…	Go to
Set up and run my first benchmark	Getting Started (./cpueval install)
See all supported test suites	Test Suites
Run benchmarks from the CLI	cpueval CLI
Find supported models	Model Catalog
View and analyze results	Dashboards Quickstart

Documentation Map

Getting Started

Page	Description
getting-started.md	./cpueval install, environment setup, and first run
cpueval-cli.md	CLI reference (install, doctor, suites)
test-suites.md	Central test suite reference

Test Suites & Models

Page	Description
test-suites.md	All suites, cpueval commands, and status
embedding-models.md	Embedding model setup guide
audio-benchmarking.md	Audio model setup guide
models/models.md	Model catalog and selection rationale
tests/	Per-suite methodology docs

Running Tests

Page	Description
ansible/test-execution.md	Ansible playbook automation
ansible/model-predownload.md	Model pre-download guide
scripts-reference.md	Utility scripts
environment-variables.md	Configuration reference

Methodology & Results

Page	Description
methodology/overview.md	Testing approach
methodology/metrics.md	Metric definitions
methodology/reporting.md	Result formats
dashboards-quickstart.md	Streamlit visualization
mlflow.md	Experiment tracking
terminal-results-viewer.md	CLI results viewer

MTEB Quality Testing

Page	Description
mteb-sweep-guide.md	MTEB quick start
mteb-timing-guide.md	Test duration planning
mteb-troubleshooting.md	Common issues

Configuration

Page	Description
platform-setup/x86/intel/deterministic-benchmarking.md	Intel platform tuning
vllm-kv-cache-configuration.md	KV cache settings

By Role

New Users

1. Getting Started — Setup and first benchmark
2. Test Suites — Choose the right suite
3. cpueval CLI — Run benchmarks
4. Dashboards Quickstart — View results

Running Benchmarks

1. Test Suites — Suite overview and quick reference
2. cpueval CLI — CLI commands and options
3. Ansible Test Execution — Playbook automation
4. Scripts Reference — Utility scripts

Understanding Results

1. Metrics Guide — Metric definitions
2. Reporting Guide — Result formats
3. Dashboards Quickstart — Visualization
4. MLflow Integration — Experiment tracking

Platform & Configuration

1. Intel Platform Setup
2. Environment Variables
3. vLLM KV Cache Configuration

Quality Testing (MTEB)

1. MTEB Quick Start
2. MTEB Timing Guide
3. MTEB Troubleshooting

Contributing to Documentation

Documentation is written in Markdown. Run pre-commit checks before committing:

pre-commit run --all-files