Mistral Small 24B speed: tokens per second on GPUs and Macs
Mistral Small 24B (ollama run mistral-small:24b) is about 14.2 GB at Q4 (Ollama default).
It fits fully on graphics cards with 16 GB of VRAM or more (25 of the 38 GPUs and Macs below); on smaller cards part of it runs on the CPU and slows down sharply.
Nobody has measured it here yet, so the speeds below are estimates until readers add their runs.
How fast does a model actually run on your GPU or Mac? Speeds here are measured by readers with Ollama and checked against the --verbose output they paste. Where readers haven't measured a combination yet, you see a published benchmark from a linked public source when one exists, otherwise our estimate from memory bandwidth, marked as an estimate.
Generation speed (Ollama "eval rate"), single user, default context. Results vary with drivers, context length and background load. Measured values are the median of all approved runs. As an Amazon Associate we earn from qualifying purchases.
Measure it yourself: ollama run mistral-small:24b --verbose
| Hardware | Memory | Speed (tokens/s) | |
|---|---|---|---|
| 32 GB | ~88.6 Estimate
|
Check price ↗ | |
| 24 GB | ~49.8 Estimate
|
Check price ↗ | |
| 16 GB | ~47.5 Estimate
|
Check price ↗ | |
| 48 GB | ~47.5 Estimate
|
Check price ↗ | |
| 24 GB | ~47.5 Estimate
|
Check price ↗ | |
RTX 3090 24GB Desktop GPU |
24 GB | ~46.3 Estimate
|
|
| 16 GB | ~44.3 Estimate
|
Check price ↗ | |
| 24 GB | ~44.3 Estimate
|
Check price ↗ | |
| 16 GB | ~44.3 Estimate
|
Check price ↗ | |
| 192 GB | ~40.5 Estimate
|
Check price ↗ | |
Radeon RX 7900 XT 20GB Desktop GPU |
20 GB | ~39.5 Estimate
|
|
Mac Studio M2 Ultra 192GB Mac |
144 GB | ~39.5 Estimate
|
|
RTX 4080 / Super 16GB Desktop GPU |
16 GB | ~35.4 Estimate
|
|
| 16 GB | ~33.2 Estimate
|
Check price ↗ | |
| 16 GB | ~31.6 Estimate
|
Check price ↗ | |
Radeon RX 7800 XT 16GB Desktop GPU |
16 GB | ~30.8 Estimate
|
|
| 36 GB | ~30.4 Estimate
|
Check price ↗ | |
RTX 4090 Laptop 16GB Laptop GPU |
16 GB | ~28.5 Estimate
|
|
| 96 GB | ~27 Estimate
|
Check price ↗ | |
| 27 GB | ~22.7 Estimate
|
Check price ↗ | |
| 16 GB | ~22.1 Estimate
|
Check price ↗ | |
Mac M3 Max 64GB Mac |
48 GB | ~19.8 Estimate
|
|
Mac M3 Max 128GB Mac |
96 GB | ~19.8 Estimate
|
|
| 16 GB | ~14.2 Estimate
|
Check price ↗ | |
Mac M4 Pro 48GB Mac |
36 GB | ~13.5 Estimate
|
|
| 12 GB | Doesn't fit in 12 GB: runs partly on the CPU (slow) | Check price ↗ | |
RTX 4070 12GB Desktop GPU |
12 GB | Doesn't fit in 12 GB: runs partly on the CPU (slow) | |
RTX 4060 8GB Desktop GPU |
8 GB | Doesn't fit in 8 GB: runs partly on the CPU (slow) | |
RTX 3080 12GB Desktop GPU |
12 GB | Doesn't fit in 12 GB: runs partly on the CPU (slow) | |
RTX 3080 10GB Desktop GPU |
10 GB | Doesn't fit in 10 GB: runs partly on the CPU (slow) | |
RTX 3070 8GB Desktop GPU |
8 GB | Doesn't fit in 8 GB: runs partly on the CPU (slow) | |
RTX 3060 12GB Desktop GPU |
12 GB | Doesn't fit in 12 GB: runs partly on the CPU (slow) | |
RTX 5070 Ti Laptop 12GB Laptop GPU |
12 GB | Doesn't fit in 12 GB: runs partly on the CPU (slow) | |
| 8 GB | Doesn't fit in 8 GB: runs partly on the CPU (slow) | Check price ↗ | |
| 8 GB | Doesn't fit in 8 GB: runs partly on the CPU (slow) | Check price ↗ | |
| 10.7 GB | Doesn't fit in 10.7 GB: runs partly on the CPU (slow) | Check price ↗ | |
| 10.7 GB | Doesn't fit in 10.7 GB: runs partly on the CPU (slow) | Check price ↗ | |
| 10.7 GB | Doesn't fit in 10.7 GB: runs partly on the CPU (slow) | Check price ↗ |
Add your result (2 minutes)
- Run the model with Ollama in verbose mode:
ollama run qwen3-coder:30b --verbose - Ask for a long answer, for example: Write a 300-word story about a robot learning to paint.
- Copy everything Ollama prints after the answer (from
total durationtoeval rate) and paste it below.
Results with the pasted Ollama output go live straight away when they are plausible; others are checked by hand first. We store an anonymous hash of your connection only to prevent duplicate submissions.
Latest measured runs
- No measured runs yet. Be the first: add yours above.
Benchmarks by model
Speed tables for each model on every GPU, laptop GPU and Mac in our list.
- Qwen3-Coder 30B (MoE) speed · 18.3 GB at Q4
- gpt-oss 20B (MoE) speed · 12.6 GB at Q4
- gpt-oss 120B (MoE) speed · 70.2 GB at Q4
- Qwen3 8B speed · 4.9 GB at Q4
- Qwen3 14B speed · 8.9 GB at Q4
- Qwen3 32B speed · 19.7 GB at Q4
- Qwen2.5-Coder 7B speed · 4.6 GB at Q4
- Qwen2.5-Coder 32B speed · 19.7 GB at Q4
- Gemma 3 12B speed · 7.3 GB at Q4
- Gemma 3 27B speed · 16.4 GB at Q4
- Llama 3.1 8B speed · 4.8 GB at Q4
- Llama 3.3 70B speed · 42.4 GB at Q4
- DeepSeek-R1 14B (distill) speed · 8.9 GB at Q4
- DeepSeek-R1 32B (distill) speed · 19.7 GB at Q4
- Mistral Small 24B speed · 14.2 GB at Q4
How these numbers work
What is measured?
Generation speed: how many tokens per second the model writes once it starts answering. That is Ollama's "eval rate" line. Prompt processing speed and load time are not included.
Where do the estimates come from?
From memory bandwidth: generation is limited by how fast the GPU or Mac can read the model's active weights, so we divide bandwidth by the model's size at that quantization. Estimates are shown only when the whole model fits in VRAM or unified memory; "doesn't fit" means part of it would run on the CPU and the speed depends heavily on your system.
What are "Published" results?
Real measurements from public benchmark projects, shown with a link to the source and its method in the tooltip. They are used only where the model, quantization and hardware match, and they step aside once readers have added three or more runs of their own. They may use llama.cpp directly rather than Ollama (Ollama is built on llama.cpp) and an older version, so treat small differences as normal.
Why a median?
The same card can differ by driver, power limit or laptop cooling. The median of several runs is a fair "typical" number, and one unusual result can't skew it.
Can I trust reader results?
Results go live automatically only with the pasted Ollama output, and only when they are close to other runs (or to the estimate when a combination is new). Everything else is reviewed by hand. The number of runs is shown next to every measured value.