Lower your GPU bill.
We compress and optimize your open-weight model to serve the same traffic on half as many GPUs.
You only pay when we meet the agreed targets.
A few models we’ve optimized.
We make open-weight LLMs smaller and faster.
| Model | GPUs needed | Score keptaverage across benchmarks | Speedper answer | Model size |
|---|---|---|---|---|
| 50%fewer | 98%+of original | 2xfaster | 20%of original | |
| 50%fewer | 98%+of original | 2xfaster | 20%of original | |
| 50%fewer | 100%of original | 1.5xfaster | 20%of original | |
| 40%fewer | 98%of original | 1.5xfaster | 40%of original |
Measured on H100 against the original in dense BF16, at the same traffic and response time. Compression and optimized GPU kernels let each GPU serve more traffic.
See benchmark scores
| Benchmark | Original | Ours | Kept |
|---|---|---|---|
| MMLU-Redux | 87.2 | 85.9 | 98.5% |
| GSM8K | 94.1 | 93.2 | 99.0% |
| HumanEval | 88.4 | 86.9 | 98.3% |
| IFEval | 86.5 | 85.4 | 98.7% |
| GPQA Diamond | 58.6 | 56.8 | 96.9% |
| Average kept | 98.3% | ||
| Benchmark | Original | Ours | Kept |
|---|---|---|---|
| MMLU-Redux | 86.1 | 84.7 | 98.4% |
| GSM8K | 93.0 | 92.1 | 99.0% |
| HumanEval | 85.4 | 83.5 | 97.8% |
| IFEval | 88.7 | 87.9 | 99.1% |
| GPQA Diamond | 61.2 | 59.0 | 96.4% |
| Average kept | 98.1% | ||
Scores on a 0–100 scale; kept is the compressed score divided by the original’s, each benchmark weighted equally. Ask us for the Qwen3-8B and Gemma 4 E4B benchmark scores. We also test on your tasks and languages.
Running another model? Tell us about it →
What you get
Use fewer GPUs for your current traffic, run a more capable model, or fit more models on each GPU.
Serve the same traffic with fewer GPUs.
Lower your GPU costs, or free up hardware for other work.
$17,520saved a month
10 GPUs × $2.40 × 730 h ·$210,240 a year
The calculator needs JavaScript; this is the 20-GPU example.
Measured on H100, at the same traffic and response time.
Run a more capable model on your existing GPUs.
For documents and tasks your current model struggles with, make room for a model that performs better.
Original
Exceeds available memory
Compressed
Fits on your existing GPU
Fit more models on the same GPU.
Make room for the fine-tunes and task-specific models used across your teams.
Separate GPUs
Each model takes up its own GPU
Shared GPU
Your models share the same GPU
How it works
We optimize the model and the code that runs it.
We compress the model, retrain it to keep its quality, then write GPU software tuned to it and your hardware.
Each row adds an optimization. Retraining recovers model quality. Custom GPU kernels increase throughput.
How we work with you
You only pay when we meet the agreed targets.
-
1A 30-minute call.
We discuss which model you run, how you use it and what it costs you today. Then we agree the accuracy you need and the savings you want.
-
2About two weeks with your team.
We compress the model and test it on your own tasks and languages, starting from your fine-tune if you have one.
-
3You pay when we meet the agreed targets.
If we don’t, you pay nothing. You keep the compressed model and GPU kernels.
Questions
Will my model’s answers change?
We retrain it to keep its quality on your tasks, then test it against your original on the languages and formats your users rely on. You set the accuracy target before we start.
How is this different from quantization?
Quantization stores weights at lower precision. We also compress model layers with tensor ring decomposition, retrain the model to recover quality, and write GPU kernels for the compressed layers. We test the combined result on your workload.
Can this work with our current deployment?
We provide the compressed weights and GPU kernels for your serving engine, such as vLLM. On the call, we review the integration with your team. We tune for your GPUs while keeping the same context length, batching and API.
What if we fine-tune or update the model?
Bring the version you serve; we start from your fine-tune. Your weights and evaluation data stay under NDA and are deleted when the work ends. A new version needs a fresh compression pass, so tell us how often you update.
We serve many models. Does this scale?
Each model gets its own pass, about two weeks the first time. Tell us your catalogue and how often it changes, and we’ll plan the order.
How do you cut LLM inference costs?
We shrink the model’s layers, quantize it, retrain it to keep its quality, then write GPU kernels for the new shapes. The smaller model fits more requests on each GPU and runs faster, so you need fewer GPUs.
Talk to us
Let’s look at your model.
Tell us about your model, your GPUs and the workload you want to run.