Three scaling routes

Cost on the horizontal axis; intelligence on the vertical axis. Black shows the current relationship between cost and intelligence. Purple extends it, green improves efficiency, and blue introduces a new scaling dimension.
Overview

Three ways to scale intelligence.

Cost and intelligence

Scaling is central to progress in LLMs. We explore three paths: invest more along existing dimensions, improve computational efficiency, and find new dimensions for growth.

01 Continue02 Improve03 New dimension
Route 01

Along the existing path.

Invest more. Raise the ceiling.

Increasing model parameters, training data, and reasoning effort can extend capabilities along an existing scaling path. As returns diminish, each further gain requires greater investment.

Parameters+Data+Reasoning
Route 02

Better trade-off.

The same capability, at lower cost.

Improving efficiency means reaching comparable capabilities with fewer resources. Quantization, sparse attention, speculative decoding, and adaptive compute offer different ways to reduce memory use, computation, or latency.

R2RTokenRouterFrameFusionMoA
Route 03

Find a new path.

Find a new scaling dimension.

Model size and data scale have long driven LLM progress. Models such as o1 highlighted another dimension: test-time compute. We explore further possibilities, including loop depth, model adaptivity, and context information density.

TaHTaH2C2CAKV

Research

Project highlights

We pursue these paths through joint optimization across algorithms and systems.

Route 02 · AdaptivityR2RToken-level routing for reasoning LLMsProject page Route 02 · AdaptivityTokenRouterA serving engine for token-level LLM routingProject page Route 02 · Model CompressionFrameFusionEfficient video token reduction for LVLMsProject page Route 02 · Model CompressionMoAMixture of attention for efficient LLMsProject page Route 03 · AdaptivityTaHAdaptive Looped LLMsProject page Route 03 · AdaptivityTaH2Better test-time scaling for looped LLMsProject page Route 03 · ContextC2CCommunicate through KV cache between LLMsProject page Route 03 · ContextAKVLet long-horizon agents manage their own KV cacheProject page