AI Vocabulary · 06 Own Your AI
What is Model Routing?
Sending each request to the model that fits it best, small and cheap for easy jobs, large for hard ones.
Model routing means not using one model for everything. A router, which can be a simple rule or a small classifier, looks at each request and sends it to the most suitable model: a small language model for classification or short rewrites, a frontier model for complex reasoning, a local model for sensitive data.
Done well, routing lowers cost and latency without lowering quality. Done badly, hard requests land on weak models and quality drops quietly, which is why routing needs evals.
Why it matters for you
Most AI workloads are a mix of easy and hard tasks. Paying top-model prices for every request is often the biggest avoidable line in an AI budget.
Own it
Routing is also a lock-in defence: a system that already talks to several models can drop or swap one without a rebuild. Start simple, with two models and clear rules, and measure cost per task before and after.
Where it comes from
Routing between models of different size and price is a young research field. Stanford's FrugalGPT (2023) showed that sending queries through a cascade of cheaper and stronger models can cut costs sharply while keeping quality. In July 2024 the team behind Chatbot Arena released RouteLLM, an open-source framework that trains routers on human preference data.
Related
Your AI Learning Hub
Learn the whole AI Vocabulary, step by step, and build with it. Build it. Own it.