AI product Open source
RouteLLM is an open-source framework from lm-sys for serving and evaluating routers that direct requests between stronger and cheaper language models. It provides trained routers, a drop-in replacement for the OpenAI client, and an OpenAI-compatible server; routing decisions use configurable cost thresholds that can be calibrated for a target share of requests sent to the stronger model. The framework supports comparing routers across benchmarks and extending it with additional routers, model pairs, and providers.
1 use taken from transcripts — each links to the moment in the video.
Select an AI model for heavier business-listing compilation and other research prompts.
1 in the library.