AI product
llm-d semantic classifier is an incubating llm-d project for routing conversations to an appropriate language model rather than selecting models only for inference efficiency. Its routing can use factors such as cost or conversation topic, including directing requests to a fine-tuned specialized model or to a smaller, less expensive model instead of a larger one. It is described as a sibling of the vLLM semantic router project.
1 use taken from transcripts — each links to the moment in the video.
An llm-d project being incubated that routes conversations not just efficiently but to the right model — right could mean cost-based or topic-based, e.g. choosing a fine-tuned specialized model or a cheaper small model instead of an expensive large one. Sibling to the vLLM semantic router project.
1 in the library.