ktransformers

What people use it for

An open-source framework for inference and fine-tuning of very large language models using CPU-GPU heterogeneous computing. It runs mixture-of-experts models by keeping hot experts on the GPU and the rest on optimized CPU kernels.

ManuAGI - AutoGPT Tutorials