Swiftlet

What people use it for

An open-source Swift and Metal runtime for running 35B and 80B Qwen mixture-of-experts models on Apple devices, including iPhones. It is available as a Swift package, CLI, OpenAI-compatible server, and iOS app.

ManuAGI - AutoGPT Tutorials
Runs large Qwen mixture-of-experts models on Apple devices by keeping the dense core in memory and streaming routed experts from SSD.

Github Awesome