mitra-finetune

Mitra-v2 Classifier is a tabular foundation model from AutoGluon and Amazon for classification tasks. It is pretrained entirely on synthetic datasets sampled from random classifier priors, including a Hybrid SCM prior, rather than on real-world datasets. The model uses a 12-layer 2D Transformer with attention across both rows and columns and incorporates in-context learning. Its recommended recipe fine-tunes 50 steps on a user's table and bags eight model copies for prediction; it can be run through the mitra-finetune package or used as a Mitra model in AutoGluon Tabular. The repository describes it as the second generation of Mitra, with a longer context, more features, and an improved optimizer than Mitra-v1. The weights are distributed on Hugging Face under the Apache-2.0 license and require a CUDA GPU for the documented fine-tuning and eight-fold bagging recipe.

Mitra-v2 Regressor is a tabular foundation model from AutoGluon for regression tasks. It is pretrained entirely on synthetic datasets sampled from mixed random-regressor priors, including a Hybrid SCM prior, and uses in-context learning with a 12-layer 2D Transformer that attends across rows and columns. Regression is formulated as classification over 1,000 target bins: the model predicts a distribution for each row, whose mean provides the point prediction and whose probabilities can also produce quantiles and probabilistic metrics such as CRPS. Using the accompanying mitra-finetune package, the model is fine-tuned for 50 steps and bagged across eight copies on a CUDA GPU. It accepts tabular features and regression targets and returns point predictions or full predictive distributions. The weights are distributed through Hugging Face under the Apache-2.0 License; the package is required because standard AutoGluon expects the scalar head used by Mitra-v1 rather than Mitra-v2's distributional head.

AutoGluon is an open-source Python machine-learning library for automated modeling of tabular data, time series, and other data types. It trains end-to-end predictive models by selecting and combining suitable algorithms, including classic machine-learning models, ensembles, and foundation models; its tabular API can fit a predictor from a training file and generate test-set predictions with a few lines of code. The library supports Python 3.10–3.13 on Linux, macOS, and Windows, with optional GPU support, and is licensed under Apache 2.0.

MiniMind is an open-source tiny large-language-model project and end-to-end training tutorial developed by Jingyao Gong. Its main dense model is approximately 64M parameters, and the repository provides the model architecture, tokenizer, datasets, inference code, and training pipeline. The pipeline covers pretraining, supervised fine-tuning, hand-written LoRA, DPO, PPO, GRPO, CISPO, model distillation, tool calling, adaptive thinking, and agentic reinforcement learning. Core algorithms are implemented directly with native PyTorch rather than relying on high-level abstractions from third-party training libraries. Its agentic reinforcement-learning path performs multi-turn rollouts, executes generated tool calls, appends tool observations to the context, calculates trajectory-level rewards, and updates the policy; rollout can use local PyTorch generation or an SGLang server. MiniMind supports dense and mixture-of-experts variants, single- and multi-GPU training, YaRN-based RoPE length extrapolation, Transformers-format models, and inference through llama.cpp, vLLM, or Ollama. The repository also includes a Streamlit chat interface and a lightweight OpenAI-compatible API server with tool-call and reasoning fields. It is released under the Apache License 2.0.

KTransformers is an open-source research framework for efficient large-language-model inference and fine-tuning through heterogeneous CPU-GPU computing. Its user-facing capabilities are provided through the kt-kernel source tree: high-performance inference and supervised fine-tuning (SFT). For mixture-of-experts models, it uses heterogeneous expert scheduling, placing frequently used or “hot” experts on GPUs and other experts on CPUs with optimized AMX, AVX512, or AVX2 kernels and NUMA-aware memory management. It supports CPU-side INT4/INT8 quantized weights, GPU-side GPTQ, native BF16 and FP8 precision, and multiple CPU, GPU, and NPU backends. The SFT capability integrates with LlamaFactory for hybrid CPU-GPU fine-tuning of large MoE models, including LoRA and full-parameter training, with INT4/INT8 quantization options.