tuner.py

Optuna is an open-source Python framework for automatic hyperparameter optimization, particularly for machine-learning and deep-learning workflows. It defines an objective function, uses a trial object to suggest parameter values, evaluates the resulting objective, and repeats the process through a study; in the described DigitalOcean agent, it applies this optimization loop to serving configurations, scores each trial, and adjusts the next recipe. Its eager search spaces support Python conditionals and loops, while its optimization algorithms can prune unpromising trials and run searches in parallel across threads or processes. Optuna is framework-agnostic and can be installed with pip; the site states that Python 3.10 or newer is supported.

GuideLLM is an open-source platform for benchmarking and evaluating real-world LLM inference deployments. It simulates end-to-end interactions with OpenAI-compatible and vLLM-native servers, using real or synthetic text and multimodal datasets and configurable execution profiles such as synchronous, concurrent, throughput, constant-rate, Poisson, and sweep workloads. It measures SLO-oriented latency and token statistics, including time to first token, inter-token latency, and end-to-end behavior, then produces console, JSON, CSV, or HTML reports for deployment tuning, capacity planning, and regression tracking. The tool provides both a CLI and API, with multiprocessing, threading, asynchronous execution, and Hugging Face, file, synthetic, or custom data sources.

NVIDIA Model Optimizer (ModelOpt) is an open-source library for optimizing deep-learning and generative-AI models for inference deployment. It accepts Hugging Face, PyTorch, or ONNX models and provides Python APIs for composing post-training quantization, quantization-aware training and distillation, pruning, neural architecture search, speculative decoding, and sparsity techniques, producing optimized or quantized checkpoints. The resulting checkpoints can be exported for inference frameworks including TensorRT-LLM, TensorRT, vLLM, and SGLang. The library also integrates with NVIDIA Megatron-Bridge, Megatron-LM, Hugging Face Accelerate, Transformers, and Diffusers for supported training and export workflows. It is distributed as the nvidia-modelopt package on PyPI and can also be installed from the NVIDIA GitHub repository or used through NVIDIA container images.

Soup is an open-source Python CLI for fine-tuning and post-training large language models from a YAML configuration and a single training command. It supports supervised fine-tuning, LoRA, QLoRA, preference optimization, quantization, automatic batch-size and GPU detection, and local training without a cloud service. Its optional layer-streaming mode keeps the frozen base model out of VRAM and feeds it to the GPU one decoder layer at a time, reducing memory use for low-VRAM systems. The project describes training an 8B model on a 4 GB laptop GPU with this mode; layer streaming is marked beta.

OPAL (OPtimizer with an Agent in the Loop) is a PyTorch optimizer wrapper developed by Xingjian Bai and contributors. It turns a fixed optimizer into two editable programs: an updater that converts gradients into parameter updates, and an analysis program that reports training statistics. A coding agent, Claude Code by default, periodically reads the goal, losses, gradients, and analysis reports, then can rewrite learning rates and schedules, momentum, preconditioning, clipping, parameter-group update rules, and related optimizer logic while training continues. New programs are trialed for 20 steps and can be reverted on errors when a parameter-and-state snapshot is available; until a change is made, the wrapped run is bit-for-bit equivalent to the original optimizer run. OPAL supports CPU, single-GPU, and single-node DDP PyTorch training, with integrations for selected Hugging Face Trainer and Accelerate workflows, but does not support multi-node training, FSDP, or DeepSpeed. Agent runs require Claude Code and an authenticated account, while the keep backend provides an agent-free dry run. The project is distributed under the MIT license.