neuron-explorer

NanoJev is an open-source 0.6B parallel-decision model and training pipeline based on Qwen3-0.6B. It accepts states, questions, and candidate sets in a single forward pass and returns complete probability distributions without output-token decoding. Shared decision heads support dynamic-choice distributions over supplied candidates, Boolean proposition probabilities, and ordered score levels with expected values. The repository includes data generation, distribution-loss training, evaluation, persistent serving, and browser replay tools. Its game demonstrations combine model judgments with controller code: maze controllers ask local safety questions and track collisions and explored paths, while Snake controllers filter immediate collisions, plan toward visible food, and use the model to break ties between remaining actions. Checkpoints and datasets are distributed through Hugging Face, and the service exposes repeated batched evaluations through a local HTTP endpoint.

Strata is a free, open-source local inference engine for running the 125-billion-parameter Qwen3.8-Flash-Next model on consumer Windows or Linux PCs. It distributes the model across the GPU, system RAM, processor, and, for some configurations, SSD storage; a mixture-of-experts design keeps frequently used experts on the graphics card while RAM and the processor handle the remainder, with speculative decoding to check batches of predicted tokens. It provides a browser chat interface, monitoring for model and hardware use, optional image input, and localhost APIs compatible with OpenAI, Anthropic, and OpenAI Responses clients for applications and coding agents. The installer selects an engine and model size based on available hardware, and the project supports NVIDIA and AMD graphics cards with at least 12 GB of VRAM, 32 GB or more of RAM, and Windows 10/11 or Linux. Strata is licensed under the MIT License; the included model and some components have separate licenses.

Bespoke Nimble is an open-source model, training recipe, and local scoring library from Bespoke Labs for making typed decisions about text. Given a context and a flat schema of boolean or fixed-choice fields, it selects an allowed answer and returns candidate logits and probabilities without generating a reasoning explanation or free-form answer. Nimble fine-tunes Qwen3.5-9B with LoRA using contrastive training pairs: two nearly identical examples differ in one relevant fact, which changes the correct label. At inference, the scorer reads the logits for one-token candidate codes and converts them to probabilities with softmax; the Python library constructs the typed result rather than parsing generated JSON. The MLX ParallelScorer processes shared context once and scores fields in parallel, while the CUDA scorer scores each field separately. The repository includes the model recipe, curation and evaluation code, datasets, examples, tests, and deployment documentation. Bespoke-Nimble-9B can run locally on Apple Silicon or an NVIDIA BF16-capable GPU. It accepts text only, does not support nested schemas or answers outside the supplied choices, and its probabilities are not guarantees of correctness.

An Apache-2.0-licensed serving setup for Qwen3.8-27B on a single 24 GB NVIDIA GPU, built around vLLM and exposing an OpenAI-compatible API with optional key authentication. Its preparation pipeline starts from a W4A16 AutoRound model, requantizes the language-model head and embedding matrices to int8, requantizes the MTP components, and applies vLLM patches for the serving stack. The runtime combines int8 tensor-core GEMMs with an fp16 recurrent state, continuous batching, split-KV verification attention, and speculative decoding through either Qwen's MTP path or the optional DFlash2 block drafter; DFlash2 can also draft from the request's cached context for document-reproduction workloads. The repository provides Docker Compose profiles for batch throughput and single-user latency, prebuilt container images, model-download and requantization scripts, benchmark and quality-test tools, and launchers for fast, long-context, and experimental KVarN cache modes. The standard configurations target roughly 64k to 150k tokens of context, while KVarN and alternative int4 KV-cache paths extend the claimed capacity to the 256k-token range with lossy KV quantization. The setup requires a recent NVIDIA driver and a compatible Ampere-or-newer GPU; the repository notes that its benchmark figures were measured on an RTX 3090 subject to a 250 W power limit.

Needle 2 is an open-source, 45-million-parameter foundation model for tool calling, device use, and structured extraction on small devices. Its weights are compressed into a single 14 MB binary and integrated with its inference engine; the repository reports that a full session uses about 28 MB of RAM and performs inference without network access after setup. The model converts text and tool descriptions into schema-constrained JSON calls using a byte-level grammar, attaches a learned confidence score for threshold-based escalation, and can retrieve the top five tools from a larger catalogue. It uses a 256-token sliding window with tools pinned as key-value sinks to bound memory, and its Simple Attention Network combines a Hadamard MLP, grouped-query attention, engram key-value memory, and multi-lane hyper-connections. The Python package supports inference, LoRA fine-tuning, and checkpoint export; the inference engine is fetched from Hugging Face and cached, while the training stack is an optional installation.