LZ4

Glyd is a Rust lossless-compression library and command-line tool with a C ABI, streaming interfaces, and Python and Go bindings. It targets object storage, data lakes, logs, telemetry, backups, versioned exports, RPC payloads, and caches as an alternative to LZ4, Snappy, and zstd. Its record mode detects delimited records, SQL dumps, JSON lines, and varying-shape logs, converts fields or template variables into typed streams, and compresses them in parallel. Base mode compresses a new version against an older version by using regions of the old data as history. The format uses independently decodable blocks and parallel paths; higher levels use separated token, offset, length, and literal streams with SIMD-oriented decoding, while long-distance matching can find repeated content up to 128 MB back. The repository also provides shape dictionaries for small objects, packs for many small objects, container and JPEG handling at selected levels, and a cold context-mixing level for infrequently read data. The separate glyd-store crate and CLI select similar stored objects as delta bases using fingerprints, cap delta chains, and support local directories and S3-compatible backends. The codec, CLI, C ABI, and bindings use the BSD 3-Clause License or GPL-2.0; glyd-store uses the Business Source License 1.1, with commercial production use requiring a license and conversion to Apache-2.0 four years after each release.

Headroom is an open-source context-compression tool for AI agents and LLM applications, provided as a Python library, TypeScript SDK, local proxy, CLI wrapper, and MCP server. It compresses tool outputs, logs, files, RAG chunks, source code, JSON, and conversation history before they reach an LLM; compression runs locally and the project states that prompt and file content are not sent elsewhere for compression. Its ContentRouter selects SmartCrusher for JSON, CodeCompressor for source code, and Kompress-v2-base for prose. CacheAligner identifies volatile content that could invalidate a provider KV-cache prefix without rewriting prompts, while CCR caches originals locally so an agent can retrieve the full content on demand. The proxy can be used without code changes, and the MCP server exposes compression, retrieval, and statistics tools. Headroom also provides cross-agent memory, failure learning that writes corrections to agent instruction files, and optional output-token shaping. The project supports Python 3.10+, a TypeScript SDK, Docker distribution, and integrations with OpenAI-compatible and Anthropic clients, coding agents, and several application frameworks. Its repository is licensed under Apache 2.0. The repository reports that compression savings depend on the input, with repetitive JSON and logs benefiting more than prose or already-dense content.

JSZip is a JavaScript library for creating, reading, and editing ZIP archives. Its API lets applications add generated content to files and folders, then asynchronously produce a ZIP archive such as a browser Blob for download; it also supports base64 data and use with Node.js. The project is distributed under either the MIT or GPLv3 license and can be installed with npm or included manually in a web page.

HatchZip is a Rust archive extractor with desktop and command-line interfaces that extracts archives in place while reclaiming disk space. It saves the archive index separately, extracts independent pieces from the end of ZIP, 7z, tar, and RAR archives, and truncates or deletes each completed portion so the freed space can hold subsequent output; split archives are consumed volume by volume, while tar.gz streams are processed from the front with filesystem hole punching where supported. Before changing anything, it checks the required space, preserves a configurable free-space margin, validates paths and archive integrity, and writes and flushes each output file before removing its corresponding archive data. Interrupted or crashed extractions can resume, and unfinished extractions can restore the remaining archive. It supports ZIP, 7z, tar, tar.gz, RAR volume sets, and split archives, with optional archive preservation through a normal extraction mode. HatchZip is distributed for Windows, macOS, and Linux, and is dual-licensed under Apache-2.0 or MIT; RAR support uses the separately licensed UnRAR source.

LLM Compressor is a tool for quantizing user-provided language models.