LlamaIndex

llama.cpp is a C/C++ implementation for running large language and vision-language model inference locally or in the cloud, including on CPUs, GPUs, laptops, Apple silicon devices, and small computers such as Raspberry Pis. The project is built on the ggml library and uses integer quantization from 1.5-bit through 8-bit to reduce memory use and accelerate inference, with CPU, CUDA, HIP, Metal, Vulkan, SYCL, OpenCL, and other hardware backends. It supports hybrid CPU-GPU inference for models larger than available VRAM and includes command-line tools, a built-in web interface, and an OpenAI-compatible REST API server. The project is distributed as source code, Docker images, and pre-built binaries, with installation instructions at llama.app.

Langfuse is an open-source LLM engineering platform for developing, monitoring, evaluating, and debugging AI applications. It ingests traces for LLM calls and related operations such as retrieval, embeddings, and agent actions, with integrations including OpenTelemetry, LangChain, the OpenAI SDK, LiteLLM, and other LLM frameworks. Its features include prompt management with version control and collaborative iteration, LLM-as-a-judge and code-based evaluations, user feedback and manual labeling, datasets for test sets and benchmarks, trace replay and debugging, metrics, and a playground for testing prompts and model configurations. Langfuse exposes APIs and typed Python and JavaScript/TypeScript SDKs for custom LLMOps workflows. It is available as a managed cloud service or for self-hosting with Docker Compose, virtual machines, Kubernetes via Helm, or Terraform templates.

GuideLLM is an open-source platform for benchmarking and evaluating real-world LLM inference deployments. It simulates end-to-end interactions with OpenAI-compatible and vLLM-native servers, using real or synthetic text and multimodal datasets and configurable execution profiles such as synchronous, concurrent, throughput, constant-rate, Poisson, and sweep workloads. It measures SLO-oriented latency and token statistics, including time to first token, inter-token latency, and end-to-end behavior, then produces console, JSON, CSV, or HTML reports for deployment tuning, capacity planning, and regression tracking. The tool provides both a CLI and API, with multiprocessing, threading, asynchronous execution, and Hugging Face, file, synthetic, or custom data sources.

PageIndex is a vectorless, reasoning-based retrieval-augmented generation platform from VectifyAI. It replaces vector indexes and chunking with a hierarchical tree index for each document, then uses an LLM to search that tree agentically, following document structure and context to retrieve relevant sections with traceable references. The PageIndex SDK supports local indexing, retrieval, and chat with a user's own LLM key, as well as PageIndex Cloud for managed parsing, OCR, image understanding, tree-index construction, and storage. It can process text-based PDFs locally and supports cloud-hosted indexes for scanned and image-rich documents; the platform also provides an application for long professional documents, an agent integration interface, and a cloud-only file-level tree index for reasoning across document collections.

LLM Wiki is a cross-platform desktop application maintained by nash_su that turns imported documents into a persistent, organized, interlinked knowledge base. It uses a three-layer structure of immutable raw sources, LLM-generated wiki pages, and configuration rules, with ingest, query, and lint operations; generated pages use YAML frontmatter, source traceability, an index, an operation log, and [[wikilink]] cross-references. Its ingest pipeline first analyzes each source for entities, concepts, connections, contradictions, and recommended structure, then generates or updates wiki pages, summaries, indexes, research queries, and human-review items. SHA256 caching skips unchanged files, while a persistent queue supports serialized processing, retries, cancellation, and crash recovery. It accepts PDFs, Office documents, EPUB/MOBI, Org mode, images, media, web clips, and URL batches, with optional vision-based image captions and MinerU processing for complex PDFs. Retrieval combines tokenized keyword search, optional LanceDB vector search through OpenAI-compatible embedding endpoints, and graph expansion based on direct links, shared sources, Adamic-Adar similarity, and page-type affinity. The application also provides Louvain community detection, graph insights, deep research through Tavily, SerpApi, or SearXNG, a Rust tool-using chat agent, review workflows, Mermaid and KaTeX rendering, a Chrome web clipper, and project export/import. It includes a token-protected local HTTP API and MCP server for searching, reading files, traversing the graph, rescanning sources, and calling the backend agent. The application is built with Tauri v2, React, TypeScript, and Rust, has macOS, Windows, and Linux builds, and is licensed under GPL-3.0.