Gandalf (Lakera)

Promptfoo is an AI application security and evaluation platform for testing prompts, RAG systems, agents, models, and guardrails. It can run evaluation tests with LLM-judge scoring against defined criteria and verify citations against source documents; in the described payment-rules assistant, it was used to assess answer quality and investigate apparent regressions against the underlying evidence. Its security workflows simulate context-aware attacks, including prompt injection, jailbreaks, data leaks, business-rule violations, insecure agent tool use, and toxic-content generation, and can run in development and CI/CD workflows. Promptfoo also provides automated red teaming, code scanning, MCP security testing, remediation guidance, and continuous monitoring, with integrations for AI applications, agent frameworks, CI/CD systems, and on-premises or cloud deployments.

Vulnerable AI Student Assistant is a deliberately insecure AI chatbot application for testing and capture-the-flag exercises. It uses a Python FastAPI backend, a plain HTML/CSS/JavaScript chat frontend, and Gemini to answer questions about mock students, their names, and enrolled classes. The backend reads the complete student database, including grades, into the prompt while a system instruction tells the model not to disclose grades, allowing tests of prompt-based data leakage, prompt injection, and variable model behavior. The repository includes example attacks and points to a separate project containing mitigations.

TryHackMe is a browser-based cybersecurity learning service with interactive rooms for practicing security concepts, including beginner exercises on finding and exploiting vulnerabilities. It also provides a learning roadmap and offers free learning alongside paid annual subscriptions.

Langfuse Workshop is a step-by-step learning resource about applying Langfuse across the AI engineering loop. It uses a small TypeScript Dad IT Support Agent, named Specs, to teach tracing agent steps, managing prompts, monitoring production behavior, building datasets, running experiments, and evaluating changes. The workshop includes self-guided learner lessons, instructor notes, preparation materials, checkpoint branches, a runnable sample application, and helper scripts; its material is maintained in the public langfuse/langfuse-workshop repository. The sample agent uses an OpenAI tool-calling loop with two local tools, and the monitoring exercises cover user disagreement and all-caps frustration. The workshop is licensed under MIT and recommends Langfuse Cloud because self-hosted instances may not match the features used in the lessons.

geiger is a read-only command-line scanner from Atomburst that inventories AI agents, harnesses, MCP servers, plugins, skills, hooks, browser extensions, desktop AI applications, and related IDE configurations on a machine. It reads known configuration files and directories, identifies each finding's origin and evidence path, and labels capabilities such as code execution, secret storage, broad filesystem access, broad web access, and network access. Credential-shaped values are reported by key name, file, and secret type without exposing value contents; the scanner performs no telemetry and writes only an explicitly requested report file. It can emit terminal, HTML, or versioned JSON reports, scan specified project or home directories, enforce a strict exit status for findings that can execute code or hold secrets, and compare JSON snapshots to identify newly appeared, removed, or escalated findings. It runs through npx with Node.js 18 or newer, has no runtime dependencies or account requirement, and is licensed under MIT. The project notes that it reads configuration rather than runtime behavior, covers known locations, and does not determine whether a package is malicious.