AI product Open source
Vulnerable AI Student Assistant is a deliberately insecure AI chatbot application for testing and capture-the-flag exercises. It uses a Python FastAPI backend, a plain HTML/CSS/JavaScript chat frontend, and Gemini to answer questions about mock students, their names, and enrolled classes. The backend reads the complete student database, including grades, into the prompt while a system instruction tells the model not to disclose grades, allowing tests of prompt-based data leakage, prompt injection, and variable model behavior. The repository includes example attacks and points to a separate project containing mitigations.
3 uses taken from transcripts — each links to the moment in the video.
A purposely vulnerable student chatbot used as a capture-the-flag exercise for red teaming: it holds confidential student grades it is instructed not to disclose, and attendees try prompts to extract them, demonstrating prompt-based leakage, variable model behavior, and why access controls must sit outside the prompt.
Intentionally vulnerable LLM application to practice threat modeling on
An intentionally vulnerable student assistant chatbot repo (FastAPI with Python and a Gemini model) that Snyk uses for training, letting practitioners practice prompt injection and threat modeling against it.
2 in the library.