PII/MNPI leak audit and suppression for LLM-as-judge pipelines
UI redaction hides sensitive data from human users, but LLM judge system prompts (e.g. an instruction to quote the agent's answer for human verification) can cause the judge's persisted reasoning to un-redact PII or MNPI. Audit judge traces for leaks, then persist scores always but persist reasoning only after a sensitivity check, or run a second judge on redacted input.
online B2B Deep Tech / Infrastructure
From AI Engineer — Shipping AI Into 5 Major Banks Without a 12-Month Compliance Wait — Al Chen, Cisco
Problem: Side-channel leakage: the UI looks redacted, but LLM judge reasoning stored and displayed can contain un-redacted customer PII or material non-public information.
For: Banks and any AI platform vendors whose evaluation pipelines surface judge reasoning to internal users
Products from this video
Cisco ClickHouse ClickHouse backup Daytona Harbor MITRE NVD Postgres Prisma ORM Redis Sysdig Wiz
Other takes on AI security infrastructure
- Agentic consent governance layer for autonomous AI systems
- Enterprise agentic last-mile identity and access control layer
- Identity-based security infrastructure for multi-agent AI systems
- AI-agent security and control infrastructure for enterprises
- A guardrail tool that enforces strict test-driven development and other engineering rules for AI coding agents
- Privacy-first document preprocessing layer for AI coding agents