A Valid Key Let an AI Agent Delete Production - Sagar Batchu, Speakeasy Transcript, AI Summary & Key Points

00:12 Hey everyone. Thanks for coming today. My name is Sagar. I'm CEO and co-founder of Speakeasy. We are an AI control plane that helps you securely scale AI within your business. So what this means is we we help take all the AI you have, put it on a governed path, and help you authorize all the actions it's taking. I'm I'm here today to talk to you about how we can actually intentionally authorize agent actions.

00:39 I'm going to recreate a little incident that was in the news a couple months ago, use that to talk to kind of the scenario of how you can authorize actions and prevent prevent such an incident. The question of whether agents are capable at a task is long gone today. The real focus in most businesses is what happens when one of them does something. In most regulated industries, every action has to be has to be attributed to someone or something and and some identity.

01:07 So that's really the the core of what I wanted to talk here. Let me first do a quick little animation I put together of the incident that I was talking about. If you remember back in April, I won't name names, but there was a startup founder at the startup who thought he was fixing a staging environment but accidentally deleted his production database.

01:35 I've got this funnel animation to kind of remind you of that story. The summary of the story goes as I said, the developer asks a coding agent to fix a credential issue in staging. This is a pretty ordinary request, ordinary afternoon. The agent starts by reading the workspace and in an unrelated file, it finds a database access token. This is an infrastructure token with full access.

02:00 It decides the fix is to delete the volume and recreate it. Except the volume it has access to is not staging and nothing is standing between that guess and the API. There's no identity on the call, no policy, no approval and really no record at all. So in one fell swoop we've deleted production, it's back up and all of that lived within the same network boundary.

02:28 So it's all gone in one instant. Took about 10 seconds to kind of completely bring down the startups whole database and and their backup. Now what I want to leave you with as we kind of go on to the next part of this presentation is that the agent had valid credentials but it didn't have valid intent. That's the real difference between the kind of right and wrong in the situation.

02:56 So my argument I'm going to make to you here is that the unchecked authority is the real failure. An agent can be wrong, it can miss context. It's pretty well understood now, you know, we've all been in that situation before. But what you didn't see in that scenario is that there was no exploit, there was no jailbreak, no prompt injection. There was really no malicious actor.

03:15 There was nothing that anyone was doing intentionally to kind of harm the system or harm the company. The valid key did exactly what valid keys do, right? The agent did what it thought it should do. So the failure for regulated business is that not not that the mistake happened but nothing stood between the agent and its judgment and the API. So there were no real guardrails in place.

03:42 Before we get into kind of the blueprint of how we can do better here, I wanted to give you three numbers that represent how this is playing out in our industry. The first 82% of all organizations already run AI agents, no surprise probably to anyone. 32% had an AI governance policy in place. So probably more surprising given the difference between the two.

04:05 Um and then finally, when a breach does happen, thanks to Shadow AI of some kind, uh the average cost is usually calculated to be about $5 million or so. So, that that that's a fun stat. Um but essentially, we can conclude that um air adoption is universal, but the control layer doesn't really exist and it's definitely not universal. Um before we go to looking at what a control layer for agents could be in the scenario, um what let's actually just start at what we've asked people our employees to do uh for as long as

04:39 we've known. We've held people to this kind of standard as long as there's been software or you know, SaaS software of any kind. Um the the things that we hold people to are you have to sign in as yourself. Uh we ask nobody we ask everyone to you know, not share passwords or keys of any kind uh and you can't approve your own escalation or changing permissions.

05:00 And we expect systems to keep track of all changes. That's pretty standard, right? That's all software today. That's what we expect. Um the question uh if you're being audited in a regulated industry, the question that will be asked is, you know, not is the model smart enough is your agent smart enough or using the latest release, but it'll just be who acted, who allowed it, and then show me the record of what happened.

05:26 So, if we're going to apply this to agents, the first thing I'll say is we don't want to stop agents, right? We don't want to uh create a block that teams are going to find a way to route around. Agents will work around. We know that blocks don't really work. Uh the answer is creating a path to action that is meaningful and uh much easier than the ungoverned one.

05:46 So, we want the governed action to actually be easier than the ungoverned action. Um let me show you with another fun animation what that path end to end could look like for a truly governed action. So, imagine you had a true control layer, you know, a magical system of some kind that governed agents on your machine and in the cloud. Um the first thing that you really care about is that it works just out of the box.

06:15 So, you join the company, you onboard, and you want all of these policies in place. Um so, in this kind of first step of the animation, we're we're showing that it's something that comes out of the box installed on your laptop the first day you start at this company. Um the policy of what should happen or not happen is written once centrally. Uh it's not something a developer needs to write on their agent or their own harness.

06:40 Uh it's written once and now kind of enforced at the at the company level. Uh for this one, let's think of a simple one. We'll go with secrets on user prompts and tool requests. And the action that we want is block or a quarantine. Um we'll take the same scenario again um of kind of what we did earlier, uh you know, from the developer accidentally deleting the database.

07:03 Uh in this case, um the user's putting in AWS key in. Um or it might even be coming from another agent. Before the model ever sees it, a hook fires, and um this happens all in the harness runtime. So, it detects that there's a key that's being sent, um and the prompt is caught by a policy that's evaluated by the control layer. Uh the policy is enforced by a detector, and one of them matches the AWS key.

07:32 So, there's there is a concept of like an interception that happens, um and then as that happens, there's also a decision that's created. And all of that happens in, you know, ideally we want it to be like 100 milliseconds or less. We want We don't want the agent or the human to really change the the path that they're on. Um so, it happens very quickly.

07:54 And then finally, um you know, once the interception happens, that's logged, and now there's an audit of that event that's happened. So, inspected, blocked, and logged, the secret never really left the machine, and in this case the developer lost nothing but just a bad prompt. Um let's simplify this kind of conceptually into what's happened. Um we will look at this as four steps that a company can model a governance architecture around.

08:26 Uh the first I like to call connect. Um this is creating a one creating a door instead of, [snorts] you know, having dozens of different integration points. All of the calls that an agent does when it connects to external systems should go to one place, something like an MCB gateway. A lot of us have used those and understand those well. Um the second is that the calls need to be resolved to a named agent or a human behind it.

08:49 Um policy evaluation doesn't really work without identity. Um this has to be the same identity all of the enterprise software is assigned to as well, so it's consistent across your organization. Um Third, there's a decide step where you either allow, deny, quarantine, or you escalate the decision. Um and then sometimes even just warning a user is sufficient.

09:11 And then finally that decision is recorded um before, you know, any action is actually taken. Um we're going to dive into couple of these in a little bit more detail so I can give you a better blueprint of how to do this uh for your own product or your own company. The first is uh identity. So, who was on the call, who actually executed uh this request?

09:34 Um if we look at this uh kind of screen in front of us, there are two sides to it. The left side is the incident that we posed. That's the log from the incident. A key was found on the file, but no, there's no actor and there's nobody who approved it. Uh so, if you're looking at this as, you know, uh an auditor, you're wondering who actually took the action, who's responsible.

09:58 Um these And to be honest, this is the default state of most systems today. Um it was even ours you know, until earlier this year. Every principal, every agent, every user, um it's all borrowed based on a person's identity. This is pretty common in most companies today running agents. Even a background job is mapped to a user or there's just a anonymous kind of system role that's created.

10:21 Even Slack assistants or tend to be running as like user permissions. They don't necessarily have their own identity. On the right is that same prompt now modeled to the lens of a control layer. The agent is a principal in its own right, so it has its own identity. Below that you see there is an an operator who drove the action. So who is actually interacting with that with that agent in real time.

10:46 In this case it's me. Um and that's recorded for attribution. It's never used to like widen access in real time. And then finally there's the authorizer. So this is the human who approved the original credential setup for the agent. In this case it says Thomas who's our CISO. This is fixed for credentials lifetime. So the right hand card is what we really want for compliance.

11:08 This is a true log of what actually happened and how we could model it. Um the next one is for kind of the set theory nerds in the audience, but it's how you actually decide in real time whether the call succeeds or fails. Um let's the kind of three parts to this. The first is the one on the left. Our credentials. This is the ceiling of what a human put down when they issued the key.

11:35 So this again is the thing that never gets widened. It lives in the credential and it's something you can't escalate past. The second is we call it A. It's the agent's own policy. It can't hold an admin privilege or any or any kind of human only scope. And then finally O is the owner's live policy. So an agent can never do more than the person accountable for it who who's actually driving the agent.

12:00 So in this case it was me. Um what's interesting about modeling it this way is that you know, the second and third A and O are read on every call. If you remove a permission, um, it's gone on the next request. So, it's re-evaluated. Um, and if we go back to that database scenario that we started with, even if the owner could delete production, an agent scoped to just staging can't.

12:22 And if someone gave the agent a broader policy, it can't exceed an owner who never had production rights. So, in kind of both directions, you're protected. Um, either way the call would stop. Uh, and then finally, an important note, you know, if an if an owner would leave a company, um, since it's connected with your enterprise identity, your IDP de-provisions them, and every request that agents make is now denied going forward.

12:51 Uh, the last step in that conceptual flow is is recording what actually happened. Um, so, on the left is again that scenario playing out. Um, on the right is what it actually leaves behind, now that we have a way to kind of model the steps that have happened. Uh, we can see kind of in this flow, as you go through the different steps, there's a denial step that happens, uh, and there's a stated intent, which was that, uh, the agent wanted to act on staging, but the action was done on production.

13:17 Um, the policy sees both sides of that. So, this is where compliance frameworks have generally landed today. Um, every non-read action is evaluated against policy uh, that considers the agent's stated intent, uh, not just the action that actually happened. Um, in most control systems, there's exactly one of kind of five decisions that comes out of this, uh, like we mentioned earlier, it's either allowing, denying, modifying, or escalating up to a human.

13:44 Um, every evaluated action produces this kind of record of what's happened, and then in generally, in a lot of systems, it's cryptographically bound to the agent identity. So, I have only one slide that's going to actually take you through how we built this at Speakeasy. So, I'll keep it limited to that. Um, as I mentioned earlier, we we Speakeasy were an AI control plane.

14:07 Uh, we actually put our product in between agents uh, that people use or using autonomously, and the systems that they have access to. Um so the identity comes the identity baseline comes from your IDP. Uh so Okta or another identity provider. Uh and then the policies of action is actually evaluated on our product or our control plane. So every call um includes the ones including the ones that we deny all go through us.

14:32 And so we write out the audit log for for our customers. There's kind of three kind of critical decisions in this architecture. Uh the first as you can see is that we sit in the critical path not beside it. Uh the record has to be of what actually happened. Uh not what the agent reported and two we use kind of first class identity uh instead of using user logins everywhere to the system.

14:57 And then finally we'd actually distribute to MDM. Uh so all of this comes working out of the box. For a user they don't have to go redefine these policies on their end. It's something that just works the moment they start uh at the company or start using an agent. So that's my only slide about Speakeasy but I'm happy to talk to you more about that later.

15:18 Um So that's the to kind of summarize what we went through. Uh in a regulated industry permission to act usually means kind of three things have to be shown for every agent or every agent action that is taken. Um who the actor was, what policies allowed it uh and the signing of the record. So knowing for a fact that this actually happened. Um this is what we've always kind of asked employees to do at a business and it's what we ask agents to do now.

15:44 Um valid credentials are not valid intent. That's something that's really important to remember and the fix is not you know just saying we have a smarter model or a better agent. Uh it's actually putting together identity policy and record of every action that's that's happening. So that's uh kind of everything I wanted to talk to you about today. Um I'll also leave you with um we're doing an event tonight with our friends here at Tailscale um launching a partnership around private put taking your MCP gateway and

16:16 putting it on a private network. Join us if if you're here. And yeah, happy to take questions. >> [applause] >> Go ahead, camera. >> I had one question on say you're using an agent that is being governed by the ethics of the planet we're talking about and disagree with the decision that it came to to deny that that action. What would you How would you design a policy?

16:52 >> Yeah, that's a that's a good question. I think um the you know, if probably the first most important thing is that you do have a way to see what decisions were taken. So, in most companies today when people are using you know, just let's just say Claude from Anthropic and connected to all the systems, as it's accessing these systems, there's really no record of those decisions taken.

17:14 And so, even if you disagree that hey, this user shouldn't have been able to access, let's say your CRM, you can't go back and actually say that has happened for sure. Secondly, you do want a way to escalate your user when that happens so you can actually decide if you want that to be passed or not. So, those I would say they're kind of more fundamental missing pieces.

17:36 If you have that in place, you can always go update a policy to actually allow that to happen. Yeah, go ahead. >> Out of the 56% without policies, I'm curious who's buying because regulators force them versus something broke. Like, what moves the budget? >> Yeah, that's a good question. Something we think about a lot. I would say compliance is a really good driver for people to think about getting a solution in place, but I think the the reason what I want to leave you with is that it you don't necessarily you know,

18:14 buying a product is to fulfill compliance, but the first thing you need to do is kind of understand what is actually happening in your business first. So, I would say compliance is a great driver of budget. When there is an incident that that often, you know, raises the priority of something like this. Go ahead. >> Do you mask PII or like the logs that come through your control plane?

18:40 >> We do. We do everything from kind of blocking, masking to redacting PII. Um, what we've found in most businesses when you turn on observability and you look at everything going through the agents, there's a ton of PII going through. So, if you are going to kind of block on PII, you'll probably grind your business to a halt, right? It's just it's hard for people to not use the agents with PII.

19:04 I think the more important thing to look at is what is happening to that PII. So, like what's the intent for the agent? Is it going to post it in a public Slack channel, send it by email, put it back into a tool called to another system? Those are the kind of policies we try to recommend people to, you know, use observability to define rather than just kind of blocking or redacting all PII together altogether.

19:29 >> I guess I'm asking that question because it's like if there is an incident, I'd ask the control plane for like our logs and to understand what happened, right? With authorization? >> Yes. >> You would have to like give those logs to me, right? But, why should I give you like the data [snorts] in the first place? Like because it's proprietary and it's our secret sauce.

19:50 >> Yeah. >> So, like what's the trust model with that? >> Yeah, I mean, that's kind of it's getting a little into the details of how we deploy, but like we do deploy our software in a few different ways. Like we do offer kind of a data plane option where it runs in your own VPC, so data stays with you. But, the more common option that customers do is they bring their own encryption keys, right?

20:13 So, we do encrypt the data, but you bring your own keys for so you can govern it yourself. And yeah, customers often will go into kind of our product and look at what is all the PII at that happened at least look at from the perspective of like all the findings even if they're not looking at the redacted data itself.