Why Are We Still Governing AI Agents Like Software? - Tim Tyler, Gravitee Transcript, AI Summary & Key Points

00:01 [music] Uh before we get started, uh my name is Tim and I'm with Gravity AI manage uh uh agent management. Uh before we get started, I'm sorry, somebody back at the office asked me to put this up real quick, so bear with me. Uh, we're hiring right now. So, I'm going to put up a job description and if you know anybody, it's in marketing operations. Um, basically no ID required.

00:32 You can just come in, no docs. Uh, there's no manager or supervisor for this position. It's got a really obscure job description. You have access to key systems, but don't worry, they're all shared passwords. Uh, and you will be expected to work 247, but nobody's checking on you, and we won't fire you if you get anything wrong. I see some smiles and chuckles.

00:59 Everybody raise your hand if you would hire somebody by this job description. See, nobody raises their hand, but we're already hiring this new digital workforce with AI agents. And if you aren't, then others are, and they're probably your competition. So today I'm going to talk about why we're still governing AI like software where we look at it in as a in a different perspective of looking at it as a digital workforce.

01:25 Of course, you need all of the software aspects. You need all of the protocols, the standards, so forth. But they're not just necessary. We need something more. We need to look at it in a different way. I had somebody meet me at a show and said, "I'm I'm I'm head of AI at my organization and I sit between I go and talk to uh my CTO and they say, well, it's not really a a software application."

01:51 And he said, "Okay, so I went to HR and they said, "Well, that's a software application." [gasps] So, we've done this before. This is how we started to look at this. Over the last century, we have been as enterprises looking at how we govern the employee workforce, the humans. So we said, well, what if we just take that and apply that onto AI agents and look at them as a digital workforce.

02:16 So we call this agent accountability. And there's a framework behind this. And I'm going to talk to you about the principles at the top of the framework, but there's controls for actually auditing to this framework as well. So we break it down into three stages. We start with the assignment stage. This is what's uh essential to establishing the agents presence in the organization.

02:34 So we start with who are you? So if when you start with a company first they got to know who you are. You got to get it into the registry into their system and usually if it's like a secure organization like many of you are in finance you have a secure organization to get into the building you have to start with a ID badge. And if your ID badge, like mine over at Verizon, if your uh when I started with them and got to get in the building, if your ID badge was like mine, it competed with your DMV photo for the worst

03:07 picture of your life. So, we need to give the agent an identity. Who are you? That identity needs to be one unique ID. Needs to be in a known shared registry. That's one single point of truth. And that identity is what everything else in this framework is going to point to. Now, the agent doesn't just work alone. The agent is part of a team. It's a team.

03:32 Even if you're using an agent just between you and you're utilizing an agent, it is a team of two. And if it's part of a team for a specific role or task or that uh is a team for um a larger team that's a department or just the fact that this agent is in your organization is part of that team. So it needs to get context on the goal, the decisions, the history of that team and it should be giving context back into that team so the humans and other agents can know what it's doing and what what what is its learnings and

04:12 that means it's a teammate and not a digital clone. A lot of people think, oh, I'm just spinning up something here with Clyde and it's doing things on my behalf, therefore it is me. That's dangerous. So the next step is we need to look at you've got your identity but you need to look at who's the my boss right who is responsible and accountable for what I do in the organization and who's going to help me to understand what I'm supposed to do.

04:36 So we need to have the agent needs a sponsor who answers for you. So our agent 409 that we've already given that identity to is now going to have an accountable party that is a named person. Sven is going to be the accountable party. Sven defines what agent 409's purpose and so forth. We'll get into that. He owns what 409 costs and what it spends. And he gets the call if anything goes wrong.

05:06 So there's one name that starts a line of accountability. That line of accountability we already have in our our hierarchy of our org chart, but the it starts with somebody where we enter that. The way I like to think of a line of accountability or even the sponsor role is when my kids were young, we had a pool and we had a rule that there was always somebody that was accountable for those kids in the pool so that we'd know if they were drowning, right?

05:32 And so we had a rule that if I I was responsible for the kids that if I needed to go to the bathroom or something that I would look at some other adult that I trusted to be with the kids, I would look them in the eye, make eye contact, and say, "Do you have the kids?" And they would reply, "I have the kids." We did that for years. And that's essentially Sven is saying here, "I have the agent."

05:58 So now we've got our assignment. we know who the and who is the agent and who who's responsible for it. So now we need to align to what is this agent scope? What is it supposed to do and how is it supposed to do it? So basically what are you supposed to do in this job? So we've all had that job where you started and there was really no job description or a really obscure job description and you just figured it out and kind of checked in with people and said is this what I'm supposed to be doing?

06:20 Um that doesn't work for agents. They they are better at give them what are you here to do a role. So that job description is a purpose, responsibilities of how to accomplish that purpose and some kind of budget. These are just high level. Of course, a role goes way deeper than this and that's what the controls on the framework help to define. But just to give an idea what is the role, these are the types of things we're putting into it.

06:46 And beyond what it's for, how to do the job, it's also what success looks like. And we'll talk about that in a minute. So there's two types of definition for the rule. We've got onboarding versus runtime scope. So a lot of what I just mentioned is that onboarding, what's the purpose and so forth. It's set once. It changes rarely unless we've got to take some recourse versus mission.

07:11 And this is a word that I'm pulling from some of those tech specs. Mission is like in the new AOTH. It's what's happening at this very moment at every time it's making a call for instance to an MCP tool. So the the agent actually defines its mission, but Sven always defines its role. So we've got we've got a role what we're supposed to do. But now we need a way to do it.

07:37 So we need to know what access and authorization we can have to to what data to what tools to get the job done. So we're going to give the agent authority. What can you see and what can you do? So sponsor the sponsors Vin decides which doors he's the one that gives the keys. Now it might be through an IT department or a particular team that runs an MCP server database or whatnot but Sven is the one that has to open it up for the agent because he's accountable and this is where we start to set those boundaries and

08:11 escalations are defined here on what how what happens with that authority. So let me take a minute and talk about that authority in scope for a second. We always work to the lowest scope. What I mean by that instead of going through specific scenario that could get confusing. I'm just going to call the access authority for Sven. He has access to A, B, C, and D.

08:33 That could be information he's allowed to see. That could be a tool he's allowed to access. But he has that scope. Agent 203. Notice because we have identities, I can call out this is not agent 409. This is agent 203. Agent 203's jobs is to converse with with Sven uh about projects. And agent 203 has a scope of C and D. And then agent 409 has DE and F.

08:55 So to get so when agent 203 calls on behalf of Sven delegating a task, agent 203 calls to 409. 409 can actually go to DE and F to get the job done. But it can only return D back up the chain here to Sven because only D, Sven, 203, and 209 all have access to. So the the scope of authority never grows beyond what agent 409 has been given. It always is constricted by what is that that lineage.

09:34 Never assume alignment meaning we have inherited uh uh alignment here with the model. For instance, if you are using a large language model built into that has its own alignments, right? Its own rules and its own uh definitions. As we know the hyperscalers, the frontier models have just said they're misaligned. We're going to tell everybody they're misaligned.

09:57 So we know that we don't have control over that and that's locked in from our perspective. Also that team context I was talking about, it's coming from somewhere else. So we don't have Sven doesn't have the capability to go change that very easily. And then there might be outside agents that are accessing agent 409 from other parties. As we grow this whole agentic era, we're going to see this more and more.

10:18 An outside agent kind of needs to be more like a contractor coming to work in your building. You give it an identity. You give it authorization when it's able to access and so forth. But from the perspective of 409, we need to think that that that uh alignment that context coming in is not something Sven controls opposed to Sven can control what is defined for 409.

10:42 And so we he sets up a set of rules that are deterministic. That's the floor. That's kind of like FGA policies and so forth where you can define specific things. But then you have intents that are probabilistic. So he intends for something to get done and how do we measure that? Oh, that's nice. And then uh so the inherited alignment, it can change on us without our control even though we can't change it.

11:08 And the defined alignment becomes a contract for 409. So how do we actually enforce that contract? Well, that's with the administration stage here. So with the administration is where we start to enforce everything before. So if as an employee we think we have trust but also our enterprises and organizations make sure that they verify that trust. So they have things like uh you know human OS and so the roles that are making sure that our devices have a mobile device management on it that they're checking what we're

11:41 doing. Then uh we have people checking in with us. We have supervisors that are watching that we're on align to what we're working on. So we need to look at that for agents as well. But one thing I want to call out here is that behaving, that's what we talked with agents. We say like are they out of scope? Are they accessing things they're not supposed to?

12:00 That does not always uh equal succeeding. So we need to think of behaving we defined in authority and succeeding we defined in the role. Are you actually succeeding at what we want you to do? So the first thing we need to do in administration is we need to know if you're doing what you're supposed to do and you're staying within those those guardrails of the authority.

12:21 So we set oversight. Are you doing what we expect? Now that's not a single metric. It's actually a scorecard. So on the authority side, we have those more deterministic rules like are you staying with inside the scoped authority? are you uh you're only appro approved for uh up to $40,000 invoices to be dealing with something like that very clear rule and defined but then on the rule we tend to lean into things that are more that those uh intense.

12:53 So is your invoice match accuracy up to the par we want? Is your reconciliation time the cycle time uh within the scope? Yes, you can put rules to these, but more and more as we start to define these, we start to look at them and say we we need to look at them as a scorecard. So, and the weights will come from the definitions in rule and authority. So, again, rules are checked, intents are judged.

13:17 So, rules like I said, these types of never approves a payment, rights only to the ledger versus the intents are probabilistic and it's like the correct PO was matched, a proper vendor uh project match. And an intent of evaluation should be based on a confidence level. So if we look at that, well, who's actually looking at that? The agents need micromanagers.

13:38 We as people, sorry, do not like micromanagers, right? We don't want that manager looking over our shoulder, but agents need that. And so we have a micromanager looking at what's going on with the model. We have a manager looking at what's going on with the context coming into that agent. And then we also look at what's the work product coming out. And so there's all sorts of things we should be looking at in the oversight constantly.

14:03 And that's why I have these as little guardian agents around here that are actually applying that at machine speed. Because if you wait for a quarterly review like we do with humans, you are about 40,000 400,000 4 million decisions too late to have that discussion. So what happens when things start to go ary? We need to take action. Well, if you think about it, if we do something wrong, the first step isn't you're fired.

14:30 Hopefully, it's not. So, you you're going to you're going to you know, your supervisor is going to sit down and say, "Hey, you know what? You misunderstood that. This is what we want. A correction, but if you keep doing it, especially if you're out of your authority, we're going to start restricting that authority." And then if you keep doing it, we're going to escalate and then you're out of here.

14:50 So, we also need recourse. The same kind of model for agents. what happens if you don't do what we expect. So we look at it as a deterministic grade uh when we're talking about like intense uh for the probabilistic judgment. We look at like if if we're pretty confident then we can take a certain action. If we're low confidence we can take another. That allows us in the definitions to be to set these steps and the steps allow action.

15:19 So, for instance, if I'm looking at the intense and I've got a confidence score going, I can look at like if I'm 90% pretty confident, then I might just redact content. If it if it's out of out of alignment, if it's less than 50%, I need to escalate to a human in the loop. If it's below 20%, block it. We're not we're not confident at all. So by putting in those steps, those stops, we can start to turn that probabilistic into something that can have deterministic uh actions that can be uh acted on in recourse.

15:51 But that's opposed to something like you accessed something you're not supposed to access and uh so we restrict authorization at the moment or suspend access if you've done it five times today or suspend the whole operation of the agent if it's been doing this consistently for days. So that we when we look at a zero trust, we start heavier at first, then we lighten with repeated success.

16:16 So this is not something you just kind of set and then forget right away. It's something that you want to be looking at and evolving. And again, at the machine speed is where you can do this practically with agents. So we have the police, the rule, we need a cop. That's where we've got something like guardian agents, AI gateways, guardrails, where they're doing restriction, prohibit, and suspension at machine speed.

16:40 And then Sven's job, what we're good as at as humans is we come in with our judgment to be able to say, you know what, if I tweak this role, if I tweak this authority definition in that alignment stage, then I can get a better result, I get more success out of the agent or I get it to behave better. So Sven adjusts those roles and authority as a coach.

17:01 If you think about it that way. So we have assignment. We need to name every agent. We have alignment. We need to bound every action. And we have administration. We need to trace every outcome. That's agent accountability. That's how we reach agent accountability. This was built from the history of humans tailored for agents. Like I said, there's a a framework behind this with controls that I welcome you to take a look at that uh you can audit in your organization and then you can apply those technical solutions that I

17:34 talked about as the methodology to achieve. So you can get started today by giving your agents an identity. And one thing I ask when I give this talk is if everybody can do me a favor, give it the worst picture of his life. Thank you very much. I'm Tim Tyler. uh and and uh feel free to go to agentacountability framework.org or scan the QR code if you want my landing page and you can check out the uh the agent accountability page or uh gravity if you want more information. We're also downstairs. Thank you. [applause]