AI generally cannot be smart, fast, and secure at the same time without trade-offs: increasing capability expands attack surface, increasing speed reduces inspection time, and stronger security introduces friction. An AI security proxy can help balance all three.
Searchable transcript of What Is the AI Security Trilemma? Smart, Fast, or Secure AI? — IBM Technology (09:03). Search for a phrase, then click its timestamp to jump straight to that moment in the video.
Captions sourced from the original video on YouTube, published by IBM Technology. The video, its captions and all related intellectual property remain the property of their respective owners; AINotes claims no ownership. Provided for research, accessibility and search — see the Transcript Notice and Copyright Policy.
00:00 Have you ever played musical chairs? You know the children's game where the kids walk around a row of chairs and when the music stops, they rush to grab a seat? The problem is there's one less chair than there are children. So someone always gets left out. Well, that's basically what enterprises are doing when they try to have AI that is smart, speedy and secure.
00:23 There seem to be only two chairs and yet we want all three players to win. Welcome to the AI security trilemma. A dilemma, but with three parts instead of two, where it seems that no matter how hard we try, you can only pick two and end up sacrificing the third. Let's trisect the conundrum to understand the problem better. First, let's start with the smarts.
00:46 This is the capability of the model, its ability to do reasoning, to do multi-step planning and think its way through a process, have a greater context so that it has greater understanding, to do problem solving. To have tools that it can use and to express creativity. Generally speaking, the smarter models though also require other capabilities. For instance, they might need a larger context window, they want more autonomy, they want more tools and more access to those tools and greater permission levels.
01:22 And that's where things start to get dangerous because the smarter an AI agent becomes, the more attack surface it creates. Speed is the thing everyone wants from AI. I'm an impatient person, so me especially. What we want is instant response time, low latency. We want real time agents that are operating without someone involved, which means autonomous actions.
01:47 But speed has an important implication, less time for inspection. If every AI interaction has to complete a milliseconds, it becomes very hard to do things like scanning. The inputs that are coming into the system, detecting prompt injections, analyzing the outputs that are going out from the system and then ultimately trying to enforce security policies.
02:11 So the faster the system gets, the harder it becomes to intervene safely. And finally, security. So this is gonna include things like prompt injection defenses so that someone can't take over the context of the system and control it. We're gonna validate the outputs that come from the systems. We're gonna look for data loss prevention so that we're not leaking information from our system to the bad guys.
02:36 We're going to do policy enforcement and enforce access controls on the tools that we are using to do all of this. But security almost always requires some other capabilities. That is things like inspection, filtering, guard rails that are gonna limit what the system can do, sandboxing, another form of limitations. Logging so that we have observability, and ultimately a human in the loop so that we have some sort of human oversight that is determining if the system is going awry.
03:09 And all of those things ultimately introduce friction. And friction ultimately slows the system down. The reason this trilemma exists is something security professionals understand very well. Capability increases attack surface. When an AI system gains ability, each one of those abilities become a new pathway for attack. So for instance, if we start with a model and then that model needs certain reasoning capabilities and other things that it's going to access, we wanna give it a memory so that it can understand
03:39 context so that if I ask it another question, I don't have to repeat everything and start from the beginning, it has that in its memory. We also maybe wanna give internet access. Because that way the system can go and get fresh information, sometimes almost near real-time information. And the model didn't have that when it was initially trained. We give it access to tools, we give it access to databases, we give it access to APIs so that it can call out to other systems and get services, get new information, make
04:17 changes to other system's databases and things like that. Another set of inputs that might happen for this system would be sensors. So it's gonna get input from the real world. Maybe it's looking at certain things or hearing certain things, and that become another way that it can take input into the system. And then the way it interacts with those things out in the real-world would be through actuators.
04:44 So a whole bunch of capabilities. The more powerful the system becomes, the harder it becomes to secure. And if we try to secure everything perfectly, the system slows down dramatically. So we've got smart, speedy, secure, pick any two, you don't get all three. Well, let's see what happens if I optimize on two of these. So let's say I start with smart and secure.
05:08 Well, then that means I'm gonna end up with a slow AI and nobody wants a slow, AI you would think, but actually there are some cases where speed is not the most important factor that we would optimize on. Let's say we're wanting the system to do some sort of deep level of research into a particular topic. It's not so time sensitive. If it wants to take 20 minutes instead of 20 seconds, as long as we get the deep level of research that we're looking for, that might be okay.
05:36 Maybe medical diagnostics. When it's looking at the x-rays and doing its analysis, we want it to really get it right, not so much get it fast. So there are use cases where that is perfectly okay. How about smart and speedy? Well, then we're leaving security out, which means we end up with dangerous AI. Now, dangerous doesn't sound like a good thing to have.
06:00 Why do we want all this risk? Well, it turns out that there could be, again, some use cases where the risk is low enough that we can tolerate that. And we really do want the smarts and the speed. So in this case, for instance, if all I'm dealing with is public information that everyone would have access to anyway, well, then I don't need all of the security maybe.
06:20 Or if the information it has, or the processing that it's doing, the output that we're gonna take and the decisions we're going to make and the actions we'll take based upon that end up being low risk, well then it doesn't matter if the AI is a little bit off. An example of this might be a proof of concept where we're trying to do some level of testing out a system to see what it might be possible, but this is not the production level system.
06:43 But we just don't want it to take forever to end, to do the job. And then ultimately the other thing that we could pick two of was we could we could picked secure and speedy. Well, then what we end up with is a dumb AI. Well, why would you want a dumb AI? We want it to be intelligent, right? So if we leave the smarts out, you would think that wouldn't be useful, but it turns out there are some cases where you don't need the smartest AI in the room.
07:10 And for instance, if it's a simple set of operations, simple questions that we're gonna be asking of it. Then we don't necessarily need a big brain in the sky. Maybe an example of this might be a home automation AI where all we're basically asking it to do is turn the lights on and off. So I do want it to be quick and I don't want everyone in the world to be able to access my lights in my home.
07:34 So security and speed would be the things that I would optimize. So you can see we've got three different things that we're trying to optimize on, but we can pretty much pick any two. So how do we deal with this AI security trilemma? One approach that's gaining traction is introducing an AI security proxy. This is a policy enforcement layer that's added outside of the AI itself.
07:58 Instead of trusting the model directly to have to do all of these things, we enforce security policy here. So this layer can inspect the inputs, look for prompt injections and things like that. It can sanitize the outputs if we see that we're leaking sensitive information. It can control tool permissions if the AI wants to go off and use those things and basically apply security policy across the whole system and do it in a more uniform way.
08:26 So the model itself can remain smart and fast while the stuff around it is secured because we're adding security as an extra layer. It doesn't eliminate the trilemma. But it does help balance it. We still have a game of musical chairs, but if we do it right, we can sneak in an extra chair so that everyone has a place to land. Right in the sweet spot.