How to build and scale multi-agent AI systems on GKE Transcript, AI Summary & Key Points

00:19 We are looking at a 5,000 share order. Hello, everyone. Welcome back to the show. My name is Olivier. I'm a developer advocate at Google. And today, I'm joined by two of my fellow colleagues, Lucia and Sophie. Hi there. I'm Lucia. I'm really glad to be here. I'm happy to be doing this today. I'm a developer advocate. As Olivia said, I specialize in data specifically, and we'll be looking at labs related to data as well.

01:15 Hey, everyone. My name is Mofi. I'm also a developer advocate at Google. My focus area is Kubernetes and running AI/ML workloads on Kubernetes. And today we'll be looking at some stuff to run on Kubernetes. Great for those who don't this is completely live. If you have any questions, drop them in the chat. We'll see them. We'll try to answer as many as we can as we are running through our content.

01:44 So let's get started. I have a couple slides that I'm going to present first. So what we're doing today is we're going to be presenting the clouds engineers AI toolkit. This is some contents that we brought in person over the course of two months, the month of May and June. But of course, we weren't able to get to every single city that exists out there.

02:10 And so we thought, why not also bring that content online and make sure that as many people as possible have access to it. So the idea behind this content, the premise is let's look at agents and GCC in various different use cases. So we have three labs. The first lab is going to be looking at how can you use AI agents to help troubleshoot infrastructure, deploy new infrastructure instead of doing things more manually.

02:42 The second lab that we have is OK, you have some Kubernetes infrastructure, maybe deploying GKE. You want to run some AI agents on there, and you want to run AI agents that is able to generate some new codes. This is maybe untrusted code. How can you make sure that the code that you're running is running securely and is not going to. Make some damage to your environment or to your infrastructure.

03:12 And then the third lab is going to be about data. So you have a lot of personal data and you want to be able to organize it, analyze it, and build a knowledge graph out of it. We're going to be doing a little bit of a shorter version of each of the labs. The reason why is because when we presented these, they were all day events. And this is a 90 minute lecture.

03:43 So it's going to be very quick. But I want to make sure that everyone has access to the material in case they want to go at their own pace at home, whether it's during the live stream after the live stream tomorrow in a week, whatever it is. So I have the full slide deck unabridged of the three labs as well as the introduction. The link will be dropped in the chat if you don't have time.

04:11 Of course, as a live stream, this is also going to be available as a video on demand, and I have links to the three labs. So each of these labs are going to be accessible anytime you'd like. There's no time limit. Do them at home whenever you have the time. Again, these links are going to be dropped in the chat. OK, so as I was saying three labs we have about 90 minutes, maybe a little bit less.

04:42 So I'm going to be taking over the first lab. And then once I'm done, I'll pass it over to Mofi to do the second lab and then Lucia to do the third lap. So my lab is about troubleshooting geeky infrastructure using AI agents. So the premise is manual troubleshooting is tedious. You have to inspect manifest for errors. You have to try to parse error messages.

05:19 You have to diagnose why failures are happening. All sorts of error messages, Warning logs, maybe alerting. In terms of logs, you might have multiple services. That all has their own selection of sometimes thousands or hundreds of thousands of logs being generated every hour. So how can we make this more efficient, faster, and easier and less tedious.

05:55 A couple of years ago, general web AI assistants started taking over. So there's been a lot of AI tools, like for example, Gemini web, that allows you to ask all sorts of questions and get answers. That's great. But the first problem that you encounter with these tools is that they lack context. You have to be very, very specific with these tools because they don't know what project you're working on.

06:24 They don't know what error messages you're looking at. They don't know what logging you do. So there's a lot of tedious copy pasting, where you open a web AI assistant and you ask it, I'm seeing this problem, and then it'll reply back, well, can you send me the logs. Can you send me the manifest. Can you send me exactly what you see. And it takes a while to get an answer out of it.

06:49 So this brings me to an idea that I had is this AI context ladder, where you start at the bottom of the ladder, where we're using AI tools that has 0 context, and then we're trying to move up where we have more and more and more context, and then we get these AI assistants that are more helpful to you. So this can be something like leveraging MCP servers, leveraging skills or agent plugins to have more of that context.

07:25 Taking a quick step back on AI agent, do the very quick introduction. What is an AI agent. An agent, very simplistic model is like an agent is a system that uses a model or multiple to reason about and decide what tools to use to get to a specific goal. It sort of looks like this. So agent it has some models. It has some tools. It tries to achieve a goal.

07:58 Anti-gravity CLI is an AI agent. So this is something that we've released at. Cloud next this year. And it's an AI agent that lives in a terminal has access to. Multiple models like Gemini 3.7 Flash and has access to a variety of tools, weather, local tools, or hosted MCP servers, for example. So that's what we're going to be looking at today. So as I was saying earlier, I have this entire material in the form of a lab.

08:39 I'm going to be running through it fairly quickly. I don't expect anyone to really follow along because I'm probably going a little bit too fast for that purpose. But the lab is accessible at home if you'd like to try it out, that's going to be available for the indefinite future. So let's start with, step 0 of our AI context ladder instead of low context.

09:15 Let's say we don't even have AI tools. What does that look like. So going to be sharing my Google Cloud Console here today I'll be using Google Cloud Shell. So this is a small VM that lives in Google Cloud. It has access to a variety of different tools. And you're able to operate in this Unix environment without having to step out of the Google Cloud UI.

09:54 So let's just go to my lab. All right. And I'm in similar thing OK. So I've already gone ahead and I've deployed an application to Kubernetes. So I have a GKE cluster. I have this application called simplebank. And I decided OK let's try deploying this. So I deploy it and I get a couple errors. I look at my pods and I'm seeing some pods running. But I'm seeing in create container config error I'm seeing in image pull back.

10:33 There's a lot of things happening. So without at this point, what you would need is you would need a deep understanding of Kubernetes, you would need to know exactly what to look for. You need to know what the various errors mean, and you need to know a variety of commands to look at what is happening. So for example, if this image pullback, we can look at the name of this and try to maybe describe and seeing what is happening.

11:10 And maybe you're seeing some events and maybe you're seeing some failures of some sort. Similarly, if we look back maybe that is running, but maybe there's some errors. And so you might still have to look at, let's say, logs for the. And again, there's a lot of things happening. I was almost about to control-c because that was a lot. But you see there's some errors.

11:40 Unexpected error. Code 403. So there's a lot of context switching because you have to run different commands. You have to take different pieces of information into context between logs and events and all sorts of other pieces. OK, let's move on from there. AI web assistance came out cool. Let's try this. So I'm in Gemini. So this is Gemini web. And I'm going to try solving one of my issue.

12:17 I'm deploying an application on Kubernetes. And one of my pods is failing with image pullback. How do I fix it. Now the Gemini web assistant only has as much context as you're giving it. So I have a problem and I'm asking it a solution for it. But the reality is that there are multiple potential solution to my problem, because an image pullback can be caused by many different things.

13:00 It can be a typo, it can be a permission issue. It could be just a 404. Maybe the image hasn't been pushed to let's say Artifact Registry. It could be a variety of reasons. And so the answer that you'll get from a web assistant is a little bit vague. It'll say, well, it could be one of these five things. And you have to keep prodding. You have to keep looking at more logs or more manifests to get somewhere.

13:30 So what's the alternative. Well, what if we had an AI agents that had the context. This AI Agent happened to be anti-gravity CLI. So still in my terminal in Cloud Shell I will type AGI. That's the shorthand for anti-gravity. And I'll try from there. So I have this prompt that pops up. It looks a lot like the web system. Should it not just give me a similar answer, but there's something a little bit different between the web assistant and this one with anti-gravity CLI that lives in your terminal.

14:15 In addition to the context that you're providing the prompt, it also has access to local context. So it has access to the directories and the files local to where you ran this command. So for example, if I have something vague what is this project. I'm not telling it that I'm looking at symbol bank. I'm just very generic. What is this project. But anti-gravity CLI has local context.

14:48 And so what it's going to do and you're seeing it here is that it's going to list directories. It's going to read some files. So I'm seeing it's reading a readme file. And let's see. No on that one. And it's going to get an answer that is a little bit more specific. There we go. So I'm just going to wait for it to generate what it wants to there. So I ask it what is this project and saying.

15:23 This project is formally known as Bank of enthuse, a sample Cloud native microservices based web application for demo purposes. It's a banking simulation. It simulates a retail bank payment. Blah blah blah. Where is it getting this information. Well, it's getting at because locally I have a readme file, and a readme file has a lot of this information.

15:47 So great. We have access to contacts. Maybe we can use that to help us troubleshoot our issues. So I'm going to say I have. A failing. I already forgot where I love live demos. I already forgot which of my is failing. I'm going to go look at that. So let's look at our pods. Oh yes, the user service. So I have a feeling user service. My user service.

16:26 Is failing with image pullback. Can you tell me why. Let's go from here and see what happens. So anti-gravity CLI in addition to local context is also able to use commands terminal commands. When it does, it will ask you for permission. It will. It's one of the more frequently asked questions, which is, well these AI agents, I don't want them to just run all sorts of command and delete all my files or run away from my data or whatever other concern.

17:10 So the answer to that is, every single time that a command is about to be run before it's run, you will get a consent prompt. So in this case here anti-gravity CLI wants to run kubectl get pods. It will ask me would you like me to proceed once or always or not at all. So in our case, I don't mind it. Doing a kubectl get pods. It's a read only operation.

17:38 So go ahead. Again describe that's also read only. I'm very happy with that. So I'm going to let it. And then it's getting events. I'm going to let it proceed a little bit the main thing that I want to. Really pay attention to here is a lot of what anti-gravity CLI is doing. I have the ability to do that too, right. I have the ability to look at events and look at logs and look at which pods are failing and all these sort of things.

18:16 I can do that, but I can't do it nearly as fast or with nearly as much good memory as an AI agent. And I think that's a lot of the superpower. It's not necessarily that these AI agents are running commands that you don't have access to run. But it's able to do it at a dramatically faster speed than you are. So this helps a lot in the context of troubleshooting where you have these issues.

18:50 And you would like to look at multiple pieces of logs, multiple pieces of information. There's a lot of context switching. Why don't we delegate some of that instead of having to troubleshoot manually. Maybe we can troubleshoot faster. And there you go. So what did that take in total maybe 10, 15 seconds. Me hitting. Yeah you can run this read only command a couple times.

19:15 And not only has it found my issue, which was a typo, there was an extra in the image name. Not only did they find the issue, it knows exactly how to fix it. It's telling me how to fix it. OK, cool. I could go ahead. I could open that file up and I could make the change myself. But I'm going to try something better. Can you fix it for me. And this is a continuation of what I was saying.

19:52 I could totally go ahead and open the file. It was pretty explicit which file to open and what to fix, but just by asking it to fix it, it's going to take, five tens and the fix is going to be there. Not only is the fix there, it also just ran a kubectl apply. So it's propagating the fix for me and very likely yeah there you go. So very likely is going to go and check on itself and make sure that the fix has been propagated and that the pot is now running and that there's no more issues.

20:31 So right now it's looking at the pot description. It's going to look for events and then it's deleting the old. Cool So that is awesome. So we had an issue. We asked anti-gravity CLI. What's my issue. We got an answer and then we asked to fix it. And it's already in the process of fixing it and getting all of the changes there. But we can go further than that.

21:07 So I'm just going to wait for this to go up. And in fact, just so I. Go with my word of being quick, I'm just going to Control C and talk a little bit more about how we can help our troubleshooting even more. So I'm going to quit anti-gravity CLI. It was in the middle of just verifying fixes, but for the sake of time, I'm going to keep going a little bit.

21:41 Another superpower that we can give to these AI agents is giving it hosted MCP servers. So for those who don't an MCP is the name of a protocol, the model context protocol, that allows you to use various tools that are not local to your machine, but rather are hosted somewhere. So various organization companies, open source software, are providing these hosted tools and hosted capabilities, and Google Cloud provides some of those.

22:18 So Google Cloud has MCP servers. So collections of tools for various of our products, including GCC. So let's go ahead. What I'm doing here is I'm just modifying my MCP configuration to add the GCC or container MCP server to anti-gravity here. And then I'll reopen anti-gravity CLI. From there, I can do MCP list. And I see that I have a MCP server called GCC.

23:05 It has a variety of tools. OK Why is that useful. This is useful because now in addition to running local commands, we can also have access to a lot more information that is relevant to our situation or to our goals. So for example. We can ask a question like can you list all of the GKE clusters in my project in, I have AI in action project, and I can ask this question where I may not have a local command that is able to explicitly get an answer to that.

23:52 Although this is funny because I do actually I do have gcloud gcloud install on there. Let's say I'm not going to allow that, and I'll be a little bit more specific. And I'll say. Prioritize the MCP tools because I want something accurate that is up to date. So it's going to be looking at the MCP tools that I have. It's finding that, oh yeah, I do have a GCC server on there.

24:30 And then. And then just to make me happy during a live demo, it's taking is little time to think. But that's just the beauty of it. Oh, interesting. I decided to do it as a subagent, so that wasn't part of the material. But anti-gravity CLI does have the ability to create and manage subagents. I'm not sure. I'm actually going to reboot this one. I feel like it strayed a little too far.

25:28 So of course, AI agents or AI models are non-deterministic, and sometimes you have to steer them the direction that you want them. So in my case, just for this demo, I will be extra explicit using the. MCP server. Get a list of my clusters in project. Yeah I see that. I see that. Gemini app decided that it did not like me today. That's all right. I wonder if the MCP should be.

26:32 Yeah, it should be there. Interesting well, that's all right. It was one side, but I have multiple sides, and I have about five minutes left to my presentation. So I'll do a last one. I'll move on from there and I'll talk about scales very quickly. So agent scales is another ability that AI agent has. The idea behind it is that for a particular given problem, we can be.

27:07 Giving a little bit more context on how we want that problem to be solved. So traditionally when AI agents first started out. The way you would do that is by modifying a agents file, giving it all of the context of how to solve problems and run with that. But the problems that quickly happens is that your agents M.D. file starts being very long and it starts talking about multiple different problems.

27:35 So what if there was a way where we could describe how to solve problems. But where that context of how to solve problems would not be ingested by the model until it's actually needed. So until you actually have this problem. So this is solved by something called agent skills. So I have a prompt here I want to create a new skill. We'll call it the Kubernetes troubleshooter.

28:04 And we want it to help diagnose issues with Kubernetes manifest and cluster state. So we wanted to be able to analyze logs and events and resources to identify problems in configuration errors. So let's try that. Agent skills have the particular format. So they live in a directory. They start with a filmed file and then they can have additional resources and scripts surrounding them.

28:41 So when I'm asking anti-gravity CLI hey creamy, a new skill for this particular problem that I'm having. What is going to be doing is going to be formatting a directory with relevant context files for that problem. So here I'm seeing that is creating a sqlcmd file. I'll let it run a little bit longer to see what else it creates I would like it if. Yeah, you can totally create that.

29:16 I would if in addition to it, it would also create some resources file because what I didn't explicitly tell it to do that, but I think it would be interesting to see if it does it on its own without me prompting. It does. Awesome, I love this. So I could have been more specific. I could have said, in addition to the skill, make sure to have some resources for x, y, z.

29:42 And probably is a good practice or best practices. That's what I could have done for my prompt. Sometimes it's better to lean towards being a little bit more specific rather than vague or generic. So what it just did here is that it created an agent skill for helping with troubleshooting, and it's creating scripts to help me out. And it's is creating resources.

30:12 And I want to go see what that looks like. So I think it might not be quite done. It looks like it's still creating some stuff, but I'm still going to go and open the Cloud editor and I'll go see what it has created so far for us. So I'm going to open this demo directory. I think I need to make sure hidden files are toggled up. There you go. So the skills will go in a H file because it starts with may not be able to see it in your editor or your ID until you enable hidden files.

30:55 So we're seeing the skills. I'll make this a little bit bigger and I'll put that out of the way. So excuse me. It has a certain format. So you start it has a name and a description. But then it has all sorts of contexts. My problem is troubleshooting Kubernetes environments. And now it has this context of how to do so. In addition to that, it also created some references on things like common errors, a kubectl cheat sheet of which commands to run, as well as a script.

31:37 This Kubernetes triage script. And I find that very interesting because a lot of contexts that yeah, you could put in your own file, but you don't want to have this context use up tokens when you're asking about a completely different problem. You only want it to be added to your context window when you are troubleshooting Kubernetes environment. So this is the beauty of skills to me.

32:04 And, being able to create these skills and only ingest them whenever they're needed I'm going to stop here. I've talked for about 20 or so minutes, and I want to make sure that Mofi and Lucia has some time to talk about their own section of their sphere. So I'll pass it over to Mofi to talk about how to run untrusted or unsecure generated code in GKE.

32:40 Yeah thanks, Olivier. So this lab, before we worked on it, I think there is a bit of a backstory why this lab exists. Last year, we ran GKE 10 year hackathon for basically GKE turning 10. And when we did that, we got submission of hundreds of projects. 400 something projects. And then we had to basically recruit a bunch of volunteers, Googlers, to go review those projects.

33:09 And when we're doing one of the things that came up, again, not every project got seen by everybody because that is not feasible because of time constraints. So one of the things that happened, there's two problems that I ran into with this. Number one is each judge is using there is a rubric, but they're still using their own biases to judge these projects based on what they think, what they see from the project, how much time they could spend in looking at the code or the markdown or trying it out themselves.

33:39 So that's one problem. The second problem is, if I'm judging, let's say, 20, 30, 40 of these projects, and then I'm downloading someone else's code, downloading that onto my machine, and then trying to run that code again. Generally speaking, you expect the people that are submitting to hackathon are doing the right thing and not submitting code that are untrusted, or not submitting code that is trying to take over my machine.

34:01 But on the internet, you can't really trust anyone. That is the kind of boils down to can you trust the code that you run. More importantly, on the other side of that same story is can you trust the code your agent writes up. So I'm going to quickly go over the slides a little bit just to go through what kind of things we're dealing with here. So it is a hackathon evaluation system built on GCC uses agents as the judge for judging hackathon projects.

34:31 But this can be scaled for other things, if you're using these agents at production for generating your company's code, or generating experiments, or generating any type of code like, again, anything that uses resources, this can be applicable. My name is Mofi. As I said earlier, I'm a developer relations engineer working in Google Cloud, mostly focused on Kubernetes and running AI/ML workloads on Kubernetes.

34:55 And over the last few years, one of the big things that has been is this very, very expensive and big workloads that are running on Kubernetes. And more recently, now you are talking about these agents that spin up at very quickly hundreds and thousands of new workloads very rapidly onto Kubernetes. So trying to figure out trust and security boundaries is one of the biggest challenges right now in this space that a lot of the operators like myself and potentially yourself, are feeling.

35:25 So what I said about hackathon judge, right. You have someone it's again, a human is trying to understand if this code is good or if this project is a good project that fits the criteria. And the goal of this is not to say, OK, the alarm is going to judge everything and going to be done with it, but you still get some sort of a baseline where the LLM with access to tools, looking at files, looking at being able to run code, as Olivia just showed with an anti-gravity CLI, we can have a better baseline of understanding

35:53 of what's actually in this code base, rather than I had five minutes to look at one project versus two hours. Now we have a very solid baseline potentially, if we can have an impartial system, look at all the projects at the same time to give us a starting point. Now, I could potentially create some sort of a automated system that would pull down the project and run some automated script to do things, but doing so opens myself up for the same amount of risk that I had before.

36:26 If I'm pulling all this code that I don't know who wrote or what is the purpose of them, if I'm just pulling all of them down to my machine, I'm just opening myself up for that risk. So code injection is a big one. Even if you're running them in container because you're trying to execute code, the container needs to have a certain level of privilege to be able to do that.

36:46 That basically means if that container somehow escapes the container boundary, they end up having the same level of access they would need to be able to execute code. And any of you who are security minded or look into that space. How scary that can be. On top of that, there also can be some resource abuse. If someone basically submits a fork bomb into as a hackathon project and you just go, OK, let me see how this program runs and just open that up.

37:13 You are basically breaking one of your machines. So with all of these are all the different things that can happen. Kubernetes does have some mechanism to help with this. So Kubernetes runs something called pods which internally is containers. Containers have fairly good resource boundary with cgroups and namespaces. But the problem, as I just mentioned a few seconds ago, is the same as if you are trying to say execute code.

37:40 Executing code. Let's say you're trying to build C binary or running go code or something like building Java applications or Node.js. That runtime need to exist on the container already. So even containers, you will have to give it escalated privilege to be able to do the things like execute code, if you're trying to run code. So isolation within potentially there could be some level of boundary, but it's been shown time and time again.

38:08 There are many different paths to escaping the container boundary. And there are a bunch of CVEs that we found over the last 12 plus years, and a lot of them got fixed, but there are still potentially 0 days that are lurking somewhere, and you don't want to find out what happens to your expensive GPUs and CPU resources when something like escapes the boundary.

38:29 And just like again, crypto mining is probably the least of your concern. If someone gets to your really, really expensive GPU machines, they could probably do other bad things to your production cluster. So this is where sandbox, the idea of sandbox is you can create this boundary like a even stronger boundary that you can give some level of permission to.

38:54 And then whatever happens inside just stays inside that container and goes away very quickly after that container has run its course to help with that. In Google, the open source technology that was developed here called gvisor. And if you're not familiar with gvisor, gvisor is a technology is like a gives your container access to system commands at the user space.

39:19 So instead of having to have root access and having to do everything at the system space, you could do things in the user space. And there is a project in Kubernetes called agent sandbox, which will take advantage of to build a sandbox that can run these untrusted code on our behalf. So this is where sandbox comes in, generally speaking, very low latency.

39:42 It can execute your code. Again, it's just running an API server inside the sandbox. As your command comes in. It just executes those commands underneath. This is not part of sandbox itself, but it works really well with gcc's snapshotting technology, where we can snapshot a running state of an agent, a running container, and then resume that later.

40:03 So it can give us a easy way to save resources without having to continuously run a resource all the time. And you have nice little APIs in Python and as well as Golang where you could talk to your favorite agent framework like SDK, LangChain, crew, AI, Lama, index, whatever you want to use. Because it's a Python SDK, you can just use that to call onto the sandbox SDK.

40:32 Obviously it's built on top of open source standards and it's part of Kubernetes sig. So it has really good support for working with Kubernetes generally. Again, I'm probably a little biased here, but Kubernetes is probably the best place to run your agents because of the proven scale over the many, many years Kubernetes have been around, and it can manage isolated and stateful workloads specifically for AI agents.

41:00 So it has to be isolated because if it executes, if it is executing code, you don't want that to be able to escape that boundary. So few resources that come with the agent sandbox. So you have the sandbox router whose responsibility is to when a request comes in to take it to the right sandbox. The next one is sandbox template. This lets our controller know for a given request what type of sandbox object to create.

41:28 And finally, this is probably the most important part of it because we're trying to build, let's say, code execution agent, which you're going to see in a second. We might need to create many of these judges to judge our hackathon projects. So we can't really just wait for Kubernetes to be able to schedule our workload. So for that sandbox, swarm pool can keep a number of these are sandboxes, pre-created and just sitting waiting for us to then claim and use it for our task, in this case judging a hackathon project.

42:01 So in our use case, what does that look if we're talking about sandbox lifecycle. So you have an agent application that some task comes in and whenever it needs, whenever it has a task available to it. Number one thing it does, it goes and claims a session which will pull apart from the warm pool and start the agent sandbox. Then we have an active sandbox.

42:29 Once we do have that sandbox, router will then be used to pass in the execution command to be able to send the execution to back onto the active sandbox. And then from there, optionally, you could have the snapshotting mechanism. If there's nothing happening, you could have some logic to say, OK, take this entire has a. Every single thing like CPU, memory, network, I/O all of that and stored that in Cloud Storage, just take that entire state and store that in Cloud Storage.

42:59 And whenever we need to restart that back, we could just pull that back out of Cloud Storage and start it from exactly where we froze it. And then we can this is optional. You don't have to do step 5 and 6, but if we want to then after that, we can just go ahead and say our work is done. Whatever we're doing is done closed session and then the sandbox just goes away.

43:20 And in the meantime, while the sandbox is taken, our warm pool will recreate a new to make sure that our warm pool size stays the same. So secure agent workload are the foundation of safe AI system. So I can do a lot of cool things. And they're going to continue to get better at doing the things they do. But until we can say the things we do are not going to escape the boundary, leak our secrets, or let someone else take control over our cluster or our computers.

43:52 All of that are not necessarily useful. We cannot really trust our production system, our data, to these systems. Again so we have the sandbox router we talked about is routing to the right sandbox claims. And then we also have a bunch of other resources that you can create in Kubernetes. We can inject secrets via ConfigMaps or secrets. You can also pass in other configuration using an env subst.

44:19 So you have all these resources being happening. The other one is cold start right. So because we have a warm pool, we don't have to wait for the resource to be created. Every time we can just grab from the pool. Once we exhaust the pool, potentially we'll need to wait a little bit longer. But even then we have some sort of stagger because we're pulling from the pre-created ones.

44:37 And while we're doing that, new ones are being created. All right. So if we're successful in this case, our system processes the task safely without leaking secrets, or letting someone execute code onto a machine that could not handle unsecure execution. So these are the same parts. So we're using GKE versus Kubernetes. Again it's an open source project.

45:01 So you have access to running them on your own Kubernetes distribution. But we have some additional support on top for example, the last one being the most important snapshot. So snapshotting as of now is GKE only feature, which lets you do things human in the loop where if you're waiting, you're waiting on your agent for a human to come and say yes, you don't have to basically keep that running.

45:23 You can just snapshot it, get rid of the resources for that, and bring it back when it needs to. So this is what the application structure looks like. What do we have three base applications in there. We have our front end React application. We have a go back end application. And we have a Python agent that is written. It can optionally also use ADK, but in this case, we're just calling a Python agent.

45:49 That is basically just a simple harness that our go back and calls into. All the data is stored into BigQuery. And for our input, what we have is a number of hackathon projects. These are projects I just made up about 9 of them or 10 of them. And then once you go to the UI, we're going to see in a second, when you click the button, it's going to go and use this harness.

46:13 Here our agent is going to call into our sandbox router, get ourself a sandbox. And inside the sandbox itself the code that is running is using an agent similar to AGI. In this case, I'm just using Gemini CLI to call and pull the code down and run the execution. We're going to see that example just in a second. So I just wanted to give you a bigger picture view of what it does.

46:35 So let's quickly jump on to talk about what we're doing. Just like the previous lab, this is an existing lab and the link is already should be in the chat. If not, someone potentially can paste it again, for folks to go see. So this code lab will be here. You can go through all the steps, the things we're talking about right now in the image. It's also here if you want to see exactly step by step what happens.

46:59 So I would recommend if you are interested in learning more can go see that the code of everything that is happening is also available right here in this GitHub repo. Pretty much the same place where Olivier's code existed. So you can go check the code out and see how all of this works. Again, this is a very simple implementation of this. We can definitely take it much further.

47:23 I just built it out as a proof of concept for us to see if this is something that we can use safely for a hackathon judging system. But potentially, if we were to run this run a big hackathon again, I could go and add even more functionalities to this. But as a proof of concept to show that we could use an LLM as a judge to give us a very nice baseline.

47:46 This worked fairly well. So one of the things I want to do right quick, right now is to go to our cluster to show you what are the things that are running right now. So I have a namespace kubectl get. And this will give us the list of namespace. All the hackathon judge related things are running in this hackathon judge namespace. So if I were to keep control get Po dash n hackathon judge, you would see that I have a bunch of things running there.

48:13 The things we saw there in that image. So I have my agent Python agent, I have my go back end, I have my React front end, and I also have my sandbox router. There's two replica of that router running. And then you also if you notice here see I have five copies of my sandbox swarm pool that are running right. So the reason being again, I wondered about some buffer when I quickly asked a bunch of hackathon is to start.

48:40 I have a buffer so that I don't have to wait for these parts to spin up. For my workload to be able to start the actual judging task is a fairly long running task. Again, long running in the grand scheme of things. Probably not the right word, but for an agent where a lot of the agent related tasks. When you saw Olivier was talking to a GUI CLI, he was asking a question in about five to eight seconds.

49:04 He was getting answers back. But the questions we're asking here is for the agent to go clone a GitHub repo, open the repo, look at the files, look at if the file is following all the criteria, as well as if it is doing things like run all the code. See the code works if there's testing check that. All told, it's like a fairly elaborate task that we're giving our agent.

49:28 Each run can take up to 3 to five minutes, We're letting it do a bunch of things to configure, to test and make sure the code is doing what it's claiming to do. So this is our UI. This is running. This is available here. And in our dashboard we have one project and the hackathon project that I made up. It's called productivity hackathon. So the goal of this hackathon is to build things or build the software tooling that works or helps us in productive in some way.

49:59 Again, it doesn't necessarily matter, but this is the theme I was going for here. And then I have some criteria that is set here because again, I want my LLM to look at the code, look at the running example, and potentially help me understand if this project fulfills this criteria or not. Again, LLMs are fairly good at following instruction, if you can give them strict instruction like that.

50:22 And so I have here about nine projects here. I already ran the judge on few of them. The reason the score is looking a little bit higher, because I did a bad job at doing the math of the criteria score. If you add everything up, actually the total score possible is 5.45. Again, that's on me doing a bad job at doing math. So again, 5.08 is a good score.

50:48 But 4.4 is out of 5.45. That doesn't matter. But that's just something to call out from my end. All of these projects have a GitHub repo. I could just open one of them up. Let's look at one we have not seen yet. Trio task. I can open that up and I can see there's a project URL that is a running example. In Cloud Run I can actually open that and Cloud Run with the cold start and everything should show me this productivity application.

51:16 The idea of this was you create a three task and you can reorder them, you can reprioritize them. And once you and the idea of this is you can only have three maximum tasks in your task list. It can only add more by finishing this task. So again, as a productivity tool, the idea is you don't want to overload yourself, give yourself small, manageable goals, and once you're done, I can do more things so you can potentially finish them.

51:45 And then once you're done, you can just clear everything. Now you can add more tasks. Again, very simple application. Not the point of building the most elaborate thing. So I have that example in Cloud Run. I also have the code available. I can go take a look at the code, but if I'm the judge for this, I'm going to look at the running example. I'm going to also look at the code here.

52:06 But the idea is I don't want to have to immediately go download this code or the other option is I just look at the code and never actually run it and give it my scoring either. Both of these options seem kind of risky to me in one way. The risk is I'm downloading untrusted code. In this case, it's code by me, so I should trust it. But again if someone else is a judge and they're downloading the code, they should not trust it, obviously.

52:33 And the other option is I don't look at the code at all, and I just give it some judgment based on just the markdown that exists in this GitHub. So I can just look at the oh, OK. They wrote down a bunch of things here. I'm just going to give them a score based on that. And then we're basically losing the integrity of our hackathon itself. So what would be good is if to be able to say, OK, I'm going to run this judge run agent, that's going to go do the judging work, and we're going to get the score that way.

52:59 Before I do, though, I'm going to run a watch here so that we can quickly see what exactly happens when I start the run agent. So keep control, get pods and hackathon judge dash L equals to app equals to sandbox. What I'm just doing here is I'm going to run a watch on the hackathon judge namespace. And I'm running a label. I'm filtering by the label where the app equals sandbox.

53:25 If I do this I'm going to I see sandbox swarm pool. So I'm just looking at these five pods here. So I'm going to go ahead and run this agent. The moment I do, my front end is sending a request to my back end, which is then calling the agent. And then immediately one of the things you see is that I have a new part that is being created. So that's the interesting thing.

53:48 So what happened is because I claimed one of the parts, I don't know exactly which part is running my request right now. One of them is I don't know which one got claimed, but the task of the sandbox, one pool is to make sure there's any given time. There is five sitting warm pools that are always ready to take in request. So I can run one task. I could also say 1, 2, 3.

54:12 I can rerun these agents as well. Yeah, sure, I can rerun. And then if I come back here, you would see that. OK, I have a bunch of more pending because what is happening is as new task has been doled out, it is taking away the existing one pool and creating new ones. So when in a stable state, I always have five warm sandboxes just ready to go. So it gives me the moment I click it.

54:37 It's much more snappy because I immediately get resources to run my workload right. So that's awesome. So again, this is happening again. It takes up to four minutes to do the thing. What I want to show you is also look at the agent's response as they come back. What you can see here is that I have different criteria given to it. Instead of just saying it is good or bad.

55:00 It can actually look at my code, look at my lines of code. And this is the instructions sent to the agent harness to say, OK, majority efficiency win is found in these lines. It can tell me exactly the line number where it found this a things why it gave the score. So it's not just I feel like it should be a 4 out of 5, or it should be a 2 out of five.

55:20 It's more along the lines of I think it is good because it called out x, y, and z things. So it is looking at the code, looking at all the different things. And it can give me kind of a way to look back upon. So if I look at this code and then see, OK, there is a line number that's given to me here. I can potentially go there. And based on what it said I forget what it said.

55:48 But I can open this up and look at the code itself to see the claim from the model to verify what it said it was. So as a hackathon judge, it would give me then potentially the path to verifying what my claims are instead of just randomly saying, oh, I felt like this should be this. So there is some proof that is added on top. OK probably broke something.

56:14 This is the day of breaking demos, it seems. OK I don't even know what that particular sentence means. OK kubectl, get dash and hackathon judge. The front end is still running, so. Gateway and hackathon. Judge what is a live demo. If the networking doesn't give you a little bit of pain. OK, looks like something was happening, but it's back anyway. So the whole purpose again, this hackathon is an example of showcasing code execution.

56:55 But now let's try to map this to anything else. Now we're getting a lot of our code written by agents. And oftentimes we're running those codes directly on our local machine. We're giving that full edit permission 99% of the time. That's probably fine, but that 1% of the time where you see people have problems with going off, deleting their code base or getting rid of their databases.

57:19 This is where a sandbox environment where you can specifically control the type of permission that potentially is hard to control on your local machine. Saying your local machine should not be able to talk to your database is probably much harder than doing that. In a container and Kubernetes container, you have a lot more control with the network policy.

57:39 Maybe if you're on a coding agent server, that network policy stops it from talking to any of the production level things. This type of control is much harder to do on a local machine where you can do that with an agent. So I'm going to stop there. Now I don't know if we have any potential question. I don't see that. But if that is not the case, what I want to finish with is agent sandbox is an open source repo.

58:06 So I would ask everybody to check out that. And the other thing I want to actually call out this is super new. And this lab was created, what, seven weeks, eight weeks ago. And in this space moved so fast. In that time we have been talking to the people in the industry, as well as customers and other people. There's another project that exists in this space.

58:27 Now, along the same vein, it's not a replacement, more like an augmentation, which is called agent substrate, which gives us even faster time to getting our agents and our sandboxes and our agent related workload to start and run by somewhat. I want to say somewhat bypassing the Kubernetes scheduler in some ways. We create some workers. This is not the talk for that, but there are some learning to be had about sandbox agent substrate.

58:57 This KubeCon, if you are not even if you're going great, is happening in Salt Lake City in about two months, even if you're not going, I would keep an eye out in that space, because I'm sure this KubeCon, I feel like is going to be really big for agent workload on Kubernetes. So you can definitely check out all the talks that happens afterwards. So I would highly recommend, if you're in this space interested in learning more about in this space.

59:21 Kind of keeping an eye out for this KubeCon that is coming up in a couple of months. With that, I'm going to pass it off to Lucia to talk about agents. Data and all the other good stuff. Thank you so much. That was great. I feel that it's very brave all of you to be doing live demos with non-deterministic stuff, and we're going to continue doing that.

59:45 So this is the last demo that we are going to do today, and we're going to be talking about data specifically about graphs. Before that, once again, my name is Lucia. I'm a developer advocate as well as Murphy and Olivier. And today we are going to talk about data because we have seen how important it is to keep our containers and clusters alive and healthy, and how AI can help us with this.

01:00:15 We see how important it is to be sandboxing are applications and how useful it is to have a proper way of running code that we feel may not be entirely secure, or that we're not sure about. And I'm going to tackle another pain point that we are facing in the era of AI, which is basically hallucinations. So at the very beginning of our AI journey as an industry, we all got the mandate or the inspiration to do something with AI.

01:00:49 And when the hype started some years ago, we didn't really know what we were doing. And basically what we tried to do was create a chatbot. Even though chatbots were not really popular before this, but we created chatbots and we asked our large language models to produce anything useful out of those chatbots. So the beginning of that was probably fitting as much as we could from text files, or if we were lucky, embeddings into this chatbot so that we would get an answer that would make sense.

01:01:28 We quickly found out that didn't work, and as an industry, we decided we needed to apply the embeddings that the transformers already using into these knowledge or sources of knowledge, so that the agents would ground their answers in some truth that we know is good, right. Similar to how Olivier was using MCP servers to get the greatest and the latest on a specific topic, instead of using whatever the model is trained on, we started pumping embeddings into our databases, and this worked very well because one of the

01:02:07 things that these embeddings do is transform, like in the case of Gemini, app a multi-modal set of inputs into these mathematical representations, and put them into a database so that we can ask questions and do semantic searches. So in this example, if we're looking for a cat napping and we have pictures of cats, chillaxing or even dogs who are similar to cats and maybe just sleeping somewhere, we will get those things as a result.

01:02:45 What started happening though, is I don't know about you folks. I'm a pet owner, I have three cats and I take a lot of pictures, a lot of videos of those cats. I talk to my family. I describe their day like they're my children because they are my furry children. And that means that we have a lot of content and a lot of volume for this multi-modal data.

01:03:08 So we started heading a computational limit of how we can process all of these things. And the other thing that we started hitting is that we know that these embeddings, when they are fetched using Retrieval augmented generation by an agent, let's say the agent decides, I don't have any information about Lucia's cat. I'm going to go and fetch it from the database.

01:03:33 That's semantic search has a limit. But I have an example for you to show you these limits. Let's say that I'm looking to figure out which of my infrastructure artifacts, virtual machines, containers, Kubernetes clusters, machines under the desktop have been hit by vulnerabilities that allow for remote code execution. And if I do that search on my database, I will get semantic results or results that are semantically similar to remote code execution.

01:04:11 So in this case, I'm getting the first CV that allows for remote processing. The second remote execution functions. The third one is a little bit further away from that semantic search, but it kind of semantically it means the same. However, what I'm interested in doing after this is understanding which of my systems are actually compromised by this.

01:04:33 And that means that I want to understand which container images are compromised by this, which virtual machines are compromised by this. And while I'm at it, I would love to know the characteristics of those things. Are there user facing. Are they having potentially access to very critical information like financial or HR information, that would make it more of a priority for me to go and look that up.

01:05:03 And that is where a graph comes in. Because the graph in the case of an agent giving me this answer, would allow the agent to not just go and look for the RAG results that it was given. It would allow it to traverse through the different components of my infrastructure because they are related. As a little reminder of why I'm talking about graphs. Graphs are composed of two things a node and an edge.

01:05:32 The node is going to be the entity, and the edge is going to be the relationship between those entities. So in the example I was given before, the nodes would be the libraries in whatever language they are, and that would be a property of that node. The node could be a piece of hardware that we have or some piece of infrastructure like a virtual machine.

01:05:56 And the relationship to them is for example this library is deployed in this container image. And then this container image is used in these systems. And the beauty about graphs is first, it allows us to search for relationships in the data that the traditional tabular data does not allow us to do very easily. And the other thing is, because they have all these connections, agents are actually faster at finding the right information because they can traverse these nodes.

01:06:33 So this means that they also spend less tokens, and they will be more accurate in giving us the information that we give them through an edge. What we are going to do today is not related to vulnerabilities, is related to pictures of our PTs. We are going to go through all of these multimedia files that we have that are pictures, videos, audios, texts, and we're going to extract the facts about those pets.

01:07:05 And we're going to build a knowledge graph out of those. A lot of companies, a lot of our customers have been doing this, and in production it looks a little bit more complicated. But most of the times in production, these businesses, they know their business very well. So they define the entities very well. It's very common to do this with manuals for manufacturing, and then also having that attached with policies or employee manuals so that you can really understand if you're say, manufacturing cars, which parts are

01:07:40 composed in a car and which pieces of information are relevant to that car based on your internal policies or your quality assessments or even whatever feedback your customers are giving you out of the usage of those cars. The entities that we're going to extract in this case are going to be hobbies, the favorite foods, and the preferred toys of these pets.

01:08:12 And the way that we can see or visualize the graph for our PTs is we have these two cats, Yoda and Madonna. They both absolutely love sleeping, which I think is not going to be a Shock to anybody who has cats. But Yoda prefers chicken, and Madonna really likes her yarn ball. And to give you an idea, if you're not coming from the data side of things, the language that we're going to use, the query language that we're using against the graph is going to be GQL or graph query language.

01:08:46 And this is the SQL version of these, this same statement. We're going to dig a little deeper into this in a bit. Why we're using GCC. Of course, we're using GCC because in this case, even though the amount of multimedia we have is not that big. Most of the times your companies will have a lot of audio files, a lot of videos, a lot of texts and different versions of them.

01:09:16 So, this is a great way of parallelizing those heavy workloads and the processing of it. And the great thing about GCC is it's going to scale the compute on demand. You do not have to do anything manually there. As Murphy was saying, we have proven time and again that GCC is the right platform for us to do this kind of processing in at scale. And finally, of course, the infrastructure is automated, which we appreciate.

01:09:47 We're going to look into the code, but we are going for a decoupled data flow. And this is by design. And this is on purpose of course, because we are going to be scanning the assets with what we will call producers. These producers are going to enqueue the analysis of those tasks via Pub/Sub, and then the workers will start to drain those queues as they process the different types of files.

01:10:12 Wear videos and audios and fun facts about our PTs. All right. So we're going to get started. And if you have questions during this talk please feel free to ask. I do have some stuff preloaded here. So I'm going to start with the lab real quick. I'm using the skills platform here because it's easier for me to provision a brand new environment to show you that this is real and it works, but the link that we got here at the beginning works is exactly the same thing, only that it runs in a different place.

01:10:55 But this is the architecture that we're running for today. We have all of our beautiful all files we have Pub/Sub that is getting the files for processing from the producer job. And then the workers are going to start draining that queue. We are going to ask Gemini in this case Gemini 2.42.5 Flash to help us detect the entities in the different files.

01:11:22 And then we're going to save everything into two tables, mostly in BigQuery Studio. These are relational tables. You'll see this in a few minutes. They are the usual tables that you're used to. They have columns and the columns have types. What we're doing is creating a property graph on top. And this is one of the superpowers of graphs on BigQuery, because it allows you to basically continue doing whatever you are doing with your existing tables and overlay graphs on top.

01:11:54 And when you delete the graph, nothing happens to the data. So you can always recreate it and add more labels, or modify it as you go. All right. So I went ahead and did all of this in the lab ahead of time, mostly because I find it boring. So I did not want to bore you with it. And I did set up the cluster, you can see, because I was running this on purpose while Sophie and Olivia were talking can see that I created.

01:12:28 I did the setup scripts. When my setup scripts finished properly successfully, you will see a unicorn having a party. And I did create the containers and I left this part. That takes a lot of time. Otherwise well, not a lot of time, but it takes a solid 10 minutes so that we don't have to worry about that. Now let's look a little bit at the code as the other labs.

01:12:53 The code is in Git repositories that you will be cloning as part of the labs if you decide to do it yourselves. But we have the producer and the worker. The producer is basically going through a storage bucket, and for each of those files is creating a new is publishing each of those files as a message in the Pub/Sub topic that we precreated during the setup.

01:13:22 And then we have the worker files. The worker scripts are basically deciding picking up the message from Pub/Sub. They are getting the GCS path from there and based on the type of file that we have, they are deciding to do the processing with this script called process file. The script process file is basically a massive switch that will call one of the many processors that you have and what the processors are doing are fetching the file based on what type of file it is.

01:14:04 And here's the magic of generating the knowledge graph. This is a very simple prompt. If you were doing this in real life with something a little bit more serious than the pictures of your pets. Most of the times you would have a very elaborate prompt, but that's the topic. Maybe for another time, or maybe for a chat offline. But in this case, basically we're telling Gemini, please analyze this video, extract the entities related to the pets that we see in the video, and create nodes and edges.

01:14:36 We are telling exactly what those nodes and edges need to look like. We're asking for a response in JSON, and we're asking it to extract the pet, the breed, and if there's any of these references, toys, food, or actions that we have as hobbies. And then basically the worker is going to acknowledge that message so that we do not have to process it again.

01:15:04 So I have already done that first part. I am going to go ahead and apply that job so that the producer. Starts working. And while that producer starts working I'm going to instantiate the worker as well. While that is happening, we don't need to monitor it. I want to oh actually, I think I messed up, but that's OK. I should have done this first. So the producer and the worker are going to try and create these other things into BigQuery, but they will not be able to.

01:16:11 They will not be able to. Mostly because OK, this is fine. Because these tables do not exist. So I'm going to go ahead and create the tables. Yeah this table created a new table named edges and another table name nodes and probably. These jobs couldn't do what they needed to do. So we are going to go and look for the logs of these jobs. And I'm pretty.

01:16:50 Let's see. If they are running. OK, so the producer job. Ran but I'm pretty sure we were not fast enough in creating the tables for the producer job to run. Oh, actually, we were fast enough. Maybe OK, I think we're going to be OK. This is the magic of Pub/Sub and the job provision taking a little bit longer than. I took to create those tables anyways.

01:17:28 This is fantastic. So you can see that the code that we were looking at before is referencing this files that we have here. While this is working we can look at the files. And you can see I have my bucket. This was created as part of the lab and it was copied from a master bucket that we have with the different files. But for example, we can see Madonna's profile picture.

01:17:59 She is really cute. We can see SQL profiles. Profile picture. Yes, my cat, one of my cats is called SQL. He's also very cute. And if we go into the additional media we can see descriptions in audio files. We can see the different videos and we can see some additional pictures as well. For example, Joe, that is one of my colleague's cats who's also very cute.

01:18:35 And well, let's see if our worker. Fetched any of these things. Nice OK, so our worker worked despite me not doing things entirely in order. But you can see that basically the it started with this, for example, SQL description, and then it starts finding the different nodes and the edges that it gets from the different files, and it just inserts them into BigQuery as it goes.

01:19:14 So if we go into BigQuery right here and I refresh. I now see these two tables that I created manually. And these tables will have. The edges remember these are the relationships. So we have the source IDs Yoda. And the target ID is chicken and the relationship is likes. That's a pretty obvious one. And the nodes are going to be a little bit more convoluted.

01:19:51 And of course, these schemas are whatever you want them to be. You could be generating time dimensions into these type of schemas so that you can navigate a versions of documents. For example, in a given graph, some people add one of the most common graphs are related to navigating distance and having the optimal path. So one node will be address and node B would be address B.

01:20:26 And then you have as part of the relationship the distance. And that could be a property there. So let's look at what plain SQL what this is in BigQuery. And we see that SQL girl likes freeze dried fishes. I'm going to close this here and maybe make this a little bit bigger as well. And while we're here, because we created embeddings for our tables.

01:21:06 We can ask in this nice SELECT statement about pets who like to relax. And we're going to base that out of the biography that Gemini app hopefully extracted. And we're going to use this endpoint. This is one of the endpoints that exists in Gemini agent platform. And the connection between BigQuery to agent platform is given by this connection here that was created by the deployment script.

01:21:39 So this was automated. It's very easy to create otherwise but this is one of the superpowers in BigQuery that is basically having the ability to talk to files or GCS buckets like we're doing to fetch any type of video and then generate embeddings and ask this type of questions directly on pretty massive tables. We're going to come back to this just for the sake of time.

01:22:06 And OK, it just finished filled my impatience. But we can see that we found out of pets like to relax Daisy, who likes napping on soft blankets, or Joel, who likes laying in the sun, or Madonna, who likes naps in the sun, or Cappy, who likes soaking in the water because she's a capybara, and Yoda who really likes cuddles with mom. Now moving on to our graph because we already have everything that we need, we are going to go ahead and create the graph.

01:22:46 And this is what I was talking about. The last time, all I have to do is tell which tables or which it could be the same table. Sometimes you could have the same table as the node and the edge and tell it, which is the key that I want to base this graph on, and then expose the different properties or columns that I have in this table for the graph to use.

01:23:18 And so in this case, we have our nodes table as the node and the edge tables as the edge. You could be a little bit more creative if you wanted and have more creative names as well. So of course, our table, our graph looks like this. This is not a very impressive but because we have the notes. Talking to notes through an edge that is an edge. To see this in action, we are going to query this graph using a multi-hop query to match different pets that are connected to the same node.

01:23:59 In this case, the. This data set doesn't allow for this, but what I like to do sometimes is to check how many degrees I have from Sundar or I don't Celine Dion. And you can do this very nicely with graphs as well because everybody at the end is connected. This is a very simple response that we have. So we have pets who like to rest, and we have one dog and one snake who like to rest.

01:24:31 I would venture that probably all of our PTs like to rest, but it was not there in the information. So that talks about the quality of our data set. And then we are going to very quickly chat with the graph, because I've been talking about how agents are able to chat with the graph. And we're just going to put it to the test. Most of the times the more mature and clean your data is, the better your agent will perform.

01:25:05 But that, of course, requires more infrastructure if you're doing this at home. This is a known thing. So I have just enabled all of the APIs, but I still do not see the conversations. So I close it and I'm going to try again. Oh it's enabling them. Yeah is it though. Are you sure. OK I'm just going to go ahead and do the good old refresh. Because of course.

01:25:46 Sometimes this happens on a live stream. So one more. There we go. We can create an agent. I'm going to create a call it chatty. You could add a description. It's not going to be offended. And I'm going to choose my graph. As the source. I'm running this in the European region. So I'm going to leave that as well. And I like to play with this. So respond with an epic tone because why not.

01:26:36 And which pets like to eat chicken. Only because I know we have pets who like to eat chicken. Well, this is going to do is it's going to create a JPQL query and have added with our beautiful graph. Hopefully sometime soon. But while this happens, and because I do not want to run out of time while you folks are looking here, I do want to call out this extension that I'm using here, which is the data Agent Kit.

01:27:18 So the data Agent Kit was announced during next of this year, and it allows me to basically do pretty much anything from in this case, the Cloud Shell Ed. But it works in anti-gravity. It works in VS Code. It works on code OSS. It works on pretty much anything that is based out of theta or looks like VS Code like in this case. And I can see the jobs and save queries and execute those saved queries.

01:27:49 I don't have any in this case, but they would look exactly like this. And as sources you could have BigQuery or you could have other of our databases like AlloyDB or Cloud SQL or Spanner. So in this case, you saw me at the beginning. I created these two tables directly from here, which is really nice. You can also monitor your data flow or Apache Beam jobs.

01:28:18 You can look at your Apache Spark serverless jobs as well. You can run notebooks from here, which is also very, very neat. And you can initialize orchestration pipelines. And if you are using the data engineering agent as well, you will get a much more rich experience here because you could generate some DBT or Dataform pipelines and orchestrate them on Cloud composer or airflow.

01:28:45 And you were looking at me before showing the Cloud Storage right from here. And of course, you have access to the catalog. So this is a pretty neat extension. It's called the data Agent Kit. And if you do not have it installed and configured, I would absolutely recommend you do. Let's see. We have a result here otherwise. And we asked for an epic tone.

01:29:11 And I do not know how to pronounce this word. I think it's the first time that I'm seeing this, but that to the tales of the beast who feast upon the sacred fowl through the ancient web of connections, these legendary pets emerge. Yoda the wise, who partakes in pure chicken. Daisy the noble seeker of chicken and rice, and Rocky, the fierce Warrior whose favorite nourishment is chicken, and who listed Eden chicken among his grandest hobbies.

01:29:44 That's really nice if you want to look at what it did. It created a graph query. It failed, and it corrected itself here. And that actually worked. Then it created a few more queries. And finally, we got the executions and the results that we wanted. And that is all from me. And with that, I think we have some time for a few questions, if we have any.

01:30:20 We have a few. I do. Great Thank you so much, Murphy and Lucia, for your labs. I collected a couple of questions from our stream. There is still time. If you have questions, whether you're watching from X, YouTube, LinkedIn, wherever else, type some questions in the chat for myself, Murphy or Lucia and we will try to answer as many of them as possible.

01:30:47 So I will start with a first question here from Sanjay. So Sanjay asks for the Gemini and MCP diagnostic workflow. How are you scoping MCP server permissions. So a diagnostic agent gets cluster visibility without becoming a lateral movement risk if it's compromised. I can take that one. My answer to that would be that in general best practices, you would probably not want to have a heavily.

01:31:19 Scope a MCP server, something like a MCP server that has a lot of different permissions. What you would probably want to do is have a gateway in between yourself. The agent, as well as the tools and the MCP servers that you're using. Something like Google Cloud agent gateway, where you are able to put in place some policies and restrictions on what the agents are capable of doing, and what they're not allowed to do.

01:31:50 And this would minimize the risk, rather than just having an MCP server that you're using that's scoped to get access to all of your resources. Thank you. Sanjay next question from Jonathan. I can do the same using anti-gravity desktop. What are the advantages of using the CLI. Lucia, do you want to take this one. I'm sorry. That got me really off guard.

01:32:26 I can take it to anti-gravity desktop using the CLI. I'm going to give an opinion and maybe not necessarily the official response. It's mainly it's mostly a matter of choice. So if you are using another Ide that you prefer for whatever reason, like I was we were using the Cloud Shell or we were using I was talking about VS Code because sometimes I do use VS code for certain use cases.

01:32:56 That is when I use the anti-gravity CLI. If I'm starting something from scratch and I do not need to set certain things, especially because at Google, we have some things that we need to use. Sometimes I am going way ahead of with anti-gravity. Especially after 2.0 because you get the two different views with the Ide if you wanted, and then you have the agentic workflow if you just want to use that.

01:33:33 So for me, it's more of a matter of preference, but I don't know if anybody has another opinion. Yeah, I think this one is a very opinionated thing. If you're more used to the flow of a desktop application, that's perfectly fine. Another thing I think recently, we announced and I could be wrong because I don't directly work on that product, but anti-gravity, we're working on a lot of extensions to other IDEs as well.

01:33:58 So if you're in the world of other IDEs like JetBrains, IDEs and all those places, you could still have the same or mostly the same harness of anti-gravity. The reason I feel like a lot of folks, myself included, like CLI quite a lot, because what Lucia said CLIs are very portable to move around pretty much every interface you will find yourself a CLI, but if you basically are used to the desktop application that potentially doesn't exist all on your workflows, for example.

01:34:32 So I do a lot of work in Cloud Workstations, I don't this is not like a full anti-gravity desktop application. I just have access to Eclipse to Code like environment. But I always have a terminal. I can always run a CLI so that one is a again, a matter of opinion, but I think CLI. I personally find it to be more portable, so I usually stick to it. But also like again, Olivier and myself are we're both infrastructure person.

01:35:02 And I feel like too much color hurts our eyes and we tend to Black and white too much, I guess. I don't maybe. Yeah, I'm much of the same. I actually use both anti-gravity ID and anti-gravity CLI. It just depends on what I'm working on. Am I doing something thermal heavy, or am I doing a lot of just editing files or answering or asking questions. Yeah and I'm not sure if I saw this or am I just leaking information right now.

01:35:37 So I probably should shut up. But can you not potentially also share your session between CLI and Ide anti-gravity, or am I just like making it up right now. Am I hallucinating. It could be hallucination, I don't remember, but anti-gravity has so many features and they're releasing features at the speed of sound, so. Well, for all if you heard it here first, don't tell anybody I said it, but.

01:36:05 All right, I'll move on to the next question. So Ebony asks. I'm curious about context engineering. How is it different from traditional prompt engineering. Do you think context engineering will eventually become more important than prompting as AI agents scale. Do you want to take this one. Yeah, I think so. At the end of the day, an LLM you are giving an input and getting output back now.

01:36:36 Context engineering. When we talk about that term it becomes the concept of how do I go about getting that input to the LLM. So prompt all these techniques people built auto write this thing in the beginning. You are give it a persona. All of this stuff, they are still useful to a certain degree. And everything we're sending to the LLM, if it's a text based LLM input, it is a prompt that we're sending.

01:36:59 But context engineering kind of gives us a little bit more control over, like typing in or copying, pasting hundreds of lines of this system instruction versus having our system generate those like snippets or things that it needs. So context engineering is a more of a repeatable, programmable way to do prompting in some way. So in some ways, yes, we're getting to the point where yes, we're going more towards the world of context engineering, but at the same time, we're also doing more to not do prompting, where we

01:37:28 have skills like grill me, where instead of you writing the whole prompt, you ask it to pose a problem and have LLM ask you a question to build the prompt itself. So at the end of the day, if you look at the thinking things or if you look at the history of your chat against the LLM, everything is the prompt. But the question is whether or not you hand typed a few hundred lines of prompt, or you had some mechanism to get to that spec or that prompt without typing a lot.

01:38:00 At the end of the day, I feel like a lot of the things we're doing is to reduce typing is number one, and also get to a more repeatable process rather than having every prompt being this one off, even last year or the year before, there was this website of prompt gallery. People like collecting a bunch of this golden prompts to get things done in 2026, most people are not going to a prompt gallery and downloading a 300 line long prompt to get something done, because most cases, you can ask a very simple because models

01:38:32 are getting better too. So yes, to answer your question, in shorter term context, engineering is probably going to become the main way we talk to prompts. But at the end of LLMs. But it is still prompt engineering, but it's just a different variation of it. I think nowadays people will not necessarily go to a prompt gallery, but instead they would go to an agent scale sort of repository and they would take the context from there.

01:38:58 Yeah, but also scales are also just a big bundle of prompt that gets loaded based on some description, which is we're saving on context by not always having a prompt, only loading the prompt when you need it. But again, at the end of the day, text prompt just get loads gets loaded onto the context of the LLM, right. Thank you very much, Murphy. I'm seeing one more question here.

01:39:23 So Daniel asks, do prewarmed sandboxes cost more. I think this is a question for you again, Murphy. Yeah I mean, so it's a trade off between do you want your sandboxes to run quickly versus do you want your sandboxes run cheaply. So yes, the more pods you're constantly having run on, your cluster is going to use resources. But there's two caveats to this.

01:39:49 Number one is if your cluster already have nodes that you are paying for pre-warm is not costing you anything extra because you don't pay per. In regular Kubernetes, you pay per node. The second part of this is if you're using GKE Autopilot, you do actually pay per. You only pay for the resources that are currently being consumed by one of your workload note that this means when you're not using them, autopilot can automatically shrink your cluster to wrap around the workload, and you basically cost a lot less.

01:40:21 In that world, you are paying the cost of cold start because autopilot will potentially sometimes have to create new nodes to be able to give you the room to run the workload in, that having a little bit of a buffer for your workload to not be stuck for a minute to a couple of minutes while the node comes up, can be super powerful for your workload to feel more responsive rather than just sitting around.

01:40:45 If it's a long running thing, you probably don't care. But if it's a thing that's going to agents going to respond within the next few seconds, if it's a user facing thing, you potentially want to have some pre warmed so that you take a hit on the cost, but you are saving on time, which at the end of the day actually turns into better response time, better revenue hopefully.

01:41:06 And so it's a trade off. You have to look at your application and see what type of SLA, SLA you're trying to hit. And based on that can set that number. Great I'm not seeing any more questions. So I think that's going to wrap it up. Thank you so much Mofi Lucia for your wonderful, wonderful labs. Thank you for everyone watching, and we'll see you again next time. Thank you. Bye bye.