Decoupling the Brain from the Hands: Claude Managed Agents (Part 1 of 4) — Anthropic Transcript, AI Summary & Key Points

00:01 [music] Thank you so much to everybody for for joining. Um I think we're going to get started given we have a have a full house. Um hi everyone. My name is Priyanka. I'm a member of technical staff at Enthropic. Uh I lead the engineering team that built cloud manage agents and I'm really excited to be here to talk to all of you uh and help you ship your first manage agent.

00:33 I've been anthropic a little bit over two years and the last 12 months I've been focused on uh building um agentic infrastructure and hope to share some of our learnings with you. >> Uh hello my name is Gabe Semach. I'm a member of the technical staff uh building cloud managed agents in Priyanka team. I've been at Anthropic a little bit over a year and I've been mostly working on this my entire time.

00:58 >> All right. Um, so here is our agenda. We will kick it off with h a little bit of an evolution story of how we built and scaled managed agents, some of the pitfalls and the learnings that we had along the way. Then we'll switch to a hands-on part of the workshop. Uh the first part uh we will build uh a very simple agent uh using cloud platforms console.

01:22 You will not be writing any code during that part. We'll just kind of prompt our way to our first agent and test it out. Uh then I'll hand it over to Gabe to walk you through like a hands-on portion. We'll be writing code using cloud code or your favorite coding agent. Um and then build a more complex financial agent and then we'll have plenty of time at the end for Q&A.

01:41 We we have a twohour slot. I know they broke it down into four parts, but it's basically one long presentation. Um, hopefully it doesn't take the full two hours. U, we want this to be pretty interactive, so if you do have questions, raise your hands. We have a couple folks around that can run to you, hand you a mic uh with a question, especially as we get into the hands-on uh portion.

02:04 All right, let's let's get going. Um, so I'll start with the basics. So, we'll we'll start with what what is an agent or what makes an agent work? Um so agent is basically uh a model in a loop with tools. So you'll see on the left you have the model that's deciding and taking the next action and calling tools in a loop until your task is done. So that's like the very basic what what an agent is.

02:32 >> Excuse me. >> Y slide deck or slide. >> Oh, a link to a slide deck. Um, we can try to >> we we can we can share it after. >> Yeah, we we'll share it after. It'll be hard for us to get it out right now. Um, but happy to answer questions uh and and share the slide deck at the end. Are you is it hard for you to see see the slides from the back? Okay, sorry about that.

03:04 Um, all right. Uh, the next thing I wanted to chat about is what a harness is. harness is typically like a layer that wraps around the model that helps you make these tool calls in a loop to get you the outcome for the task that you're the task at hand and it's making assumptions about what the model cannot do and trying to sort of fill those gaps. So it's running in a loop making tool calls managing your context encapsulating everything that the model cannot do.

03:30 A good example is our set 45 model had something what we called uh context anxiety. So whenever sonnet 45 felt like you're getting to the end of the context limit, it would try to wrap up the task really quickly. So when you're building a harness around sonnet 45, you would add something like a context reset to get around the limitation of the anxiety.

03:52 But with the next model like opus 45, opus 45 did not have this limitation and you did not need to do this like context reset for opus 45. So when you're building a harness, you want to encapsulate all these assumptions for different models and keep up with it with every new model that comes up, which is why building a good harness is really really important.

04:13 And that's something we've tried to do with cloud managed agents. And then the last bit uh you want your agent to have access to a sandbox or a computer essentially uh a place where it's going to run the code it's writing to execute the task. Um, so typically you're giving cloud access to like a sandbox or container where it's executing the code it's writing.

04:31 It's able to edit files, install dependencies, etc. Cloud code is a really good example of this. It's running, it's a harness essentially running locally on your computer, has access to your files, and it's executing, updating your files as it goes. Um, it's also writing uh code that's untrusted. It might be buggy. So you want to make sure when you're deploying production agents uh your a the code your agents are writing is sandbox and contained within a container.

04:57 Um and then you want your sandbox to be isolated and the blast radius to be limited. So typically you want to execute every task in a separate container so you're not mixing up and you're not um uh increasing the blast radius for for your tasks. Next up, I'll walk you through um the first architecture that we actually built managed agents with. This was in December of last year where we shipped our first architecture to our early access customers, got a bunch of feedback, and then actually pivoted to a new architecture.

05:28 So, I'm going to walk you through some of like the pitfall pitfalls and like issues that we saw with the first one. So, with the first architecture, what we did was we put everything in the container. So we had the session log of everything that's happened within that session. So test session is essentially your um unit of work. Um the harness was also running inside the container and then the sandbox execution was also happening inside the container.

05:53 The benefit of that was like because everything was in the container there was no service boundaries to design for. Everything was local um easy to edit. File edits were essentially just like sis calls. But what happened was we realized we adopted a pet. Um so if uh if you've not heard about the pet and cattle analogy from like the old infrastructure days, um pet is when you essentially have a named and handtended entity.

06:20 When it gets sick, you have to nurture it back to life. Otherwise, you you basically cannot afford to lose it. And cattle is more interchangeable, more disposable. So when one fails, you can replace it with another one and move on. So a good example is imagine you had like a single database shard. It goes down, your product is down, you have a pet. But imagine you have a master replica set up.

06:43 Your master goes down, you can bring up a replica. That's a cattle, right? So you have many instances. One goes down, that's not a problem. You can bring it back up. So what we had with our initial setup was essentially a pet. Um so few few issues with this debugging was really hard because everything was running in the container. We didn't know what was causing the issue.

07:03 So a a drop packet a dead container would show up very similarly and we had to essentially like websocket stream into the container to debug. Um the other issue was investigating meant we had to uh essentially investigating was hard because customer data would live on these containers. So we didn't have a way in essentially to go in and debug. So that was like the other issue we saw in our early access.

07:26 And then the last one was if a customer wanted us to connect to let's say an M MCP in their VPC, it meant peering networks or shipping them our harness and having them run it. So there were like a bunch of challenges with that architecture that that we weren't able to really have a good solution to overcome. So what we decided that we wanted to essentially decouple the brains from the hands and I'll dive into what that means.

07:53 Um so the agent essentially became three separate interfaces. On the left you have the session which is essentially an appendon log of everything that has happened in the session so far. In the middle you have the brain that's your harness or the agent loop along with cloud. Um and then on the right are the hands. So sandboxes your MCP tool calls etc.

08:13 Provisioned on demand uh gone when the work is done. And if you need if you lose a sandbox you just spin it back up. So every component essentially became a cattle and interchangeable. So when the sandbox died, uh the harness or cloud could choose to just spin it back up, right? The error essentially went back to cloud and the sandbox to cloud was essentially like any other tool call.

08:35 So the tool call fails, sandbox fails, it decides what to do and it might decide to spin it up again or choose to continue to move on. um when the harness died, you could bring up a new instance of the harness and continue where you left off by essentially just reading your event log from the session and continue from there. So work resumes from the last event as if nothing went wrong, nothing happened.

08:57 Um so the only thing with protecting here was like your session log. Everything else was interchangeable, could go down, you could bring it back up. The other thing that we did was credentials never entered our sandbox. With our previous setup, your credential would be in your sandbox, so you'd be more susceptible to prompt injection. The change we made was if the sandbox ever needed to call out um to uh something that needed a token or access control, it would be routed through a proxy that would inject your token

09:31 before it called the external endpoint or external MCP. That way the model never sees your credentials or your tokens which means you can never prompt inject claude to say like hey give me your tokens or print out the tokens or write them to a file. You can never do that because cloud actually never sees your your tokens. The other thing we did was because we had this like append only event log in the session um our compaction strategy could stay flexible.

09:57 So nothing is ever thrown away. We're not compacting and changing your event stream. We're always storing each and every event that happened and then the compaction strategy is the one that's deciding oh do I want to look at my entire event log uh summarize and put it in the context do I want to only look at like the last n events and put those into context.

10:14 So we we could have a more flexible context strategy because of this like interface design that we chose. And then the other benefit we saw is because we could immediately start our inference calls, we were able to bring down the TAM to force token by about 90%. So the cloud would only spawn the sandbox when it needed and it would not wait for the container sandbox to spin up for you to get your first token back.

10:41 So we were able to get like a much snappier and faster experience with this uh architectural change. Um, and then here's sort of like the illustration of like one brain can reach many hands. So your agent loop or harness can now call a sandbox container. It could call into a customer's VPC. It could call into an MCP server. So all these like hands are essentially interchangeable.

11:05 You can call and plug in many of these and it made our architecture a lot more flexible. Um, so if you take one thing home from this, it's that harnesses encode assumptions and assumptions can go stale. So we designed interfaces to outlast these assumptions and made the architecture more flexible. So this allowed us to like I said change our compaction strategy, change out the harness completely but the interface and the architecture remains uh the same and we can really have a more pluggable way of operating this.

11:43 All right, so next we're going to get into a little bit of a hands-on portion. U we got you some complimentary API credits to help you build with us today. So, please scan the QR code, either sign in to your cloud platform account or create a new one and you should be able to um get in there and get those credits so that you can build with me and and then with Gabe.

12:08 And I'll I'll leave this for for a second. Hi. >> Sorry, what? >> Last slide. It's basically what we talked about here, right? This is the this is sort of like the three interfaces we designed for, right? So the first one is the the the question was like what were the interfaces that we designed that ma made things outlast uh changes. So we have the append only session log.

12:41 You have the harness that's again interchangeable and then the hands where you can plug in a sandbox tool calls or whatever the model might need access to and that that's what kind of makes it more generic and extendable. All right, going back to the credits. Uh, does anyone else need to scan this before we jump in? Okay, still seeing a couple phones.

13:05 Okay, going to give everyone about 30 seconds. Thanks for your patience. Oh, there are problems with the internet. Oh, no. >> Uh, Gabe, do we have a mic? Because I I can't hear some of the questions. Yeah, the the credit should arrive once you once you sign up in hopefully a minute or two. Um, yeah, we're getting a mic so that we can hopefully walk around and uh get get some of these questions.

13:48 But yeah, I'm going to walk you through building a really simple agent that tells you jokes. Uh, and I'm going to cover some of our core APIs, our concepts first, and then we'll hop over to platform.entthropic.com to build your first agent. Um, so our first step is going to be creating our our environments API, which is v1 environments that helps you create the environment or template for your sandbox.

14:14 So this is going to have uh your networking setup. So do you want uh unrestricted networking or you want to restrict it to specific domains or MCPS? you can define what packages you want installed on your uh container or sandbox. Um so this is going to be like the first step we're going to take to set up our environment. Next you're going to create the agent.

14:37 So agent is essentially uh you're you can give your agent a name. So in this case we have we're going to call it joke teller. Um we're going to choose which model you want to use for this agent. So you can use opus 48, you can use haiku uh or sonnet uh your choice. Uh and then we're going to provide the agent with a system prompt. In this case, we're going to say you're a comedian.

14:58 Find a fresh joke on the topic. Deliver it well. Uh and then you can provide this agent with tools. In this case, we're going to provide it with agent tool set, which is sort of like our basic tool set. This is like your bash tool set, read, write, grap, etc. And you almost all agents will need need this. Um so that's going to be your step two. Step three is going to be finally creating the session.

15:20 Um so session takes an agent ID and an environment ID and creates a session. This is your task uh unit of work or your task. Um and when the session is provisioned it's idle. No work has been done right until you send an event. And then this is sort of what an event stream on your session would look like. Um you're going to have events that you send and then events that the agent will come back to you with.

15:46 On the left side you'll see you have user message. You can say, "Oh, tell me a joke about distributed systems." And then the agent will come back to you with a funny joke. Um, you can use something called a custom tool. Custom tool essentially lets you define a custom tool that you want to handle how you uh how you respond to it and claude will essentially put a custom tool use event on your event stream that you can listen to and then you can process that event and then return the result um to claim tool result.

16:14 And then there are some other examples in there for what type of events you would see. So you'll see session error if your session ever ends up in an error or sees an error. Status idle is when your session is idle, not working. And this is typically when you're not really have a sandbox that you're using. Um and then interrupt on the left is when you want to stop or kill the session and don't want to do the work anymore.

16:36 Uh this is sort of like our full feature list. Maybe if we have time at the end, Gabe can kind of come back to this and dive in. But Gabe will be kind of working with you on his part of the workshop going deeper into some of like these concepts like multi-agent and memory that we also provide. All right, before we go to that one, we're going to hop over now to platform.claw.com.

17:00 Um, and hopefully you can use your API credits and see those. Now, if you don't, I have Travis walking around with the mic in case you have questions or you need to just like flag him down for for anything. Um but if you go to your console you'll see in this manage agent section there is something called as a quick start. This is sort of like our um guided tour of how to build your first agent. [music]