What Is Loop Engineering? Building Better AI Agents Transcript, AI Summary & Key Points

00:00 Loop engineering is a term that's related to working with AI agents, and it's kind of like the the newer, cooler version of prompt engineering. But with prompt engineering, that's all about writing the one, perfect, ultimate prompt and then sending that prompt to a large language model. Then the model comes back with a response and you need to write the next perfect prompt, and so forth.

00:25 Now, loop engineering is the practice of designing a system that does all that prompting work for you. So what loop engineering does is it takes a prompt and sends it to an AI agent. Then loop engineering checks the agent's work and prompts it again. So we go back and forth, back and forth like this, looping until we're done. But we're not having to manually engineer every prompt.

00:58 That's the job of loop engineering. There's minimal human intervention. Although, we'll be coming back to the the word minimal a bit later. Anyway. Let's give an agent a job. Let's say there is a checkout service with a bunch of failing unit tests. And the job is to get the failing unit tests to stop failing. So for that we're gonna use a loop. Now, the loop starts with a goal.

01:31 In this case get the checkout unit tests to pass. Now the AI agent takes an action. That's the next stage in the loop. And for a coding agent, that's something like maybe opening a file or editing some code. Then, the next stage in the loop is the agent observes the output. And the last stop on the loop is to adjust based upon that. So if the test went green, yay!

02:10 That one's done. It moves on to the next failing one. And if the test is still red, then they would try a different fix. And then it kind of goes back around to the goal. Are we done yet? Nope. Okay. Go again. And we keep going around the loop. So, specifying a good goal is quite important because the goal is going to get evaluated every time the agent goes around the loop.

02:37 It's what decides whether the agent keeps going. So, a goal of "make the checkout faster" it's a pretty bad goal in much the same way that "be happier" is a pretty bad New Year's resolution. They both have no measurable test, so the agent could go around forever, or it could stop after just one pass and declare victory, whereas a goal of "stop when every unit test in the checkout sweep passes" is verifiable, so the agent will know for sure when it's met.

03:09 A good goal is specific, with an end condition, a stopping condition. So that's the loop. Basically, these four stages. Not much to it, huh? Doesn't really seem like we need a whole engineering discipline named after it. So, why do we? Well, loop engineering is really what gets built around this loop. So let's take a look at that. So, something has to kick this loop off.

03:35 Now, something that might kick it off is a schedule or something that could kick it off is called a hook. So a schedule is just something simple like run the check out loop every night. And a lot of teams would do that with just like a plain old cron job. And then a hook is event-driven, meaning the hook is invoked when something specific happens, like a file changing or a or a pull request opening.

04:05 Developers largely use hooks for security and qualities such as to enforce policies. Now, every trip around the agentic loop is going to generate something. going to generate context. Stuff like test output would be context or file contents, and context can really start piling up, which can make it hard for an agent to focus its attention on the important stuff.

04:32 So, another engineering context, engineering is choosing which pass actually the loop gets to see. Now, that could mean that we need to do some operations on this context, like compact it. compaction would be one thing we can do. Or we could use the progressive disclosure of agent skills, which contain instructions for how to do something, how a particular recurring job happens.

05:04 Now, inside the loop is something that I'm going to call a spine. And this is really persistent state. So maybe that's some markdown files where the agent writes down what it did and whether it worked. And the information stored in the spine can inform future iterations of the loop on what to do, what worked so far, and maybe what hasn't worked. Now, for this stage here, the action stage, the agent might need to call out to external resources.

05:36 So it would do that using tool access. Now that might be using standards like MCP, the Model Context Protocol. Or maybe would do tool calling directly with APIs. Agents use tools to execute code, to access file systems, to run commands in the terminal, to query databases, all that sort of stuff. That's action. The observe stage, rather than have the same agent that wrote the code, check the code.

06:10 Good loop engineering brings in a second agent. We can think of that second agent as being a sub agent. And that kind of acts as a checker and a coworker to farm tasks out to. And the sub agent has its own instructions and its own context and it may be powered by a different AI model than the main AI agent before sending its results back again. So that's the loop.

06:44 There's a trigger to start the loop. There's context and skills to feed it information. There's a spine to remember what happened. There are tools to let it act on tasks, and there are sub agents to verify the result, which still leaves the question of what the human is doing. While all of this is looping away. All right, so back to minimal human involvement because I said loop engineering requires minimal human involvement, compared to prompt engineering, because an agent writes its own prompts rather than having a

07:22 person submit them manually. But even a well-built loop with a checker and a sub agent and a spine and all the rest of that, it still needs human involvement. A human in the loop approach helps avoid the most dangerous pitfalls that can happen with automated code generation And which pitfalls am I talking about? Well, I've got four for you, starting with unverified code.

07:55 So a checker sub agent is still just an agent. Human developers are ultimately responsible for all shipped code, and that means a real person has to read and verify that code And the gap between the total amount of code in a system and the degree to which humans understand it... that's known as comprehension debt. As agents write more code and humans check less of it, our comprehension, debt is going to keep going up and it can build up pretty fast because, well, agents write a lot of code, and if left unmitigated

08:28 comprehension, debt can be a big problem. When human developers need to address urgent production failures, or humans need to architect a migration. And there's a... there's another debt as well we can put under pitfalls and that is intent debt, an agent can read what a system does, but working out why the system does it can be a bit harder, unless that something has been written down somewhere, perhaps in an agent.md file.

08:58 Without that, an agent can make a change that is technically sound, but goes against a project's objectives. in our checkout loop example, maybe the agent fixes a failing test by, let's say, relaxing the validation on a discount code, which is a perfectly good fix as far as the tests are concerned, but a perfectly bad one as far as the business's bottom line is concerned.

09:21 So the human's job here is to write the intent down, keep up to date and keep it visible to the AI agent. And then, just a fourth pitfall that I want to mention is the rather severe-sounding cognitive surrender. Yeah, we definitely don't want that. But it's very easy to do. Offloading work can lead to cognitive surrender when a person just blindly accepts whatever came out of the loop.

09:47 So if the AI did it, I'm sure it's right. Yeah. When that happens, comprehension debt is also gonna go up because now nobody is even trying to understand the generated AI output. So. Minimal human intervention turns out to be quite a specific amount of human intervention. So, the human, they need to write the goal with the stopping condition. The human needs to write the intent where the agent can find it and the human needs to read the output from the agent.

10:25 Loop engineering works best when there's a human in that loop. Are you engineering agentic loops in your own work? Let me know in the comments!