Agentic Engineering: 5 Terms Every AI Developer Should Know Transcript, AI Summary & Key Points

00:00 Imagine being a software developer and not having to write any code. Boy, does that sound nice. Well, on popular search engine and music streaming teams, that's the reality with 80%, or more of their code being written by AI and these new AI coding assistants. I mean, that's just incredible, but it's the realty with the rise of these AI-assisted development practices and what's known as agentic engineering.

00:32 But the question is, how does it work? And what should you know? Well, in the next few minutes, I wanna explain the five most important terms that you should know for building and using AI models, with examples and best practices so you can leverage AI just like those top teams do. So, let's start with the basics and work our way up. You've likely heard about prompt engineering.

00:59 That is the practice of crafting the right instructions for a model to follow. So, what type of instructions would this be? Well, this could be, hey, you're a technical support specialist, or maybe something like only answer based on the provided documentation. But the thing is what you might not know is that even if you have an LLM that is extremely capable, I'm talking the most frontier, you know, best rated model.

01:30 Well, knowing how to communicate, to be able to explain what you want from the model and what the model should be doing for you is more important than the model itself. And with that ability to communicate, is the need for domain expertise. So domain expertise, in whatever the field might be. A lot of people think, oh, I need to be a developer to use these models.

01:55 Not necessarily, but you do need to be experienced in what you're working with the model to do. And some people, in order to communicate properly while they're doing prompt engineering, is uh, use their microphone. So they might talk for several minutes with one of these LLMs using dictation. And it works spectacularly, because these advanced models are really capable of picking up what you're trying to do.

02:21 Now, let's talk about smaller models, models that you can run locally or on the edge, even on your mobile phone today. So, what we typically see for prompt engineering with these smaller models is providing examples of few-shot prompts, so questions and answers that you would like to see the model also replicate. And so you give these examples, known as few-shot prompting, with the prompt that you want to give to the model.

02:51 So hey, I want you to solve this or take this new user's information and give us a response, and even provide what's known as chain of thought examples. So COT, chain of thoughts, that make a really big difference when you're getting a response from one of these smaller sized LLMs that you could even run on your laptop or a smartphone today. So, based on what type of model you're aiming to use, your instructions might be a little bit different, but at the end of the day, my friends, it's all about communication.

03:22 But the model doesn't just see the instructions you give it, and that is where context engineering comes into play. So, I want you to think about everything that the model is seeing in its context window, right? This is going to include the system prompt, or, hey, this is what the model is supposed to be doing for this use case. Uhm, this could also include, for example, tools and MCP servers and skills that are loaded in.

03:48 It also might be PDFs, information that you're pulling in from retrieval augmented generation to answer questions about uhm, this subject, for example. Or, for example, with coding agents, you're going to see your code base, added to the context window. And the thing is, even if you have a model with a huuuuge context window, right? The thing is quality is going to degrade even before you hit that advertised limit.

04:16 So, this is where you see techniques here like compacting or summarizing the contents of your context window into a new and fresh prompt to the model. So here, for example, we could summarize some of the activities that happen in the conversation with the model, in a smaller portion like this, and hand that off to a new fresh prompt, to the model. So, these are just ways of managing your context window.

04:45 You'll also commonly see sub-agents being used, right? Sub-agents allow us to do things like research, or check out a library, or documentation, without adding much to our original context window, and once these sub-agents, which are typically smaller, and more purposeful agents, uhm, finish, they can add that information back into the context window and save us a little bit of storage, as we're doing our work.

05:14 And this is what context engineering is all about. But even with a well-managed context window, you're probably noticing that there's something missing, because with all that relevant information, well, this incredibly capable LLM is gonna be able to output a very accurate answer. But how do we know that that answer is actually going to work? That, my friends, is the entire idea of harness engineering.

05:41 And harness engineering is everything around the model that lets it act and verify, then what it did, and continue to get that output and input back. It's the runtime that actually executes this command, that you want to say edit files by using tools, right? But also to manage the context window. Let's say if we overload that context, uhmm, window that we originally had, how do we, you know continue to have the same memory and the same prompt and the same execution.

06:12 At the same time, we want to have a sandbox because we don't want the LLM to be able to do everything to our computer and our machine. And the harness can provide these capabilities, as well as orchestration. So, orchestrate those different sub agents that we mentioned earlier. So, the harness is that capability that allows us to interact with our own machine, but also to tell the model, hey, for example when we use that tool, did the test suite pass after the change you recommended?

06:42 Or, say for example, do we have a merge conflict now that we're trying to add that code back into the repository? So, this is the feedback that lets an agent iterate on its own, instead of you having to approve every step. And most AI harnesses out there, already connect with tools and utilities like your CI pipeline, your observability stack, and all the components that you already have.

07:06 Now, you might have also heard of loops and loop engineering, for example, and the best use case I can show you is the agent loop itself. So, from an initial goal, or a lot of people just use prompts, this allows the agent to act using various types of tools and be able to observe that output, and at that point, decide if that's a success or to continue in that loop.

07:34 And my friends, that is all essentially an AI agent is, is just this loop continuing on and on. But then, there's the idea of loop engineering, where instead of a human like you or me continually prompting this agent, well, we attach a trigger. And, this could be any type of trigger. This could be every morning at 9 a.m, that's, that's what's known as a cron job, it could be someone opening a poor request in the repository, it could be an issue getting filed about a bug or an alert.

08:10 But, loop engineering just means that what we're building is a system that orchestrates the agents for you, with that intended goal. Oh, and typically, there's a human in the loop to approve these changes that happen. So you could have a either perhaps after the agent has started to act and use tools, or after it's observed its output so you can also check what's happening in this workflow and work alongside an agent to ship software more efficiently, more high quality and faster.

08:41 Now, you have several agents that can do this work and the concept of graph engineering. So, graph engineering is where we take this same loop concept, but we tie in parallel multiple agents to each other. So, from an intended goal or start or trigger, we can have two sub-agents, one that might do market research, uhm, one that is figuring out how our repository is structured, and they can pass off their work to an agent that's actually going to implement.

09:14 And so now we can take this loop idea and branch out into this kind of various, uhm, kind of graph structure. But the thing is, unless you have unlimited tokens, you can see how this implementation would get a little bit pricey, so you might not always be doing this, but this is the concept of loop engineering and graph engineering. Finally, is what's known as a software factory.

09:37 This is all about automating the software development lifecycle using AI agents. So, essentially setting up robust pipelines for traditional software development tasks, like task management, or code implementation, or testing, whatever it might be. And instead of having just one general purpose agent that can try to handle everything, we use this software factory with individual task-based agents.

10:08 So each of these though need components such as isolation. So isolation using whatever might be containers or sandboxes. Each of these agents also need identity. So, who are they and what can they access? So this could be with Git or with our cloud environment. And each of these agents, as they're doing work, also need observability. So, if they were up working at 3 a.m last night, we should be able to trace everything that happened by one of these agents in the factory.

10:39 So, a lot of teams are starting to build out these AI software factories. And as models become better, I see it only becoming more common. So five terms, and if you've been working as an engineer, you're gonna notice that these aren't exactly new ideas. Graphs, feedback loops, distributed systems, these are all computing practices, but they're now being reinvented.

11:03 And here's the thing though. You don't have to be an expert in all of this. If you're just a solo developer, you don't need a software factory. But concepts like the prompt, the context window and harness engineering, well ... that's applicable to everybody. It doesn't matter if you're a student, designer, a manager, you name it. But please let me know in the comments, is this what the future of engineering looks like?

11:26 Or what works for you? As always, I hope you learned something today from today's video. And please leave a like to hack the algorithm. Cheers, and we'll see you in the next one.