How I Built a Server That Runs AI Agents 24/7 (Full Setup) Transcript, AI Summary & Key Points

00:18 If you started using AI coding agents, not if, you have, right? And you probably fell into the pull to run more than one. You have several going at once in parallel, each chewing through a different task while you orchestrate the lot. That's the goal we're working toward. But the moment you reach for it, you run into a handful of problems, and solving them is what this whole video is about.

00:40 The first is persistence. Those agents run on a machine, and machines sleep, reboot, lose power. The instant that happens, every agent stops dead, and our software can vanish with them. The second is accessibility. The agents run wherever they run, but we're not always sitting right next to them. You close the laptop at home, you're working from an airport cafe an hour later, you're over an ocean by nightfall, and through all of it, you still want to reach them, to check in, to redirect them.

01:10 Those two are the big ones. There are also a couple of bonus problems, the kind that aren't deal breakers necessarily on their own, but quietly making everything worse. One is dedication. If the agents are grinding through builds and tests on every machine you're trying to work on, everything ends up fighting over the same CPU and RAM. You and your agents elbowing each other for resources.

01:32 The other is isolation. Agents execute code. They run commands. They install things, sometimes things you'd never install yourself. Keeping all of that well away from your daily driver machine limits the blast radius when something inevitably goes sideways. So, how did we end up wanting a fleet of agents in the first place. It's worth retracing the steps very quickly because where we're going only makes sense once you've seen the path that got us there.

02:07 It started with chat. We would open something like chat GPT, describe what we wanted, and get back a block of code. The intelligence was there, right? But the plumbing was missing. The model couldn't run anything, and it couldn't reach into our files. So, we became human clipboards, copying code out of chat, pasting it into the editor, running it, copying the error back, round and round and round.

02:30 Then came magnetic loops, and that changed everything. The agent could finally act on its own, read our files, edit them, run commands in the terminal, look at the output, and try again. But we didn't trust it yet. So, most of us run a single agent and watched it like a hawk, right? Approving every single action before it happened. Now, as that trust grew, we loosened the leash, and something clicked.

02:57 Once you're no longer babysitting one agent, you can run several. You start the first and let it rip, move to the second, give it the task, then the third, the fourth, and by the time you've briefed the last one, the first has usually finished. It's ready for you to review, redirect, and set off again. And it kept escalating. We started handing agents tasks big enough to take hours, sometimes even days, and experimenting with whole swarms of them attacking the same problem together.

03:27 Now, let me be honest that frontier is messy. It is. Coordinating a pile of autonomous agents is genuinely hard. They trip over each other. They make conflicting assumptions, and plenty of people are rightly, rightly skeptical that swarms are ready for prime time. But the direction is unmistakable, and it leads to one stubbornly practical question. Once you want to run several agents at once, some of them for hours or even days of a stretch, where and how would they actually run?

04:02 Almost everything else, everything else hinges on that one decision. So, let's work through it option by option. The obvious first thought is the cloud, right? Spin up an instance, done. The problem is what that instance costs because we want those agents working, which means the machine runs around the clock at a real spec. Price out something with, let's say, eight cores and 32 GB of RAM on AWS and the computer alone, only computer is around $300 a month.

04:35 Add a terabyte of storage on top of all that and you're knocking on $400 a month, which works out to the better part of $5,000 a year for a single machine. $5,000, 1 year, one machine to run agents. Now, AWS is pricey end of the pool. Drop down to a leaner provider like, let's say, UpCloud and that same eight cores and 32 gigs costs you a little over $200 a month, roughly half for hardware doing the exact same job.

05:10 Now, to be honest, for the amount of work a fleet of agents can churn through, even that is perfectly perfectly reasonable price to pay. It's cheap compared to the time it saves. But, here's the thing. We can do far, far, far better than that. So, let's see that. Now, what about a real server right in your own data center? If you got one, sure, go right ahead, right?

05:35 But, let's be honest, most of us don't have a rack sitting in a basement somewhere with our name on it. We don't. Then, Mac, right? I'm a Mac person through and through. In this house In this house alone, there's a iMac and Mac mini, a Mac Studio, four MacBook Pros, maybe something else. So, believe me when I tell you for something that just sits in a corner as a server you connect to, rather than a machine you actually sit in front of and use, a Mac is simply too expensive for the job.

06:11 It's amazing hardware wasted on a role that never sees a screen. And that leaves us the PC. Unless you genuinely believe a mainframe is on the table, a plain PC is the last option standing, and it turns out to be exactly the right one. A small, boring, unassuming box tucked away somewhere, somewhere out of sight. And here's the good news. It does not need to be powerful.

06:35 No GPU, you don't need it. The model lives in someone else's data center. We're not running inference locally, at least not in this scenario. You don't need a mountain of RAM or some monster CPU either. Even if you end up running dozens dozens of agents, they're rarely all grinding at the same time. The agents themselves are never the bottleneck. The work they kick off, writing code, building, running tests, is what actually costs something, and that work rarely lands all at once.

07:02 32 GB of RAM and a decent, unremarkable CPU will be plenty. Now, you can go lower if you want if if your budget is tight, you can go below that. The one exception, naturally, uh that's Java. If you're working with Java, none of that is enough. Just disregard everything I said, right? For everybody else, that's enough. Now, think of it the way we think about people.

07:26 Every engineer on the team gets their own laptop, right? Nobody is expected to share. Now, we're not going to go quite that far and hand each agent its own machine, but the principle holds. The agents get the machine of their own to share instead of squatting on my machine. They stop fighting my daily driver for CPU and RAM, and everything, everything, everything, everything they run, everything they install, every bit of damage they might do, happens nowhere near the laptop I actually depend on.

07:56 So, though that's two of the problems we opened with dedication and isolation, and that's taken care of just by deciding where the agents live. And that's where that earlier thought pays off. Remember the cloud bill, right? Somewhere between $2,500 a year on upcloud, and the better part of 5,000 on AWS, right? The machine I actually bought for this, a small box with exactly those specs, cost me around $900, once.

08:24 It pays for itself against the cloud in a matter of months. And after that, it just sits there. It It's just sitting quietly being mine. It's mine. Now, if you're as steeped in cloud native as I am, your instinct right now is to reach out for Kubernetes, right? Now, resist it. Resist it. Don't do it for this. The most important reason comes down to what these agents actually do once they're running.

08:50 Agents don't just write code, they run it. They run integration tests, they run end-to-end suits, and a huge amount of that work means spinning up containers, sometimes a whole throwaway cluster with something like kind, right there, in the middle of the task. Inside a pod, that means running containers inside a container. Docker in Docker. And Docker in Docker needs privileged mode, which effectively hands the workload the keys to the host.

09:17 Modern Kubernetes really does not want you to do that. Don't. Pod security standards block privileged containers in everything but the wide open profile, and just about every managed or hardened cluster forbids it outright. So, the moment the moment an agent needs to run a container, the pod approach quietly, or not so quietly just falls apart. Now, that alone would be enough to walk away from Kubernetes, but it isn't the only problem.

09:44 Kubernetes also wants you to declare CPU and memory up front requests and limits, and an agent makes a mockery of that. In a single session, it goes from idling while it waits for the model to pinning the CPU through a build to chewing through memory on a test suit and so on and so forth. There's no honest number to write down. Set it low and you get throttled or killed mid-task.

10:05 Set it high and you're hoarding capacity that sits idle most of the day. And pods were never never never built to live for hours or days in a first place. They get evicted, drained, preempted. A pod is never really moved. It's destroyed and replaced with no promise that your working state comes along for the ride. On a single server with a normal operating system, every one of these problems simply evaporates.

10:32 Docker just runs, kind just runs, the kernel time shares the CPU and hands out memory on demand the way it has for decades. Now, to be fair, the ecosystem knows this is a gap. A Kubernetes effort called agent sandbox is trying to fix it with stable identity, suspend and resume, state backups, warm pools, and so on and so forth. It's it's very very promising, but still very green.

10:56 And notice what it really is, a pile of machinery to make a pod behave like a single long-lived stateful machine. Which is to say, like the boring little server we already picked, right? The mini PC. So, the machine is settled, a humble little PC. Now, what runs on it? Not macOS. We already threw out Mac on cost, and the operating system leaves with the the hardware.

11:24 Hardware is gone, operating system is gone. Windows? Why on earth would we use Windows? Practically nobody runs Windows servers anymore, at least not by choice. Even on Azure, Microsoft's own cloud, the supposed home turf for Windows servers, there are far more Linux machines running than Windows ones. And that tells you everything you need to know.

11:45 That leaves us with Linux, right? Bingo. And honestly, any distribution will do. I went with Ubuntu, mostly out of laziness. It's what I know, and it's been ages since I last bothered to shop around, right? Well, what whatever you pick, grab the server edition, the version without a desktop environment, because we don't want a graphical interface anywhere, anywhere near this thing.

12:06 You don't need it. Forget about GUIs. And to hammer that last point home, no graphical user interface. Again, none. It's a server. We connect to it. We don't sit in front of it clicking buttons. So, with the machine and the operating system sorted, we get to the heart of it, the agents themselves, and the form they take matters far more than you would expect.

12:28 You can have a desktop app, something like Cloud Desktop, right? Please don't. Please, please, please, please, please do not. You're going to want many, many, many, many agents running at once, and desktop apps simply aren't built to run as a dozen parallel instances. And the resource cost is offensive. Cloud Desktop sits at over a gigabyte a gigabyte of RAM doing precisely nothing.

12:53 Before you even type a single word, you lost 1 gig of RAM. That over there, that's Electron for you, one of the worst things worst things that happened to our industry, right? But but even setting all that aside, it's a server for agents. We don't run graphical desktop apps on servers. Well, unless you're a Windows server person, in which case you should probably change your profession and become a farmer or a lawyer.

13:16 Both are likely going to be easier for you. Then we have IDEs, something like Visual Studio Code with an agent bolted on, right? Same story, for all the same reasons. It's a graphical Electron app. It's heavy. It's built to be set in front of on a machine with a screen. You're not about to run 20 copies of that headless on a box in the corner, right?

13:36 It's simply the wrong shape for the job. And then we have a terminal agent. A TUI like Cloud Code and Code X and Open Code. And now we're talking, right? We can spin up as many of these in parallel as we'd like without breaking a sweat. They barely touch the CPU. Now, memory is another story. Each one cheerfully comes on a few hundred megabits of RAM because, of course, the people building them decided to write them in JavaScript and TypeScript, right?

14:05 I have nothing against those languages. I use them myself, but for a terminal app, come on. And yet here we are. Most of them are built in exactly that. It is what it is. Still the best choice. So, with the agents chosen, what else actually needs to be on the box? Less than you think, actually. Linux already ships with most of what an agent reaches for.

14:27 It's all just there in Bash. A good chunk of the tooling problem solves itself the moment you pick a sensible operating system. Then we have agents themselves. Obviously, that's the whole point of this exercise. And we should have a container runtime, Docker or whatever you prefer because, as we already established, a lot of the work A lot of the work involves building and running containers.

14:50 And then there is just one more piece. One more piece. Ready? Devbox. Unless Unless you've gone all in on NixOS, in which case power to you and you already got it covered. So, why Devbox? Because instead of trying to figure out up front every single tool that every single project you work on now or might work on later could possibly need, you drop a devbox.json file into a project, you run devbox shell, and every tool that project depends on is suddenly right there.

15:23 There's no juggling five versions of the same tool across five different projects. It's easy to manage, properly isolated, the whole package. It generally great. Now, I went deep on Devbox in a separate video. I had It's somewhere there on the screen, check it out. So, if any of this is new to you, go and check it out. Now, agents also need secrets.

15:43 They need API keys and tokens and credentials for whatever they touch. For that, there's Vault. The beauty of Vault is that it talks to almost any secret store you can name. Vault, AWS, Google, uh Azure Key Vault, in physical, one password, SOPS, dozens of them. So, wherever your secrets already live, Vault can pull them in on demand without you ever ever ever copying them onto the server in plain text.

16:09 So, right. The box is built. Now, how do we actually reach it? Now, remember the accessibility problem from the start? The agents run on that PC in the corner somewhere, but we're not always in that same corner. We're on a different laptop, in a different room, sometimes in a different country. The answer is, of course, SSH. But, here it comes, tunneled through Tailscale.

16:34 Tailscale stitches all your machines into one private network, no matter where they physically sit, and SSH gets your shell on the server from any of them. From a cafe, from a hotel room, from the other side of the planet. It's the same single command, and it just works. There's a catch, though, and it goes straight back to the persistence problem. If we SSH in, fire up Cloud Code or Code X or Open Code, and then close the laptop or lose the connection, the session dies with it.

17:02 The agent's process gets killed the instant we disconnect. All that work, gone. Which is exactly the nightmare we set out to avoid in the first place. The fix to all that is a terminal multiplexer, tmux or zellij. Pick whichever you want. And because a single multiplexer session can be split into a grid of panes, you're not limited to one agent. You spin up a whole fleet side by side, hand each its own task, and watch them all work at once.

17:34 The session lives on the server itself, completely independent of whether we happen to be connected. We attach, we watch the agents work, we detach, we close the laptop, we fly across an ocean, and then we come back and reattach. Everything is still right where we left it. The session never even knew we were gone. Okay, so far we've been talking about running agents in parallel as though it just happens.

17:57 But it's worth pausing on how we actually coordinate the whole fleet of all of them. Cloud Code, for example, recently grew its own multi-agent orchestration, where one agent can spin up and direct a team of others. And it's a start. But because real problems is too simplistic, it lacks visibility into what the agents are actually doing, and the list goes on.

18:20 Honestly, picking that apart properly properly would take a whole video on its own. So So, I will leave it at this. It's early days, it's not there yet, and we are still waiting to see where it goes. There's a better fit for this particular setup though, and I should be up front about it, right? It's my own pet project, it's called Agent Deck. I built it to scratch my own itch, and it works for me.

18:45 Right now, it drives Cloud Code and Open Code, with more to come. It gives you one place one place to see every session status, running, waiting, or idle, and to orchestrate the whole fleet from a single view. And here's the neat part. Because Agent Deck runs as a daemon and keeps those sessions alive itself, you don't even need the multiplexer we set up a moment ago.

19:05 It handles that for you. It can also drive agents on remote machines, but that's a rabbit hole worth its own video, right? So, I'll save it for another day. Anyways, agents become dramatically more useful when they can reach beyond the box and talk to the outside world, and for that, we lean on well, obviously, MCP, the model context protocol. Remote MCP servers, remote, specifically, not local ones, right?

19:30 Running an MCP locally on a remote box you've already SSH'd into would be a bit silly. For local execution, skills do the job. Although, honestly, even a remote MCP can be overkill. If a well-known CLI like GH or GitHub CLI already talks to the service you need, just let the agent use the CLI. Just do it. Don't reach for an MCP server when a command you already trust does the job perfectly well.

19:55 And finally, finally, notifications. When an agent finishes a task or gets stuck or needs a human, we want to know about it, wherever we happen to be. And again, no need to reinvent anything, nothing. MCP already has us covered. There are MCP servers for Slack, for, let's say, Teams, if if your company has inflicted that one on you, right? If it's a punishment.

20:17 Or for Google Chat, if if it hasn't yet joined the long list of things Google has quietly taken out back somewhere and shot, right? Anyways, the point is that there are MCP servers for all types of communication devices or tools or services or whatever it is. You wire one up and your agents can reach you on whatever channel you already live in. But, here's the line I draw, right?

20:43 And and I draw it very firmly. Notifications should mostly flow in one direction, outward, to you. The chat is there to say, "Hey, hey, hey, this is done, go take a look." Or, "Hey, I'm stuck. I need you." What I'm far less keen on is the reverse, actually, trying to talk to your agents through Slack. And plenty of people are betting hard on exactly that right now.

21:07 There are agents built to live in Slack first, and even Anthropic has wired Claude code into it for a quick yes, go ahead on some small task. Sure, fine. It's It's okay. But for real work, for steering an agent, reading through a diff properly, course correcting halfway through, a cramped little chat window is a miserable place to do any of that. Dropping straight into Claude code in the terminal is infinitely better.

21:36 So, I treat chat as the doorbell, not the workshop. It tells me something needs my attention, and then I go to the proper interface and actually get to work. So, now is the right moment to come clean about something, right? Way back when we picked the agents themselves, terminal tools or desktop apps and IDEs, I deliberately skipped over an entire category, and this is exactly why.

22:02 There's a class of tools, Open Claude being the obvious example, that flip the whole model around. You don't sit in front of them at all. They run as a gateway on your machine or server, and your interface is quite literally a chat app like Telegram, WhatsApp, Signal, Slack, whatever you're already carrying in your pocket. You fire off a task from your phone, approve an action, read back the result, all from a thread.

22:25 It's generally clever, and it's about the purest expression of the chat-first bet you will ever find. But not where this really shines and where it stops. Open Claude is brilliant at exactly what I said chat is brilliant at, dispatching something quick and checking in on it from anywhere. And it stops where chat stops. Nobody's reading a 500-line diff or steering a delicate refactor through a Telegram message.

22:50 So, even the boldest chat-first tool in the room lands on the very same line I just drew. A fantastic doorbell, a handy remote, but not the workshop. And that's the whole stack, a boring little PC, Linux, a handful of terminal agents, the tools to keep them fed, secrets pulled in safely, a private tunnel to reach them from anywhere, and sessions that flat out refuse to die when we walk away.

23:18 Every problem we opened with, persistence, accessibility, dedication, isolation, all answered, and for the price of a machine that pays for itself in a few months. That's a server for agents. If you end up building your own, tell me how it went in the comments. I'm generally curious what setups people land on and what you would do differently. And if this was useful, you know what to do, right?