How to Stop AI from Ruining Your Codebase Transcript, AI Summary & Key Points

00:00 Kent Beck once said, "I'm not a great developer. I'm a good developer with great habits." Now, Kent Beck clearly is a great developer. I mean, he's well-known for his work on JUnit, extreme programming, test-driven development, and he's a valued contributor also to this channel. But, what he says about habits is important, I think. These are the behaviors that we fall back on when we're concentrating on something else.

00:27 The reliable things that we will always do anyway. So, one of these habits or behaviors that Kent was referring to is test-driven development. But, these days that inner loop of detailed development is being done by coding agents. So, what kinds of habits and behaviors do we want to see the agent using? I mean, we really have to avoid the slopocalypse.

00:53 >> [music] >> Hi, I'm Emily Bache. I'm a software developer, creator of Salmon Coaching. Welcome to the Modern Software Engineering channel. If you enjoy the video today, please hit like. Consider subscribing if you aren't already. We are several presenters here with a common goal to bring you great advice on the technical aspects of modern software engineering.

01:15 This video is another episode in our AI briefing series. You can check out the previous ones on the Modern Software Engineering channel homepage. Today, I'm interviewing an expert TDD practitioner and early adopter of agentic AI, Yvette Eddog. Yvette is an independent consultant. She created Lean Developer Experience, which I can recommend. It's a really fun workshop for a team learning about continuous delivery.

01:39 But, today we're going to talk about agentic development and code quality. Recently, I've been talking to quite a few technical coaches, and it seems there's a lot of variety and innovation going on in how to use these agents. Yvette is one of these innovators. And today, I'm going to talk to her about her tool and approach, which she calls Habit Hooks.

02:01 So, can you tell me a little bit about why you created this tool, Habit Hooks? >> Yeah, so basically, what I noticed was that I was repeating the same thing over and over again. And that same thing was, "Hey, this code doesn't look really nice. I don't understand it. Please refactor it." >> So, as Evgeny notes, it's really easy today to create a lot of code with LLM agents like Claude Code or Codex or something, but it doesn't always look very well designed in code.

02:29 And you can imagine if this goes on for a while, it compounds. And you could achieve a slopocalypse, where your code becomes so badly designed it just collapses under the weight of all the technical debt. And the amount of entropy and bad design means that not even the agent can make progress anymore. I hope most people would react and do something before it came to that, but even moderately poorly designed code costs more tokens, is slower for the agent to work with, and makes it more likely the agent will make coding

03:01 errors. I mean, there's research that really shows that. And we all know that bugs can be very expensive and painful. So, we want to get well-designed code from our agents. That's part of what a user-defined agentic harness will do for you. I talked about this idea previously on the channel. Basically, you're using guides and sensors to constrain the agent to go in the design direction that you want it to go in.

03:25 In a moment, we're going to look at a short demo of Evgeny's approach, Habit Hooks. But first, I would like to pause here and thank our sponsors, Equal Experts, Transposit, Octopus Deploy. We are grateful for their support. These are companies that are offering products and services very well aligned with the kinds of topics we discuss on this channel.

03:45 Please click on the links below to find out more. I'd also like to thank all of our Patreon supporters, both for this channel and me personally. We couldn't keep doing these videos without the support of you as well as our sponsors. Okay, so let's look at a demo of Agentic AI doing some design. >> Yes, so basically you can see here we have a on screen a function that is kind of long and hard to understand.

04:09 It's not super long just for the sake of the demo, but the point is that it's it's not nice, right? And when I run the linter, the output I would get from that is something like this. Like it tells us that there are too many lines here, maximum allowed is 12. Good. Now, what did the what does the agent do when it sees that? If the linter fails the build, then it's going to react and I really want to gauge your opinion on the thing that it does by default.

04:40 >> Right, right. So the original function, I mean, you're right, it wasn't that long, but it was clearly kind of, you know, doing a couple of different things and it's now split it into two functions which are each, you know, within the line limits, but it the function name of the build lines and total, what is that? That's not a very good method. It's got an and in the middle of the name, that's a bit of a clue.

05:00 So it's it's like it's just kind of pulled out a chunk of thing in order to meet the the line limits, basically. >> And that's not even the worst of it, there's also my favorite one was when the agent simply cut the function in half and then named the second half of it the same name too. Which was like, at this point this is this is not even helpful in any way.

05:23 And and this is my main problem. If we use the linter as a signal that we need to refactor and then the agent refactors just to appease the linter, then we have a bigger problem than we started with because until now we had a signal that we can use to detect the problem. Now we don't even have the signal anymore. >> Right, this code looks worse and and we can't we can't use the linter to tell us that it looks worse, really.

05:49 Can you just tell us what's what has happened here with the habit hooks? >> Yeah, so basically what habit hooks tool is doing is running a linter with JSON output and then it uses a JSON output to render a prompt that it gives to the agent and that is the prompt that we see here. >> Yeah, so this is like uh three or four paragraphs here about what to do about this code smell, I guess.

06:14 >> Yeah, it basically tells the agent like, "Hey, this is a long function, but it's not just that it's a long function. Long function usually means one of these things. It's either doing too many things or it's just not cohesive." And then it gives some examples on on how to how to fix these kinds of problems. So, it kind of shifts that the agent's attention from number of lines to what is the smell here?

06:37 What is it What is the problem that I'm trying to fix? >> Right, and then we can see here the the refactoring that the agent does given this output from habit hooks. So, we can see that the original summarize function is is shorter now, but it's like broken out three smaller functions. Each is just one line and it looks to me like this code is much more decomposed into pieces that make sense.

07:03 >> This is code that I would be happy to ship. It's pretty good. I haven't seen many code bases that are just like this level of quality. >> A lot of people have noticed that coding agents will do a better job and use fewer tokens when the code they're working on is better designed and decomposed into modules. It's this modularity and separation of concerns that I and people like Dave Farley have been going on about for ages.

07:27 And if that's experience confirms this, what we've got here is an additional tool that's part of your user-defined agentic harness. Bekkes Bakkelaar calls this a sensor. >> And what habit hooks does is basically brings the sensor right next to the the guide. You have the sensor result, this is the part that you need to focus on and the the guide on what to do with this this sensor.

07:50 And I think that's why it works better because it just it's more natural for the agent to react to this. >> Excellent. So, this is the thing, it detects the problem and it gives you the prompt for how to solve it right next to it, the guide right next to the sensor result. >> Basically clumps that information together, which works really well because the agent also tends to clump its attention together and have just one one part of the like if if it has to focus on just the end of the text, it's going to do a much

08:20 better job. >> That's why it works. Excellent. I'm recording this part a few weeks after the original interview because I wanted to add some evidence for what we're saying here. An independent developer from Finland, Leena Suoninen, did an evaluation. And I looked at her study and although it's quite small, it seems well done to me and it shows a really positive effect from a technique like Habit Hook.

08:40 So, what she's done is she's got a a set of 18 Python functions as a test set with two different code smells. So, either it's code that's overly complex or code that swallows all exceptions. And then she compares three different prompts and evaluates the outcome. So, did the AI genuinely fix the code smell or failed to fix it or alternatively it could just game the metric.

09:04 So, then she tests two different models, HiQ and Sonnet, and shows this graph of the results showing the percentage of cases that are properly fixed in each case. So, the control where the prompt just says improve this code without pointing out any specific issue, as she does pretty well, both models are solving the code smell about 30% of the time.

09:23 For the prompt that states the bare metric from the linter only and just says fix it, HiQ is doing better than Sonnet, considerably better than Sonnet. And our hypothesis is basically that the stronger model, Sonnet, is better at gaming the metric, not actually fixing the problem. However, the much better result than both of those, which is properly fixing the code smell over 80% of the time, is the prompt that includes guidance about how to refactor that specific code smell.

09:54 So, I think this is actually pretty good evidence that adding refactoring guidance to your linter outputs is worth doing, which is exactly what Yvette is saying. I hope you enjoyed listening to this interview with Yvette as much as I did talking to her. Yvette's given us a concrete approach that will let your agent gain better coding habits, just like Kent Beck recommended. And I think we need all the help we can get if we're going to avoid the slop apocalypse. Happy coding. >> [music]