Does Code Quality Still Matter? It Depends on This — Max Kanat-Alexander, Capital One Transcript, AI Summary & Key Points

00:01 [music] >> So, today I figured I'd just pick a nice uncontroversial topic that nobody has any opinions about. Right? Nobody nobody said anything on about on this about social media, you know, and of course every take about this on social media is like really deeply nuanced, you know, like people have lots of like depth and like, you know, really big thoughts.

00:31 You know, maybe there's a little bit more to it than like yes or no. And you know, of course if you ever ask a senior engineer a question, we always have the same answer to every question, every senior engineer's favorite answer to every question, which is it depends. Now, that's not a very helpful answer though, frankly, because what does it depend on?

00:55 So, let's start with this. Why did code quality ever matter in the first place? Like I see a lot of people be like, well, it made me feel good. That's not really a sufficient justification. So, let's see if we can inject some nuance into this conversation and help everybody at least be able to have a basis for reasoning about it while we're discussing it.

01:11 So, let's see if we can make a framework here and that's what we're going to try to do with this talk. Okay. So, we're going to start off with everything you want to ever accomplish with a piece of software uh depends on two factors and two factors alone. And that is understanding and executing on that understanding, whether that's aligning a model, scaling a system, driving adoption, anything.

01:30 All software success is determined by these two factors. This is why we have things like debuggers and observability and evals and even tests. All those things actually are there to improve your understanding or ability to execute on that understanding. People think that they're there for other reasons, but fundamentally they're there for that reason.

01:48 Everything we have ever done in the field of code quality is exclusively and only for these two things. It's not for anything else. Not doesn't have something to do with correctness or something else. It has to do only with these two things. Now, there's two understandings that are the most important to have if you're going to be a software engineer.

02:05 There are other understandings to have about software if you're going to be a product, if you're going to be in sales, but if you're a software engineer, there's the two that are the most important. And the first is what is the behavior of the system? Like con- the concretely, not what does a doc say the behavior of the system is, but what is the actual behavior of the system?

02:22 And the second one is how do I correctly and safely modify that behavior, okay? On a very small system, if you want to understand what the system does, I just tell you what it does. It prints hello world. You say, "I want it to say hi world." I say, "Okay, I will make it say hi world now." There's very little to understand about it other than purely what it does.

02:45 That's all of the understanding that you need. However, any real software system is enormously complex. This is a real map of the code base of Chromium, which is the open-source project behind Chrome. And if you think about all of the different ways that these components can interact, they have essentially infinite possible behaviors. Like, I could not describe to you in words the behavior of Chromium as a whole as a system.

03:17 However, what I can describe to you, and as we all know, everybody's been a software engineer for a while, I can describe to you components of the system. I can successfully describe to you how a component of the system behaves. So, this is the URL parser, the actual functions and stuff for the URL parser of Chromium. It figures out all the different parts of the URL.

03:37 And this is the whole reason that code exists. The whole reason that we have code at all in the first place is for structure. It Because it What do I mean by that? Okay. So, both of these two things on the left and on the right will print Hello World if run on a Linux machine. However, on the left, you can see that it is Hello World. It has the identity and structure of Hello World.

04:07 On the right, you just see a series of instructions with no particular identity, recognizable or understandable identity. It's not a component, it's just a bunch of instructions. While that's understandable there, if that was millions or billions of lines of that, it would be much harder to grok. In the past, we've talked about things like readability or testability or things that seemed like human concepts about code, but what actually mattered the most always was that the system had some sort of understandable

04:37 structure. So, it's the structure that actually matters. And when this modifies when this matters the most is when you want to modify the system, the second uh understanding that we talked about. All modifications to a system carry two categories of risk. The first is that you will introduce some sort of problem. And a problem isn't just a bug, it's anything bad that could happen.

04:59 Like your project could take much longer than you expect. Uh it could not behave in the way that you wanted it to for your users. The project could fail because you have budget overruns. Any kind of problem could happen from just you trying to go through the process of making this change. And the second is you could make it harder to make future changes.

05:18 In particular, what that means is you can make it more likely to have future problems. You could increase future risk of the system. These two things are caused by under problems with understanding. So, the first has to do with the difficulty of understanding the system in the first place, and the second is you make the system harder to understand. I can reduce every postmortem I have ever read for any incident to these two factors in one way or The the production configuration was too hard to understand, somebody made

05:50 something impossible to understand, there were emergent properties of the system that were impossible to understand, whatever. Point two feeds into point one in a vicious cycle that compounds in a non-linear fashion. So, the difficulty of understanding does not grow linearly, it compounds over time. You will eventually reach a point, and all systems eventually tend to reach this point, where every modification involves some sort of problem.

06:21 Uh and you know, some some systems get to the point where like you can't modify them at like if you think of like, you know, some of like your giant 100 million 500 million line mainframe code bases and that everything, every change is some sort of problem. You know, you'll see it in organizational issues, you'll see it all over the place, you'll see it in timelines, whatever.

06:41 And look, there's a lot of arguments about maybe future models will fix this. Look, maybe they will. Maybe future models will have infinite ability to deal with complexity somehow. Uh but we don't have to discuss that today because I think models are doing something else really interesting right now. They are creating a new kind of software, a new category of software.

06:59 This category has always existed, but it's about to become a hundred thousand times more common than any other form of software. And that is disposable software. So, what is that? Okay. So, imagine a world in which the consequences of failure for your system are very minimal or non-existent. Imagine that there's very few people that depend on it or and almost no other systems depend on that piece of software.

07:31 Change management, very minimal, non-existent. Imagine the cost of replacement went to near zero. You could replace the whole thing in hours or days. And it had basically no cost of maintenance. In that world, code quality does not matter. Because it all of those things we were trying to prevent about the compounding risks over time don't exist. Traditionally, we've always built durable software.

08:04 You know, it lives for years. It's depended on by millions or thousands of people. There's huge consequences of failure, potentially business catastrophic consequences of failure. And it requires enormous maintenance. Most of the software engineers in the world spend most of their time maintaining these giant durable systems. I expect this to continue to exist.

08:21 So, this is one of the areas where I think sometimes the conversation goes a little astray, okay? Durable systems will continue to probably be the most important systems in the world. And there will be probably more of them. But the most numerous systems will be disposable software. They will be replaceable in hours. They will be depended on by a few or change management just won't matter.

08:50 And look, if it doesn't work, you will give everybody agents. They'll make a version of it themselves. You know, maintenance will be unimportant or nearly free. Don't like the calculator on your OS? Make your own. Want to plan a party for your kids? Write a party planner software just for that one party. Throw it away. Code quality doesn't matter. Nobody cares.

09:12 In the future, we will apply engineering rigor to the degree of disposability of the software. On the left, nothing. None of our practices of engineering rigor matter at all. On the right, all the practices of engineering that we have built up over the last many decades matter greatly. We can determine where you are on this scale by those four factors that I have been talking about.

09:36 Consequences, cost of replacement, dependencies, and cost of maintenance. If they're all low, it's disposable software. If they're all high, it's durable software. And look, any person who's written software for a long time knows you should err on the side of it being durable. You should err on these all being higher than you expect them to be. But, there's also a new thing you can do in the world.

10:00 You can make software more disposable. So, like, what do What does that involve? Well, come up with better isolated environments, right? So, hey, you can only deploy to this environment that has access to only your data. No customer data, no personally identifiable information, whatever. You can enforce independence. You can say, "Okay, no inbound API calls from other production systems are allowed to this system in this environment, right?"

10:26 And so, everything deployed to that environment is disposable. Redundancy. You can have an old clunky backup system that is like hard to use. And then, like, look, if people want to vibe code their own like more efficient front ends, you let them vibe code their more own more efficient front ends because there is an old durable clunky backup system that they can use if their, you know, crappy little vibe coded thing fails.

10:47 Which Which it will fail, right? That's fine. But, it doesn't matter if it fails. Uh and you can limit scope, just like I talked about, right? It's just a calculator for this one person. It just plans this one party. There's so much more we could do here and talk about ways of increasing disposability. In fact, I think this is a whole new discipline of software engineering, not to think about how we improve the rigor of durable systems, but how do we take our systems and how do we encourage disposability or allow more

11:18 disposability, allow people to throw these things away. Interesting thought. Now, I want to re-emphasize that durable software engineering is not going away because this is the biggest confusion that I see on social media today. In fact, I believe the absolute amount of durable software in the world will increase because AI coding tools make it easier and easier to make more durable software.

11:44 And another important thing to understand is that disposable and durable are not black and white. There's a spectrum, like I showed before, but that spectrum what changes across the spectrum is how much effort you're going to put into engineering rigor. Okay? And that's both immediate effort and effort over time, just like we've always talked about for software.

12:05 However, what you want to do, and I think what you want to care about the most, is reducing the cost of effort, reducing the cost of rigor. Like, if rigor is cheaper, if it's the the easier and easier you make it to be rigor, the less you have to worry about where you are on the scale. So, this is another approach you can take. If you change your mind, it's much easier to go like you're like, "Oh, I thought this was going to be disposable and now it needs to be durable."

12:31 The cheaper you make it to make it durable, the easier it is to change your mind or to be wrong because you're never going to be 100% right all the time on that estimate. So, that involves a different set of practices that are not engineering rigor themselves, but that are about how to decrease the cost of engineering rigor. Now, I could talk about this for hours.

12:54 I have spent a tremendous amount of my career doing this. Uh I don't have 10 hours. I have about 5 minutes left on that timer. So, I'll tell I'll talk about what I can, okay? Let's talk about standardization. Why do we do standardization? People are like, "Well, I don't know because the boss said so." Well, now, we do standardization because it removes possibilities.

13:17 If you don't have any PHP in your system, you don't have to figure out what are all the linters for PHP, what are all the practices, what are all the security holes I have to fix. You just eliminate possibilities through standardization and it just reduces the cost of rigor. Uh same thing with like frameworks. Like why don't you want to have every front-end framework or every back-end framework in the in existence?

13:36 Because it makes rigor more expensive. Like yes, rigor is still possible. Yes, you could still have the agent code up all your tools, but you have to do it every single time. Uh you know, proto- uh protocols. Why do you want to have standardized RPC protocols? All the same thing. Standards are all for this reason. Okay? Uh what I call inward flow tooling.

13:54 So, inward flow tooling is tooling that just occurs. The programmer doesn't have to choose to run it. They don't have to understand how it runs. It just happens. If it's there, it it's feels and seems free and it is reliable. Reliability is very, very important. Like it doesn't have a ton of false positives. It's fast. It just happens. And the more that's true, like you can apply it to your disposable software, too.

14:15 If it's free and fast, who cares? Uh making testing easier. So, this is not just about like, "Hey, how do we do better testing?" Look, how do you make it so that nobody has to think about how to do better testing? How do you make it so that every test is fast and deterministic without anybody having to be a deep expert in testing? There's a ton of work that you have to put into test infrastructure to make that possible.

14:40 For example, you know, one of the great dreams that I've always had as a as a test owner is that everybody would provide in-memory fake versions of their services and I could just run those in-memory fake versions of their services. Guess what? The coding agent can do that. So, you can just do that now, right? But you have to choose to do it and make it available to everybody.

14:58 And another thing is, look, maximum strictness. You just turn the strictness up everywhere. You turn You set all your lint rules to maximum. You You never compromise on anything and it just eliminates It's very similar to standardization. It eliminates a ton of conversations. Sometimes the lint is wrong. Like as long as the false positive rate is relatively low and it doesn't have catastrophic consequences, it doesn't matter because all of a sudden there's a bunch of future lint errors or future issues that you don't

15:22 have to deal with. Like, ask anybody who's been like, "Oh, I suddenly want to turn on my linter in this million-line code base." And they're like, "Oh, wow. Now here's my 100,000-line change that I need to make, right?" Much easier to just do that up front and have it on for everybody. Now, look. Like I said, I could literally stand here and talk to you about this straight for 10 hours.

15:39 There is much, much more you can do here. This is just a few areas, but also a whole area of software engineering discipline that you can apply on the other axis. So, this lets us now answer our question, not just it depends, but yes or no. So, does code quality even matter? For disposable software, no. For durable software, yes. And we get to answer a third and even more interesting question, which is does it have to be expensive? No. Thank you very much. >> [applause] [music]