Software Architecture's Biggest Enemy (Not What You Think) Transcript, AI Summary & Key Points

00:00 The ability to change your software is I would say the single defining characteristic of its quality. Think about that for a moment. If you can't change the code, then the only way to get everything else right, correctness, performance, security, whatever, is to be perfect the first time. And nobody's perfect the first time. So, the real question for any system is how expensive is it to change?

00:23 And the answer to that question is dominated almost entirely by one thing, coupling. Coupling is the enemy, but the catch is you can't get rid of it. >> [music] >> Hi, and welcome to the Modern Software Engineering channel. My name is Dave Farley, and if you haven't been here before, please do hit subscribe. And if you enjoyed the content today, why not hit like as well, and maybe join us in the comments.

00:53 I think that coupling is the biggest single challenge in the whole of software engineering. It's the thing that decides whether your system stays soft and adaptable, or slowly calcifies into something that nobody dares to touch. And I want to give you a really useful way of thinking about it today. Built on a lovely piece of categorization from my friend Michael Nygard, the author of Release It, who breaks coupling down into five distinct types that I think makes it a really useful tool.

01:24 Because once you can name the type of coupling you're dealing with, you can actually do something useful about it. But before we get to the five types, I need to clear up the most common misunderstanding about coupling, which is that it's simply bad, and that less of it is always the better option. That's not true, I'm sorry. Let's start there. If there is no coupling in our system, the parts can't talk to each other.

01:48 We need coupling, it's essential. Coupling is just the connections that turn a pile of separate parts into a working system. So, the goal is not to achieve zero coupling ever, really. The goal is to be deliberate. The discipline is in choosing which coupling we choose to accept and how we're going to manage the rest of it. If you'd like to learn more about managing coupling, I have a new free how-to guide on that topic out right now.

02:17 I've also got a free tutorial on designing to manage complexity. Find them both out in the description to this video. The useful rule of thumb is that strong coupling is perfectly acceptable when the thing you're coupling to is stable. Think about something like SQL. SQL barely changes from one year to the next. So, depending heavily on SQL is a perfectly reasonable choice.

02:40 It's not going to move under your feet. It's going to still be the same as it was last year and the year before. But, depending just as heavily on this week's version of your own application's database schema is probably a very different story because that thing is going to be changing all of the time if your application is under development. Same strength of coupling, but an entirely different kind of risk.

03:04 The problem isn't coupling as such. The problem is coupling that's unstable, unintended, or invisible. That's the stuff that really hurts us. Right. So, let's name the enemy. Mike Nygard gives us five types of coupling. The first is operational. This is about run time. The consumer can't run without the provider. The classic example of this is imagine an application that won't even start up when its database isn't running yet.

03:33 The application is operationally coupled to that database. Sometimes that kind of thing is unavoidable, but it bites hard when lots of systems lean on one another or a shared provider. One flaky shared service and everything that depends on it goes down with it. So, it can be the enemy of fault tolerance and get very much in the way of our ability to gracefully degrade service when things start going wrong.

04:01 Instead of allowing our system to collapse and be completely unusable, that is. The second in Mike's list is developmental coupling. This is about change really and is one of the most pernicious forms of coupling in terms of its impact on our ability to develop software at any kind of scale. This one bites as soon as we start working with more than a handful of people on the same system.

04:24 Developmental coupling is present when two components have to be changed together. A modification in one forces a coordinated modification in the other. This is the coupling that shared code creates. The moment two services share a function, neither of them can evolve independently of the other without dragging the that other along in lockstep. That doesn't mean that shared code is inherently bad.

04:52 Sometimes it's very good, but that goodness always comes with this cost that the coupling must be managed if you are to work effectively. This problem gets exponentially more complicated as systems and teams grow in size and complexity. The next in the list is semantic coupling. This is the subtle one. Two components share a concept, what a customer means, what an order contains, and even if they don't share a single line of code, they still have to agree on the meaning in order to be able to work together.

05:23 When the meaning changes, they both have to change in step and nothing in your tooling will warn you that this is the case because there's no code dependency to be visible. Sometimes we can helpfully trade off semantic coupling with developmental coupling, but as usual we get better results when we are more conscious of what's going on. On one big project that I worked on, which was in flight when I joined, we had seven different approaches to mapping data in and out of our system to a database.

05:53 Seven. There was no rationale for which mechanism worked where. There was This wasn't a conscious choice of any kind. These seven distinct mechanisms had evolved and had been organically adopted by different teams. Often, we'd use different mechanisms to access the same data from different parts of the system. Everyone was dissatisfied with this, of course.

06:15 Everyone knew that this was a dumb place to end up, but the amount of work to normalize the approach was by now so high that we lived with it for quite a while. In the end, we did an audit of how much time we spent maintaining this horrible mess and made a case then to our bosses showing that it was worth the time for us to fix it and and and do the work that was necessary to get rid of six of these options.

06:42 But, this coupling cost a lot of time, money, and degraded quality when changes didn't happen in step for a couple of years until we finally fixed it. The fourth type of coupling is functional coupling. This is when different parts of the system address the same problem in different ways. Two slightly different implementations of calculate discount, two competing notions of what valid email address is.

07:08 In other words, the system has more than one answer to the same question, and these answers tend to drift apart over time. And then you get bugs and systems that are unpleasant and inconsistent. And the last incidental coupling is the version that exists for no good reason at all. A module reaches right across a system to grab a value that it has no business knowing about, or our system crashes in production because someone dug up the network cable with a backhoe in the car park and tore it apart.

07:36 And yes, that that last one really did happen to my team once on one project that I was on a few years ago. This kind of coupling buys you nothing at all. It's pure accident, pure cost, and it's everywhere. The best we can realistically do to shield ourselves from this kind of coupling is to build more resilient systems so that they continue to do useful work even if that means degraded service when bad things start to happen.

08:04 This episode is brought to you by sponsors Equal Experts, Transfix, and Octopus Deploy and through the support of our patron members. We'd like to thank all of them for their ongoing support. Our sponsors offer products and services that are extremely well aligned with the topics that we discuss here every week. So, if you're looking for excellence in continuous delivery and software engineering in general, please do check out their links in the description below to this video.

08:34 There's a great quote from computing pioneer David Parnas that I think positions these five types of coupling rather nicely. The connection between modules are the assumptions which the modules make about each other. That's true of all of these types of coupling except that perhaps the last one. Except assumptions about what other stuff that you don't know about might happen, I suppose.

08:55 This is true because each of these five types is really about hidden assumptions in one part of the system and what it makes about another. If we're doing a good job, then coupling is at the heart of many of the structural and organizational decisions that we make when building more complex software and you can see its impact everywhere. For example, I think it's useful to think of event-driven systems as deliberately trading off operational and developmental coupling against each other.

09:26 Components don't call each other, so they don't need each other to be up and running and so they don't need to change in lockstep. They concentrate the coupling instead that's left into the shape of messages, which is a perfect example of the real game here. You don't eliminate coupling, you move it to where you can better manage it. That's what happens.

09:47 The really tricky thing about coupling is it's often invisible until it hurts. So before we talk about fixing it, let's talk about the symptoms, the way that coupling actually shows up day-to-day, so that we can spot it more easily. Slow builds. Build time is a physical measure of coupling, and everything depends on everything architecture always builds slowly.

10:08 One of my clients a few years ago had a very highly coupled system with many circular dependencies, to the extent that their normal build process involved compiling the whole system seven times in a row to achieve one outcome, because that was the only way that they could get a working build from all of the pieces. Absolutely crazy. The next in the list of warning signs is complex test setup.

10:37 If you need three paragraphs of scaffolding before you can exercise a single behavior, the design is screaming at you to say it's not good enough. We fix this by reducing the coupling between the parts in our design, not by sharing the complex test setup between tests. Brutal tests that break when you rename an internal class. That's a test coupled to implementation detail instead of one that asserts desirable behavior of our system.

11:02 Teams that are blocked on each other. A tightly coupled team moves at the pace of the slowest team it depends on. Lockstep platform releases where shared code forces every dependent service to take a new version whether they want it or not. And long parameter lists. A method with eight parameters knows too much and depends on too much. I get nervous above four, and I generally prefer to design my builds to refuse more than six parameters to a function.

11:31 Coupling is a slippery concept, though, and it's really more about how information works in general than it is about technology. And so is equally true about organizations as it is about code or design. And as Conway's Law tells us, any organization that designs a system will produce a design whose structure is a copy of the organization's communication structure.

11:56 That's a sideways statement about coupling. I think it's important to recognize that that fundamentally there are only two tools for coping with coupling: design and speed of feedback. We can survive a more coupled system if our feedback on the viability of our changes is fast enough. In fact, that's exactly what continuous integration is there for.

12:18 It's a tool for safely working on coupled code. We bring everything together continuously so that the moment our changes collide with somebody else's, we see that. We notice the problem in minutes, not days or months. And from the other direction, we can survive slow feedback if our system is sufficiently decoupled in terms of design. Independently deployable services don't need anyone else's permission to release, leaving teams free to do whatever they need to keep change flowing and keep themselves working

12:51 efficiently. The downside of this is that it requires a reasonably sophisticated approach to design to achieve the necessary looser coupling. We can think of this as a classic two-by-two grid. On one axis, how coupled the system is, and on the other, how fast the feedback. Three of the four corners are survivable and work fine. Decoupled with fast feedback, wonderful.

13:14 Decoupled with slow feedback, fine. That's classic microservices. Coupled with fast feedback, also fine. That's good continuous integration doing its job. But there's one corner that does not work, and unfortunately, there are many, many systems and teams that fall into that corner, strongly coupled with slow feedback. This is almost the definition of what we would think of as a big ball of mud legacy system.

13:39 This is the danger zone. Changes are dangerous, releases are fearful, and nobody is ever quite sure what's going to break next. The tragedy is that most teams don't choose the disaster zone. They drift into it by accident. They started treating a tightly coupled system as though it was a set of microservices deploying the pieces independently from one another, even when those pieces actually share state, schemas, and concepts, and so are coupled in multiple dimensions of Mike Nygard's model.

14:12 This approach hurts very badly indeed. So how do we get out of this disaster zone, or better yet, never drift into it? There are only two honest strategies. We compartmentalize by design, or we protect ourselves with fast feedback. Pick one deliberately. My advice for new systems is to always begin with continuous integration, even when the long-term plan is to split the system into separate services, because until you actually understand the right boundaries for that, continuous integration is our best defense against

14:49 the coupling that we haven't yet learned to see. And then there's the design side. A handful of techniques do most of the heavy lifting here. Hide information behind APIs. Services should keep secrets because a public interface is what gives you room to change the insides without breaking the consumers. Treat the boundaries between modules, services, and teams as special guarded places in your design.

15:13 Translate other people's data at the edge and validate it. Treat it as something that could easily break the system cuz it is. Announce, don't command. Instead of calling another component and telling it what to do, announce that something has happened and let listeners react to those things. Event-based design. Be parsimonious in what you consume and generous in what you produce.

15:37 Every field you read from another system is a coupling that you've accepted. So, take what you need and only what you need. But, when you publish, tell the whole story. Design from the consumer's perspective. Use messages between decoupled parts of the system like services unless the semantics are very stable and indeed and essentially fixed. Remember adding a field later to a message is easy and removing one is a breaking change.

16:06 The golden rule is to tighten what's stable and loosen what's uncertain. It's okay to couple tightly to things that you understand and that change very slowly. But, always couple loosely to the things that you're still figuring out. And if you want a single brilliant detector that will help you to highlight coupling better than anything else that I know, use test-driven development.

16:30 If a test is hard to write, don't power through it. Listen to it. A hard-to-write test almost always means that the code under test depends on too much. The difficulty is coupling made visible before it ever reaches production. The cost of software is the cost of change. The coupling is the biggest single factor in that cost. You can't eliminate it.

16:52 A system with no coupling isn't a system at all. So, all we can do is to decide which coupling to accept and manage and mitigate the impact of it as far as we can. Here are a few tips to finish with. First, name your coupling operational, developmental, semantic, functional, incidental. That can help you think about it. Each has different remedies and you can't treat what you can't name.

17:18 Second, strong coupling is absolutely fine when it's stable. It's the inevitable, accidental, and invisible coupling that really hurts you. Continuous integration is by far the best tool to mitigate the costs of this, and you should treat it as your friend when you have more tightly coupled systems. And third, above all, stay out of the disaster zone.

17:38 Strong coupling plus slow feedback is a combination that never works. Escape it either by decoupling via design or by speeding up your feedback or maybe both, but never ever drift into it by accident. Modularity, cohesion, separation of concerns, abstraction, every one of these ideas that we talk about all of the time on this channel is in the end in service of this one goal: keep your software soft enough to change.

18:07 I think that the ability to think clearly about coupling, to see these five distinct shapes and use them and treat them as distinct things with different cures instead of treating them as one big blob of spaghetti is genuinely one of the things that separates the great engineers from the rest. Thank you very much for watching to the end, and if you enjoy our stuff on the channel, please do consider supporting us by joining our Patreon community.

18:33 There are lots of perks to that. There's information about what you can get and how to do it in the description below. Thanks and bye-bye. >> [music]