What Your Coding Agent Can't See in the Code - Tom Moor, Linear Transcript, AI Summary & Key Points

00:01 [music] >> Thanks for turning up to a talk about agents and context. I, you know, I I didn't think these were keywords that would really hit today. Uh, I'm sure it won't be the the last the last time you hear about it. But I'm going to try and kind of get out of the the inner loop a little bit uh and think more about the outer loop of agents. So, um excuse me if it feels a little off-topic at times, but we'll get there.

00:37 So, yeah, I I work at a company called Linear. Um I'll start with a just a quick intro for those in the room that might not be familiar. Uh Linear started life as an issue tracker. Um it was born out of the frustrations with the existing products in the space. Uh how poor quality they were, kind of slow, hard to use, buggy, slow. I won't name any specific products, but you may have used other products in the space.

01:04 Um So, you know, the founders very selfishly uh wanted to work with great tools. You know, the grea- the the the best crafts people always invest in the highest quality tools that they can afford. Uh and we felt that at the time, you know, engineers had a distinct lack of good tools. Uh now this this was around 2019. This is when uh Linear was founded.

01:27 Um And you know, you you might say that the world has changed a lot since then. Um Some some things have happened since 2019 or since 2020. Uh but as you know, as a company we're we're still striving to to build build high-quality software and and build systems that enable that for everybody else. So, what what changed? I hope you guys will just humor me for a second here.

01:53 Um this is not new information to you, but I I I want to just like bask in in how fast this stuff moves for a split second. Like we've we've had a complete reinvention of software engineering in the time that that our company exists and is only half a decade old. Um it's not that long ago that tab completion came out. Um at least AI assisted tab completion with like GitHub Copilot in 2021.

02:19 Um coding agents did not exist in any functional sense until 2 years ago. Um the first coding agents came out in 2024. Um these were really early. They were not that great and you definitely had to be kind of cutting edge to be using them. Um so coding agents themselves are both like long in the tooth and like super early right now. Um I'm assuming that if you're in this room, you're probably using a coding agent as the primary way to work right now.

02:55 Um that is the assumption of this talk. If not, come to me afterwards. I'm intrigued what you're doing. Um you may have you may have moved to to a world beyond this and definitely come and talk to me then. Um so, you know, we hear a lot talk about software factory now. I don't know if that's the right terminology. You're also going to see our co-founder did a talk a couple of weeks ago about how we shouldn't call it a factory.

03:17 I'm not going to harp on that particular piece. Um but I do think that we are you know, one of the important pieces here is that we're kind of quickly moving from a world where you're hand driving an agent on your local device um to orchestrating more agents or uh having them run in the cloud automatically and things like that. And so that that does uh change how it does change you the how you use the context, what context you need.

03:46 Um yeah, if we look forward like this this this is just the beginning, right? Like that that speed that we've seen, it's we're still at the the very very beginning. There's a lot of folks on Twitter that are like naysayers saying that oh, the free ride's over. It's all going to go up from here, like they were just subsidizing. I don't believe in that at all.

04:07 The cost is going to keep dropping. Like this this graph just is just an example of, you know, 3 years ago what the intelligence that would have cost you $100 now costs you 5 cents. So it's is you can extrapolate that into the future and kind of get very interesting results. In a few years like ultra-fast will be common. You'll You'll snap your fingers and a feature's worth of code will appear.

04:31 And then on top of that like task horizon is exponential too. So excuse the graph pitch. Pitch is great software but it does not let you It does not let you do an exponential axis. But it makes it look more impressive. So you know, the important thing here is we're now leaving these coding agents running longer and longer. So think back to 2024, you had a coding agent but you were definitely kind of very in the loop tight loop there, right?

04:58 And now these things can run for hours, days. You hear You hear folks leaving things running over the weekend having success with it. I don't know how much truth there is in that, how much of that code actually ships. But no doubt it's increasing and will continue to do so. So why am I mentioning all this? Like what's what's this got to do with agent context?

05:17 I I think that one of the main reasons is that the context kind of moves out of that inner loop and a largely into the outer loop. If we have like super-fast systems completing tasks for cents and running for hours or days, you're not going to have humans manually delegating and starting and stopping them. So we need to start building systems to manage this chaos and manage the context.

05:46 So yeah, that this sounds great, right? Like we're going to have tons of code produced. It's going to be really cheap. We're going to build software factories. We're going to pump it out like features coming out left, right, and center. And I think like to to be a a little trite like you know, code code is not the bottleneck now. Implementation is not the bottleneck now.

06:07 Um the failure mode here isn't an inability to build. Um the failure mode is like an inability inability to maintain coherence, um to have a product that makes sense as a whole. Um what makes a product great is something something we care about a lot and we something that we are trying to build a product that helps you build great products too. So, it's very meta.

06:30 Um but what makes a product great is consistency, like restraint, uh trade-offs, and like lots of tiny decisions that reinforce one another. This was a Codex generated image of a uh >> [laughter] >> a settings page that's gone a bit crazy. Uh don't look too closely at it. Um but it's a good example, right? Like just add another setting, you know, what could I have done to do?

06:52 Um So, you know, a a coding a coding agent today will will read the code, but it can still and that's great like that there is a lot of value in the code, obviously. Um but it can still miss a lot of the context behind it. Um so, some examples would be like um the intent the intent, what what problem are we solving here? Why does this matter? Uh why does this matter to the end user?

07:22 Um which decisions were made previously? What were the trade-offs that resulted in this design or in this feature? You know, um how should this look like visually outside of the the code? Uh how should this feel like in the product? Um maybe things like is there other work underway in other parts of the organization? Um and like who who is the decision maker on this?

07:44 Who's leading this project? Who's ultimately responsible for the decisions that happen here? And then are there any like additional risks? Like you can see some things from the code, right? But you can't always see oh, um like a revert happened 2 weeks ago in this area of the code base may not be immediately apparent. So, the idea here is that we carry this uh we carry this context through Um from the initial idea through planning, implementation, and review.

08:16 So, I'm I'll I'll talk a little bit about our system for doing that. Most of this is in production in some form and most of it's in production in a way that other people can use, too. This this isn't a linear ad. You you can use linear for this, of course, but the principles apply broadly and you can you build this into any system that you're using.

08:36 And it's it's agents all the way down, I'm afraid. This is one of the reasons I'm I'm bullish on AI as a whole. Like, every time you deeply dive into a problem, the solution is more AI. And it's definitely the case here. So, the first thing we did is automate is automate intake. You have, you know, as if you're developing a product you have ideas coming in from Zendesk, Slack, Teams, Gong, Twitter, your internal product team, your CEO.

09:12 All of these ideas kind of need to be collected. We feed it into this concept called triage. Data dog alerts. Like, this this so many things. I this is like 1/5 of it. Then we have something called triage agent. It labels, it assigns, and groups [snorts] the incoming work into themes. It closes duplicates. It roots things to the correct people and the correct teams.

09:42 And it serves serves as decisions when more input might be needed to clarify the problem. It also investigates. The This is this is part of like the investigation it does. And when the investigation is done, it's it's left on the issue in our case. Issues are kind of the center of our world still. And it's left as detailed context for the next human or the agent to use.

10:07 So, by the end of that investigation, we have, you know, the problem that came in or the idea that came in, who requested it, it's been enriched with the customer information. Hey, this is a high-value customer. This is the third time they've requested this in a month. Any slack stack traces maybe if it's a bug, logs from Datadog that are associated.

10:30 It's hooked up to your internal MCPs so you can have like internal account IDs and things like that. And then also related things that have previously touched this area, related issues, and decisions that were made on those. This is all surfaced on the on the issue itself and kind of forms this this base context. So, you can have something very simple that comes in like this crashes in this scenario and then by the time the triage agent is done, you have a really a really filled out context.

11:02 A few examples of where this was useful for us recently. Um I don't know, you probably can't read this. It doesn't matter too much, but, you know, this is an example where the video captions kept getting turned was it on? Yeah, kept getting turned on. Um and here like our triage agent went beyond finding just the setting or something like that. It actually reproduced the bug and discovered that there was an automatic browser action that was being mistaken for user intent.

11:29 Um This level of detail left on the on the issue context it was key in success of like automatically fixing the issue at this point. Um Here's another example uh where the research checked the existing code base. It found some similar similar shipped work, made sure it wasn't a regression. In this case, you know, is this a missing feature? Is this a bug?

11:58 Um it found all of the customers that are associated with this request uh and realized it's high value. It also found a likely point in the code base to get started. So, as you can imagine, this is just like a really great starting point for your coding agent at this point. It probably massively increases the chances of success. So yeah, we have a pretty nice base context.

12:18 The next step is to build either automatically from the triage agent deciding to deciding that the issue is quick enough. In our case we we have a if the bug is small enough we automatically fix it, but you could do this manually. It could be locally via MCP or it could be remote in the cloud cloud. I'm not going to harp on this slide too much. I'm sorry if this is where you expected most of the talk to be, but in my experience this is kind of a solved problem today.

12:51 The most important thing here apart from the base input context that we just created is the ability to verify and the ability for CI failures to feed back into the agent. So between those two pieces I see a lot of success between the base the base context being able to verify and being able to see failures we see a massive amount of success in our team.

13:22 Lauren at XAI you would probably potato on Twitter. She has a great set of skills you can use to generate the verification skill for your coding agent. Really makes a huge difference. So the next place where context matters and I I think it's still ill-considered is in review. It's also where it gets quite interesting. Like if you think about your setup today, what does it look like?

13:50 Do you do you just have codex running on your on your PR's? Code rabbit, reptile, one of these products? What are they considering when they review your code? It's it's likely that they're getting all of their context from the code base itself. And mostly focusing on logical correctness. >> [snorts] >> That's okay. That's good. The coding agents the the review agents are really really great.

14:15 They catch a lot of bugs. But at this point we make internally we make much more context available than just the code base. We dispatch another agent. Hey agents. We have a review agent. It starts with that original issue and that base context that we created with the customers and the original intent. And then it goes and looks at product history. So things like previous decisions, reverts, regressions and incidents in the same area.

14:47 Now we have code to compare against to write. So we know which code has been touched and we know the intent of the PR. We can we can further enhance enhance the context with this. We also need to refine related issues. Hey, we now we now have the code context. This might be weeks later. It might be minutes later, but it might be a long time later too.

15:06 We also want to catch overlapping and conflicting changes. And we also have the full agent session. All of the decisions that were made in that. Maybe it was a one shot. Maybe somebody went round and round and made some critical decisions along the way. And finally ownership tells us who should make the product trade off. And where review coverage may be missing.

15:29 You know, it's not always people that are in the code base. It could be your product lead. It could be your CEO. Could be your head of product that needs to be heard too. So now we're thinking like less about is this code correct and more about is this the right change for the product? So I have a couple of concrete examples of this. These are all within the last few weeks of our product.

16:02 In this case our review agent found several relevant changes from six months ago and seven months ago. where This was marked as an incident risk um, due to holding a DB transaction open. Um, this is actually really great. Like, I you know, a review agent uh, a standard review agent may have been able to find that, potentially. Um, but in this case, the extra context really helped.

16:30 And Codex did not find it. Um, this is a pretty visceral example where we surface decisions made uh, 1 year ago, uh, where we dropped some UI because it was busy. Um, this is the kind of decision we make a lot in a product, right? Like, ah, this this doesn't feel right. This doesn't feel good enough. How do you get that back into the coding agent's context?

16:52 Uh, in this case, like, a PR attempted to re-add that same iconography uh, that'd been rejected multiple times before. Uh, we were able to surface that and then of course, these review comments get fed straight back into the coding agent um, as additional context. And, uh, one more here. We we have the the concept of team and workspace labels in our product as a as a hierarchy.

17:18 Um, and in this case, it looked like the workspace labels were being displayed below the team labels. Um, I don't think a code code review uh, tool would generally pick that up. Uh, but in this case, yeah, it had a a previous revert and an issue um, related to this and was able to surface that. So, this is where it's able to like maintain that sense of coherence.

17:42 There's no place in the code that defines that. There's so many places that labels are listed, for example. Um, we also built a UI around this that will guide a human reviewer through the change set uh, and acts as a place to surface those decisions. Um, and places where the judgment calls are needed. You can use any review product for this, of course, but you know, we we we want to build tools.

18:05 We we we're we're a big fan of like, um, handcrafted tools and we wanted to make the the best one we could, so we built this surface. Um, and then the final stage is like distilling that feedback. So, along the way, these comments have been stored um, on issues, on PRs, changes have been merged into the code base, a lot of context has built up. Um this this is like multiplied at the organizational level, not just in the code.

18:35 Um this gives like the human corrective influence beyond the the current change. Hopefully we spend less time making the same correction twice. And then on top of those changes that end up back in the code, end up in the issue tracker, end up in the pull requests, we also have a specific loop set up. This is an automation that runs once a day, but it could could be less or more depending on your PR volume that will troll through all of the code that has been merged in that day.

19:10 Um it will find actionable lessons, places where um these types of comments happened in the PRs and changes were made or reverts happened, um extract learnings, and then inject them back into the code base, um either in the agents.md or in the related documentation, um just to make sure that those learnings are con- continuously compounding. Uh the exact prompt is something you you just have to experiment with and um obviously these changes aren't something you want to build up forever.

19:38 You don't want your agents.md to end up like a million lines of context, so this is this is something that that you have to uh it's very time >> Is this prompt always actually handled by the >> This is this is a linear automation, but you could use any automation product for it. Yeah. Um you know, I I think every product has has some sort of automation trigger at this point.

20:04 So, what does carrying the context through the whole workflow buy us? Um hopefully I've kind of described it, right? Like fewer changes that solve the wrong problem to begin with. Um at intake, understanding the intent helps us with that. Um during implementation, the the product history and conventions help us build on decisions that we've made before.

20:27 At review, seeing the related work helps us catch conflicts in the product and with ongoing work. I didn't give an example of that, but it exists. And decides like whether the change like fits the product as a whole. The aim here is to to is a few avoidable mistakes, less rework, and with that we get faster iteration faster faster iteration. And then at that final distillation loop, that final automation, you know, human judgment that happened during this whole process becomes context for the next loop around.

21:02 You know, our our judgment matters more as code gets cheaper.