Stop Doing Agent Archaeology on Your Team's Code - Richard Feldman, Delta Transcript, AI Summary & Key Points

00:01 [music] Welcome everybody. This is from sharing code to sharing agents. I'm Richard Feldman. I'm assuming everybody can see everything on the slides yet, but actually as I look, I don't see anything up there in terms of slides. So maybe we have to press a button. We press the button. All right, cool. Yeah. So, um, imagine this. It's Monday morning and you get a DM from a co-orker.

00:27 It says, "Hey, so I made a bunch of progress, but I got to head out for vacation. Uh, here's a gist with the branch name and my transcript with the agent. Uh, bye. I'm off off to vacation." So, what do you do with this situation? Like, think about how you do this at your job. If somebody hands something off to you, like, how does that go? Like, they've been going back and forth with an agent.

00:49 I mean, I guess at some point you're like, "Okay, thumbs up. Uh, what else can I say? Let's just see how it goes." So this is the the classic handoff. Uh you know whether it's you know someone goes on vacation or maybe it's just across time zones. You know you're you're working with somebody and they want to hand off the work that they've got in progress to you or maybe they just want to get you involved.

01:04 So here we've got you know the issue description so I at least understand what their goals were like what they're trying to do. Um I've got some code snapshots so the commits like you know as they went along uh to some extent. I've got the agent transcript which might just be the sort of copy pasted as maybe a markdown document or something like that.

01:19 Um, I would like to have how the code evolved maybe because I could see like what they already tried and like maybe they went off in some directions. Some of those did or did not work out. Um, hopefully those are captured in the commits, but they might not be. They might agent might have just said, "Oh, I'll not commit this because it didn't work out."

01:35 But I would still like to know that if I'm picking it up and I would love to have their thought process, like not just, you know, the transcript of like what they said back and forth with the agent, but is there any other context that might be relevant like other directions they explored, things like that, like rationale behind their decisions? These last two are just kind of fuzzy when all I have is like this gist and maybe a transcript from what they were going back and forth with the agent.

01:54 It's kind of like, you know, there's this whole puzzle that they've got in their head and how all the pieces fit together, but I'm not getting the whole puzzle. I'm just getting like some puzzle with some pieces missing. And maybe I can go back and forth with my agent to try to recover some of that and do a little bit of detective work and try to figure out like, okay, what what does go there?

02:10 But unfortunately, when you when you have pieces missing, you don't have the complete picture. Sometimes it's hard to answer things like I've got this frog over here that looks like kind of a frog in the in the puzzle and like what's what's the frog got his hands on? Is that is that like a climbing wall? I mean what what is that? A purple thing with these like dots on it.

02:25 But if you have the the full picture, it's like really easy to answer a lot of these questions like oh no it's a mushroom. Okay, now now I see what it is. And this is sort of like how we generally tend to work when we collaborate with one another on our teams while agents are involved. like we've each got our sort of our own silos where we talk back and forth with the agent and then as soon as we need to get another co-orker involved who we collaborate with all the time, we start to have these lossy transmissions where

02:47 where we send them like little snapshots of like what we were doing and we leave them to sort of piece together the missing pieces. In the worst case, we end up having [clears throat] to go back in time like let's say that you know from 6 months ago there was some decision that was made when a feature was made and now we're trying to build on that but we're like hm why was this feature made in this way?

03:04 I can see how it was made, but there's a problem now. And I don't know if they considered what I'm about to try and decided not to do it for a good reason that maybe I want might want to know about, but I don't have that. That's that's just not available to me. It's buried in some transcript that didn't get copy pasted to me. It's not mentioned in a commit message or maybe there's like a throwaway line in a commit message that I'm really trying to read into to try to figure out what the rationale was.

03:25 We're just sort of doing this agent archaeology where we're trying to dig back into the the history of what happened using these little clues that we have in little commit snapshots. Um, this is just a big waste of time because the information was there at some point when somebody was going back and forth with their agent or maybe multiple people were collaborating but you know they were doing it in Slack or they were doing it in a meeting but it wasn't actually captured in this context.

03:46 It's a waste of time to just go back and try to guess at what happened when we could just have the full information of like what actually did happen. It reminds me of this meme from back in the like the early smartphone era where it was like, "Oh yeah, you know, in the old days you would have like this gigantic video camera and you'd have like a corded phone and you could have like a boom box for playing music and all these like CDs for storing your music on and stuff like that and then like a a computer which was a

04:11 full desktop computer tower back then. It's like, yeah, you know, you can get by with all these things, but they don't quite capture the feeling and the experience of using a smartphone where everything is put together, and it's all designed to work together, and it all works together really nicely and seamlessly. There's just so much less friction when you can just take out your phone, record a video, play it back later, as opposed to having to have all these disperate pieces of equipment that sort of trying to

04:35 connect them together in a way that just never quite adds up to the same experience. [clears throat] So basically what the theme of what we're talking about here is sort of like human agent human collaboration. It's humans collaborating together with agents involved in the mix. I want to give you a cool example of of the tool that I'm about to show you um and and what happened to me when I was actually in the process of making these slides for this talk.

04:54 I got a little notification from a co-orker uh through through Delta this tool that I'm going to show you and he at mentioned me and he said hey the agent thought I should check with you about this thing that he's working on. Um are you still working on conversation undo like the undo feature in the middle of these uh conversations with the agent? And I said, "Oh, no.

05:09 Actually, I started on that a long time ago, but actually what I have right now is is totally stale, and I suggest, you know, just just taking it over if you're up for it. It's not something I'm working on anymore." He says, "Okay, cool. Thanks. I think it's tangential to what I'm working on here, but I'll keep that in mind." Now, this is a back and forth that could have happened anywhere.

05:22 It could have been on Slack. It could have been like, you know, out loud or something like that. But because it was in Delta in this tool, this was not out of context. This was not out of the blue. This is actually connected to what he's working on. So, this is actually in the middle of his thread of conversation with an agent. He just was like, "Oh, I just realized that Richard worked on this in the past.

05:39 I'm going to atmention him and just I got a notification. I came in. We had this interaction and this is now part of that conversation that he's having with the agent." So the agent can see this convers conversation that we had and it knows that as now part of the whole overall thread. If you zoom out, this is a screenshot of Delta as a whole. You can sort of see like this was just the lowest friction thing you can imagine for him getting his question answered in a way that's not disconnected that doesn't require any

06:04 archaeology or putting together disperate pieces to connect after the fact. It's all just right there in the conversation. You can have more conversations. You can have more comments and all that stuff in one seamless experience. It's sort of a very natural way to collaborate in a way where the agent can actually participate in and get access to all the stuff that you're trying to collaborate with.

06:24 So here we've got an example of like code changes. So he, you know, of course in the context of his agent, you can kind of see a little diff down there. He's got like about a 2,000 line diff going on there. Um, you can see that he's got a whole lot of, uh, you know, code changes happening and I can see those code changes. I can participate in that. I can jump in and do stuff with my agent if we want to like work on this in real time together.

06:41 Um, but also we've got the conversation and of course this is now a big part of how we make code changes. you know, back in the olden times, the ancient era of like five years ago, um, you know, there used to be a lot more like time spent on the actual code changes of like typing things in by hand and whatnot, but now the conversation and the code changes are pretty inextricably linked.

06:59 They're they're a major part of how software gets made these days. So, the question then becomes, why do we keep them siloed separately when that just causes problems and makes the agents less effective? Why don't we just have them be in the same place? So, this is Delta, what I'm about to show you. This is the tool that we've made that helps you collaborate with one another while using agents.

07:20 So here's uh here's Delta. This is a native app running in Mac OS in this case. We've got it for like Linux and Windows as well and a mobile app as well. Um and basically I've got my thread here. Uh this is something I've just been working on that's like basically because we have OS level notifications. One of the things that can happen is maybe you have your OS configured and like the notifications are allowed for this app but not that app and you want to just have a little like press button to say like send it send

07:42 me a test notification so I can confirm that I'm actually able to get notifications through Delta. So I sort of you know went back and forth with the agent here and uh one of the cool things about this is that you can sort of highlight anything that you want and just like comment on it. So I can say like uh you know that the agent said Delta doesn't ask for notification until the first real notification fire files.

07:58 I can say like uh is that avoidable? I just ask the agent that and I press enter and I submit and then the model's just going to, you know, go do it. In this case, I've got Opus 5.5. Um, we're we're pretty uh, you know, model and harness agnostic here. We've got like a cloud code plugin in the works. I can do it via my chat GBT subscription. We're going to roll out allowing you to do it through your um, cloud code subscription via ACP, all that good stuff.

08:18 This is really about collaboration. So, you know, bring whatever harness, whatever API keys, etc. Uh, you want. Okay. So, my agent responds and all that good stuff. Cool. Now, the other thing that I can do is I can bring in one of my co-workers very easily. So, if I look through these things, I might say like, okay, um I'm I'm looking through and like reading what the agent says and I says, okay, uh let's say, okay, is this error going or should this error be a uh toast or a inline error?

08:44 An inline error. And I'm just going to actually um at Bennett. So, this is my co-orker Bennett. Oops. Um Bennett. I'm just going to ask him like, you know, what do you think? Now, Bennett when I just at mentioned him there, he's going to get a little notification in his app. Uh, so he should be able to, you know, see this and then join the thread. And I don't know if you noticed, but now, hey, there he is.

09:09 Oh, cool. He's, uh, he's typing and he's responding. So, you know, wherever Bennett is, he's able to just like hop in and respond right there. He says, "I like inline better." Like, oh, cool. Yeah. Okay, sounds good. Um, so this is a good example of where now again we've had this really low friction collaboration, this really easy conversation here, but again this is now part of the conversation.

09:27 So all my agent when I continue working on this and also anyone else's future agent if they if they're if I give them access to this thread, they can find out this full context of how these decisions were made. So they can see that like this was kind of a huh we just decided, you know, no big deal. The agent can actually know like was this a really deep market research question or was this just like h what do you think?

09:46 Okay. Yeah, that seems good to me. Um, I mentioned earlier briefly that, you know, we've got all the code changes in here. So, of course, you know, the agent's been making some code changes which we can see very easily. Um, but also this is a full-fledged like, you know, we before we made Delta, we made the Z IDE. So, we've, uh, we're very familiar with how to make like a really nice code change experience.

10:02 So, if you are in the like reading the code, you know, camp, I know some people have different feelings about this. We make it really easy to get in there and like do a line by line code review. So, I can just like see all this stuff. I can say like, uh, you know, is this going to work on Linux? Um, right. I can just like comment here and like press enter and like that'll get submitted to the agent.

10:19 The agent understands that I'm asking about this line of code. You know, all that commenting stuff just works on there. You can do sort of like a traditional like, okay, I'm I'm I'm done with this. I've done reviewing this. I can do that. We've even got if you want to invite other co-workers to, you know, give you more of a formal review, you can do that and they can hit like, you know, approve or reject um, etc.

10:34 on those things. So, you know, whether you're going like down to the individual line level, you know, if Bennett wanted to hop in here and like make edits to the code directly, he would be able to do that. Um, [clears throat] we can do that like really nicely. So again, the the theory here is like, you know, if you're doing a single player thing, Delta is just a very nice UI for that.

10:50 I've got all my threads over here. I could do like, you know, sub threads that are, you know, broken down in here. I can jump into any of these and see what's going on with them. Um, and then if I want to do like code review, all of that works very nicely inside Delta. We're just trying to reduce friction um, you know, for whatever workflow you have.

11:04 Now, another cool thing that I can do um is I can ask Bennett, and I'm not not even hands on keyboard. I can just ask Bennett and say, "Hey, Bennett, uh, could you would you mind inviting me to a thread?" Um, and what you're going to see is that I can actually just get a notification from Bennett. Oh, there it is right there. Now, the reason I was able to do that without my hands on my keyboard is that Bennett's in the audience.

11:21 Hi, Bennett. So, [laughter] uh, that's, you know, that's a little bit of cheating. Um, now what you can see here is that Bennett's got this thread and part of his thread is actually he's got a terminal running. Um, so you can see the output of his terminal right here in line. Um, and if you're, you know, watching the animated thing here, Delta NYC AI engineer, um, this is all just his terminal syncing in real time with my machine.

11:42 So, I can just see the output of his terminal. I can't type in it because that's a security risk guy. You're not going to get like he's not giving me remote code execution on his machine. Um, but everything I'm seeing here is actually like right there directly like replicated in real time between these threads. So, that's how serious we are about this real-time syncing thing.

11:56 Um, now if I want to, I can press exclamation point here and bring up my own little terminal. Now, Bennett's able to see this in real time as well. And if I just do cargo run um while this is building um you know he's able to see in real time like you know oh here's how my thing's going. Now of note this is you know so now I'm running this version that says RT Feldman because this is running on my machine like what it shows here is just like you know who's actually uh doing it.

12:18 So this is his machine. This is my machine. I've actually got like you know you saw how quickly I was able to join this thread that just immediately set me up. So on my local file system I now have a complete mirror of his entire code repo basically. Um these are not get work trees these are delta work trees. We we made it more efficient so that you know you can really seamlessly hop between these things.

12:36 And basically the overall theme here as as you probably tell is just we don't want you to have to experience any friction to collaborate with people. It's it should just be this easy to just like hop into what the other person's doing. I now have a local checkout. I can use whatever agent I want. He can use whatever agent he wants. We can talk back and forth.

12:50 Um the whole idea is just to eliminate that friction. Just make it really easy for people to collaborate on code. So this is Delta. Um, basically, you know, to to wrap up, like we have this whole example of a handoff, but you can imagine this in all sorts of different scenarios. You know, you're working on something, you want to bring a co-orker in.

13:08 You know, I showed it with one co-orker, but of course, you can do it with as many co-workers as you want. Um, you can go down these threads to sort of isolate the context. You could have multiple threads, multiple subthreads, you can have sub threads within subthreads, all that good stuff. Um, and this is just so much better than having only an issue description and the sort of the crude code snapshots with commits and just hoping that the commit message captures everything.

13:27 um you know, issue description, code snapshots, agent transcripts, all well and good, but knowing like how the code evolved and the thought process behind it and being able to go back and see that in the past and and giving your agents full access to all that means that you're no longer working with this just sort of puzzle with pieces missing. It means you're actually working with the full puzzle and you no longer have to worry about those pieces sort of being blurred out because nothing is blurred out.

13:50 You just have full access to all the information that can give you and the agents the best ability to do your jobs possible. you don't have to do this agent archaeology thing anymore where you're sort of like digging up in the past and like really hoping that that context is available somewhere and piecing it together because that stuff's just a waste of time when we have all the information.

14:05 We should just be able to just work and collaborate effectively together. Um just like we did, you know, in the pre-agent times. So basically, if you if you're interested in Delta, you might be somebody who's like, "Yeah, I don't want to keep working with all these disperate things that sort of like glued together. I would rather have something that feels like a cohesive, lovely experience where everything just feels seamless."

14:24 And that's Delta. Thanks very much. I think we may have a couple time a couple minutes for questions. Um, but there was another speaker coming up, so I have to be uh cognizant of that. But, uh, yeah. >> How high does this scale up? How many users can >> how high does this scale up? How many users can be online concurrently? You know, we haven't done a like really big stress test, but I can say that we do all our standups in Delta, which is like usually like 30 40 people at a time.

14:52 Um, and we haven't had any problems at that scale. Uh, that's all in one thread, I should say. So, of course, if you're collaborating across a whole company, that's going to be a kind of an unusual case where you have that many people in in one thread at once. Um, but you can if you need to. So, I would say I don't have a definitive answer for you, but I would say that scaling in terms of like, you know, if your whole org wants to get on Delta, that shouldn't be a problem.

15:12 It's really just about individual thread. >> Is this deployed on cloud or is this local? >> Is this deployed on cloud or is this local? Um, so what I just demoed to you is deployed on cloud. uh you you can have either like uh if if you want you can use it through a um like I was demoing on my local machine but if you want to switch to like a cloud runner that's also an option like for as to like where the agent's actually executing.

15:31 Um the infrastructure for the syncing is currently run on our cloud but we have our currently we have a a uh our first on premises pilot going on. So if that's something that you're interested in because I've talked to a lot of people here who are in regulated industries and they need to have everything running on their hardware that's something that we're absolutely into.

15:46 So u just come talk to me if you're interested in that. >> Thank you. Yeah. Other questions? >> Yeah. >> That's fine. Go ahead. >> Uh, so if I use this, is there any additional token expense? >> Ah, if you use this, is there an additional token expense? Um, so we haven't published pricing for this yet. Uh, because it's in private beta right now. So right now there's no expense.

16:13 It's free. Uh, but of course the there there will be a cost in the future. But, uh, short answer is to be determined. Uh we haven't like set our exact pricing yet. Um but when we do delta.dev is the website and you'll be able to find all that information there. >> Yeah. >> How do you think about life cycle of this? Like you have a s bot that responds to issues and like it be kind of nice to tag and like talk to it if you're awake or just wake up in the morning and see what step it like get deployable.

16:48 How do you >> Yeah. So, a couple questions in there. I think I heard three and I'm going to just tackle them in uh in any order. So, at the end you mentioned uh like CI and like how do you um deal with that? So, we actually have stopped using pull requests. We just use delta for everything now. Um so, basically like if I uh you know finish what I'm working on, I can just say like hey you know agent like go run CI on this and then once CI passes I can say like hey just go ahead and land this.

17:11 And we actually have a dedicated flow for like landing things and it'll like report in like you know green check mark like this landed. Um that type of stuff. So yeah definitely as as far as like the CI life cycle and like landing things. Um and I know like if if people have like sock 2 compliance they need to be able to say like oh there was like somebody signed off on this and gave a review and we also have a workflow for that as well.

17:30 So you can like officially sign off and check like I approve this you know. Um and that all works you know seamlessly real time just the way I demoed everything else. Um, another question you asked was about um, like S bots, like you have like a bot or something that you can appment mention. Um, that's a great question. I mean, so we uh, obviously like the appmention system can be extended to work on that.

17:47 We don't officially have like a bot API at this point, but if that's something you're interested in, like definitely please come talk to me because obviously like the metaphor of like, you know, appmentioning co-workers works just as well if you want to appmention anybody else and bring them into the thread. Um, it's just not something that, you know, we've heard people ask about yet.

18:00 You're the first. Um, but yeah, definitely something that is, you know, a possibility. So, let's talk about it. >> Yeah. >> When you tagged Bennett, did he have to read everything or is there like summary that >> Oh, great question. Uh, when I tagged Bennett, did he have to uh, you know, read everything that was above there? Um, or did he uh did he get a summary?

18:24 So, actually, I'm just going to give you another example. This this literally happened uh when I was on the train on the way over here because I'm from Philly, so I just took the train over. Um, my co-orker was working on uh these like this other feature that he knew that I'd done some stuff on. Um, and so he brought me into this conversation and literally what I did is I said to the agent, "Hey, I'm new to this thread.

18:39 Can you give me an overview of the approach here?" Um, and so here's this like subthread that I went off. Now, he was able to keep going on the main thread without, you know, dealing with my sub thread over here. Um, but basically, let me full screen this. Um, basically the the idea here is just like I wanted to catch up on what he was doing and so I just did all the same kind of stuff I just uh, you know, showed in the in the demo.

18:57 Um, you know, I'm new to this thread. can you you the agent give me an overview of the approach here? Um so yeah, like the typical thing that you would do with agents, right? Just be like, "Hey agent, can you give me a summary of what's going on here?" And then I ask some follow-up questions and so forth. Um so all this stuff, you know, uh is designed so that you can sort of compartmentalize like if I want to go off on a tangent with my agent and not interrupt the main conversation, you can do that.