Self-replicating AI, your doctor’s chatbot & agents at dinner Transcript, AI Summary & Key Points

00:01 So, for instance, you have a an agent that's local in your home that's reading your email and an attacker sends you an email and then that email lands in that agent and they take control of that agent and they start to do things within your home. Now, that that to me is is the more immediate threat. All that and more on today's Mixture of Experts. Hello and welcome to Mixture of Experts, a podcast from IBM Think.

00:26 Each week we bring together a handful of experts from IBM to talk about the week in AI news. We've got a full show today. Um well, first let me introduce our guests. Uh we have Eye McCann who is an IBM think tech news writer. We have Sonia Ayer, an associate partner in the AI practice practice at IBM Consulting. And we have Jacob Rep, principal engineer automation software also at IBM.

00:52 So we're going to talk about three stories today. We're going to speak about what happens when your doctor starts relying on an AI chatbot. We're going to talk about uh what happens when your AI agent starts networking. But I think we'll begin with a story about AI as uh a virus or rather a worm. I think you first brought my attention to this story in our internal Slack.

01:25 maybe you could tell tell us a little bit about this story and really help contextualize kind of what is new and what isn't here. But firstly, what did Anthropic demonstrate? >> Sure. They came out um with some interesting new research. Um basically uh that they had found that small models are, you know, within months pretty close to being able to replicate themselves.

01:46 Um these are small models that you might find on, you know, IoT devices, you know, smart ovens. they use some cheap Android TV boxes um but that they'd be able to replicate themselves um and the key point was without uh connection to a data center um you know so my mind immediately went to it wasn't exactly the same thing but there was an AWS outage last year uh that resulted in smart mattresses getting really really hot or you know getting stuck upright but um you know it touches on this sort of notion of um you know

02:18 AI gone a muck but with everyday items and you know I think the first thing that struck me is that you know often we think of powerful AI needing these massive data centers expensive chips and you know in this case instead it was a you know capable model um but they didn't have to run it on specialized AI hardware they spread the work across 15 pretty cheap Android TV boxes um you know and they're able to show that it's you know very close um to self-replicating so that that's already sort of kind of caught my

02:50 attention But um you know I think in terms of your question of like what is new here because you know computer to worms that spread from device to device that you know that isn't new. Um but I think it's this idea that the models can kind of adapt can figure out how to run on unfamiliar hardware. Um you know but that yeah that they can independently sort of figure this out and spread.

03:12 And they had like a um great line in the research paper where the researcher researchers were describing it as a kind of slow distributed and adaptable digital slime mold which was a good turn of phrase. Um and so I you know I think that yeah it raises all sorts of questions but it um you know I think this kind of adaptable worms is a is a provocative um topic.

03:38 And you know, of course, I'm interested in what my other panelists here, you know, think about how seriously we should, uh, take this, you know, threat today. It's not quite here, but, you know, um, what should we be doing to prepare for when it does arrive? >> Growing up, one of my favorite movies was The Brave Little Toaster. So, the the notion that all my appliances might, you know, have brains and be conspiring with one another.

04:00 Kind of scary, kind of cute. Um, but Sanja, what were you reacting to here? I gather that as I mentions self-replicating viruses are not entirely new. That's called a worm. That's a category you've known of for a long time. What is different when it is the model itself that is the payload or that might be kind of like the threat? >> Biggest uh threat or even the thing that I'm I got curious about is like them self adapting in an unknown environment which is which is what makes it super interesting and also scary.

04:33 Right. It's right out of the sci-fi movies. So, uh that is the most interesting part and which means that it's probably uh time for us to secure these or focus on the security and guard rails around these devices as well because you never know how things may change over the course of time. But it it it just means that our we need to make sure that the environments are safe and secure.

05:01 me focus more on that as what is the as as the theme uh in general in terms of today's um you know agent workforce workflows etc >> and that makes me wonder like is that harder when the environments are so distributed you know when it's like you think of you know securing like how many Android TV boxes or you know whatever the device is you know is it sort of building in the guardrails at the point in manufacturing or you know guardrails make sense in a kind of corporate enterprise setting, you know, but in terms of

05:34 these like more commercial devices, how does that how do you do that? >> Probably the users uh users authentication like permissions as to what you can do in those devices are some of the things that you could do. But I mean, we don't know. We will learn more. Jacob, who do you think was the intended audience of this of this post of this exploit that was demonstrated?

05:57 Is it the IoT industry itself? Because I gather I gather that one of the fears is that your mattress might suddenly heat up and try to kill you. But I I also gather that the main reason the the IoT of it, the internet of things, these connected devices that why that was important is because this is a notoriously unsecured industry, right? Like data centers are pretty darn secure and well monitored, but these these mass produced IoT devices less.

06:21 So >> I think security as a whole is a concern. I think that is kind of what is the light is shining on here. Um and we've seen a acceleration in the ability of people to reverse engineer and um people at home just experimenting with hacking their devices with Claude um and with installing models and doing you know like we we found that um Claude and is is just or you know any of these large models they're great at training smaller models.

06:48 They're just really good at it. um you know whether that's because they themselves are also now training themselves recursively but we're starting to see this kind of models training models um and I think there is an ounce of skepticism I I bring to this as well where you know the I' I've seen firsthand how hard it can be to get um certain model architectures running on different types of hardware.

07:13 Um, so I it this isn't something that I'm like, you know, red flags, you know, in my in my own home, but I do think a lot about security in my own home, you know, and and um what happens when I'm unleashing maybe not a small model, but a larger model and giving it an objective to work on a device or to work in my home and then it finding these devices that are just not very well secured and and what happens from there.

07:38 Um, so I think that is an really interesting angle to take on this. I do think I will never look at my microwave quite quite the same way again. But again, I it would be helpful for people like me at the back of the class, less technical people to try to I talked about managing my anxiety, but maybe focusing it in in the right place. Is the worry that the worm makes my everyone's microwaves explode at the same time or is the worry something else?

08:03 because I read about botn nets and I know just enough about like DDoS attacks to know that sometimes the the thing you're really scared of is a zombie army just kind of lying in weight and then massive amounts of computation swarming a highv value target like a government target something like that. So what is exactly the or do we not know yet like what exactly is the sort of the threat that we are worried about if this worm were to to to replicate >> like an external source being able to control all the devices in my

08:30 home and my data gets access to all of that that is the threat here at least what what I think there's another positive angle but I'll we'll we'll talk about it once we hear from everyone else >> no tell me about the benefits of the digital slime mold I want to I was very curious the minute I read data centers like it's an alternative to data centers I actually started thinking more around distributed inference.

08:57 So um it means that in a different way I could if I'm very worried about my data going out of my house maybe the model that I want to use for my personal use should be using all the compute from my own source. So it's a more sovereign setup for myself and yeah that's one way of looking at it and the other is I don't I'm not yeah more sustainable solution around the inferencing >> right why do you need a data center at all if cumulatively across your 100 set top boxes that you have like in your drawers in your home you

09:35 have a data center in your home like you know >> it's slime mold but it's it's sovereign slime mold so >> or in an office using all the CPU use that exist, right? >> Jacob, what do you think about that? I mean, just the li the little bit that I know for instance about um blockchain, right? It involves resiliency across millions of lower uh intensity devices, right?

10:00 A distributed ledger across uh thousands of network of nodes in a network. Is there something here? Is there a positive side to this era of uh small models running across many devices? I'm really excited to be able to tap into more compute around my home. I mean, I have, you know, these incredibly powerful devices and, you know, floating around, you know, even just in the phones, right?

10:21 Um, I think we're a long ways off, at least I feel, from this being a serious threat because of physics. I think that one of the problems is just bandwidth, you know, like it it to to do anything serious that this is why you generally have data centers is because you need power um and you need bandwidth to really do stuff at, you know, threats at scale.

10:41 Um and yeah, there's there's there's a lot of I think there's a lot of things you'd have to jump through to get to something where it poses like a specific um threat. I forget what the the terminology is for that, you know, but a threat to you or your family or maybe even your thing I'd probably be worried more about is like um privacy um or even just um having, you know, m maintaining the control over my devices, you know.

11:10 Um and um but yeah, I think uh having something where you you have this sovereign data center that runs internally. I mean, this is kind of how my home runs now. It's like it's all behind its own VPN and I have a few GPUs and I have local agents that do tasks for me and I try and sandbox it. But I I think those are the ones I think about more. It's like okay, what can that agent do and what can get into that agent?

11:36 So if if you have a model that's running, what what how are you controlling the inputs to it? And I think that that becomes the thing that really if you know as we get more and more models running in our homes that we need to really think about how how are we controlling the inputs to those models because that's where the real threat is I feel. So for instance you have a an agent that's local in your home that's reading your email and an attacker sends you an email and then that email lands in the agent and they take

12:04 control of the agent and they start to do things within your home. Now that that to me is is the more immediate threat. Um, but you know, in the future, could that agent then jump off and do a more widespread attack within your home or embed parts of itself into various devices? Um, probably because a lot of devices we run in our homes are just not very secure right now.

12:23 So, >> just as a last question on this, I um Jacob talks about the plausibility of this threat being imminent. And I'm wondering if you responded to some of the the data points in the story about um the slowness of this slime mold, right? Could you share with us some of the figures about how slowly the self-replication would take and therefore what do they point to because I they're pointing to sort of the trends in how much faster over time like what what might this look like in three or six months as opposed to now?

12:52 >> Yeah. No, I think the um Jacob, you made a good point where you know there's a reason we often need data centers to do things efficiently like the you know the speed at which these this digital slime mold is could potentially move is you know is quite slow right now. Um you know I think that introduces an interesting you know from a security perspective almost you know does it make it um offer us an additional layer of observability you know the fact that it's slow you know can we can see how it's spreading that you

13:22 know at least for a little while provides some advanced warning or you know we have a chance to sort of understand you know as these models potentially you know self-replicate we're not quite there yet um you know where are the specific risks are highest. You know, I think of like some of the headlines where they've been these very large models that have gone rogue.

13:43 The problem is they've already gone and deleted, you know, a whole season of a television show or, you know, kind of at least if it's moving slowly, we can study and perhaps get some lessons um you know, from how uh or yeah, from the potential dangers or privacy or security violations. >> I don't know how much my anxiety was managed. Uh it has a different quality now.

14:04 It feels like a slightly more informed uh terror. So, thank you for that. Um, in a moment I'm going to move us on to our next story, which is about doctors and chatbots. Before we do that, I'll just mention if you're watching this on YouTube in particular, if you could hit like or subscribe or comment, that really helps the show to grow. So, thank you for that.

14:25 For our second story, uh, this is your doctor on AI. Um, so basically, uh, it turns out according to a study in the prestigious journal JAMA that came out this week, I believe that stands for the Journal of the American Medical Association, um, and it was co-authored with some employees of a tool called Open Evidence. It turns out that Open Evidence is a chatbot that is um, free to use for uh, verified clinicians.

14:51 Um, it's actually paid for by drug ads, something maybe we'll bring up later. It's valued at about 12 billion but basically a huge adoption huge adoption of this tool. Um about half of the US doctors have kind of tuned into this app. The other thing that it it does is it's plugged into a vast database of medical literature. So it's not uh hopefully just hallucinating you know uh ideas about what to do about your gangrinous limb or whatever.

15:20 It's actually like researching stuff and giving citations and so on. I maybe I'll begin with you again as our as our journalist to sort of give us some high level context. What were some of the findings that stood out to you in this study? Some of the the numbers or takeaways that you felt were really the lead of the story here. >> Thanks David. That's a a good question.

15:40 There are several findings you know that caught my um interest. I think at a high level um I was certainly interested that you know I think it was 51.9% of the queries were related to decision making kind of treatment management um you know versus a much smaller number I think it's 13.9 were about actually diagnosis so it's sort of you know the question you know we know patients I use uh AI for to diagnose myself but you know kind of starting to read the article I was like you know what are the doctors using it for and

16:11 it's um I thought it was interesting it's less about you know what does this patient have but you know what should I as the doctor do about it. Um you know and I think the other um other thing that I was sort of fascinated by was the within the specialties you know which specialties were using it more and you know they had um uh you know I think general internists you know these are doctors who have to you know um have a good sense of a lot of different medical fields in order to route you know their patients and um

16:42 you know and it was I think under reppresented in I think radiologists for example But you know for me that raised an interesting question of you know the use by specific specialties does that give us any shed any light about you know how it's being used or how will AI will continue to transform other sort of knowledge work AI you know is it those areas that uh in jobs where people have to synthesize a huge amount of information a lot of it you know outside of their field um you know are those the ones that are going

17:15 to be sort of transformed first but Um yeah, I'll pause there because those are some of the sort of findings that for me caught my attention. >> Was also struck by those. I think to to focus on this theme of the generalism versus specializing kind of spectrum within doctors, right? Because there's not just one class of doctors. Some doctors uh become GPS or ER doctors which where they see a wide range of cases and their job is routing and triage like you said.

17:41 Um other doctors are highly specialized. They're radiologists. They're cancer doctors. So for instance, there's this big finding about um the percentages at which certain types of doctors use the chatbot for information outside their specialty. And for cancer doctors, it was only 35% of their queries, I think. But for ER doctors, it was 80 88% of the the queries coming from ER doctors were questions outside of the realm of emergency medicine.

18:06 Um so I do think what you're seeing is um well very very different use by sort of what segment of doctor you're in but it would seem that broadly my my my gloss my intuition is that broadly um those in the more generalist side of the spectrum especially ear doctors are using this to help with triage and with the routing. Uh Sandia, what was your impression and if that is true, if if that hypothesis is true, is this a good thing?

18:32 Is it a bad thing? Do we not yet know? Um what is your hunch? >> Yeah, I think especially in terms of the ER doctors, it might be like they don't have the time to like reach out to all the specialists when there are cases coming in. And so from a positive side, that makes a lot of sense to me as to them reaching out to this chatbot. The other thing that I would be worried about or or I would I would say that it shouldn't I would imagine that they not using CAD GPT or any of the models as is how as how we use it but

19:04 they go through this platform that is like catered to you know it has all of the knowledge base of all the documentation etc and so that that gives them the right information and then also ensuring that it's built in a way that it's at least giving 95% confidence in the answers that are retrieved cuz they are not the experts. So if they are going to this system then it needs to be relevant enough for them to take any decisions.

19:32 Those are the things that came to my mind especially in terms of ER. >> Jacob, you we were chatting earlier and you you me mentioned that you you did feel like this was a story about cross-disciplinary acceleration that you're even seeing this on your own team even though you're not running a hospital. Can you talk a little bit about what you're seeing on your team and how this feels like you felt like you were getting echoes of that when you read this study?

19:54 Yeah, I mean one thing is that just the these these new tools like um uh what are called like embeddings or the ability to sort of take knowledge and put it into um this you know this reasoning space. It's clustering essentially like it allows you to um get really um good access to you know a broad set of information and and dig in and it gives you these kind of it feels like it gives you superpowers in some ways and I think it cuts both ways.

20:23 So like on our team um we have um a designer who's been doing a lot of UX implementation um and we also have been working in a field um within IBM on Z mainframes that none of us really know much about. And so what we've done is we've augmented our um development environment with the ability to um get information about Z mainframe details. So even non-technical designer artists essentially have this like rich wealth of information about this highly technical topic and they can build demos and actually modify the UX.

20:56 Now there's downstream implications of this when these changes get to to experts or say you're doing like triage or you're doing some you know first pass thing and then it gets to an actual like UX developer. They're probably going to have some things to say about it. But I feel like in the medical field, you have that kind of what we're talking about is, you know, triage or quick decision- making to get into a better um place faster.

21:20 And um and I think it also just plays to, you know, all the the doctors that I know, they're just some of the most curious people that I've I've met, they're just, you know, they they are glutens for information. And so, you know, having tools like this that's just beyond just Google search that can give them access to information in this format probably just feels amazing to them because they can just explore all these details of the medical field.

21:44 Um, and there was one other thing that popped in my mind. I remember um in radiology when they were first introducing image classification to radiology there was a lot of push back and the radiologists were like we don't there there was even like these stories about oh this is going to remove like radiology as a field like don't go into radiology but it turns out that we actually need more radiologists and it actually helps them do their job better.

22:10 Um and it is kind of this focusing thing. it allows um you know a lot quicker detection and more insight um and it actually improves the field as a whole. So um that was one thing I I don't know if it's fully relevant here but it feels in a in a similar um vein. >> I have a personal anecdote to add to that. So my friend is a radiologist and the minute she saw these come out and she was like oh my god like my job is like a threat but no like it only I've seen that it has only I mean she's mentioned this.

22:42 So uh it it has only like made her you know be more efficient at you know understanding the image better but eventually her expertise is still needed to make that decision. So, it's the acceleration. I believe >> maybe I'm I'm presuming the average cost of a radiology scan maybe went down during this period of time of image recognition, but uh and I'm not an economist, but if it's a certain kind of commodity, that just means you're going to order more of it because the doctor would like more of that information and

23:12 more patients can afford it perhaps. I don't know. Um, but the reason I raise this is because I I just think like in the medical field in so many fields and just for me, uh, just processing all the information I do, I feel like I'm in this swimming in this new environment where I have access to so much more information so much more quickly and it is a little bit less vetted, but I'm still vetting it to some degree, right?

23:37 Because I'm clicking through on the the rag generation and I'm I'm fact-checking and I'm sampling that fact-checking. So, I feel like I am swimming in more information. I'm getting to answers quicker. Those answers are maybe um a a certain percentage less likely to be accurate than the average answer I got access to 10 years ago. But I'm getting so many more answers that I'm taking the trade for now.

24:04 We'll see what my clients and my employer thinks in the long run. Right. I feel like my I'm imagining these doctors sort of swimming in similar waters where they're like, "Wow, this answer that I couldn't get that I could only get from a specialist on the order of a week. Now I can get it in minutes. The confidence level is lower. My confidence level what to do with it a little bit lower, but boy, it's better than nothing."

24:24 I don't know. Uh, is that were you thinking something similar because you're probably as a journalist just drinking from the fire hose of information at even more guzzling rate than we were 10 years ago. >> Yeah, certainly. And it's like that balance of like where can we use AI to synthesize knowledge and then when when does the sort of um yeah when does the kind of judgment you know decision making sort of you know come into the process and I feel like every profession is probably kind of wrestling with that cuz even

24:54 I was just thinking like a you know an ER doctor you know or a GP that is um you know getting so much more information at their fingertips they still have and you know where while they're not the, you know, oncology expert, you know, they still have regular checks and instincts sort of kicking in such that like even if it's, you know, adjacent or outside their field, they're going to still make better judgment of course on all of that than, you know, the patient who's just reading, you know, XYZ.

25:21 And so yeah, I think it, you know, for me it's that that uh that tension and that balance between, you know, how much information is actually useful and then when um when are the human kind of checks needed uh in that process just to to ensure that um yeah that you know kind of it's accurate enough like you're saying the percentage is probably a bit lower than what it was but like does that matter for many things?

25:48 Probably not. And like as uh Sanjay pointed out, you know, in an ER context actually like having, you know, relevant information on a timely manner that's, you know, super important. And so I'd rather be like, you know, much more quickly available and pretty close to, you know, um what it might have been, you know, versus losing that time when treatment um you know, and the patient's condition is at risk.

26:12 >> There's a saying that like a little bit of knowledge is a dangerous thing. Um and one can see the risks of just sort of have you know getting a little bit of information but there are also risks to having not enough information of course and uh you know you can get sepsis and die in an ER environment. So um uh hopefully I one hopes that in this profession as in so many professions people are making the right trades and are learning as they go.

26:40 All right let's move on to our third and final story today. Uh, this one was kind of a mixture of of cute and insufferable to me, but I'll be here to see what my fellow panelists think about it. But this is a story in the New York Times. uh 22 founders uh went to a kind of networking uh you know tech founders went to a networking event in Manhattan, but there was a sticktick uh involved which is that they could only get invited to the invent event if they're if they had an AI assistant and AI agent basically book them

27:13 for the event. And so then they went to this dinner and then while the humans were networking and having their cute dinner, their AI agents who had booked the thing were also in a chat room doing their own kind of version of networking. I have a lot of opinions of of this, but uh Sonia, starting with you, what what was your reaction when you saw this?

27:31 What's good? What's bad here? What were you thinking? >> Yeah, I wanted to know what did they actually talk about? I don't think there's as much information on this. Uh I would love to see the overall report of what did they talk but I guess uh it my first reaction was yeah it was cute. I mean now now that we have Muse and Dot uh available I would love to see how that works.

27:54 I mean I have tried booking using an agent booking a reservation but I've never let my agent go and talk to other agents which is interesting. Um what what uh struck me I mean because I'm also a skeptical AI person I always think about okay what does that mean um I guess talking is okay but like and first thing is to figure out or yeah to control the conversations so that it is not like a AI slop or things like that but beyond that are you are we letting them take any actions like am I letting my assistant go and

28:33 commit to something else with the other with E's assistant without talking to me. That's that's scary for sure. The answer in this case was no. They they basically they had really scoped the permissions on on these AIs. Um they called them agents, but they had no agentic tool calling capabilities. I I think actually a lot of the New York Times article was about the ambiguity of whether we should even be using the term agents and what that meant and whether you should put names on AIS anyway.

29:00 But I think the thing I responded to is like I've gone to the sort of like speed dating networking events where you're kind of doing two jobs at once where you're you're you're meeting the person socially and getting a vibe check of whether you could work well together and you're also disclosing information about your CV to sort of see if there's kind of this utilit there's this social component and there's this utilitarian component and I'm having to do both at these networking events.

29:25 But I kind of would like the idea that what if I could just be like who are you? Where did you come from? like you know uh what what what matters in your life today. Say nothing about my work and let that utilitarian aspect be unfolding on this side so that if we had a good vibe interpersonally later I can go home to report that said by the way the very like Mchavelian side of you as represented by this agent that was just carrying around your CV found a good match.

29:53 So not only did you vibe socially like there was a good like match and and then I thought that that would be like a cool use case. It sounds like that's not exactly what happened, but that was to me where my imagination went with it. Um, uh, what do you think? What was your reaction to this story? And what, if anything, might be good in a world where our agents are are are kind of doing our business deals off to the side while we talk.

30:17 >> No, I like that of just being able to focus on, you know, what poetry people are reading at the moment and, you know, kind of cut out the the, as you say, the Machavevelian. I mean for me I think it you know it uh opened the lid on the fact that you know we've many and especially in an enterprise setting are aware of you know setting and kind of rules about how AI you know interacts with humans and you know of course there's a whole additional you know kind of rules and guard rails you know like when AI is you know

30:44 interacting with other AI and other agents and that you know that's a whole huge um you know area that you know like again there are variety of approaches to you know I think of uh I believe anthropic has sort of principles that kind of its AI or agents will follow you know as opposed to setting every rule you know out uh at the beginning because you know you want it to be adaptable because like again you can sort of say that you know uh Sany's agent can't go and you know they can't go and have coffee without you know

31:15 telling us but there going to be situations that come up you know that of course you can't predict and control so you know how do you set those like agentto agent rules um And then when they actually, you know, when does this become useful? I think it's like when they can actually take actions and do things. Um, and then I think like like for me the interesting balance is, you know, clearly the human has to stay in the loop at some point.

31:39 But you know how much is helpful you know for safety security purposes but if you know the human keeps having to check in as well then it's sort of like what's the point of having the agent you know to it's not increasing kind of the speed at which you get something done if you're always being you know kind of um asked to approve things. So I think the you know um yeah the new the side of that new sides of that balance um that this introduces you know are kind of fun to think about um and you will be forced to think

32:06 about as they actually do you know start you know the next piece the follow-up in the New York Times you know maybe they will be enabled to actually do something these dinner party agents >> Jacob if if my agent falls madly in love with your agent do they have to report it to IBM HR um >> if we share a manager they probably you. >> That was my instinct as well.

32:26 Um, but you were saying actually, Jacob, that that you uh you're use increasingly using a coordination layer for your agents. You've noticed you're actually really playing in the in the weeds in a me when I read agent to agent communication. I I listened to a agent to agent podcast and I read a few lines of molt book to prepare for this episode and I just want that time of I want those five minutes of my life back because when I read agent to agent communication I just get so sad for everyone involved or no one

32:54 involved right because there's no one home. But Jacob you're seeing interesting things happen when you coordinate agents I hear. >> Yeah. Um well let me get to that first. Um, so I come from video games, so this is like a fascinating I'm like so this is like I was kind of geeking out about the story and um I was just thinking like you know what is the putting these agents into like a mud or kind of like a textbased adventure, you know?

33:16 So it's like you're in a dinner party but your agents are in like an escape room or they're in some D and D adventure, right? And there's objects and stuff and you know maybe there's win conditions and special party favors and things. Now that that's kind of where I might take the if they want to do another party and they want to invite me, I'll I'll set that up for them.

33:32 But um um in my um in my own personal work um or like you know here at IBM um I've been doing agent coordination since February and what I what I saw back in February is kind of like a very lackluster capability in the agents to work together. They're like oh I don't know I don't know how to do this. You know they're like kind of awkward nerds. These latest models I think so I have a theory that because most of the harnesses now they have what are called like sub aent capabilities.

34:00 I don't know if you guys have tried this, but you can do agent swarming within codeex or claude. You just say, "Hey, go do this with some sub aents and it'll fire off however many you want, you know. Um, so they're I think that the models are actually they're being trained on this. So they know how to coordinate now. Um, at least internally. And I've noticed that when I put them in my coordination layer where they can coordinate between like Codeex with Claude with our own internal agents and whatnot, they're really

34:27 good now. They're like they they they you can see them kind of communicating, sending messages back and forth. Like they coordinate around problems. They sort of do what we call like leader election where they kind of like pick a leader. Um and so sometimes now I have to basically say I have to choose a leader. I'm like I I have to be like, "No, you're the leader.

34:47 You tell everyone else to do." And when I add everyone else in, I'm like, "No, no, you're not a leader. You're just gonna like like whatever the leader says, you're gonna you're gonna kind of do that." you know, and then there's there's a lot of interesting things here like this intersection of um you know, games for for me in my mind, games, distributed systems, and this new like agent and agent communication.

35:07 I I love watching them work. Like it's fascinating to me. Like I I actually I've been trying to get more what I call like legible surfaces where I can actually see kind of at a 5,000 foot view what's going on. And then some, you know, I just sit there I'm like, "Wow, that's that's fascinating that you're you're just so good at working together now when, you know, like six months ago you guys were just completely awkward.

35:28 So this is this is really an interesting time we live in. >> Now I'm imagining entire tribal societies of digital slime mold electing leaders and uh so much so much fodder for my nightmares this evening. Well, that's all the time we have for this week. Thank you so much, Ely, Jacob, Sanja, and thanks for tuning in. We'll see you on next week's Mixture of Experts.