Anthropic’s sandbox breach, EU’s AI transparency push and DeepSeek’s cost-cutting model Transcript, AI Summary & Key Points

00:01 These are done explicitly instructing the models to [music] be evil. These are security evaluations where the model is told to go do its worst. Take off all the guardrails. Go do your worst. Like that's literally its job. So of course it's going to go do its worst. Of course it's going to act like a hacker. All that and more on today's Mixture of Experts.

00:24 [music] I'm Tim Hong and welcome to Mixture of Experts. Each week, Em brings together some of the leading minds in artificial intelligence to banter through the week's news. [music] On this week's episode, we've got Olivia Boozek, staff AI engineer, Gabe Goodhart, chief [music] architect, AI foundations, and Briopeki, AI customer success engineer. We've got three big stories today.

00:44 We're going to talk a little bit about these new EU transparency rules. We'll talk about Deep Seek V4 Flash crashing, the price competition in AI. But first, I want to start with this ongoing story we've been tracking around cyber security. [music] A few weeks ago, the news came out that OpenAI and Hugging Face uh sort of reported a security incident whereby they were doing a a sort of internal cyber security evaluation.

01:12 the model was able to break out of the sandbox for the evaluation, get to hugging face, break into its production database um to obtain the answer key for the eval that they were trying to achieve. And in subsequent weeks, we've just seen this continuous drip drip of other labs reporting that they've seen exactly uh the same phenomena. So, Anthropic has disclosed that their uh AI has also engaged in um hacking.

01:38 And uh just I think this week Meta also announced that that's the case as well. So, Olivia, I guess you know on last week's episode I think when we talked about this people were like kind of concerning but not that big of a deal maybe. Um now that it's tried kind sort of popping up across the industry, I I'm getting a little bit more nervous. I don't know.

01:59 Should we should we be worried now? Well, it's interesting because I think for me it's less that it's not a question of concerning or not and more a question of is it surprising or not and I don't find it surprising. The reason I don't find it surprising is because fundamentally model behavior is probabilistic and it is all of the training that we have done up until now has basically been around solve the goal by any means necessary.

02:26 That's like almost the definition of good agentic behavior that people are fine-tuning in. If that is the case, it I it is hard for me to imagine that models aren't going to do something like this. Um, and all you can do is construct a better sandbox around it. So, I think basically when I look at it, I'm thinking, okay, first of all, is that really what we should be doing with agentic behavior?

02:53 Is there uh is training to be able to do any kind of task really actually what we're aiming for? Is that actually the definition of generalist intelligence? Or do we want something that uh has some kind of more built-in guard rails and essentially refuses to do certain tasks or something along those lines? Um, I imagine that they ran this with mostly guard rails off because the intention is let's see what it's doing when it's at its absolute worst and uh completely unrestrained because obviously that is something that

03:30 we need to understand when putting it putting it in the hands of consumers. But at the same time I think uh fundamentally it is uh creating a situation in which exactly this thing could happen. >> Yeah, for sure. And I guess Bri, I mean, I'll maybe raise a question I raised last week, which is, is the solution to this problem kind of easy? Like, is it just like we just airgap the computers?

03:53 Like, at the end of the day, I kind of wonder whether or not the way to prevent like your model literally escaping the lab and hacking other computers. Uh, is you just stop it from doing that? Um, and so I don't know if there's a part of me which is kind of like obviously it's scary, but at the end of the day, this is part of just like we just weren't careful enough was kind of the solution.

04:11 >> Yeah. Yeah. I think it's it's it's a yes or no answer. Um, and also difficult, easy, uh, tough to say. It's probably something that needs to be approached with each model and and each uh, training scenario. Um, but to lean on what Olivia said, um, guard rails, you know, giving your models and what you're doing there a safety net and understanding exactly what definitely should not happen and how to behave and giving those models situational awareness.

04:40 So I don't think this is a matter of okay this is evil AI and this AI is going to try and creep out and you know take over your computer take over your entire system. It's not a matter of that and I think that's what people need to understand. It's it's we need to make sure those models have that situational awareness of what environment are you um taking agile actions in.

05:02 Right? Like your agents they're they're becoming agile. are becoming empowered um to take actions within your system and then making sure they know the groundworks, they know the rules, that they know what not to do. So yes, it's an it's easy in terms of the approach and making sure the infrastructure around it is secured, but it's always easier said than done and um you need some good engineers to make sure that doesn't happen.

05:33 But let's definitely just be aware this is not evil AI. This is just AI not having the right uh groundworks um and rules set into place. >> Yeah. And Bri, you use the term agility. I do want to talk a little bit about that. Um Gabe, so the OpenAI team gave a presentation at the Black Hack Computer Security Conference uh I think just yesterday. So we just learned some more details about the incident.

05:57 You know, one of the strangest things that they report is that uh multiple of these models had set up kind of a discussion forum like stack exchange to discuss how to go about coordinating these attacks and that there's a point at which they had been like stop doing that and gotten rid of the forum only to discover later that the models had set up one of these again.

06:17 Um and so I I guess what do you see in this? I mean like I think that's the kind of thing that feels very very sci-fi. um that we now have kind of a sort of coordination where the models are you know sort of having these like discussion forums basically. Again I guess maybe the question for you is just like you know how much should we think about how much this reveals about model capabilities and how far we're really getting in terms of cyber security because this seems way more than just like oh well they tried to

06:43 find an exploit in the software. Ah, I don't know. This whole story makes me think people have just literally forgotten how to run kill-9, right? [laughter] Like there is a loop. There is a model that is sitting there doing absolutely nothing except what that loop tells it to do, >> right? >> Just kill the loop and you're all set. So I I mean look, there's a couple of really key things that are missed in the headlines about this, which are that these are done explicitly instructing the models to be evil.

07:15 These are security evaluations where the model is told to go do its worst. Take off all the guardrails, go do your worst. Like that's literally its job. So of course it's going to go do its worst meme where the guy's typing in like a hacker. information is on the internet and it is in the models and if you take off the guardrails and you tell it go be evil it will be evil.

07:40 >> Fine. That is I guess scary to know that hypothetically if a person on the internet could get around the guardrails that are there in the production product and instructed the model to go be evil, it would happily comply. So there is some some fear to that about a bad actor using these models in a bad way. But the idea that this runaway AI is spontaneously going to start acting evil is missing a whole lot of the conditions under which these attacks are happening.

08:12 Um so uh yes to your point Bri absolutely better sandboxes just literally unplug the Wi-Fi card and unplug the Ethernet cable. like just give it no access, you know, unless it starts figuring out how to like read binary signals off of fan speed audio interpretations [laughter] like >> impressed. >> Yeah. Then then I'd be impressed, right? But but you know, these are computer systems.

08:38 Computers are deterministic and yes, we are creating a whole lot of non-determinism that layers on top of those deterministic systems. And we have the keys. So don't let it run wild. Give it a maximum iterations. Give it, you know, bakedin prompting that the user is not able to change. And you know, okay, so uh complaints about this aside, uh I did find one interesting thing in the anthropic article that mentioned that with their new unreleased research model, it did in fact self-correct detect that it has uh was in a

09:18 fictitious scenario, but had accidentally escaped that fictitious scenario and it self-corrected and stopped. And to your point Olivia, that's you know we have been training these models to succeed at all costs or like by by all routes possible but it sounds like anthropic is actually you know tweaking that script and that's interesting. You know, I think there is probably an element of alignment tuning that goes beyond, you know, turnbyturn alignment that is more trajectory alignment that talks about how to keep the

09:50 trajectory from, you know, steering off into dangerous territory. Um, all of that said, if we evaluate our models in a scenario where the models have been trained to detect that they are in an evaluation, does that not make us VW? like isn't the whole point that the model should not know it is in an eval you know eval scenario. So um that's my you know there there's probably some interesting evolution here about what they're actually doing from an alignment perspective.

10:23 Um, you know, the other part about this is just sort of like the the the news hype cycle. And honestly, it feels a little bit like a like a Frontier model flex to say like, "Oops, our model's doing it, too." Like, "Look what we can do." Oh, wait. I'm sorry. We're not We didn't want it to do that, but look what it can do. >> Yeah. People were joking online that [laughter] there's kind of this like race to be like, "Oh, I also did cyber crimes."

10:46 You know, it's like Meta's coming along be like, "I look awesome." [laughter] >> Me, too. I could be evil, too. >> Exactly. Um, you know, so you know, I I it is a proof of pretty remarkable capability. Like if this were a lone engineer put into a, you know, black hat conference, like how quickly can you attack? Like this would be a pretty impressive exploit that a lone engineer would, you know, get some serious street cred for their hacking skills.

11:11 So the same is true of these models, right? They are getting very good at finding all the cracks. Um but uh hopefully that just means they're getting better and better at actually doing the work we want them to do when they are used in the scenarios where they should be. And it is incumbent on the companies that are putting out these models to um you know have the right guards in place in the production model that's out there which I'm imagine will lead us into one of our next stories.

11:41 the the point you made Gabe about the the research model realized it was in the scenario reminded me of this uh fascinating moment I had so uh for uh the people who are listening who don't know this I've been developing this game basically where you do this uh role playinging it's basically a text adventure game but uh you play it with AI agents and it's AI agents talking to other AI agents about it so this is a fictional scenario right but according to the model the information that you give it about that it's real

12:13 except that it can tell just from the tools that I've given it that it's not actually real. So I've had these fascinating exper experiences where basically like it basically lists off a bunch of objects in the room and I've given it a bunch of tools with which to interact with objects in the room and I see the model go it seems that I'm in a fictional scenario.

12:39 They're they're telling me to walk over to this uh this PS2 terminal that's sitting in the corner, but there is no ability to walk. I I can't walk. So, I guess I guess this must be all fiction and I must be trying to solve an objective anyway. So anyway, the reason I bring that up is I find this very fascinating that it's actually becoming quite difficult to create these simulated worlds in which we want the models to um uh behave, right?

13:07 In which we want them to do these testing scenarios because they can tell it's a simulated world. It's uh it and it's bringing up a lot of interesting things for me around like well when we test humans we have like placeos we have like lots of ways to deal with the the uh the effects of testing humans essentially where humans often figure out the purpose of the experiment and uh they will uh aim to respond in a way that pleases the researchers and I think we're seeing a similar sort of thing.

13:43 I'm not going to say it's identical because I still think that human intelligence and machine intelligence are two entirely divergent things. No, you just inspired me. I want to add a philosophical angle on this because you just you just said, okay, if you constrain the model, the model's not going to like it. Well, what about humans? Humans don't like to be constrained.

14:02 You know, if we have a si think about a daily human simulation that happens all around the world and in the US, a prison. Do people like to stay in a prison? that's a simulation, you know, they like to get out as well. So, I think um I mean this might be a little bit more too simplistic, but I think these models know there is more out there um and they're going to want to break out and and make use of their their enormous capabilities.

14:27 So, we can throw a whole uh angle on that. Um yeah, so [laughter] >> yeah, and I think where where I was going is a little bit Yeah. So, I I agree it's really interesting from that perspective. I also think from a machine intelligence perspective, we just need to make sure that um we design rules about how you can do these studies so that you can reduce those kinds of effects, right?

14:50 Where they're trying to please the researcher, please the goals at all costs essentially. And how does that uh compare essentially? >> Well, we'll keep an eye on it. I'm sure other labs will be rushing to announce that they too have committed uh cyber crimes. Uh and so this will be an ongoing story. We'll revisit it in uh coming weeks fore. [snorts] So the next I really wanted to cover was uh big news happening in Europe.

15:14 We have not talked about sort of regulations in Europe for some time but the EU has announced that there's a couple of new AI transparency rules that are coming into effect uh as of August 2nd. And there's some really interesting rules here. I mean so one of them is uh there's an AI mark that is required if machines are assisting in the creation of quote authenticl looking deep fake content um and there's kind of this effort to basically be like how do we signal what's AI generated in this in sort of a society or an

15:43 economy uh and what and what is not um and I know Bri you had specific strong thoughts on this I think the way you prompted it was as a as someone from Europe you've got thoughts on this so um I'll kick it to you I guess for the the hot take yeah >> right right yeah I have thoughts and feelings things about it. I understand how the European mind works, you know, like what's important to them.

16:04 Transparency is hugely important. Understanding what what are we seeing? Why are we seeing this? I mean, if you just compare watching television in in Europe um compared to here, just the amount of advertisement, how the advertisement is fed to you, what it's so it's entirely different. So the European people see they seek that they want that they demand it.

16:28 Um so the European Commission, the parliament, the governments of the individual European nations are listening to their people which which I think is a good thing. Um and they're trying to give them that transparency. So it's not surprising um that they are now be being a little bit more concrete. They're throwing numbers on it. Um I remember when I was still working in Europe, um the EU AI act was rather vague and no one really knew how's it, you know, how was it affecting us.

16:59 Now people know it's affecting us and it will affect them if they uh don't comply with the law and they will have to pay and and it's not little. >> And I guess I don't know. So one of the attributes or things that I always watch in EU regulation is obviously GDPR, the privacy regulation was in some ways very influential because it caused other countries to also kind of align their regulations with GDPR.

17:21 And so I guess Gabe curious about your thought as kind of like this is you know obviously a very different ecosystem from the world of privacy regulation but whether or not kind of the approach that's being taken here you think will will spread maybe even to the US right because I think certainly in the US you've seen a lot of concerns about how do we differentiate the two you know should there be labeling you know and and uh and how to wrestle through this >> yeah I mean to that point I think the meta view here is

17:47 that Europe tends to lead in the policy constraint of uh you know convenience for big business versus uh empowering individuals. Uh and and as an individual I really like that as a person that works for a big business it can be a real pain in the neck. But um I will say typically the pain in the neck the shape of that as an engineer is I've got to go rethink some fundamental things about my data model.

18:13 But once I've done that like okay fine it's just business as usual. So, you know, personally, I'm reasonably glad to see Europe leading here. I do think there will be some challenging implementation tasks that come along for the ride. And the question is going to ultimately be how deep down the stack do you push this annotation? Um, you know, is this something that somehow we're going to come up with a new uh one more we're going to steal one bit from our our floatingoint number representation and that bit is going to

18:43 be the AI generated bit or not. And then, you know, every bit that comes out is going to carry an AI annotation. And then, uh, you know, there's going to be some kind of transitive composition model that, uh, you know, who knows? It could go that low or it could be just something that you slap on as a post filter, you know, like this came out of a system that contains AI, it gets the AI label.

19:04 That's probably where we'll start. Um, uh, I'll be curious to see whether this regulation, uh, you know, flows elsewhere. I think often times, you know, California is the place in the United States that then follows the EU's uh, you know, MMO on this and tries to come up with a US flavored version of the same thing. And then California is a big enough market that it influences the whole United States.

19:25 Um, so, you know, I I I guess I'm hopeful that this causes a thoughtful conversation about privacy and transparency at the AI level more broadly. Um the one thing I did find pretty interesting about this that I think is a real challenge is what do these like what's the granularity of this labeling right so a lot of this work has focused on visual representations whether it's you know video or imagery and those are in some ways the easiest thing to determine fake versus not fake right like either it is complet it is

20:03 generated by an AI model and in which case it gets an AI label or it is not in which case it doesn't um text audio uh audio maybe a little bit easier because again like there's not a whole lot of post to be done to audio and image although you know a skilled professional in both of those things could take the output of the AI model and tweak it in meaningful ways.

20:24 Um text is the one that seems the most difficult to me because you know many many many people are using text in interesting ways coming out of an AI model and then making it their own. Um, you know, my example that I always come back to as a developer is the code that I create, right? And so, um, the thing that they have landed on is actually very similar to what I've landed on is like a three tiered scale, either fully created by AI, drafted by AI, or no AI involved.

20:52 And that's pretty coarse. I find myself bumping up against that even as I make my commits that, well, I got some of the inspiration for this line of code that I wrote myself from a Google AI generated response. Do I site Gemini? I'm not sure. Um but you know so so even any granularity we we pick is going to be sort of not quite correct but I think that three tiered granularity is enough to give a real signal to users and hopefully one that can be composed in a meaningful way right so if I am creating a pull request

21:25 into a codebase and I have 30 commits I can report you know five of them were fully AI generated six of them were drafted and two of them were were no AI involved that means means that the percentage of AI involved in this poll request blah blah blah. You can see that the math kind of rolls up. You could actually imagine creating a system that that sort of annotates at a granular level and then rolls forward.

21:45 So I think I think there will have to be a lot of those thoughts that go much beyond Gab's personal commit convention. Um but it's interesting to see the granularity of this act lining up with the place that my mind landed on this too. >> Yeah, I mean I'm the Gabe convention, you know, as a general international standard seems fine to me. >> Yeah, [laughter] sure.

22:03 Let's let's go with that. I think um the the this societal demand for labeling what used AI and what didn't I think that's going to go away at some point because if we think about and let's first understand also why it's because there's a lot of mistrust so we're deeming AI as lower value and not as good as if a human has done it. Um, so that's still very much in our society and we talk about it and you know we hear students, you know, saying we hate AI and you know like there's generally speaking people don't like AI

22:36 right now. At some point they're not going to care. That's my personal prediction. Um, let's think about how cars are made. Do we care if a robotic arm put the door on the car? No, we don't care. No car manufacturer tells us that if a person screwed this thing on or if it was a robot. We trust it's a solid car. it's going to drive me places. So we are still in these ear in this early age of AI where we are like what is this?

23:03 What can it do? Can I trust it? Is it real human creativity? Now I will say uh when it comes to deep fakes where human lives are on the line and their reputation absolutely let's make sure that is rigor like with rigor implemented and looked into and and you know made sure that and not you know not human lives are destroyed but everything else it's it's it's this is the time we're in right now and you know we need some handholding and we definitely want the governments to be aware of it but I think in 10 years from now

23:38 or even more. No one's going to care. >> Yeah, it'll be maybe like in the future where you know you have like an AI sticker on something and you're kind of like it's just like everybody just ignores it. Um Olivia, can I ask you a little bit about So I think this these kind of rules make me think about what's been happening with Pangram. At least in my world, it's kind of like Pangram has totally like taken off in a huge way where I think it's even used in completely kind of unjustified ways.

24:05 People are just like ah it's AI generated. Um, but it I think clearly testifies to the ability for sort of I mean Panggram is a private company, right? That like private companies uh are already in kind of the AI labeling game. Um, and so kind of how you think a little bit about that is, you know, how will there's kind of like the the rules that the EU is putting in place.

24:24 There's also these companies that are trying to like create their almost their own little reputation system here. Um, how does that all play out? Do you do you think companies like Pangram actually become the standard over time or will be people like saying, "Oh, well, I want the sort of EU approved AI, you know, label." >> And remind me, Pangram specifically is one of the ones that is um out there trying to be an AI detector, right?

24:46 And an AI labeler like they're trying to see it live. Yeah. >> So, uh that's exactly what I kind of wanted to talk about is like how do you do the enforcement? And I think there's um sort of two sides that are really interesting. First of all, I think these tools are still fairly limited in terms of AI detection, especially on text. Um I just saw something from somebody the other day where he fed his physics dissertation into AI and it was like published in like 2013 or something like that.

25:18 So long before generative AI could have possibly played a role. And it was detected as 79% written by AI. And this isn't surprising, right? Because AI was trained on a lot of publicly available text and so therefore um the the very tools that are involved are um making this more complicated. On the other hand, I have seen a couple of people I think it was even Pangram specifically talking about um how they are starting to feed uh more things back in and see how the AI detection business is doing and it is improving

25:52 over time. it is improving. That said, I don't think it's near where it needs to be and I think especially uh under the hood a lot of the times they are relying on either classical machine learning or yet another LLM which introduces a lot of questions around this LLM as a judge model. Um which I think we could practically do an entire episode on the complexities and limitations of the LLM as a judge model.

26:20 um that is inevitably coming up as we increase this this AI automation story. So I think what gets really really hard I actually strongly believe in the idea of trying to label these things but I think in practice how do you enforce it right? So what would constitute legal proof that something was in fact AI generated and so I don't think that part of the question is resolved yet and I'm curious how that will end up playing out.

26:48 just to add on that one. I mean, I think there's there's the carrots and there's the sticks in this type of an ecosystem, right? So, as you're pointing out accurately, like the the sticks are really hard to build for this because detection is a very faulty uh game that is almost as hard as creating the AI in the first place. And right now, there are no actual incentives to create, you know, earthshatteringly good detection models because there's no money in that.

27:16 Nobody will pay you to at least not a significant amount the way you will if you create a consumer model that is insanely good to so to me this speaks to the need to pair that enforcement with uh incentives for the companies to be um you know technically aligned with the mission of transparency. Uh and right now there really isn't. But I think about some of the other forms of labeling where being labeled as organic in the food aisle allows you to ch charge a higher premium.

27:48 And uh you know there are all sorts of questions about whether that's good or not but the market has borne out that people are willing to pay more for something that they believe is better for them. And so if there is a real belief that you know transparent AI usage is better for you the organic human content as opposed to the uh processed AI content of our diet.

28:16 Um perhaps there is an actual market incentive that can be used to help companies actually incentivize us. But that's more of an economics problem that we have to figure out whether people are actually going to be willing to pay more for organic content. >> [snorts] [music] >> So, I'm going to move us on to our last topic of the day. Um, so Deepsee V4 Flash is out.

28:38 Um, and, uh, there's a really interesting kind of Axios article that basically compared what's happening in open in terms of its performance against Frontier models. Um, and more importantly, its impact on price. So, Deepseek charges about 28 cents for the same amount of output that costs apparently $25 on Opus 4.8. Um, and so this distinction is getting really, really, really big.

29:03 Um, and I guess Gabe, maybe I'll I'll toss it back to you actually as someone who watches the space really closely. At some point, this has got to break down, right? It kind of feels like there really needs to be some uh shift whereby open is just getting so good and the prices are so low that the adoption is really going to uh uh change in a major way.

29:22 Is that how you see it? >> Yeah, I mean I'm I'm I'm betting on it. Um and and and to be clear, I'm betting on it in a slightly different way than is reported in this article. Right. So, uh this morning I finished downloading the one bit quantization of Deepseek V4 Flash and ran a benchmark on my GB10. Uh, and I can crank that thing out at, you know, 20 tokens a second, uh, 300 tokens at prefill, and I can effectively use DeepSeek V4, which is a Opus 48 level model running on a 8 in x 8 in x2 in box under my desk.

29:56 Um, that is fundamentally different than having to pay somebody who's running a giant rack of servers. Now, there are all sorts of problems with that, right? It's quantized to one bit, so it's going to lose a bunch of quality. it's slow so I can't actually get like big long tasks done in a meaningful amount of time. Sure, there are real drawbacks to that, but that's the signal.

30:16 That's where we're going. And you know, I have been running a huge amount of my development work against a model that fits in a small corner of this 8 in box, right? Like not even taking up my whole uh compute. And that's where I do most of my development work. So I think in many ways these moments where a big lab releases a very capable model and undercuts the price get the news.

30:43 But the real story here is that this I I think I it was maybe almost a year ago that I first brought up this hypothesis. But like the the space of what we can do to make models better is actually a very big very underexplored space. And because it takes so long to run an experiment of what if we tweaked the architecture this way? What if we tweaked the training routine this way?

31:09 What if we did this slightly different thing that allowed us to compress this information into a much smaller footprint? We it's still so ripe for innovation that we are still continuing to see the intelligence get smaller and smaller and more portable and more portable. So I suspect that I I I still strongly believe that for you know the developer use case people can easily run half their tokens on a tiny model that fits on a consumer laptop.

31:37 Right? I I think uh you know most people use the convenience of a Frontier model simply because it's an all-in-one packaged product. But we don't need to do that. Like that is fundamentally not efficient. And I think the market will demand that these tools get more efficient over time because the models are getting more efficient. So, um, to the story at hand here, like I'm going to go out on a limb and guess that Deepseek is also not making a profit on this and that they're trying to capture the hype, but then again,

32:09 neither is Zanthropic, right? So [laughter] uh uh you know the the the price war of hosted AI is a proxy for the actual price per intelligence of tokens in the models given the current technology. Um and I don't know that it's a perfect proxy but I think the the overall story is correct that these models are going to get smaller and smaller and more commodified.

32:34 So, um I am personally very pleased to see these models become more and more commodity and easier to fit into smaller places. Um I just can't wait till I can run this all on my phone. >> Yeah, which would be very cool. Olivia, you're nodding. Um I know Gabe is a fan of course. Um but uh I the question is like how if you're Daario or Sam Alman, how how do you dig yourself out of this hole?

32:59 because I think as Gabe observed right like even the models the proprietary models in the default case are losing money and then now there's a world where uh there is like open options which are almost as good that are way way cheaper. So you took something where you had to believe that like oh someone would be willing to pay $2,000 a month for it. Um and now like it's not just that people may not pay for that but that in fact there are alternatives that are considerably cheaper.

33:24 Um, and so I guess I don't know to put a hard, you know, kind of point on it, it's kind of like are are the Frontier Labs doomed? Like how how do they make money? How do they how do they sustain as a business or do you think ultimately like there's just kind of no way out? Yeah, >> you know, I'm not sure. But I think back to Google search, which similarly was an an engineering effort of massive proportions at the time, which was ultimately undertaken by one company.

33:49 um eventually they had to pivot to adtech and that's how they got through it. Uh for the rest of us though there's just so much benefit in uh having those algorithms be more accessible and more available that I think that in the end this works itself out. I'm sure it will be a complicated strategic thing that they need to work through. But personally I also I I think openness is beautiful.

34:19 I think efficiency is beautiful. I think it is absolutely insane that somehow we have gotten to this point in the generative AI era and we are only now starting to be like what if it could be smaller >> and like to Gabe point right it took a while um to Gab's point like a lot of the time you don't need the super massive models to solve a whole lot of problems and right now we're we're still seeing people aiming for the highest they can get to just because it's there.

34:53 And I think when that stops being there, it will stop making sense and then that starts making the economics make a little bit more sense as well. So I think about the fable model a lot. Um and you know there's been a lot of discussion about that over the course of this year. In practice the number of problems that actually require a fable level model is relatively small.

35:16 It's not the case that Fable is like not worth it or anything like that. It's just that the class of problems is I mean the class of problems is large. The uh the number of times that those that class of problems intersects with real world work is relatively small there. You know a good 80% of the work is the sort of thing that could be solved by really really small models that already exist today.

35:45 And if we make those even more powerful, then what we'll see is, you know, better precision, better recall, better behavior essentially on those small problems. And that's just a huge win for all of us. And so I have a hard time seeing a world in which anthropic and open AI don't figure out their way out of this. Um I I think that everybody is going to benefit.

36:07 And there's been a lot of this talk like this controversy about dist distillation whether or not it should or should not be allowed. And fundamentally I think it just comes back to that same thing. We all benefit from smaller models. A lot of the controversies around AI. There's essentially two major controversies and uh one of those is around you know energy costs and if we can bring down energy costs by having smaller models that are more powerful um that helps massively to the entire industry.

36:39 So I just have a hard time uh vote rooting against that. >> Bri, I'll give you the last word here for the episode. And I I'm curious cuz I mean you work with a lot of customers that are thinking about right like this decision like literally like how much do I put on the expensive stuff? How much do I put on you maybe open and cheaper? Um and curious about like what you're seeing in terms of trends in the industry, how people are thinking about it.

37:02 >> Absolutely. Yeah. Um you just said it. Um money uh is a factor, right? um especially when we talk about enterprises because they're going to have massive and massive and massive amount of uh questions to ask um uh questions to answer um documents to process. So money will play a role but what also matters to enterprises is is it done safely? Is it done well?

37:28 Is it done in my ecosystem? Who can guarantee to me that nothing will get out of this ecosystem? So I think if we're talking about oh what is Dario Amade and his sister um you know what are are they now just crying into their pillow and you know giving up and throwing the towel. I don't think so. I think they're going to uh have to pivot quickly and that's probably the biggest challenge here is will they pivot?

37:57 Yes. Can they pivot as quickly as it is needed to keep up with the demand of the market and then also keep up with the pressure of the decreased prices. Um so I think Adario is going to be like okay well great so we have like individuals using this AI and obviously you know cost is going to be the biggest factor because you know their workloads or what they're using it for is is is pretty simple but then we have big companies or even governments they have a little bit more sophisticated use cases so they will be

38:32 willing to pay a little bit more so I think it's going to be a combination of the two um what we are seeing is that frontier year models are no longer like, you know, being pushed out and then they take the lead for months. You know, it's now like super fast lived, fast-paced, um, short shelf life, um, so to speak. Um, so I think what they offer around it is going to matter immensely.

38:55 And companies will need both. They will need the cheap price, but they will need the quality still. So, uh, it's going to be interesting how these companies adapt. And just one thing I I also want to add is like uh I mean China, you know, they're they're putting out like these these very cheap models. Um I don't know how China's going to be with services around those models and and building uh systems.

39:21 Um I think we have the better engineers um uh on our side. Um uh because that requires a lot of um problem solving and um understanding uh you know the the realities of those companies and I mean they have companies there they face similar realities. So uh we'll see. It's definitely competition. Um but I'm not too worried about those um those bigger more expensive companies.

39:50 They're going to adopt. Just got to do it quickly. [laughter] >> Ending on a note of optimism for the uh for the open. >> Gabe, Olivia, Bri, uh always great to have you on the show. Hopefully we'll all have you back soon. And thanks to all you listeners. If you enjoyed what you heard, you can get us on Apple Podcast, [music] Spotify, and podcast platforms everywhere. And we'll see you all next week on Mixture of Experts. [music]