Microsoft’s new AI models & bots dominate the internet Transcript, AI Summary & Key Points

00:01 Bots have outnumbered humans on the web for quite a long time, right? Web scrapers, different things like that. A lot of people are interacting for research tasks through their AI agents these days, and I think that's what they're capturing here, rather than people going directly to a site. All that and more on today's Mixture of Experts. I'm Tim Hwang and welcome to Mixture of Experts.

00:25 Each week, MoE brings together a group of the sharpest thinkers working in artificial intelligence to walk you through the week's news. On this week's episode, we have Ambhi Ganesan, AI Transformation Leader, Sandi Besen, AI Engineer, and Rynne Whitnah, Lead AI Ecosystem Engineer. All right, we've got three big stories today. We're going to talk a little bit about Microsoft releasing its own models.

00:46 We're going to talk about a nice report coming from The Verge on the Tribeca Film Festival. But I want to start by talking about bots on the internet. Kind of an interesting story that popped up on our radar. It was first reported on CNET, but I think it went around a couple of places online. But specifically, it was a study that suggested using Cloudflare's tools, which really is, I think, kind of like a pretty representative set of the internet at this point.

01:14 And reporting really now that generative AI bots generate 57.4% of web requests globally, with humans accounting for just 42.6%. Maybe I'll start with you. I saw this and I was like, I thought this was going to be a few more years off, but it sort of feels like now the internet's already becoming mostly agents. I don't know, are you are you surprised by these results at all?

01:36 Not really, but that's also because this is very much, there's a lot of asterisks on that number. Right? So, for instance, bots have outnumbered humans on the web for quite a long time, right? Web scrapers, different things like that. That's always been a big part of, you know, data being vacuumed up by the big AI companies, search and search engine indexing, all of those different pieces.

02:01 So, like, automated requests on the web are a pretty big part of it. And I think this is also direct requests to websites, which is a lot of people are interacting for research tasks through their AI agents these days. And I think that's what they're capturing here, rather than people going directly to a site. So Google does this, right. When you go to Google now, you get a summary back of the top pages.

02:29 We're probably seeing some of the AMP traffic drive that down to where they actually serve that traffic themselves. So I think a combination of all of those things makes this make sense to me. But I, you know, it's definitely, a bit of a shift, but I don't think it's a seismic one. Sandi, it looks like you might want to jump in. I mean, I guess, I don't know, maybe.

02:49 Maybe I should not be so worried. I saw this and I was like, oh, yeah, I gotta make it the top of this episode. But I think to your point, you know, maybe we've just lived in a mostly robot internet for a long time already. And I think Rynne's spot on with that. The UX is shifting, like the way in which we interact is shifting. It doesn't mean that web scrapers were happening through the, the the catalyst was the human, right.

03:19 The catalyst was a human that sent off the web scraper to go scrape something. And it's kind of the same type of UX. But at this point, if you search something through your research agent, it's hitting like 20 searches, right? It doesn't mean it's interacting with all of them in the same way. It doesn't mean the engagement is the same. But we're even seeing the format in which information is served change in order to account for that.

03:49 So, for instance, a lot of doc sites now have like an LLM text file so the LLM doesn't have to go and parse like the HTML, but instead it just can read the LLMs text file. So I think there's an element of that as well, the easier that we make it to provide context to these agents, the more they will consume. And, and I don't know if that's a, a bad thing necessarily, because there will always be a space for humans to communicate.

04:22 It might just not look like it does today. Well, I think that's it. And I mean, maybe you've got some thoughts on this because I think, you know, Rynne, I think your point is well taken and I'm calming down now. I think my blood pressure is going down. But, you know, I think the kind of I think the study is sort of picking up on certainly something that I think people are maybe concerned about, but I think it's certainly true.

04:37 Right. Which is like a world in which we mostly interact with resources on the internet through an AI system, is a world where at Sandi's point, you may not even need to write websites for humans anymore, right? We just have to make them kind of widely accessible to LLMs. And so I guess, I mean, do you want to think maybe it's just pie in the sky? Kind of like where where all this kind of goes.

04:58 Like, will we live in an era where I kind of never go to a website anymore? I just have sort of AI agents and I kind of interact with it purely through this interface and sort of what we remember as like the old school web. And by that, I mean, I guess the 2000s is maybe no longer around. Yeah. So if you think about where all of this is heading, there's broadly two flavors of impacts, right, or impact areas.

05:20 So one is how do you serve up ads. And then the other is the broader agentic e-commerce, agentic e-commerce, ecosystem or workflow, if you will. Right. One is more of how do you serve up information to humans, right. And then the other is how do you help humans get some tasks done, right, which is, you know, and a lot of this that we are talking about is still in that first category, right?

05:51 Whether you have the, the scrapers, the bots, the agentic bots, right now, all of this, like Rynne was just mentioning, right? It's all in terms of how do you serve up information for humans ultimately to consume in some sort of a distilled manner? Right. There is still the whole promise of how do you get to an agentic e-commerce, which I think this is still the the underlying substrate on which we'll build that.

06:17 Right. I go and shop on websites, right. So I have to go and let him shopping for a shoe, and I have to browse through ten different shoes. And, you know, I know my, personal taste. And I look at the pricing, I get some offers, I put something in the cart, I ship out. And I'm already tired. Right. The the the holy grail problem is, is that, hey, you know, there is a world in which agents can go and do that, right?

06:45 And we've saw we've seen some of those glimpses of it with Perplexity and some of the others saying, yes, we can go build it. We obviously haven't reached there yet. I think that is still a couple of years out. So all of this that we are talking about, the bots taking over the internet is not in that category. Right? It's still in this first category of how do you serve up information for folks to consume?

07:10 You know, you go on ChatGPT, you go on Claude, you go on Perplexity, whatever it is, you're looking up for information, and then the bots are going to go and scrape the websites, crawl the websites, get the information distilled, and then give it to you. You're still the person who's still deciding, which offers you want to take and then go shop around.

07:27 Right. That piece is still happening. Coming back, right, there's there's two impact areas. Like I said. So in this particular focused area, right, the the immediate impact is on the ads. Right. So how do you serve up ads is I think is going to be a very, very tricky. And that's a very fast evolving situation with, with, you know, if you extrapolate this trend, you know, you you keep going from 57% to 70%, 80%, whatever.

07:54 Right. Now, how does the ad ecosystem evolve? Right. So yes, there is a question of, well, should websites exist, should every website just have text? And I think you can you can take it to that, extrapolated state. But then the real commercial impact is what does this mean for the ads business? I don't think we have a clear answer on that yet. Right.

08:16 I think we all crystal gaze, crystal ball gaze at it. But there is a serious arena where, things are things are going to have a profound impact. I think, just to extrapolate from that, the reason why the ads matter is because a lot of the content on the internet that we care about, like news and journalism and things like that are funded by advertising, right?

08:39 They have to be, so that the model for that has to change. And we've seen a lot of explorations of, you know, pay to access and things like that for premium content. And, you know, I don't know where that's going to land. Right. And I think, I think figuring out a modern funding model for how does an agent give back to the places where it's getting content from is not something that's figured out yet.

09:08 Yeah, for sure, because I can imagine even like I mean, the most extreme version of this is you kind of pair up like the current agent access with, you know, agents being able to kind of code on the fly. And you can imagine, like, you actually live in your own internet, right? You're like, I wish there was a website that did this. And like, it kind of just compiles with the resources necessary.

09:26 And how the ad ecosystem works in that kind of universe is is really unclear. Sandi, did you want to jump in, any predictions on how to how to save news media? Well, it was more on what Ambhi mentioned in terms of like, having a split between AI native browsers and AI native web and the web that exists today, that we're trying to, to, to jam, like extensions and plugins into in order to make that happen.

09:56 But I, I wonder how that the, the advertisements, the, the consumption of information, how it's displayed, how it's, ingested changes for both of those and whether they converge or whether they remain separate. And, and one exists in one place and the other exists in another, or they both exist in both, and there's just preference as to how the information is accessed.

10:29 Yeah, I think time will tell on that and how it evolves. But, it's an interesting thing to think about. Well, I'm not worrying just yet, but we will keep an eye on these stats as they go. And super interesting. And I appreciate you all kind of talking me down from from the ledge on, on the study itself. I'm gonna move us on to our next topic. This one was sort of an interesting story.

10:54 We have not, you know, admittedly, on MoE, by and large, talked about Microsoft in in quite a while. You know, I'd say about early last year, there was a lot of activity and a lot of discussion about what they were doing, but they've gotten a little bit quiet. And so this story was actually pretty interesting to me for their big developer event. Microsoft's kind of announced really for the first time, I think that they were really kind of doing their own foundation models and announcing sort of that they were releasing

11:16 a set of them. The two really kind of headliner models they were releasing. One was called MAI-Thinking-1, which is a 35 billion parameter model, and then MAI-Image-1, which is basically kind of an image generation, sort of model that they were kind of putting out there. And, you know, I guess maybe, Sandi, did you want to take this kind of first, as I was sort of interested.

11:38 Right. Because I think early on the idea was, okay, Microsoft was in this alliance with OpenAI. OpenAI was going to provide the models and Microsoft was going to provide the infrastructure. But now we have them kind of touting their own first party, you know, models in the space. I think they're trying to capture a certain part of the market that OpenAI doesn't seem as interested in at the moment, although there's lots of efforts at OpenAI to, talk about safety and regulation and guidelines, and they do work with the

12:13 different governments to kind of monitor policy around AI as well. There's a hole in where IBM also tries to, to compete in it. Right, which is in safety and in something that has been regulated. And, we can count on the information being used in the models being clean in a way where it's not distilled from another model. The data, the data sets that are used for pre-training are all licensed.

12:50 And you don't have to get into the big legal battles that you would, especially if you work in a litigious industry, which to be honest, every industry is a litigious industry now. Many, many, many such cases. But but especially thinking about those industries that are the most concerned and the least, or the most risk averse, like the legal industry, accounting, anything that, has a history of being litigious.

13:20 They're very careful around, how to use AI. And this gives them an extra safeguard that they're not going to be sued because they're using copyrighted information or information they shouldn't have access to. Rynne, do you think this makes a difference? I think, and, you know, I guess I'm trying to look for kind of what the the bull case is for Microsoft here.

13:40 Right? Because it's sometimes so easy to be like OpenAI and Anthropic are so far ahead. How can anyone ever catch up? And you look at these models, and they say, oh, 35 billion. What's a small model and all this? But here's about what if Sandi's mentioning is kind of an edge in the space? Yeah. So it is one clarification though. This is a trillion parameter model that does mixture of experts down to 35 billion.

14:03 So so so it's a very, very like it's a much larger model that has many different versions of that that it can activate based on the task. The other thing you know about this particular model is right. For a while, Microsoft had a pretty exclusive partnership with OpenAI, right. That that was very much a thing where, they were the exclusive compute provider and other things like that, and they've both outgrown that relationship.

14:32 But, you know, when they renegotiated that, that was obviously like they were seeing that that was going to change. And they needed to have their own thing too. So absolutely, the indemnification aspect is super, super useful, right? You know, we've leaned into that for a while, but, it's honestly great to see more people actually leaning into this idea of clean data going into these models.

14:56 I, I love that, right. You know, we know that the people who generated that data got paid in the original, and that's, that's super cool, actually. So I absolutely agree that it's a differentiator. I think this is interesting because it is a very large mixture of experts model. So you actually think that this actually is. Yeah. And sorry I, I misread it.

15:17 You're absolutely right. I think this actually is kind of like frontier in a real, genuine way for what Microsoft's doing. Sort of. So I would say a frontier model is going to be many trillions, right? This is this is somewhere in between there where this is a very solid thinking model that for certain agentic tasks can do a really, really good job.

15:40 You know, on some benchmarks, I'm sure it'll be competitive and then some benchmarks. I'm sure it won't. But we'll we'll kind of see where it lands there. It's an interesting experiment as well in terms of are people willing to give up SOTA or sort of state of the art models for something that is, safer and still performant, but maybe not as performant in certain areas?

16:04 But I just wanted to, to drop that in there. Agreed. And one of the things that's always interesting about cost optimization with models on this is there's kind of two schools of thought where you use the cheaper model and then you go to the bigger one if you need to, or you try and one shot it with the more expensive model so you save the person's time.

16:24 And I think both are interesting approaches and I'm not quite sure where we've landed yet. Yeah. And I think that's actually getting to. It's great because it tees me up for the question I was going to ask Ambhi. Was basically like, you know, typically the way this discussion goes is like, well, do models really matter anymore? They're becoming more and more commodified, open source is getting so good, blah, blah, blah.

16:42 But I think the more mature way of thinking about it is, what you're mentioning is, like, you see these architectures now, which is what we we either try to do the fancy thing first to see if we can get it done in one go, or we try the smaller models and then if we need them, we use the bigger models. And I guess I do want to, I'm curious if you've got thoughts on kind of how how this all kind of evolves overall.

17:02 And obviously there's lots of edge cases and nuances here. But really I think in this balance between the the biggest, baddest, most expensive model and maybe the more open source, like still quite significant, like 1 trillion parameter model. You know, I think I'm curious about how you think the usage is going to kind of evolve over time as people really kind of get into, I would say, the next stage of maturity on this stuff.

17:24 Yeah. I mean, the first piece is always choice is good, right? So more models is, it's always good, right? It gives folks, choices in terms of which models to pick and the more the merrier, right. That overall yes it is. You know, you can view it as a little bit of that commoditization, which also drives down the costs eventually for the, the ecosystem, for the industry as a whole.

17:47 Right. So the cost pressures, yes, there are going to be some constraints from the compute like we've talked about before. Right? There are going to be some physical limitations that are that in the short term will impact certain, you know, ceiling prices. But overall I think, you know, we'll see the the costs go down as the choices become wider. Right.

18:12 Rynne is correct. It is a frontier-ish model, I would say. Right. It is on the caliber of the O series models of OpenAI. And, you know, you're probably like a nine months to a year, probably behind the current state, which is still pretty good for an overwhelming majority of the tasks that you may want to do. Right? Not everything needs a, a 5.5 caliber model at X high.

18:43 Right? So, so the choice is good. I think the cost will definitely come down. I'd be very keen to see how Microsoft is, driving the pricing changes of the pricing models for this category of models compared to, say, hosted OpenAI models on Azure. Right? So how that shapes up and how that trends over time would be a very interesting pattern to see. Given that, you know, they are developing the model as well as they are hosting it as well as their inferencing.

19:19 So, you know, hopefully that verticalization gives some amount of cost economics for them to do that. I would be very keen to see how that trends over time. Right. And I think it's coming at the right juncture. Cost is on the top of minds of a lot of enterprises. Given all the recent snafus that have happened this year with, you know, tokens blowing out of budget and, you know, organizations just blowing through their budgets for tokens.

19:46 Literally going bankrupt because they spent too much on tokens. Yeah. So it is on top of every organization's mind. And so I think, you know, this playing into that is probably, you know, I think it's coming in at the right time. Like I said, it would be very, very, would be very curious to see how the, the cost trend shapes over time. Right. It is, you know, to Rynne's point, I don't think it's a settled case of, you know, should you go for whether, you know, you you just do a one shot versus you do, you know, a

20:16 series of different models. The, a good chunk of prevailing thought right now is, you know, let's go to some sort of a model routing. Let's go figure out, you know, there are, you know, for sure there is a lot of different queries and tasks that are low fidelity, low effort, that you just really don't need a frontier model. Right? You can definitely do it with, much less capable models at a, a smaller cost.

20:53 Right? And that piece is known. There is a good prevailing thought that, okay, you should probably do some sort of a model routing and pick, a mixture of models to do a task. Right. So I think this fits well into that equation overall. Right. This raises, I think, kind of a really interesting way of sort of dividing the market, I think where, you know, it feels like there's kind of be a strata of customers for which like kind of cost is no object.

21:17 Right? We have all the money in the world we're going to get, you know, the, the most frontier, most SOTA kind of model we can. We will we can spend an amount of money that would bankrupt other people in terms of tokens. It kind of doesn't matter. Right. But like, that's like a that's valuable but kind of a tiny, small slice of the market. And then there's this, like, vast median of customers for which you know, Ambhi, to what you're saying, like cost really does matter.

21:41 And it feels like one, one competition here in terms of who wins that middle will be who can deliver at the lowest cost. I guess maybe one thing, just because of, I think the folks on this panel, you're all ideally positioned to answer this is, does everyone have thoughts on like what the other levers are to kind of win in the space. Right. Like I think, Sandi, you pointed out one, right.

21:59 Which is well, maybe it's safer, right? This is maybe one reason why you might want to use this, because you might not have a team internally that you can just stand up to deal with the safety of an off the shelf SOTA model. I'm curious if there's other things that people kind of think of, and it looks like you might want to jump in, but I'm just kind of these kind of factors of success is like pretty different when you talk about this middle versus these like cost is no object type customers.

22:22 Yeah. So there's a lot of different ways to save costs on AI. And I'm going to mention one that's kind of rarely spoken about because I think it's interesting. So when I started programming, I was actually doing so in Ruby on Rails, and Ruby was designed with a very unusual principle for a programming language, which was it doesn't matter how hard the computer's job is, as long as the programmer's job is easy.

22:51 Because the programmers are more expensive than the computer. So that was how it was designed. And, you know, those economics are changing a little bit with some of these frontier models, right? Like, we're seeing we're seeing just the speed at which you can burn through tokens is kind of remarkable. But I also think that, like, you know, as Ambhi was saying, you know, sometimes you do need to route to the big model, right?

23:15 Sometimes you do need to route to the state of the art, because you're doing something difficult, and coming back and redoing it later is much more expensive than doing it right the first time. There's a whole bunch of other, you know, pre compilation we've got, you know, probably a hundred different techniques for saving tokens, like, we've seen the caveman skill.

23:35 We've seen all of those very silly techniques around, let's talk to it in single syllables. But, you know, I, I actually think where we're going to land on all of this is that there is a middle ground and it will be on whichever one does the best cost savings while making the human adapt the least to that workflow. I think that's that's really interesting.

24:02 And yeah, I never really thought about it as like how expensive it is to use the machine versus the engineer, but, absolutely. A big factor in the space. And, Tim, I'll tell you, I know you mentioned, you know, there's a tiny, I can guarantee every major enterprise is cost conscious, right? Every client of ours that I talk to, this will come up as a conversation.

24:20 You know, everybody has a budget. It is definitely, it is definitely a consideration. Right. The the fraction that you're saying you may just throw caution for the wind is going to be a very, very microscopic minority of VC funded companies that might do it, right. But the overwhelming majority of the Fortune 500 companies definitely care for cost consideration.

24:45 But I think where you are going is it's not just a just a cost equation. It's a cost and safety equation. Right? Because we have had choices before, we have had the choice of, okay, let's go and use something like, you know, DeepSeek or a GLM, you know, can we go and use those? And then in, you know, in regulated industries or even in non-regulated industries, you know, you're going to go through vetting of the models.

25:13 You're going to go and look at the lineage of the models. And are you, or, you know, are you good to use those models? And, you know, you're weighing the pros and cons of the open source models that have a clean lineage versus open source models that have, you know, that are that are matching frontier capability, but they have come through a distillation mechanism, right?

25:40 So you've always had some of those constraints, which I think Microsoft is trying to wedge in a sweet spot over there saying it's completely trained and we can reach almost frontier capability. And by the way, it can be cheaper, right. So it's it's an and and and it's sort of how I think they're trying to carve out a sweet spot over there. Well, we'll keep an eye on them.

26:00 And I promise to report on Microsoft more regularly because as it turns out, they're a significant company in the space. And so more to come there. And we'll keep an eye on the Microsoft AI team. All right. Next story we're going to talk about kind of changing gears quite a bit. I read this sort of interesting review of the Tribeca Film Festival from a reporter named Charles Pulliam-Moore at The Verge.

26:26 And it's a fun article. It's kind of worth reading. You know, it's just sort of some kind of domain of AI implementation that I have not been keeping my eye on. And this is specifically the world in which people are using generative AI to submit films for film festivals. And I thought this was kind of quite striking because, you know, the reputation I have for AI video is like kind of negative, right?

26:44 Like I think people are like, ah, the most slop of AI slop content is short form video. And so the notion that people are really trying to take a crack at, you know, doing real kind of critically recognized art at a film festival with AI is pretty intriguing. And, you know, Sandi, maybe I'll kick it over to you first. I think, you know, this reporter's kind of view of this is.

27:12 Well, the jury's not not out just yet. Right. That basically that there are some films that really seem to be, you know, kind of rough and some films that seem to be kind of getting to the edge where you could kind of kind of squint and take a look and see them becoming, you know, much more critically acclaimed in the future, something we would recognize as something that kind of quote unquote, belongs in a film festival.

27:35 I'm curious what you think. Like, I don't know, in five years time, you know, ten years time, do you think we'll see films that are completely gen AI that, like, win a critical award? There might be their own category for it. Even, I could see the industry especially kind of separating things out and being like, this was not touched by that. And this was.

27:55 But I think the reality of it is no humans involved. Right. But I but I think that the reality of it is, is even in the movies that won Best Picture today, there are lots of elements of AI, and that seems to be really forgotten when we are talking about AI's intrusion in to the film industry in general. So much of CGI is AI. And if we look at movies that were produced, 50 years ago versus today, unless you have a special niche and you absolutely love the old, you know, the the old effects in those movies, I think the

28:38 majority of people would agree that they prefer the CGI that we have today, where you can see things like, dinosaurs coming out of, I don't know, the sky or things like that. And it doesn't look like a big sock puppet, right. And so I've gone a little bit on a tangent, but what I, what I'm trying to say is that, I think that there's a difference between using it as a tool, as almost like a developer tool.

29:16 It is a developer tool, right? And saying, hey, I want to use this to refine this artistry in this work and saying, I'm just going to put three sentences into some prompt and something's going to pop out the other end. And I can only edit it by giving it another sentence of what to fix. Right. I think there's going to be a big evolution, and there's going to be a big split in.

29:47 We've started here because that is the in some ways most impressive way to start because it's like, oh my gosh, I can do it all by itself. But what we've realized is maybe all by itself isn't, what the majority of people in or out of the industry prefer. Maybe it's a, it's a, it's a mixture of both, a mixture of experts in a different way. If you will.

30:09 Yeah. And I love this episode has been me being like, isn't this cool and crazy and new? And then all three of you are like, actually, now this is like the way it's always been. I guess we're in. Any thoughts on this? Like, I guess I take the point that, you know, maybe we actually shouldn't see this as a new kind of category in some ways. I mean, certainly I think in filmmaking people are debating what's the line we should draw, right?

30:32 Like, should we create a new category just for this? But it's hard to know where where that line should be drawn. Yeah. So to Sandi's point, there was, I had a mentor at IBM, a guy called Grady Booch, who said once that the the entire history of software development is that of rising levels of abstraction, right? Where you where you level up, the tools you're using, and you can do more with them at once.

30:57 And there is some aspect of that. I also do think, we are seeing a pretty big backlash still. Right. Like to a lot of these models that are attempting to create what would traditionally be known as art. Right. And I think, with that, that's a really interesting area where, you know, for instance, I recently actually, I'm at a conference right now, and I wrote my own DJ software and picking where to put those lines because I wrote the DJ software with AI assistance.

31:36 But choosing that, you know, I made the explicit decision that I didn't want it to replace me as the DJ. I wanted to still be able to do the fun parts. And like, the art form of DJing is all around, like, what mistakes do you make? And how do you work that into the set? And what choices do you make on the fly? So, I kind of wonder a little bit about, you know, is it the same art form?

32:00 And I'm not saying you you can't make something impressive with it. I'm not saying you can't do something really cool with it. I, I do wonder if it is, exactly the same thing. Like the, the the format. Right. The delivery format might be the same, but I actually wonder if we are creating almost a new a new class of experience. Yeah. Is that where you think it will kind of, you know, kind of shake out in the end?

32:25 Because I think that is that is part of the stress that I think people have about this is you're saying this is a film, but the back end here looks nothing like what I know filmmaking to be. And, you know, it's disturbing to people. I think it ticks people off for sure. I think both of those are true. Rynne. I like the word art form. I think there is there are there's both going to be a new art form, and there is going to be an evolution of the existing art form.

32:51 Both of those I think will happen. What I think most people tend to focus on is the first one, right? Oh, you know, there is going to be a new art form, which I think is true if you think back to the early, early 20th century. Right. Like, you had surrealism pop up as an expression form. When movies came into the picture, you realized, oh, you could do cool things with the camera.

33:16 You could juxtapose different images and then create completely, you know, mind bending tricks. And you could create very surreal images and surreal forms of storytelling. Right. So the storytelling medium, that storytelling art form, I think is going to open up into a completely new dimension, which might not be just the AI generated videos, right?

33:40 It's likely going to be a combination of AI generated video form plus world models. Plus maybe, you know, you you add in the the metaverse aspect of it. Who knows. Right. So I think there is some shape taking. I think we're going to see a Cambrian explosion of different types of, art forms. And then some of those are probably going to win out there right in, you know, combining generative videos plus world models in some shape or form.

34:09 I think this is sort of being seen as an early precursor to that. Right? If I just, logically just do an experiment of letting AI generate videos with very minimal steering and guided, then how does it lend to, doing this? Like, you know, Sandi was saying, right, you're going to have very surreal types of images and motion scenes that you can put together that just completely don't make any sense at all.

34:38 And that might appeal and that might become an art form. Right? I think we saw some glimpses of it. Even when the Sora app was live, like, you could see a whole bunch of really zany type of art. If you saw, I know it got kind of a bad reputation, but it was like one of my favorite, you know, products. You know, I hope I hope eventually the compute justifies bringing it back.

35:00 Ultimately. Ultimately. Yeah. Yeah, yeah. So so, yeah, I mean, in a long term horizon, I think that'll happen. The second big piece, which a lot of us probably don't think about, is the evolution of the the existing art form. So you still have movies as they're made, you know, as you view it today, as you have viewed it over the last 20, 30, 40, 50 years.

35:21 That video format, right, the way you make it is going to change. And that's what is changing, right? Like, I mean, when the likes of Scorsese go and talk to Black Forest Labs and think about how to do it, I think there is you're seeing some the parallel that I see is, again, with the, the software engineering world, right. You've had multiple different personas.

35:42 You've had product managers and you've had developers and, you know, you've had to go and figure out product managers have some sort of a vision in their mind, and then the developers have to go and somehow translate it into tangible output. You, you could sort of find a parallel in the moviemaking world as well, right? You have the directors with their vision in their mind.

36:04 You convert it into some sort of a storyboard, and then you've got the, you know, the, the actors and the, the photographers and everybody else trying to make meaning out of it. Right. That workflow, that equation, I think is getting much, much more accelerated because now you can, the director now has a very strong impetus to get their vision into a hard, tangible output very, very quickly for someone to go and then take and then polish out of it.

36:36 Right. So I think we're seeing some of those parallels between the software engineering world and the moviemaking world, where I think, you know, your your time from germ of an idea to some sort of, a quick, tangible output, I think is getting very accelerated. And then you still have the craft. I mean, folks who know the craft, right? That piece doesn't change the way you tell the story, the way you express yourself.

37:00 That piece doesn't change, right? Like, you can't take that out of the equation for the movie to be a success. So that story taking that, that craft piece I think still exists, those who know how to wield it and who can apply their craft to fashion a good story out of it that I think will exist. But now, you know, they just have one more tool to accelerate from the germ of an idea to something that is tangible.

37:24 I actually gave a talk on the craft of software engineering recently. So, I, I agree with most of that. Right. I would say one of the things we've noticed is we've there are three phases of like traditional software development, right? You have a planning phase where you're figuring out what you're going to build. You have a building phase where you actually implement it, and then you have a review phase where you make sure it's good.

37:45 And with these tools, we have compressed the build phase down to something much shorter, right. That goes much quicker. And then there's this weird thing as humans where we think we can also compress the planning phase because because the iteration cycle is so much faster, and then you end up with a review phase that I call the wall. So, I think, you know, one of the most difficult parts about working with these tools is the same problem that we've had working with people for a very long time, and that is alignment,

38:16 right? You have to have a shared understanding between not only all the humans involved, but also all of the agents involved about what you're going to build and why, to reduce the complexity of the review process at the end. Because, you know, I've seen videos of people using like generative AI for concepts, and then they have to go clean it up for hours, right where there's it's kind of a question of where do we want to do the work there?

38:44 And, I view actually doing the review piece is the most infuriating part of the work. So the more you can actually move that up to the front side and do a good job of planning and a good job of understanding your artistic vision for the thing you're going to make, whether that's code or anything else. You're going to save yourself a lot of pain in the long run.

39:04 Yeah, I love the idea of that. In the future You know, movies will take like one hour to create, but it will take like a year or two to get edited and cleaned up at the end. So. Well, this is great. Incredible discussion. That's all the time that we have for today. Ambhi, Sandi, great to see you, as always. And Rynne, hopefully we'll have you back on the show at some point.

39:21 And thanks for joining. All you listeners. If you enjoyed what you heard, you can get us on Apple Podcasts, Spotify and podcast platforms everywhere, and we'll see you all next week on Mixture of Experts.