Help enterprises select, integrate, govern, and operationalize multiple AI models rather than committing to one model provider. The consultancy combines provider partnerships with forward-deployed engineers who adapt models to client-specific use cases, data, geography, legacy systems, and regulatory requirements.
Behind this: 11 build steps · 6 tools and how each is used · how to validate demand · 4 more real examples · 5 things the video never answers.
Provide a unified layer through which an organization manages models, agents, MCP servers, tools, and related AI traffic. The gateway routes requests to the most suitable provider, applies access and spending controls, caches requests, exposes observability data, and helps enterprises manage the total cost and risk of their AI fleet.
Behind this: 12 build steps · 6 tools and how each is used · how to validate demand · 4 more real examples · 5 things the video never answers.
Anthropic Cursor Granite Granite models Mixture of Experts OpenAI OpenRouter watsonx Governance watsonx Orchestrate Watson X Orchestrate
Searchable transcript of Stripe buys OpenRouter, Ramp’s AI Index & IBM’s OpenAI deal — IBM Technology (35:52). Search for a phrase, then click its timestamp to jump straight to that moment in the video.
Captions sourced from the original video on YouTube, published by IBM Technology. The video, its captions and all related intellectual property remain the property of their respective owners; AINotes claims no ownership. Provided for research, accessibility and search — see the Transcript Notice and Copyright Policy.
00:01 Yeah, I mean you know this this market is you know eban ebbing and flowing you know and you know we just have to trace where the heartbeat is coming from right and it seems like that u right now the heartbeat is around routing all that and more on this week's mixture of experts. I'm Tim Huang and welcome to Mixture of Experts. Each week, Moe brings together a panel of technologists working at the frontiers of artificial intelligence to lead you through the week's news.
00:28 On this week's episode, we have Koutar El McGrowi, principal research scientist, AI native systems, Aaron Bachmann, IBM fellow, and Mihi Crevetti, CTO Watson X orchestrate. Welcome to you three. We've got a bunch of big stories to follow on uh today. We're going to talk a little about Stripe buying open router. We'll talk about some really interesting business data out of ramp.
00:47 And then finally, we'll talk a little bit about AI for legislation uh in Congress. Uh but first, I really want to talk about one big announcement coming out of IBM itself. Just this last week, there was a big announcement that IBM was going to launch uh a big partnership with OpenAI. Uh and this is sort of interesting. It caught my eye just because uh not too long ago, IBM had also closed a very large partnership with uh Enthropic as well.
01:15 And specifically the partnership pertains to essentially training up large numbers of people to uh facilitate basically an open AI practice uh within IBM consulting. And um I guess Aaron maybe I'll throw it to you first. You know I'm kind of interested in how this kind of world of consulting is evolving around AI. Um because normally I think we've thought about like oh okay it's going to be open AI versus enthropic and it's just going to be you know the sharks and the jets.
01:42 Um, but this kind of partnership really almost suggested a a different world is going to emerge where it's not going to be so black and white and in fact that there will be a lot of kind of professionals that work to integrate these systems that are kind of much more ecumenical with these models. Um, and so curious about your take about, you know, this partnership and and where you think it all might go.
02:02 >> Yeah, I mean I'm I'm very excited about it. Um, I think it's a potent combination, you know, to have open AI's AI capabilities plus IBM's and our ability to put them within a Fortune 500 company and and beyond, you know, um, within this new practice that we're creating, we're going to have four deployed units and four deployed engineers are going to be equipped with these kind of capabilities that can go out and meet clients where they are, right?
02:26 um you know and this is a big signal that we IBM we're not trying to pick a model winner you know we're we're actually uh working and we're going to become the owner of the enterprise AI control plane here you know we're we're betting that the enterprise AI value it's shifting upwards from owning the model to now orchestrating governing and and integrating and even you could argue operationalizing you know these uh types of models and what what I also think is fascinating right, is that you know both anthropic and open
02:58 AI they they somewhat own the market right and and because we're we have a partnership with both both in different ways um it's quite important that we're putting together anthropic right uh where we have anthropic is really geared towards IBM software engineering you know you know where we have IBM Bob built around that ecosystem while open AI is really for IBM consulting right and going out and and having these four deployed units and uh engineers there's you know so now we're really this neutral enterprise um AI
03:29 integrator where we have the options right of both of the you know leaders anthropic and open AI um but yet we still maintain our own you you know granite you know type series models which I think would become even more important right because this gives clients the freedom of choice and the granite models are the much smaller ones right where we can route simple requests to those so they're smaller, they're cheaper, whereas you could argue that the open AI ones, you know, they would handle the more complex type of
04:00 task planning, right? Um, and so and and so by having, you know, these model gateways, I know, uh, Mahai works works on a lot, you know, gives us the ability to go in any direction that our clients would want to go in. >> Yeah. And beh, I think, um, you know, I think one way of looking at it is almost like everybody's competing with everybody across every single layer at the moment.
04:24 Um, you know, this partnership makes me think a little bit about uh OpenAI not too long ago both launched its own deployment company which is kind of a consulting arm that would help people integrate OpenAI. They also signed a partnership with McKenzie which is like another consulting org. Um, you know, at the same time Aaron as you mentioned right IBM's got its own models that are kind of playing in the space and also has its own consulting arm which is now partnershipped with OpenAI.
04:48 You know, it kind of feels like again it's sort of like everybody is going in every single direction. I guess the kind of question is like what's open AAI strategy here because it feels like they are sort of almost like choosing every single route and sort of seeing what sticks uh when it comes to this world of working with clients to kind of integrate this technology.
05:05 >> Yeah, I think open AI needs system integrators which is boots on the ground. This is where consulting organizations come in and have the ability to really integrate specific models or specific model providers into use cases which are company specific client specific. It's also important to note that there is no such thing as the best model. The best model for what, right?
05:25 Different large language models and models providers offer multiple models with different cost, different latency as in time to first token, different performance as in things like tokens per second, different performance as in the outcome. And even that differs for use cases, which is the best model for coding, which is the best model for information retrieval or summarization.
05:47 uh different models and model providers have different geographic availability and in many cases you're only allowed to use as an organization's models in your geography. If you don't have open AAI in your um you know London data center you're going to have to use what's available there and even there you have limits with these models which is what most companies tend to find out the moment they scale AI.
06:08 Hey, can I just give AI to my 100,000 employees tomorrow? And the answer is well, you can't buy it. >> Like, no, you can't buy it. You literally can't buy it. You go to these organizations and you're like, hey, this thing only does 50, 150, 200 requests per second or the inference performance is not sufficient. And no matter how much money you throw at it, you realize there's not sufficient model to go around.
06:33 So then you either do model routing or you go to multiple providers and you find out hey I need to prompt this model differently. So this is where again that relationship and partnership with system integrators with you know the forward deployed engineers and so on comes in to be able to help you consume the right model for the right task in the right way in the way which is governed and secured and controlled and prompted the right way to get optimal results.
07:00 Yeah, I wanted to also mention there's a cost factor involved as well. You know, it's very costly, right, for over 100,000 employees each to begin to use these types of models. And I think to Mahai's point, you know, uh there's no no free lunch, right? Uh where you have to route the appropriate problem to the appropriate model. That's like this multi-objective issue, right?
07:20 Where you want to have the best capabilities that match the problem, but you also want to minimize cost, right? Because because if you think it's like 10 bucks or $10 US dollars per employee and you have hundreds of thousands of employees per month, you're looking at millions of dollars a month in cost, token cost, which may not be sustainable for a business, right?
07:40 And so by having these partnerships, you not not only get access to, you know, some of their leading edge models, but you presumably also get, you know, um economies of scale pricing, right? which does help you know to bring down the cost of your workforce so that you can equip your forward deployed engineers and forward deployed units right so um I do think it's a competitive advantage for both IBM and also for for open AI to uh you know saturate the market with their capabilities >> yeah I think there's another also
08:10 point that's very important which is kind of the distribution and the legacy plumbing that we have you know at at IBM which is very strong and I think those be troll benchmark edge If you look at what's happening in the Silicon Valley, there's a lot of belief that maybe having a 2% higher score on the MMLU or the human eval benchmark wins the Fortune 500.
08:31 But in the real world, 80% of enterprise value is locked inside legacy mainframe cobalt databases, uh maybe SAP ARPs, Epic Healthcare Systems, uh HIPPA compliance vaults, etc. And IBM owns those enterprise relationships. Open AAI and Atropic needs IBM's distribution and consulting pipes far more than IBM needs their specific model checkpoints. So I think here this is really a great partnership because you know at IBM I don't think we need to spend 20 billion training frontier uh foundation models to compete with maybe
09:06 Microsoft, Google or Meta. Instead IBM's winning strategy here is becoming the trusted vendorneutral enterprise orchestrator through the Watson X.governance. IBM provides the audit trails, the bias detection, the prompt monitoring and the regulatory compliance layer while letting clients kind of hot swap models based on cost, speed and capability. I agree with Mihi.
09:31 I mean there is no model that fits everything here. So it depends the the best model depends on so many other other uh dimensions here and you have to tune it to your enterprise to your data etc. So I think it's very important to have that layer that orchestration and those pipes that are so important with the regulatory compliance with the audit with the governance which is really important um because the bottleneck here in enterprise AI is not a shortage of intelligent models.
10:00 It's really change management system integration and and also dealing with dirty data. And I think IBM consulting is kind of weaponizing open AI and anthropic models to accelerate that enterprise digital transformation kind of converting legacy code bases and automating banking workflows at scale. >> Yeah, absolutely. And I think the question being asked is it's almost a little bit like what um you know how much will the model matter in the future?
10:28 Um, and I think, you know, you've seen this debate, we've talked about this a lot on the compute side, where it's like, okay, well, if everybody can get great open source, maybe it's really the compute that's going to matter. It feels like the consulting side is actually the other part of this discussion, which is, well, okay, when a world where there's lots and lots of models and there's there's lots and lots of options, then ultimately the kind of winning factor is going to be the the sort of like trusted consultant.
10:51 Um, Aaron, do you want to get a final thought in before we move to the next topic? >> Yeah. Yeah. I I really do like um you know this constellation of partnerships, right? So there's there's three uh in my mind right now that really stand out. It's IBM plus anthropic plus open AI, right? Um because the the anthropic deal and partnership was announced around October 2025 and the idea was to bring in AI to change enterprise software development and integration, right?
11:15 Um and that's that's how IBM Bob, you know, somewhat emerged, right? And then this open AI partnership is different. you know, that's about equipping and training thousands of consultants uh through IBM consultant advantage so that we can go to clients, right, and meet clients. So, so we're taking the full stack of the business with these two partnerships.
11:35 Um, and I'm really excited about the future and what we're going to do together uh within IBM. >> Great. Well, it's a great segue to uh the next topic I wanted to cover. News broke this week that Stripe, the payment processor, um is finalizing an agreement to purchase a company called Open Router for more than 7 billion. Um and again, if if any indication of how crazy the market is right now, the the stat that was kind of going around the internet, which is that 7 billion is more than five times the $ 1.3 billion
12:09 valuation uh that Open Router had uh just 82 days ago. Um and so this is like a really wild story. Um and you know I think for a lot of folks who spend their time focusing on the models and hearing about what open anthropic are doing open router itself may be kind of an unfamiliar name. Um and so I guess Mihi for for folks of uh who are maybe less familiar with what open router is or does uh do you want to just give a quick intro like why it's such a big deal and why it could be possibly worth 7 billion.
12:35 So, as somebody who's written his own model gateway, MCP gateway, and A2A gateway, and gave it away for free, I'm kind of u >> my life choices, my life choices. But, uh, all joking aside, it's like I believe the future is multimodel, multi-model, multi- aent, multi-framework, multi- harness, multicloud, multi- everything. There is no one sizefits-all for models.
13:03 There is no one model which is good and you can't even get enough of one model to fit every use case. So with platforms such as open router, you have the ability to access a large variety of models from every vendor through a single platform which gives you access not only to the models but you can do things like for example semantic caching or you can help optimize some of that work.
13:29 And some of the future that we will see in this space is integration into unified gateways. gateways that have the ability to access not only models but MCP servers, agents, skills, civals, controls with semantic routing, the ability to route the right request to the right model phops and cost management, the ability to say, "Yep, we're going to limit and cap your cost and spend with things like virtual API keys where you're not accessing the key of the model directly, so you can lose it or somebody can steal it, but
14:03 you're actually accessing a virtual key which is tight to role-based access control. So a lot of these capabilities are musthaves in enterprise. They are the capability which allows an organization to distribute these models, MCP servers, tools and so on safely to their enterprise but also get things like metrics and observability data from those interactions.
14:30 >> Um Kar I guess so now we know what open router is. Why stripe? You know, this is also kind of like the other part of the the transaction. We've been hearing, of course, a lot about the the big acquisitions that the Frontier Labs do. You know, Stripe is, you know, I use them to pay for subscriptions and stuff online. Um, why why are they getting into this space, you think?
14:47 >> Yeah, that's a very good question. I think the way I see it, models are cheap. The toll boot is where the money is. Every 6 months, models get smarter and way cheaper. Token prices have dropped over 90%. Building a raw model is a tough, expensive race to the bottom. But the router gets paid a tiny toll on every single request, no matter who wins the benchmark race.
15:08 But AI billing isn't traditional software. It's kind of a payment headache. Old software kind of charged say like a $30 a month per user on a credit card. AI doesn't work like that. It's millions of unpredictable micro tokens flying around like 24/7. Stripe already runs online payments. So buying the router lets them handle metering and payments right where the code runs.
15:33 And and if you look at AI agent, they need their own wallets. Down the road, AI agents will do the jobs on their own. They can't sign contract or swipe physical cards. An agent needs a digital wallet to say, I'll use a cheap 5-cent model, for example, for a simple lookups and the 50 cent model for deep thinking. Stripe wants kind of to be the Visa card for these autonomous software.
15:56 And another thing, Stripe is also it's neutral. The big the big clouds aren't. You know, Amazon, Microsoft, and Google want you kind of stuck on their servers. Stripe doesn't care where your code runs, whether it's on AWS, on an old laptop or a GPU cluster. They just, you know, they just take their small cut of the transaction. And you know, I think that the the routing strategy allows them to be very flexible here and then kind of have these fallback mechanism, switch between various models.
16:22 So they don't care where things run or what models you use, but they want to control that orchestration routing layer which is becoming super important here. >> And so Aaron, I guess I to bring it back to the original topic, it feels like we're we're back in the same place we were before, right? Where we were talking about, well, you know, people don't care about the model so much anymore.
16:40 It's going to be the trusted consultant. You know, it seems like, well, maybe people don't care about the model so much anymore. It's going to be the router and the gateway. Um, and I guess kind of question for you Aaron is like we're almost looking at a postmodel market now. It feels like where a lot of the money is going is to things in and around and outside um the AI to like the AI model to wit um and um I don't know like how should we think about how that market's evolving, right?
17:05 Like is the future Open Router buys a consulting firm or is it you know Nvidia gets into the consulting? You know, I'm kind of really interested in kind of all of these weird configurations we're about to see as the the core thing we've been paying attention to AI almost seems to become kind of commodity and we're all kind of looking for like what else is the sort of uh economic niche that you can stick into.
17:28 >> Yeah. I mean, you know, this this market is, you know, eban eb and flowing, you know, and, you know, we just have to trace where the heartbeat is coming from, right? And it seems like that u right now the heartbeat is around routing, right? being able to route traffic to the specific place of which it you know um optimizes or minimizes your criteria that you're looking for right but I suspect that this routing it's it's going to become commoditized at some point you know it's a very it's going to become a very
17:54 crowded capability space you know I just quickly looked and I mean there's many competitors to a to open router there's light lm cloudflare port key helicon together ai fireworks AI so on right so >> context forger I wrote it. Hello. Yeah. Yeah. Yeah. I mean, it it's it's incredible just just the amount of space, you know, that's where all of these little companies I won't say little anymore, but these companies are coming in.
18:21 And to me, Stripe bought uh or rather uh yeah, Stripe bought OpenAI based on growth. You know, they saw that you over the last what, you know, 6 months or so that the tro token growth by 5x from 5 trillion to 25 trillion, right? So they're betting the farm I think on that type of growth and they want to position themselves as the infrastructure for AI agents and agent commerce just like they are today with you know um just commerce in general.
18:48 So they're trying to make that leap right saying that you know AI agents and models are going to be the drivers of commerce therefore we need to get into it but there is risk in this transaction right is this commoditization part so and so do I think that this is going to be the underpinninging I think it you know of the economic drivers and decision space I think today it is but you know you know after it becomes commoditized there will be something else you know that that'll come in so you know will consulting you
19:16 know groups want to buy you open router you know typeish or build their own organic u today yes but I think you know there there'll be other things that are going to come up right where they want to focus in their capital yeah I think they also have a large number of users which was part of the acquisition like it's um I don't know the exact numbers but some you know different sources claim they have 8 million or 10 million users across free enterprise whatever else 150,000 active I mean I don't know the exact numbers
19:48 but this is not just an open source project somebody buys it's more of they're buying the users they're buying the commitment they're buying the wallet they're buying those paying customers as well with this transaction yeah what's what's really interesting is that operator has like 50 to 100 employees I mean it's pretty small you know but when you look at their market valuation I mean you're looking at maybe 70 million per employee right that that they contribute towards their valuation of which they are out 70
20:14 million per person right um that's that's big and so yeah I mean to Mahai's point this is a you know a bet on growth uh but they do have the foundation to make that growth you know and if that trend line continues right right you know I think strike might have done a good job you know but again commoditization is always the enemy of these types of deals >> yeah I also like maybe to bring here a nice analogy uh like the the tall boot analogy so I think Stripe didn't buy this for just the AI hype especially the the open
20:46 router I think it's Like think of it as they bought the toll boots. Like while model makers kind of bleed cash competing on benchmarks, I think Stripe sits at the bridge and collect two cents on every car driving across. So um and I think it's a very interesting economy here because they want to get margin of all of those transactions happening, you know, all of these tokens being consumed and they it's a very good place where they're sitting.
21:14 uh you know the routing right now is becoming very important place. >> Well maybe just maybe a final question I had I know it's a sore topic because you uh released one of these gateways for free to the public um but um what is the open side of this the open source side of it like do we feel like you know that there will be more kind of open gateways that kind of are operating outside of the businesses like what's that ecosystem look like from your perspective as someone who's built something like that?
21:39 Yeah, I think there's already more than 100 of these g gateways in some shape or form out on the market today in the open source space. You know, I think Aron has already mentioned like five or six of them, but definitely there is a need right enterprises and users and so on need to have a way to manage a combination of agents, MCP servers and models and so on with things like semantic routing and caching of permissions.
22:04 And I think a lot of where this space is evolving towards is that of agent management platforms or agent control planes where AI gateway or model gateway is just a small component of the capabilities enterprises need whether they're using open source or commercial software to provide governance trust observability optimization cost control phops run and manage capab capabilities for their entire AI fleet.
22:41 >> Really interesting data coming out of RAMP uh which as you may know is one of the most kind of widely adopted you know sort of business credit card payment management uh sort of platforms. And so as a result they have a really good bit of traction on understanding how you know businesses down to the very micro level are using things like AI. And so they released the post which is basically their kind of August 2026 review of what's happening uh in the AI space.
23:07 And some really interesting stats here. I mean I think the first one that I'm really kind of interested in uh to build on some of the themes that we've been talking about today is that there really does seem to be a lot of evidence um that open source AI adoption is is continuing to rise over time. I guess uh maybe Aaron I'll kick it over to you. Uh you know this is still you know a jump from 4.5% to 6.1%.
23:26 So not a huge amount of the overall market but they are seeing this kind of really measurable rise in people using you know model serving inference platforms and not really depending on the kind of proprietary models. How far do you think that's going to go? Yeah, I mean, you know, I I do enjoy reading, you know, the ramp and looking at the 2026 RAMP AI index because it it gives you a nice um you know, sort of pulse right on on what people are doing and what the businesses are doing because it's you know, it analyzes
23:57 about 70,000 businesses and um what what I found and just by looking at it, it looks as though AI adoption is it's really uneven, right? Because if you look at like the top 1% right of these AI adopters, you know, they spend about let's say 7,400 per month on an AI, you know, per employee, whereas the top 10% it drops down to 650, right? Um and then the median, you know, or you know, the number that's like right in the middle, right, of all this is around what?
24:26 1212. Um and this is, you know, when whenever I quote monetary values, it's uh USD. Uh but but this is per month on an AI employee. So that spread really shows you this huge spread right across uh companies. Um but even so there is growth um but there's a ceiling to the growth right um it it's it's as if companies are saying hey let's give everyone access to AI but we we we have to meter the use right right we need to cap the use in some way if it's getting out of control especially as we get these u digital workers
24:59 right on online right um and and so there is this this economic consequence and what I find interesting right is that the top 1% % right it's starting to approach the cost of AI per employee starting to appro approach the actual salary of the said engineers that are using these tools so so now the question becomes well if there's you know if if we're paying as much for AI tools as we are a person right u what do we do right do we need to focus in on productivity versus revenue right what's what's the most important
25:36 aspect and so businesses I think will eventually have to uh grapple with that, right? Um, assuming the the economics and costs, you know, stay where where they are, you know, at at the moment. I >> I feel kind of the the end of this AI for more tax uh because what happens in 2023 2025 CFOs approved this broad AI software budget and they're kind of the fear of falling behind.
26:01 But in 2026, the era of the unmetered experimentation is kind of officially dead and CFOs now demand kind of measurable unit economic payback. So either an AI tool demonstrabably kind of reduces the headcount requirements, accelerates kind of cycle times. So for example, PR's merge per engineer or you know drives you know kind of net new gross revenue.
26:27 So, so I think what RAM's data is exposing here is the kind of the vulnerability of this 10 wrapper startups the which point solutions that merely summarize kind of PDFs or generate marketing copy you know these are are seeing kind of catastronic catastrophic churn because foundation model updates and native OS browser features do these things for free.
26:47 So I think the true cost metric here is how do we measure the cost per unit work versus the cost per seat. So evaluating AI spend um on a per seat basis. I think it's kind of an obsolete legacy SAS framework and you know I think the strategy has to shift here to evaluating AI on cost per resolved maybe support tickets for example dropping from $18 human cost to maybe like a few cents AI cost or cost per audited contract dropping like from say $400 to $12.
27:21 So the high spend per per employee is actually kind of a an indicator. It's not a good indicator if it's if it's it doesn't correlate with the multiffold productivity gains and I think that's where I think we need to start focusing on it's what the return on investment with these spends per employee in terms of productivity gains otherwise it's it's not going to be useful just just to have these spent uh but I also feel things are kind of segmenting uh initially it's like okay let's start let's give everybody access to
27:55 these AI tools and and and and LLMs etc. But some certain teams like especially the engineering teams, the software developers, they're actually using them substantially versus maybe other teams that might be maybe losing them less. So, and I think that maybe we will start probably seeing segmentations per departments, you know, who really is benefiting the most and getting the uh ROI versus other teams that might probably don't need as much.
28:24 >> Yeah, for sure. And I think I mean uh underlying that Mihi is sort of the idea that we actually may be spending too much on AI. I don't know like I guess there's there's one way of reading these numbers which is that the skeptics who say hey this might be actually kind of a bubble are maybe actually right right because as we start to take a much harder look at say productivity per dollar in it may end up being much smaller than we think.
28:45 Do do you think it's like too pessimistic as a way of reading these numbers? I think the way I would look at it is that you know over the course of the last two years most of the client conversations I've had have shifted from help us use AI or you know we want to buy your product or we want to do something with AI we've done a PC with AI to help we've done too much AI we've got 60 random acts of AI we've got agents written in longchain long graph autogenb crew AI we've got agents on Microsoft copilot studio agents on
29:14 what's next orchestrate we've got agents everywhere we've got models we've got every model we've about model routing. The problem we have is one, we don't even know where these agents are, where they are built, or how AI is being spent because it's usually not that employee that's, you know, asking chat GPD for advice on one or two things that's really burning down the tokens.
29:32 It's these hundreds of agents, the developers and everybody else who are, you know, they have 50 open terminals or they've got an event stream going into one of these agents and consuming them at scale and they use the most powerful and the most expensive models and so on and so forth. Second comes regulatory pressure. Regulators are asking, "Hey, you know, there's a thing called the EU AI act.
29:56 So, are we factoring into the total cost of ownership, one of those big fines, for example, or the cost of regulating those models?" Cuz the more you have and the more AI you have spread all over the place, the higher total cost of ownership is not just the tokconomics of how much you're spending on an individual model provider. It's everything else to manage it.
30:13 the people, the resources, the infrastructure, the governance, the security. And then you've got the cost runaway where today organizations don't really have a good mechanism to manage things like budgeting, even tokconomics like understanding how much they're spending uh let alone tying that into quotas, enforcement, tying it to return on investment for example and value.
30:39 So this is one of the areas I'm most excited about. It's one of the areas where um this is kind of what we're building with Watson X orchestrate. So that's the product product I'm uh I'm technically responsible for to try to get alignment between return on investment model cost AI spend total cost of ownership guardrail security controls everything else in between.
31:03 So I would say AI is easy and this list shows us is true. It's getting control over AI and having predictable costs and having predictable outcome and a good return on investment is proving extremely difficult in these early days. Yeah. Yeah. Yeah. I found too that it's becoming harder and harder because as AI is getting cheaper and more capable, you would think that the cost would go down, but just in this report, you know, you can see that just over the last year token, you know, spend has increased 13x.
31:38 So, so it's like there's this AI splurge going on. You know, companies can't afford to do more, so let's do more because it costs less, but then the overruns happen because you have all of these agents that are compounding the problem that we don't know what exactly they're doing, right? And how to put the guard rails on, right? So, so it's quite an interesting conundrum, you know, that uh I think the field is in.
32:01 >> Yeah. And I think maybe this is kind of similar to what happened with the cloud comput when cloud computing matured between 20 uh 2012 and 2015. Companies stopped mindlessly spinning up idle EC2 instances and and they started introducing the PHOPS. So the ramp data here shows enterprise AI is also entering this PHOPS era and really pushing the companies kind of they need to start thinking about how do we spend smarter not less.
32:31 So because now we're entering this maturity age and now the economics the the reality is hitting us harder. Uh so we I think a mental model here of how do we monitor these things meter these things properly make sure that we're spending where things need to be spent properly. Uh so like if you look at engineering teams it it seems that they're the only department where AI spend continues to grow exponentially without push back.
32:56 tools like cursor cloud codes, specialized code review models, etc. They've crossed the threshold from novelty to being indispensable developer infrastructure. So that's spend there. It I think it's justifiable, but maybe in other areas not as much. For the final topic today, because we're almost out of time, uh this is just a fun final item that I wanted to to touch on and I guess maybe high maybe I'll give you the last word on this.
33:23 Um, Politico, which is a a publication that covers uh DC and DC politics, uh had a really interesting article that the Congressional Office uh of Legislative Council in the House of Representatives is awash in AIdriven uh slop quote unquote bills. Um and it reminds me a little bit of I think we've talked about it before. I think one of my favorite analyses that anyone has ever done was observing that after TAC GBT launched, the use of certain types of phrases really exploded um in the in the houses of parliament in the
33:56 UK. Um and I guess behind we've all used AI occasionally to take certain shortcuts. Um and so maybe it's no surprise that we're finding people trying to use AI to draft legislation. Uh should we be worried about this? What do we do about this sort of thing? Uh and I'll give you the last word here for the episode. want to introduce a loadbearing bill with a lot of PM dashes and smart quotes.
34:18 I think it's no surprise that just like we're seeing an explosion on papers, publications, homework, emails, Slack messages, Teams messages written by by AI. We're seeing the same thing everywhere, including legislation. It's part of it is a question of education. And it's educating people that AI will make mistakes that you need to have governance security control.
34:44 You're still responsible for the end result and giving them a way to do it that is I would say secure and govern not just everyone going to the for instance of CH GPT or online or somewhere or you know just posting the information there and posting back whatever they see. Um, so I think legislation is no different than any other field and rather than trying to fight it, there needs to be clear guidelines and rules on what's allowed, what's not allowed, what's safe, what isn't safe, and empowering folks to use it in a
35:18 smart way as opposed to having AI write law. >> Words of wisdom to end on. Uh, Mihi Kowar, Aaron, thanks for joining us on the show. And that's all the time that we have for today. Um, and thanks to all you listeners. If you enjoyed what you heard, you can get us on Apple Podcast, Spotify, and podcast platforms everywhere. And we'll see you all next week on Mixture of Experts.