IBM’s mainframe chip collab, NVIDIA’s Poolside deal & Ox Alpha’s reveal Transcript, AI Summary & Key Points

00:01 This is a clear line in the sand from Nvidia that they are going to depend, their future is going to depend on a strong, thriving, many model ecosystem. All that and more on today's mixture of experts. I'm Tim Hong and welcome to Mixture of Experts. Each week, Moe brings together a panel of brilliant technologists working at the frontiers of artificial intelligence to lead you through the week's news.

00:28 On this week's episode, we've got Gabe Goodhart, chief architect AI Foundations, Ash Minhas, technical content manager and AI engineer, and Skyla Speakman, senior research scientist at the software innovation lab. Welcome to you all. I'm also joined by my co-host today, Eileene McCannan, who's a staff writer at IBM Think. I always say this like there's a lot of news this week, but uh this week it felt like there was maybe like an overwhelming amount of news.

00:50 Um, so we're going to cover a little bit a new announcement about IBM's new next generation dual processor. We'll talk about uh ox alpha and the kind of mystique around it uh which just uh as of today actually was revealed who is behind it. Uh but I actually wanted to start by talking a little bit about what's happening with Nvidia, its recent line of acquisitions um and uh and why it's doing what it's doing.

01:19 The headline here is that uh HuggingFace announced that it was going to be acquired by Nvidia for $12.9 billion. Um this follows on sort of just another deal not too long ago of a $6 billion kind of acquisition transaction with Poolside. Um and I guess Gabe, you you flagged both of these stories for us because you were just like this is all about open.

01:42 Um and so maybe I'll kick it over to you to give your your hot take first. Yeah, I think this is a clear line in the sand from Nvidia that they are going to depend their future is going to depend on a strong thriving many model ecosystem and right now that is open and right now they see that that is a critical lifeline for their company. So they're willing to put money behind it.

02:07 Um they've clearly started down that track with their work on the Neotron model line. Um and the poolside investment is clearly a further investment in making Neotron a strong model line that people can rely on. Um and then this acquisition of Hugging Face is kind of just taking it to the max, right? Hugging face is the cornerstone of the open AI ecosystem right now.

02:32 And that's true of the models, but it's also true of the software. Uh the Transformers package is owned and operated by Hugging Face. It is the reference architecture of literally every LLM on the planet that is released in open source. Um, so every downstream architectural implementation follows from the Transformers one. If I try to get something merged in llama.cpp before it's merged in hugging in transformers, they say, "Hold up, we can't count on this until that other PR is merged."

02:59 So this is not I mean this is clearly a play for the center of gravity, right? So Nvidia is making a clear statement here that they want to be the center of open source AI. Um and that I think has some some pros and some cons, right? So um a big company like Nvidia that is clearly well-moneyed, wellrespected and in a very strong position of power in this industry, it's great to have that support behind the open ecosystem.

03:28 For those of us that live in that ecosystem, it means we've got backing for years, which is amazing. On the other hand, a big piece of the OK open ecosystem is innovation that drives uh intelligence per weight downwards uh and in fact incentivizes making the model smaller and easier to run on a wide variety of hardware. And putting that behind a vendor that is clearly biased in what hardware models should run on and the size and cost of that hardware creates a bit of a perverse incentive.

03:59 So I'll be really curious to see how this plays out um in terms of is it uh you know I think the obvious analogy that a lot of people are drawing is GitHub and Microsoft where GitHub is the de facto standard for where code lives in the open. Um and thus far it seems like GitHub and Microsoft have done a pretty decent job of keeping a firewall between the incentives of the closed source Microsoft ecosystem and the open source code ecosystem in GitHub.

04:26 There's obviously product cross-pollination with Copilot. Um, but they haven't detracted from the open-source ethos of GitHub. So, the hope here is that Nvidia would do the same with Hugging Face and that HuggingFace would remain a a shining light of open-source uh championship and openness. Uh, and that, you know, Nvidia would simply be putting their bet behind it.

04:50 So, we'll see what happens. I'd love to push more on what you've uh underlined the sort of my question is sort of who benefits more in this uh is it the open source community or Nvidia you know what in the best case scenario how do they each help each other and you know what would be some of the concerns and we can uh you know Skyler or Ash if you want to jump in to join the conversation please.

05:10 Yeah, remember when Microsoft bought GitHub? It's it's it's basically that again. Now, Microsoft done a very good good job of of how that acquisition's gone. I think that um one of the interesting things with Nvidia and and them acquiring uh Hugging Face is going to be that there's a lot of inference providers already providing inference to Hugging Face.

05:32 And so, how will that work now? Are they reselling the same hardware that Nvidia were providing then? It's going to be some interesting dynamics start to appear from that perspective. Open weights, open software, but closed hardware. Uh Nvidia's play here is to get as many models trained and running on their CUDA cores. That's that's the larger draw here.

05:56 So on the one hand, we do benefit from having easier access to a wide range of models with an asterisk as long as they run on Nvidia chips. Um, so I think that's that's what's really going on here, both with the hugging face purchase and with the model factory from Poolside. They want to turn these models out as fast as possible because they're still in the business of selling hardware.

06:17 Well, I think on that note, you know, it really leans into why this is important to Nvidia and and why Nvidia is well positioned to be the one making this investment. um pretty much every other significant vendor in this space is in either the model space itself or the software space above the model. So if you look at any of the neoclouds that are providing that inference if you're looking at any of the um the big hyperscalers like their goal is to get you running not it doesn't particularly matter what the chip in the

06:50 computer is. It matters what the API surface you're hitting is and what metering gateway you're going through. Nvidia doesn't care. it cares about the chip under the hood. So there's no disincentive for HuggingFace to continue providing a wide range of inference providers as long as all those inference providers are buying chips from Nvidia to power the inference that they're running through Hugging Face.

07:09 So I do think there's a incentive alignment here that's fairly unique to Nvidia. Again, I think you know the big losers in this deal will be the uh companies trying to make commodity chips that are not Nvidia, right? So, you've got I think one of the articles I read about this clearly positioned this as a defensive play against all of the labs that are trying to do vertical integration and build their own chips.

07:35 Uh because if the big labs stop needing to buy Nvidia chips, Nvidia needs to sell their chips somewhere else. And so, they're really going to aim at that commodity hardware like go broad rather than deep type of uh sales play. And uh the the ones that really lose out are the others attempting to sell broadly to you know general data center consumers.

07:56 >> Yeah. Can I play bad guy here for a second? Because obviously there's some discomfort in the room with you know Nvidia's nefarious plan here to keep everybody on their kind of closed hardware. You know is there another view here which is like get real right like basically these commodity players were never ever going to catch up. The video was so far ahead.

08:13 We're just recognizing reality which is that like the game has been lost in terms of certainly open hardware but maybe even commodity hardware here just given Nvidia's like extensive lead. I don't know if any do people agree with that like this is in some ways realism versus them really kind of closing off an avenue for commodity catchup. I think to me, yes.

08:34 And I also think it's important to note that the the smaller hardware vendors that are trying to do commodity hardware that are not Nvidia, they're trying to soak up excess demand, frankly. Like right now, we are in a very skewed supply demand ecosystem. And so, this isn't going to change that dynamic. Now, if the demand really does drop off, that's where you're going to start to see this become a bit more of a food fight.

08:57 And you're absolutely right. you know, Nvidia is going to throw its weight around and uh you know, maybe it's just calling a spade a spade. But I think right now all of the possibility of other vendors losing out is just speculation because frankly, you know, if a data center needs chips, uh they're probably going to look for Nvidia first, but if it's a, you know, 9month waiting period, they might look elsewhere.

09:16 And Nvidia is probably okay with that because they're still selling as many chips as they can make. >> I think it's also interesting whether this is like a new, you know, taking this news. You mentioned obviously the um licensing of the poolside model factory. Is this like a new you know bigger broader playbook for AI consolidation because like in that case what struck me was you know they weren't acquiring it.

09:38 They you know bought the license they're hiring the team. Um you know so it's sort of Nvidia kind of grabbing land in a variety of different kind of manners but uh perhaps that's too simplistic. >> I think a key part of that poolside Nvidia conversation was it's a non-exclusive license. So Nvidia is not necessarily doing a land grab there. They are going to be one of perhaps multiple companies that are going to be using pool using poolside approach to model creation.

10:04 Um so that uh that was kind of in the details there of that agreement. U so yeah, we'll be interesting to see who else might be in line to pay $6 billion to Poolside for their license. I don't think there'll be too many customers left in that. Uh but it this is not exclusive Poolside and Nvidia in that particular arrangement. Yeah, and that's maybe where I want to kind of like do the final few questions on this segment is uh you know normally when you spend 1920 billion you're usually a little bit short on the money,

10:32 but we're also talking about Nvidia here where they could go for a lot more. Um I guess the question is if uh Ash I'm curious if you have any thoughts on like who's who's next up here. Uh where else would you go if you were trying to kind of consolidate in the open space? Um, do you think Nvidia is gonna kind of make another another play or is this kind of like it for a little bit?

10:53 >> When we look at sort of like what what does the AI world look like? I mean the stack is like you know chips and infrastructure then it's like the cloud providers providing that and then you've got the application layer. I think with both of these things it's like infrastructure lockdown you know they're in that middle layer now and like they want to make really good models.

11:15 are going to use obviously that poolside acquisition to make their own Neotron models better. I'm curious and I mean I'm I know I'm responding to your question Tim with a question but the question is is like are they going to go even higher up the stack and start looking at some of that application layer and maybe going for a slice of that somewhere and may make that open or you know is it is it like you know going to go grab open web UI they've already taken a small step in that direction with Nemo claw right so they

11:42 have put they're probably one of the largest companies behind an open-source agent harness at this point So, um, you know, that there was a big announcement a little while ago about Nemo Claw, and it's kind of I I I wouldn't say it's it's petered out, but there hasn't been as much hype about the claws as of late. Um, I think a few other agent harnesses like Hermes uh have taken up a little bit more of the oxygen for the generalist agent, but uh you know I I guess I would be surprised to see Nvidia try to go the actual

12:17 like uh for sale you know buy this as a consumer uh software platform uh route. You know I don't think we'll be going to like ai.invia.com nvidia.com and you know having our agent chats there the way folks do with chat GPT or claude um it just doesn't seem like that incentivizes the right things because ultimately their business is still built completely around the chips that are under the hood.

12:42 So I think for me the two recent deals we're talking about here are just a clear sign that they are going for uh as many flowers blooming as possible right and if they can put their money and their mouth behind specific projects that they think amplify that goal um they will uh but I don't see them you know trying to then also carve off a corner of the market and say but forget all those other flowers come to our specific flower.

13:11 I'm going to move us on to uh our next topic which I guess is related in some ways but it's a it's a different side of the market entirely. Um we've talked about the mainframe market uh here on in the past. Um and one of the things I love about the market is that it's very different from what we know of usual you know things around AI where the general approach of the AI industry has been just like all right let's just launch it and see what happens.

13:33 With mainframes you really kind of can't afford that in some ways. Um, and I I know you flagged this one. I don't if you want to introduce it, but sounds like IBM is basically out with a new dual processor in the mainframe space. Um, which is a kind of big deal just given, you know, how cautious people are in mainframes. Do do you want to talk more about it?

13:50 >> Sure. Sure. And, uh, yeah, they presented this at, uh, this week was hot chips, which is a big kind of semiconductor annual conference kind of chip makers um, researchers come together, unveil new processors and designs. And um you know many of the participants were you know some of those Gabe that you mentioned the frontier companies who've all come up with their own kind of custom chips that they're they're sharing but uh in this context IBM was announcing kind of a different type of trip uh chip you know this

14:16 dual process architecture and um so this one they've developed with ARM the AI software ecosystem and it what's different about it is it combines IBM's Z so the mainframe um workload with ARM and all on one chip. and with the ability for the ultimately to be able to switch between those two workloads you know in nanconds. So I think the bigger picture you know thinking is that there's a growing challenge in enterprise AI where organizations you know want to run their inference uh closer to where that data resides you

14:52 know and the reality is that much of that data sits on mainframes you know many of those IBM and Linux one systems um and then at the same time you have this ARM ecosystem where a large share of the modern AI software you know is being built. So I think that um you know the idea is rather than moving the data between these two places IBM is sort of you know proposing this chip to bring those workloads closer together and maybe Skyler if you're game to you know jump in on this first I'm interested you know from an

15:21 infrastructure perspective why does this matter you know how big of a deal is this >> if you get all the way down to the nitty-gritty details about how these processors work there's two different philosophies one of them is doing multiple simpler instructions. So if you want to add two numbers, there's one instruction to go read a a number from memory, there's another instruction to add it, and there's another instruction to write it.

15:46 The other philosophy is a single more complex instruction that does that all end to end. And so computer science over decades has evolved from kind of those two different philosophies. And it's really interesting now to see these worlds somewhat collide in the IBM ARM space. Um, I do not know the technology behind it about how they're doing both of those instruction sets on the same chip.

16:08 Kudos to them. Very smart people working on that. Um, but it really I think it's going to be interesting to, you know, ask questions like, is your bank going to go to the app store and update its core banking software? Um, you know, are can you go to an ATM and do a transaction and say, wait a minute, our our backend is training a model right now. Give us give us 20 seconds.

16:32 And those aren't questions we had asked before because these things have been completely separate ecosystems. And now IBM and ARM are saying still mainframe functionality but the ability to run these two different instruction sets. And they have again what I want to emphasize here is two different philosophies of how you go about writing code coming together under one place.

16:53 Um, and the thing that it kept bothering me when I was reading this is on the one hand, it's cool to see these fences being brought down, but there's also this approach where you should be asking why were those fences there to begin with? You know, do we really want to have ARM software sitting in the same places as for example our uh our government services, banking uh banking services and the things that have relied on mainframes for decades.

17:17 Uh, so watch this space. I don't know who all good else can be can comment on that but um this is I think this is bigger than just a single chip sharing two different instruction sets um it really is two different worlds colliding and it'll be really cool to see it play out in the coming months >> and I think it sort of yeah I think to your point of it being a larger question too it's sort of this looking at enterprises trying to integrate you know AI with these mission critical systems these you know established

17:44 systems that you know they can't just totally rip out and then you know kind of bring in, you know, something totally new. Um, and the analogy you used of the app store, someone as I was talking about this news, it explained it like that. Like suddenly, you know, it's like mainframe consumers, you know, have this whole um app store of options, you know, if they're an enterprise, you know, hoping to to bring in um the inference kind of closer to where their data is sitting and it's, you know, harder to get out of.

18:12 >> I I think both IBM and ARM are winners in this case as of now. I think I think right now I think this really is a I don't think this is a zero- sum uh game here. I I think there's opportunities for for both of these kind of established players. It's cool to see an announcement back in April of the partnership and then a few months later the announcement of the chip.

18:30 We'll still have to wait for its actual release. Uh maybe maybe I'm a bit doubtful for that, but um it's cool to see that sort of quick turnaround on this type of partnership. >> Yeah. So I want to bring in the perspective of a software engineer on this which is um this is going to make a lot of people's lives a whole lot easier. Um so you know a few months ago there was a big uh kuruffle in the market that certainly affected us at IBM where Anthropic announced their migration tool off of Cobalt.

18:56 Um, and the market worried that that meant the death of the mainframe. And we had a lot of well articulated responses out of IBM. But I think the core of all those responses was look, Cobalt is a means to an end, but the end has not gone away, right? The reason mainframes exist is bulletproof reliability. Like I was talking to I forget who it was uh earlier this week who was literally part of uh the team that said you could literally take the mainframe out back and put a bullet through it and it would not stop

19:30 processing transactions. And that's just not true of any other piece of hardware, right? Like you can you could shoot a bullet through a RAM stick on a Z system and it wouldn't blink. Now that reliability is what the foundation of many of our most pieces critical pieces of infrastructure is built on and so yes the there's a lot of code trapped in cobalt which is a pain in the neck uh and is you know probably very difficult for companies to find programmers that can actually handle that code but the need for that

20:00 reliability has gone nowhere and so this is an attempt to say well we can solve that problem in a different way rather than bringing the mainframe code out and and sacrificing that reliability that you've come to depend on, why don't we bring the modern code in, right? And as someone who has tried to cross-co compile code for arbitrary random architectures, um, if you're thinking about import torch as a Python program and you're like, what does cross-co compiling mean?

20:26 Well, let me tell you, uh, Python is a C program. Every single library that you import in Python is in fact delegating down to a C library if it's got any kind of performance behind it. um torch itself has backends compiled against every single uh accelerator architecture out there. So it is a massive pain in the neck of uh you know ecosystem targeted cross compilation and the idea before this announcement of taking something as complicated as an LLM that is backed by all of this complicated acceleration logic and

21:01 cross-co compiling it all to a completely bespoke bespoke is the wrong word completely uh firewall and separate uh chip architecture both for the accelerator and for the standard CPU processing was a daunting task and this was the the the purview of you know potentially years of work to get uh you know individual LLM architectures ported over to a Z system.

21:23 Now with ARM and the fact that a huge number of people are running ARM workloads on their laptops on their phones even starting to be on their desktop processors and server processors means that a huge amount of software has already been adapted to the ARM ecosystem. So that work is done for us. Um, so this is hopefully going to open up the floodgates of bringing a ton of interesting workloads to the mainframe that just could not run there in any reasonable amount of effort before.

21:53 Now, to your point, Skyler, it'll be interesting to see what that does for the reliability of the other workloads that are sharing the processor. Um, so the proof will be in the pudding there, but um, at least on the surface of it, this looks like this is going to make a lot of people's lives a lot easier. >> Yeah. And should we talk a little bit about the future?

22:10 Uh Ash, like I'm thinking about like in 2050, you know, will we still have mainframes? Uh it feels like this technology that people like really gripe about and are always like the main frame's on its way out. The main frame's on its way out. Uh but to Gab's point, right, like it is it is rare to find reliability of this kind. And you know, I guess the question is I guess in some ways you could almost read the trend line.

22:31 It's like actually mainframes will be here and sort of bigger than ever. Um but do you buy that? I guess because I'm kind of curious about like whether or not this like pretty dusty kind of ecosystem in some ways is like much much more robust uh and interesting than it looks. >> I think that um there are certain workloads where latency really makes a difference.

22:49 Okay. And the the latency um um issue just doesn't really exist in in mainframe. you know, you're getting like transactions processed in such a fast time that uh we will always need the ability to do that with certain types of transactions. And so I think that we will trend towards sort of bigger, more powerful computers of this kind pretty much for forever really.

23:15 um the the the the point that Skylar and both gave made. I I also like sort of have some um questions I would say right now and curiosity. Um you know this is a great sort of thing that's come out of this collaboration with ARM and you know I guess the when I think about this from sort of like a computer architecture perspective they could have just put some ARM cores into the chip and they didn't okay it's all combined in one and that's really really cool and really really interesting.

23:45 I would love to like go and talk to the people who made that architectural decision and go why did you decide to go the hard way because it could have been easier if you did it the other way. Um and yeah also that then raises questions of like well you know we're trying to process these things on on mainframe today which are like sort of millisecond type transactions and so now if you've got you know I don't know let's say PyTorch running on there and something else okay what does that do to the behavior of the overall

24:13 system and is that going to mean that it will like it will mean a much much bigger computer in 2050 >> yeah that'll be a really funny outcome is just like we're living in 2050 and it's like giant main frames Um that that'd be really interesting to see. I'm going to move us on to our kind of last topic of the day. Um you know, I think it's an adage in 2026 that it feels increasingly like we're living in a a cyberpunk novel of some kind.

24:38 Um this story I think was very much like it. Um basically on open router there's this mystery model uh called uh ox alpha that kind of hit um and people were very impressed by its capability and most interestingly you know at least in the announcement it was claimed that it was able to serve 100 trillion tokens a day um and you know which immediately raised questions about like what kind of compute are you running to be able to offer this kind of thing.

25:05 I think as of this morning it has finally been revealed that it is GLM 5.3 um open source Chinese model. Um and uh I guess maybe the first thing is we should just do the usual vibe check test. Um Gabe uh Skyler Ash, I'm curious if any of you kind of played with the model. What do you think about it just from a you know almost kind of like a wine review?

25:25 Like what what do you think about it? Is the hype justified? >> Benchmarks are one of these things that I think in this like probabilistic world are going to just be contested forever. So, well, some point, I don't know, about a year ago, I just put together sort of like, you know, my own little like benchmark suite, whatever. And I ran that through Open Router with it, and I thought, you know what, this is actually pretty good.

25:46 Pretty good. I mean, it's great that it's free. It's not free now, but it's still dirt cheap. And um I I was pretty pretty impressed with with the output that I got. Yeah. I mean, I gave it the smell test as well. Um it smells like a good model. Um, but like with wine, I at this point can't actually claim a connoisseur's pallet. Uh, because most of the workloads I want to run can be satisfied by a sub 30 billion parameter model on my DGX Sparks.

26:12 So, thank goodness. >> Yeah. So, you know, little little plug for small models there. But, um, you know, my my feeling is that it is fantastic to have competition at the top end. Um, love seeing it. uh very curious whether this throughput number that they are claiming is due to breadth of scaling and just raw compute or whether they've done something truly novel in the model architecture that enables massive throughput improvements while maintaining this top-end quality.

26:41 Um that would be genuinely very cool if they figured some tricks out about how to you know get the bits through the attention mechanism faster. Um, but uh, at the end of the day, we have a glut of very, very, very good models, and that's awesome. Um, and I'm still going to try to run as much as I can off my local machine. >> My interest in this story is as much about how they decide to go secret, and, you know, I'm interested in your thoughts on, you know, presumably intentional.

27:08 Does it build hype? You know, is there more to it than that, that they, you know, it was going to be revealed, you know, at some point, you know, why the why the stealth mode? >> I think the stealth mode worked. I mean, in between the time where we had these topics chosen for this uh for this podcast and when it's actually recorded, they came out as GLM 5.3.

27:26 Would we be talking about the release of GLM 5.3 if that was just how it would came about? Or is it so much more fascinating to have this idea of a hidden model? As Tim pointed out, it's like a cyberpunk novel where you can have these, you know, mysterious people showing up and competing in a I don't know, a medieval joust with a with helmets on. we don't know who they are.

27:48 Um, and so I think definitely well done uh uh to the makers of the model to release it in this way. They got the hype they wanted to a few days of great speculation u and then they released saying it is just you know an improvement not just it is an impressive improvement over one of their previous models. Um so I think I think they really did a great job with um the the hidden reveal um letting letting the internet talk about it for two or three days.

28:15 And I I will say another thing to that which is that timing a model release is very hard. Uh you know we at at IBM just released the granite 4.2 models which um you know we are proud of. Uh they are certainly not going to win the benchmark race but we think that they're going to be very useful for the the target audience. Um however uh a day after we did that we had Quen drop yet another benchmark busting amazing model that also runs on the same local hardware.

28:44 Um, and you know, picking when you want to release and how you want to release is a a very difficult game of speculation on who else is going to be releasing competitive models in the same size, in the same space. Um, so so like you said, Skyler, I mean, kudos to the team for choosing a um different route to release because if this had just been just another amazing Chinese model, like what what a world we live in, by the way.

29:10 Um, you know, would we even bother be to talk about it? But um choosing a release strategy that almost was uh competition proof because nobody else was doing it. Now I don't think anyone else can probably pull this off again for a little while. I mean we had it kind of with Nano Banana for a while. Um this isn't the first time a lab has sort of stealth launched a model with a a catchy name, but um it is a good strategy to mitigate against oh shoot is one of the other frontier labs also going to release their new model

29:39 on the same day or the same week? Um who knows? or sometimes within, you know, I'm recalling one uh opening anthropic pair of releases within like an hour of each other where it's sort of it becomes uh the sort of playground battle of who gets the most attention. I >> I think that they did have like a like an announcement to say, oh, they wanted to release it in a sort of stealth mode so they could get unbiased actual real feedback from developers to to know, you know, how good their model is in in all of those use

30:10 cases. I do think there's probably an element of that in like why it was stealth, right? You know, they didn't want to say, "Hey, it's a Chinese model." And for it to like introduce any sort of bias into determination as to the the model's um performance. Um I I do also think that there's probably 50% of it is marketing, right? It's a great marketing thing to do.

30:32 >> That's great. Well, that's all the time that we have for today. Um Gabe, Sky, Ash, it was great to have you on the show and uh it was great co-hosting with you today. >> Thank you, Tim. And thanks for joining all you listeners. If you enjoyed what you heard, you can get us on Apple Podcast, Spotify, and podcast platforms everywhere. And we'll see you all next week on Mixture of Experts.