Why Top Founders Are Racing Into AI Infrastructure Transcript, AI Summary & Key Points

00:00 We have a whole new technology. That's the most important technology ever. And you need a whole new infrastructure. >> Normally when we talk about the infrastructure world, we're talking about the servers on the storage and the network. Here it goes all the way down to the mines of copper min. That's how widespread this thing is going to be. >> It used to be when you built something, it was an engineering problem.

00:17 And here it feels like it really is a resource limitation. So whether it's tokens or not, we're pouring a ton of money into systems and then those systems are producing a result. And right now we're bottlenecked on the systems ability to actually match the resources we're pouring into them. >> The leading memory vendor said the demand they have today will take them 3 years of capacity to supply.

00:38 >> If this fund does what we think it will do, how do we see the world in 5 to 10 years? >> America wins in the infrastructure and that would be awesome. >> Ben Martin Ragu, welcome. >> Thank you. >> All right. Thank you. >> I want to start with a Mark quote to introduce this new fund. This is the biggest technological revolution of my lifetime. This is clearly bigger than the internet.

01:00 The comps on this are the microprocessor, the steam engine, and electricity, or maybe the wheel. Guys, the machine age fund. Please introduce it. Ben, start us off. >> Well, um, basically what's uh happened is we have a whole new technology that's the most important technology ever. And what happens every time um there's a dramatic new way of using all of the things that we love infrastructure um you need a whole new infrastructure and never has it been more high impact as it is on this one.

01:38 So not only do we need new chips, new system software, we need new ways of doing power, we need to replace copper. I mean like it's absolutely everything. So it's a very exciting time. So you know particularly for the kind of hardware aspects of of this new era um we needed a new approach. >> Yeah I would agree. I mean normally when we at least in the computing when we talk about the infrastructure world we talking about the servers and the storage in the network here it goes all the way down to the mines copper mines.

02:16 That's how widespread this thing is going to be. Um that's number one and number two I think what we have seen over the last 3 years is the steady increase of the capabilities of the models where the model is no longer the bottleneck and in fact using AI these models are getting better faster and faster and faster. Now the bottleneck is all what I call south of the model and so that's why we need to work on that.

02:40 You know, the only thing I'd add very quickly is like we tend to follow founders and we've been watching over the last couple of years is the number of very strong teams going after complex hardware problems has increased. I don't know the actual numbers but I was trying to estimate it over the weekend. So I think we'd get you maybe 5% of the deals from top founders would come in would be hardwood before.

03:00 Now I would say north of 20% or 30% right now. So like the founder community which tends to be much smarter than the VC community has identified this as a very active area for innovation and they're responding. I think 5% is probably generous. >> Yeah, it's very low. Is very low. Yeah. 3%. Yeah. >> And explain some of the macro conditions that have led to this this trans this change in terms of the surplus of founders pursuing these idea.

03:20 Like what are they seeing that's that's enabled? >> Well, I mean the obvious is like you know the the demand for AI is basically infinite and as a result of that every part of the supply chain is under under duress. I mean everything including like materials used to make things like like memory. Um it's also very interesting whether there's something unique about AI.

03:41 Um which because the demand is infinite and growth is infinite. Um uh what you tend to worry about is the margin of companies which is how efficient it is. Like normally you worry about growth like can I just you know can I just get people to buy this stuff. You don't have to worry about that here. The question is is can you do this in a way that's profitable?

03:57 And a lot of the um efficiencies are actually strictly a physical limitation of hardware. And so even the business model of the AI wave is really putting a lot of stress on the existing systems because they weren't built for AI. They weren't built for those workloads. And I think there's just this you know this global observation that we actually need to change the core components to get that efficiency to help drive the growth and to drive the value of the businesses.

04:23 >> Yeah. And how did we know that demand is actually outpacing supply here rather than this being you know another hype cycle? Well, I mean, >> yeah, there are any number of cities today. Um, firstly, it is that some of the smartest judges of demand are cutting huge purchase orders. I mean, if you look at the hyperscalers, right? >> Yeah. >> Their capex spend has been exploding.

04:50 Next year supposedly, it's going to reach a trillion dollars collectively across the big hyperscalers. This year, it's about 700 billion dollars, right? And if you think about the hyperscalers position in the industry, they see demand from everywhere, right? They see obviously the frontier labs um wanting their compute, they see the AI native companies, they see the enterprise, they see the US geography, the international geography.

05:17 So if anybody has visibility, it is down and they've been jacking up their capex like it's never been seen before, right? So that's a clear clear sign. And secondly, if you look at the companies that we see on a day-to-day basis, they are all ripping. All the application companies, the growth is insane. The frontier labs, the growth is insane. It's been documented.

05:39 So I would say on the demand side, the signals have never been clearer that this is not a hype. It's to top it all, all of it is just and prices are going up. >> Like we've never seen prices go like up on chips. >> You keep your prices went down. They always go down. Yeah. It always goes down. If you look at the price curve, it it went like this and then went back this.

06:04 >> Yeah. >> And we know only like 5 10% of the address for the market to stop today. >> I mean, the the the supply, if you look, if you look at the supply across the board, it's basically all booked out to 2028. I mean, it's so bad we've actually seen multi-day auctions for a few thousand GPUs. Um, you know, the other side of that, of course, is demand.

06:24 And as Ragu said, we've seen the fastest growing companies we've seen in the history of the industry. >> But also the unit of work that AI can do, the value of that unit of work keeps increasing. But underneath the covers, the number of tokens that are consumed is going by orders of magnitude, right? If it's one token for I mean 100 tokens for chat or an agent, it's thousands of tokens, right?

06:49 So you got expansion of both sides of demand. One is the unit of work is becoming more and more consumptive >> of tokens and then secondly the number of people therefore that are going to be benefit it's not just the developers it's going to be all knowledge workers and then all of beyond that so that's that's what we see >> you said the key components in supply are sold out to 2027 maybe in 2028 >> what does it mean for an entire industry to be sold out that that far like >> I don't know this has ever happened before

07:18 do you guys recall I mean remember in the Um, in in in the internet days when we were doing massive buildout, the majority that was actually being put in the ground was speculative and was dark. Remember the dark fiber? And here basically every GPU that's being created is already pre-sold. So >> yeah, we weren't quite there. I mean there was there was a lack of bandwidth like in the 989 time frame, but there wasn't there there wasn't that much real demand for it because there just weren't that many people on the

07:55 internet. So like it was a two-sided thing and the companies were all rushing there and needed more bandwidth theoretically, but there weren't the users on the other side to consume it necessarily. And then to really consume a lot of bandwidth, you have to do high bandwidth things like video, which weren't really viable for, you know, a number of reasons that that had nothing to do with how much bandwidth was in the data center.

08:21 So it smelled similar, but it wasn't this. This is like we're flat out and people are reselling GPUs for four times what they bought them for and this kind of thing. Like it's just not. And then, >> you know, we're we're also out of power and cooling. Uh and then on top of that, it's really hard to build because there's these incredible political headwinds going into it.

08:49 So, it's it's really unprecedented in my career. uh that we've had anything anything like this. No, >> I want to give you a quick anecdote. So, this I was talking to a CFO of a large company, large public company who had historically been very resistant about going into the cloud. So, they had a lot of servers and they were doing an inventory uh check and they realized that the memory in their servers had increased so much it could uh fund the entire migration to the cloud.

09:15 So, I just feel like we're in a very unusual situation. >> Yeah, that's right. We're out of many things. power, cooling, memory, GPUs, like you name it, we're out of it. >> Yeah. So, the flagship uh conference for the industry is the one called hot chips is going on in Stanford and the leading memory vendor said the demand they have today it will take them 3 years of capacity to supply it.

09:44 It's just today it's not even future demand. So in terms of being about everything simultaneously, is it because people just underestimated how good the models would be, how how useful they would be, they just couldn't have foreseen the demand? >> Well, I don't even think it's that. I mean, this stuff came out of nowhere, right? We we're only four years into this.

10:05 So even if we had a perfect oracle, once it started working, I don't think >> we could have built the capacity. >> We could have built the capacity. There's no way. And we're talking about like chip cycles, which tend to be three to four years. We're talking about breaking ground and building data centers which is you know four to five years we're talking >> and connecting breaking down and building them and having uh power source.

10:25 So like you either have to build your own power or like usually both you've got to build your own power and have a power source which is not easy. Yeah. the ML industry that's used if it's growing at 20 30% it's a great growth rate right and it's being connected to an AI software industry that's like triple digits is the base you know >> so you can see the disconnect right so it's it's just wide bank >> and so why didn't this fund exist you know five years ago or seven years ago or why was it not a great category to to

10:58 invest in in in the same way Brad >> well I would say we're probably I'd like to think for just in time, but you know, we probably would have been uh well suited to have it at least a couple years ago. >> I will say you could actually point on on on basically every epoch to uh an independent company that came up, right? Clearly the move from the client from uh in mainframe to the client server, we saw a bunch of companies come up.

11:22 Um the move to the internet, this we got Cisco and Juniper. uh even in the mega data centers which by the way was largely driven by the incumbent cloud providers verticalizing you saw the uh arising of Arista so there has been the ability to invest in you know silicon and hardware but it's been relatively minor because the change has been relatively minor like one chip company one switch company where here everything is and so I I agree with Ben we're probably you know we probably could have started a little bit

11:52 earlier um but the amount of change is so high now that it's just an obvious thing And the other thing is the demand for intelligence is so vertical um with really no end in sight. I mean cuz every company that's adopted it is growing very fast in its usage and then most companies haven't adopted it to a high degree and then consumers are just getting started.

12:24 And so it's going to probably the demand for tokens is probably going to grow close to a,000% a year, which you cannot grow supply that fast. Like like we're not >> like the amount of just work we're going to have to do across the board to get to the point where we can grow like infr at that kind of rate is is pretty vast. So I think there's a lot of investing opportunity on the way.

12:52 And by the way, the other thing is like all the architectures of the hardware systems were built for a whole different era of computing. And so more than just we need more capacity, we need capacity to build there there's lots of opportunities to build different kinds of infrastructure. Yeah, they're they're all reaching their the physics limits for what they were designed, right?

13:21 Like what Ben was talking about copper and so on and so forth. And you could go across every one of these categories and you could find, okay, this is the limit of this type of technology. So now you got to get some technical breakthroughs to get to the next one. >> Yeah. I want to dive deeper on the demand side for a second. As we've moved from chat bots to reasoning to agents to to multi- aents, each step has multiplied the number of tokens a single task takes up by orders of increasing orders.

13:53 >> Yeah, nobody likes to use AI more than AI. >> So, um why does that keep happening instead of a leveling off? Do you just see that happening you know indefinitely just continuing to >> uh well so there's a couple as that the first one is is for sure right now if you look at like the the way we're achieving scaling the way we're doing it is through a lot of inference so through a lot of token right if you think about what RL is you know it's it's a lot of inference if you think about chain of thought it's a lot of

14:21 inference u think longunning agents of course it's a lot of inference and so that's just basically been one of the approaches that we've been using to um uh to scaling um I think if you want to step back and say kind of what is the macro trend here it used to be when you built something it was an engineering problem and you throw a bunch of engineers at it and that doesn't scale and that would have a natural law of engineering physics uh which is what the mythical manmouth came from and here it feels like it really is

14:48 a resource limitation so whether it's tokens or not we're pouring a ton of money into systems and then those systems are producing a result and right now we're bottlenecked on those position those systems ability to actually uh match the resources we're pouring into them. And so I think like tokens right now is probably where we are on the scaling curve, but we don't have a natural regulator like engineering like we did before.

15:09 So I think we should expect this to continue and we have to build a supply to support it. >> Yeah. Like the simple way to think about it is any problem that you have can be solved with enough infrastructure >> basically >> GPUs and power and money. Uh and so until we run out of problems, we're not going to run out of demand. And that's the that's the uh challenge.

15:34 I >> mean AI's answer to getting better and better is to use more AI, right? inference is one basic building block that it keeps using over and over and over again and so that's why these tokens multiply at each step. >> Yeah, even the autoc catalytic effect so even the idea of using AI to create more AI like creating a GPU kernel of course is just using more AI um as part of the process.

16:00 So again, one way that we think about it is in the past money would come in, you have an engineering problem, we know that it takes two years, normally fails, you know, it's a national governor and then you get the product on the other end. This there's there's there's nothing between the money going in and then the hardware, you know, creating intelligence.

16:19 And so now we're just limited by our ability to create supply. It's a very very different >> be more GPUs and our sol. >> Yeah, that's right. >> So that's the cycle. as long as you have the money, the GPUs, and the data, you know, for the foreseeable future, you'll be able to scale these things. >> And it's it's fascinating because, you know, over the last decade, it feels like there so many, you know, people and the pervasive sentiment was there's too much money going to startups.

16:43 We're overfunding these startups. There's too much money in in in in venture capital. Say more Ben about what that means because there there used to be this um sort of skepticism that the more money you put into into the industry that you know there would be bigger outcomes and now you know we were saying at the offsite that there's to some degree the the market is as big as we collectively contribute to it.

17:05 >> Yeah. So, this is this look, the one thing we all knew with in startup world is that if I have a two-year lead on you and you try and catch me by hiring a thousand engineers, you're going to wreck your company. Like, that never works. It's a mythical man month. Nine women can't have a baby in a month. That like that's it. Like, that never works. Okay, now that works.

17:31 But it's not hiring a hundred thousand engineers. It's taking $3 billion and like lighting up a magnificent cluster and then all of a sudden, you know, whatever Grock can come out of nowhere and like, oh, all of a sudden it's real or or a Kimmy or or what have you. It's just like these leads um you can throw money at the problem and you can throw money at almost any problem and that works.

17:57 And so that is just completely different than anything we've ever lived through. So we're, by the way, we're all psychologically adjusting to this. The Chad GBT app has a billion weekly activives. There's about 30 million uh developers um who are using, you know, relatively a big portion of compute demands. How do we think about compute compute demands needs now and in the future in light of what people are actually doing with AI?

18:26 >> That's the progression, right? So Chadic was a casual app and it's coding for professionals right now using coding you have now built amazing tools for knowledge workers so that's the next frontier and now there are over a billion knowledge workers in the line right and with that it's a long ways to go for that demand and by the way the work that they do all this workflow and automation and so on and then you get to the back office which is all the agents So progressively each of these things unlocks uh I would say

19:00 order of magnitude more demand. I mean just at the start of this >> well and now you have Grockbot which is kind of uh you know what uh happened with coding is kind of happening with all use of computer viaot and so we're in a whole another wave of demand and most certainly there's going to be more to come. Uh so it does seem quite unlimited at the moment and we haven't even gotten into embodied AI or robots uh which are going to be another source of demand.

19:36 >> Martinez is expert but my understanding is that part uses computer use which is just like >> human being sitting inside the computer typing away. >> I literally used it over the weekend to to update my credit card with a bunch of services that I'd been like lazy to do and cancel a bunch of subscriptions. I mean, this is not coding or whatever. This is true computer use.

19:55 >> All of a sudden, you're creating like half a billion knowledge workers except they're all sitting inside of the computer doing what? >> I I do I do think that that that Mark Mark Andre is right. It's like the right analog here is like the steam engine or electricity in the following way. Like we've we've introduced this new thing that you can turn to work and there are some very obvious applications now, but there's probably 30 40 years of throwing computer at problems.

20:19 anything with a clear reward signal and we we're just starting like we've got language and code that's it and just starting computer use but like what else are we looking at we're looking at uh in terms of science materials biology I mean of course creativity is a massive use and so listen we're at the very very early part of a very long journey and we've removed this key bottleneck which is you know traditional software engineering now of course you know bottlenecks will move and there'll be kind of more complexity

20:45 elsewhere but I think we're at a very early in a very long run of throwing computer problem. So let's expect you know this compute need to persist for decades. >> But because we mentioned it u Marty talk about grabbot um because we um we're talking at the offsite about how you know what what struck you about it. Obviously we're involved in every possible way you could you could be involved but what um yeah what what did you find so interesting about it?

21:07 So I think we've I think we've as an industry gone through kind of multiple realizations for how AI enters our lives, right? And and uh very early on we're like okay well you add AI to a product and it's like whatever it's like a search bar and then you kind of you know you you do chat with it and it chats back because that's kind of the traditional way to do it.

21:26 Um uh and then Open Clock kind of showed up and that was it earlier in the year. And with Open I say, "Okay, well maybe like it just being like Google but better. Maybe that's not the full embodiment of it. How about we'll have it be a standalone thing but it'll be an extension of you and it'll share your keys and it'll know your passwords and it'll just kind of do stuff that you would do, right?

21:45 So it's kind of an extension of you but it's more like a human, an extension of you." And then what I think Rockbot got really right is no, how about it's actually an employee. So now you have this thing that's an entity and it doesn't have like special access to your keys or whatever. It has its own computer and it has its own browser and because these are the smartest models in the world, it can do whatever an employee can do.

22:06 And it's kind of interesting because now actually if I if I want something done my first thing I I think is like well can do it for me and and often the answer is yes even if it's something you wouldn't you know expect it. So, the obvious ones are like whatever. It'll like manage my calendar. It'll like book a meeting, but there's also non-obvious ones as well.

22:26 Like, so, for example, I'll have it um uh read through my email and do triage. And and I did I don't tell it how to do that, but it will know to check with me before actually doing the triage. So, like these things are sophisticated enough that you can give a relatively high level task and it'll do kind of, you know, like sophisticated things as a result.

22:46 >> Ben, I know you're thinking a lot a lot about this and how this you know, works in the organization. you think a lot about culture of course. What are your thoughts here? Well, I mean, I think if you just look at us, um, you know, it's like having a new kind of employee and there's going to be a lot of them and we have to just like, you know, with our we spent many, many, many years figuring out how to work with our kind of regular human employees and now we've got these other kinds of employees and, you know, there

23:18 is a learning curve with them. So um they can burn a lot of tokens and spend a lot of money and get nothing productive done. Um they can forget stuff. They can make stuff up. Um you know they can have good behavior. They can have bad behavior. They can >> they can create security problems. So like there there there's all those aspects to it, but like they can also be like super duper productive.

23:46 And so I think figuring out how to integrate them in, have them work nicely with the people that they're working with, um, the actual humans, uh, is all something that we're learning how to do. I mean, I I don't want to sit up here and say I've cracked the code. We've got this more of a sloop and the whole firm is just completely automated now, and I'm going to slowly get rid of all the humans because I can.

24:12 Like, that's not at all where we are. were much more going like, okay, how do we make all our humans superhuman um without like wrecking the place um because the bots got out of control? >> Yeah. And it's interesting because we've done we've tried a couple of different ways as how best to get agents into the system if you will and eventually it was Martin's insights just treat them as people and get it done.

24:38 And that's what we're doing. that's turned out to be the most durable way of getting this thing going inside of an organization. >> I want to go back to the supply side and go deeper into the the bottlenecks. You we were talking about how you know in terms of the data centers, the chip architecture, system software, facilities themselves that none of them were designed with with AI in mind.

24:57 What would it look like for them to be designed with AI? Like what is sort of the mental model for thinking about what that could mean? >> Yeah. So um I mean if you start with a statement that you just say hey original model um of infrastructure on any of these models has to change you can go se category by column and see how it bricks right and then you start uh unlocking the bottlenecks in each one of these things.

25:27 So eventually you have to get to a system where if you look at what an inference engine does, right? It takes up a lot of memory, it generates new tokens along with the compute. And so you can just think about how do I optimize all of this? What does the memory need to be? What does the compute need to be? How do they need to talk to each other? How much power does each of them need?

25:49 And if they need all of these power, how do you cool each of these? Right? And then how do you put the collections of these things together? That is the exercise that's underway in the industry right now with a lot of the founders. So they're breaking down the problem into its fundamental components and saying what is the exact nature of the compute that's getting done.

26:07 Okay, it's how it's going to be matrix multiplications. How do I optimize my compute around that kind of a scenario and then they all need very progressively to generate these tokens? What is the best way of hierarchically arranging this memory right and then how does the power consume I mean then you got to connect it together what are the ways of connecting it on the same chip but across chips and across data centers how much power does each of these data transmission take so you have to progressively break it all

26:37 down and rebuild it from these fundamental building blocks and that's what we see underway and that's where we see the opportunity >> let me give you an interesting mental model to think about how the landscape's changed so um uh so today to build a frontier model costs let's say $3 to5 billion right so and let's you know and and that's to train it and so the inference has to pay back at least that of course right you know in order for any of this stuff to be viable let's say two times that so let's say that now

27:07 inferred has to to make $10 billion so if you can save 20% of efficiency on that that's $2 billion and you can easily build an ASIC for $2 billion Right. So, so we've actually gotten to this interesting point in the industry where it actually makes sense to build an ASIC per model just because the amount of capital investment in that model and then unlike traditional software, traditional software has a lot of state and a lot of you know is very dynamic.

27:37 These models are fixed. The model weights are fixed. And so we don't know if the world goes to per model A6. But it gives you a great mental model of how you would evolve the architecture to be far more bespoke for these massive capital investments we're doing. Like I don't think in the history of the industry we've ever created a digital artifact with something like $5 billion that went directly into that artifact.

28:01 And so you know like this I think is going to put the greatest demands on hardware that we've ever seen. Well, to that end, rack power requirements are moving from roughly 5 to 10 kilowatts to 100 to 50 kilowatts. Compute density is climbing something like 70x. >> Cooling is moving from air to liquid as a requirement. What are the investment opportunities as a result of this?

28:24 >> Well, first of all, when you get to that level of power per rack, AC power doesn't work anymore. So, like that's a pretty wild thing. Um, so now now you're into DC power, which by the way also requires its own cooling. Um, and is like, uh, by the way, super dangerous. Uh, which is kind of ironic because this was, um, Edison promoted DC power by claiming how dangerous AC power was in demonstrating it by like electrocuting animals and things.

29:00 >> The horse. Yeah. >> The horse. Yeah. So but he was right but around his own kind of power which is extremely powerful is the good news. Um so you know just starting with power uh yeah that's going to be like very very different I think with cooling. So and this gets into so yes we're going air cooling to liquid cooling. I think we're already at liquid cooling for any state-of-the-art data center.

29:28 Like that's already kind of a done thing. >> But it gets into okay, you know, given the political environment and so forth, you like liquid cooling isn't enough. It's got to be eco-friendly liquid cooling. Um, and you know, kind of DC power is not enough. It's got to be um power that contributes uh to the power of society, not takes away from it. And so you have data centers who have been behaving badly.

29:58 Um small percentage actually probably 10% wasting a lot of water. Um you know not as much as pistachios or almonds and so forth as people demonstrate on the internet but like they could be a lot more efficient with that. Uh and then there are ones that you know kind of are uh parasites of power and don't contribute power back. I think all that's going to end.

30:22 it's going to have to end just because like like that we we've kind of gone through a one-way door on that. Uh so that requires like a level of engineering um that you know many haven't invested in yet. So that's coming. Uh and then you know like if racks are that dense um there are other things that like the way the floors are designed have to support that kind of weight.

30:51 uh you know that kind of thing is is actually for real. Um and then you know I I think that there's you know you just need a lot of everything and so there's going to be you know kind also by the things are really loud so you have to build the data center with thicker walls or you're going to disturb the peace in the neighborhood which is not going to be acceptable like I don't think any state's going to allow that.

31:18 >> And so a lot of the ways people have architected and designed the buildings themselves are already completely obsolete. Like once we get to Fineman, um a much smaller percentage of the data centers that we have today work. In fact, I mean everybody talks about memory prices, but one of the fastest areas where prices is increasing is reinforced concrete from for their sake.

31:43 >> The other thing that happens when these data centers are uh sending 800 volts to the whack is it's become so dangerous number one, but secondly, we don't have enough electrical contractors that have the expertise to deal with the 800 rules inside the data center. because this is high voltage. Only 2% of electrical engineer or uh electricians in the US have been certified on DC power.

32:11 So like that gives you an idea. Now Meta's got a whole program to train people up and so forth which is great. It's like a new job court where they train people for free to do this job. But you know it's funny AI is taking all the jobs. AI is going to create a lot of new electricians. >> Yeah. I think we're just doing something in space too. the big guys that own the big cloud data big data centers, they all are furiously experimenting with the robots, right?

32:41 To do the work of assembling or putting servers into the data center, etc. And so you will see that increasing as a result of the evolution in AI. >> By the way, to be to be clear on the um on the actual fund that we're raising, our focus is on computer science infrastructure. So anything a model runs on that's computer science right so think you know chips network interconnect storage all the way down probably to the electricity >> yeah and and saying more about the robotics arm in terms of what we'll be doing versus

33:16 maybe American dynamism or how to think about that >> yeah yeah for sure so um you know again we we think that any platform that that AI will run on like one of the the the great breakthroughs that AI does is it allows computers to interact with the physical world, right? It can see, it can hear, it can talk, right? And this means new platforms, right?

33:34 And the simplest way people say edge device, but that doesn't really mean anything, right? I mean, it could be a mobile device, it could be a CDN, it could be a laptop, but it also could be an embodied, you know, device that goes around. And so again, we we um as you know, as infrastructure focused investors don't do heavy regulated industries um or more verticalized industries, but any sort of computer science platform that's going to push AI further out, we're quite interested in.

34:04 >> Yeah. Going back to the data centers, by 2028, new data centers are going to need something like 44 gawatts of additional power against maybe 25 gawatt of expected grid additions. >> Hold on, hold on. We use that word gigawatt. >> No, it's like it's like we'll have 100 gigawatt, >> Martine. What's a gigawatt? >> I mean I mean how big is it? It's multiple football fields.

34:25 I mean it's massive. It's 50,000 50,000 people. >> What do you mean? What is it? >> Like like >> the equivalent is like 50,000 houses. >> City >> 50,000 50,000 homes. 50,000 homes. I I I grew up I grew up in Flagstaff, Arizona, which is a town of 40 to 60,000 people, depending on the universities. We have less than a gigawatt of power consumption. I mean, this is >> so you can basically light up and air condition your entire town for a gigawatt.

34:54 >> Yeah. I mean, this is >> just throwing them around. >> No, but by the way, everybody talks about the gigawatt. There's very few gigawatt data centers that are actually up. I mean, we've got a long way to go, right? But then why can't utilities and hyperscalers just build faster? Oh, there's so many things there. Well, there's first of all, right now, you need humans to build them.

35:13 So, there there's just like the regular construction, but much more than that, you need permits. Um, you need access to power uh that you can plug in. So you're either you're doing a combination of you've got to get access to power which is a massive kind of regulatory bidding struggle. There's very limited kind of amounts and things you can tap into in terms of n natural gas power grids, what have you.

35:45 Um but then you also have to build your own power and guess what? We've got shortages of transformers and turbines and everything that goes into that. So, it's just, you know, like you've got to get all that stuff. It's not this is not a software problem. It's not just like a bunch of engineers can't you like work weekends and that type of stuff like this is not that that works anyway, but uh there are real bottlenecks in this and these these lead times are not that easy to compress.

36:19 And look, we have the best minds in the world trying to figure out how to compress them. Uh but I it's not easy. It's not easy and the demand is not slowing down. So we're already behind. The demand is growing, you know, 10x a year right now. And you the supply just can't grow that fast. >> By the way, it is so bad that right now if we have new companies going for GPUs, it's often in Mexico or Australia or another country just because it is so difficult in the United States.

36:50 Yeah, we're creating huge both job and long-term economic opportunity in other countries by uh banning data centers here. I think look the the right answer would be to set a standard where a data center contributes back to the community like that power gets better, there's no noise, there's no water issue, um and it's adding jobs like that ought to be the standard.

37:19 Yeah. And then everybody ought to be just held to that center. And by the way, like there are there are data centers that do that now. Like that's not a you know like a a futuristic dream or something. Rates energy rates have gone down like every year they're there. And the reason is they provide their own power. They give power to to the state during the day and then at night they borrow power from the state when the state doesn't need it because you're always the the way power plants work is you're always generating

37:51 peak uh capacity and since a data center has steady capacity during day and night and a city goes way up in the day and way down at night um that's a symbiotic relationship. zooming out. Uh why do we think the you know we we've been batting around the name for for a little bit. Uh why do we think machine age is a is a compelling term for for what we're doing here?

38:16 >> Well, listen, let let me let me take it. So, the first one is I think Ben's absolutely right. Artificial intelligence was the wrong word. Like we shouldn't have called it it's machine intelligence. Um >> say more about that. Why is that? >> Uh because it's not how humans think necessarily, right? I mean, it is a cache of how humans thought is a collection of humans thoughts.

38:35 But like to date, we don't know how to take a um an AI with no knowledge and put it in out in the world and have it reconstruct language, right? Like that's not what we've done, right? We've we've built a something that can learn off of everything we've already learned and then use that in a productive way. And listen, a a AI is a general term that goes back 70 years in computer science formally that applies to many different things.

39:00 And of course it's got a lot of baggage either from science fiction or from you know Nick Bstrom who wrote about it or whatever. And so so the first one is just an acknowledgement like this really is machine intelligence. And then you want to emphasize the machine part of it. I mean there's kind of this deep irony and this is from you know the uh the software's eating the world people that you know you've really come to a place where you pour money into something and then you're limited by the actual machines below it.

39:23 And so I think it is a kind of a nod to like the hardware component is so significant in this wave and and we want to acknowledge that. >> Yeah. I think that's what's going to create the next breakthroughs in intelligence is the quality of the machines underneath. And so that's that's basically a reason for the name. >> It's also a cool name. >> That sounds good.

39:44 >> Futuristic. >> Yeah. the um given how much has been spent on AI infrastructure to date and how capex intensive these businesses can be are we past the point where new companies can break break in at at at sort of material levels uh you know you know in why not incumbents like Nvidia Core etc just take the line share of these markets they're all doing well there's no question about it right but to our discussion earlier when you're get you need funly new innovations to keep the growth both continuing or the pace of

40:18 improvement continuing whether it's tokens per second per dollar or tokens per watt uh tokens per rack right um or power you take any metric if you want to have a 10x on those metrics you got to have new innovation and new innovation traditionally comes from brilliant founders thinking about solving the problem from first principles in a different way right and that's what's needed here for the next jump in innovation.

40:47 >> I mean this is the law of markets, right? I mean let's assume that the the existing silicon incumbents are multi- trillion dollars in market cap which is absolutely the case. >> Even 5% of that is a massive private company, massive private company, right? We're talking, you know, annual. Um and you could say, well, but Nvidia could do that. They could, but why would they if they're focused on things that are in the 90% which is also driving the same amount of growth?

41:13 You always ask these questions. We ask these questions during the cloud days, right? Like why wouldn't Amazon did this? You would ask these questions during the Microsoft days. Why wouldn't Microsoft do this? There's a very natural law of markets is once you get to a certain scale, there's tremendous opportunity for innovation um at at the margins. >> Yeah, there's a funny uh quote from our partner Alex Rmpelle.

41:31 He had this startup called Trial Pay and he was trying to sell it to or sell the it his services to to Meta and uh then Facebook and Dan Rose who was the head of corp dev at the time said Alex that's great you're it sounds like you can collect a lot of silver bricks but I'm like I have so many gold bricks I can't even pick them all up so the last thing I'm doing is looking at a silver brick and I think Nvidia is in that position >> 100% yeah we were talking about as relates to the model providers that if you're, you

42:03 know, in the sweet spot of of what open airropic can can do, you know, one of their main sort of interest areas, that might be a tough place to be, but anything outside of those maybe, you know, three to five areas might uh, you know, as as markets expand, they fragment, right? And it happens all the time. And remember in the early days of Ford, there was the 1913, there was the Rouge River plant.

42:24 Literally this, you know, this was like made cars like in went like water, coal, and rubber trees and out came cars. >> By the way, he bought a whole >> rubber plantation in the Amazon jungle, >> right? And and there's a great book called Ford Landia where so because he wanted to like own like the complete vertical thing where he created this city called Ford Landia in the Amazon jungle um which had like was all Americanized band stands and ice cream and all this kind of stuff.

42:57 And it actually worked for a while until uh he made people like show up to things on time and then they were like screw this get the out of here. So, so now if you look at the car industry, of course, there's multiple levels of supplier and there's a bunch of companies and this always happens. So, you know, as markets expand, they fragment. There's a lot of and then and then once that growth slows down, they tend to consolidate.

43:18 The consolidation can either be acquisition or it can be like new challengers rise up and that that is the, you know, everlasting cycle of private markets. >> Yeah. the use cases are are multiplying and there's no way like if you're the biggest company you can get to the biggest use cases but there's so many use cases and all as Martin was saying very valuable use cases that it's just very hard to get to in a great way >> yeah even inference used to be one simple architecture right and it no longer is like it's so

43:48 complex now so it's inevitable that you can optimize things in a different way >> by the way here's a very interesting thing like people don't um uh often don't understand that like the margins kind of fell out of the standard way of doing the technology with software, right? Like it wasn't really a technology problem. Like once you got the business working, you tended to have pretty good margins because that's just kind of how software works.

44:11 certainly when you shipped it but even as a service uh and that's not necessarily the case with AIS we may actually be entering an era where the optimization in the hardware is absolutely meaningful to the upside of the business in a way that we haven't seen in the past so there's a lot of opportunity here let's get deeper in talking about the types of companies we'll be investing in um maybe we could start by either illustrating the the the subsectors or if we can talk about a few um or a couple investments that we've

44:41 I know there's some that haven't been announced yet, but Robert, do you want to take us down? >> Yeah, I mean the sub sectors as we've been talking all is every one of these categories, right? The obvious ones are um uh computer chips, but these days it's not enough to build a chip. You need to build a full system, right? And then therefore, what goes into the system?

45:01 There's potentially memory innovation. There is potentially networking innovation. There's potentially power chips and so on and so forth. So each one of these categories are categories where you can see public company style companies emerging and those are all things that we are looking into. Um and then once you put it all together there's a layer of software around it to automate all of these things to manage these fleets and so on and so forth.

45:27 So that is another important area. So these things keep building on each other but every one of these categories is important. >> Talk about what's different about these kinds of companies from from the usual company. I mean one thing you can tell from the companies we announced is their first rounds have been massive. You know hundreds of millions.

45:43 Is it a different kind of founder or what else is different as we think about just the practice of you know building and investing these kinds of businesses relative to our traditional software? >> Well I I think the big thing is you hit on one of the big things which is a lot of money goes in um before they get to a product. Um and that's just kind of the nature of it.

46:02 Now that's true on big models too, but I I would say that's a little more of a known path. Uh whereas this has got a little more risk and a little more money than uh than some of the other things that we've done. But >> um and you know, look, a lot of a lot of the chip founders um are here from the past. >> You know, like the guys who know how to make memory, they're not young.

46:31 Yeah, you know, so it's uh you know that part is different too, but it's kind of exciting, you know. >> Yeah. The other thing about these founders, they have all got to be systems founders. So what I mean by that is you can't just be a researcher or a great computer scientist, right? You got to be able to architect and design the chip or the system, whatever it is.

46:55 Then you got to think about how is this thing going to actually get manufactured, right? who's going to be supplying this and a whole bunch of these downstream things which normally if you're building software you don't have to think about all of these things. So the to really the best founders um and of course Jensen is the Michael Jordan of this right they think the entire ecosystem right from the get- go before they start designing the chain right because of the nature of the bottlenecks and all these things that

47:28 have to come together. So that's that's a big characteristic that is different. >> There there's two environmental factors that are important too. The first one is the labs are so desperate that they will engage with startups. And so like we actually have quite a bit of signal early on because they're you labs are inking deals with companies before they actually have hardware available.

47:46 And that's a big big shift than you know 5 years ago, right? Like you just didn't go and you know sell your kind of janky hardware thing to Google or whatever. So that that's a shift. The second one is the the capital availability has loosened up a lot. I think there's general consensus that that you know it is the time that to reshape this stuff and so follow on rounds there's a lot of capital available which you know of course you want to be investing into areas where there's capital available and so the atmospherics

48:10 are also just different. Patrick Carlson, you know, remarked a few years ago said, "Hey, it feels like there's less younger founders today, you know, in the way that you know, Zuck, you know, in college building next Facebook or or Gates, um, you know, in the same way with Microsoft and of course, you know, the Michael Trolls of the world, there there's still, you know, some young founders building iconic companies, but it does seem, you know, to your point that there's more older founders building these these these

48:36 companies or or less 20-year-olds. I'm I'm curious if you resonates and why. Well, I think it's Ragu's point that if if you're building something that has like a very complicated supply chain, has to manufacture things um and is technically complicated that you know some experience helps uh and you know if you look at Elon or Travis Kalanick their companies when they were young were software companies.

49:08 It wasn't until they got like a lot even those guys the best guys um needed some experience in building a companies building technology and so forth to kind of graduate to the much more kind of complicated or I would say elaborate domains you know there's just much more there many more moving parts in these things and so look when you're learning how to build a company it's hard enough if you completely understand the product.

49:39 If you don't completely understand the product and have to learn it while you build the company, um that's just such a steep learning curve for a brand new entrepreneur. So, I think that what we're seeing is you see Michael on the one hand, um who is a very young guy, brilliant, but what he built was kind of a pure software AI thing. >> Yeah. And then on the other end, you have like an Elon or a Travis who can who's got enough experience.

50:07 I think Michael could probably do that, you know, 10 years from now, but today that would have been hard. >> And it's important to remember like it's been defocused by the entire industry and academia for the last 20 years, right? It just there just hasn't been the same opportunity. Like it's been there, but like it's never been a growth area. The growth areas have been, you know, software, um, networking, things like that.

50:30 And so I also think we just have a posity of people coming out of the universities or having experience at large companies that have have done this. I there just not that many. >> Like you don't go intern and like build a chip. So but a lot of that's changing now. like listen we're going to create a whole generation of of you know founders that come from these new companies that will know how to do this and you know they'll be hired in much more junior and like I would say actually one of the greatest legacies of Elon

50:54 towards this is is of course he's created these great companies but the amount of entrepreneurs that have come out of SpaceX that that have are changing the entire industrial complex maybe even greater um legacy than than the companies themselves and I think we're going to see the same thing uh for for computer science and hardware >> y as a matter of All of our investments was started by uh two founders in their 20ies.

51:18 But if you go watch one of their offices, you see the experienced people as well. So it's ideal combination here. >> Yeah. >> Yeah. Yeah. It doesn't necessarily have to be the founder with experience, but that founder better be able to tap into that experience with it. >> Yeah. Yeah. Well, and then be able to work with them and and and they have to be good and and all these kinds of things.

51:40 It's complicated. Speaking of experience, this is a a big new fund we're we're launching and there's no new GPS. Um yeah, we're sort of collecting. It's because you guys have a lot of experience and the rest of the group, you know, in in this field that it's been kind of latent and uh dormant. Yeah. Well, it's kind of funny. I think we almost had to be warned against it almost just because like our backgrounds are from is hard.

52:02 And I think the reason that we needed a reminder is because all of us have spent so much in our careers existence and hardware. We're kind of drawn to that. And so listen, we've been clearly invested um in harbor over other years, right? We're in SpaceX, we're in Andrew and all. These are very early checks. We're in astronomics, we're in Whimo, you know, so we even even early on we did a number of those investments.

52:22 Um but like you know this is because it's so much in our DNA and so I don't think this is necessarily need to increase the team competencies just bonus. >> If this fund does what we think it will do, >> how do we see the world changing or looking like in 5 to 10 years? Well, you know, hopefully uh America wins in the infrastructure game. Um and we have lots of like super eco-friendly efficient data centers out there and lots and lots an abundance of chips and abundance of memory and abundance of power.

52:53 Uh and you know that would be awesome. Uh, and I think we look we we, you know, it goes back to like we really think uh America is a special place and um we're important not only to everybody here but anybody in the world who wants to kind of make a contribution and do something bigger than themselves that it's kind of the best place to come with nothing and do something profound.

53:22 So we'd like to keep that going and I I think that doesn't continue to go if we lose our lead in technology. Think I think we'll be in another era and it'll be another country and maybe they have a different set of values around that. >> Thanks. That's a wrap. >> Great for seeing me fun. Martine Beni. Thank you. Thank you. >> Thank you. Thanks. Yeah.