Tokens Got Cheaper, Your AI Bill Didn’t: AI Cost Economics Transcript, AI Summary & Key Points

00:03 [music] So, I'm going to talk a little bit about uh the bill. Uh for those who've been enjoying the talks today, yesterday, there was a lot of conversation about all the great things that we're building. I think we really have gotten to a place where we're building an enormous amount of great things. Um but the bill comes due at some point. And so I wanted to share some of the things that I've been seeing uh my company's been seeing and uh and uh make some wild ass guesses about where I think it's all going and you all

00:33 can hold us hold me accountable to that. Um but I can tell you quite frankly that it's a mess. Um so who am I? I'm Eric Peterson. I'm the CTO and founder of a company Cloud Zero. started the company here in Boston in 2016 and uh coming up in August almost on 10 years and it's been a wild and crazy ride over the last 10 years. Thank you. Uh crazy. Nobody could have predicted what's going on right now or maybe somebody did.

01:02 I'm sure somebody did. Um Cloud is the financial control plane for cloud and AI. And uh I'll I'll share some of the data. But, you know, uh, you know, 6 months ago, we kind of noticed something interesting happening in the bill. Um, and we are hiring. I'd be remiss to say, uh, if I didn't point people to our careers page, please, if you want to join us right here, downtown Boston, great place to work.

01:27 Okay, we are here. I mean, really, we we are here. This is what is happening, right? Um, it is an amazing time that we can live in that uh if I want a money flaming meteor and pounding it into the earth, I can just have whatever I want to create in front of me. How much did that cost to generate? Going to say this a few times today. I used to say every engineering decision is a buying decision.

02:01 We are now in a world where every decision is a buying decision. I need to do some research. I'm going to go to perplexity. I want to write some code. I'm going to go to claude. I want to build some images and maybe I'll go to chatbt. Before I send that email, before I finish that thought, I am putting that information probably through some LM somewhere.

02:25 And that costs money. Somehow we've gotten ourselves in a situation where every thought we have is now monetized. It sounds a little dystopian, but maybe it's the evolution of the human race. I certainly hope so. But we are here and it costs money and the bill comes due and the bill is a little crazy. We have amazing statements like this being made.

02:52 How many people saw this? Think about that. take whatever your salary is and if you don't spend half of that salary by the end of the year, Jensen doesn't think you're a great engineer. He's going to lose his mind. I think this may have kicked off or maybe it was right in the the the heart of the token maxing universe. Well, token maxing, what does that rot upon the world?

03:23 Well, this is single-handedly probably the most expensive vanity metric in the history of it. And it's really hard to exactly estimate exactly what that is, but it's somewhere on the order of 5 to$15 billion. How many people had a a boss or manager or a CEO who was looking at the AI leaderboard? They're calling it a leaderboard, by the way. To figure out Yeah.

03:53 few of you, a number of you to figure out who in their organization was doing what they hoped they were doing with AI. Did they know? No. But they wanted, they knew for a fact that if you weren't spending money, something must have been wrong. This is the craziest thing ever. And here we are. Um, this term I think was invented back in March. Um, it was declared over in May.

04:19 I don't know. I think it's still going. I think we're still living this. Um, but what has happened over the past couple months is is pretty pretty indicative of what's what's to come. Oh, and by the way, I think AI is coming for your jobs, too. If that meteor wasn't scary enough last year, I I I would say this definitively is not true. I was wrong. Last year, I would have said, uh, you know, you're going to you're not going to lose your job to AI.

04:47 You're going to lose your job to somebody using AI. Um, but I actually think what's happening is we are getting a promotion, but it's a force promotion, and we're going to have to accept it and change. And um nothing drives this point home for me more than actually a story uh from my past, one of my first jobs in IT where I was working at a computer support job and I had a couple of nice ladies walk into my office and they said, "Oh, we used to work in this office and then you people with your computers showed up."

05:27 We were six of us on typewriters. Typewriters had a real feel to them. computer soulless. You guys took our jobs. Typewriters shown up in the AI era. We've got to go figure out what promotion we're going to get. And I think next year we're going to be really wondering like what the hell is our job now. It's definitely not going to be about doing the work.

05:51 It's going to be probably about managing the work. And our taste might be the only thing that matters. So, this is going to color the whole conversation we're going to have today. This is the promise versus the reality. You know, AI, don't worry. AI is not coming for your job. I think it's coming for your job. Don't worry, took cheaper. Actually, tokens have gotten cheaper.

06:15 Um, but we're spending a little bit more. In fact, they've gotten a lot cheaper. AI makes us all more productive. How many people here feel more productive? I feel pretty productive if I look at the number of lines of code, the number of PRs, the number of commits, the number of it all is up and to the right. But some days I wonder my churning. We're still bad at measuring developer productivity.

06:43 We haven't really quite figured this out and I think it's going to only get worse. And don't worry, the cost curve only goes down. Actually, our spend is pretty much tripled and only going up. So here's the story line for today. The confession, the eclipse, the inversion, the aftermath, and then hopefully some things that we can take with us after this that will help us.

07:10 So I got a confession to make. I uh built a company that built the category, and I can't actually fully trust our bill either. I spent all day long looking at the data that comes from the cloud providers, the AI providers, random people on the streets sending cost data into our system. It is a mess. And anybody who tells you that this stuff is clean and easy and it's just data is forgetting something.

07:37 We all know that just data is there be dragons there, right? And when we look deeper into the AI bill actually like what's behind that? Is it just inference? Is it just tokens in and tokens out? Well, there's a lot going on inside of there. And a lot of it is buried and hidden and not even ever revealed to us. It's just charged to us. At least we have the benefit when we look at the Amazon or the Google bill to see that I spent this much on EC2 and this much on S3 and this much on who knows what.

08:10 But at least I have an itemized list. We have somehow reverted to the world where right now I get a bill that is the equivalent of what I've been getting from my power company for the last 100 years. You spent $500 on 10,000 kilowatt hours doing what? Getting what? Providing what? Did I leave the lights on? Was the oven on? AC. I don't know. Somehow we're back to where we are 100 years ago.

08:39 That hasn't changed, right? 10,000 kilowatts, a million tokens. And there's a lot to try to unpack behind invisible. And so much so actually that um folks at uh um University of Maryland had to go come up with an entirely new approach to get at it. They call it co-en, the cost of the invisible. And they found that there's just a ridiculous amount of invisible spend happening behind the scenes.

09:10 What you get shown on the screen isn't necessarily what you're paying for. Sometimes it's built around changes in the model, changes in the weather, changes in things that aren't revealed to us. It's 10,000 kilowatt hours. Is it because a tree branch hit the power line and suddenly the power got a little weirder? We don't know. It's actually gotten a lot worse.

09:34 Nobody agrees on the unit. If we try to look at the data that's actually coming in, nobody can agree on like exactly what it is or what it means. We got different formulas for code, for images, for audio. Some of this stuff charges us for silence. Some of this stuff charges us differently. It's all a mess. And they use different terms, different formats, different schemas, different everything.

09:59 And it causes huge problems when you try to bring it all together. They're all random vendor implementation details. We've rolled back 100 years. The prices themselves also don't make any sense. Um, I really want to know why 01 still exists. It's 7.5 times more expensive. There's zero reason for you to use it. There's vastly superior models for vastly less cost, but it's still there.

10:34 Somebody, some enterprise somewhere has an agent running in a basement that is consuming tokens that's probably keeping open AAI aloat. They thank you for your service. Other providers, I think Google turns off models after a year. Maybe they're doing everyone a favor, right? But OpenAI is leaving stuff around for a really long time. Or maybe that was Microsoft.

11:00 Um, pick a region. You'd think, oh, I just keep it in the US. That'll be cheaper, right? Instead of like multi-reion. Multi-reion sounds expensive. No, that's that. Pick a region is more expensive. Don't pick a region. You're going to spend more. That's a feature. They change how they tokenize things. Token costs go up 35%. All this stuff changes and none of it is really under your control except maybe the model you pick.

11:30 The APIs themselves, the ones where we get this data, they're broken. We don't need to tell a story about uptime and availability these days, which is fascinating how unimportant that number seems to have become if you've been a GitHub customer for a while. Um, but it's happening everywhere. It's like the old days. Ah, AWS is down. Let's all go out for a a bite to eat.

11:55 Claude's down today. I'm not going to get any work done. When we're trying to get the cost data, daily 500s. When we're trying to get um information from OpenAI, daily 500s. Cursor, one bad record in there, the whole thing just blows up. It is a mess. And this is just what happened in the last 90 days. If you think that this is happening over some, you know, long period of evolution, no, it's not.

12:20 It's happening in real time. This presentation actually was the hardest presentation I think I've ever had to create because I'd write something down and then I'd look at look at the information the next day and I realize, uh, new stuff out. It's wrong already. Probably by the time I get off the stage, there'll be something wrong in this deck as well.

12:41 March, we didn't even have a proper API to pull anthropic enterprise billing data down until March. By the way, if you on claude.com, you could get your billing data. But if you're on claw.ai, two vastly different platforms, you could not except screenshot it and download a spreadsheet from their website. All of this churn just in the last 90 days. So what [sighs] is happening?

13:08 Well, it's pretty simple. AI is going to eclipse the cloud bill. Cloud and all its growth, all its craziness, all the billions trending towards trillions. If we look at just what's happened in the last couple months, that was the warm-up match to what's coming. Anthropics like up and to the right is truly insane. Nothing like this has ever occurred before.

13:38 This is the warm-up match. And I believe all cloud spend is going to be absorbed by AI. There's only going to be one thing. It's going to be AI. Just a few days ago, just days ago, all these crazy things happened. We're talking billions, if not trillions of dollars here, exchanging hands, flowing, who knows where it's going. People making huge claims about how compute is going to be spent.

14:02 It is an entirely different ballgame. We thought cloud was expensive. No, that is a rounding error to what is getting spent right now. And I think that is going to lead to a disruption timeline that is moving faster than any of us here are prepared for. I thought I was crazy. Actually, a lot of people thought I was crazy. I started committing myself to only using AI to create code in January of 2025.

14:34 And by the middle of the year, I was convinced by the early beginning of the mid I was convinced that all code is going to be written by AI. If you talk to anybody in California, they were convinced, too. Everybody outside was like, I don't know. I don't know how many people think that all code in the future is not going to be written by AI. We'll we'll come talk about that in a year.

14:59 It's all going to get written by AI. And I think it's going to happen faster than anyone thinks. I think that what we'll end up with is something I call the fully autonomous cloud, which is AI writing the code, AI deploying the code, AI managing the code. It all is running in an autonomous fashion. The cloud as we know it is going away. It is going to disappear.

15:19 And my bet is that it's all done and figured out by 2028. We'll see. you can take the safe bet, but I don't think I've been aggressive enough. And we'll be the financing mechanism. Interesting kind of point of fact is that like every AI startup combined is still eight times short of like the total AI capex spend of all the cloud providers. There's still this huge gap.

15:45 So where's that gap getting made up? I mean 80 billion here, but that just seems like 80 billion is such a small number. What the heck is going on? And they're both burning cash if we look at both of the, you know, the giants out there, but they think about their numbers drastically different. Anthropic says that they're profitable 5% return based on how they built their numbers and built their books.

16:11 OpenAI is is got a negative 122%. Crazy, but they're both looking at about 80% margins on their API calls. They are making money. This AI cost crisis kind of feeds the narrative as well. This is probably the most profitable supply chain in in tech ever. These are real numbers. What you get charged and what we're paying. And we're so so driven to grab this right now that I don't think we we stop paying attention at some point.

16:56 And we've seen the circle chart a few times. The circle is real. And every time I turn around, there's a new box to add to it that's feeding the circle. I think as long as it just keeps flowing and spinning, we're going to be okay. [laughter] Just keep spinning. Just keep spinning. We'll be okay. I hope. Otherwise, it's sticks and rocks and stones. So, the inversion is this magical thing that some CFOs have started to notice, which is the cost to build is now the most expensive thing, not the cost to serve.

17:36 Cost to serve was what I was spending to deliver cloud computing and deliver services to my customers. The cost to build is what we're spending to build this thing, to maintain this thing, to deploy this thing. Somehow the cost to build became more expensive. Now, I know a lot of you have probably heard of Jevon's paradox, but I don't think a lot of people actually stop and like talk about just how freaking old this thing is.

18:05 1865. I love it. We're talking about a power bill that looks like what Westinghouse would have given me 200 years ago. And we're talking about Stanley Jevon's like, you know, worrying about coal. Just really internalize this because Jevon's paradox is at the crux of it all. The more efficient we make a thing, the more we consume of that thing. And we're making it very efficient, or at least attempting to.

18:34 This is what it looks like in reality. Cost per tokens are down by a,000% but our enterprise spend is up by 222%. That's the inversion. This is the actual numbers that we're seeing. This is the impact, the reaction, the the result of this. It's happening. It's happening right now. What Cloud Zero is seeing is pure evidence that it is very real. 14 times growth in Gai spend in the last 5 months.

19:12 80% of that seems to be overhead based on all of the crazy requests, [snorts] systems, R&D, humans, you know, cloud code spend. You mix it all in there. A new model comes out. We see customers go bananas. Z one customer 0 to two million in a month. Just a new model appears. Nothing else really changed. Just that. And then who's using the cheaper models?

19:42 About less than 2% of you. We all rush out to grab the new thing. Haiku still does some good stuff. Actually, Haiku is the only one that can answer the question, how many How many days in the week have the letter D in them? Go ask Opus 48 that question yesterday. It did not give a good answer. And I preface it with yesterday because somebody's probably hard coding it as we speak.

20:12 The CFOs are doing the math. This is what we have to think about. Tokens or humans. I have caught a couple of CFOs with spreadsheets that literally compare your paycheck to the cost of tokens. Don't believe me? This is why we have to think differently about our jobs. Nvidia is talking about it. CFOs are talking about it. We're seeing it happen. I want us all to enjoy this future.

20:42 Should be a very happy future where we all get promoted. But we got to start thinking differently now. So what's the aftermath of all this? Well, we've heard about Uber. Everybody's talking about Uber spent so much money, but they're not alone. JP Morgan did it, too. Microsoft just literally days ago, I think it was this was I had to update this in my slides yesterday said stop using Claude.

21:09 Of course, they got plenty of reasons why they want people to stop using Claude, but It's real. People are starting to freak out. Although, interesting enough, Uber is still spending because maybe they're getting stuff done. I had one one story that came to me. Somebody said, "Oh, we're going to spend about 4 million this year. We just in this quarter spent about 40."

21:39 Who does that and survives? My reaction was, "Wow, who got fired?" said, 'N no, I think some people are getting promoted. What the heck is going on? Uber CEOs though stepped in and said, you know, if you're not actually Wait a second. What's the connection between all this money we're spending and the outcome? This is the important question to be asked.

22:03 This is the question I want us all to think about. How do we connect the money we're spending with the business outcome? Love vibe coding. I was out at Sasser. I heard Jason Lempin actually say this words up on stage. He was talking about how he had rebuilt his whole organization. He had a staff of like 20 or 30 people and now it was just two people and a dog and they used replet for everything.

22:26 And he was like, "Whoops, dollar a minute." And then people got up to ask questions and said, "Ah, this vibe coding slop is just driving me absolutely insane." They said, "Yeah, but we're all here, aren't we?" In fact, this is the most successful conference I've ever run. Two people. It's scary. We also are packing more and more, at least trying to, into the context window.

22:53 Oo, I can get a million. I have a million context window in cloud now. That's fantastic. Let me pack more into there. Let me link up 30 different MCP servers into that into that system. Let me link everything and let me just dump it in there. Wait, why am I spending all day long fixing the bugs that I just created? Right. MCP tax is real. But there was just I don't know days ago again another thing on what Netflix was working on trying to compress the token consumption on the in on the input.

23:27 Try to break this down. If you're not spending time in front of whatever system you're using and regularly sharpening the saw, so to speak, you are absolutely burning through tokens. Not everyone's token spend is equal. It's fascinating. I've been watching every single one of our engineers use these tools and looking at how much they spend and then looking at what they produce.

23:47 Wildly different spend, similar outcomes. Some people are doing things differently than others. A lot of it is in how you pack that context window and how you think about planning starting tasks or just vibe code the oneshot wonder wagon to value train whatever you want to call it. Just made that up. It's not just the coding that we're doing. It's also the systems that we're building.

24:20 I've seen so many loops, things that break, that just keep going for days and days and days spending money. This is an example taken out of um one customer where I saw this. I gave a talk at CubeCon last year about million-dollar lines of code. I think I'm getting closer to billion dollar lines of code in some of these things. Security is not to be forgotten either.

24:45 The first UCON talk I ever gave, I think in 2018, I I coined this term called denial a wallet. It is alive and well. I think the next attacker is going to come to us through our own systems, get access to our AWS environment, go find bedrock, and then use our AI tokens to then absolutely own us from the inside. And when we walk away, we'll have no data and even less money after the fact.

25:13 that attack is coming. The attacker won't even have to pay for it. They'll bill us. Every company that we've looked at, they had plenty of dashboards. Check out this amazing dashboard I built in Graphana or Data Dog or wherever or even Cloud Zero. They had budget alerts, but nobody was paying attention to them. In fact, a lot of people when they get a budget alert go, I don't know.

25:38 Can I turn this thing off? Will I get in trouble? The CEO said token maxing is a thing. I've talked to friends who have literally watched hundreds of thousands of dollars go out the door over a weekend and they felt like if they said anything they were going to get fired. That's terrifying. We've got to have the conversation about what is too much. There's no per call, per user, per anything limit out there.

26:06 We used to be afraid of this in the cloud, but ah how much damage can be done? Multiply the ability for every engineer in your organization to spend $5,000 in a 12-h hour period and then start to think about how important a limit actually is. It's very easy for any company of any size to spend the total of its capital in a couple of weeks if they let this thing go.

26:37 And there are no limits. Everyone here is absolutely convinced they're secure as well, right? Denial wallet's coming. There's going to be some companies are going to be put out of business by that. So, we have to think differently about this. Every decision is now a buying decision. Everything. When was the last time anybody googled something? No disrespect to Google, but seriously, Googling was free.

27:09 When I planned my date night on the weekend, I went to Perplexity. When I went to Claude, I didn't go to Google. When I needed to figure out the directions, I didn't go pop it into a search engine. I asked an LLM. I paid money for that. We're all paying money for that. Stop for a moment, everyone here, and think about how much time, questions, things you've done that you would have done for free that nobody would have bat batted an eye before searching the web, but you've popped into some cloud or chat GBT or

27:49 perplexity window. And then ask yourself, was that a company thing that I did or was that a personal thing that I did? Right? There's this report that comes out every year that talks about cloud waste. It says cloud waste and it's been stuck there for like the last decade and a half, which is it says about 30% of all cloud is waste. Well, if we just all stopped collectively doing free Google searches and just started pumping our every question that we have into the AI thing that we have now at work.

28:21 Um, what does that waste number look like and is it waste? And when the companies figure out how much money is being spent on that, you want to really wonder what big brother looks like in a corporate environment. Gez, how many people are ready to have their chat history shared with the world? Even if you try to use anonymous, maybe you're using a proxy.

28:49 Every decision is now a buying decision. So, we have to think differently about this. So, let me walk you through a couple of things. So, we have to stop asking what we spent. Actually think this obsession with cost is costing us. Um, when we think about just how much we spent, we kind of forget about the thing that was truly most important, which is was it worth it?

29:14 What am I trying to do? What am I trying to build? Why am I trying to build it? Who am I trying to serve? What is the purpose of this? And I think that's also rooted in when we think about what our jobs will evolve to be. we have an opportunity to get way more focused on the outcome, way more obsessed with the outcome. Now, if you are obsessed being a crafts person building some of the finest code that has ever been written, I have nothing but sympathy for you.

29:52 Honestly, I do. I I used to dream of writing some of the finest code ever written. I have given up on that dream. It's not much different if we think about it though in terms of like you know I'm I'm not really after the compiler is done going and looking at the machine code to make sure that it interpreted it properly. No, we're just going to speak whatever we language we speak.

30:16 We're going to let it get translated into code and we're gonna stop looking at the code. I got a confession to make. I don't look at the code anymore. That's the easiest way to deal with it. I care about the outcome. I care about the tests and the validation and the proof that the stuff that came out of it met the demand or the the use. But I don't care so much about the code anymore.

30:44 Somebody's already wants to chime in. I don't know. He cares about the code. will eventually all code that is produced will eventually be whatever the LMS can produce in the most efficient manner possible that may or may not be assembly not a lot of LMS really are good at assembly they're good at JavaScript and Python right now apparently today I think over time the code produced will be some language that is super efficient for for the LMS to write that is inevitable I Um, so we have to stop thinking about what we

31:28 spent just like we have to stop thinking about what was created because we need to think about the outcome. What did we spend? No. What was was it worth it? And then us as builders, creators, we have to think, well, is the reasoning, the model, the the the stuff that I'm building, is it actually necessary? Am I getting to the right outcome? Was it did I really need to like rush out and switch everything over to Opus 38?

31:54 Can I mix some of these things together? What are these models individually like actually really good for? It's not like, you know, a 286 and now I'm upgraded to a Pentium. Like no, they're they are different beasts for different tasks. We're thinking about it wrong. What's our rollback plan if we just rushed out and actually burst all these things up?

32:17 Did we go back and just look at the outcomes? Um, I didn't talk enough about it, but I uh I probably could have given a whole talk on just evals. People have been spending more money on evals than sometimes their production systems, which is a funny observation, particularly for people using LLM as a judge stuff. How much money are we spending on on that?

32:42 There are different ways to achieve quality. What happens when these things loop and run off and explore the universe without us? Are we aware of that? Do we get an alert? What business outcome is this delivering? And we should rejoice in that. It is something is a chance for us to get closer to the customer, closer to the value that we're here to produce, closer to the the businesses that we hope to see successful, hopefully, or maybe even better individually, we can all start businesses that are much closer to our

33:19 customers and be much more in charge of that outcome than we've ever seen possible. I'd like to see more companies started, frankly. Uh, why is that slide in here? Oh, [laughter] >> I actually had a better slide than this, but I'll talk to it. Talking is better. Funny thing about building this presentation. I wanted it to just start with text and I did.

33:46 Um, there's just too many AI uh image presentations out there. So when we think about how we are going to change our jobs, I want us all to think instead of dealing with being the perfect crafts person to to think about how we are going to drive that herd of agents drive the AI employees that are going to be working for you or I. Um there's a funny joke about this.

34:18 This is where this came from. This is a farmer explaining to a horse, "Hey, don't worry. You're not going to be replaced by the tractor, but you might be replaced by a horse driving a tractor. [laughter] So, you better go learn how to drive tractors." We all need to learn how to drive the tractor. We got to also measure the right things. If we measure cost per token, your team is going to optimize for cost per token.

34:50 This is good arts law like uh in full force on display. Amazon had a leaderboard. They took that down recently. Good for them. I don't know how we explain this to our bosses that that AI leaderboard is a bad thing because I definitely still hear the questions. How much are people spending? Are they what? Wait, why is this person only spending $2? What are they doing?

35:19 Are they do they even work here anymore? They only spent $2. That is not healthy. Hopefully we get better at thinking about the outcomes through through all of this. Um good friend of mine, very senior engineer, um worked with him for years, you know, dealing with this same thing. And I asked him, I said, "Well, what's the problem?" He's like, "I got into this business because I love writing code.

35:44 I don't want AI to write the code. I want to write the code. It's fun. I enjoy it. Like, I got nothing but sympathy for you, man. I don't know what to say. You're going to be just like those friendly ladies who came into my office 20 years ago, 30 years ago. God, I don't know how old am I? And uh and said, "Ah, there used to be six of us in here typing on typewriters."

36:15 And then new computer people came along. Change is inevitable. And the outcome driven metrics honestly are better. They drive better outcomes. These are four that I think you should think about that you should internalize. Um I am a true believer in unit economics. What does it cost to do the thing that it is that you do? If you don't know this, you should know this.

36:43 It's probably one of the most important performance metrics for any on demand cloud-built system in the universe. And it is the secret to understanding the 1 million token bill that you got that doesn't tell you anything about what you actually did. It is the most uh obnoxious anonymized uh source of information that tells you nothing about what you achieved.

37:07 And you need to figure out how to break that apart. So, what feature did this drive? What functionality does this drive? What outcome to this drive? What's the cost for the autonomy in our environment? Engineer plus cloud code. Are they producing 10x for 5x less? Great. You got to think about the economics. your CFO, I guarantee you is. And I've seen a lot of the conversations lately move out of the engineering org.

37:44 You probably heard of the term PHOps. PHOps is in a lot of times inside the engineering or they care a lot about the cloud bill. I care a lot about the cloud bill, too. Still, it's big. But the AI conversation has shifted to the CFO's office because it's so big. It's not a budging issue. It's a P&L issue. Right? And what that means is the person who also has free access to everybody's salary is also looking at all the other things side by side.

38:18 Those spreadsheets exist. So get ahead of it. What's the efficiency for the model that we're using to achieve the outcome? That way maybe we can actually understand when the next model comes out if it's even worth upgrading in the first place. What's going to be the margin yield or margin improvement for all this autonomy? And don't think about it like personally.

38:47 It's not personal. It's just business, right? Um, I think we actually need to start thinking about that unit cost or that margin data flowing into the systems themselves as a signal for how the systems operate. Is this a top tier customer? Maybe they shouldn't be using the most expensive models. Maybe they shouldn't be using exactly the same system.

39:14 What's the lifetime value of this customer? Is this a free tier? Do we have the highest margin you know tier on some the same model as our lowest tier route according to margin or your profit objective? Right? It's just unit economics. It's not optimization really. This is just business. It's good business. So here's what I want you to stop doing on Monday.

39:46 If you take all this together, stop thinking a dashboard is going to save you. In fact, in the future, nobody's going to be looking at dashboards. If you're still looking at dashboards, I don't even know why you're looking at dashboards. I don't look at dashboards. The number one user of all these systems is going to be AI anyways, and they'd rather look at JSON.

40:05 They don't want to look at a dashboard. Yes, you can paste a screenshot into them, but it's way more efficient if you do it a different way. Dashboards are just postmortems of like a horrible event that occurred because this stuff is moving way too fast. A lot of the CFOs are freaking out right now and actually starting to talk about something they never used to talk about, which is real time data, which is crazy.

40:34 Um, because they think in terms of months and quarters. Get out of this cost per token as your primary KPI. Don't waste time. Um, don't forget to don't forget to think about the unit economics. What does it cost to do the thing that it is that you do? All these things right now are just decisions that have been made for you. Nobody's thinking consciously about these things.

41:01 You need to think consciously about these things or they will cost you or they actually are in fact costing you right now. Oh, it showed up. Why is it here? I don't know. That was the joke. I think I told it better. So, I'll leave you with this. By 2027, I'm convinced that this is going to be true. I'm probably going to happen faster. The primary buyer of things, technology, cloud resources, stuff in your environment is not going to be humans.

41:38 It's going to be an autonomous AI system. Remember that Jevans chart? Cost of tokens going down, total spend going up. Right now, we're seeing that spend driven by the cost to build more so than the cost to serve. At that point when this happens, the cost to build and the cost to serve will start tracking together again and we'll be looking at an even steeper curve because we're spending tokens to write the code.

42:19 We're spending tokens to deploy the code and the code itself will be picking and choosing what it runs on and how it runs and when it runs. We need to be prepared for a world where we can govern this reality. So think about that. Those are the systems or thoughts or culture or ideas that we have to embed into our engineering practices right now into the software development life cycle right now.

42:48 The thoughts that will dictate whether or not I believe your company is profitable and survives or doesn't. That's what's at stake. And the folks who master this, can ride hard over this, um, are going to be the ones getting a promotion. And I want that for all of you. So, thank you. >> I was checking if you have an MBA, by the way. U, sorry. I was checking if you have an MBA because a lot of this is economics, etc.

43:24 So um to what extent do you think the workforce needs to be educated in like these basic concepts of unit costs etc etc >> education gez that's a whole another conversation the cost of knowledge is is rapidly approaching zero but the cost of wisdom is going to go near infinite but [laughter] okay um I mean yeah yay I my daughter's going into college in the fall I don't know what that means for her right I don't know what that means for for any of us like it's a it's a kind of an odd place.

43:58 Um if I could go back in time I would have studied economics and not computer science. Um that's for sure. Um I think economics is at the center of everything. But um um but I think we have to elevate ourselves to think about I think the the the path to applying our very technical mind to to this problem is by understanding the the the cost to do the thing that we do being focused on the outcomes like that that path will set you on the right path ultimately.

44:31 Um and you should think about those things. If you're on that path, you will get better and get that promotion. Um, here's a funny thing. CFOs, it's it's it's it's kind of it's kind of an odd observation, but it's true. Finance people and technical people often seem like oil and water. They're actually just two different alien species that speak a completely different language, but they're both actually highly technical, highly disciplined, highly like geeky, like in the weeds detailed kind of like cultures and they

45:09 have way more in common than you would you would think. Like just ask a CFO to open up Excel and show them the cool thing that they've like done and you'll be like, "Wow, this person knows their stuff in front of a system." Right? It's the same way you would feel if somebody said open up your ID and like bang out a few lines of code, right? So, you have a lot more in common than than you do than you know.

45:30 And um I think if you find that you know you find yourself having that conversation, there's a bridge that you can establish there probably more so for the engineering side of the house with finance than any other part of the business. Honestly, >> my question is um AI seems to be one of the first technologies we see in in tech that is that adoption is being driven partially top down in a lot of ways right it's not engineering saying hey there is this cool technology let us all move to that right it's uh various types

46:05 of sea level people saying hey and right now most of the companies are at the stage of >> hey you everyone engineering must become AI native right the second the second step of how much this cost and what's and and and why and prove that it's um that needs to be coming that that's probably coming for most of them uh but hasn't come so my question is uh from your perspective what it is you you are seeing what are the trends you are seeing in sea level uh decisions and how how people look at that >> yeah um so AI is is

46:41 definitely felt very tops down it's also felt grass grass roots at the same time. Um what um what I'll say is you first we kind of did this to ourselves. A lot of technologists myself included like have had that moment where we walk into the to the room with the business conversation going on and we say Kubernetes will solve our problem. Oh crap. Nobody knows what Kubernetes is in this room.

47:09 Ah right. and they go, "Nah, just keep running it on that thing that's in the closet or whatever, like why are you talking about this?" And um and I've worked with people where I literally took the whole like proposal that they've had on the table like I can't why can't we save some money? Why can't we do this the right way? Why I go into the room I say if we build this Kubernetes pipeline data thing super containerized yada yada yada it's going to it's going to do wonders for our efficiency.

47:35 And I'm like okay so what let's dig into that wonders for our efficiency part. What does that mean? So, well, I don't actually know how to articulate. All right, let's go figure that out. Well, what is it going to do? Well, okay, it cost us 25 cents per transaction. It's going to go to 10 cents per transaction. Okay, just walk in and say that there is not a single finance business person in the world.

47:55 If you go in and say the thing that we do cost 25% 25 cents per transaction, if we make this change that I'm not going to explain to you because you wouldn't get it anyways, it'll go to 10 cents per transaction. They go, why are you still standing in front of me? get out and build it right now. And that's the opportunity. Get out and build it right now.

48:15 If you can talk in terms that the business understands, they really just don't care about Sorry to keep picking on Kubernetes, but you know, it's an easy target. Um so you know from a trends perspective um the CEOs are still out there saying things that are wildly disconnected from you know the other parts of the CIO and the CFO seem to be having a moment.

48:42 CTO's are like oh shoot am I going to get fired? Did I do the right thing? the CIOS who are like wait a second everybody's building stuff internally again like we rolled out just clouds as a small company we're about 180 people but we like rolled out quad code and co-work to every single person in the business and a couple weeks later my head of HR asked me for a GitHub account what she's like I don't know Claude told me I needed a GitHub account in order to do this thing and I was like do you know what GitHub is?

49:18 She's like I don't know what GitHub is but but it said that I needed it and it said that I should ask my IT person and you're the technology. So like every and that was my observation was like everybody in the company became a developer very junior developer in that moment. The CEO was like this is awesome everybody's doing stuff. The CTO brain was like okay I think we're getting better.

49:43 Oh I'm getting the snob. and everything blew up. So, anyways, I could keep talking about that. I'm going to respect the the the clock, but thank you. You've been a wonderful audience. I really appreciate it. [applause] [music]