How Jev Turns AI Into Software That Gets Things Done Transcript, AI Summary & Key Points

00:00 Where the [ __ ] is all the automation? AI is so unbelievably smart and yet it's so useless at all other stuff. >> It doesn't matter how much AI coding agents you use, the software actually isn't getting better. Maybe you're writing it faster. It's like arguably getting worse. >> OpenAI has been trying to automate customer service since 2020. What I want instead is smart software.

00:17 I want to expand what software itself can do such that things that should be automatable can then be automatable. My favorite thing that you guys say is we build prod not god. So good >> because if we had any other kind of like big lab leader even if they had joy they would cover everything >> and then your view is so different. You're like no we're going to create a way better world >> for nuanced reasons.

00:43 I don't think we are on the path of RSI in the SAS apocalypse story. Um today we have the founder and leader of Typesafe, Diego with us. Um who uh is a bit of a hero to both Martine and me. Yes. >> Um he is not only building like a really interesting product uh but creating what we think is a very important movement. So we're super excited about today.

01:15 >> Welcome. Thank you for coming. Yeah. >> Thank you. >> Yeah. And uh maybe you can give us kind of a a brief on um just you know what is Jev? What is type safe? Why is it important? >> Is this a curse friendly or no? >> Okay. [ __ ] are you talking about? >> Okay. Okay. Okay. Cool. So I was actually asked for like a elevator pitch which I tend to ramble on and I don't do well but like I realized my favorite elevator pitch for Jev is where the [ __ ] is all the automation?

01:45 Like this is like so unbelievably tragic, you know, so much intelligence. AI is so unbelievably smart and yet so not that I hate on chat bots or coding agents. I love them myself, but it's like it's so useless at all other stuff and it's it's tragic. It's tragic that, you know, we have so much like diamond in the rough but not polished for work. That's that's a but type safe is making AI for software.

02:12 You know, we want to make AI powerful not just for humans in the loop, but to actually build real software and JIE to us is our first model in this whole space to make it way way better to like make automation. >> Yeah. And so it's been interesting because it's kind of caught fire in software world. So, you know, one of the things that made us go, "What the hell's going on here?"

02:36 is like every developer we know is calling us and going, "Oh, this is freaking awesome. It's great. It's fast. It's it's great. Everything's better. Um, and h then how does that uh cuz because everybody thinks of well we've got cloud code you know we've got codecs don't we already have that? Like what's the difference? And then how does that lead to real automation?

03:00 >> Oo I wish I had like a some slopp visuals because I have like a favorite sloped visual for this. So I like cloud code and codeex. Um I love the description from Gary Tan on them. It's just in time software, you know, incredible way to describe what they're doing. It like it makes software on the fly and you can like program software in natural language, but it has the same expressive power as software.

03:24 >> Um, what I want instead is smart software. Like instead of like automating software engineering, I want to expand what software itself can do such that things that could should be automatable can then be automatable. and like in a more um flowery language like I want to express things like intent. I want to like expand the vocabulary of what we can do and I can talk about like all sorts of like weird sci-fi things I want but like programming is like hyper specifying like valuable things and then infinitely

03:54 replicating them. It's so freaking cool and I want to just make that more, you know? >> Oh, interesting. So, so one way to think about it is instead of kind of a tool that uh somewhat replaces a software engineer with a faster, maybe not even as good software engineer. What you're saying is no, no, no. We're going to supermpower the software engineers we have to write way, way better, more interesting things.

04:25 >> Yeah. I I actually I mean Yeah. Yeah. So this is by the way I just think so many people miss this point and it's such a subtle point and it's so important to actually tease it out which is um if you use something like cloud code or codex which is great or cursor which is great >> they write code but that code is the same thing a human being would have write maybe it's better maybe it's worse but it's basically still code just like code looked 10 years ago.

04:46 >> Yeah. And and the thing with Jev is whether or not you're cloud code or a human, you have this new primitive, this new thing that you stick in your code that actually expands like the power of software. So instead of like writing code, it is something that you include in your code >> which go ahead. Well, which which by the way is interesting because it's this very powerful primitive which would be great if you explain but it's also a little bit different than like you know how programmers think.

05:12 for example like it has this notion of like you know probabilities or you know and like so you know so >> so an intelligent layer inside the software. >> Yeah. Think of like like a li yeah like a library that you can like use natural language to describe what you want and you give it kind of a state machine and then it will will choose what to do with some confidence levels which we kind of haven't really had before like so ubiquitous.

05:36 So maybe >> ooh there's a lot of tricks there. I will jump into one thing first which is I love the first thing you said like >> you know in the direction of where the [ __ ] is all the all automation. I love software so much. I wish I could be writing it all day. It's >> would not recommend being a CEO to people but whatever. Um, and also like it's wild that AI is so cool and software has been unchanged in 10 years, you know, like like like that to me like no one can like square this together and the most we can do is

06:08 add like a little chatbot in the side sometimes that can take actions but not all actions cuz some of the actions are not reliable now. So I just want to give like that tiny aside. Um, >> I love the point I'm gonna jump back to the point about like this is a little bit of a different way to think about it. Um yes I think that machine native doesn't exactly match bits perfectly and like that's actually the art form that we are trying to do like I in our onboarding on day one I draw like the vin diagram of like what AI

06:37 is good at what is valuable in code we are in the middle so you know we don't output like you know extrapolated floats for example cuz like AI is just bad at that you know but things like probabilities are not exactly novel And it's similar to the is Jev just a classifier argument. >> Jev is absolutely a classifier. You know, like classifiers are sick.

07:01 Classifiers were designed to be useful. >> Yeah, they're designed to be useful. And actually, it's the same interface as like some of those ML concepts because these came from like practical people who are trying to make systems work. And what I'm seeing is happening now is that Jev actually I my guess is that Jev probably is better than having like an MLE team from 2019 making the stuff for you and you can just program it on the fly.

07:27 Who knows what could be built cuz like there were not that many good ML team MLE teams in 2019 to build like narrow things and to be able to like collect data sets and measure it and all of that. And it is just the beginning. There's I feel like there's way like like by the way to this point do you think there's a slider bar here where like on one end is like language in language out like we have today on the other end is like a like an existing imperative program and then you can kind of move between the two or do you

07:50 think like this is like the point in the design space which is language in kind of state machine out which is going to like solidify as a general purpose thing for programmers. >> Ooh that's a tricky one. So I will say the answer in my heart yeah the answer in my heart is that it is a slider. So in and actually when I design for our the properties we have I might have made mistakes due to my personal preferences but like intelligence per dollar is my northstar right now and it could be wrong just to be clear

08:18 intelligence per second might be more valuable in the short term but like even like our interface like calling the input state this is intentional like it's to say >> oh that's great I didn't catch that >> it's meant to be the insides of programs so >> so in my heart because like so we are really optimizing a lot of the work I do is for even more complicated arrangements of the internals of program state.

08:40 Can you put intelligence in there? I think this is going to be an everpresent battle to have I, you know, we're very intentional about our design and also pragmatically I think certain things happen like it's easier to make an AI at these milliseconds. So it'll be more like a database for a while than like a standard library thing. But I would love it to be a standard library thing too.

09:01 >> Can I can I just pull back like what is the alchemy that creates a do yogo? I mean like you speak when a man and a woman love each other but like listen you speak like an AI researcher you speak speak like a systems person you speak like a programmer and normally these things have been like not super overlapping and like you're you're taking you know AI which we've been pushing towards you know being a being and you're making it a programmer's tool.

09:24 So maybe a little bit about your personal journey that >> my history into AI is somewhat unorthodox. Um, I was a mathlete. Um, I was a award-winning mathlete. Um, the way I describe it is I was good enough at This is cringe. I was good enough at math to get girls. So, that's quite good. >> No, that was a thing. >> YEAH, YOU HAVE TO get you have to get quite good.

09:45 Um, >> and what kind of girls do you get when you're that good at math? That's an >> Oh, that's our audience needs to know. >> Oh, no. >> We have to inspire the youth here. >> Don't do it. Youth, don't do it. It's not worth it. Just be cool and chill and interesting and >> don't overcompensate. >> Wow, I can't believe I said that. Um, >> so I was a math athlete, >> but I actually never Oh man, this also is a little cringe.

10:12 I never really liked math. I never really tried. I was just like big fish in little pond. And to me, math was act I math was always the path I was set on, but I hated it because it was always about like winning competitions. But then computer science is actually a lot like math. It's basically like math but cool and useful and fun and interesting and I still love giving algorithms interviews.

10:35 It's the best thing for me to do. I don't know. But do I love it? Yes. And does it like allow me to like sus people out really well? Yes, it does. So I love computer science. I consider myself to be computer scientist much more before AI researcher despite my history. And um like what actually got me into it was I also won a kegle competition not from sophisticated math but from like just automating like the [ __ ] out of it.

10:58 Um >> you know like just like more nested loops more you know like I solved it like a systems problem you know. So that eventually got me um like I was forced to speak at Nurups normally an honor but I hated it cuz I just wanted to be in the mines. >> Was that from the Kaggle team? >> Yes. >> Oh wow. Yeah, actually the Kaggle host of it was Isabel Guong who was the co-inventor of the SVM.

11:23 Actually, I think the first author of SVM. I'm not 100% sure first author. Um, and she just basically saw that I was like this person who really didn't fit into the research community and then adopted me and showed me like like it got me to meet all the AI people and that you know my career was just pushed into that direction. >> And from there open AI?

11:44 No, it was like a startup with Jeremy Howard. Um, >> no kidding. Yes, I love Jeremy. Fantastic. >> Cool. >> Um, and then Google Brain for a while. >> Wow. >> And then, uh, retire for a while. >> And then eventually I was like just kind of tired of not doing anything. I was like, you know what? Actually, AI is pretty damn fun. And I joined OpenAI because of that reason.

12:12 Uh, and it worked out really well. >> Amazing. >> Really, really well. >> Yeah. Incredible. So, you you said something there that is so um unusual in today's world, which is AI is really, really fun. And then the company has such a different demeanor and view of AI than every everybody else. And my favorite thing that you guys say is we build prod not god.

12:39 So good. Because if we had any other kind of like big lab leader, they'd be like trying to even if they had joy, they would cover and then your view is so different. You're like, "No, we're going to create a way better world and it's going to be awesome and there's going to be not only are there not going to be less jobs, there'll be more jobs and there'll be way better jobs and everybody's going to have a great time and like just be being around you like you clearly believe that."

13:10 So, so tell us about that and like what this because for us, you know, type safe Jev, it's it's more than a company. It's a it's a whole movement towards a positive future that most people in the AI world kind of don't like. >> Yes. >> Or or or or they're not with it. >> I think they don't get it. Yes. You know, like the it's just a classifier complaint.

13:31 It's like an ML level concern while everyone else is having like a Jeff party >> because it's like holy [ __ ] like we can do all the things that we wanted to do and I don't I think if you don't like get developers it'll be hard to understand what's really going on. So 100% I agree with that. I do think that there's like a pretty negative world painted that I obviously disagree with.

13:54 I think it's really comes from this like um you know mono model Kool-Aid that everyone believes. I think >> right one big brain to rule them all. That's one that's one way to the It sounds much more ominous. >> That's what people hear. >> Yeah, for sure. That's what people That's what people hear. Yeah. >> But you know, like will that one is that one brain really on the path to rule us all?

14:18 Like we have not automated really basic things that I don't think we want people to be doing, you know, like it there's lots of really really basic stuff. And I think that oh man it it it pains me when the world is discordant with the reality and like part of the pain is you know on the where the [ __ ] is all the automation like how can we have AI be so freaking smart and >> like there's so much so much financial incentive to automate stuff like yeah you could make an excuse for diffusion I don't buy it at all I

14:48 shouldn't name names but like that obviously is not true part of the problem is like the discordance with the reality and the fact that AI has like so much potential is what made it really tragic for me that we had not released this. So now it's a now it's like a little bit of a party for me. But like I was afraid of >> all dev users are like >> there's the happy AI the people on Jev and then there's the morose AI the people who are not.

15:15 It's it's really uh it's quite a kind of fascinating dichotomy. It's it is really well I give you and and to your automation point. I had a funny conversation this morning with David George who runs our growth fund because we're talking about the new tools. I was like have you tried the Muse thing? He's like oh it's awesome. I was like what'd you do with it?

15:35 He said, "I finally canceled my New York Times subscription." And I was like, "That that is hard to do." >> But yeah, you know, it's a kind of a it's a very tip of the iceberg of the things that are horrible things to do that that we need to automate. I think that if we were going to be really intellectually honest and we really aiming for the north star of automation, we cannot fall into the same anti-atterns that AI has fallen into which is really um focusing on outliers and demos, right?

16:08 Like a lot of people ask me like what are your favorite use cases and I'm like I'm not sure if they work. I want them to work in the background such that like someone would trust that to run and not page them and like people can build on top of that too and like >> composable composable >> composable but like other things like safe right like it's a different type of safety where like if you want it to actually like run with resources associated with it with access to things you need guarantees for that or like at

16:34 least statistical guarantees and um >> so it doesn't go rogue breaking face that type of thing >> well I I don't think our models will be doing anytime soon unless somebody like does the software to do that which would be very cool flex very cool flex I should figure out how to give credits for that but um but not in a way that we're not responsible >> right right >> I'm just curious like how long has this intuation been percolating cuz I remember talking to you maybe in was it 2017 >> we did talk about that yeah >>

17:08 yeah and like and then like a lot of these ideas were in your you know you were talking about data being important and you're talking about like you want to focus on the task and like but like so I I just like you know was this like did you know that this was going to end up being a classifier or was this just an intuition that like there was just kind of another way to view this entire kind of AI movement >> you know so actually a fun story about that chat in the talk from 2017 I think my talk was >> actually in a

17:35 very similar theme I think it was called something like AI modular in theory and flexible in practice which is very software. So, I'm a little bit consistent in that. I >> I think that this really started right before chat GBT. >> Um like right when we released these things, I did not have intuition about this and honestly I was not even >> I was very very pleasantly surprised by the generalization capabilities of RHF.

18:01 >> When is this? >> Must be end of 2021 like a >> fourth quarter of 2021. um like we were it was really really general like if you read the paper it's unlike other papers that are like trying to prove their point. It was us actually you know scientific methodish trying to disprove like is it cheating and uh >> you know my favorite query was why is it important to eat socks before meditating?

18:26 We made sure that was not on the internet beforehand and like the models were able to like make plausible humanl looking answers for this and that to us in the team was the thing that clicked like this is not cheating which you should always be afraid of cheating in ML and >> then what really got me burnt was I we released it um you know we did a >> you know I'm obviously a big capabilities guy I did a lot to release that model I I really thought that that model had like a decent chance of being AGI and when it didn't

18:56 that was like when my whole world came crashing down and I was like why? >> Oh so you were kind of on the other train for a bit which like the crazy train or >> well no I'm just like under RL generalizes like maybe we have AGI like >> RHF generalizes pretty well. RLVR is the thing that doesn't generalize as well from what I've seen. Um and AGI in to >> well I I was just saying more I mean like you know you were beh in chat GPT you were behind like these early GPTs that was a very different goal which is like creating a

19:30 chatbot that will talk to the human being was not a programmer's tool etc. So I'm just wondering like >> oh well actually early early like 2020 um openai when we talked about AGI people used to describe it as Ilia and every if statement. Um, so it's not it's kind of like but like is that like part we were talking about open AAI culture part of it is that it's like intentionally vague so it's a wide like tent so that everyone can be inside of it but like I am not >> for nuanced reasons I don't think we are on the path

20:02 of RSI and I still don't think we're in the path of RSI and I did then I do think that what OpenAI defined as AGI is extremely doable automating most of the world's economically valuable actually sounds >> like uh >> oh man I don't like there's a there's a lot of work out there a lot of it is very wrote and simple and like by volume in order to be able to like outsource work you need like simple instructions that like basic people can do and as far as I can tell the intelligence of that has been available in the models

20:33 for like quite a while now and >> like my oh man you know chip on my shoulder is like why is this not available and and then the the and then since RHF the AI industry just like kind of bifurcated into gigantic overpromise underdeliver I think GPT3 was actually quite calibrated back in that day but because humans evaluate how good the models are it looks really good cuz they are the judge but we've been optimizing that judge instead of the automation part and that has been the missing thing so I would say it was really

21:03 really then that it like hit me you know like why is this thing not more useful >> and so you think that the measure that we should have is to what extent Can you automate actual productive tasks? Is that the >> the over when you say overpromise and underdel? That's the dimension in particular you're talking to. The ability to automate tasks. >> I like I think in my heart it's like cool sci-fi.

21:27 Um you know and I think that I think that that is the canary in the coal mine for cool sci-fi. Like, are you really telling me that we like math is solved or like even like 2 years ago GPQA that Google proof question answering is solved, but we still can't handle a drive-thru, right? Like it's it's it's it's a very hard thing to hold in your head at once and I think a lot of people don't have good answers to that.

21:52 >> Yeah. Can I can I just test one thing which which may which may not make sense but I want I mean I mean isn't there an argument though that like >> the the the real the distribution of the real world is is is different than the digital world right it's heavy tailed there's a lot of exceptions we don't have all the data and I mean it couldn't it be the case that the reason we're not doing productive stuff in the real world is just like we're not we don't have the data for that distribution we're not training on that

22:18 distribution and this is why it's just been basically relegated to like these is lower dimensional manifolds like whatever math or code or >> um I don't entirely buy the data argument in my opinion. Um I do believe that there's a a long tail for sure like that that would be kind of crazy to deny and I don't think that in my like canary in the coal mine situation we need to automate that long tail.

22:43 Like I think that I think we need to be incredibly pragmatic on everything and like building reliable software is always an investment, right? Like it like you know what were the three great virtues of a programmer? Laziness to not to do it again, hubris. And >> there was a third one. >> Yeah. Yeah. No, I remember this is from the Pearl days. Yeah. Yeah.

23:05 >> Yeah. There there was a third one. >> Very well. Yeah. I I wish I wish I could remember it, but like it's about like the laziness to like spend, you know, like 10 hours to do like the five minute task instantly and to never have to do it again. Like it only make like it should be an ROI decision for people who like automate stuff. Like I would just like it to be automatable and I think that people will just make like new kinds of work, hence the Jev in jeans, new kinds of work once that stuff is doable.

23:32 But uh like as like a a benchmark, I feel like it's useful to see can we actually automate the stuff that it really really looks like AI should be able to automate. Open AAI has been trying to automate customer service since 2020, you know, like and it it it's you know like it's not >> which is pretty amazing. >> It's it's wild, you know. It's wild.

23:56 Well, and then inside I mean inside companies um >> there's very little that's automated right now like and the projects haven't worked >> other than programming has worked amazing. >> Can you can you maybe classify the types of problems you think that are easier to automate now because it it was kind of interesting. So we've actually looked at support before the current generative wave and it was interesting.

24:17 you'd meet a company and the company would say um we uh we answer 95% of all like you know like help desk calls and like that is so many >> but then you actually look at the data >> and you realize it's all password resets >> and then like but if you did it by like the uniqueness it was only something like 50% or so so it just feels like when you're dealing with humans and natural systems like there's just kind of this very kind of >> you know like long tale of exceptions and so to what extent did like I every but

24:49 probably every hour I have somebody ping me and like I'm using Jeb for this new use case. I'm like I had no idea you know like you know and so like to what extent did you even predict like the broad range of use case for it like did you assume that was going to happen and have you been surprised by that or >> extremely surprised did not assume it would happen.

25:05 Uh this launch was >> like not something like if anyone expected this they are probably insane. Um right like it is I I don't think someone could expect a chat GPT for developers cuz chat GPT was for you know like you know the normal users and >> it's weird. I actually don't even know what percentage of the people who are part of the JF party are developers themselves.

25:31 I can't imagine non-developers using it. I don't know how they would use it. Um but even my non-developer friends are just like part of the party and Twitter and memeing and everything like that. So, number one, phenomenal. Number two, um this will be hard to convey in this short message because like it's been like blood, sweat, and tears for years now.

25:52 Like the amount I care about reliability is um it's it's a lot. Like reliability is what this thing is. If you don't understand that, it'll be very hard to make like a copycat that's benchmaxed. Like it's >> I feel like every nine of reliability is going to be so valuable for everyone even if it's not the most valuable thing market capwise because it will just enable new applications and like we are fighting for like all sorts of like weird nines of reliability that like we don't even fully understand because we are

26:27 just like you know like really getting this like electric motor of AI of intelligence like into people's like workstations and they can figure out what to do with it. What does reliability mean in this context? Is this is this like availability of the model or is it like I call the model and it returns the same thing or like how do I think about reliability?

26:46 >> Yeah. So >> for something that's inherently kind of stoastic. >> So uh not so much the former thing and the second thing is closer like I would describe the first thing as kind of like uptime or SLAs's. The second thing I would maybe call closer to determinism. The something thirdly I would consider more like robustness. So robustness I would kind of describe as um similar intelligence every time.

27:09 >> Oh, interesting. >> Yeah. So like not exactly determinism because I think determinism it's useful for unit tests but not real systems. Think about like if you add a UU ID to a prompt, it should be the same cuz it's the same functionally but it's not exactly deterministic. I think that there's a another layer of it that I don't really know what it's called yet.

27:28 Like maybe this is what I would call like some form of intelligence which is it doesn't have to be the similar function every time but it needs to be smart every time you know like if you were in that situation would this be an understandable thing for a human to think because a developer can program around that and actually to me the highest honor of reliability will be to get to the point where people can program against Jev without making example queries like when you just trust it you'll be in like peraflow state

27:59 And that's where I think it's unrealist feel free like if if it's too weird just feel free but um it occurs to me that actually the value of things like coding agents goes down if you have a primitive like this in a way which is like you could be like you know whatever some you know codeex builds all the software for me but it doesn't actually use Jev and so like the software itself it creates is somewhat limited or you can be like okay I as a human being I will write the software without using a coding agent but I've

28:33 got this very generalized primitive that makes writing software easier. So like do you feel like see a future where it's like the coding agents using Jev and then you're telling the coding agents and then do you have like redundancy or do you feel it's like humans hip? >> This is more of like a coding agent question than it is a Jev question. Oh yeah.

28:49 So my vibe is that um I'm not in the coding minds as much as I'd like to be. So you are you two might be in there more than I am which is sad but my experience is that they are really good at syntax and really >> they're bad at semantics. >> Um I would say incredibly bad at architecture. >> Yeah. >> Um and so like to me architecture is like the most human creative part of software.

29:16 So I love using coding agents. I think that uh Jev is almost certainly not in distribution. that would be spooky if they trained on our user data. Um, so it's probably not. Um, but uh I think that when it is in distribution, I I see no problem with like having it do the syntax. And the thing with architecture is that maybe the models are actually like not just crap at architecture, but maybe they're 50th percentile architecture.

29:40 And if you don't know anything about architecture, it would be fine. So these are all like gray area tradeoffs in order for you to navigate. And sometimes speed is the the knob for your company or project to turn like you're willing to do a 50th like a 50th percentile architecture instead of a 60th because you want to move faster and have like codeex work overnight or something like that.

29:59 Right. >> Yeah. Actually kind of along those lines one of the interesting things or phenomenons in the market already is that you know when the coding agents came out it was the SAS apocalypse and all their values dropped through the floor and then when Jev came out every SAS company is like this is the greatest thing ever. So explain that. >> I I don't know what else to say, right?

30:24 Like I I think it's like quite natural. Like in the SAS apocalypse story, um the the story that I feel like has panned out really poorly is that software is very cheap and perhaps easy to replicate, which I think I could I could believe the former. I could not believe the latter because a lot of the stuff happens beneath the hood. Um I'm maybe overly a software fanboy here.

30:46 All of us. >> Okay. Okay. Okay. Um I didn't know where might be the coding. >> We have a lot of legacy around that 100%. >> Yeah. So I don't think that really panned out. So SAS seems like maybe the markets don't agree, but like I think SAS is providing the same value it used to. Maybe the markets are just scared. Um but I think that SAS will be one of the largest winners of like the whole AI game.

31:13 And I want to like work really really well with like all the biggest most boring most like in the know of user problem SAS companies because I think that they are the best positioned to know what workflows to automate what do people need like that's what their bread and butter is and to spend the big like you know software is always a capex investment and but like you spend it ahead of time in order to make this experience even better that gets you know like distributed to all of that mass of users.

31:40 So, I think that it's going to be I'm I I'm not going to forecast anything about the financial markets, but I think for as far as like a capabilities games goes, it's going to be like an inverse apocalypse. And I am so jazzed about it. We I should make a name. >> Yeah. Yeah. That Yeah, that that should have a name. >> SAS Palooa. >> Oh, that sounds a little too fun.

32:04 >> Well, all the SAS applications are going to all of a sudden get like dramatically more useful. And by the way, you know, the the the kind of uh capital investment like so much of a of a SAS company's capital investment is like actually getting to all the customers. And so if you've gotten to all the customers and then you make, you know, not just put a chatbot on your SAS product, but actually make the software like way way better, um that's a hell of a thing.

32:29 I don't know if this is a realistic dream or not, but I think that there's a world where like the multi-choice choice forms just disappear, you know, like I I feel like they are like they are always like something mapping natural language that usually the software already has into like a Jeike output and I think >> it's literally it's literally from the 80s.

32:51 It's like it's called we used to call it 4GS. Do you remember >> fourth generation language? Actually, also I think this is from the 80s. This might be an insult. I was more than um the like I think that do what I mean is going to like be taken to the absolute next level. If I could shout out one Jev application, I don't know if it's reliable, so I can't promise anything, but it was so freaking cool.

33:12 Someone was using like a voice to control your computer and it was basically constantly making decisions on like is this a command or is it inserting text? Where's inserting text? Like like that sounds so unbelievably cool. Like like I feel like interfaces could like just completely change and maybe we're going to have to make it like cheaper and faster, you know?

33:34 >> Yeah. Then you're at Star Trek. >> Well, you know, there's there's just such a profound intuition here, which is um um you know, if if you use AI today to generate software, right? you're still creating the the same software that you did before, but if you actually look at like the average PR for a large company, it's like 10 lines, right? Seriously.

33:56 Nove. We actually did the study. So, it's like 10 lines. So, like you're automating 10 lines. And by the way, those 10 lines are are like, you know, part of a learning from a customer or something. So, like you're you've kind of optimized something that's actually pretty minimal. Um, but what it doesn't do is provide new capabilities to the software, right?

34:12 It's kind of automating this thing which in the limit ends up being relatively minor. And now like there's actually a new capability and so like it could just be the case that just software just actually gets better. Yeah. And and by the way it didn't even before Jeff like just it didn't even occur to me that like it doesn't matter how much you know AI coding agents you use the software actually isn't getting better.

34:31 Maybe you're writing it faster. It's like arguably getting worse just because like there's less oversight. So I think this is >> well and and often more insecure. >> Yeah. For sure. For sure. But you're but you're actually now can make an argument like like like apps will have new functionalities as a result of this because there is this new primitive that you're providing that I mean like in in a way like it speaks natural languages and it can reason but it marries that to a state machine.

34:57 I if people take that as a takeaway that would be like the greatest compliment ever to what we are doing like I actually feel like it's almost too grand of a vision to expand beyond the three logic gates that we have into like you know it our types are kind of like one of the same logic gate but like one that's like a little brain in there like that would be the greatest compliment to like the the type safe legacy cuz like that is that's a very non-trivial huge thing for the world.

35:24 Um, I'm not going to like overpromise underdel that, but I will fight for that. >> Yeah. I mean, listen, I mean, there's I think pretty open questions to what like how deep can this get as far as like like really serious stuff like state consistency or durability or like real systems level stuff where you actually need to like provide strong guarantees.

35:44 And so 100% this will change things like whatever analyzing logs, analyzing emails, providing a UI, talking to the human like that for sure. But like you know you could argue that over time this becomes like a smart database like you know and so >> and also >> air traffic control system >> which we really need. >> True true >> a little scary like I think automate the easy work before the hard work is always my philosophy but I also think there's going to be like an entire era of probabilistic programming that's opened

36:12 up like my >> by the way you know there's a huge history of proistic program that basically died in like the 70s or I'm familiar with it. I I I actually think it's going to be like with the same like you could also call Jeb like neuros. >> So your co your co-founder Eric came from that background. He was telling me kind of basian. >> Oh yes yes yes.

36:31 He did a lot of biology stuff that goes up and down but like what I mean is a more >> I'm not a fan my brand is pragmatism. Incredible pragmatism. I'm not a fan of like biologically inspired stuff um at all. >> This never worked. Have you ever noticed that? I mean I I think it's never worked. It's useful to motivate crazy people to work on things for decades until it works and then they refine it into like the engineering >> the story of AI neural nets for sure.

37:01 >> Yes. But like you know a lot of the stories about how it worked were not accurate right so like the hierarchical features of applications really did end up working cuz like otherwise resets wouldn't have worked. longer story. Um, I I do think that it opens up like like from a systems perspective. I'm not excited about this part cuz it's really I'm excited for the world, not about me programming this cuz it sounds like really complicated.

37:22 But I think that as we have like lots of intelligence at lots of uh like different cost and speed trade-offs, the super systemsy types will be be making trade-offs at like you know like dev is going to be like a thousand times too smart for them. They just want like an approximate link to have an approximate guess to like optimistically route here and there.

37:41 It's gonna be like so crazy the the type of stuff that's available in the extreme systems >> and the and the good news is we get to like rebuild systems again which is great right we have a new no seriously we have a new primitive it's kind of a new way think about doing software like I mean we did listen we did this with the internet you we did this main frame to client server I mean we do this periodically and and by the way just because of the um cyber security issues we probably have to rebuild almost all the

38:08 systems uh to to just make them safe. I I would think >> I think it's pretty clear that there like there's not no >> or at least the critical the the critical infrastructure for sure. >> Yeah. >> Yeah. Yeah. Do you do you think about this more in terms of like apps, SAS, analytics or more in terms of like systems foundations or all the above >> for what I would think of or how >> just general application for this when you think about like like you're working on Jev and like and you kind of envision that people are

38:39 adapting it like how you know like do maybe do you even have an opinion? I have a little bit and it's so the way I think of it is a little like uh like deep into like the TCP guts, you know, like UDP TCP, you know, like it's unreliable, too reliable, >> speaking my language. >> Exactly. And like I I so like when I think of AI no I mean like when I think of AI and this is why I care about intelligence per dollar to be clear when I think and how I got to this conclusion I work backwards from AI based economic revolution

39:11 AI everywhere know sci-fi and everything like all the software has AI all over the place and I asked myself the question what percentage of the calls to AI imagine it's like a function which what percentage are like for human consumption where you need style and everything and yeah and it's going to be like many nines and actually from that same question, how many will be at the first layer versus like deep in the guts, right?

39:33 And I think that it's going to be many nines in the guts, but it will start at the first layer. But like we need to if you don't aim for the guts, right? That sounds weird. If you don't aim for the guts, you it's it's going to take you a while to get there, right? I think people don't understand to what extent like AI was kind of shipped in the night with software.

39:49 Like even if you try to embed AI in software, it kind of like didn't behave, right? because software doesn't really take natural languages and you like do all this weird stuff like you stick in the prompt like here's the JSON output that you want and here's the schema and it would never listen to it and so and so what you ended up doing is just taking the output and giving it to a human you're like the hell with it right or another LM that is what a while loop is like the agent while loop right so it's like it's like

40:12 from first principles it needs to be human in the loop which is chat >> or an agent which is the while loop because like the natural language needs to be fed back into another >> and I will say I I have watched this h there was almost like this kind of like five stages of grief that like you know people would pick up AI and like I'm going to use this you know within my software right and then you know and then it would like go with like you know whatever denial like try to make it work and like anger then they go to

40:37 acceptance which is like okay never mind I'm just going to give this to another LM to a human being so it's been very ships in the night I think this is the first time I have seen it's almost like actually you can take an LLM you can take AI and you can actually map it to like like a state machine and you can do that productively And I hope so. I will not want to overpromise underdeliver as well.

40:56 Like I don't know if it's ready for all the applications that have been overpromised. I really really want it to and my team will fight for that obviously. Like we >> really really care about reliability. We could have released so much sooner. I don't think people realize that and I don't think that honestly I don't think that they will based on what I see off the Twitter discussion.

41:19 I think it's people will never get it, but like it'll just have like that good vibe of like, oh, I can trust this. So, >> it's the anti- frustration machine. >> It's it's a I hope so. I hope do what I mean, right? Like to me that is about like smoothness in the world, like having everything like just move more smoothly together and interlink like a gears.

41:42 I actually have my whole like AI utopia on like different axes that I really really want and like do what I mean is a huge part of this, you know, like imagine if all technology just did what you mean. >> That is like that's not sci-fi. Look how smart AI is, right? >> Yeah. No, it's a it's amazing. And and maybe that's the thought to close on. Oh, >> do what I mean. >> Yeah. I love it. >> Thank you, Dio. This has been a great conversation. Really enjoyed it.