Kavak's Playbook for Rebuilding a Company Around AI Transcript, AI Summary & Key Points

00:00 I'm investing more today in tokens than in knowledge workers. We could build superhuman agents. This means that by every dimension that matters, our agents would outperform the best human if we had ever hired. >> The most ambitious companies listening to this will decide to follow suit, which is you decided to build an agent per customer. >> Yes. Every day between 100 and 200,000 agents get instantiated specifically for this customer with its own virtual machine.

00:28 There's a lot of people worried about how the organizations of the future are going to look like and the role that humans are going to play. If you haven't faced fear before, if you haven't felt it, then you haven't tried AI. We launched a program inside Kavak that's called the Jedi Academy. From the CEO to like AI engineers to mechanics, we train everyone and after 6 weeks, they launch state-of-the-art agents to production.

00:53 >> What advice do you have to future founders or first-time founders that might be listening? What works right now is >> welcome back to the ACC podcast. Uh today we have Ali Masa the head of AI at Kabak. We're going to discuss today the transformation that Ali led within Kabak to turn it into an AI native company. Thank you Ali for being with us today.

01:15 >> Thanks for having me. >> Before starting at Kabak uh you were running a company called Oppy Analytics. >> That's right. >> And you were very much into AI before Chad GBT. You want to tell us a little bit about that journey? >> Yes. Yes, of course. Well, we called it machine learning back then. It was a different family of of algorithms and and we founded a company with this very like like ambitious vision there that that new machine learning models would be so powerful that they could solve any complex problem.

01:45 This was pre-transformers, right? This was like 2013. So, we started building the company that way and I think we were like 10 years ahead of time. uh but we built a great company. We served like Fortune 500 companies uh around like risk algorithms, logistics, forecasting, marketing and but like really the the the power of what Transformers and then like the Chady moment uh when it arrived make things like very clearly that that we could now build a whole new uh company and and and way of of of building companies and

02:26 we joined Kavak to and Carlos to build that. >> Amazing. All right. So, we're going to spend the bulk of this podcast talking about exactly how you've identified Kavak. But maybe just to start, what does Kavak do and what is your role there? >> Kavakh started out as a use case as a used car marketplace. So, we buy cars, we refurbish them, and then we sell them and finance them.

02:52 But to do that, we also had to build a fintech and a logistics company and the Carfax and like basically all the infrastructure for this to work didn't exist in in Latan. So we had to build everything vertically so we could serve our customers the right way. >> I'm going to sort of start with the framing of what the architecture looks like. So a consumer comes in and says I want to sell my car like how many agents do they touch?

03:18 Like what's the harness look like? Like ground us in how you design this. >> Right. So, so, so we bet the company in transforming to a company run by agents. The questions we ask ourselves is how would we build Kabak in 2035 with fable 10 or or or GPT 10 level intelligence and actually that company looks very different than than what we had built or what we had back then.

03:42 So when a customer comes in right now um agent will get spawned specifically for this customer with its own virtual machine. It'll remember years of interaction of this customers with with Kavak what they visited in the web page or a call they had two years ago. remember everything like in its memory. Come up with a strategy and set a long-term goal to maximize the lifetime value of this customer and do whatever it takes to make the customer happy and convert them into like all our different products and like across

04:19 time and this is a completely new and groundbreaking architecture at scale. Uh I think because like people are still building multi- aent system with with experts and and we realized to bet that longunning agents with hard goals not just workflows uh could could maximize our our customers uh satisfaction and obviously their their lifetime value. >> Awesome.

04:44 Okay. So we're going to jump to the nuances that but maybe versus many companies that say hey we want to be a gentic and they try some workflows you guys took the just rip like we had to make this work you had to downsize dramatically it didn't work for a year >> right >> so do you want to talk through obviously you had to >> tune a lot of things to make that work like describe the harness at that time and like what models you were using and sort of specifically yeah >> so so there there were like three main decisions

05:15 that that we had to make. The first and this is where I think many companies are stuck right now is the first instinct is okay let's adopt AI and you you basically leave your structure as it is and just give chat GP to your cloud to to your team and then there's no efficiencies your customers have the same problems and and nothing happens right and so so you need to redesign your whole company around the agents and around the future capabilities and this means really like rebuilding most of your APIs, rebuilding your

05:48 system so the agents can use them to to perform. >> Then you need to start generating the data and the feedback loops to fine-tune these agents. The only way to really make them work is if you teach them. And how do you teach them? You you put them out in the open. You you put them in front of customers. You get that data. You get those evals and then you train your your your agents.

06:09 And this is the second bet that we made that that we could build superhuman agents. This means that by every dimension that matters like conversion, lifetime value, uh customer experience, our agents would outperform the best human we had we had ever hired and we put them in front of the hardest problems. >> Um and finally you you start to change how you measure the success of the company.

06:37 Kavak was a transactional company. We used to measure how many cars we bought, how many cars we sold, how many brakes we we needed to to brake pads we needed to buy. And we moved to a relational company where now I have 10 million customers in my database and I have agents assigned to most of them with the task of maximizing their lifetime value. Now, we're selling cars and and and personal loans and very high ticket items.

07:04 So just activating 1% of this customer base, it's like hundreds of millions of of dollars uh if we do it the right way. So so it's a bet that made sense for us because of our industry, because of the ticket, and because at the end of the day, customers need to build trust with with a company because they're buying a used car. And the way to build trust is to to know them and to and to plan and and nurture a long-term relationship.

07:35 >> Ali, I just wanted to double click on something. You know, evos over agent demos. Yeah. >> Um you probably get pitched a lot of new agents and you know, it's never been easier to build things like before. But um one of the questions is like how do you how do you guys go about evaluating this? Because not everybody test them across 90% of the customer interactions to see if they're really working.

07:55 And you know you guys I believe is it about 98% of the interactions or something yes like that are now handled by agents. >> Yes. Totally. So so to give you a sense of the scale 90 like 96% of all interactions uh are handled by agents. So so no humans there. Um like 95% of all transactions are completely handled by by agents. Obviously, you meet a human when you pick up your car, like there's someone physically there to give you the keys.

08:26 But the rest of the of the of the experience of the journey is handled by by an agent. Every day between a 100 and 200,000 agents get instantiated in a day. They wake up, they work sometimes for three minutes, sometimes for eight hours, sometimes for three days, and they like set an alarm clock for their next task and they go back to sleep. So the scale of this is just is just amazing and and it's working.

08:53 Now, how do you get this to work at scale? And the answer you mentioned it is is evol like I like to move extremely fast, but in order to move fast, you need to have brakes, right? Imagine a car. Uh you'll hit on the gas just if you have the right brakes. And AI is super powerful. And I've seen many companies get this wrong because they try to go slow because they they don't have the right brakes.

09:22 So, so I thought about it the other way around, like how fast can we go? Well, it depends on the quality of our evils. So, a good rule of thumb here is we spend about the same amount of time, engineer time, tokens, and and and money on building the evils than building the agents. And this is how you get better and better and better, not not letting evils as an afterthought.

09:48 So, what do we measure first and foremost? Like the the the results for the business. Like if my customer is happy, they'll buy a car, they they'll get their loan approved. Uh they'll sell a car to us and and that's the like first check like did it convert? And that's where most things break. Like I I see companies like measuring number of calls or minutes during the call or or some like superficial KPIs that give you some information but that doesn't really work.

10:19 Like the important thing is did this customer convert? Is it bringing value to the customer and is the customer happy to re-engage with us after a while? And once you get those evals connected, then it's just optimizing the right agentic architecture and giving the agent skills to to scale this and and cater to millions of customers. It's really really amazing and you know related to this is like okay so you create the right evos it's working you know some people some companies still feel a little bit risk averse and

10:53 putting them in front of of of the customers and being able to perform >> the highest leverage task which in your case would be selling. Do do your agents really sell to customers? >> Yes. So so we never built customer support or customer service agents. We we built like sales agents. It's extremely hard to sell a car in in Latin America. So imagine someone wanting to buy a car.

11:18 They can choose like like amongst like 20,000 SKUs. Then they need to pick like financing and go through the financing process, insurance and and coverage. And then they're probably trading in their car. So So we need to quote that car. So, it's a process that if someone does it or or the way Kaak did it back in in in 2020 2021 was you need to be extremely good at 15 different things and have 15 different experts in 15 different teams.

11:46 And usually the person would go and speak with the expert in financing, the expert in car advisory, the expert in buying, the expert in insurance, and they'll build a package and buy a car. That's extremely hard to do. But like the first thing we did was okay, can we get an agent to be better than the expert in each of these things >> and then put it together and have like a mega expert that's an expert in insurance, financing, etc.

12:18 And that's who we put in front of the customer. So the experience for the customer is amazing. We tripled uh NPS and customer satisfaction score by putting the the agent in front of the of the customer. And it at first it converted like 50% more than our human team and now it's converting over that like 2.1 uh x more. So it's a completely different concept.

12:45 >> Your agents are better sellers. >> Totally better. And and and you get this right because they're experts and they're infinitely patient and they know all your history and they they they can plan for the long term and they never get tired. So, and if they make a mistake, they learn it and the next day, not just them, but the other 200,000 agents will have learned from that mistake.

13:08 So, that's the feedback loop that we engaged and and that's showing in the growth and results and satisfaction of our customers. >> Yeah. One of the well, one of two of the very cool things I think about Quebec is I think I think the world has gotten comfortable with AI can do customer service. It's still very hard to do well. But you know, as Gabe said, there's still a view that well, customers aren't going to want to buy expensive things from AI, and you are proving them wrong.

13:35 >> The next layer on that is, well, you're not actually going to be able to do regulated financial services end to end with AI. >> But if you walk through what you're doing, you are underwriting a thin or no file customer. >> Yes. >> Pricing them correctly, doing servicing. So, so maybe talk through how did you how did you write the evals to get comfortable with that and then versus I don't know going to a bank branch or or even a fintech sort of how how is that experience?

14:03 >> Yes. So much better. >> So the first financial product that that we launched was a car loan and usually in in Mexico and and in some emerging markets it'll get like two months or or or or more to get a car loan approved. Um, we usually approve it in under three minutes. Uh, which is like pretty cool because we have all this data around the customer and the car.

14:28 And if the customer can't pay for the car anymore, they'll just return it to us and we can give them a cheaper car and and then the they're pay a smaller amount each month and they like get out of the water, which is amazing about the the vertical integration of of the business. But then like when we started launching other financial products, we we realized that this is a very important decision for the customer, right?

14:53 Like like they usually take three to four months to make their make up their mind and buying a car and and and and getting a loan or getting a personal loan like a large personal loan that that we also um do. So if you get to know your customer throughout this process and make the the the process easy for them, then just your conversion and retention metrics start going through the roof.

15:20 It's not just the transaction is understanding each customer personally >> and get them to to convert when they're ready with a very deep personalization of the interest rate, the the risk, the max amount of the loan. in a way that makes sense for the portfolio as a whole obviously but that's optimized to the risk level and and probably the other offers that the customer is is getting >> and then maybe give us just to be um you know eval are always a very hot topic.

15:54 You kind of led with that. What is a like what is an example of maybe a a hard to design area for evals or one where you had to spend extra amount of time with just given the fact that like there's real money PII at risk. Yeah. So when we decided to to to to redesign the company around AI, you ask the question, okay, is is AI going to be able to do this job like even the CEO job or or jobs where the leadership is?

16:21 And the answer honestly is probably yes. Like in 2035 with a rate of improvement, it will be able to do. So we said, okay, let let's try it now. Let's try and build an AI CEO. So we carved out a city in Mexico. It's it's Quavaka. And we put like an agent in one of our harnesses as a CEO and it starts learning and it starts making decisions and evaluating on those decisions and it's only been running for for six weeks now.

16:54 >> The goal of the first month was to double the the the profits of of Guavaka. >> It didn't reach it but it was 1.5x like 50% more profits. just like managing the the city which is it's crazy right it's it's amazing and and it's it's the CEO like people were like that was the last job AI was supposed to take and no it isn't really and how did this happen and and it's like uh very smart person like like fields metal level smart like going into every single number every single customer making the perfect forecast and

17:33 going to micromanage every single things that needs to be executed every day to reach a plan. So he'll literally send messages to all the physical workers in in Cornabaka with their plans for the day and ask them to send voice notes back to to know their their progress. So customer satisfaction grew. Uh we got a better inventory. We rotated better, better financing penetration.

17:59 like every KPI started to to improve. So, it's super cool. It's super exciting. Now, what are the jobs where where we think uh we're still like training and hiring humans? Those are related to the physical world. >> So, when we talk about mechanics, Karak has around I think in Mexico around 800 uh mechanics. There's lots of dexterity and and senses that's super hard to to substitute.

18:29 So there we also build these agents with the exact same harness that's scaling and the the mechanics have the sidekick. Um I was telling you guys earlier it's like the the movie Ratatouille like the the mouse that's actually a chef collaborating with a with a human. It's kind of like that. So it's a sidekick. We call it El Mike. And it tells them how to inspect a car and gives them tips and and shows them the way to to do it.

18:58 And the quality of inspections again went through the roof. We're inspecting faster. We're repairing faster. It's cheaper. But most importantly, we're delivering higher quality cars. Um warranties came down around like 20 26% since we since we launched and customer satisfaction again went up. So it's about this like how would you design your organization from scratch?

19:29 uh with with with abundant super intelligence that's that's cheap and just go build it. >> Now, this this is a good segue to a key topic right now in Silicon Valley where you know there's a lot of people worried about how the organizations of the future are going to look like um and the the role that humans are going to play. Yes. In this and I think you touched a little bit on that.

19:52 So would love to hear yeah like how you guys are thinking about that and >> Yes. and the organizations. Yeah, >> totally. So, um we we took that question very seriously three years ago and the truth is that everyone's job will change. So, and what we were doing a couple of years ago will probably be be be performed better by an AI agent, right? So, what does this mean?

20:24 We need to train everyone. So, so we launched a program inside Quebec that's called the Jedi Academy where anyone from Quebec like from the CEO >> to Yeah. And it's awesome like from the CEO to like AI engineers to mechanics like going to the academy. It's super hard like I I I've led >> I led them myself. >> You designed the program. I designed the program >> but constantly >> constantly because you you need to be upgrading the program because everything's changing so fast and there's >> like you can't send these

20:58 people like outside to Stanford to to learn this because like it's new stuff right so we train everyone and after six weeks they launch state-of-the-art uh agents AI agents to production and it's mechanics and and finance guys and engineers is like everyone can do it and what this generated is maybe this person won't become an AI engineer some of them have but they they know how to collaborate with this new technology right so the way we looked about it was guys there there's no way back like this is the way kabak is

21:37 going this is the way the company will look like these are the changes for the engineering team the finance team the product team like this is what's going to change. You have the choice to like train and and get the skills to perform in this new reality, in this new world. Um, or maybe leave Kavak if this is not for you, but this is the way we're going.

22:02 >> And it were great like like we we we strengthened the culture. Everyone was super excited. Um, people really know how to build this agentic systems. And then if you look at kadak now any process it's really a collaboration of agents and humans and sometimes like agents are the bosses or of of humans and sometimes humans are designing the agents but I think we we managed to really build this and and change this and it's through this idea that we need to be learning every day and things will continue to change and the

22:39 only way to to continue being relevant is to upgrade your skills uh every month or every couple of months. >> But you do have or did have you know thousands of people now agents do most things. So like what is the org structure of Kavak? Like does the middle management concept even exist anymore? Like what does your org look like? >> Right? So the way it looks like now is very flat teams, very senior teams, super empowered.

23:05 If you look at a team, you'll have engineering, AI, like operations, like everything. And they're either building the agents, working for the agents, or being in the physical world in front of the customer. Yep. >> Like most of our organization looks like that. So, so it's really built around um around the idea of how organizations will look like in the future and around AI and really harnessing this this new technology.

23:35 Obviously, this required lots of retraining because in 2023 >> or 2022, no one was building agents, no one was helping agents or taking orders from from agents. And the way you cater to the physical world or the customers was in a different way than if an agent's telling you what to do or helping you make your job uh better. >> So it's a completely different structure than than we had just two years ago.

24:01 >> Yeah. Explain um we talked about this before what working for the agents look like. Like I think the way you described it was you knowic system and then sometimes when it fails it's like oh that's kicked out to kind of a human queue right >> but then that's lost. And so how have you brought that together? So, so like the we see human in the loops and and and most of these agentic systems in production right now like large scale agentic systems usually if an agent hits a wall or can't perform anymore it'll like send

24:28 uh this case or this customer to a tier 2 support and forget about it. That doesn't really work because you don't close the loops. So you don't generate the data to train the agent to do this better. What works right now is we have an agent that's obsessed with each of the customers like millions of this. They have access to every single API, every single skill like and we have agents building those humans building those skills for them.

24:54 And then if an agent hits a wall or cancels something, it'll call this API saying I need help. And on the other side, it's not an agent or software, it's a human helping them out. But if you map this out in an org chart, it's really human teams that have an agent. I'm getting better results. It's super clear like it makes sense. >> That's actually a perfect segue.

25:18 I I know you get lots of of leaders at larger institutions inbounding to you. So So maybe this will save you many phone calls, but I think rationally many leaders of companies intuitively understand this. It is very hard still to deploy AI through their organization. like the models are good enough you know that it's a it's an or problem it's a psychology problem like what advice do you have or what what have you seen I >> think it's two things uh the first is it has to be top down because of this like >> if you just

25:50 get adoption it won't go anywhere because it's it's hard to generate this taste or or strategy for people to to bottom up decide what to build and what not and and come up with something that works for the company. So the transformation has to be top down and leaders need to adopt and leaders have to have a very clear plan on what to build. I've seen so many companies it's just like oh like we're doing a hackathon people are coming up with use cases.

26:18 We're sponsoring some of these use cases. That doesn't work. It's like be very clear on what the company will look like in three or five years and then start building that and be like very vertical in in guiding your troops towards that. Like an army doesn't really work if everyone comes up with ideas on the strategy and tactics and goes to the battlefield and like does whatever they want.

26:42 Like you need a very clear strategy and that's what we need now. It's a like transformation stage. The second one is you need to measure what what really matters and it's evolves but it's also the right evolves. So I see a lot of companies spending now huge amounts and they say okay I got adoption I'm just spending like hundreds of millions of dollars in in tokens now >> what about that like there's quality in the tokens.

27:07 So have a framework here that's also useful like level tier three tokens the most valuable are this agents where you can get the ROI of each specific token and I can do that now. That's great news for me because I'm growing and because I know the ROI of each token because it goes to agents that are performing the job of the organization. Right? These are the best tokens.

27:33 Yeah, tier two tokens are things that you can measure indirectly. Do I see devs in the in the codebase and I can evaluate the value of these tokens at least indirectly and then push those to productions. Tier one when most companies are is people are just using plug code or chach or co-work or whatever. What happens with those? I have no idea. So it's not just about adoption.

27:57 It's really about having a very clear vision and then measuring that each token you spend is bringing you those benefits and just iterate iterate iterate from from there. And I want to uh and we touched on this a little bit, but I think it's worth a dive as um maybe the the most ambitious companies listening to this will decide to follow suit, which is you decided to build an agent per customer >> versus per task and then discovered along the way that each one of those agents needs its own micro virtual machine.

28:28 So maybe kind of walk us through >> yes >> those decisions that architecture >> and and I think we're seeing these results now but it was a really risky bet because people usually go from workflows >> like if I could advise everyone don't build agentic workflows to graphs or or functions or or objectives and we built that that these are multi- aent systems that can perform a whole function for a complex goals like the ones I told that to sell a car you need to do financing, purchasing like recommendations, uh, etc.

29:05 >> And we had thousands like tens of thousands of these agents working at scale running the business back in December. But then Opus 4.5 came out and I realized like this isn't the right paradigm anymore. like the the intelligence now doesn't need like the graph and the multi- aent >> lattis work and harness because it will constraint this level of intelligence.

29:31 So we decided to like destroy everything we had been building for for two years that was working that brought us to profitability that brought us amazing growth >> and start over with a harness that we thought would be robust and scalable and and leverage recursive self-improvement or new models more intelligent models coming out every month. So the way this looks like it's it's a virtual machine with an agent with access to memory and evals and the CLI where they can access every tool and every API in my company and

30:07 the long-term goal and I instantiate hundreds of thousands of these each day with long-term goals like maximizing the the lifetime value. >> Yeah. >> The self-improving organization. >> Exactly. The self-improving organization. And I think people are super obsessed with with RSI now and this will improve the the the models. But if you look at it this way, economic value in humanity for for the past 4,000 years has been delivered by organization, not not by individuals.

30:40 So what you want to self-improve and to engage in that loop is the organization that can deliver more economic value. Right? So that's the loop that I think companies will start to to focus on because if you get that loop working and it's an organization that is really self-improving and harnessing the the newer models and the better intelligence that we're getting every couple of days now uh then like you hit the exponential not just in intelligence but in the value uh that you can generate as a company.

31:13 So so that's really exciting. That's what we're working on. You mentioned that because of all the challenges on adopting AI, you saw the biggest opportunity on net new companies being formed working on this new way and then disrupting markets like you want to talk a little bit about that? >> Yes. Um there there's this concept in economics about creative destruction from from from Joseph Schumper and what it says is that the the the way innovation hits the economy isn't by companies adopting the the new technology but

31:46 by companies remaining the way they they were and incumbents with the new technology destroying the old companies. So this destroys value in the short term in the economy but in the long term it's better for everyone because this new more efficient more effective uh companies will provide better products and services for for the economy as a whole and this has happened in the past like industrial revolutions and this has always happened and this a great opportunity for entrepreneurs and and people today because it's

32:20 hard to adopt AI deeply h it's really hard for a CEO today, especially of a large company or public company, to go and say, "Hey, like I'm betting everything on AI. Uh the company has to look this way. I'll like destroy and rebuild everything I've been building for the past 40 years to become an AI native company." Like how many CEOs will will will do that in a company at scale.

32:48 So while they they adopt, new companies can be formed that are built around the the the strengths of of AI and and take over and and and and bring new products and services to to to the masses. And um this has happened before like this happened with electricity. This is a story I always tell tell my team. the the the technologies for Ford's production line were developed in 1879 and 1881.

33:15 Edison started commercializing electricity in New York and then London and he invented a dynamo that was extremely efficient. So you could have built Ford's factory 40 years before Ford. The technology was there, everything was there. But the way people adopted electricity and and forge dynamo was okay, I'm gonna leave my factory like four floors, shafts and belts and just change my coal engine for an electric engine.

33:48 And this will bring you benefits, yes, but like 6% efficiency. What needed to be done was like to destroy that factory, build it in a flat surface, not in the center of New York, but in Connecticut or New Jersey, and redesign your whole factory around small dynamos and and electricity. And then you get like the 3x uh improvement in productivity that that like powered the US during the 20th century.

34:18 And the same happened again with a computer. And the same is happening again today. People want to adopt it, >> but they're not willing to redesign the whole company and they just adopt it superficially. And in the end, that'll give you a 6% or a 10% improvement, not a 10x improvement. And it's like the innovator's dilemma at an industrial uh scale.

34:43 Again, >> I think you've just made an amazing case for any future founders out there that it's time to build. >> It's time to build. And and maybe a great a great place to to end is, you know, you've built and scaled your own company, you've now turned Kavak fully agentic, like what advice do you have to future founders or first-time founders that might be listening?

35:04 >> So, this is the most exciting time in human history. I I believe that like like we're living in the most exciting time in human history and it's the most exciting time to to be a founder because it's the first time that anyone has access to the most powerful tools and intelligence in the world like for almost for free or for $20 a month. So, so literally there the democratization of the tools for people to build has never been uh this way in human history and there's so much problems to be solved and a new reality

35:41 to be built around this this new paradigm. So say like just go for it but go for it deep like imagine what the future around AI will look like. Mhm. >> It's just a uh it's not even an exponential. Just just map a trend that's linear. If things keeps getting like AI keeps getting better at a linear scale and just build for that and and you'll come up with with wonderful ideas that will like bring a lot of value to the world. >> Amazing. Ali, thank you for joining us. >> Thanks for having me.