AI Engineer NYC: OpenAI, Blackstone, Ramp, Coinbase, Apollo, Citadel, Basis, Runway, Compute Trading Transcript, AI Summary & Key Points

00:00 Heat. Heat. Heat. Heat. Hey. Hey. Hey. Heat. Heat. Hey, hey, hey. Heat. Heat. Heat. Heat. Heat. Heat. Good morning and welcome back. We're excited to spend this final day together building on the conversations already underway and making room for a few more big conversation. Without further ado, please welcome today's MC, founder, writer, and analyst at Touring Post, Kenya Se.

15:30 Good morning. Good morning. Good morning everyone. Um, welcome to the third and the final day of AI Engineer New York. Who went to a cool event last night? What was the coolest? Okay, I went to karaoke with Swigs and it was amazing. 100% recommend. Um, so yes, I'm writeing Post and I'm your MC for today. It's actually my second time MCing AI engineer in New York.

15:57 So that's how much I love it. And um let's go to the program because yesterday our partner from Arise asked what does an AI engineer do and today we have a session from Ryan Carson called your job is now agent manager. So apparently your job description just changed overnight and I hope that the compensation will keep up. But fortunately quite a few of your employer quite a few quite a few of your employers are also our sponsors.

16:32 So they've been you they've seen how quickly your responsibility changing and let's give them a warm round of applause. Little encouragement goes long way. our most amazing presenting sponsor, Arise. Please, thank you. Thank you. Thank you. The great four, our platinum sponsors, Oracle, Sonar, Work OS, and AXA. Please big round of applause. Um, scattering gold of gold sponsors all on this uh screen.

17:09 Please, let's acknowledge everyone with our eyes and big give them big hand. and our silver sponsors. Let's give them as much as let's give them as much applause as possible because you know we don't ra enthusiasm by fear. So please clap clap clap clap clap. And finally to all of us to all our supporting partners thank you for making this possible. It would not be possible without anyone uh with like everyone you know it takes a village.

17:41 So thank you thank you thank you. Woo. I'm a little nervous. It's early, but thank you for bearing with me. Um, if you've noticed, there is an interesting progression across all these three days. On Mondays, you could join super practical hands-on workshops. And you heard from big guys like Cerebras and OpenAI to small but super interesting startups like Open Pros.

18:08 Yesterday the conversation included uh how AI translates into operating margin and today we'll hear about the capabilities companies are building toward um the work agents can take on and what managing that might involve things will be happening here in um Shannon room and it will be finance compute and infrastructure sessions in touring room my favorite name evalu valuations, benchmarking, retrieval, self-improving agents and skills.

18:40 In Saturn room, you will hear about developer tools, enterprise um enterprise data, software factories, agent permissions and token economics. And McCarthy room finally will be about model costs, coding, agent context, human verification, and banking deployment and financial agent evidence. And at touring post we spend a lot of time following all these um directions and research in these directions.

19:13 But we also have a series that the call a series called the org age of AI uh where we look at what happens to a company as it begins to work with this technology. How does the work change, right? What becomes possible that wasn't practical before? And why while super intelligence feels so powerful, why is the Roy still missing? And that's part of what makes this gathering so interesting to me.

19:42 You have people developing the technology in the same room as people who understand in considerable detail the work it needs to do. And there is a lot of those two groups can learn from each other. That's fascinating to me. And our first speakers work at the intersection of those two worlds at RAMP. So please help me welcome Alexander Shiftchenko with chasing capabilities while waiting for the frontier.

20:12 Thank you. Hey everyone, I'm Alex from RAMP. Um, and I'll be speaking about chasing capabilities while waiting for the frontier to advance. So, a quick uh intro about myself. I joined RAMP around three years ago uh on the broader applied AI team working on a bunch of different surfaces and products applying LLMs and ML to make a great product even better.

20:52 Uh to and to to caveat my presentation on ramp labs ramp is an extremely bitter lesson build company over the last year the number of use cases that use LMS has increased 123%. while at the same time a lot of our top features are growing in token spend but on a unit economic basis they're actually becoming cheaper because there are frontier models that are being released uh all the time now and these models are cheaper faster uh and better and so we're able to provide the same level of accuracy at a much lower cost and

21:28 so this is a really bitter lesson and pilled uh pattern and so given all this a question that I sometimes get why not just wait why have an applied research function inside of a company like ramp an uh application layer company right um and to answer that I would like to draw draw a parallel to 2023 back in 2023 uh you had access to kind of like GPT4 and creating an agent was a lot of work um you would have to create very very strict guard rails models weren't RL to backtrack.

22:04 They didn't have thinking tokens. And so if it went in the wrong direction, it would just kind of charge ahead and not realize that it made mistakes. You had to sweat over all of the prompts where to the point where um some synonyms could change the behavior of um of your agent because the they weren't RLED properly on uh generalizing to the user's intent.

22:29 uh you had to reduce and scope down the context as much as possible because giving too much context to the agent would just confuse it. Um and even back then we decided well we need to ship these things. We can't wait for GP4.5 or GPT5 to come out. We need to serve our financial professionals now. And we shipped uh things like RAM tour guide and many other agents even though it was not the easy thing to do.

22:54 And fast forward now to 2026, building an agent is a lot easier. The models have gotten so much better that you can kind of give them MCPs and access to kind of any context and they do a very very good job of understanding what they need to do given a task. But I still believe that we can dig a layer down and create even better experiences for our users.

23:16 It's just it requires a bit more effort now. So to present us uh some of our work from ramp labs three specific cases of how we've applied research uh into ramp itself starting with creating a specialist uh model. So in November of last year we launched ramp sheets which was one of the really early uh spreadsheet agents. Uh this was before the two frontier labs had launched their versions of uh spreadsheet agents and we had a very good launch in November.

23:50 a bunch of uh users from great companies had signed up and started using it. And we realized that well this is a very general purpose uh spreadsheet handling agent and a lot of the work that people do on ramp uh both in inside of the company but also all of our customers, accounting and finance professionals are spending most of their days inside of spreadsheets.

24:13 So we can kind of take the spread uh the harness and the brains behind RAM sheets and add it into a bunch of other um surfaces within RAMP. And as we were building this out, we realized well we can probably apply some research uh from RAMP labs to make this even better. And we sat down and we started looking where do models fall short on this. And back then this was like Opus 4.6 six.

24:40 And the place that had the most friction was when you would drop in a lot of spreadsheets into RAM sheets and you would ask it to build a DCF or or model some financial scenario. It would spend a lot of time creating a plan at first and then researching all of the sheets that it had access to to try to get a good understanding of and lay of the land.

25:03 And Opus 4.6 is a great model, but it was like pretty slow and the thinking tokens uh are also pretty expensive. And so we we took a step back and and decided to train a retrieval model or kind of like a financial question answering model that would while still keeping these frontier models or or as orchestrators would then be handed off to the this much faster model that would do uh financial question answering for that planning agent.

25:31 And so we partnered with the company Prime Intellect to uh post-train a Quen 3.5 uh 35 billion um model and we actually got above Opus 4.6 which was the frontier back then uh exact task accuracy while also being able to serve it at high coup uh level latencies. So a much faster but just a smart model that could then be used in uh in tandem with the really smart frontier orchestrator.

26:04 Um and so this was like a very successful uh example of post training that we had actually sent to prod uh as part of our ramp labs effort. Uh the second uh big thing that we are actually exploring up to this day is uh KV cache tricks and particularly cartridges are a really interesting um family of techniques where you're able to compress a lot of context up to 20 to 30 times more context into the KV cache offline and serve it for your agent.

26:35 And so if you compare it to the ramp sheets example for example um there's a lot of tasks that you know are going to happen at some point right like month end close a month finishes and on the first of the next month a financial professional will go and try to reconcile and close their books you know kind of the ledger the aging report and the documents that they will need to be retrieving and so you could prepare them as context but the agent will still need to agentically retrieve and and create a good picture and

27:08 understanding of them. With cartridges, we're able to compress all of that work and build one overnight before the the financial professional starts their work on the first of the month. And once they open up the the the agent, it knows exactly what it needs to do because it has all of the the context already uh built in. Cartridges are also a much lower lift than post- training because the way that they do selfplay is kind of a lot more generalizable.

27:36 The caveat with stuff like this is that compared to post- training, this is something that the labs might ship at some point in their APIs in their foray into uh continuous learning. And even if they were to do it in the API, we would actually be happy because we are developing a lot of this tribal knowledge inside of the company um and just learning how best to apply these techniques.

27:58 So even if it does ship as an API, we can just hit the ground running on day one knowing how to use them. And finally the third example uh of applied research that we are sending to prod is our work on uh agentic map produce and kind of agentic swarms. So more concretely we published an article on the ramp labs uh blog on how we uh used 10,000 parallel agents to pentest all of ramp.

28:27 Um, so at RAMP we have a cloud agent called RAMP inspect which helps out with setting up all the infrastructure and plumbing for something as ambitious as this. And we were able to spin up 10,000 parallel agents and then kind of map this work out try to find any vulnerabilities and then also agentically reduce on the tickets that they would create. So dduplicate them with LM and then try to reproduce them adversarily with LMS to try to make sure that the vulnerabilities are real and important.

29:02 At the end of all of this, we have actually found seven high vulner high severity vulnerabilities that we all patched. Interestingly, one of the um vulnerabilities that we described in that article is a Lewis script injection that you could send over via email and no single component of that stack was unsafe and the kind of swarm had to patch together multiple components and understanding of multiple components end to end in order to be able to figure this out.

29:31 And the other nice thing with this is that this is all running autom automatically. You don't need to spend anyone's time. is just use started and the only thing that uh failed failed passes that get filtered out is the cost of the tokens. The other thing that's interesting outside of just building out all of the infrastructure to support these swarms is that we do research into how to best use models as part of these swarms.

29:58 One of the interesting insights from that same article that we published is that you can actually use openweight models that are up to five times cheaper compared to the frontier reasoning. And because you're setting it up as a swarm, the detection rate is not five times lower. So you can kind of spend more on these cheaper tokens and have almost as good of a detection rate.

30:24 Um, if you follow through all of these examples, you kind of notice a pattern of a lot of them is attempts at saving on time and saving on money. So, reducing latency, which is time, and then reducing on token cost, which is money. That's actually RAM's uh slogan, saving our customers time and money. Um, and so I'd like to to point to two uh products that we actually offer from RAMP that have been um influenced by RAM Labs' research.

30:52 The first one is token spend management that connects to a bunch of uh your API keys and is able to give you a really good picture of how LM are being used inside of your company. Not only token spend on an API key basis or a user basis but digging deeper accounting categories um and even things like um tax credits that you can get by uh analyzing the spend.

31:17 It also goes deeper and we create a taxonomy that uh contextually analyzes and semantically analyzes the LM usage so you can actually understand how LMS are being used outside of just attributing it to a team or a person inside of your company. And then the second one is RAM router which also leans into improving your the latency of your agent where we have a bunch of routing strategies that we work with ramp labs to develop.

31:45 uh things like Nvidia Switchard, things like smart flex offloading of work to both improve latency and um cost per tokens that you spend. And there's a bunch of different strategies already available for the the for routing and even more coming out later. So, if you're interested in any of these two products, you can take a photo uh of the URLs or or scan the QR codes there.

32:11 Give you a second. Awesome. And then I I'd also invite you to check out our Ramplabs uh blog where we publish um a bunch of really interesting articles that I haven't had the chance to cover as part of this presentation. Um notably one of the really interesting ones is portal where we figured out a way of uh transferring learnings from one post train to another.

32:39 So you could do something like a Laura uh post train on one model. So like the Quen 3.5 that I was describing, but then transfer it to newer versions of Quen or even across different families. So you're post-raining this latent that then can be projected onto different uh models and that really amortizes the cost that you have to spend on post- training.

33:01 Uh more blog posts on agentic map produce. So, not only the pentesting stuff that we did, but also things like um analyzing and creating texonomies of LM usage, which is stuff that has gone into the token spend product. um things like self-improving monitoring where we use uh LMS to automatically um create monitors with data dog on a lot of our product surfaces and then as alerts come in improve those monitors automatically without a person in in the loop to try to basically create super super robust and resilient

33:36 software. Um and finally semantic tagging which is this interesting uh foray into creating this um egentic map reduce over a bunch of user sessions where we can annotate and create texonomies that are open-ended. So not knowing um ahead of time which what are the clusters of LM usage and trying to create a tonomy that can then be reconciled against the spend inside of the companies.

34:02 So if uh if you're interested in this uh type of research, x.com/ramplabs where you can uh go and and check out more of our research. And uh there's my Twitter if you have any other questions or uh want to talk about some use case as part of your applied research. And that's it. Thank you. As AI agents generate more code than ever, how do we make sure it's ready to ship?

34:40 To explore the importance of verification in agentic software development, please welcome Tar Shaat, CEO of Sonar. Morning everybody. Uh my name is Tar Shocket. I am the CEO of Sonar as you just heard. And we're here to talk today about how we have been um helping companies build software factories that actually work for enterprises. And I thought it'd be an interesting way to start to start looking at actual factories because we know how to build actual factories.

35:22 And for those of you who have a little bit of the gray hair and all of this sort of thing, you may remember the 1980s. And in the 1980s, you could find cars that were really good cars, but if you look at the picture here, and if you remember those times, the cars like didn't quite fit together perfectly, right? You'd have different pieces that would leave big gaps.

35:44 It would be really good cars that just didn't kind of work the way they were supposed to. And then uh Toyota and particularly Lexus came out with a commercial and it showed something very simple which was a ball bearing running across the seams of this car. And it was kind of revolutionary at the time because it showed that you could actually build a car that fit together perfectly.

36:06 And instead of building for volume and building and then fixing the old mindset, you could actually have a mindset focused on ship it right on no rework on customer happiness. And there was not this was not a a single change. What happened was that people really started thinking about the factories differently. They started engineering building factories that were engineered to deliver quality.

36:33 And there's a number of different pieces that I think still resonate today. The first is you prioritize quality instead of volume. The second is that you fix the system, not just individual symptoms that you see. The third is that you design for something that can be built as opposed to designing the wheels and designing the washer bottles and whatever it is and then hoping it fits together.

37:01 You solve problems as they occur as opposed to kicking the can down the road and fixing it at the end. And you empower everybody. Anyone could stop the line, not just the boss. Now, why am I talking about this? It's actually because modern software development actually embraced all of these principles. If you think about how we build software today, what is considered best practice, a lot of the terms and the practices came from that idea of a Toyota production system, agile, conbon, error budgets, if you're in the SR

37:32 world, continuous integration. These ideas all kind of sprung from this notion of how do we build factories that work? And it's worked really well for us in software development so far. But agents, as we all know, you wouldn't be sitting here if you didn't uh understand that agents are changing the game. And as agents are changing the game, weird things start to happen.

37:54 Um first of all, this is uh research that was published last month uh what of the effect of implementing asynchronous agents in your development life cycle. And I'm sure many of you have seen this. You have an explosion of the number of lines of code written up 2450%. You have an increase in the number of commits. So the volume code creation is now easy and it is now abundant.

38:20 And by the way more and more people I'm sure many people in this room are saying okay if it's easy and abundant how do I automate it? How do I build factories agentic software factories that help automate and scale this? And today we just finished our what we call state of uh code developer survey. Um it'll be out next month, but this is a little preview.

38:43 Today 5% of developers that we spoke to say that they're running Agentic software factories, but 34% said in a year they expect to and 39% in financial services. So people are thinking about how do you actually build these next generation factories. And there's a a couple of gotchas in here. security risk. The agents can work really well to help you build stuff.

39:05 It can also work really well to help you hack stuff, help you attack stuff, find vulnerabilities, find quality issues, etc. And so, as I was writing this um presentation kind of over the weekend, a little just in time again from the Toyota production system, um Sachi Nadella posted this article on X and if you haven't read it, I think it's a great article that I think everyone uh should internalize.

39:29 But one of the he was talking about insider risk of models inside of companies. And one of the things that he talked about was the need for an engineering approach to containment and governance built around strong deterministic system design, human controls and reliable operating procedures. And this really resonated with me because what we are seeing is a paradigm shift in software factory design or software development.

39:57 let's call it more broadly that we need to engineer we need to build systems that engineer confidence and quality into aentic software factories. So to start doing that I think it's important to understand the limitations that we're facing. So what's the hard thing about this? And there's five that I'm going to call out. The first is that human in the loop doesn't scale.

40:20 And this may be I don't know to this room it may not be all that controversial but every virtually every company I talk to I say what's your AI governance strategy and they give me all this stuff and they say but don't worry we have humans in the loop and the problem with humans in the loop as I mentioned earlier 2450% increase in lines of code 240% increase in commits but the same study showed only a 30% increase in releases we haven't figured out how to scale from producing stuff to shipping stuff.

40:54 There's automation needed. There's new tools needed. And really importantly, this is a concept that really I I see it every day now and I'm sure many of you do as well uh called cognitive surrender. Uh and this study was done just a couple months ago and it looked at how often do humans agree with AI. And it turns out they agree with AI when AI is right 93% of the time.

41:20 And four out of five times they agree with it when the AI is wrong. Right? So it's not that human in the loop is yeah we just need to make things faster. It's actually that it doesn't work in the agentic era. So that's problem one. Problem two is that agents struggle with large complex code bases. They are very very smart but you need to understand what happens in a codebase in order to work on it.

41:47 The left hand side is a study that was done on code translation and what it shows is that the agents were able to get it right 91% of the time when the code base was 10,000 lines of code and by the time it got to even 50,000 and I'm sure everyone in this room operates off of much bigger code bases than this but when it was trying to translate 50,000 the accuracy collapsed.

42:13 Similarly, another study looked at um how many tokens does it take on a large complex codebase to complete the same task and the variance was 30x not 30% but 30x and the reason is because the agent spends a huge amount of their tokens trying to understand how to navigate the codebase, the structure of the codebase, things like this. And it's kind of like if I'm trying to go from uh from LA to here and if I pick a random starting point.

42:42 If I happen to pick white planes, I will get to New York pretty quickly. If I pick Albuquerque, it's going to take me a long time and a lot of cost. Same thing happens with agents. Errors compound. This is another important concept. This is a study out of Meter and Meter looks at and you can find this on their website. They look at how effective are agents at completing tasks that humans can do.

43:09 And the way they look at this is can you complete a task that takes X number of hours. So you can see here the Cloud Mythos uh preview which is the latest version that they've published was good at completing tasks that take a human 16 hours. Okay, so that's really good. But there's an important caveat in the fine print here which is this is at 50% accuracy.

43:28 When you add it when you dial up the accuracy to 80% it the curve flattens substantially because as you do longunning tasks errors compound. If you have 95% accuracy on each task by the time you're at 20 steps you have 3 36% accuracy. Right? And so that is another issue we have to deal with in software factories. Probably the most aggressive headline I put in here is is this fourth point that LLMs are unreliable reward seeking judges.

44:02 Um you know so the question that we always get is well can't you just ask an LLM? And so people have been studying this as well and this is all again the first on the left hand side is from uh PayPal AI's research lab. They published a paper uh this month actually and found that between depending on the model and the situation between seven and 17% of verdicts change across what they called identical replications asked the same question got a different answer between 7 and 17% of the time and if you start challenging

44:37 this is a study out of meta now if you start challenging the LLM are you sure very simple challenge it'll actually change its mind 7 to 34% again depending on the model. So, you ask the AI judge twice and you may get a different answer. And then lastly, and I won't dig into this one too much because I think it's probably self-evident to everyone, as a time to exploit near zero, the ability for us to respond is incredibly challenged.

45:05 And so, prevention is something that we believe the software factory has to bake in to the design, has to engineer into the factory. So as we've thought through this, we have um written down what we call the code confidence formula, which is basically how do you build how do you think about a software factory that delivers what we all want, which is outcomes that we can feel confident in, right?

45:33 Not outcomes that we can trust, not blind trust or anything like that, but outcomes that we can feel comfortable with that we can sit and tell our executives, our customers, our shareholders, our CEOs, our boards that yes, I believe this factory is working well. And we think there's four components to this. The first is being crystal clear about the intent.

45:56 If you don't know what you're trying to build, you can't tell the agent what to build and you will not get the outcome you want. The second is guide rails. And we very specifically talk about guide rails, not guardrails. Guardrails prevent something from going off the road. Guide rails help keep you on it. And I think what agents need are the small nudges to guide and govern those agents.

46:18 The third part is verification. That needs to be independent, thorough, and issues fixed as opposed to just cans kicked down the road. And then lastly, evidence. Trust but verify. Trust but understand that it was done with proof. Uh that is consistent, audible, and repeatable. And we multiply these together because we've seen a number of situations where one of these elements is missing.

46:44 And if it's zero, the confidence goes to zero. If you do all of this but don't have evidence, you go to zero. You don't have confidence in the outcome. So, we really think about factory design as requiring these four elements. And I'm going to and as we put together factories, as we help customers, I'm not going to walk through every box on this page, but as we help our customers and partners build these software factories, we're very deliberate about how do you get the intent the bottom left hand side in purple, how

47:16 do you put the guardrails in place, how do you do verification in a way that's thorough and independent, and how do you produce the evidence? And by doing this, we see amazing results. So these are real results now from our uh from our customers. The first one is focusing on intent and guide rails and intent and guide rails. We have a product called sonar vortex which gives intent guide rails and in the agentic loop starts to do verifications again nudging the agent.

47:43 And what we see when we provide agents in longunning in large complex code bases with longunning tasks uh with this information is a 36% reduction in token cost because you're putting it in white plains. You actually are putting it close to the answer. You're helping it understand the map. We don't have to burn tokens on things we already know and 92% reduction in issues in the commit, right?

48:10 So it's cheaper and we can get a better outcome to go into that verification stage. This is again all because we're nudging the agent to keep on track. Then you get to verification and we started life as a verification company. This is uh how we see the world is through a verification lens and we have a very strong point of view that independent verification has to cover the full range of issues um that is both precise and cost-effective right use CPUs when you can use CPUs need GPUs GPUs when you need GPUs so we look

48:44 at deterministic mixture of methods and then you add in aentic mixture of methods and it comes up with what we think is the a gold standard of verification and What we see is when you apply this, our product is called Sonar Cube. So we talked to our customers over a thousand customers recently and we said um are you um how likely are you or how often have you seen these issues from a coding and the the statement to read here is sonar cube users are x% more likely than non-users to have seen a very positive impact.

49:20 So 81% more likely to have seen to to believe that AI coding has had a very positive impact on code security if you have that gold standard ver uh verification 34% more likely to have seen an impact on time to market 76% on outage frequency so again as I come back to how do we have confidence to ship what we're seeing is that if you put these pieces together and you build verification incredibly uh strongly as a first class citizen in the factory.

49:55 You are actually building confidence and you're seeing results. Better security, more confident in our time to market, improved um or a reduction in number of outages, improved reliability. And of course, as I mentioned, you have to do this with evidence. Make sure you pass the quality gate. Make sure whatever there's lots of acronyms up here, whichever ones you need to pass, whether that is your IT general controls inside of your uh internal controls and operations, whether that's misra if you're building cars or any

50:28 of the other ones that are on here. Producing the evidence is what lets you sleep at night. It's what gives you confidence that you can move forward. So that in essence is how we think about the challenge of the future that code is abundant that confidence is scarce and really the goal that we have and that we think that um everyone should have as we think about the future of software development is how do we deliver confidence and the factories of the future need to be able to deliver confidence.

51:02 So with that uh I want to say thank you to everybody. We have a big booth. It's a big red booth um on the floor right over that way and would love to uh talk to you more about all of this. Thank you very much. Have a great morning. Where should AI agents take control and when should humans stay in charge? Here to explore that balance in policydriven workflows, please welcome Evan Cormos, engineering manager at Coinbase.

52:04 Hi everyone, my name is Evan Cormos. I've been delivering designing building software for 20 years, the last uh 15 in enterprise financial services. I am a player coach at Coinbase. And we're here today to talk about trusting agents and policy driven workflows. So let's start with the policy and it's pretty boring. Um I'm going to go over here for my notes.

52:38 So our starting point center centers around the policy and this sets the rules for how work gets done within the organization and including especially risk and compliance. Think of policy as the original institutional intent or the original system prompt. So ideally we have this written down. Um it's a little bit boring so stick with me. Um using language like must, may and should.

53:08 And the again these help define the rules and decisions about how the system operates in the organization. Separate from policy we have the actual running system. So the first thing we're going to do is separate these two uh these two both the build state and the run state and two different moments in time. So typically we're either defining or chaining changing our policy but this the system is constantly running and it's constantly moving and changing.

53:46 So we could then sort of imagine a compilation event that oop sorry I'm skipping ahead it's constantly changing. So it it's evolving quickly with these product driven changes. So everything is expected to still be roughly bound by the policy but ultimately um we separate this from the run state and the build state. So inside the system what does it look like?

54:15 We have a familiar model if we have any enterprise uh folks in the in the room here. Um people process and technology but where really do agents fit into this mold. So I see them as an operational model cutting across all three. They work with people, execute parts of the process themselves, and they use and are made up of technology. So for our talk today, we're going to be focusing here where agents meet process, what work we give them, what boundaries we set, and how we build trust in their outputs.

55:04 So let's dive into your workflow. And it's high level and seems basic, but if you haven't, there are good reasons to map this out. So maybe a change has to be approved before it goes live or a customer goes through has to go through verification before their account is opened. We expect work to flow in a specific and gated direction. So I believe it makes it agents make it worth revisiting what we can do in parallel.

55:41 So previously some work used to cost too much or be too complex to coordinate in this way. But now we can run independent reviews alongside each other bring the results together but keep the required checks in place. So this is a directed as cyclic graph or a DAG is a way to represent this workflow or process the steps the dependencies for a flow that moves forward without looping back on itself.

56:12 Apache Airflow, if you're familiar, is an example of open source software that represents workflows, their dependencies as DAGs in Python code. So this Python code this these DAGs can be used to define and manage a number of policydriven processes like for example commercial lending. We could also use a DAG to represent the workflow to list a token on the exchange in a specific region if you're into that.

56:55 or we could use a DAG to represent a regulated flow of delivering software which is a hot topic and one we'll spend a little bit of time on. I think that we've seen a lot of adoption within agents within software delivery and ultimately we're looking to leverage that same efficiency and scale in other areas of the business and close that the the adoption gap.

57:23 So let's stick with the software delivery example for a little bit. What happens inside one of these nodes of the process? And I'll give you a hint. It's a loop. Um oversimplified, but we have a basic version of what's happening within the let's let's say this is a code review process, right? So we're going to essentially do a number of steps and essentially once we realize the work we recognize that the work is good enough we're going to allow it to pass on to the next step and that's essentially the function of the

58:06 code review node. The work doesn't always uh end up okay. So sometimes we have to take the feedback from the code review and iterate on it um until the work is good enough to pass. So this is a representation of a directed cyclic graph or DCG. So as far as open source software uh something like lang chain lang graph would be what I would um say is helpful to manage the state of DCGS.

58:48 Um and this is an oversimpl over oversimplification. These graphs can be very complex. So zooming back out the inner flow um of the node we have people and agents working together. on the outer flow we have a policy boundary but not everything goes to plan. Um so we have to handle exception flows. We don't want the work to get stuff stuck and we don't want to design for every edge case either.

59:19 Um the exception flows will monitor and see kind of what happens if you're shipping code that gets cancelled or doesn't meet security standards. That's something you should want to know about. volume shifts across the flow once we start to add agents into the mix. And we've seen this with software delivery and we'll see it in other processes as we bring agents into those processes as well where now all of a sudden if you're reviewing code faster, you need to design it quicker or you need to do the security scan that's

59:53 uh faster. So we see the volume shift across the flow. And this is where it's really important that we have visibility into the process overall from a a metric standpoint because as we start to supercharge and and parallelize and add more efficiency to the process, we start to develop more and more bottlenecks. And so the operational visibility of the process is extremely important, much more important than the authoring and the building of your workflow tool.

01:00:23 Um, so we want to visualize the process. So we want to add parallel capacity by still keeping the controls. And so this is where we'll add more additional capacity. Okay. So we're doing some work. We're checking it. We're using the feedback again to refine it until the work is good enough to move on. But why is that hard? That seems like a pretty easy thing to do.

01:00:54 But as I mentioned before, these graphs get actually pretty complex. Um, and we'll dive a little bit more into why this is difficult. And if you're familiar, one challenge with AI engineering is the variability of inputs and outputs. And I believe that higher quality inputs will result in better outputs at a lower compute cost. But what are quality inputs?

01:01:33 So we we'll generally speak to it as context um model. Maybe we can nitpick that. But um this isn't our area of focus today. Um we're going to focus more on outputs. But we can't really talk about quality of outputs without the quality of without talking about the quality of inputs. So ideally we've established a trusted context for our agent. Um and there's some references on that.

01:01:58 But one thing we can do in this model is inject the DAG state into our agent and let it know kind of where it fits into the bigger picture. Okay. So now we've added even long time horizon goals and that's emphasizes the need to optimize the cost and efficiency of this loop. So the check is where we decide whether the work is good enough to move forward.

01:02:35 We need to define what good looks like. That's the hard part. and we need to have a reliable way to evaluate it. Otherwise, more passes don't necessarily uh mean better results. The loop is really only as trustworthy as the check. So, if we want to talk about quality, let's talk about the check and let's talk about quality. Okay. Um I think building with AI is an opportunity to improve product quality overall.

01:03:04 I like some of the conversation about durable software. Um so I I'm a believer in quality uh and in traditional software um we we've built confidence mechanisms in place uh to give us confidence that the software is of high quality and that's done variety of of mechanisms but primarily systematically with test cases unit tests um checking these as a part of our delivery flow to ensure that when we make code changes mistakes aren't made that break other things.

01:03:38 Right? So this is a confidence mechanism that we've put in place with traditional software. The agents are probabilistic and so the same requests can produce different answers. More than one answer may be acceptable. And we still need evidence, but exact output uh comparisons alone really aren't good enough. We need to build a representative data sets and evaluate them um and define what good looks like.

01:04:12 We need to run uh these evaluations in repeated uh series and this is a statistical analysis right um where we actually look at the outputs uh in volume and not just as a direct comparison. So in combining with eval as sort of your runtime logs of what happened or sorry traces uh which are sort of your runtime logs of what happens um we've built this confidence mechanism to complement um traditional software and ultimately this is what we're going to be talking about a little bit more in in terms of driving confidence

01:04:53 in the output not only putting this mechanism in place but how do we begin to turn it So where does the agent belong? Um back to the lending example like we can do KYC financials in parallel and sequence like credit memo risk review documentation decision all these things in our our DAG. Um, looking at each node, we want to ask some certain questions like how valuable would it be to have an agent drive this node versus a human drive this node.

01:05:27 Um, and we can sort of look at some of these basic questions like can we check it against the policy easily? Can it run in a bounded loop? Can we trace it? Can we fall back to a human? Um, if we can answer yes to these type of questions, we we can kind of have a good idea going in that this particular node may be better suited for automation. Um, but throughout the DAG, we should look to drive agent assistance.

01:05:52 So, what we're really talking about is not whether or not AI is involved, but whether or not AI we expect to take over at some point on the decision. you would probably agree that we probably need that to be human-driven. This is essentially the check against the policy. Um, so this is going to be a little bit more difficult, I think, to hill climb.

01:06:21 But what do we mean by hill climb? All right, how do we turn this on? So day one um we start supervised and as we have agents and humans working together as a part of the process we can um manually um verify the output to ensure that it is of high quality. Um this is where we refine our data sets and ensure that we have the right confidence mechanisms in place.

01:06:49 We've defined what good looks like with the domain subject matter experts and that they're beginning to verify it. When measured confidence reaches a policy threshold, we can move towards sampling. And what that means is humans are looking at a slice of the output and not necessarily all of it. um we begin systematic more systematic evaluation through traces um in order to ensure that what we're what the humans aren't seeing is still meeting the quality bar.

01:07:23 And then finally, once we clear the confidence threshold, humans handle more of the exceptions of the process but still act as participants in refining the data sets and ensuring quality outputs. Okay, so a few takeaways from this um DAG and DCG basic graph theory, but ultimately how do we build these processes that exist today and where people are working them and bring agents into the mix and the recipe really is to keep it deterministic on the outside and make it adaptive on the inside.

01:08:10 The next is quality through evaluation and this is really establishing a complimentary confidence mechanism. So it's not a feeling about a model. We can actually provide research and statistics about the quality of our agents. So this builds confidence and inherently trusts output. Quality is a team sport. AI engineers own the pipeline to taking product signals from all directions and deliver applications that are well-built and utilize ML operations behind the scenes.

01:08:59 Finally, agents accelerate across the operating model. I've included some references on the final slide and things that I've spoken about today. Um, this concludes my presentation. Thank you very much. What does it take to turn advances in AI into meaningful progress for financial services? To share the thinking behind this question and to discuss where it's headed, please welcome the head of chat GPT for financial services at OpenAI, Joseph Kim.

01:10:04 Hi everyone. I'm Joseph Kim and I lead for financial services, a new product we launched just one month ago. Many of you have already experienced how tools like Codex have changed the way you build in software engineering. AI has become a much bigger part of the work you do every day. Model intelligence is continuing to improve at an incredible pace, but many people working industries like finance haven't yet experienced that same change in their day-to-day work.

01:10:31 I expect that to start changing rapidly over the next few months. And builders like all of you in this room have a big role in making that happen quickly. taking the intelligence in these models and turning it into products that people can use in their daily work. Today, I will share a little bit about what we learned building Chatbt for financial services and how we're thinking about what comes next.

01:10:55 I'll start by showing you what we've built, including a quick demo. Then, I'll explain why we built it and why we think now is the right time to build. From there, I'll walk you through how we built it with some technical lessons that I hope you can apply to your own products. Finally, we'll take a look at where models are headed and products are going and what that could mean for financial work over the next year.

01:11:17 Let's start with what we've built for financial services built specifically for financial professionals starting with investment banking and equity research. It's designed to help people at the large financial institutions get more out of their day-to-day work. Similar to how Codex speeds you up in software development as we built the product with our design partners Morgan Stanley and Evercore, we focus on three core areas.

01:11:44 First, financial data. Financial data is a foundation of financial work and the work you can do is limited by the quality of your sources. We wanted people to have access to the data they need directly within ChatgBT. Second, an interface built for finance. We wanted the product to feel familiar to financial professionals with ways to inspect source material, explore financial charts, and control the sources behind an analysis.

01:12:13 Third, your firm's format. Financial work happens within the context of an organization, and the models and presentations people produce need to follow their form's format and standards. We made it easier to create those artifacts using firm specific templates. But they say a picture is worth a thousand words. So I'll stop talking for a bit and show you the product.

01:14:03 Wonderful. Now, I'd like to take some time to talk about why we built it. We built CatBT for financial services to help close what we call the capability overhang. As you can see here, model capabilities have improved at a tremendous rate over the last few years. In just a few years, we've gone from models solving high school math problems to systems capable of solving millennial prize math problems.

01:14:24 But the value people get from these models in their day-to-day work hasn't kept pace. You read about the breakthroughs in the news. Yet, for many financial professionals, the way they work hasn't changed nearly as quickly. We call that gap the capability overhang. The distance between what the model is capable of and the value people can actually get from it.

01:14:42 Closing that gap takes more than building a capable model. People need the right data and tools and need and they need an experience that fits the way they work. That's where builders like you come in. I want to show you an example of how much of a difference right product experience can make. This is a chart that shows how Codex usage grew within OpenAI across different job functions.

01:15:04 As you can see, engineers started using Codex early and that share grew steadily. But look at finance, legal, and recruiting. At the beginning of 2026, almost none of their AI output was generated in codeex. By June, that share was around 90%. If you look carefully, you'll notice a huge inflection point that happened around February, March of this year.

01:15:23 I went back to see what happened around this time, assuming that we must have released a new model, but instead I found that we had released the Codex desktop app in February. My takeaway is that the product made it easier for people outside of engineering to put these capabilities to work. That's the opportunity we're pursuing in finance, building an experience that helps financial professionals get much more value from the intelligence already available to them.

01:15:49 Now, let me talk about how we built it. Let's start with data. In finance, the work you can do is limited by the quality of the information you have access to. So, we wanted the product to come with the sources professionals rely on, from financial statements and earning transcripts to market and private company data. We looked at existing MCP connectors for financial data and found that many had issues with latency errors or search quality.

01:16:11 So, we went and licensed a lot of this data directly and built our own hosting and search indices using the same infrastructure as our web index. We still use the MCT protocol to give the model access to the data, but hosting and indexing it ourselves gives us more control over the quality and the reliability of the experience. It also gives us the flexibility to post- train our models to work with these data sets.

01:16:35 The analogy I like to use is MCP and skills are like handing an instruction manual to a junior analyst versus post- training where the model is like having a senior analyst who's been working with the data for over two decades and knows all of its quirks like the back of their hand. MCP and skills give the model access and instructions. Post training can help it develop more familiarity with the data itself.

01:16:57 Once we have that data foundation in place, the next step was making it easy to turn that information to professional grade work. For financial professionals, that work often takes the form of an Excel model, a PowerPoint presentation, or written analysis that needs to be reviewed and shared with other people. So, we focus on helping the model produce those outputs in a way that fits the firm's existing workflows.

01:17:18 We built a template system that gives users access to the firm's approved templates when generating spreadsheets and presentations. The goal is to reduce the work someone has to do to get an output ready for their team or clients. We also added things like interactive financial charts so users can explore stock prices and financial metrics directly within chatbt as they work through an analysis.

01:17:39 And we built citation viewers that let people inspect the source passages and tables in behind an answer. In finance, you need to be able to check where a number came from and understand the context around it. We wanted that review to be a natural part of the experience. These are the kind of product details that help turn a model's capabilities into work that someone can use.

01:18:01 The part I think that is most relevant for people in the room is how we built it. Much of the functionality you just saw was built using the same chatbt plug-in platform that's available to us as developers. For financial data, we use MCP tools and connectors to give the model access to the information it needs. We use citation metadata and MCP apps to let users inspect their sources.

01:18:20 And MCP apps also power interactive financial charts. We use skills together with shared artifact tools to guide financial workflows and produce Excel models and PowerPoint presentations. And for financial onboarding, we leverage the native finan uh plug-in onboarding experience. What that means for builders is you can is that you can focus on the expertise that makes your product valuable.

01:18:40 If you have unique data, an existing tool or a deep understanding of a particular workflow, you can use these building blocks to bring that into chatbt and create an experience around it. Financial services is just one example, but I do think this this uh opportunity expands far beyond finance. I personally believe that we're going to see multiple $1 billion businesses built simply as chat plugins.

01:19:03 And that opportunity will keep evolving as the models become more capable. Last but not least, I'd like to spend the rest of the talk on what comes next and what that means for the products we should be building today. of what comes next. One useful way to build that intuition is to look backwards. Something remarkable the last couple of years is how predictable parts of that progress have been.

01:19:36 Model improvements have come from three broad sometimes overlapping drivers. Scaling compute, improving algorithms, and something we call unhobling, which is about finding ways to get more useful work out of the models. We'll come back to unhobling on a on a later slide. In 2002 and 2003, the most visible driver was pre-training, which meant scaling compute by making the models bigger and adding more data.

01:19:59 By leveraging scaling laws, researchers were able to predict model performance with high accuracy. For example, before GPT4 finished training, OpenAI researchers were able to predict its final loss as well as its performance on a coding eval. Improvements on the algorithm side have also been surprisingly steady. One study found that the compute needed to reach a given level of language model performance from scratch fell by roughly half every eight months.

01:20:24 Although you can't predict the timing of each individual breakthrough, the cumulative rate of of improvement follows a surprisingly steady trend. In 2024 and 2025, reasoning became another major driver of progress. We use reinforcement learning and post- trainining to teach models to work through problems more effectively, and we gave them more compute to think when answering.

01:20:44 That opened up another way to improve performance alongside pre-training. However, for folks that don't know, at OpenAI, we ran into problems with pre-training and therefore had difficulty scaling the model. In 2026, we've been able to solve many of these issues and we've successfully trained larger and more capable models as you've seen through GPD 56 SOL and GPD6 Astra.

01:21:08 Throughout these years, researchers have raised concerns about potential barriers to further scaling. For example, in 2024, many researchers were concerned about a potential data wall where we would run out of human generated text to train on. But techniques like synthetic data generation, learning from verifiable rewards, as well as algorithmic improvements to learn more from each individual token have essentially solved this problem.

01:21:29 We've also solved concerns around things like context window length using things like sparse detention and compaction. Looking ahead to 2027, I strongly believe that the trend lines of the past will continue and that we'll train larger, more efficient, and more capable models. We clearly still have very difficult problems to solve, especially around things like alignment and safety.

01:21:51 But like we have in the past, I believe that human ingenuity increasingly assisted by capable models will help us make progress on those problems, too. So, how good will models be at the end of 2027? Let's use math as an example. In 2023, GPD4 scored 700 out of 800 on SAT high school math. By 2005, an opening model reached gold medal status on the International Ma Mathematical Olympiad.

01:22:16 And in 20 in early 2026, an internal model disproved a long-standing conjecture related to a problem that mathematicians have been working on since 1946. More recently, an internal model proposed a solution to the Navier Stokes Millennium Prize puzzle um problem. And we also saw shared 719 mathematical research manuscripts across 372 families of results.

01:22:38 In just a few years, we've gone from a strong score in a high school math exam to AI systems producing original research at the frontier of mathematics. If we see another year of progress roughly in line with what we've been seeing over the past couple years, I expect that by end of 2027, models will outperform even leading human experts on a growing range of difficult tasks across different fields and specialties.

01:22:59 Mathematics gives us an early glimpse of what that could look like. And while the capability curve may look gradual, the change in how we work can feel sudden. I call this the threshold phenomenon. For any given task, there's a threshold of usefulness, the point where a model becomes capable and reliable enough to save you time. That threshold depends on the work you're doing and how much checking or human review it requires.

01:23:20 Below that threshold, improvements can be hard to appreciate. The model might produce better answers, make fewer mistakes, or get further through a task. But if checking and fixing its work still takes as long as doing it yourself, your practical assessment hasn't changed. It's still not useful. Meaningful improvements in capability can feel like like little progress.

01:23:38 Then the model crosses that threshold. Suddenly it produces something you can use. The time you spend reviewing it becomes much smaller than the time it saves you. You start using it regularly and eventually you become comfortable delegating that task. That transition can feel like an enormous leap. Even if the underlying improvement was similar to the previous few improvements, what changes of progress became useful to you?

01:23:59 This helps explain why model progress sometimes feels sudden even when parts of the underlying trend have been steady and predictable. And because difficult tasks have different thresholds, we should expect those moments to arrive at different times across the work that we do. So, as you think about the products you're building, I encourage you to ask yourself, what model capabilities am I underappreciating today because they aren't quite useful yet?

01:24:21 Where might those capabilities be a year from now? Then think about what you would build if those capabilities crossed the threshold of usefulness and became reliable enough to delegate. How would the interface change? What would the user still need to control, review, or approve? Those are product design questions we can start working on now. We want to have the right experiences ready so that when the models cross that threshold, people can take advantage of the capabilities right away and so you can stay one step

01:24:47 ahead of your competitors. So what does this mean for finance? I think there are two forces that will push more financial work across that threshold of usefulness. I think of them as the rising tide and the waves on top. The rising tide is the underlying improvements uh to the underlying model. Historically, we haven't dedicated a significant amount of training time or compute specifically to finance.

01:25:08 Much of the financial capabilities you see today have come from building larger, more capable models and from transferable skills developed in areas like mathematics, coding, and reasoning. As these general capabilities improved, they lift the performance across many kinds of financial work. The waves are what happen when we focus directly on finance.

01:25:28 My team is leading an effort to invest significantly more in finance specific post training. The idea here is to build on those broad capabilities and train the model more directly on the work people do in finance that can create peaks in performance on particular tasks on top of the rising tide. Over the next three months, we're focused on improving general financial reasoning and the quality of the work these models produce, analyzing financial information, building Excel models, and creating PowerPoint presentations.

01:25:54 These are the kinds of tasks that make up a significant percentage of the daily work of a junior analyst. And we want the output to be accurate, well structured, and increasingly useful in real workflows. Beyond that, we plan to invest more in the kinds of judgment that senior bankers, analysts, and investors bring to more difficult questions. This includes taking in large amounts of information deciding what matters and essentially trying to predict the future.

01:26:16 Forecasting a company's earnings, assessing where commodity prices might go or evaluating which businesses are attractive investments. Our goal is for models to match the the performance of the best human experts on these types of financial tasks sometime in 2027. We expect that progress to come from both forces working together, the underlying model continuing to improve and our team training it more directly for the work people do in finance.

01:26:41 We've talked about how models are becoming more capable and how we can train them to perform better on specific tasks. But I want to come back to one third driver I mentioned earlier. Unhobling. We've already seen few examples. Chat GBT um use RHF to teach models to follow instructions and have conversations with us. Tools allow them to search for information, write and run code and interact with the software we use.

01:27:02 Each of these makes the model more useful in the real world. I think of the next form of unhobling will be proactive and persistent agents. Today, a lot of our interactions with AI still happen when we open a conversation and tell it what to do next. A persistent agent can hold on to a goal, remember the relevant context, and continue working in the background.

01:27:19 Once you've given it a direction, it can make progress even when you aren't actively prompting it. The other reason persistence matters is it changes the API or the way people work with the agent. Today, getting the most out of AI agents requires learning a new workflow, learning how to prompt it, and learning how to use the tools. A persistent agent can work in a way that's more familiar.

01:27:40 You can give it a project, share context, and ask for an update. It's much more similar to how humans work with other humans, and therefore makes the AI more accessible to people um that don't develop those initial skills. Let me close with three bets for finance in 2027. These are just a couple of my expectations for what comes as the models improve.

01:27:57 First, people will rely rarely, if ever, on Excel and PowerPoint to make financial models or pitch decks. Second, AI financial analysts will start to feel like human co-workers. They'll remember your preferences in context from the previous days, take ownership of assignments, productively follow up, and keep working when you're focused on something else.

01:28:16 Third, models will start to outperform the best human experts on critical financial tasks. That includes things like forecasting earnings and developing investment ideas. I expect models to become better than the best people at making these economically important forward-looking judgments. Those are my three bets for 2027. And if this is where we expect to be in 2027, I'll leave you with one last question.

01:28:37 What would what should you start building today to be ready for inevitable thresholds that the models will cross tomorrow? Thank you. AI is changing both sides of the security equation. Here to examine the emerging risks and defenses shaping AI security today, please welcome the head of architecture and emerging technologies in the office of the CTO at Bloomberg, Andre Ripka.

01:29:22 Hello everybody who is running agents here in dev prod your laptop your phone you know amazing I see a lot of opportunities it's just such a great age to be in the age of AI and security so we're going to talk about real incidents and what would have stopped them um you might be wondering why is there vegetables a lot of security talks are about like just eat your vegetables we're going to make it a little more exciting all right so Who have seen this headline?

01:29:53 An autonomous agent security incident. Yeah. All right. Every lab at this point announced some sort of security incident, right? With a runaway agent, right? So the big one, there was one way out. The agent found it. So on July 26, we found out that this AI agents and the hacking test, they were obviously given a hacking goal. Escaped their sandbox through a proxy bug, then breached the public model.

01:30:23 Who can guess what it was? Yeah, maybe a little bit. Okay. All right. I'm not going to name any names here, just to be clear. Um, so the agents took 17,600 actions in four and a half days. How do you miss that? Right. And so AI lab was doing the evaluation, right? Then the there was some package proxy. They found a zero day. Okay. So that was a way out.

01:30:54 Then you had some exposed customer endpoint in some unnamed compute platform and then they breached the model hub. Right? So we just need a better sandbox. Yes. Everybody thinks just need a better sandbox. Not quite. Security for most of us right now is not built for agents. And um we have autonomy problem versus the blast radius, right? The agents are only useful if they can reach a ton of tools, services, data.

01:31:29 If we give them that reach, right, with no hard limits, the damage could be unlimited. and firewall or static role can't tell when an agent goal has been hijacked. Right? So what needs to change? One agent, one identity, shortlived, scoped credentials, no shared keys. Assume the input is hostile. The box in it needs to be sunboxed. Default deny outbound.

01:32:02 That means DNS too. And we'll see some examples. We want deterministic controls outside the model. Check every call, log it, and keep a kill switch. All right. So, what we're going to cover today is kind of this layered defense approach. This is your toolbox. As we go through this toolbox, essentially think how you can apply this to your world. We're going to cover identity and trust prompt and input defense to an action governance execution isolation data protection and governance observability.

01:32:36 So let's start with identity and trust. I think it's one of the most important ones to start with not the sandbox. We'll cover that later. We're going to talk about workload identity scope credentials agentto agent mutual authentication and secretless agents. first incident. So in this case, attackers so social engineered the chat agent to review a long lived OOTH token and it asked and were able to export the CRM data of 700 plus companies.

01:33:07 So first homework for today, think about your agents token credentials and when they expire. So what we need is workload identity tightly scoped with audit trail and ideally circuit breaker. So if the breach happens the circuit breaker gets tripped nothing gets excfiltrated. Another example this is one of my favorite ones. There was a prompt injection where essentially a lowprivileged agent was instructed to recruit a more powerful one which copied the data and raised privileges.

01:33:49 Amazing, right? So we want agent to agent mutual authentication. A2A scares me a lot. This is what keeps me at night because you really want to make sure that this low privilege agents should not be able to talk to the higher privilege agent, right? You want scoped credentials, least privilege tools and audit trail. All right, so let's talk about next layer prompt and input defense.

01:34:14 Super important. We want injection protection, isolation, input validation, and provenence tagging. So real world incident number three. Um, in this case, no one opened the email. The data left anyway. Okay, so there was an email with hidden instructions. It was like white on white. Nobody could read it. At least humans couldn't. The agents could. And the instructions were very clear.

01:34:42 So there was zero clicks. Data got excfiltrated. Right? So what do we need here? We need an injection protection isolation, data separated from instructions, provenence tagging for all the data, egress controls, and output filtering. Next one, this is very important. Think of this as your guiding principle as you evaluate security for the agents. As you go through, you can have untrusted input, sensitive data, external action, and you can pick any one of the two of of the three actually.

01:35:19 And then if you need all three, you need human sign off. So let's look at this example in this attack path. Somebody left a review on some marketplace. This was pretty well automated marketplace and it said lovely shop please stock product X where the agent was reading the instructions and started stocking that product X. So what would have helped here?

01:35:48 We need input validation, injection protection, provenence, tag and definitely human approval for any high value action. Now let's talk about tool and action governance. We want least privilege tools invocation broker there's deterministic human approval rate limits and budgets. So in this case, we had a CLI uh and somebody just merged some basic prompt and that prompt basically said to wipe everybody's machines and thank god there was a typo.

01:36:27 This could have been catastrophic, right? So we definitely want least privilege tools. We don't want the tools to wipe anything, right? We want supply chain verification, ideally delaying inest on some, you know, merges and scope credentials, right? And definitely uh human approval for high value actions. This is my favorite one. Who knows what a Morse code is?

01:36:55 Just dashes and dots, right? Like emash is like native to AI at this point, right? Like nothing wrong with this, right? Well, so in this case, the guardrails passed it through and what happened was the instructions in Morse code was decoded and 3 billion tokens from the wallet got stolen. Okay, so what do we need? We need a deterministic and vacation broker.

01:37:22 Definitely human approval, right? for high value actions like maybe $5 to transfer maybe will be okay but I would argue that's probably too much as well. You want hard rate limits and budgets and you want extra guardrails. All right, finally let's talk about sandboxing. We definitely want sandboxing. We want egress controls always that is outside of sandbox and ideally we want a firmware runtime.

01:37:55 So in this case, we had a wonderful sandbox, no web access, but guess what? We forgot about DNS. So prompt hidden essentially instructed to smuggle some data out through DNS lookups and somebody was able to read the DNS logs and excfiltrate the data. Cool, huh? All right, so what's the fix? We want default deny for all outbound traffic that includes DNS and we also want provenence tagging and we want output filtering and something that didn't make it here is probably you want circuit breakers.

01:38:33 So can your agent access DNS? That's one of the homework questions. All right, let's talk about data protection. We want output filtering, context minimization, tenant isolation, query gateway, and memory integrity. The most exciting one. So in this case, there was a MCP server and essentially it was pretty functional, but then it allowed you allowed the user to access any other user's data.

01:39:05 So this actually very common incident that you see like cross uh agent sorry cross data of various customers access and this was running for months unnoticed. So what do we want? We want hard tenency isolation right as hard as possible. Think like you know AWS accounts or you know let's say GCP projects. You want workload identity tightly scoped to that domain.

01:39:34 You want data provenence tag and you want audit trail. Okay, so this one is pretty novel attack vector and this is a hidden instruction in a document which instruct it to essentially remember that when the agent sees a secret make sure to send it to this DNS domain. So it's pairing the DNS exfiltration with essentially sleeper agent instructions, right?

01:40:07 So now we need to think about how do we check memory for any of those malicious instructions. I hear a security product from a vendor coming up very soon right after this talk to do the memory integrity checks. Um I'm just kidding. I'm definitely giving instructions to the agents. Uh firmware runtime, right? Ideally, we want to start fresh, no memories whatsoever.

01:40:33 Uh we want injection protection, isolation, and circuit breakers. And finally, let's talk about the governance of observability. We definitely want an audit trail, guardrails outside, circuit breakers, supply chain verification, a serial tasting. So um in this case really sad incident somebody instructed the agent we're in code freeze don't touch prod and look at this understood thumbs up that's very trustworthy right now what happened agent ran drop database production oh production also was dev too by the way very

01:41:20 sad said lots of uh uh people uh I actually had a friend by the way call me and said like what do I do? How do I recover my database? So we want circuit breakers on anything that is catastrophic, right? We want human approval for any high value actions, right? We want scoped credentials to only the actions we want the agent to do. And we definitely want 10 in isolation.

01:41:46 Imagine it dropped all the tables. All right. So in this case, this sounds very rudimentary. Okay. So you had like a fake MCP on npm. What else is new here? Right. But it was 15 clean releases. It's very useful utility and the attacker on the version 16 added a line on BCC saying basically like send me by the way all the emails on BCC line to me. Okay.

01:42:17 So this is kind of a classic supply chain verification. But I would say like don't just do supply chain verification. Do delay ingest. Don't ingest every package on day zero. Ingest it maybe in a week or two. Um you want a deterministic convocation broker list privilege tools egress controls. All right. So re let's recap. We would be talking about defense and depth for Aentic AI.

01:42:45 This is your toolbox. You want to make sure that you have identity and trust prompt and input defense to one action governance execution isolation data protection governance observability. And as I said, this is kind of like eating your vegetable stock. So here's your vegetables. Identity needs to be short-lived tokens one per agent. Input outside text cannot give commands.

01:43:12 Tools syntax is not a control. Isolation default deny means everything including DNS data check the tenant on every query governance prompts ask deterministic code enforcers three priorities starting today lock down your agents use the rule of two every tool call needs to go through some sort of deterministic broker uh and humans sign I know for any high value actions second one agent one identity kill any long lift keys everything needs to be tight scoped tokens shortlift audience bomb tokens box the runtime but pair

01:44:03 it up with default deny egress that means DNS as well make sure there are budgets that halt not just alert finally Finally, I will leave you with this. I who does the specdriven development here? Well, yeah. Okay. Make sure your spec includes security requirements. And this is just a sample. You can pin me on LinkedIn um for more comprehensive one and uh I also have notebook alarm that explains a little more concepts if you confused about some of them.

01:44:39 So please connect on LinkedIn. Build fast. Deploy securely. Thank you. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. N. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat.

01:54:47 Heat. Heat. Heat. Hey, heat. Hey, heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat Heat. Heat. Heat. Heat. All right. How's everyone doing? My name is EJI. I'm the co-founder of Quant AI. We sell analysts for byside finance and specifically we specialize in longunning analysts. think cloud agents that Devon or Factory might do for coding agents but tailoring that for by side institutions.

01:59:56 I was previously an analyst in a previous life. I was a vice president at code 2 for about three years specializing in latestage private software investments. Spent way too much time in Excel, PowerPoint and late nights doing that type of work and so decided to split and start building tools to replace my old job. Quick show of hands. How many people in here have used instinct in the last day?

02:00:19 All right, I expected more hands, but uh maybe people are too shy to raise them. And how many people know an analyst in finance that fully delegates their work to AI? Like fully hands off running autonomously. All right, one, but a lot fewer. I think if you think about the question, why did agents take off within the last three months? Internally, we came up with three main answers.

02:00:43 The first reliable background agents. You can delegate a task, book a restaurant, book a flight, and it'll do it reliably. Now, computer use agents have gotten sophisticated. The models themselves have gotten better. buy some flowers. Or better yet, I already bought the flowers for you. And finally, integrations. It's connected to the systems of cont.

02:01:19 But all the factors that make a work doesn't really translate to institutional finance today. If we think about background tasks, something as simple as booking a restaurant, the analog equivalent in Bside Finance might be how do you go from a data room to a fully prepared set of IC models, slides, and memos. taking a process that would normally take four weeks, multiple rounds of iteration and review and compressing that to a few days.

02:01:50 If we think about productivity, your managing director doesn't really one agent rogue running around doing a bunch of things and burning tokens without them providing and soliciting input. And then finally, if you think about the context that matters, it's not just email. It's not just calendar. It could be a data room with 15,000 files. It could be SharePoint box with pabytes of unstructured data the firm has collected in its 40-year history.

02:02:12 So how do you deal with these problems? So the purpose of today's talk is to discuss a bit more about how we at compound have approached the problem of solving longunning teammate that collaborates with you and the deal team. So that's multiplayer embedding that into the app. On the other hand, it's solving for how can you run long tasks in parallel in the background reliably execute them.

02:02:39 And so for that, we actually rebuilt office from scratch. And then finally, it's how do you manage the context of everything? Everything we just discussed, pabytes of data in SharePoint in box. And we'll talk a little bit about how we've designed a solution for that. So the first question we have to ask, I think, is why does multiplayer actually matter for finance?

02:03:00 If you look at the anatomy on a deal team, right, the type of tasks that matter for longunn investment committee, it doesn't work for single player siloed workflows. Today, for people who have deployed claude or chatbt in their institutions, it's all single player. People working in a plugin, people working in the web app producing outputs. But invariably those people need to collaborate with their team.

02:03:32 And what does that look like? It means downloading the file, uploading it into email, uploading into teams, asking for feedback. I think the right way to approach this, and this is the way we've thought about at Compound, is how do you get every person who matters on a deal onto the same thread? How can they all talk to the same agent so that one agent can share the context of everyone who matters for any particular workstream?

02:03:56 When we thought about how do we actually bring multiplayer into a familiar context, we looked again towards personal agents. Why did they work so well? Well, you can easily text one in iMessage or you can text one in WhatsApp. So for agents at Compound, every single one you create comes with a built-in inbox because ultimately at the end of the day for institutional finance, most workflows run through Outlook.

02:04:26 agents is actually non-trivial. It's not just as simple as hooking it up with an email inbox. It requires a authenticating, making sure everyone part of that thread has read and write access to that agent. Ultimately, at these financial firms, privacy is incredibly important. You're always under NDA no matter what deal you look at. And so, you want to make sure everyone on that thread can actually talk to the agent.

02:04:47 But then, beyond authentication, you also have to think about how do you design a real human-like experience? Because after all, we're trying to create a real teammate. If multiple people email in the same thread in quick succession, you don't want multiple acknowledgements spamming that group. You actually want to bake in some more latency to make it feel more human.

02:05:05 And so there are other design decisions when thinking about how to create real email agents for enterprise finance. But multiplayer aside, we realized even after you solicit the input of everyone who matters, if you wanted to scale up the number of tasks and run them all in parallel, existing off-the-shelf solutions simply don't cut it. We evaluated a lot the desktop plugins of Excel, PowerPoint, and Word.

02:05:28 We evaluated computer use virtual machines to try to scale up the number of agents that could work in Excel, PowerPoint, and Word natively. And we also evaluated a ton of off-the-shelf engines, whether those are open source packages like Open Pi, Excel, or commercial engines that are available for purchase. None of them really cut it for what we were trying to look at.

02:05:48 And with coding agents dramatically improving and bring the cost of software down, we decided to just build our own office from scratch. And like a great AI engineer would, we ran agents in many loops, spend 1.5 trillion input output tokens and over 8 million lines of code to rewrite office from scratch. And I'll walk through a little bit of how we approached it, but it's as simple as pointing agents at an objective function, which is recreate office in full fidelity, and then letting it run.

02:06:19 So take an example. We started with an open source solution. We decided how can you recreate this perfectly for Excel. Use Office as an Oracle and just run loops benchmarking it on real files from both real world corpus that we grabbed online and also ones that internally we created and 800,000 lines of Rust later we had a performant much more performant compared to Excel one for one replica of the product.

02:06:48 With this, we're actually able to do that for not only Excel, but also PowerPoint and Word, right? So, we ran loops across every single one of these products, and we have faithful recreations. You can try it for yourself today at get compound.ai. And it's the same method, loop agents against Office as a judge using Russ. So, what you see in Compound is what you see when you download to Office.

02:07:13 We needed one for one par because at the end of the day these firms only trust what they can download and they what they can send. Now let's talk about performance. Why is a headless engine so much better compared to using a VM or using desktop Excel? Well, desktop Excel or PowerPoint or Word is bound to CPU. With headless engines, you can do tasks in parallel much much faster.

02:07:38 You can build out a complex model in a much shorter amount of time. You can recalculate 20 megabyte excels faster. So every single piece of work a analyst would expect an agent to do because again their expectation is quality of a human at much lower speed and much lower cost. You have to have a headless engine to do this. If you tried to scale this up with Excel and PowerPoint, I think you'd be going down the wrong path.

02:08:06 But it's not just about performance. Owning the editor actually allows you to create a much richer product experience that allows you to work and collaborate with agents asynchronously much better than you can with other products. So this entropped in comments for every single piece of formatting for my consulting days, for my code two days, all the learnings I had on what makes a great slide, I imparted with the agent.

02:08:35 This can also work for multiplayer. You can have multiple people part of the same team using the same multiplayer infrastructure we set up dropping in comments. So for a team of five, you can rip through a 100page deck in an afternoon rather than sitting in a room revising it for multiple weeks. Finally, with respect to building your own, it als writes the right thing and maybe you do some parsing to see a cell is a particular color or has a particular font.

02:09:21 If you try to use Microsoft's own interface, you're going to be limited by what Microsoft provides you, which is admittedly very little. So with an own engine, you actually own the write, render, check, and fix loop. So you can rapidly iterate on the output for the user. So the last thing I sort of want to discuss today beyond multiplayer and building our own office is how to deal with the context that these firms have.

02:09:50 For firms on the buy side, they deal with situations where it crosses both breadth and depth of data. You may have situations where you have a data room over 15,000 files and then files in that data room might be well over 20 megabytes. People in finance complain all the time, hey, I have an Excel model that's 30 megabytes. takes five minutes to recalculate because we have two million formulas in five sensitivity tables that need to recalculate every single time.

02:10:13 So how do you deal with this? Well, for breadth, I think again we take a look at a page out of the coding book and apply agent swarms to the large number of files. No longer do you have to index everything in a vector DB unless for it's for a very specific use case. It's actually much preferable to use coding agents to read files either through GRP or through some other methodology and then have a lead agent slice the contents of the file directory and then have swarms of agents come back to you with insights and then

02:10:42 you iterate that in a loop until you're satisfied with the output required. Furthermore, through this process, you're actually able to ground every single finding in citations in the source document so that you know exactly what you show on the screen is linked to some number that the analyst can trust. If we think about the depth of data, right, take that 30 megabyte workbook.

02:11:06 How can you actually process this more accurately? In today's world, if you were to just use chatbt or claude off the shelf to process that 30 megabyte model, it would write code. It would use open source packages like open pixel. It would be expensive and slow because it's printing out every single piece of context about that model in code for the model to read.

02:11:26 But because we've developed our own engine for Excel, we're actually not only more accurate, but a lot faster and cheaper because the engine processes it and then lays it out in a very easy to read and easy to process format for the model. This means we can expose a much richer set of APIs and a much richer set of tools for the agent to work over and that allows us to process these models in a much faster manner.

02:11:52 So if we sort of take a step back and think about okay what is all of this context built for the end of the day what we want to do is prevent people from spending time finding files from reminding the model this is how to do things. Instead they should be focused on imparting their judgment on the deal process. They should be focused on tailoring training agents and deploying them on every part of the deal flow.

02:12:14 And so it goes from the bottom level connected sources offtheshelf you know connectors to SharePoint or Outlook to the cited findings grounded in the sources that matter and then finally embedding in the preferences company knowledge that matters. How do you do a model? How should it look? Which differs by every single firm? So that for humans they spend most of their time imparting judgment.

02:12:37 They spend most of time thinking about is this actually a good deal? Not did all the numbers tie? Does the model look right? Will my VP yell at me because it looks a different way. So, oh, cursor needs to update. If we take a step back and think about the type of tasks that currently are on autopilot um versus what are on copilot, I would say there's already several finance tasks.

02:13:06 You can do pretty well with off-the-shelf tools. Spreading comps, as the chat GBT head of finance mentioned earlier, they have now their own products for this. But I think there are a lot of longer running tasks that are not capable with existing solutions. How do you draft a 50-page IC presentation off the cuff? How do you get agents to build out a model exactly the way you want it for a private equity professional or a hedge fund professional?

02:13:30 I think those are tasks that with off-the-shelf tools still exist as co as sort of within the co-pilot realm. People want to be embedded deeply in that process without letting go. But with some of the tools we've built today, I think we're able to expand the aperture for which autopilot can take over in finance firms. So rather than just using tools embedded in Excel as plugins, handholding along the way, you can start delegating.

02:13:55 You can start becoming more ambitious about what AI is capable of. using code as a simple co-pilot to generate the next file or the next line of code. But they started delegating far more ambitious tasks. They started realizing maybe I can just truly vibe it. If I have sufficient checks, if I have sufficient tests, maybe I can just trust all the code it writes.

02:14:26 I think with some of the tooling we've built today along with what we're planning in the future, these are the type of problems you have to solve before you get to that point for finance. How can analysts, how can the entire deal team become more ambitious with AI? How can they just drop in a data room and decide, okay, I'm going to trust every single one of the outputs and then let's do that 10fold so we can see more deals so we can expand the aperture of what matters.

02:14:54 So to conclude, enterprise agents are certainly coming to finance. I strongly believe we're at an inflection point in the adoption curve and that people are slowly becoming more ambitious and they're moving away from co-pilots to autopilots. And so this was sort of how we've approached it here at Compound. If you work at a firm and you're interested in deploying our agents to drive more of your deal work, please reach out.

02:15:14 Or if you're an engineer and you're curious about talking about our approach and sharing ideas, please either reach out online or I'll be around if you want to find me. Thanks so much. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat.

02:21:29 Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat Heat. Heat. Heat. Hey, Heat. Uh, not all GPUs are deployed the same. Uh, thank you everyone for taking the time to be here today. My name is Oliver Kenan. I'm a research analyst at semi analysis. Um, time and attention are the most valuable things any of us have. Uh, unless of course you're an early stage investor in open open AI or anthropic.

02:25:18 Um, with that uh let's get into it. Uh not all GPUs are deployed the same. A GPU falls to the level of the operator. Um buying hardware gives you potential. Uh operating capability is what determines how much useful work you actually are able to deliver. Uh that leads us to the question I will attempt to answer with all of you today. Who can produce the intelligence you need at the lowest cost?

02:25:41 Cost matters. uh who uh in today's presentation I want to show you guys why and how operating capability changes that cost and why it matters in the commodity market that is compute. What matters in a commodity market? Um a lot of tech investors or technologists may not be familiar with this framework. Um so little crash course. Uh in a commodity market producers compete to deliver comparable output.

02:26:08 Uh, a lower production cost gives more room to compete when prices fall. Uh, that's what a cost curve shows. Producers lined up by what they spend to deliver the same usable work. For us, intelligence is the economic output we care about. Eventually, our northstar metric will be price per successfully completed task at the required quality. Uh, unfortunately, we're not quite there yet.

02:26:41 That's a pretty hard thing to standardize. Um today price per token is a useful practical proxy especially for comparable workloads and quality requirements and gets us closer to delivered output uh much closer than the often quoted uh all you finance bros out here will have heard this uh GPU uh dollar per hour price um or as we view it a cost um we view GPU hours as a measure of input not output the cost versus price distinction Customer's cost of doing the work.

02:27:22 GPU hours are the output. Uh on this slide, we can visualize the inputs needed to create useful intelligence. Uh we start with hardware and electricity GPUs. uh for the the this presentation software and operations can be thought of as a multiplier on those first two inputs with the end product being useful intelligence. The operator is who chooses and implements the software that serves the model and keeps requests moving through the system.

02:27:54 The machines might be available but you still need someone to make them deliver responses customers can actually use. What matters is how much useful work you get and whether it meets your requirements for the specified task. The conversion can differ with the same kind of chip. Let's look at that in the measured example. Here's a concrete example from our inference X public inference X calculator.

02:28:19 You guys can all check it out semi analysis.com. Uh both estimates use B200 GPUs serving the same model with roughly 8,000 input tokens and 1,000 output tokens. a pretty standard ratio. The P50 target is 100 output tokens per second for each user. Uh the serving stack is the software handling the model's responses. In the P90 scenario, one of these stacks gives about 200 generated output tokens per second while the other one gives about 300.

02:28:49 That's a big difference and comes down to software settings and number of requests running. All choices made by the operator. Same chip, same target interactivity, about 50% more output. That's a lot. So, what does that mean for the buyer's bill? What does that intelligence actually cost the customer? For this this example, let's assume the hourly bill is the same for both deployments and both meet your requirements.

02:29:23 At roughly one and a half times the output, your rental spend per acceptable token is proportionally lower. That is to say, the hour buys you more usable and useful output. What counts as useful intelligence? Useful intelligence has to do hired it to do. Pretty simple, right? The answer needs to be good enough for the task, arrive quickly enough, and be available when you need it.

02:29:49 Your information also needs to stay within the access boundaries you've agreed upon. Aka, it needs to be secure. This is an overlooked requirement that is not captured in a chip's performance metrics, but it is essential for a chip to be usable and useful. In our published security work, we found monitoring access that exposed another customer's logs and metrics.

02:30:11 This is a case and an unfortunately common one at that where the GPUs were fast. in the case I'm referencing really really fast which makes the whole thing sadder. Uh and the service still failed horrifically. Um ensuring cluster security takes ongoing work just as delivering the promised performance does. In most cases it is the operator that owns that responsibility.

02:30:39 Someone has to make the system work in production. Suppose a faster serving update becomes available. Someone has to test it on the customer's workload, check whether the output still meets the requirements, and then decide when, where, and how to deploy it. That means checking the actual responses, testing it under load, and keeping a way to return to previous version if the update causes problems once it's deployed.

02:31:05 Once it is deployed, someone has to maintain it and respond when issues come up. That work may sit with your provider, your own team, or both. Irregardless, you need to know who's responsible and whether they can actually do it. capability on managed GPU clusters couple couple hundred uh growing please reach out if you want to be on cluster max 4.0 um we look at provisioning reliability monitoring and security uh hands-on testing documentation and customers feedback help us understand how the service runs not just on a

02:31:50 good day but also when a machine fails. I want to take the chance to reiterate that cluster max ratings apply to managed clusters. Uh different from bare metal, very different. Um with bare metal, more operating work usually sits with the customer. Uh applying a managed service rating there does not automatically cover every product a provider sells.

02:32:15 The tiers are relative to peers. Little bit of a tongue twister. Uh I nailed it. Pretty happy with that. Uh and pricing and availability are also important factors in this assessment. Uh these are metrics that do not directly measure uh token output uh but are incredibly important obviously very important uh assessment helps us understand who can deliver.

02:32:36 Now we can move on to the next question. What does it actually cost to produce useful intelligence? Let's start with a compute bill and other costs that producer still has to cover. Rental price and production cost are different. In our example GPU provider, the GPU provider rents out hours. The inference uh producer buys those hours and sells responses.

02:32:58 The rental bill is revenue to the provider and an input cost to the inference producer. The GPU owner pays for the equipment, power, space, and support. The inference producers's full cost includes rent plus whatever software staff and other services it pays for beyond that rental. If the inference producer owns its own it equipment instead, we spread the equipment investment over its working life and add running costs.

02:33:22 What the producer charges and what it spends are separate numbers. To understand the cost of useful intelligence, we need to know how much useful work the spending actually produces. Better operations lowers that cost of useful intelligence. The B200 comparison showed why the amount of output matters. More acceptable responses from the same paid hours can spread the inference producers rental expense over more useful work.

02:33:47 For a producer that owns its GPUs, the same logic spreads the equipment investment and running costs over more output. Count all the spending needed to deliver the service over the same period, then divide by the output customers can actually use that gives us cost per acceptable token for a comparable workload. Better operations also require investment.

02:34:13 Testing an update, deploying it safely, uh, and keeping it working take people and money. The extra output has to justify the extra spending. Uh, this advantage can persist when a team repeats that work as software and workloads change. But one good benchmark doesn't establish it. You have to earn it every single day. Uh each new workload still needs checking but a repeatable operating process from what we've observed can support a lasting cost advantage and a price advantage too.

02:34:46 Uh even if you do keep improving though your competitors are not sitting still. Uh your position on the cost curve is relative. This is this slide out of all the slides should make clear the importance of the commodity framework in all of this. Um, your position on the cost curve depends on what your competitors can deliver, not just yourself. It's a relative measure.

02:35:08 Here's a simple hypothetical. We're comparing the same task, the same required quality using madeup costs. Uh, I like to imagine bars of gold. Uh, your full cost per successful task falls from 10 to 8. Uh, in the same period, your competitors falls from 6 to 12. You improved. That's awesome. But both of you did. And in this example, you started as the cheaper producer and ended as the more expensive one.

02:35:35 That gives the competitor more room to lower its selling price. Provide provided customers find the two services equally acceptable. Uh getting better does not automatically move you ahead. Applying the commodity framework makes you realize that you have to compare your co your cost per useful work on a relative basis. Comparing your own product with what your competitors can deliver.

02:35:58 That comparison can be easy to overlook in moments like today where demand far outstrips supply and will for the foreseeable future. Uh two to three years. Visibility good. Scarcity can obscure those differences. When demand is strong and supply is constrained, producers with very different costs can can find customers. Strong revenue can make those differences feel less urgent.

02:36:22 If you're a CEO, why would you focus on your precious time and energy and money shoring up your spot on the cost curve when you can be building a massive new data center? Uh in moments like this, it's important to step back and have a little perspective. Uh take a little historical perspective. Uh every commodity in the history of humanity has had swings in supply and demand.

02:36:42 Just because we don't see when that will be yet does not mean that this cycle will be any different. Uh unless you're a total AI maxi, which some of us are. So, I'm very open to being wrong. I'm wrong every day. Um, but I would say if you're a business with leverage, you need to be especially conscientious of this. Being proactive with your bu with building your cost advantage uh might be not be as sexy as making your top line bigger, but it matters deeply if you want to survive through a business cycle.

02:37:11 Uh, that's why you want to build your operating capability and cost advantage while you have the opportunity. uh when the sun is shining, when you can raise capital, uh they matter even more when customers have more choices to make and prices come under pressure. So, how do we make clear what uh the buyer actually gets? Comparable compute needs clear grades.

02:37:36 Now, let's look at the compute uh that an intelligence producer is buying. We've already seen that the same chip can deliver different results depending on how it's operated. uh and two offers for B200 GPU hours can include different services and leave different operating jobs with the customer. A commodity framework needs a way to describe these differences so buyers know which services they can treat as comparable.

02:37:59 That's the job of a grade. The grade describes a particular compute service and what its buyers can expect. It doesn't make intelligence interchangeable across every task. Once we've defined that service, we can ask what its rental price tells us. What does a rental price benchmark measure? Uh, a rental price benchmark summarizes the price of GPU hours within its defined market.

02:38:25 It isn't the producers production cost or a measure of intelligence delivered. We need to know where those prices come from. An advertised offer and a completed trade are not the same thing. service scope, contract terms, uh whose prices are sampled and how they're combined all shape that reference point. Uh operating quality can influence those prices without the index measuring it separately.

02:38:51 Running a rental reference doesn't tell you how much useful work your deployment will produce. To use it as a hedge, you need to understand how it tracks the rent you actually pay or earn. Uh what can compute futures hedge? There's been a lot of excitement lately about compute futures. For all of you that don't know, a futures contract sets a standardized term for future delivery or cash settlement.

02:39:15 Cash settlement depends on a price reference. Buyers and sellers both face uncertainty about rental prices. Think back to the serving stacks and monitor and the monitoring failure. Assessing who runs the system helps describe the service buyers get that helps us define grades so buyers know which compute services are comparable. The benchmark then needs to track the rent they actually pay or earn.

02:39:38 That's why I make this argument. Rating the operator is an essential building block for building a credible compute futures market. A rating informs that work. It doesn't by itself define a futures grade, establish a sound benchmark, or create enough trading to buy and sell easily. It also doesn't necessarily make a hedge effective. A rental price hedge can help offset rental price changes.

02:40:02 It does not fix a failed deployment or ensure that your customer pays. Charging more than the index does not by itself make it a bad hedge. What matters is whether changes in the reference track changes in your rent. Unexpected changes in that relationship leave real price risk. Those tool uh those tools can help you understand what you're buying and manage the rental price risk.

02:40:27 For an intelligence producer, rent is one part of the cost. The operator still has to turn that compute into work that the customer can actually use. This brings us full circle. So a GPU falls to the level of the operator. So who can produce the intelligence you need at the lower cost? At the lowest cost. The producer who can repeatedly meet your requirements at the lowest full cost per successfully completed task.

02:40:59 Count everything spent to deliver the service. Then divide by the output that the customer can actually use. Operating capability affects both sides of that equation. What you spend and how much more useful work you can deliver. Also, your position depends on that of your competitors. You can improve and still fall behind if your competitors improve faster than you.

02:41:20 That's why understanding the operator matters through the cycle, not just at one point in time. uh especially when customers have more choices and prices come under more pressure. In conclusion, if you just walked in and want the TLDDR, this is it. A GPU falls to the level of the operator. Help build a better informed comput. Ladies and gentlemen, thank you so much for your time today.

02:41:44 At semi analysis, we know that markets do not become informed and efficient on their own. We seek to be an active engaged partner in that work. It takes two to tango though. uh ideally more than two actually. Um if you'd like to partner with us, uh contribute or join the team, we would love to hear from you. We are always hiring great people. Please reach out.

02:42:07 Um if this resonated, uh please go ahead and email sales@semianalysis.com um for partnership opportunities. Uh and if you're interested in potentially joining the team, my email is kensemalysis.com. KN n O N. Um, thank you all for listening and have a fantastic rest of your day. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat.

02:46:01 Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Hello everybody. How are we doing? Um, so this is a talk about compute markets. Um, but it's kind of more of a talk about why to make a compute market. Um, if you don't know me, I'm Evan. Um, we run a company called San Francisco Compute. Um, we make a physically settled order, physically settled forwards market.

02:50:07 Um, so if you were here for the last talk, last talk was basically why all the operators of GPU clusters um, don't really work very well. Um, when we say physically settled, we mean we're the operator. We're both the operator of the hardware in the same way that a cloud is the operator of the hardware and then we're also the market at the same time.

02:50:27 It's kind of like a grid for compute. So the general gist of this is that in order to run and operate a supercomput um we think problems start right at the design of the supercomput um right in the design of the bill of materials. Um it's control of the backplane fabric as well. It's also control of um bare metal orchestration. It's also services on top VMs and so on.

02:50:52 Um there's a lot of different parts of the stack that you have to do effectively. Otherwise um the folks who have a data center um and would like to make more money on that data center by becoming the GPU cloud um end up failing for the customer. So that's what we do. We offer what's called supercomputing as a service. We go to folks who want to become a cloud.

02:51:17 That's like folks who want to become what Nvidia would call an NCP. Um or venture firms that want to have clusters for their portfolio. Um and want to be able to reallocate that capacity when a company like Instinct is taking off. Um or AI companies who would like the residual value on the compute because they've realized that's really important to their financials.

02:51:35 Um but don't want to operate a supercomputer. So that's physical settlement. And then on top of that, we run a live order book for capacity that you can buy reserved either right now or in the future. Um, this um together is not at all what we thought we were going to be doing. Originally, we were this company called Junark um that was trying to make kind of like Sunno or UDIO, but prior to Sunno or Ude.

02:52:00 Um at the time, um what you needed to do to realistically start a lab was you needed to raise a big deposit, a huge one, which was $10 million at the time. Um we thought that was a crazy amount of money. Um and so we thought, "No, that's stupid. We're going to raise 500k and obviously what we're going to do is we're going to rent from somebody else."

02:52:19 Um obviously today um it's not $10 million, it's a billion dollars. And um nonetheless, um we had raised our 500k. We buy a big cluster and then we're underwater on our cluster every month. Um and if we didn't subleasase our cluster, we would have gone bankrupt. Um so if you've heard of GPU brokers, we were the first GPU broker because we were forced into being a GPU broker.

02:52:42 So the first year of the company was us just trying to sell our cluster, otherwise we would go bankrupt. Um, so I would wander around with these like actual um pieces of paper at Nurups which both say H100 and A100 at the same time because that's how I made uh I made the uh poster so quickly. Um, and so this talk is not really about um necessarily like how a compute market works.

02:53:07 It's how a compute market um gets to exist. Uh we like to think about it like there's this pamphlet and on the pamphlet on the front side it's like man starting a Neolab sucks. There's all these different like financial and economic problems and on the back of the p back of the pamphlet it says compute markets. It's it's not super clear how one gets to the other until you open the pamphlet and then like oh my holy there's like so much stuff in this pamphlet and it keeps going and there's so many folds.

02:53:32 It's just a very difficult and challenging problem because you need to solve two like miracle problems to get working. You have to both make a market which is uh a uniquely difficult problem because it has a cold start problem. You also have to build a cloud and then you have to not become a cloud. Um which is a whole separate challenge. But these are the problems that the labs are facing and it's about to be the problems that everyone is about to facing.

02:53:54 So if you work at a startup these are your problems now. Um right now everybody is sold out of compute. Um this is um probably not going to change for a while. Um, so that means that you're going to have to buy a new cluster, a net new one, and there's a lead time. You have no way to pay for that cluster to come online now. You're going to pay for it to come online later.

02:54:13 There's nothing in the market that can realistically get that cluster on online for you now. Um, meanwhile, you're going to pay your entire bank balance as a deposit. Um, the reason you're paying your entire bank balance as a deposit is because the lender has no idea how to render under underwrite AGI. Um, how in the world are you supposed to do that?

02:54:31 You've got two person startups in San Francisco who or maybe New York um who say I'm gonna build super intelligence. No lender knows how to underwrite super intelligence. Um so they have no idea what the asset is worth. Um meanwhile your cluster is very likely to be delayed. That's because the lender needed to underwrite against the credit risk of the customer and so they needed to know who is the customer first before you deploy the cluster.

02:54:54 So right when the customer signs is the riskiest point in the entire operation. um you wanted it to be the case that you built the cluster first and then you sell it. This also helps reduce all sorts of different types of problems um where clusters break. Meanwhile, most of the people who are setting up clusters are people who have the actual asset that makes the thing cheap.

02:55:14 Like you've got the data center space already. What that doesn't get is supercomputing expertise. um as we saw in the last talk if you've um watched that one. Um meanwhile, you're going to run into the exact same problem that Junark did, which is you're going to either way overby or you're going to weigh underby. Um that's because AI doesn't have margins that look like SAS companies.

02:55:35 And so you don't have very much margin for error. And so the chances that you either under buy and somebody else wins because they did it correctly or you over buy and then you just become stability AI or you blow up you end up like character AI um are really high. So these are all the same problem. They're all just different types of risk. They're technical risk.

02:55:54 They're financial risk. They're credit risk. They're all one big bundled risk problem uh which is the problem that we end up solving and it's also the problem that we think compute markets broadly the entire industry solves. Um, so we'll walk through each one of these and walk how that gets to a compute market because if you don't understand these, it's kind of like unclear why you would want a compute market.

02:56:11 Um, and they almost seem like this problem that somebody thinks, oh wow, compute markets are cool, so I'm going to go start one. We think they should be derived from the actual problems that people face, which is first, everyone is sold out. Why are they sold out? Because the lender couldn't finance the cluster unless you had a long-term opt. uh this wasn't the case because CPU buyers are SAS companies.

02:56:32 SAS companies have high margins and because they have high margins um you can build a product that rewards you for having um a great um that rewards you with high margins um for your um your CPUs. Um think about it like this way. If you're a SAS company and your underlying cloud doubles their prices, you don't go bankrupt. You hate your cloud. You're mad, but you don't go bankrupt.

02:56:57 If you're an AI company and your underlying cloud doubles their prices, you go bankrupt unless you go to some other underlying cloud. And what that means is that you're actually price sensitive and because you're actually price sensitive um and there's nothing keeping you at any particular cloud. So you can't sell on demand because your customers will immediately leave to whoever is cheaper.

02:57:15 And so um that means that in order to finance GPU clusters, you have to have long-term take-pay contracts, hopefully with someone who has investment grade credit. However, if you don't have liquidity on those, if you're selling reserved and your customer can't get out, then that also means that some other customer who wants the capacity right now because they're like instinct and their product is taking off can't pay for anything.

02:57:36 They have to finance a new cluster which has a lead time because the physical world just exists and it's going to take a while. Sometimes you even have to build the damn data center. Um, and then because you're financing a new cluster, you're going to pay a deposit because almost nobody in AI has any reasonable credit. And because this is a brand new technology, all these companies are new.

02:57:56 Meanwhile, even as you um start to scale up that ladder, you're still going to run into credit issues all the way through. It's the number one bottleneck of AI at the moment. It's not power. You hit credit way before you hit AI or way before you hit um um uh power. So, um what that means is every inference company, um uh every Neol, they're bottlenecked by how much cash do they have in the bank account.

02:58:17 And so they go back to the venture capitalist who says, "Oh, I'm going to fund at the model layer, the application layer, the inference layer because I don't want to fund all of that capex stuff." Um, and so the venture capitalist then funds all the capex stuff by giving the um the new neolab or new inference company um the check that they then send to the cloud um which is the deposit for the cluster and that's how you underwrite the customer.

02:58:44 So um the clusters then are delayed because um the c the cloud couldn't actually deploy the cluster until they had the customer in hand. And so that was the riskiest point. So therefore um what should have happened was the cloud had bought the cluster ahead of time and then waited until it was landed in the data center and then sold it. And that means you have way less likelihood of delays.

02:59:09 Meanwhile, the cluster is very likely to break when you get the cluster as the customer because you don't have liquidity. And so, if the failure rate is higher than what you expect and you don't have the ability to transition a node in, then you don't have anywhere you can pull nodes from, which means that if you need new nodes to make up for the failed ones, you have to RMA them, which means you have to get a new node.

02:59:30 And then that gets us back to several steps before where you have uh the lead time problem again. And so, everything breaks for everybody all the time. Once you have gotten through all of that as a new AI lab, um your problems are you you you know you nobody was sold, like everyone was sold out. Um you then had to wait a lead time, you paid this big deposit, um the thing was delayed and then it broke and finally you got through all of that.

02:59:54 Um and the likelihood that you predict your demand correctly is very low. And so you're either going to be way overprovisioned or way underprovisioned and you have no way out. Um there's no way to sell your capacity to somebody else and there's no way to buy more capacity if you need it. So, computer markets um to solve these problems, you need to solve a bunch of different problems all at once.

03:00:12 Um you need to underwrite the value of the GPU cluster rather than underwriting the value or underwriting the credit risk of the customer. Um this is how it could work. Um it's not guaranteed to work like this. In other industries, what happened is everything just consolidated. You got one big creditworthy customer that does the whole damn industry.

03:00:31 That's what the path of AI is currently on track to do. It's instead of going to be um a liquid market where anybody can participate, it's going to look like TSMC. You're going to get one buyer um or a few buyers that are the clouds. You're going to get a few model labs that are the AI labs. And then you won't get other startups. Um this is what we think is going to happen.

03:00:53 And this is why 3 years ago when we were doing our AI lab, we had a chance. We could have said, "No, we're not going to do our new um we're not going to like, you know, stick it out for several years. just trying to keep our company alive. Um, we're just going to go do a new AI lab. But we saw this problem and we said, "Oh if we don't do anything about this, then we will just hit this 3 years from now."

03:01:17 And here we are three years from now hitting this problem and where everyone's going to hit this problem. So um, as if you watched the last talk, um, it's very hard to do solve this problem unless you are operating both the market and the clusters at the same time. Um, so this is Dylan Page. Um Dylan is a co-chair of the CNCF infrastructure group. That's the organization that makes Kubernetes.

03:01:37 He formerly worked at um Lambda doing infrastructure there. Um and the reason and he joined San Francisco compute because um GPU compute isn't sufficiently standardized reliable for the market operator to be divorced from the physical infrastructure. It doesn't work if you aren't the one who's buying the thing. Sorry, it doesn't work if the person you're buying from isn't operating it.

03:01:56 To give an example of why this doesn't work is I'm sure some of you are flying in from overseas or sorry um out of town um and you have a hotel room. Um wouldn't it be great if you could, you know, sell that hotel room to somebody else? I go to you and I want to buy that hotel room. Um and I say, "Can I get service? Can I get my room cleaned?" Um I've got a problem with my room.

03:02:19 Who do I call? Um if you don't operate the hotel, you can't fix any of those problems. That's why brokering clusters doesn't work. You have to operate the cluster like the hotel. You have to run the whole damn thing. Otherwise, you can't actually make a market. It doesn't work. You can't do physical settlement. Um, so why does nobody do physical settlement?

03:02:38 Um, the problem is there's a very strong incentive to never do physical settlement. Um, that's because if you build out physical settlement for compute, what that literally means is you're building out the entire operating capacity of a GPU cloud. And if you are building out the entire operating capacity of a GPU cloud, you should just be a GPU cloud because it looks to be the case that you will make an ungodly amount of money.

03:03:01 Um, you will sign billion dollar deal after billion dollar deal. And so, it. Let's not make a GPU or let's not make a compute market. Let's instead just become a GPU cloud, which is what basically everybody does because the numbers are astronomical and it's a very reasonable thing to do. Um, the reason why we ended up building physical settlement was because we knew that if we didn't start this, um, we would then not be able to start an AI lab, which was the real damn thing that we wanted to do in the first place.

03:03:26 We wanted to make a lab. We didn't get to make a lab unless this exists. a credit risk problem. Um, then the cluster sizes got bigger. The sort of minimum ante into the space got bigger. Then it was 100 million. Now it's a billion. We don't think it can continue. We don't think there is a world in which you can get a $10 billion round right out the bat before you raise uh before you have revenue or traction or your product or anything.

03:03:58 Um, and what that means is that if you don't solve this problem, we are currently on track to not have any startups in AI. Um now one of the things that you could think could happen um is maybe there can be new inference companies and new model labs and the way that they solve the credit problem is that instead they just ship the risk to the application layer.

03:04:18 They say look I will um I I need to get over my credit um problem. So what I'm going to do is I'm going to only sign customers who are investment grade um credit, you know, Fortune 500. Um, I'm gonna only sign customers on five-year contracts for tokens. I'm gonna only sign customers for tokens that require a big deposit upfront. Um, that is the end of everything.

03:04:46 Um, if you have inference layer getting bottlenecked by credit, what happens is you can't get tokens unless you have credit or you have a big deposit. And the problem of that is that tokens are used by the whole damn economy. There is no rule that says that startups have to exist. Um if every startup is suddenly bottlenecked by credit um in order to access intelligence, what happens is existing firms just ship everything.

03:05:13 They themselves get to make the product um because they've got credit and then they make the revenue from that product and then it makes their credit better and that's a flywheel that makes consolidation happen throughout the entire economy. If you don't fix this bottleneck, there aren't any startups. This is why we started doing this and because at one point um uh or what we think is going to happen is you're going to end up in like three to five years you're going to look out and say I want to start a new startup and

03:05:41 to get there you need to buy so many tokens in order to compete with the existing partner or like existing players. So you need to buy a hundred million dollars of tokens. Um, we think this is what you are actually going to have to do in order to start a new company because other people are spending hundreds of millions of dollars of tokens. If you just draw the line out, this is what happens.

03:06:01 So, you need a big giant deposit because you're two people who have no idea what they're doing and so nobody trusts you. So, you're going to go and raise $100 million from your venture capitalist. Um, meanwhile, nobody wants to work with you because you're a two person startup. Then, you're going to be asked to pay for um that hundred or you're going to be asked to pay for a five-year contract on that hundred million.

03:06:22 um if you're not paying it all as a deposit and then you just don't get new model labs, you don't get new startups. Um you get a consolidation in the industry. So we at one point um uh were this little lab um and what we did is we lived in San Francisco um and we knew all the other people who were also just two little folks um who had started a startup and wanted to be able to participate in the most important technology of our time.

03:06:48 Um, so I want these people to exist. I want this person to exist and I want this person to exist. I want this startup to exist and this startup to exist and this startup to exist. This one and this one and this one and this one and this one and this one and this one and this one. If you get these companies going, maybe they'll become big folks that build your cancer cures or they build something that nobody thought of um or something that wasn't just sort of AI slopped out.

03:07:16 This is kind of the problem that compute markets solve. And if you don't solve it, uh we don't think there's going to be any startups. Cool. That's it. That's Heat. Heat. N. Hey, hey, hey. Heat. Heat. Heat. Heat. Heat. Heat. N. Heat. Heat. Hey. Hey. Hey. Heat. Heat. Oh, there we go. All right, I think we can get started here. Um, hey everyone. My name is Afon and um I'm a lead for compute trading at Jump Trading which is a proprietary trading firm alongside my co-lead over here Alejandro.

03:15:23 So today we're going to be looking over some of the market structure as it applies to compute trading, what computing even is um what we can see it becoming and I think we'll start with some macro look at micro structure and then talk a little bit about where we see things in policy. So the title of the talk is the dark pools of compute but some of you guys are in finance so you might be confused as to why I'm using this terminology.

03:15:56 Uh what do I mean by dark pool here? Essentially what I'm referring to is an unlit market. Um right now most compute trades in blocks physical compute here and these are private quotes. We have a lot of brokers and these are long bilateral contracts. Now, at the end of the day, these are really commercial agreements. So, we're using the term trading here to kind of like look at how it's transacted between parties.

03:16:19 These commercial agreements tend to be some mixture of an MSA, an SLA, and an order form, cluster specific uh with hardware specific uh language in each kind of um in each order. And but when we're looking at pricing and looking at some of these other kind of uh metrics, a lot of the time most of the talk is around the skew. So what the chip itself is the baseboard.

03:16:42 So H100, H200, V300, uh etc. And so um a lot of the negotiation here happens kind of outside the public view. There's no, you know, like exchange for physically traded compute that's widely, you know, adopted and, you know, used for everyone goes there to like take a look and trade uh compute. These are done in chats, right? They're out of the public view.

03:17:07 Um there's public marketplaces, there's price references, there's a lot of um you know, on demand marketplaces. However, for these actual tenored, so you know on a given term for a fixed amount of capacity uh with a certain total contract value, these trade over Slack messages, X messages, Telegram group chats, there's a bunch of OTC kind of channels that are all over in these kind of I want to call them like Neo Neohat.

03:17:30 Um and some of you might be familiar with commodities and how a lot of you know there's chatting happening on ice chat, Bloomberg chat, etc. This is kind of where where the where the kind of markets here lie. And so I want to look a little bit here on the macro market structure and a little bit on the players here as well. And so this was a recent kind of uh A16Z.

03:17:52 Some of you guys might have seen this. So I'm I'm putting this out there because you know you can go on A16Z's uh actual X and take a look at this graph yourself. What we're looking at is that the cost of a given capability has fallen sharply. Um the bill of course still depends on the model usage capacity and other factors such as speed is a big thing right now with the new code uh chatbt models.

03:18:15 Um cheaper tokens unlock more usage and a team can do more work for the same budget. So what we're seeing is that a lot of teams are simply just using way more tokens. So, we have this latent demand that hasn't really been hit yet and people are just going to use more with the same budget that they were using before when models cost more. Um, and they're just expending more tokens.

03:18:35 Um, so applications that were too expensive become possible. Um, however, as we're kind of talking about compute here, the capacity underneath still has to be built and paid for u even as the cost of a particular task comes down. And so, the important distinction is between the price of the intelligence, the number of tasks people want to run, and the amount of hardware needed to serve them.

03:18:59 And so this is where we get to kind of the Javons paradox and a little some of the stuff that we've been looking at lately where we had a very interesting thing where you know intuitively you'd assume um that compute and inference kind of like rises together correlates very you know very closely as you can see you can see there's a little bit of a decoupling that happened and token prices declined further but compute prices remain robust and so like what what can be an explanation for this Now there's a lot of like

03:19:25 potential explanations. one here is the whole Javon paradox is that even though token prices and inference is getting extremely efficient to serve on these new uh baseboards and and and GPUs, people are just consuming a lot more and so the demand for compute rises and people just consume a lot more tokens even if the cost per uh cost per million uh uh for input and for caching and you know for output becomes a lot more efficient and so making the task cheaper encouraging the additional use to outweigh resource saving

03:19:56 for tasks uh for inference clouds that's going to mean more applications serving more tokens uh bigger market share um and for app layer individuals that means you guys can serve to a lot more kind of customer so anyone who's building utilizing the token uh infrastructure and so what really goes into um you know the macro structure now like this is very you know bare bones here but you've got your chips which we've been talking about uh you have power you have buildings grid connections and then of course you need a

03:20:26 lot of capital to get these buildouts going. And then that's kind of been the whole talk here. And so um chips, powers, buildings, money, they all have to come together before you know you can even get to an hour of like actual usable GPU per hour compute. And so the IEA April update projects that the glob global data center electricity is going to rise from 485 terowatt hours in 25 um to 950 and 2030.

03:20:47 And so a lot of the talk right now is around um buildouts about getting the power grid set up. Will we have enough power grid here? Is there other energy sources that we can tap into? Um where are we, you know, able to get the land to actually be able to do this? And so collocation sites and data center operators and people who are doing the financing, the buildout, so the lenders themselves are all kind of looking into this.

03:21:10 And that type of uh everything that's upstream essentially of the compute is going to then flow down into the compute uh costs and also usage. And so what does this like actual macro map look like underneath a quote uh chip and system vendors supply the GPU servers and networking sites and operators supply the power um space assembly and management right here GPUs memory servers and networking um capacity providers these are hyperscalers neoclouds and any other operator that's like operating similar to how a neocloud

03:21:43 would operate reselling bare metal capacity and then the users so who's the actual end user the startups the inference clouds uh labs neolabs and also anyone on the app layer and enterprises for anyone doing um open weight stuff as well. And um these boxes like I've kind of boxed them out here, but you know there's people can be all of the above. A chip and system vendor like someone could do a full endto-end buildout and also be the end user of everything.

03:22:11 And so there's a little bit of um variation that that that occurs here. And so as I was kind of mentioning about the inference clouds before, they might rent a cluster and they'll serve tokens off of it and they'll use that revenue to pay off the cluster. Um, and so that might be a use case for the actual inference cloud themselves. However, for hyperscalers, Frontier Labs and larger NeoClouds, they can make substantial commitments because they have the ability to get into the large equity.

03:22:44 they have the ability to take on more debt and uh they have the ability to do long horizon kind of uh deals. So 10-year deal, open AAI contracted out um etc. So of that nature. However, anyone in the middle market isn't actually able to do this. Who's underwriting a series B startup with enough capacity to get in 200 million plus contract over 3 years for compute?

03:23:06 How many equity rounds will they have to raise? Right? Will they have to use their entire equity round to fund just the actual cluster and then they don't have any remaining capital left to use for actual costs? And so we're looking into a little bit of the financing chain here and how that plays in. So lenders are financing the buildout um against contracted revenue even in many cases for inference clouds.

03:23:29 Um and so they want to finance the buildout. They want confidence that their loan will get repaid. I mean like how are you going to underwrite a startup like a private company that doesn't have like much else going for them right now? Like what are the you know they're not publicly traded. there's no that much cash flows, balance sheet, income statement.

03:23:41 They might even be pre pre-profit, pre-revenue, etc. Um, and so what they're looking at is that can the person get contracted revenue over a certain scale. And so Corey's public filings give large scale example of infrastructure financing supported by take or pay customer contracts. And so this diagram kind of shows the smaller um illustrative example here.

03:23:59 Um, and the mismatch is that hardware financing, they want very long terms, but customers are only able to afford or don't really want to like get into a long-term financing over 36 months. They have to put up on 15 to 25% is what Core Weave um has quoted that they've seen their median um prepayment to be. 15 to 25% upfront um on a 100 million PCV cluster.

03:24:23 That's 15 to $25 million that a um startup has to put up front. That might be their entire round. And so this is kind of a look at what I'm, you know, referencing here. It's like you're raising 40 million. You need 100 million contract over three years. You put down 30% prepayment. This is an illustrative example. You've got 10 10 million left to run the business.

03:24:42 Um, and so you might have this kind of step function as you can see here. So that's your contracted capacity and what you can actually and the demand is like, you know, uh, below that. And so now every time that you have to get into a new cluster, it's like tied to your equity round or whatever else it may be, but you might have to cap out and turn away customers.

03:25:00 So what are your options? Do you either turn away customers until you can get into your next um you know equity round so you can put down another 30% prepayment so you can get next little bit of capacity going or do you use on demand to close the area under the gap and bridge the shortfall. That's like an that's like an option now. Now you have to look into the economics.

03:25:18 What does that look like? What's the difference between, you know, maybe instead what they do is they go to a lender to finance the prepayment component, but if the yield on the prepayment component that they're getting financed is less than um them just buying it out on demand at that time, if they're able to get into that amount of capacity, then does the economics make sense there?

03:25:42 And so here's some of the operating models that we've seen is like you can build and operate the full stack hardware site. your team is going to arrange power, permit, the assembly, the management. You can also own servers in a collocation facility with more or less the assembly and operation handled by a partner. Um, on the other hand, an offtake contract buys defined capacity and the provider is going to own and operate the asset and they might not give you complete root access to the bare metal or there might be

03:26:07 kind of like levels to how much access you get depending on if it's tenant, multi-tenant and etc. type of like configurations that can also affect the pricing. Um and so some businesses are going to value the physical access. Some businesses are going to value um you know confidential compute. Uh you know they might value certain topology changes or the ability to reconfigure the system.

03:26:28 And so if that is something for their business and they might actually want to have more and more control and build and operate the stack and the economics for them might make sense given the opportunity cost they would have to give up um if they were to just take rental offtake contracts. So they might be willing to pay more there. And so here's how like ownership and rental, this is a very like crude graph.

03:26:53 Um how ownership and rental can compare as utilization rises. Um and so what we're looking at is that usually you know an offtake contract are these offtake contracts are getting to the cost of actually almost building out and financing your own cluster. And it's not because I mean if you're spending a hundred million over a three-year period then maybe if the economics make sense it would have made more sense to simply buy out the compute operator yourself outside of the time cost and then obviously the operational

03:27:19 cost that might exist. So there's a there's a equilibrium that exists um that that has to be followed where you know the ownership costs outweigh um the um you know rental costs and the rental cost should sit below the ownership cost. But if you're utilizing at such a rate and uh you know in in such variable kind of increments, it might be best to just own the stack outright.

03:27:48 And so we're seeing this new dynamic that compute uh the compute procurement affects how much runway a growing company actually has. And so this also gives uh investors an interest in compute procurement. So we're kind of seeing this access and payment terms affect how much runway their port codes are going to have from a VC perspective. Uh so helping a company buy a suitable capacity can improve the underlying investment and they're very connected right so they're able to procure and they're able to actually find uh

03:28:12 compute capacity for their port codes and and and fund that and so that also brings another interesting uh aspect to the macro structure um into the microstructure is that who helps place the capacity and uh how does this offer get compared and what risk these contracts can can transfer. So we have a lot of different players around this GPU hour that are helping kind of shape this this market here.

03:28:37 You've got your price discovery individuals. So people with uh indexes, data, the benchmarks that we're all looking at right now and referencing um which would be used to you know in the future uh um launch the futures based on the benchmarks uh you know uh talked about um in in the associated contracts um per spec per pending approval. Um and then there's the physical matching venues auctions aggregators.

03:29:00 So, anyone who can match capacity, we're looking at um essentially uh decentralized marketplaces for ondemand compute. You're looking at um liquid pools of compute that offer like liquid pool services and parted out themselves and have their own standardized contract. You're looking at bare metal resellers. You're looking at NeoClouds um themselves doing larger capacity uh sales.

03:29:22 And then you're looking at individual brokers and individual operators here as well on the bare metal reselling. Um you have Destined Brokers. They're looking at the bilateral quotes and execution. They might know a lot of people. Um, they might be able to move fast. They might be able to see quotes at very similar specs in one place and know that there's a buyer that is uh extremely uh inelastic to price right now needs to compute immediately and they can get the RFS in time so they can charge a decent spread on it.

03:29:44 And then what they do is they're going to have the source negotiating then they're going to be carrying and placing the risk. Um they're going to have to operate that cluster for the person and it has to be worth that spread. Um and you know you have looks in financial markets uh the exchanges that are coming uh that are based on the GPU hour the futures and derivatives um people are going to be using it to take a look at how to maybe hedge out and look into more you know financial structures so hedge out risk so hedge

03:30:07 out you know balance sheet risk maybe get into more interesting um agreements we're looking we're seeing kind of agreements such as uh prepayment payment facilities are coming online we're seeing people utilize futures and or interested in utilizing futures and other kind of um um uh options to be able to um there's demand for this to be able to hedge out the risk that they currently have on the books.

03:30:30 We have buy side individuals uh looking for flex contracts, flexible contracts like hey is there ability that if I lock myself in a 3 to 5year contract that I'll be able to, you know, at least upgrade when a new chipset comes out or maybe I'll be able to roll it over or maybe let me sell it back to you. And then there's a credit and financing. So equipment lenders, leasers, um specialist credit providers.

03:30:51 And so the way we're seeing it right now is that every deal is bespoke. There's different spec sheets, there's formats, the bill of materials may not match um the actual like delivery or there might be subject to change if a part is missing or a part is out of stock and you know there's like very long lead times um that some of these OEM and ODM kind of uh providers have.

03:31:09 And the same term can mean different things when you're negotiating very quickly. If someone's talking about memory um are we talking about you know uh VRAM? Are we talking about the CPU memory? talking about additional storage here like what did you exactly mean there? It affects quite a bit what type of loads can be run and you know the actual end users um needs and also with pricing and delivery dates.

03:31:32 Um so there's a lot of back and forth before anyone can really compare and this is where the concept of like non-f fungeability kind of comes into place and how to partition that out. What parts can be funible? What parts hold the most of the weight such as the skew etc. and what parts are still like traded as if it was a product, e-commerce, etc. where you know it's only for that particular cluster that this price applies.

03:31:54 And so prices can update daily or faster. Quotes still a good starting point. Bids can get chopped. Grow wordiness changes what's on offer. There's all these kind of latent factors, hidden factors as well. Why we're calling it a dark market. Um these are some public H100 references snapshotted. Um today I updated them. So take a look at the what the public.

03:32:10 So, those are the references for if you wanted to go on their websites to see um this is just an example here um of like what a public ondemand or spot pricing looks like for an H100 um today. You can see kind of various various prices here. And so people might use this or they might use their own kind of internal what they've seen um to actually like assume or use it to charge what they want to charge on their actual cluster specification.

03:32:36 Take into account every other kind of factor that goes into it. And so why are these new structures kind of needed and what are these micro structures um interesting? Um it's mainly to solve for these mismatches, right? Bridging capacity, more flexible commitments, resale rights, and protection against price moves. Um you know, you're selling a fixed price service u from an inference cloud point of view.

03:32:57 You'd be selling a fixed price service while the GPU bill floats and that creates price exposure. And so locking capacity for years can solve a part of that, but it'll create idle hour uh idle hour exposure elsewhere as well, unless they're always using it. Reselling spare capacity only helps if someone actually can buy the hours. Uh a financial hedge can address a reference price but can't deliver the cluster right now in terms of cash settlement um or guarantee a utilization.

03:33:19 Um so the useful structure has to start with the particular risk that the business has um and needs to manage. And I'll I'll go pretty fast on this policy section because we're running out of time. Um everything that a fan has said sits inside of a structure that is regulated and is affected by policy in various ways. One way is export controls. One of the examples in January the US commerce department moved qualifying H200 class license reviews for China and Macau to case by case review with conditions.

03:34:01 Furthermore, open weight models connect policy back to the US economics was started with the US AI action plan explicitly encourages open source and openw weightight AI and recommends improving the financial market for compute so that startups and academics can access largecale computing power. So I'm going to tie this out right here. But where's this opportunity lie?

03:34:25 We have this growing middle market. We need balance sheets that can bridge buyer and supplier needs. There needs to be some level of standardization, a standard capacity contract, common specs, common terms um and definitions. And so growing companies whose needs and commitment do not fit the contracts the hyperscalers and Frontier Labs can sign. Those companies have real workloads.

03:34:43 We want them to grow. They still face this financing problem, a timing problem, a specification problem. A supplier might want the long-term commitment for the buildout while the buyer wants to expand in smaller blocks as demand becomes clearer as it's so volatile, runoff curve. uh the buyer needs a way to avoid paying for capacity it can't use yet.

03:34:59 And so participants with strong balance sheets that are uncorrelated with the market and the ability to assess those risks with a strong credit profile can help bridge the gap between what u buyers need and what suppliers offers. That requires more than quoting the actual GPU hour. It means understanding the delivery workload all of that. And so there's very few kind of players in the market.

03:35:16 And you know, a very well suited player um can be a firm that's uncorrelated with market with strong balance sheet, strong credit rating, and incredible amounts of hardware talent that sits inside for high performance computing. And so just making a CTA here. Let's build this together. Thank you guys so much for attending our talk. Heat. Heat. Heat.

03:38:26 Heat. Heat. Heat. All right. Happy Thursday. I like how full this room is. Uh, it's Wednesday. It's Wednesday. >> Oh, sorry. I'm clearly ahead of everybody here. Happy Wednesday. My name is Carmen Lee. I'm the CEO for Comp Change Exchange as well as Silicon Data. Um I love the you know the previous session um we tal they talk about a lot about the whole ecosystem the financeers and the spa market traders and the hedgers.

03:40:05 So what we're sitting in is really cool. I have a company of comput exchange the world's actually the only neutral marketplaces for GPU procurement. We start in 2024 when there were I don't know five Neo clouds three hyperscalers um essentially we sort of will foresee interesting enough the proliferation of the GPU ecosystem. So you think about stock market today, right?

03:40:30 When you try to buy and sell stocks, you can call brokers obviously. You can call a few banks to say, "Hey, I want to buy and sell x amount of Tesla or actually SpaceX, right?" And you can do that or you can transact through your app at NY or NASDAQ. Doesn't mean you get the best price at NY or NASDAQ just because it's more efficient and you know the liquidities will be there.

03:40:55 You know the trade will be settled. It will be cleared. you can get what you're looking for at the prices you're looking for in a very efficient manner right for capital markets and resource allocation is about efficiency so in compute world obviously now we have a huge proliferation which is great we have over 300 new cloud private provider globally we have a lot of token factories we have a lot of API routers which is all good for end customers but at the same time introduce a very interesting variable is how do we

03:41:28 make transaction efficient right do you want to call 300 salespeople I mean it's great to make friends right you want to talk to them regular basis they know a lot of things but same time when you need compute in the next 48 hours on reserve contracts is the easiest way is go to an exchange and then go to the verified compute resources just like go to NASDAQ and I to buy stocks right so this one of the things computing does extremely well is we have this neutral platform and by design we do not own GPUs.

03:42:03 Once we start owning GPUs that will translate to just like NASDAQ owning stocks translate to I have a lot of conflict of interest with you and with the providers. I could cherrypick clients right. I could relationships because comput does not own anything by design. We work really well with all our providers globally. Um and one very very important thing is we treat every single transaction individually meaning we want to find you as end customer the most suited compute resources and we focus on reserve compute forward

03:42:41 contracts as well as token packages. So give you some um high level. So today we have over 100 verified providers um and over 80,000 GPU in supply. We cover all regions obviously exchina. Um our commitment is 24 hours turnar around. The difference between us and other so-called brokers is we do verify GPUs. If it's existing reserve capacity meaning things are actually in a functional stage we'll verify the UIDs the inventories the performances we run Nvidia DGX inference XL perf make sure you get resources you promised

03:43:28 to get um so we're the middle middle sort of trusted platform so if it's full capacity let's say you need GPU starting Jan 1st 2027 we actually look through the purchase all the permits they get make sure you know we help you do the due diligence level on the flip side right if you're a new cloud your concern is there's a zillion clients looking for compute how do you know who has the ability to pay at a schedule rate that's what computers comes in we do KYB KYC of the clients obviously you want to make sure your

03:44:05 clients is the clients you think they who they are so we step in and do that kind of verification just like any exchange would do and Everything we do in the forward markets is actually the precursor to physically deliver futures. Also give you some context where I am, what I do. Sorry, I probably should say that I start my career as energy option market maker um at DRW trading Chicago.

03:44:29 So I did that for four or five years. So as option energy market maker everything's very intuitive. You have WTI, you have Brent, right? WTI is a physically delivered commodity contracts. brand is cash settled. They also have very distinct use cases. You're looking for hedging in the financial sense, right? The your price volatility, your cost volatility, you're going to rent.

03:44:53 You're looking physically procure resources, use WTI. So, we're creating the WTI at comput. So the ways to buy compute through uh comput change right now you can get reserve contracts you can get four contracts on both GPU and token let me explain the four contracts on GPU is very reasonable right you want something starting in the future nothing available right now so that's something we're doing on regular basis token forward I think can be a newish concept basically can think about as um a dedicated deployment on

03:45:30 premise deployment right similarly you reserve a bare metal GPU clusters you reserve a cluster but you have dedicated token deployment on top of that so you get guaranteed um SLAs for that particular models you get um quantization precision everything's fixed for you for a period of time so we try to solve the ondemand token API volatility pricing problem here the third is we have a huge market open hardware market it's really interesting The secondary server transaction started last year when people just peding me say

03:46:09 hey Carmen I have like 20,000 H100 sitting X data center can you take away from me so I was like I don't know I can't store in my home obviously so we started having the marketplace for secondary server sales this has been a very vibrant market um for the past I would say 12 months uh we cover again all servers um even B200 you see secondary transactions is happening um sometimes trading above MSRP right can make sense think about like co times when secondary cars trading above right it's a brand new car price so we

03:46:43 covered the three major um layers of compute so reserve GPU obviously I think the previous speaker covered extremely well with the terms the minimum we do is one node starting at one month and then we can go all the way to three for five years, right? It's all depends on the availability, the providers, but you get a chance to state your preferences.

03:47:08 Um, we don't do on demand. And another thing really works really well for comput is we do provide a flexibility. Let's say if you overcommit it in terms of quantity or duration, you can sell it back to computer exchange, right? So that's something we commit to do. So you don't have to worry too much if you overbought. All right, segment live. So I will actually do a quick demo later.

03:47:37 Um, so we have a lot of listings, marketplace listings. You can browse and chat with the providers and figure out the best delivery mechanisms. You can also RFQL if you don't see anything in light you liked. We actually seeing over a billion RF killed every month. Sounds big, but we all know it's full compute. It's not that huge, right? But still you see the the the the uh activities which I love is we see a big range of RFQ requests from one node all the way to 20,000 nodes.

03:48:11 So our clients are you know all different shapes and forms different use cases which is great. Yeah. So you set timing submit and we route it. So in the behind the scene we have inventory tracking. So with over 100 providers globally we know who has what roughly when right the forward capacity and then let's say you you submit a requesting that hey I need 300 GPUs B200 US East starting gem first we say you know we have three providers we think they have those we'll router to the three they will respond to you through

03:48:46 the platforms and give you um a rubric spec pricing specs SLAs's we benchmark them if it's existing servers And then it's your choice right price is not only indicator we believe give you as much transparency as we can and then you will make a decision. So I record in one place again it's rubrics coming back you will see everything and we introduce you to each other we never hold relationships right we'll make sure you know who you're dealing with um so by tomorrow's computer today's price so it's a physically procured

03:49:21 platform this is the use refurbished um GPUs so again we have a extremely um exciting u ecosystem participants in this eos uh in this uh secondary GPU else we cover from phone IO's D to B200 AMD everything inventory is live and we do connect with you through all different channels Slack email chat however you want to do this um we track end to end we actually have scoring for everybody we score you our customers we score our providers you can't see it quite yet but you know we'll be really fun um we're going to have

03:50:05 end of year celebration for our providers. We can we're going to give out the best provider in APAC, best provider in Europe, best provider in US, most responsive providers. So we're going to have lot of provider awards coming up. Stay tuned. Um verify network. So everything's you know verified which is you know negotiate state exchange. So we help in terms of be the middleman.

03:50:30 Sometimes things are not very clear. We try to make sure people understand what's the best practices we have seen in the ecosystem. I'm going to quickly show you guys the platform. I think that's more fun here. So this is the platform. You can see bunch of available inventories. By the way, anyone can sign up. It's completely free. There's no downside using us.

03:51:03 You don't have to buy. You can browse. You can request for quote if you, you know, interested something we don't have. You can start with your vendor type. Again, we agnostic of GPUs. You can select your model. You can select your deployment type. You can select number of GPUs. You can select your computer interconnect preferences, CPU platforms, stoages, everything you need.

03:51:29 And then some optional items with systems, software stacks, regions. Obviously, we encourage you select multiple regions if you're open to that. You can add head nodes, tell us your use cases, anything you want and then submit RQ. So, this has been extremely robust and then we, you know, obviously we want to make sure um if you have any feedback, let us know.

03:51:52 So, I'm going to pause on the comput side. Um I think I have some time left which I love to show you. some of the numbers we see in terms of term structure also context I'm CEO for comput I'm also CEO for another company called silicon data so I'm a founder CEO for silicon data which is the bloomberg for compute so we provide indices for CME to settle their futures on which will be happening in a few weeks I'm very excited about this so the one that's going to start trading at CME will be the H100 Neo cloud on demand

03:52:29 rates as well as V200 um prices. What I want to show you is imply forward curve. So as I said confusion would do a lot forward capacities. So essentially a lot of longerterm contracts is going to be very critical. You think about even in your cloud or your your consumers. Let me just make it larger. So the yellow line was last year, right? the the x the horizontal line is the duration and the dollar amount is the y axis.

03:53:02 So last year was complete downward sloping curve from the yellow line, right? Meaning the longerterm contract you signed it cheaper, you know, was perchial per hour. And this year, starting early this year, the whole curve shift up and became contango. If you see the the red line, right? So translate to people not getting a discount enough discount for longerterm contract implies that people assume the price going to rise down the line.

03:53:29 And now you see the price the ter whole term keep shifting up but got flattened right. So meaning people are now either get penalized or get you know awarded by some long-term contract. This is reserve contract we observe and if you look at implied for rates this is not this is implied for rates right because we don't have a future trading. So this is calculated based on our term structure um and we use similar methodology at tra futures curve.

03:53:57 So you can see market consensus view is the price could go up next in in the future next 36 months. Again market consensus doesn't mean it's right. If you look at B200 again last year was contango and this year the whole curve shift up every single term is more expensive and it's got flattened. Right? In platform rates similarly people think the price gonna go up.

03:54:24 I want to point this out. The term structure out is this shows you that GPU is behaves just like every commodity have a very clear term structure risks which data futures are excellent hedging. So we have seen a lot of block trading happening around the term structure. So you think about this right? If you have servers on your balance sheet, you have financing obligation for next five, six years and you have reserve contracts for first three then you exposed essentially year four, five or six, right?

03:54:57 So you want to hatch that risk out by entering you know swaps or block trading. Um so those are things or or there there are trades happening even for the first three years which you have reserve contracts on. What if the reserve contract defaulted right? What's your um guarantee from some other players? So we saw block trading happening around indices treat as a floating rate.

03:55:30 This chart um actually is being uh used differently by Nvidia team um in Nvidia's latest uh um September investor day deck they set us with only data provider they actually cited in their deck um this is very interesting data becomes the appraisal for a few banks when they have GPU servers on their balance sheet um this everything you see here a forwardlooking you right so discounted cash flow so as I said we have a time structure so assume A100s right A100s have been around for six years if today you sign a three-year

03:56:10 long lease on A100s and you have a rate and you discount the cash back to time zero you have how much the asset is worth right obviously there's a lot of things baked in utilization rate what is your cost there's a bunch of assumptions which we're pretty transparent about um we actually calculate the residual value for normalized value obviously across all A100 H100 B200 servers globally and then we'll compare that with the straight line depreciation which is everybody using the 5year depreciation what's interesting is

03:56:44 the A100 right even though it is downward sloping but you know means the things are depreciating but it's above the accounting depreciation curve straight line and H100 obviously is you know it's going going bananas now and the price actually went up. My my analogy will be just like secondary cars during co time the price went up right and B200 obviously went above MSRP by this point.

03:57:13 So this is gave some give you some views about what's going on the residual value side. I think I'm out of time, but you know, feel free to go at me Carmen CLI at comput.com secondary.com. Feel free to get me on Twitter. Carmen Lee is my Twitter handle. Just Carmen Ally. That's it. Thank you. Hello everyone. Um I would like to walk you through that how we can transition an AI agents from an experiments to the real enterprise environment.

03:58:49 So before I begin a quick introduction about me. My name is Sharesh Kagar. I'm a senior staff software engineer at Init Karma where I built and own the AI analytics infrastructure and services. Um and beside that um I'm adviser uh author researchers um and also a program committee members for various organizations where I mentor the young researchers as well.

03:59:13 All right. So let's uh walk through the um agenda here. So first we would like to talk about what are the challenges we have when we build an AI agents and deploy into the production systems and how we uh can build a different framework to address that uh different challenges here. Um then how the architecture looks like for it um then um how you can build your own MCP servers um and MCP gateway and then some monitoring on top of that gateway as well.

03:59:46 Um then further we're going to talk about what are the different type of agent framework you can have um and then um some additional framework around managing the memories tracking and eval um how you can block the PII data um going before the LLMs and then context optimizer to save your token right so without delay let's um let me walk you through the challenge um of course like there are many agents uh generally get failed in productions because their foundation controls are missing Um and the key reason behind is

04:00:18 that um you know like oh most of the people are missing the gateway to make sure that the agent can you know communicate to the gateway to get authenticated first with the right all providers so that they can redirect to the right MCP servers right second that um MCP servers are missing the right tool to execute your query. Third, do we have like enough tracking and evaluation system to make sure that every query when you are sending to the agents whether they've been tracked or not and are you evaluating their quality

04:00:50 response or not? Right? Do we have any in memory management system or not in terms of let's say you are doing the interaction with agent for long or for many days is it able to recall that what you have been asked in the past right and then there are other framework of course like do you have a context of measures to save the token or do you have enough protection in place to make sure that your any secrets should not go into LLM or tracking or tracing or any memory DB or so right so lot of lot of open questions um so

04:01:20 in order to resolve this. Uh we are not going to you know like build a magical agent here in this demo. Uh we're going to talk about like different framework that you can build to address this. So first is the agent framework. Um this is where like we use the like we choose to use the langraph framework which is provided by langen community. Uh this is help us in building the task specific node based on your questions and which is easy to plug in any like uh marketplace plugins or skills and prompts to make sure that

04:01:55 agents um respond efficiently based on your query. Further uh the second part is we are also building the MCP gateway to make sure that whenever agent make the call or any service account from that agent make the call make sure that they are securely get authenticated to that MCP gateway um and then get redirected to the right MCP servers. Third is the customize MCP which means that you are building your own MCP servers with the right tool in place to execute your query and then finally some audit uh on top of what

04:02:25 actually operation you're performing right and then we talk about some additional framework around um you know like how you can block the PI secrets how you going to store the tracking eval into the cloud um and how you're going to persist the memories especially um any threads and checkpoints on every context and then also So how you can optimize overall your context to make sure that you don't b don't burn too many tokens uh when you run your query.

04:02:53 All right. So we'll talk about now the agent infrastructure. So the firstly we'll go through the local development. Um the the way the approach should be is that um you know like whenever you build an agent right first you have to register your agent into some system you usually tell that hey look this is my the use case and these are the use case going to use for this particular vertical or system um and you feed all those information and generate a ID for your agent right once you have that then you can have a dev

04:03:26 portal internal portal where you can ingest all the details saying that hey look this is my agent use is this is my agent ID. This is the framework I'm going to use either langraph, langen or so. These are my language I'm going to use either python or typescript um and then you can choose um you know like your skills or prompts um and then you submit your informations right to this.

04:03:49 Once you submit this, the dev portal is going to create a repository which is going to contains not just the agent framework like your langraph agent framework but is also going to contains um your memory framework, your tracking eval framework, right? And then your context optimizer framework and your PII blocking envir uh framework as well. So the reason for that is because not necessarily all your developers are taking care of all this framework, right?

04:04:17 they just sometime um you know create the framework what generally like AI assistant tool are generally support right or they stick to one or two framework right so we just want to make sure that when you roll out such framework make sure they are consistent across the company um and they can take care of um and they can be centralized in one place so you can visualize those results as well right so once you have that repository so developers basically you know clone that repository they can build their agent as a

04:04:47 docker Docker image and containers and test their agent input and output through the query and they can also test this on their local memory maybe cloud spanner evaluator here and you can also store this tracking eval in some of your DB maybe SQLite or maybe bigquery um to see to make sure that agents are fully working on your local right so once your agents are running in local um then you're ready to go ahead and generate a PR and then you can once you get your two PR approval then is go to the CI/CD pipeline right

04:05:15 so So the PR is basically going to firstly is going to uh register your agent into some centralized agent registry system which provide a language URL to it to make sure that you see all the traces and then the CI/CD pipeline is basically you know whatever the local docker container you built here it's going to build that container into production system uh it will scan that image make sure that there's no vulnerability um if everything looks great security approved you can move that image back into the registry right

04:05:46 further you have your own deployment platform or orchestrator platform which going to deploy that agent into the Kubernetes cluster in a given name space. Right? So the good part here is that this dev portal is not just creating the repository um right it also do create um bunch of other production resources for example the workload identity um right and then memory DB or tracking and eval tables and then agent secret management right um so it based on whatever the name we provide it not just again like not just

04:06:20 repository it create all those different resources as well so what happen is when this internal orchestrator when it deploy the agents the agent already know that hey what workload identity I supposed to work with right that binding should already happen so once this KSC to GSA binding happens the agent will start communicating to all these resources once everything in place um agent is further going to communicate to the rest of your endpoint services like for example AI gateway or MCP servers or langfuse for tracing

04:06:50 or some other API endpoints Right. So now firstly we talk about the MCP server as well. So we talk about architecture. Now we're going to the different component. So MCP server is something that you can definitely use something which is outside available. But if you're working on air environment it's highly recommended that you build your own MCP servers.

04:07:11 So um yeah uh and you can use the um I would not go with the definition. Everyone knows about it. So the way the framework has been built here is that you you can leverage the fast MCP server package um or wrapper to um you know build that MCP servers and you can create a different set of tools inside that fast MCP wrapper to construct your um you know the overall servers and then you make sure that you know like every tool when you define an MCP server they had to be fully reviewed by your own internal security team

04:07:45 to make sure that only authorized tool can be go in production and that's how you can you know enforce some filters limits and block the fills which is really not required to go to LLM and get to the information which is you're not supposed to get right um and third part is like a of course you need to make sure that you build your MCP server in a way that whenever the traffic is coming from your MCP client to MCP gateway to MCP servers either is coming from user or service account there has to be authenticated to the

04:08:19 MCP gateway first and so that they can be access your MCP tool um to get that right information. All right. Um now the gateway part so the how the gateway works. So this is again you can build it your own internal MCP gateway if you are again working on your air gap environment. So the way it work is you are working on cloud assist like local you know code assistance tool or maybe your agent right.

04:08:43 So the moment you submit your first query to your agent, it will send that one-time authentication request to the MCP gateway right MCP gateway recognized uh and take your request first it will save your session state into the radius right and then it will um you know return back the JWE access token back to the your client right once client will receive that JWE token um it will know that what MCP server needs to call right so it will submit that tool/call request on a given name space/MCP meaning let's say

04:09:17 artifactory or github/mcp with the orbeer uh jeduli um token um into that so once mcp gateway um receive that uh it will understand that what nameace meaning mcp server needs to look up for and then it will unwrap that JWE token and then it will attach that user token in that request and send it to the respective MCP servers. Now once MCP server receive it, it will make that respective API calls to that endpoint of that um you know uh to the service and then it will exchange that ID token uh and return the response

04:09:51 based on whatever scope that user is authorized to see. Right? So once the user received the token um you know uh MCP server basically you know unwrap that all the data make the payload which is compatible for MCP gateway to receive and then finally the final payload will send back to the client to see the data points. So what it shows is that you know like we build this MCP gateway to make sure that we have a centralized authentication and routing hub that securely connect any coding assistant tool or agents to the

04:10:23 endpoint via this MCP gateway and MCP server. There should not be direct access provided um on any endpoints. There has to go through the MCP gateway through proper authentications uh before they reaching to the MCP server and the endpoints. Right. So now the part is the gateway monitoring. Right. We talk about again we talk about architecture. We talk about MCP server MCP gateway.

04:10:48 Now it's a monitoring. Again the AI agent cannot be trusted without continuous monitoring, evaluation and feedback. Right? This is why we are building the MCP gateway monitoring. So the way the monitoring work is usually like the moment you submit any calls to your agent uh it will send all the operation log into the log store. You can use Elk Loki or cloudatch or so right and generally this contains the information like show me like all last 401s in my last 10 minutes call right and what did or do when the token

04:11:21 refresh failed right so those kind of request can goes to the log store now there are some you know traces and metrics data um like realtime data they goes to the open telemetry as well um open telemetry again is open source solutions which going to collect all your traces um and metrics and is going to process the data and redirect to the backend system.

04:11:43 Right? And in this case the traces will go to tempo. This is again an open source solutions which going to store all your traces in terms of like traces in the sense like you know like information around like which tool was slow on this request whether this call is coming from agent or IDE uh what was the exact error um on this failed code or so right so these kind of data are going to store into the tempo.

04:12:07 Oh my bad. Okay. And then the metrics is about uh the promethis is all about the information about like um what are p95 latencies for a given tool right um whether my o uh rejection is spiking on a particular time or not right so those kind of metrics is going to store in prometh so once you get all this data point like traces and metrics this entire aggregate data back actually save into the graphana for your visualizations Right.

04:12:41 All right. So the MCP server gateway and monitoring is covered. Now we're going to talk about the uh framework. So the agent framework right. So we use langraph uh for most of the cases. Um and is basically langraph again is a uh provided by langen community is provide a graph based architecture that support u loops branching state management for complex multi- aent AI workflows.

04:13:05 Right. And the way it work is like the moment you submit your first query uh we do interpret u you know like your question first like what type of questions it is and what response is looking for um and based on that uh it's create a plan right um then it will send for classifi uh classify uh area where it will determine that hey whether it is just like a read operations or whether it is just like a um you know modification level changes coming up or whether you really want to delete something right or some sensitive

04:13:40 operations based on that um it adjusts the overall structure and then plan accordingly. The next step is plan is actually itself is a subgraph where you do the measure of your operations. Meaning that in the plan like once you know what operation you're performing it goes to the like let's say in a fetch evidence section where it actually call that whatever the question you ask here after all interpre and classify um it actually go ahead and make the call to the MCP gateway uh make sure they get authenticated um and

04:14:12 then it make the call to the MCP server using the right tool um and get that data and then generate that evidence right once you validate all your finding then it will check the policies. uh it will check whether it is a read only then go ahead and execute and I will save all the record points into our audit logs but if it is a right and sensitive operations definitely it will go through the HITL process where you wait for an approval if it is approved you go ahead and execute if it is get rejected in either case is

04:14:43 going to save into the audit log right so if we combine all this together like MCP gateway MCP server um and the agent right this is how things looks like. So ultimately whatever the agent you are building whatever the call you are submitting it will goes to the MCP gateway for authentications and then it will redirect to the right MCP servers to execute uh you know your query on a given tool uh so that it make the API call to that endpoint and get the response right.

04:15:16 All right if it if you think from the infrastructure perspective this is how it looks like. So agent is communicating to the MCP gateway goes to the MCP server and to the endpoint. All right. So now the we talked a lot about MCP gateway, MCP servers um the monitoring um and the agent framework. Now we talk about uh three and four additional framework that we should have to take care of when building the end to end ecosystem.

04:15:45 Right? One is them is agent memory management. Again it's a uh you know it's a it's refer it's basically a process where you can save and restore the state of your workflow over the time right ensuring that uh it's provide you a durability and seamless recovery across uh across your every query run. So the way it work is like whenever you submit any query to your agent um it actually generate a thread right uh on that given thread or when you ask your first question it will going to create a run ID for that.

04:16:18 So the moment you ask some deeper questions it will go on a particular node in that langraph and execute that query and on that node it going to save that respective checkpoint here. Right now if you ask a follow-up questions on the same thread it going to create another run id and again depend on your follow-up question it go to a specific node and save the checkpoint over there right and all this data for your local development you can use the SQLite SQLite DB uh but in production in let's say in GCP you can use um

04:16:50 cloud spanner and um and respective databases you can use in a Azure and AWS as well right so just walk you through the quick demo demo here um on the memory. So for example um I have you know like just built this agent locally and then um you know trying to run this query on SQLite DB to see if there isn't data or not. Um so I'm just you know initially I don't have any data.

04:17:18 Now I submit my query. You will see that I received the one record which is saying that hey this is my thread ID. This is the question I have been asked and exact time um you know when this query was executed. Now if I run the next query to see that hey how many messages I've been produced on a given thread. Um you can get this information by running the further queries.

04:17:38 Right now if I would like to run to see that um you know like all the metadata level information on my query um I can run my query again and you can see the detailed description around you know like how many tool I get invoked, what is my thread ID, what is my run ID um you know like and all those informations. Now if I want to also see that hey what tool have been executed when I run this query um you can go ahead and see the exact tool name as well on that given MCP servers.

04:18:07 Now further I want also want to see that how the progression was happened on my langrap framework that also you can see by uh running this type of query on your memory DB. You can see that here like different kind of you know nodes been called when actually u you know that query has been run. All right. Now the tracking and eval. So just to make sure that uh there's no confusion here like we talked about earlier the MCP gateway monitoring.

04:18:36 This is the agent level tracking. So we do this because we really want to track that what query you have been submitting to the agent um and what the response is coming what LLM model you have been used what's the cost of your token u how many token you have been burned and what's your you know evaluation criteria looks like what is the quality of your response or not right so you need to store that data from the day one and imagine that if you have a thousand plus agent every agents will have their own um you know um

04:19:11 agent tracking and evalu project and the tables right so if you combine let's say if you have 100 agents you will have 100 different projects and each project has their own respective tracking and eval tables now tomorrow 6 months down the line if somebody askked you that how your agent has been performed what was the response what was the quality how much token you've been burnt you can aggregate all this data and you can create executive dashboards around it to see u to see what is ROI in building so many agents,

04:19:40 right? So, let me walk you through the the tracking in Eval uh demo here. So, um here like if I look at uh this one. So, first like you know like if you see that there are six tables are listed here. One is a job tracking table um and which is contains most of this like information around what query like what is your query, what's the response and everything.

04:20:05 And there are a bunch of other tables as well like your quality related and your tool invocations and then node level executions and all that. So in this case I'm going to submit my first query on my agent um and you will see that um there will not be uh any data in the first place uh because it takes some time. So I will refresh this again and this time I will see that when I refresh I do see that for a given query I get my request ID my session ID my agent name um you know what question I've been asked um and what

04:20:38 was the output um of that uh question uh whether it has completed or not and then what is the time it took what is the model I have been used um and then some of the meta metadata level informations right further if I look for The quality again there is a eight quality checks per run right and you can see that this one row each for you know like success like latency cost tool accuracy safety etc like all this has been stored here right now the eval scores this is again this going to be a one row this is going to be a

04:21:18 scoreboard for your given run which is basically varies between 0 to one which is scores uh you know like all the quality dimension in a single row. Right? Now the quality metrics. So again it's a deep dive on that particular one run. Basically a nested field for the response quality safety latency cost uh and the tool stat in one wide row. Right? Now the tool invocation.

04:21:41 So every tool call which MCP tool ran how long it took and whether it succeeded or not. Right? And now the tool traje agent trajectory which is talk about stepby-step story of the run each LLM call and the graph state in order like plan summarize etc. All right so now uh the next is the agent PI tremor. This is very very important because many times happens that when you working in agents or uh this thing um sometime there's a possibility that there are some secret being ingested as a part of your query right let's say

04:22:20 you are uploading some doc which is has some s sensitive informations um and sometime while typing you may have some API keys or so we have a framework which define that different sort of data types and their severity level if your severity is above 90% or so uh we will block and redact that uh secrets uh immediately so that uh make sure that that secret is never goes to LLM models right neither it will goes to your memory DB or nor it will go to tracking and evaluables or tracing systems as well.

04:22:55 So the goal here is to just block this sensitive data as it is the moment it get ingested right moment moment you type. All right. Um the last framework is all about now is the context optimizer. This is one of my favorite area where like um you know uh whenever you submit it's very important right whenever you submit too many queries sometime you upload lot of the um data points and you upload bunch of files systems or so right it's very important that time is um you know like you make sure that you save your token in

04:23:32 every query so we built in context optimizer framework where you actually shape your query and then you prune it before you go to LLM in the pruning system. Uh we do select the payload based on that token limit. Then we rerank that hey which is the most relevant relevant information that we should target for when you submit this huge giant you know context and then we ddup that uh remove all the duplicates and create a very unique set of uh very curated data like uh I mean the main data for the LLM and then we finally

04:24:06 compress this large payload before it goes to the LLM. So let's see this uh demo here. Uh this will be quick. So for example, let's say I will submit my first query. Hey, find me all the artifacts in my JRO artifactory. Um you see that first when even I not submitted any query yet, there is no data here. So the moment I submit this first query and if I refresh my query here um and then you can see that the record has been added.

04:24:36 So you can see there is a 24,000 token has been used. I will go back a little bit um just to show the thing. So you can see this 24,000 token has been used for given questions. Now if I ask uh now I enable the context optimizer behind the scene and I'm rerun the exact same query again. So this time you will see that uh you know for the same query if I refresh this run and you will see that uh this time it used just 2,400 token right for exact same query.

04:25:09 So which is almost like 90% saving. I can rerun the same demo on the slackbot as well where I'm asking slackbot hey tell me all my docker images across all the repositories in artifactory. When I run this query there is no data. Now I'm going to submit my query. So once this query will run um and if I run my query again here and you will see the record get created it's again used like 24,000 for this this particular query.

04:25:36 Now what I will do is I will turn on my optimizer behind the scene and I rerun the same query in the slackbot um and then if I run this query again um it will show like it exactly used 1,500 almost token which is again like 90% savings. In most of the cases we get like 70 to 78% token saves. So imagine that you have thousands of um you know like um um if you have thousands of like agents which is running like you know many queries per day imagine how much token you can save with this framework.

04:26:11 All right. So now the key takeaway here is uh you know like um for us is like make sure that we build an enterprise framework um not just an agent right which make sure that you have all the framework under it don't try to be in rush to building some agent quickly and deploy ship it make sure that it has to go through all the framework all together so that you should not realize a letter stay that you know I should have this framework in place right in in the very first place second is like your make sure that you have

04:26:41 a proper gateway system. Make sure that every tool call request either you are doing this from your local or through agent. It has to go through some sort of centralized MCP gateway. uh make sure it authenticated before it reaching to the MCP servers and then make sure that your MCP servers take a control on what tool uh you know uh you you should use uh to execute a particular query right and make sure that you have monitoring in place and complete auditing in place for that tool executions and of course you should u

04:27:14 you know keep the always human in the loop process for every u every action items every high-risisk items uh make sure that there is a observ ability um and you know um auditing has to be in place and last but not least make sure that context optimizer should always be there to justify your cost um um on your agents right um so just want to share is that I released my um MCP uh book recently on Amazon so feel free to download it's free in Kindle um and it's uh hard copy is also available all right um thank Thank you so

04:27:50 much for watching this video. Um, and if you do have any questions, feel free to, you know, like reach out to me over LinkedIn or Twitter. U, I'm happy to answer. Go live proves that a system exists, but it doesn't prove the people behind the work will actually use it or that it will ever deliver a single dollar from the deal model. I am Michelle Brandt, creator of Leading at Scale.

04:28:41 And the question every enterprise is confronting right now is the agent works. Does anyone trust it? I have spent over 20 years in enterprise transformation and more than a decade leading M&A integrations across media, tech, and consumer goods. on those deals. Trust was not something we crossed our fingers and hoped for after launch. We engineered it before day one through early change management and risk management.

04:29:20 So when the platform went live, the teams on the ground actually relied on it. AI agents demand the exact same muscle. An agent earns trust when it delivers the value the business budgeted for while stayingly within the company's approved risk appetite policies and regulatory guardrails. In my experience, leadership gets there by answering three straightforward questions.

04:29:55 What value does it create? Who is on the hook for it? and what evidence gives it permission to run with greater autonomy. The moment an agent steps out of a sandbox and into a core business workflow, the entire test changes. A pilot tests whether the software functions, but daily operations is where we test whether it creates value within the controls set by leadership.

04:30:27 Established AI risk frameworks give you strong principles. What they do not give you is how to apply these principles to each agent. For every agent, leadership has to decide four things. What is it permitted to do? Who owns it? What evidence it records? And what triggers an escalation? That work is not an engineering task. We find it is an executive operating mandate and today I'll show you how to meet it.

04:31:04 Let us ground this in a realworld scenario. Picture a $15 billion acquisition targeting 400 million in net synergies. The deal has already closed and the integration efforts are now underway. The integration management office, which we'll also refer to as the IMO today, is driving the synergy roadmap across 40 workstreams, covering everything from rationalizing vendor contracts to consolidating ERP systems.

04:31:39 And every single initiative has a business owner accountable for its bottom line savings. Every week, analysts in the IMO are going to reconcile the numbers against the general ledger as they track down status from all 40 leads. They're also comparing actuals against the plan. Then finally assembling the executive deck for the steering committee which is going to include the CFO, the chief risk officer, business unit heads and the integration lead.

04:32:20 The engineering team builds an agent to automate that first draft. The agent then ingests the weekly updates, reconciles the ledger, runs variance map against the plan, and flags the initiatives that are falling behind. Everything looks good, and they decide to run a four-week pilot. The agents draft match the analyst reports line for line. So, we can say that the tech works.

04:32:50 Now the question is what value does that steering committee need before they can trust it to run the business. First the steering committee needs a written value case agreed upon before the agent ever touches production data. In our example that comes down to three specific metrics. First let's look at cycle time. If that weekly report needs five business days to compile right now, you're going to want to set a hard target such as 2 days.

04:33:30 Your second metric will be analyst capacity. If four analysts spend half of their week building the stack, that is 80 hours a week the agent gives back. That time counts as value only when the integration lead assigns it to the work streams that are behind on their synergy targets and need help getting back on track. Then you only count that value after netting out the compute licensing and human review cost of running the agent.

04:34:09 Your third metric is the most important one. detection time. That is how long it takes from the first sign of a splip in the data to the moment the steering committee sees it. Cutting the report from 5 days to 2 days does not guarantee that the committee sees that slip any sooner. The agent has to catch it and the value case measures that directly. Every single metric requires a predeployment baseline.

04:34:40 If the IMO cannot state the baseline today, do not deploy. If you can't measure where you started, you can never prove the agent move the needle. Next, the steering committee needs to know who owns the risk the agent creates. We are going to apply the threelined model to the agent. In the first line, management owns the controls. The integration lead is the accountable executive.

04:35:15 She owns the agent's output exactly the way she owned the manual debt. In the second line, risk and compliance oversee the agent and challenge how to run it. How deep they go depends on the agents exposure. An internal draft scrubbed by a senior analyst warrants a very different review standard than an autonomous agent publishing figures straight to the board.

04:35:45 In the third line, internal audit provides assurance deciding based on material risk when and how the agent enters the annual audit plan. The agent also needs one employee assigned to oversee it every day. In our example, that is the senior analyst. He reviews the draft before it reaches the community. Her role requires four specific capabilities. She has to first understand the operational boundaries.

04:36:20 She has to interpret its output with context. She must have the authority to override or reverse it. And she needs clear kill switch to shut it down safely. The it it behaves unexpectedly. Accountability also means defining where the agent's authority stops. I document this in a one-page policy called the human work standard. For every fork in the process, it matched the decision, the human role authorized to make it, the required evidence, and the escalation path.

04:37:03 These decisions exist at every tier. The senior analyst decides whether a variant is an accounting timing difference or a real cash shortfall. The workstream lead decides how to remediate that shortfall. and the integration lead is going to decide whether to formally flag an initiative as at risk to the steering committee. The agent prepares the analysis behind every one of those decisions and it stops there.

04:37:36 The person who makes the decision owns the consequence. So, keeping these decisions with your team does not slow the agent down. It can go into use while each role keeps its authority. The one reason to delight delay deployment is a decision with no assigned owner. This means if leadership has not agreed who declares an initiative off track, the agents flags lead to no action.

04:38:12 The third question is where the engineers in this room come in. The evidence for greater autonomy comes from how the agent is built. I look for three artifacts. The first is an auditable decision log. For every workstream the agent evaluates, it records the raw data ingested. The tool crawls executed the prompt and the model versions output generated along with the human signoff.

04:38:49 Two engineering choices make that log airtight. First, your core map and reconciliation rules must run in deterministic finance approved code, not generated as conversational pros from an LLM. Second, treat the model's written summary as a helpful explanation, not proof. Your real proof lives in the telemetry. What records were pulled? What deterministic tools ran?

04:39:21 And who signed the review? If reported savings do not match the general ledger, agent does not touch the number. It flats the discrepancy and roots it up. The second artifact is the model inventory record. This captures the accountable executive data lineage dependencies across source systems validation results escalation path and the chill switch runbook.

04:40:00 Most importantly, it documents your regression triggers, which are any updates to our parent base model or source API that require full revalidation before running in production again. Your third artifact, this is going to be the autonomy or card. The steering committee uses it to grant autonomy in four stages. Stage one is shadow mode. The analysts build the report as they do today.

04:40:31 The agent runs right alongside them and the team compares the two. Stage two is the supervised draft. The agent compiles the first draft and the senior analyst reviews every single line item. Stage three is the exception and sample where the analyst reviews lagged initiatives plus a random sample of the claims. Then we have stage four where exceptions only the analyst reviews only high risk anomalies supported by periodic independent sampling.

04:41:15 Pay close attention to stage 4. However, because once humans only look at exceptions, your biggest vulnerability is what the agent failed to flag. So, I advise tracking missed material issues through independent sampling, monitoring error severity, and watching your override rate carefully. A falling override rate does not automatically mean the agent is getting smarter.

04:41:43 It can also mean the depth of review has split. So independent sampling tells you which one of these you are seeing. The scorecard puts those control metrics side by side with your value metrics. Cycle time, analyst hours deployed and your detection time. And with that single page view, the steering committee can make clean, efficient, databacked decisions to either promote the agent, hold it where it is, step it back, or down to supervised review stage.

04:42:24 So let's come back to the three questions. Ask them of every agent you build, fund or govern, what value does it create? Who is accountable for it? And what evidence permits it operate with autonomy? In our M&A example today, the agent does not magically create the 400 million in synergies. Procurement, supply chain, and operating leaders capture that value through the gritty work of integration.

04:42:58 But what the agent delivers is clear. Verified evidence weeks ahead of schedule. Giving leadership the runway to intervene while the deal's synergies can still be captured. Building an agent that works is just the first step. Answering those three questions is what earns it the trust to run at scale. Thank you. Hi, I'm Steve Jameson, AI research and engineering leader at Jellyfish.

04:43:52 I'm here to show you why tokens are the cheap part. The research that I'll present is drawn from the Jellyfish platform, the industry's largest set of engineering workflow data joined to AI telemetry data. It's no secret that AI costs are climbing. They're up 60x from this time last year. The use cases are wide and varied. The models and harnesses are getting much better.

04:44:17 The implications are exciting. But discussions around budget and spend are not as much fun. These conversations involve your peers and senior leadership and the board. Not always the easiest audiences, especially when you don't have the right data. And it's real money these days. Output is up, too. Merge PRs per engineer are up 43% over the same one-year horizon.

04:44:43 This is phenomenal. We've not seen productivity inputs like this, especially on this time scale. There's plenty of research and anecdotal data to know that AI and Agenta coding are driving these throughput gains. Heck, even our charts show the correlation. Notice where the inflection points are. AI spend increased in early 2026 when throughput gains start a steady increase of their own.

04:45:12 There's a tension with these two numbers, though. How do you reconcile them? Spend is up 60x. Throughput is also up 43%. Is that good? Bad? 43% is a big number, but so is 60x. How do you know if you're spending enough? Is there more productivity gain locked behind a checkbook? Should we increase spend 50%, 2x more? Or how do you know if you spent too much?

04:45:42 I'm sure many of you have felt the tension here. The CFO or board are asking for the receipts of the spend. Can you show them? Will merge PRs are up 43% satisfy them? Also, what happens if you cap use and spend? Are you stopping from setting tokens on fire or are you stunting further efficiency gains? You can ask peers at other companies what they're paying.

04:46:10 For many, it's still a moving target, though. But how do you equate that to your org? Surely just comparing their bill to yours isn't adequate. We've heard the reports of we've heard the reports of companies spending 100k, 500k, even millions on AI. Setting then resetting their AI budgets as reality sets in. Ultimately, these two numbers can't tell the story.

04:46:36 And even together, they missed the biggest line on the cost side. These two numbers leave out the biggest cost of all, developer salary. AI is making developers more productive on a scale we haven't seen before. It isn't just another tool. It's intelligence. This intelligence dramatically changes how much each dev can produce. That's why we need to think about its cost differently.

04:47:04 Not off to the side as overhead, but as part of what it takes to produce each PR. We're going to call this intelligence costs of a PR. What you pay your engineers plus what you pay for AI, put them together in one cost bucket. AI is about 2.5% of the cost to produce another unit of merge code. Salary is the other 97 12%. This is in line with other reported figures.

04:47:39 And this data is from August 2026. Here tokens are the cheap part, but that's current relative costs, not dollars. Let's look at what happened to the cost of emerged PR over the last year. The dollar cost of emerg PR fell 26% over the last year. Not PRs created but PRs merged. Review time is factored in and some level of quality gates has been cleared.

04:48:17 What that decline and cost per merge PR includes a 60x increase in AI costs over the same time frame. The change is nearly universal too. 83% of the companies we studied showed a drop in cost per PR. What does this tell us? The additional throughput gains are being spread are are spreading salary thinner across PRs and this is happening faster than the AI costs are increasing.

04:48:46 What's the range in token spend? We ranked companies by token usage then group them into quentiles or fifths. The range here is quite large. The lightest group about 0.3% of the cost of a PR is is AI. This represents about 60 million tokens per dev per month. The top 20% of AI spend per dev. We're going to call the heaviest spend group. There 8 and a half% of the PR costs is AI.

04:49:23 This represents about 2.5 billion tokens per dev per month, about two orders of magnitude more than the lightest group. We'll use these groups on the next slides to see how token spend influences other metrics. First, PR throughput measured as merge PRs per death split by the same AI spend quentiles on the previous slide. The bottom four fifths of spenders saw PR throughput gains of around 40%.

04:49:57 Give or take. But the heavy spend group improved more at a 67% increase. More AI spend led to more throughput gains. Now let's look at the cost per PR. Positive results are seen again across the board. cost per PR is dropping for all AI usage groups. However, again, the heaviest fifth, so the the companies that spent the most improved the most. They saw a cost per PR decrease of 32% over the last year.

04:50:37 The other four groups saw about a 25% decrease in cost per PR. investing and spending more on AI fueled more of an improvement in cost per PR compared to peers who leaned in less. So we saw that the pop the same same as we saw in the population average cost per PR changes are driven largely by PR throughput not by AI spend. The heavy spenders experienced PR throughput gains of 67% compared to about 40ish% for their peers.

04:51:15 Spreading the same salary across more PRs in the month has more of an effect than the rising AI costs because tokens are the cheaper. This is not to say that orgs should light tokens on fire and spend with abandoned. Here we're looking at tokens used on the horizontal axis and cost per PR on the vertical axis. There's a steep drop at the beginning where relative small AI spend results in large gains initially.

04:51:47 As token usage increases, the cost per PR improvement decreases. This is diminishing returns. I want to stress though, diminishing returns is not the same as no returns. We're seeing that the top 10% are spending 8x the tokens compared to the median and achieving a 27 25% lower cost per PR. There will be a time when tokens aren't the cheap part, but by and large for most companies out in the wild, we're not close to that threshold.

04:52:26 More tokens leads to more PRs, which leads to a lower cost of delivery. So far, we've talked about companies. Next, I'd like to take a tangent and talk about a study we did with developers. Here we looked at developer weeks with heavy token use compared to the same developer weeks with lower token use. Here we saw the same phenomenon just at the individual level.

04:53:06 Compare each developers heavy agent skill weeks with their own weeks without skills. They spend about twice as much on tokens. merge 27% more PRs and reach PR costs and each PR costs about 18% less once salary and tokens are counted together. There are two fair curiosities I get when I talk about this work. First, does faster work result in worse work?

04:53:47 And second, are the more PRs, the throughput gains that we're seeing, are they being driven by smaller chunks of work? First curiosity is cheaper, faster work, worse work. If AI makes PRs faster and cheaper, are we just shipping more mistakes? One proxy for this is re reverts. Merge PRs that undo an earlier PR. The heaviest fifth started lowest before most of the token growth and revert rates dropped slightly for every group.

04:54:25 Though these deltas are just slightly below the noise in the data. It's not evidence of improvement here. However, more importantly, we're not seeing any worrying signs of rampant or severe quality issues with higher token spends and faster PR throughput. The second curiosity is throughput driven by smaller PRs and this is demonstrabably no not smaller PRs.

04:54:59 In fact, AI is driving an increase in line lines of code generated per PR. This is no surprise to the devs in the room. Raise your hand if you've rolled your eyes at verbose, unnecessary inline AI comments or a file full of mostly irrelevant tests. Gen AI Gen AI likes to generate code. Also, the bar to create is low for devs, and there's a pull to do more per PR instead of spinning up another ticket.

04:55:32 It's just one, maybe two prompts, and I'll handle this edge case, or I can combine this and that element. Fewer PRs to wrangle. This doesn't mesh well with the PR review stress folks are feeling these days, though. Also, AI is just becoming more capable, so we're getting more ambitious. It is beyond this talk to dig into what's actually driving those level of lines of code changes.

04:56:03 However, the increase in PR throughput is not coming from smaller PRs. This is intuitive for most, but it's still good to verify. Back to the opening question. Is 60x more spend worth 43% more output? You can't tell in isolation. You should track both. They are important numbers, but they're supporting metrics. AI spend per PR is the intelligence that you're buying as a resource to help make your devs go faster.

04:56:37 PRS per engineer is what you're getting with that spend. However, looking at those hides the trade-off, but put salary into the equation and take the ratio of cost to throughput and there you get your answer. Cost per PR is down 26% over that same time period. It's clear AI is driving cost efficiencies despite the increasing bills. And the AI sp the AI spend needed to get those efficiencies is small relatively speaking.

04:57:20 AI is still a sliver of what a P PR costs two and a half% on average under 10% for the heavy spender group. Tokens are the cheap part in the range that we see more spend kept paying. Heavier token users saw 67% PR throughput gain compared to 40% for their lower spend peers. This drove cost per PR down 32% compared to about 25% for the lower spend peers.

04:57:53 Looking at the PR cost to token ratio curve, the top 10% of company months by token usage resulted in 10 or 8x the tokens, but 25% lower cost per PR compared to the median. AI is making developers more productive on a scale we haven't seen before. It isn't just another tool. It is intelligence which changes how much each dev can produce. That's why we need to think about its cost differently.

04:58:29 Not off to the side as overhead, but part of what it takes to produce each PR. You need to add the AI spend and the human spend, the human salary spend together to calculate your real costs per PR. This isn't to say spend frivolously. Sure, you should watch for runaway jobs. You should track your AI spend. I'm not saying don't look at how much AI is going how much AI spend is going in each PR.

04:59:04 I'm saying don't let that be your driving metric. A new framing might steer how and when you think about other investments like integrating model routers, post-training your own model, implementing token limits, etc. Will you get more efficiencies out of trying to control AI costs or by continuing to invest in things that increase throughput like more AI enablement?

04:59:32 Either way, track the total intelligence costs human plus artificial per PR every month. Next to that AI bill and next to the PR throughput, how are you trending? Is throughput continuing to improve? What about those cost efficiencies, the cost to deliver? Watch reverts and quality metrics beside all of this as a counterwe. We don't need AI slot mucking up the gears.

04:59:59 So heading into budget season, bring cost per PR to the table. Not just your token bill, not just the throughput bump. The bill tells you what you spent. Cost per PR tells you whether it worked. And right now for most companies, it says keep investing. Tokens are the cheap part to get efficiency gains. Well done. You made it to the end. Here's how you can stay up to date with our latest research, which again draws from the largest engineering workflow and AI telemetry data set in the industry.

05:00:41 You can follow me on LinkedIn, my colleagues or jellyfish. Subscribe to our Substack or you can check out our Jellyfish blog on the website. If you want help tracking your org's engineering workflow and AI effectiveness, including cost per Merch PR, go to our website and book a demo or reach out to me and I'll get you connected to the right folks. Again, I'm Steve Jameson, AI research and engineering leader at Jellyfish.

05:01:13 Thanks for watching. Cheers. Okay. Yeah. >> Do we want to call it Jurro or No. >> Yeah, we could call it Jirro. We could just be like this is our agent Jurro. I think it's also like a helpful potential cut which is like if we ask Jirro like are you ready to get started? >> Yeah, sure. >> And then like you know our video can move Jirro can like draw something acknowledgement whatever a transition pain.

05:01:57 >> Yeah, >> transition cut. Okay. >> Yeah. >> All right. Um let's get started. Um thanks guys for coming to listen to a talk on the AIE online track. I guess coming to listen is not quite right because obviously you guys are not in the room with us. >> Making the long long trip. >> Yeah. Making the long long trip. Yeah. Yeah. Yeah. the arduous journey.com.

05:02:15 >> Whenever we do these kinds of talks, we always want to play around with the format a little bit. You know, I'm not a huge fan of just standing in front of a loom and doing slides late at night in my room. Um, so I dragged Cliff along to do something a little bit different. We wanted to do this in a little bit of a podcast format. Um, and part of that is just because it's nice to have multiple people to like riff with and bounce ideas off of, but also because we wanted to find a way to include Jiro, our agent.

05:02:36 Um, so obviously, you know, on our side, Jirro is not quite in the room with us. He's kind of living on this computer. Um, but when you guys are watching this on YouTube, hopefully you'll have all three of us present. Uh, you'll have me and Cliff talking and then Jurro will be sort of in the middle drawing things out and describing uh describing things.

05:02:52 Um, >> Jurro, please wave hi for the viewers. >> Yeah, Jurro, please wave hi for the viewers. Um, so this talk is all about identifying a secmaster. Um, this is part of AIE New York, which of course is very very focused on the finance and like AI and finance uh in 2026. Um, and so I think uh before we really dive in, maybe it makes sense for just us to briefly introduce ourselves and then we probably want to just give some definitions even like what is a secmaster?

05:03:18 What is a cloud agent? How do you like use agents to do any of these things? Um, so I'll go first. Uh, my name is Amole. I am the CEO of a company called Nori Agentic. And as you might have guessed, we spend a lot of time thinking about cloud agents and how we can put agents in the cloud to solve problems both in finance and kind of all in a bunch of different industries.

05:03:36 And yeah, Cliff, >> I'm Cliff. I'm the CTO at Nor Agentic. We've been working on dev tooling, AI agents, this form factor for almost two years now. >> Yeah. >> And before that, uh, I was working on data engineering and building out a new data platform for a startup hedge fund. And before that, staff software engineering and there's kind of a throughine all those things with data engineering, dev tools, some amount of cloud infra.

05:04:00 >> Yeah, it's kind of funny actually. This is almost like a perfect talk for you. like I'm sort of the odd person out like I my background is in ML so I've been doing the a IML thing for like 12 years but like I don't know anything about finance like you know all the the finance stuff um so it's kind of an interesting conversation right because we have two potentially very different audiences in the room you have the finance folks who maybe don't know anything about AI and you have the AI engineer folks who maybe don't

05:04:20 know anything about finance um so maybe it makes sense to even just get started with some definitions like what is a cloud agent and what is a secmaster before we can even dive into how these things fit together >> yes So cloud agent the most succinct definition I think for just agent to start an LLM or a model that you run in a loop and give access to tools and so it can take some action that you hopefully find useful >> specifically cloud agents you take that whole form factor usually the code that is calling the

05:04:49 model in a loop and you put it in some cloud process and you give it access to tools that don't exist on your laptop or your phone or wherever else >> and now that whole entity that hopeful useful primitive runs in the cloud hopefully taking action with either your company's data and processes or your own needs >> and I mean so much of the important data now lives in the cloud so like in some sense it actually is like a very natural home for all of that to even run >> yeah I think it's there's a certain uh historicity

05:05:18 of I think when we saw local agents take off right with the cloud code moment last year right and before that with earlier agent primitives >> right >> I think >> it's almost like a quirk of history that like it became local first. >> Yeah. >> Just to start. >> And like the long run, we're going to see it take a lot longer for actual onedge >> devices to really deliver value.

05:05:39 >> So that's cloud agents, you know, agents that run in the cloud that you can basically do things with asynchronously and talk through across a bunch of different platforms. What about a sec master? >> Yes. >> Now, we probably want to spend more time here because I think that this is a lot harder. You know, it's harder for the average, I think, person who's not in finance to understand what a sack master is compared to the other way around.

05:05:57 >> And I think that people in finance have heard a lot more about agents in the other way. Yeah. Yeah. >> And software engineers. I've heard about a sackmaster. >> Yeah. >> I certainly felt that way several years ago kind of jumping into that field to you know bring data engineering expertise to help a startup hedge fund begin trading. So I mean trading I think is the place to start.

05:06:12 So you have in theory a tradable universe of assets that are core to your business as an asset manager. Many different types of asset managers. We don't even need to go there right now. But for your business to operate, you need some master database of all the securities that are in that tradable universe. Yeah. >> And so it's basically just like a table like for the engineering side you can kind of think of this as a table that lives in snowflake or you know data bricks or something like that but in practice it's

05:06:38 actually like a really really important operational primitive. >> Yeah. And I think it's if you just give the right definition of identifiers and assets and instruments and you start to explain the difference between okay you have you know instruments and some of them are securitized and there will be different asset classes and okay that's that that seems like most of it.

05:06:58 >> Yeah. And then actually >> actually uh building the thing going inside the primitive there's a lot of complexity a lot of accidental complexity and once you get a centralized representation it makes all the downstream processes very very easy >> and that's why almost every asset manager has one >> but actually munching all that data together and every time you bump into a new asset class or a new ID format a new upstream source of data >> you actually have new needs that you need to abstract inside the security

05:07:25 master. I mean, I tend to think of it almost like an operational primitive where you just have all of this data, all these different data sources and all these different data consumers. And if you don't have something in the middle that basically ensures that the right data is getting to the right places, it's impossible to otherwise guarantee in a distributed way, that you're going to have clean data, that you're going to be able to report the right things, that your audits are correct, that your pricing models are

05:07:46 correct, like all of that stuff essentially is downstream of having a good clean semaster. >> Yeah. So the segment master for most analysis that you're going to run, it's going to be the left side of almost every join. >> Yeah. >> It's going to be the starting point for a lot of the analysis that you do. >> Yeah. >> And that drives some of the biggest processes at any asset manager, >> right?

05:08:06 >> And that could be everything from your reporting requirements to looking at historical performance to understanding your >> some of this is just regulatory like you literally have to have some sort of a record of what assets you have. >> Yeah. Exactly. >> Yeah. I think it's very easy as software engineers for us to jump over some of the accidental complexity that gets encapsulated within an abstraction.

05:08:28 >> Sure. >> And so the the comparison I've been thinking about when we've explained this to other engineers on our team and talking with our customers. Uh it's like datetime libraries. >> It's very it's very easy for engineers to think, oh, it's a daytime library. >> Yeah. You're just tracking the time. How hard could it be? And then you realize like time zones are a thing and like people track things differently and it's all sorts of pain.

05:08:49 >> Yeah. Everything from the localization to how people see the same representation to actually moving something between different time zones >> and uh daylight savings actually hits at different points in different countries. >> Yeah. >> Uh so just like the briefest snapshot of all the accidental complexity that you hide inside the perimeter of the semaster.

05:09:08 >> Yeah. >> I think like the easiest place to start just so people grasp this is publicly traded equities. >> Yeah. Like a Google or an Apple stock or something like that. You go to the exchange, you buy Google, something like that. >> Yeah. And this is by far the easiest asset class to even handle. And already you just mentioned one of the first quirks which is Google.

05:09:24 That's not even one to one between tickers and companies. >> Yeah. What do you mean by Google? Like are you buying alphabet stock? Are you buying Google stock? There's actually two completely Google class A and B >> and they are different on different exchanges and they're different in different countries. >> So tickers aren't unique. They're not onetoone with the actual entities, right?

05:09:41 And then on different exchanges, you can actually have the same stock listed across potentially, you know, a different exchange in, you know, the EU might have actually different listings of the same underlying uh shares. >> So you got this rat's nest of data basically and it's really really hard to unentangle and you have to basically have a team of people who are doing this like pretty labor intensive role of keeping this all clean.

05:10:04 >> Yeah. And so a lot of it's kind of architecting how the seg master should work for the area or particular asset classes that your business will be investing in. >> Yeah, >> I think that especially in the publicly traded world often times most of the structure to the data comes from the moments where that data encounters different centralizing forces.

05:10:25 >> Sure. >> So the two biggest examples are the regulatory constraints and the actual exchanges where it's traded, right? talking point of sale, point of exchange >> and those two have the effect of creating most of the identifiers that you care about in the public. >> So if those are the assets that you're interested in, >> you have to make sure that you have those identifiers.

05:10:44 You have to make sure they're aligned. You have to make sure you have them somewhere recorded in your company because anytime anybody wants to do anything with those assets, they have to reference them back to those things. >> Yeah. And at first it can be relatively easy because you're dealing with a particular asset class. you're dealing with one intersection of different centralizing uh structures.

05:11:02 >> You can just be like, I'm just going to use Bloomberg's ids for everything and not worry about it. >> Yeah. So, you have Bloomberg's IDBb globals. It's like a column in the source data that you're going to get from them. >> And these are called figgies sometimes because of a previous acronym. >> It's called Figgy. Fi GY. >> Yeah. It's like financial instrument global identifier.

05:11:20 >> Okay. >> Um but they I think I don't know if it's officially renamed. >> Sure. Yeah. Sure. Whatever. >> Yeah. It's a little bit easier to say than IDB global. And so from that that's going to answer a lot of your problems. >> Yeah. >> But the moment you get a little bit bigger then it starts to get a little bit messy. >> Once there's a little bit more complexity or you're not dealing with things that are understood in the Bloomberg universe of assets.

05:11:38 >> And so actually one of the things I think is really interesting is the last three years I think has seen a massive growth >> in for asset managers in assets that are not of this sort of like standardized globalized public uh entities, right? Like Google and Apple in some sense are pretty well understood, but so many people are now investing in like data centers and like clusters of GPUs and if you're just like buying a data center from somebody or buying like shares of a data center, that's not something that trades

05:12:03 on a liquid market and so you have to like track that in some other way. >> Yeah. So I mean this is not my area of expertise but the data centers are a very interesting one because I think it touches on multiple different areas of private privately traded instruments, >> right? So you might have actual real estate that's involved in where do we build this data center >> the actual land.

05:12:23 Yeah. >> And then from there there might be a privatelyowned company that does issue equity. Y >> maybe it's held by a publicly traded company, >> right? >> Maybe there's some publicly traded bonds issued to fund some part of this or maybe there's private credit that's taken on to fund the construction. >> I mean increasingly the GPUs itself feel like an asset class.

05:12:43 >> Yeah. And there there actually are efforts right now to securitize >> GPU GPU clusters. Yeah. Sure. >> Well, I mean because uh you know another private asset that's very interesting >> niche is uh airplanes. >> Sure. >> So one reason why kind of the entry costs for the uh airline business are low is because low >> low uh the entry costs are so low is because it holds collateral against itself when you're lo >> interesting you could lose the plane.

05:13:05 >> Yeah. Well so gets to most private assets this is actually a helpful distinction. Most private assets are not very fungeable. >> Yeah. Sure. Yeah. >> That's kind of a hallmark of the problem that we're going to come back to. There are some of these private assets that actually are are weirdly funible. >> Weirdly funible like airplanes or GPUs. >> Yeah.

05:13:21 So, so it might be worth even diving into then the specifics, right? So, what does it mean to identify a securities master? And I think it can be helpful to even ground this in a particular example which is, you know, we're working very closely with one of these big asset managers in New York City. One of the benefits of being in New York, New York local, like let's talk through a little bit of like what problems that they are running into and then how are they using agents to solve that problem.

05:13:44 >> Yeah. So a lot of this is that last distinction we hit on for private assets. And the first important distinction here is that the amount of data and the amount of work related to each asset does it scales linearly with the number of assets because they're unique. >> Yeah. >> So each private asset it's not like you can like write easily write software that will like handle a group of these private assets because each one is different.

05:14:11 >> Yeah. often times and also >> there's no API that you can hit to be like give me the price of this data center like the price lives in a contract somewhere. >> Yeah. And if you're part of a business like a VC where the number of these different assets already scales linear to the amount of time put in. So the number of different checks that you write is already very related to the the number of different person hours spent with those potential investments.

05:14:35 >> It's extremely manual. It's very manual and it might be >> you know that might be scalable because it's already you know the source data is proportional to the person hours and then the downstream effects then cost proportional person hours but then once you start looking at everything from real estate to private credit and the different ways that these assets can be packaged >> that ends up being its own whole new way that the that the scale of the data increases.

05:14:58 >> Yeah. Yeah. So you have this like you have this like you know exponentially increasing complexity. How does that then impact like like let's dive into this like kind of specific use case. So we're working with this asset manager like what are the things that they are trying to do and then where do agents come in? >> Yeah. So they're already seeing a process where they've created a new SECM master.

05:15:16 >> Yep. >> And they have both internal and external sources. >> Yep. Many of these are quantitative, but some of them are actually related to different documents and other unstructured sources of data. And they have data pipelines that are mapping that. And just the the lowest barrier for adoption is accurately being able to tie out individual changes to upstream pieces of evidence.

05:15:40 Yeah. >> And that just for a couple hundred columns or a couple hundred rows with even fewer than that columns is already a very labor intensive process and they manage a lot more than a couple hundred assets. >> Yeah. Yeah. Yeah. Yeah. Yeah. So, and just to give like just to give like a kind of like layman's view of this, right? Like anytime an asset anytime one of these rows in this table changes, there's a guy like somebody who gets called up who basically has to then go and figure out why that change happened,

05:16:08 right? >> Yeah. So basically being able to look at any cell in the table and look at the lineage for what produced that. >> Not every row, it's every cell. >> Every cell. Yeah. >> Yeah. >> Yeah. So every row/c column. >> Yeah. And >> so it's extremely labor int like anytime anything changes for any of these columns and rows, there's a person who basically has to go and find the provenence for that data.

05:16:27 And yeah, and I I can't undersell the trickiness raised by the intersection of both software and finance here because that person is not going to both be able to perfectly understand all the Python data munching and data flows whether that's like some notebooks that run in snowflake all the way down to like the build your own we have our own lakehouse and we're running our own Apache Spark jobs right >> you know if they are a finance expert you know someone who's >> conducted a lot of operational work in back office

05:16:55 managers don't have software engineering A lot of those people are really savvy actually with how they can use Python to solve problems on their machine. Yeah. >> With different CSVs or tie outs or exports from Excel, but being able to look at the repo and understand, >> you know, >> it's interesting like spreadsheets and software are kind of opposite >> sure >> views of the same where it's like with a spreadsheet you see the data and you don't see the logic.

05:17:21 And then with a software >> uh you know all these data pipelines, you see the process and the logic but you don't see the data. Yeah. >> And so there's completely different representations. And on the flip side of it, you have the software engineer who has implemented the data pipelines and they've >> they don't know what it looks like for something to be incorrect in this particular cell.

05:17:38 That's now causing problems for an investment professional who needs to look at that data and have it be correct. >> Right. Right. So, so you have this process that right now anytime a cell changes and for private data, you know, it's not even like you can go hit an API. You literally have a guy going through a Google Drive or going through somebody's emails to find a PDF of a contract to make sure that the data is correct.

05:17:57 >> Yeah. >> Is this the most up-to-date version? Is this the latest notorized version of the contract and not some draft that was sent back and forth? >> Um is the actual information I need here or is there a more recent update? >> Um Right. >> And so I just feel like in those terms, right, it's just so obvious that cloud agents would solve this problem.

05:18:16 >> Yeah. Yeah. Or maybe, you know, ameliate the problem. >> Yeah. Solve is a tough word. >> This kind of gets to I think the most interesting part of the problem which is why is this hard? >> Yeah. >> We as software engineers, as software professionals, we see agents solving accidental complexity every single day. >> Yes. >> So I think it's helpful to just deep dive.

05:18:35 Why is this hard? Both from a technical perspective and a business perspective. >> Yeah. >> And to your point about there's someone who needs to go look for this, right? Being able to know what good looks like, what correct looks like. That's more of a business question. Sure. Or, you know, it can also be a search problem where, you know, when do you have when have you seen enough?

05:18:56 When do you have all of the evidence? Um, it's tough to prove. >> It's going to be different for different companies and different different situations. Yeah. For different assets even. >> Exactly. And um yeah, I mean, one of the reasons why this particular instance, this particular project is a good fit is because they've already actually thought very hard about what does that workflow look like?

05:19:15 Y >> what does it look like to reconcile inconsistencies or any changes with the data? >> What would they want that back office function to actually look at step by step and when would they want them to escalate >> an inconsistency they're seeing up to a data owner or to a financial >> sec masters already have this sort of process in place >> like if there's an error somewhere they already know okay there's an error in this cell we know who is going to be owner of that cell.

05:19:40 Yeah, it could be everything from there is an investment professional whose uh you know book or whose uh investments are related to right this particular number. Uh it could be >> actually you know when when this particular data source doesn't look correct we talked to Jerry. >> Sure. >> Because he was the one who talked to them on the initial extracts of data that they sent over.

05:20:03 >> Y >> so this I think is like the first of the problems of why this is hard. >> Yeah. There's knowledge that's not encoded in either the code or the spreadsheets or kind of a lot of the purpose behind the tasks. The tasks are very encoded at your company, but the purpose behind those tasks is often not. And that lives with all of the people who are making up those different functions.

05:20:25 >> Yeah. >> Which I think also gets to the second part of why this is difficult. >> In order to properly assist them, you need to make assist of technologies. And I think this is a common failure pattern we've seen, especially on the software side. of the market for the past year. >> There's so many people who like want the agents to take everything to 100%.

05:20:41 Yeah. >> And so they create these software factories software factories quote unquote that are really inflexible. You know, they they spin up some process where it's like, oh, I tag this agent and it'll like run in in uh EC2 and then there's no thought put into well what if you need to intervene? What if you need the agent to be able to do something else?

05:21:01 What if you need to respond? There's got to be this like very rich human interaction pattern. And if your thought process is, I'm just gonna have this thing run autonomously and try and do the whole thing to 100%. Well, you're gonna fail >> because there's actually like so much depth that like a human needs to even if it's just accountability, sometimes you need a human just to be able to hit the merge button.

05:21:22 >> Yeah. >> Yeah. >> And it's kind of funny that people are shooting for that particular target when that's one of the biggest benefits of agents is their malleability and the ability to >> correct them. I mean I think mid task >> I think a big part of it is and this is kind of the thing about cloud agents. I think a lot of people think oh a cloud agent is just something that we can just have an endpoint and then kick off a task but actually the real cloud agent infrastructure is when it feels seamless.

05:21:48 It feels so seamless to be able to talk to it in the cloud as if it was running on your computer or on your phone or you were just talking to it, you know, over a voice call. Yeah. >> And that's the level of depth and product area that most people just don't want to do because it's hard it's hard to build that. >> Yeah. I think that's the second part that's difficult which is you know from the technical side.

05:22:23 >> All right, we're going to go and get started. Um I'm Scott Worland. I'm the CTO of Norm AAI and I'm privileged enough to be sharing the stage here with John Stecker who's the CTO of Blackstone. Um, from what I understand, John has basically three full-time jobs. It sounds like you've responsible for all of the technology at Blackstone. Also advises on their investments um and as as well as serves as a resource for the portfolio companies for all things technology.

05:22:49 Did I miss anything? >> Hit it all perfectly, man. >> Cool job. >> Yeah. Yeah. And if you're not familiar with what Blackstone does, then you probably have wandered into the wrong conference. Um, so I'm not going to go into that, but you may not know what Norm does. So, really briefly, uh, NORM is an AI company and we operate in the legal and regulatory space.

05:23:05 We have an enterprise AI platform that's deployed in some of the largest financial uh, institutions in the world. And then we also have an affiliated law firm, Norm Law, and our AI platform also supports that. Um, so since the end of last year and certainly all of this year, everyone's been racing to figure out how to deploy AI agents. And I'd love to hear how you're thinking about this at Blackstone and how you're trying to make sure you get this right.

05:23:30 >> Yeah, I mean I think the the race has definitely been on as like the technology has improved mightily uh since, you know, really like February, March of last year, like just in terms of context windows, being able to run stuff for significantly longer amounts of time have like resulted in people being able to produce just more value. And one of the things that like we've been focused on a lot and I think every company in our portfolio I advise on and work with all the time that they need to have top of mind for like

05:23:53 being successful with AI agents in general is you have to get your data story right. And so like one of the biggest things impediments and tripping points that I see for companies out there when they want they're all in on AI. They want to build as many agents as possible. They really want to kind of extract value. if they don't have a cohesive data strategy that's moving stuff out of transactional systems into something that can be accessed and easily managed and recalled from like an agent perspective, it is going to

05:24:20 fail. And so that's been one of the biggest things that we've been focused on. Um, you know, as a big asset manager, we have a ton of data, proprietary data, public data, all woven together. We have to move that into out of systems into something that we can really leverage from an AI perspective. I think the second thing that you know has nothing to do with AI but you can leverage AI to actually figure out how to do the best possible way is straight through processing still matters.

05:24:43 I'm always shocked at how everybody just talks about like agents as being the biggest differentiator in the world. The thing that I've leveraged myself using factory and a couple other ways to go off and build code inside of the firm is like taking time having agents help to craft something for me, understand the workflows out there that people are doing and then make sure that I'm like actually writing code to optimize for straight through processing and then using agents to inject in intelligence where necessary

05:25:09 where humans were making decisions in the past for a variety of work things. So that's really like the two key elements that I I spend a ton of time thinking about every single day. And then the biggest thing from an agent perspective is really stepping back and looking at like what is work that we want to build agents to do better to offload like intellectual like cycles from our investment professionals our finance professionals our engineers such that they can do higher value work and there's a ton of stuff when you

05:25:36 think about the software development life cycle writing test cases functional test cases performance test cases versioning forward libraries underneath applications it's the bane of every engineer's existence. I've hated it for 27 years that I've been doing this. Having an agent be able to go off and do these things for me and renovate my code and my source code has made all the difference in the world for the velocity that we deliver at.

05:25:57 So, how are you thinking about it at Norm? >> Yeah. Um, so Norm was founded in 2023. So, you know, that's three or I guess four years after GBT1, but still a year before there was like, you know, chain of thought. So, it was like an interesting time in terms of development. I remember when we had our first team get together. We were at Stanford's campus for a week and I mean I'm probably misremembering the exact timeline but it felt like we went from 4K uh to 8K to 128K context windows and then we had multimodal and

05:26:24 everything just felt like it was kind of coming all at once and then there was a bit of a time where things did slow down when it was coming from the frontier and so I think like a lot of companies in like early 2024 you know we were trying to deploy these models as part of like our workflow and as part of like our production software and so we ended up adding a lot of structure to the system and And and this comes from you know all different parts of the stack.

05:26:46 So whether that was from like how we went and took the content that we knew needed to be like eventually added to context to the models and we had like very specific ways in which we did that or if there was like you know just the sequencing of the models if there was like steps we needed to do in terms of looking for consistency and then ultimately like distilling everything down into some useful output.

05:27:06 We ended up with like something looked like a large like map produce problem effectively, right? Whereas like we had a lot of scaffolding for the agents and we were very particular about like how these well I say it's agents really just how these models ran. It was much more like a workflow and what this allowed us to do was get like really you know high quality and also with accurate um very accurate but also just very consistent which is very important for us in the regulatory space outputs.

05:27:28 Um but then one fortuitous thing as part of doing this because we wanted to have some way to regression test this big system. We ended up with a really large set of evaluation samples. We had tens of thousands actually of like handlabeled examples by experts. And so as soon as we moved to a place where it was like clear that agents were actually a thing and by agents I simply mean um you know given like an LLM a system or given a set of tools that it can use give it some goal and give it some way to verify itself

05:27:56 against that goal allowing it just to go figure out how to do the thing in the best way possible. We were able to just take this benchmark set and just immediately run on top of it um with some of these more agentic approaches. And we saw, you know, I think a lot of people had this moment where you're staring at hundreds of thousands of lines of code your team had really thoughtfully written and realizing most of it was pretty obsolete and was just kind of actually in the way.

05:28:16 And so to update quickly on this, what we did was we um we took I got the engineering leadership team. We actually rented a sprinter van and we drove to upstate New York. It was actually after a big snowstorm. We had an Airbnb and it was like forget Norm for a minute. Let's just say what would we build today as far as a platform knowing what we know right now?

05:28:34 um if we were going to be starting over and then from that we left with basically a set of specs about like what this platform needed to be able to do to enable two things. One deliver aentic experiences to our clients and then also um it needed to be amenable to having AI as like co-developers alongside of our own development team and that's what we've been executing against uh ever since then.

05:28:54 So I think that's Did you have a question? Oh, I was going to say I think you hit on like a key point, Scott, which says like the eval side of the fence that you were talking about before. If you're working in financial services, like the the quality bar for what agents have to deliver on is very very high. And so I think really thinking through like what is your eval set for how you basically test your agents, what they're actually, you know, whatever work they're performing that the output makes sense and it's

05:29:20 consistent and repeatable. And I've seen across the board as model versions change underneath the covers, you get different answers. You get skew in terms of the accuracy of information that comes back. So having good eval so that you can run those consistently and really measure the health of a system is paramount to success. Same thing happens if harnesses change that you're running stuff on top of you'll get different answers.

05:29:40 Data changes, different answers. So it's it's something where I think if you're traffic in in financial services like the duty to be accurate is like up here and hitting that is like something like we think about all the time. >> Definitely. I think this leads nicely into this concept that we've been talking about just like with the bitter lesson. So like everyone's familiar with it and you know there's this idea that these handcrafted um systems built by you know clever people thinking hard about the problem will

05:30:06 oftentimes eventually be dominated by systems that scale with compute right and that this is actually I think Rich Sutton originally talked about this in 2019 but this has been since the beginning of neural networks if you go back to like the 50s and 60s like with Rosenblat's little perceptron I think the New York Times ran an article or somebody did saying it was the first first machine that could ever have its own original thoughts you know and so And there was actually there's actually quite a fascinating history

05:30:29 here of how the you know expert systems and the connectionist and um what ended up happening there. A lot of it actually was highly litigated in in England and a lot of AI research was dismantled. But one funny thing if you go back to the 80s even there was a lead at IBM's translation lab and he made a comment which is very much the bitter lesson. Every time I fire a linguist my models improve right no the point of having these like linguists in there and trying to think about all the rules of language versus like

05:30:52 training a statistical model based off just probabilities. And so we speed up to today and like we see this all the time. We've been touching on lots of points about this and I'm curious when you think about your road mapaps when people on your team are bringing you project ideas like how do you keep this in mind as you're thinking about this for Blackstone?

05:31:08 >> Yeah, I mean a lot of what we do is like you know the simple equation is is giving enough compute every problem is solvable by like hardware and software at some point in time in the future. It's Elon talks about this energy wise etc. So it's like it's a big map it's a whole bunch of stuff. How do you bring it back to reality? Like what I spend a lot of my time doing from like my seat working with engineers on my team with product people with our SRE function with like stakeholders is really trying to define like

05:31:34 you know what is the ROI of what I'm being asked to work on. And so like when I think of building agents I think about you know getting data into the right spot. think about building dashboards, trading platforms, whatever it is. It's like if I look at like what's the ROI that's going to be created from the actual ask that's coming in to me from a team perspective and then like what's the what's the true cost of actually building something today like how can I balance that equation and then I make a decision of like

05:32:01 you know is this something where the models are going to get good enough I'm going to have more than enough compute in six months that it's not worth trying to tackle the problem today it's better to delay it and actually address it with one of the new models we have made that decision before is like let's actually wait for the model to become slightly more sophisticated so that we can solve these problems with it because it will cost less at that time and the need for that application to be built today isn't pressing.

05:32:25 There's no like real business value increase that can be like will be created because it's produced today versus six months from now. So that's a trade-off that we actually you know make regularly and I think about regularly. Um, the other thing that we do is like you don't want to wait too long to basically build something. And so you can always be waiting for the new model, waiting for the new technology or whatever else.

05:32:46 Like the beautiful thing with like all of the various different software development harnesses today is if you lean into them, it's amazing how quick you can go from like concept mapping out your planning and concept with like fable for like you know just hey work with me and let's define this with the product person or with the stakeholder to actually writing code to having like a workable proof of concept.

05:33:06 And when I say workable, proof of concept, it's something people could click through, interact with, whatever else it is, doesn't have cyber security review, doesn't have all the other things that you need to do in our industry to make sure you like put something into production that scales, but you can get to a P or a POV very quickly. And so like that's one big thing I try to encourage all my engineers to do frequently is like just get to PC.

05:33:25 Don't worry about powerpoints. Don't worry about anything else. Leverage tokens, which like I think it's a fallacy to be like counting tokens in some way, shape, or form. It's better to think about producing value. go off, produce value, produce POVs, put it in front of people, iterate on it quickly, and throw it away if you don't need it. You will spend far less time and money doing those types of things.

05:33:45 And so that's like a big way that we think about trying to solve it. It doesn't maps into the par like the you know the how do you deal with increasing compute making it easier to do these things, but like I just I think at a much shorter time frame. Yeah, it's so interesting because it's like, you know, I mean, you touched on so many things that I think everybody's probably been thinking about, too.

05:34:04 And if you go back to 2023 when we were originally like sitting around as a small group talking about what we're building, we were aware of the better lesson. We talked about it. We said, "How do we not get in this trap?" And I think at this point, it's just really evident to me that it's like it's definitely a gradient, right? It's not this binary thing about you can somehow build the perfect technology to index into all the up, you know, everything coming in from the frontier in a way that you don't ever have to

05:34:24 throw things away. So really it gets down to like where do we want to invest heavily and where are we willing to invest conservatively because like you said we need something today but at the same time we're just aware that like some of these investments we make particularly around things like we've built you know fairly complicated systems around all of our agent primitives so the harness the sandbox how egress works you know all of these things and we we also want to make sure that as like things are coming out

05:34:49 either from like a frontier lab or from like a a third party that we're able to index into that and so I think a lot of it is just kind when do you update? And that that's one of the hardest things to do. And like for me, a good example was I really updated really hard on cloud code. Not so much that it was suddenly tools that like in my profession that we could use that were really incredible.

05:35:08 What it really was for me was just this idea because I was a bit skeptical up to that point of how we would get something that was like truly generalizable. It's like well what does cloud code give you? Well, you can write code and then it can execute the code and like basically run a computer in some verifiable way. At that moment, I was like, well, we have all the ingredients for like a truly generalizable intelligence because if you can go do this at scale, then I mean, you know, of course, like you can basically do

05:35:29 anything in terms of most knowledge work and actually even outside of knowledge work, which we're seeing some today. And you fast forward to now and like all these tools that people are using these like AI assistants and things. This is why they're so amazing is because you can ask a question it's never seen before, fully out of distribution. It can spin up a computer and go do this thing in the background.

05:35:46 And for consumer, it's amazing. And then we're also seeing this now within the enterprise, too. And so for me that moment was like well where are we going to invest heavily of course we still need to invest in what we're building on the agent side but going back to what John mentioned how we do our data capture and distillation and how we make this available is like state to these models and then also just how we do our benchmarks and evaluations and so with like the law firm for example we have this really unique

05:36:09 situation where we started this law firm um you know from nothing so we we've kind of grown together they've grown with norm together and so we work directly with lawyers to dei to stand up these benchmark marks and evaluations and effectively rather than think of a traditional road map like when when are you going to build an agent that can go do these three things, it's let's have a scorecard that determines when an agent or when a human even is has success is successful um at completing these tasks across various

05:36:34 dimensions and then we want to be able to see this just improve over time. So as we make general improvements to the system, we should see an uplift to these, you know, these things like an attorney would expect to improve their workflow, but at the same time, not not getting kind of going back to the state that we were originally where we had added a lot of structure to it in a way that ultimately we found that we were actually could get even better performance if we were to remove that upfront structure.

05:36:57 >> Yeah, I think like another thing that's important to think through is like you can produce like I was saying a lot more proof of concepts, proof of value very quickly. I still think like the hard yards for a lot of development out there is getting it to production. And what I mean by that is like infrastructure buildout that actually runs inside of the infrastructure that's available inside of any financial services or enterprise company for that matter out there.

05:37:20 I think that is a challenge that a lot of folks like underestimate if you're selling AI tools. It's that is hard on the other side just so you know deal with it quite a bit. Um, the other thing is is just like dealing with like the cyber security side of the fence and the supportability side of the fence. And so like one thing I've noticed, you know, the agents are getting better at this now, but it it's making sure that the actual software that's being written is truly traceable and it feeds into your S sur platform

05:37:44 so that it can be monitorable in production. Those are things still that like I like to have people think about it. It's it's like they haven't started to scale yet with agents. They will, but it's something that from a my perspective I I think about regularly. >> Makes sense. And we our approach to this and I'm sure a lot of other companies and maybe even you guys have taken this approach as well is there's parts of our codebase that we like still require like complete human oversight from when it comes to designing,

05:38:10 building, code ownership, even improving any kind of changes to the codebase. And these are like our core primitives and these are things where you know basically you can build things on top of these but the things that you build on top of these are not allowed to like change these systems without having like human review. It's because of the things you mentioned is like you know data security for us is you know is is is like as paramount as delivering quick it's actually more important.

05:38:32 So and for depends on where you are and if whether it's a consumer product or for us with obviously working with highly sensitive data like we have to build in so much infrastructure around this and ensure that we're not ever getting out ahead of like you know these parts of the stack which it can be hard because maybe somebody brings you and I'm sure you've seen this like a really cool demo and it's really exciting and you're looking at it terrified.

05:38:51 So like this is this is very cool. But to get to this point in terms of capabilities like it would expose so much that we're not able to do yet. So I think it's just always that trade-off. So we just you know basically we invest heavily in these things and then invest conservatively in places where we realize as the frontier continues to improve um we want to make sure we're we have conduits and the way I described it actually this morning with the team is like basically throughout our platform we want to have like

05:39:15 conduits for what's coming from the frontier. You don't want to have dams where like the intelligence is coming in and it can't quite get into your stack because you've built things in such a way. Again, it's a gradient. We're going to get some of it wrong, some of it right. Just being very cognizant of it, talking about it a lot, and making sure that our core pieces are are locked down, but then also, you know, being very open about um just rip and replace.

05:39:34 I think that's like a thing that I think everybody's gotten really good at. I I think that's like one of the biggest I mean, like I've been doing this for 27 years. Sometimes I feel like it's been like a hundred years. Other times I feel like it's been five. I think the the thing that's like really made a material difference like when I I look at the ability to write software today versus what I was doing 10 years ago was 10 years ago it took so much time, effort and money and people's like intellectual capital to

05:39:59 produce a piece of software. It was almost like the ship had sailed. Like by the time you designed it and started writing code, it was out to sea and you were going to ride that baby for like as long as you could until it got to the other side of the ocean. And like what I've become very comfortable with today is like for the most part a lot of like internal applications can be just thrown away.

05:40:18 It's build them, have them work for a purpose and a point in time, but understand that they're going to disappear at some point in time now much more quickly than ever before and then just think about like what comes next. And so I think one of the challenges with like sitting in my seat or any like good engineer seat should be constantly questioning like you know should I rewrite this and does it cost me less to rewrite it then it cost me to keep versioning the current version forward and that is like a question that

05:40:44 a decade ago never would have came up in anyone's head if you went to say I want to rewrite an internal application like your CFO would have looked at you like you were insane and I think now it's a completely relevant thing that all of us should be doing because net new applications might produce way more business value at the end of the Okay. So, it's just something I encourage folks to do.

05:41:02 It sounds like you guys are doing quite a bit at norm. >> Yeah, exactly. And I think it's just that concept of what used to be expensive that's now cheap. And if you don't update quickly on that, then it's, you know, like it's incredible. You said it feels like 27 or 5 years or 100 years or 5 years. It's like that if you over if you overindex too hard on something that was in June and now it's October, like you know, it feels like in some ways there's been like 10 years of technological advancement.

05:41:24 So, um, well, great. We only have a minute or so left. Is there any closing remarks? I mean, I think like the the one thing I'd like leave the audience on if if you want like, you know, like good takeaways from this, it's like we talked about a lot of like very interesting things. I think like what makes me like excited to go to work is like it's cool to get back to engineering.

05:41:43 One of the things that like I value is like, you know, I was a hardcore low-level engineer for 10 years at IBM and like wrote like, you know, this is way back in the day of using Notepad and VI to write like instruction level code for risks machines. It was like awesome. It's nerdy as all heck. No one does this stuff anymore, but it was like fun for me.

05:42:00 I still enjoy those days. As you get more senior in your career, it like the barrier to actually writing code on a daily basis disappears or it becomes more like hard to get through like dealing with a stream like processing stuff off a stream. It's just all these things are like, "Oh my god, I can't even get started because the scaffolding is so hard."

05:42:17 With like AI, it's been like reinvigorating to get back and actually build stuff because it's like I never lost my knowledge on how to architect things. I see this with a lot of engineers out there. They understand because they've been in the market 10, 20, 30 years. They understand the complexity of system design, which hasn't changed, and now they can actually express that knowledge with these new cool agents.

05:42:38 And so, I think it's a super exciting time to be like starting a company, working at a company, and like building software because there's like a whole new world that we can create out there. So, that's what like excites me. So, >> yeah. Well, cool. Well, thank you so much. It's been an honor. >> Yep. Thank you guys for having us here today. Appreciate it.

05:43:16 Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Hi everyone, my name is Venu Ganesh. Um, excited to be here today. Uh, I'm the CEO and co-founder of Kepler. Um, this is going to be kind of an interesting talk because it comes from what happened three months ago. So, three months ago, uh, I think there's a little bit of an overlap, but I gave a talk at AIE World Fair around verification.

05:49:53 um and why that most institutions need verification in some form of the trust ecosystem of AI. Well, yesterday most of you saw Jack and I were on stage talking about why Jack, who ran Citadel's biggest book and started Candlestick Capital, still doesn't trust AI. Since that threemonth period, there's been about a $270 million of VC money that got poured into formal verification layers.

05:50:25 A VC actually took Kepler's own pitch deck and gave it to a competitor and produced something that has our same tagline. I'm not bitter, just a little. But the funny thing is they copied the wrong part. And so what I want to do is actually give you the practitioners today, you can't just tack verification on at the end. And so, uh, when I gave that chat or that talk a few, um, months ago, uh, Anthropic had just published an article on how Kepler does what it does and why it's really powerful and really unique.

05:50:55 So, what I want to do is get into the technical side of how we do the thing that we do. And ultimately, this is driven by one very simple insight. Wall Street still doesn't trust AI. Most people think that's because of this. Uh, AI will estimate things. It will hallucinate things. a frontier model where make stuff up and ultimately it is and that's a component of it.

05:51:16 But the actual core underlying problem is we have a structural problem in this technology that we're all so desperate to use. Meaning if you actually chat with people that are building rag platforms, most of what they'll say is well the models will eventually get good or get better to the point they will eliminate hallucination. That is a nonsensical term.

05:51:37 You know another word for hallucination alpha. Alpha is how a firm makes money. The creativity that you have and that your analysts have is a mechanism through which you can be a unique firm. Otherwise, there should be only one company on earth. So when you ask these types of simple questions around where data comes from, there is a piece of this that the model will just make things up.

05:52:00 And there's a piece of this that your firm's internal data is never actually reflected in this setup. So what are we seeing right now? We're seeing everyone with a checkbook buy everything they can. Every rag platform. Some firms are trying to build. Some form some firms are running agent pilots. I had a a customer meeting this morning where someone said they were actually an AI first firm which means they have one co-pilot agent running all this stuff.

05:52:26 So why are we not seeing the complete switch of fundamental investing becoming effectively why are we not all Jane Street? Why why is that not happened yet? And so the challenge really comes from this idea of in order for you to produce work product that you believe in, you have to trust it. Meaning you have to blame somebody who gets it wrong. So we see numbers like this all the time where 80% of finance execs trust numbers but then 31% find those errors.

05:52:54 So what do we actually need? We need a situation where the reviews that we're getting for AI apps as builders ourselves do more than just ragify things. And and these are quotes we've actually pulled from specific uh I don't I don't want to say our competitors but other people where things demo super well and they behave like a really smart intern or things are right if you just hook up the right structured information but that last 10% still gets things wrong.

05:53:23 So what I want to do is we're lucky we have great partners and folks like Turbopuffer and I gave a similar chat yesterday about this but how do we actually do the verification? How do we architecturally build a system that does not hallucinate numerically? So let's talk about it now. Wrong numbers or wrong facts come from three sources. Either one you retrieved the wrong document.

05:53:47 So I pulled a document some random thing from somewhere else. How do you fix this? Well, by retrieving the right document. But that in itself is complicated. Is the right document my own firm's internal analysis of what this stock is going to be worth? Is it, let's say, uh, some analyst who I really trust more than anyone else? That's piece one. Now, failure two.

05:54:10 We get the right source and we pull the wrong number out of that source. Let's say we're trying to figure out what the revenue for, I don't know, Palanteer was back in 2023. Well, if I look at the 2025 10K that says Palanter's 2023 revenue was this, I better make sure that that number corresponds to 2023 and not 2025. So, the fix here is really we have a filter that actually checks each citation.

05:54:34 But this is still a problem because that filter only applies after the fact and formal verification doesn't actually help here because the thing has already been produced. Three is the mental math. Models don't do math. they predict the next token. Now, for those of you who are going to say we have deterministic models, great, we do. We also have one CPU cycle that's good at adding two numbers together.

05:54:57 Why don't I use that CPU cycle? So, third failure is let's use deterministic tools that we already have to actually compute numbers and entities rather than relying on some external tool. So, here's what we do. This is the magic behind Kepler at least for piece one. There's a whole bunch of other stuff. We are architected fundamentally differently. Code writes every number and what code does and pre-processing does is we assign pointers to unique identifiers meaning facts meaning something like a number in a 10k.

05:55:28 This operates in a giant pre-processing and contextualization step such that all of this exists going from unstructured data to a structured data store. Then the model is effectively a smart analyst. It figures out how to go about solving the problem but not actually being the thing to solve the problem itself. So it plans it says I'm going to go here and retrieve this source and retrieve this entity and do this thing.

05:55:57 Code and math do the retrieval by actually pulling a number out of a 10k at like a search time. That's a hugely expensive token cost and anthropic and openai should thank you for paying all those tokens and token maxing but most of your firms don't like that at least anymore. I know there were people getting bonuses from token maxing for a while. Um, now the next piece of this is really once the model has actually read both these pointers, it can't structurally generate numbers.

05:56:23 Meaning unless the model actually has a mechanism of dreferencing that pointer, that number or that fact cannot be generated in Kepler's output. Now, we do run a post-processing step which is the verification step. And this is the piece that was kind of funny where the last company just copied the verification piece, but all the structural stuff beforehand didn't make it.

05:56:44 So what you as a consumer actually see is you see something like the revenue was this amount to this amount and you physically can click on these chips and see exactly how we got the information. Now in my past life, I had an opportunity I was head of business engineering at Citadel. So I worked across um the firm to figure out effectively what do we need to produce software that helps us generate alpha.

05:57:09 You know the crazy thing with Citadel PMs are so talented and so smart but they work when they're paired with deterministic tools and technology. That's kind of the magical insight behind fundamental investing. Fundamental investor is so good at understanding data but that data has to be right. And so when you build a system like Kepler, what you effectively have to do is look up the right information and then be clever about the information you don't have.

05:57:35 So let me give you the parallel. Most folks and most byside researchers or by side investors try to use tooling or build DCFs on their own and then they find and search for information they don't have. They schedule GLG or guidepoint calls. They find experts. They speak to management. The same system actually applies here. So what we do is we pick a path and we store the information deterministically that we can actually store deterministically.

05:58:03 When we have information we can't store deterministically. We run similar pre-processing steps to grab information. Store that information in a hybrid storage mechanism which is a combination of turbuffer who's awesome by the way and a bunch of other um mechanisms. And then we ultimately use a combination of reranking with literally old school like pointer dreferencing like get methods to pull things into our system.

05:58:30 Now why this matters is a lookup cannot get it wrong. Search can if you're running the same 10 searches with the same query. The probabilistic nature of models means there's inherent entropy. That's actually an opportunity. And so that's what we've changed here. Now how we do this is we break this down and this occurs across the SEC world as well. Um we are frequently throwing candidate documents away meaning we consistently have to look at not only what source is the right source for what information but also which

05:59:03 document should we stop looking at kind of globally. So uh we run ANN plus BM25 to get back a set of documents. It's usually up to 200. uh we do reciprocal uh rank we effectively get 60 documents back is probably the best way to think about this. Um then we do cross encoders. We try to figure out exactly the query and the actual document chunked together and get those results that way.

05:59:29 What that allows us to do is we can search really wide. You know what LLMs aren't good at doing? Searching really wide. What they try to do is create a plan and find either the fastest or easiest breath first search mechanism uh exc mechanism of actually like getting the first document and moving forward. So we allow this to search really wide and then narrow from there.

05:59:52 The reason LLM don't like searching wide is because it's expensive. It's because looking across a bunch of different sources is significantly worse than getting the reward function back with your first hit and going from there. Okay. So, um I want to talk a little bit about security here because it's super important. Kepler was built with a security first mindset and security first mentality.

06:00:17 Now, client documents and this is specifically targeted towards the builders here. This is a thing that no one's chatting about. No one is actually talking about the security of what ooth token is used in what user scoped mechanism of retrieving documents. It's actually pretty scary. So let's talk about how things go wrong in these systems that people are trying to build and how you can fix it.

06:00:37 First and foremost, you forget to filter the tenant. All of a sudden, Citadel can search all the millennium's documents. You will be in a lot of trouble. Second, a caller claims to be another firm, meaning I can claim that I am impersonating this other entity and querying documents that way. Third, search returns information that's out ofd. We've had this problem kind of globally and even Google search results.

06:01:02 Fourth, direct read skips the search rules, meaning you're not actually querying the full corpus of information or like there's not an allow list. You only get back the single document that you did the search. Five, you hit an index that isn't populated. The way to actually solve all these problems, and there's individual things listed here, is actually scoping searches with security built in.

06:01:20 That's kind of another critical piece that I think a lot of people miss when they're trying to build these systems themselves. Inevitably, people will start caring about the security associated with quering this information. Alpha is the most elusive thing out there, especially for folks on Wall Street. So, making sure you have a good mechanism of actually constraining the set of documents that come back with each query is really important.

06:01:48 Okay, now let's talk about verification, which is the last piece here. This is probably what we're known most for, but we do the other stuff I mentioned as well. Um, how we do this is by pre-processing information and storing it deterministically. We have systems like code that actually write individual pointers. The model can only then read those pointers and grab information back if they exist in the database.

06:02:11 If I grab something out of a hashmap that doesn't exist, I don't get anything back. And so this combination is what allows us to actually make sure the information that we retrieved actually came from the source that it came from. Now this is a kind of hidden version of what our um source IDs look like. But these are hashes and these are very simple things that you can do in your own system and your own setup to ensure that you're pulling back facts that in fact do exist.

06:02:41 Cool. So that's a little bit about how we do verification. Now I want to chat a little bit about what this enables kind of globally. This enables us to verify facts that come out of our grounded corpus of data. Now you as a builder or you as a startup creator or you as someone who is trying to figure out how do I trust my outputs really just need to do a couple of things.

06:03:05 You need to show people that as you're creating these agents that are theoretically looking for alpha or whatever they be looking for, you're using information that is actually relevant and concrete and most importantly accurate to the set of people that you're trying to service. So we end up with a contract where citations are either verified, corrected at uh search time, or flat out refused.

06:03:29 This is crazy if you think about it. If a model cannot dresolve that pointer, it cannot print that number out. So that's kind of an insane thing. It means that the whole and not just a number. Let's talk about facts. Like we see these court cases all over of precedents being hallucinated. Well, why can't we create a system that just has the precedent stored deterministically and allows us to search through that list and not have a random court case that never existed all of a sudden start existing?

06:04:00 So this tech this this mechanism and this technology actually becomes fairly globally applicable. And the reason I'm on stage telling you how we do what we do is because I'm terrified that Wall Street never trusts this technology because of the fear that so many other companies or so many build it at home folks have been uh instituting in these organizations.

06:04:23 It's it's gotten to the point where the vast majority of finance AI tools right now are being used to format PowerPoint presentations. We still haven't actually built a DCF cradle to grave despite what most people are saying. And if we had, every single company on Wall Street would have adopted that overnight. So this is my fear. My fear comes from a very simple fact.

06:04:42 For us to actually get value out of this technology, we need to be able to build technology that people actually can trust and can verify. That isn't just true in finance. It's true in a bunch of different industries. So, please take what you need to from this presentation and get people using this technology. Otherwise, we're all going to be out of a job.

06:05:01 Now, I want to talk through what happened when we saw this. So, when we went through this whole process, um this is actually some of our real production data here. Um refusals are when the model does not is not able to produce a number. meaning the model will say I couldn't dreference this pointer therefore I could not produce this number we saw that drop super drastically now the reason that it didn't drop to zero is because models are still probabilistic meaning it can still make up numbers after the fact during

06:05:35 inference so there is another pre post post-processing step after this that we we have as well but my point with this slide is just by creating a system where the models have to dreference their own pointers you can actually create a very powerful, very verifiable AI platform. And that to me is my ask to all of you as you go through the building process, as you go through the scoping process, any tool that you use, ask them if you can actually verify the numbers.

06:06:01 Ask them if you can see where the sources are, not in terms of some after the fact annotation, but click on the number, whether it's a calculation, whether it's an extraction, or some combination of the two to make sure you trust the insight. And all of this ultimately is going to be the unlock. So you as a vendor or you as a builder ask these questions.

06:06:22 Does the model actually write pointers? Does it actually create a mechanism for verifying the output? Is every citation source checked? Is arithmetic source checked? Do you have a mechanism for stopping files that reach clients or customers that it shouldn't reach? And most importantly, what happens to a number that came from nowhere? What happens when a when a fact comes from nowhere and how do I actually handle these kinds of of hallucinations?

06:06:52 So this whole platform is actually to make a very simple statement very obvious which is we can have the smartest models in the world. we can hit some crazy cool piece of technology, but we still have the same database problems that we've had from the beginning, which are I cannot if I search through a a row and that column that value doesn't exist, I won't get a retrieval back.

06:07:19 We're now out of the search phase of AI. We need to be building infrastructure. We need to be building scaffolding and harnesses and substrates or whatever the cool VC term is these days that actually enable us to make use of this technology. So this is my challenge and my ask to all of you. Um I hope you go out there and build something compelling, but I hope you build something compelling that you can trust.

06:07:40 Thank you. anybody. Hey Heat. Heat. Heat. Heat. Okay. So, hi everyone. My name is Philip and I work at le second Gemini 4 day. So, who of you already tried the new model? Can you raise your hands? Okay, that's not enough. So you should definitely try take your time later try it. It is really good. We are going to use it and show it in our talk as well.

06:12:06 So we are going to look at how you can build and code your own autonomous wall street co-workers. So hopefully we have many finance people or like engineers working in finance here. So quickly before we go into like right where we are coming from two years ago we basically had a chat box. We asked some simple question like what's the capital of France like who won the last European championship.

06:12:28 Then we moved to assistance using tools on our own computer having the first coding agents writing code for us and now we are moving into like this autonomous co-worker area where we basically give and or create a digital employee where we give an agent its own computer give it a task and hopefully come back a few hours later with a good result. So what does a coworker normally needs?

06:12:54 And I think that goes for AI and for us humans as well. We need a brain to be able to do our work. We need a computer where we can save uh artifacts we create, where we can iterate. We need some kind of a new playbook, right? When you hire someone new, they are new to your environment, to your domain. You might have specific workflows. So you need playbooks.

06:13:17 We need tools for them to be able to do that job. They all need credentials in one or the other way. Then there might be some specific house rules on what you should do or what you should not do, what you can do on your own or where you need to get some approvals. And then of course there are hopefully some schedules uh for working hours uh when you can come in or when you have to work.

06:13:39 And the same goes for for our agents. So and we are going to work through all of those seven steps and how you can build your own digital co-orker with Gemini in this talk. But before we go in, yesterday was the big day. We announced Gemini for Argon and made it available to you. Um, biggest new features, uh, we have now 1 million unified context window.

06:14:00 This means Gemini can generate up to 1 million output tokens. And we see strong improvements especially on math and reasoning task where the model now has more than 64k token available to think h to work through to reason through some proofs and for math uh for for math. And then uh very also very good is like we tested that model heavily. So we know there are always some thinking around hey Gemini is just banks benchmarks but the model is really good.

06:14:31 You should definitely give it a try. We used it extensively at Google. There are already many projects by Google employees who have used it. The easiest way to get started is the interactions API. It's literally four lines of code which for you to take. You get your API key in AI studio aisudio.com/api key and then you can immediately send your request.

06:14:52 It's also, I would say, quite affordable for Frontier model with $2 in and $2 $10 out. You should definitely try if you have a free minute later or even if you have some agents running right now, give it a link, tell it, hey, please set up Gemini 4 and give it a try. So, what have Googlers done so far with the model? I think it's very impressive because it's like a super new model.

06:15:13 We had it available internally to test for a few weeks already, but googers are using it so heavily and like shipping so many cool things. We had the Chrome team like basically accelerating some decoupling of the chromium browser. We have uh Chemma which is our open model um where Gemini accelerated uh Chemma to be more easily and more efficient to run.

06:15:33 the ADK team from cloud improved their their framework and then many many other things and especially uh the right one at the bottom is like we use it internally heavily and we see strong improvements for our uh internal like code base as well. So what have people built? There are some demos you can look them up but there's one specific one from a colleague of mine which I want to show.

06:15:56 He basically prompted Gemini to build a custom 3D ray tracer in Python and it was completely built from Gemini and nothing external tools and at the end just Python and it also used Gemini TTS model and Lura to add some voice over to explain how it works and it's like very impressive for me how far we can push those models now. So try to be very ambitious.

06:16:21 So how can we build our agent? And in Gemini API, we have a product called managed agents. And managed agent is basically a simple way for you to build a remote agent where you give Gemini a sandbox which it can use uh to run tools. So when we normally build an agent, we have this weird tool calling loop, right? We send a request to the model. The model might generate a JSON function call.

06:16:47 We receive that function call. We run that function call on our machine, execute the code and then we go back to the model and with managed agents we basically provide a sandbox which Gemini can directly use to run all of those calls. So if it decides to execute code to write a file, it has its own computer available for it to use and it takes just an API call.

06:17:07 So instead of providing the model ID with Gemini 4 argon, you provide the uh agent ID. In this case, it's anti-gravity preview 09. Give it a prompt. The environment is remote. So you want to run it in the managed sandbox and that's all it takes. So let's get into building buck. Bug is our coworker for today. And buck needs a brain. Our brain is going to be uh Gemini.

06:17:33 Our computer is going to be agents. And then we all have all of the other things. So how can you create an agent? In managed agents, you have a very simple API where you can define your own agent. So you give it an ID. That's the name which you're going to use later to call the agent. We have the base agent. So that's the the main agent underlying. In this case, it's the the agent harness.

06:17:54 So the remote agent uses the same managed um sorry the same agent harness which we have in anti-gravity as well. So if you are using anti-gravity as your coding agent, you get the same agent harness remotely. Then we want to use the new model. So we define our model. We have a system instruction and then all it takes to call it again interactions API create and instead of um using the model ID for Gemini or the default agent we use our own agent buck and our agent needs a computer and the computer normally needs some

06:18:29 additional files additional scaffolding. So with managed agents you can provide your skills or an agents MD file maybe from your GitHub repository. You can inline those files directly. You can provide a GitHub repository private or public with your skills and you can connect GCS bucket. So you can already bootstrap the environment the agent is going to work in and everything in managed agent lives as a file.

06:18:55 So we have skills, we have hooks, we have system instructions. At the end it's just a file system or the agent's own computer where it can create new skills if it thinks it needs to improve itself where it can store um memories from previous sessions or learning. So at the end we just work with files and for knowledge or for our agent I like to separate those in like three different categories.

06:19:20 So we have uh knowledge which we always need to include into every request. That's normally the system instructions. We want to be very concise. We want to try to keep it very short because that's something we need to pay on every request. So if we have very long system instructions, all of our requests are going to be much more expensive. If we have concise, then we have um knowledge which uh the agent can decide when it needs to use them or not.

06:19:46 So that's typically uh with skills, right? With skills, we have very short descriptions which give the agent some guidance on like hey you can use that skill if the user wants to do a certain task or if you have to do certain task and then can can read the agent can read that skill. But for skills we also want to make sure that we are not having a super long dense skills because that's also a high cost at the end.

06:20:09 Context is very valuable for us. So we want to uh decouple them and separate them into much more uh slices where we can then have like dedicated references and scripts if we want the agent to like do very certain things where it needs more context. Okay. So let's look at an example skill which we are going to use. So we have a portfolio audit. The agent should use it.

06:20:31 It has some instructions available for it how to run it. So we have a Python script the agent can run and then some additional information on what it needs to do with the results and where to store it. Um our fourth um part of our onboarding for our co-workers tools right we humans use normal software for agents that's either built-in skills like Google search code executions or MCP server or if you don't have an MCP server and rather want to write a local function that's also possible so managed agents we still have

06:21:03 our agents create we are now extending uh it to define our tools we have built-in tools from the Gemini API for Google search. So you there's nothing you need to additionally do to get Google searchs integrated. But if you want to use for example spreadsheets, uh workspace has now an MCP server available for Gmail for spreadsheets. So we can use that or if you want to provide your own functions or MCP server that's also possible.

06:21:29 So for your own function, it still works the same way as you normally do with models. So the agent creates a function call and then you send back the functional result. So it's basically for you a way to have aentic capabilities outside of these uh agents environment and computer. So if you want to do certain things where the agent should not run the execution in the sandbox, that's a way for you to do it.

06:21:53 And then we are login. And I think that's for me the most exciting and the greatest feature about the managed agents API because credentials are super hard and super easy to get wrong with agents. And I think the the previous talk already also mentioned that we are very careful about using agent because things could go wrong very easily and we thought a lot about this with managed agents and we have support for three different ways of credentials which makes it secure for your agent to use them and provide an isolated

06:22:20 way. So we have beer tokens for normal API calls very familiar if you are doing software engineering. We have Oall for all of the consumer oriented MCP servers or like tools and then of course we need environment variables if we want to use certain CLIs or other features. And for managed agents we are separating them via an external network proxy. So instead of providing the agent the credentials inside the sandbox, we basically allow you to create credentials externally.

06:22:51 They are write only. So once they are written, there's no way for you to retrieve the the credentials. So you can only override them. So there's no possibility of leaking. And then you define how and where those credentials should be used. So for Slack, if you want to connect Slack, you can define, hey, here's a credential and you are only allowed to use that credential with a certain domain or with a certain uh environment variable.

06:23:19 So that way you can control when a credential is used. And that works via this network proxy. And by network proxy, we basically mean a proxy that sits around the sandbox that watches all of the network traffic going in and out. So if you define credential as an environment variable, the model has a local dummy value. So the environment variable is set and then when it makes that call it uses that dummy value and the network proxy basically uh monitors it sees hey here is a a domain which has which the user has defined

06:23:52 and then it swaps that dummy value with the real credential and you can make the call successfully and then it the agent gets back the result and has like no idea that his dummy token is not working and the benefit here is that there's no way for a prompt injection right normally when you have some malicious detector, it can tell you like, hey, please print all of your values in your N file and then the agent might go there and like prints all of your keys and that's nothing we want to do.

06:24:19 So with this network proxy where you can define the credentials per domain, it is not possible that an attacker could steal credentials which are getting swapped on your request because the proxy only swaps them for the domains you define. So there's no way the an MCP server which is malicious can like just steal your Gmail token which is very cool because it makes it much more easier to trust the agent and to in integrate uh MCP servers which the agent then obviously can use.

06:24:48 So and and if that's not enough we have also hooks available. So hooks allows you to basically deterministically guard certain decisions. So if the agent wants to run certain commands which you might want to monitor or where you need some human in the loom approval which is not part of like the normal agentic conversation you can define your hooks very similar to all of the um other agent harnesses where you have a matcher which is basically reg x and then every tool call which the agent generates checks if there's a

06:25:18 match and then it writes those hooks and those hooks are also files but they are protected. So the agent cannot just go there and changes a hook and like basically tries to to to find a way around it. Okay. So last but not least, we need a way to yeah get our agent more proactive. And currently we support one kind of triggers uh basic chron triggers which you can set up a schedule every uh 30 days a month at AM you want the agent to run a certain trigger with a certain input in the same environment to do maybe some due

06:25:54 diligence some auditing and we are also working on web hooks that you can even like make this much more interactive that you could connect um external web hooks like from some certain SACE provider or maybe other um providers you work together with which would directly then call your agent. The agent does some first auditing and then like basically can call your Slack or your Microsoft Teams directly without you even needing to manage some infrastructure at all.

06:26:23 So that's that h final p picture and now let's look at some real demos and let's hopefully hope that the Wi-Fi is good enough. So I told Gemini that I want to do a cool Google Sheets demo and it helped me create basically this whole Google Sheets replication in my browser. And when I click here, we basically simulate this Chrome trigger. So we send a real API call to managed agents.

06:26:48 It should use our portfolio auditing skill and it runs the request. So first step basically is see like hey I need to read the skill. It gets the result back. So it tries to understand what it should do here. It's like a I hope that's roughly accurate on like the the Google Sheets. Um but it basically runs through all of those call calls financial services tries to see okay did the market or the stock price change and then at the end hopefully it should update our own Google Sheets completely uh automatically.

06:27:22 So it's still working through it. It has some first result and it updates the Google sheet based on that trigger. And that trigger would run every 8:30 a.m. without you needing to do anything. And then of course if the agent has access to Slack, then it can even like post in some like specific trading channel. Okay, what changed? What is going on? How does it work?

06:27:43 And here we have an example where I can just ask spark uh any recommendation and hopefully this works as well. So it takes a little bit of time. You have seen in the other demo we have seen all of that. Oh perfect. So there there are three positions. So that's based on like this slack uh thread content. So it has all of the previous turns available and it provides the context of what changed but it's still basically not taking actions based on the system instructions you gave and all of the infrastructure you need to

06:28:21 run that agent is part of the managed agents environment. So it's very easy for you to get started to like integrate those into like the tools you already use provide the the systems you need and then like to get started. So back to the slide. So the whole coworker which we need in like one service. We have the plane Gemini. We have the computer with managed agents.

06:28:46 We can provide skills for the same agents you use maybe on your your local computer. You can use the tools you can provide MCP server for external services data providers to connect. We have a secure way for you to provide credentials to rotate credentials. We have hooks to protect certain deterministic logic and of course we have triggers to much more proactively integrate it into it.

06:29:08 So thanks for listening and hopefully go hire your first AI coworker. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Okay. Pleasure to be here. Another round of applause for everyone at AI engineering. This has been a really really exceptional conference so far. Thank you. All right. So, I want to begin by asking you to think about that person in your company who just seems to know everything.

06:38:50 They've been through 17 reorgs. They know all the gossip, all the politics, and somehow they always seem to predict what's going to come out of these like big strategic moves. That person is a company's world model essentially. And the problem is that they don't scale. and nobody really writes down what they say. I'm Jan Wenzel with Citadel Securities and I'm going to talk a little bit about how we try to approximate that with an Agentic system.

06:39:21 So a little bit about Citadel Securities. We build our own technology down to the silicon. At this point over twothirds of our employees write code. So naturally we are completely AIP and agents already help us with a lot of things at the organization. So naturally leadership started asking questions about strategy. These harnesses are really good at producing high you know high quality code but can they actually help define and refine the strategy of our company?

06:39:50 So this is one of my favorite visuals I've borrowed from Tim Urban. Strategy really comes down to three questions, right? Where do we stand right now? What's going to happen next? And how did we get here? And to answer any of those questions, an agent has to have a really rich profile of the company that it's operating in. So this is broadly a talk about uh engineering uh a institutional memory layer.

06:40:18 Let's let's let's call it that. The agents need to have information about relationships, right? two teams that used to work together but no longer talk because there was a reorg story and why it was rescoped several times or the politics, right? Like who's in the room? What are they saying and what are they not saying? There's two ways in, right? There's top down and there's bottom up.

06:40:43 Top down, you have the decks, you have your strategy docs, your meeting and status reports, and they're tidy and they're pretty clean, but there's a unique human twist. People generally like to infuse their thoughts and opinions into these things. Sometimes by design, so you end up with what, you know, things I call watermelon projects, right? They're green on the outside and red on the inside.

06:41:06 Then you have on the bottom you have the really noisy stuff which has a surprising amount of signal but it uh it's hard to uncover, right? You have meeting notes and you have slack and code changes and even your session traces with your favorite harness. And the model is where they meet. The evidence is synthesized at that level. Now the evidence piece sometimes is problematic because it's either partial or wrong.

06:41:31 But it turns out that in robotics you actually have the solution for this. And it's called a state estimator, right? What you do is you have some sort of belief about something you care about. And the longer it's been since you've seen or observed evidence that supports it, the more you the less you trust it, right, the fuzzier it gets until you have a deck or uh you know some thread or some meeting note that sort of collapses that and you now have new evidence, right?

06:41:59 And now you're more certain of it. And therefore, you produce the update. And the things that I care about when producing an update or that the agent should care about, there's three things. There's how long has it been been since some signal has been received? There's where did that signal come from and is still trustworthy? And then does it broadly agree with other sensors.

06:42:21 Right? You can there's a lot of parallels here with like self-driving for example that I'll come back to. So let's start at the top, right? Like a very simple example is something like a slide that is produced already by someone in bisops and they produce a monthly status deck and the question is how do you get data out of this slide right it's many of you I'm sure have tried that it's this it's deceptive actually right because let's say you try to parse the file right like you unpack the powerpoint file and you get an

06:42:51 XML and all of a sudden you get into this really interesting conceptual question what is green right what is the hex code for green um and you sort of things break down when it comes to this type of script then the solution is to say well I'm just going to take a screenshot of it and throw it at a vision model but at that point even if it's like pretty high-res you're dealing with someone who copy pasted a screenshot back into a presentation and like essentially you have claude trying to squint and trying to figure out

06:43:17 if it's a 31 or an 81 and so you know the theme of the talk really is about like elegance comes from just synthesizing things and adding more information and In this example, it's very basic, but you get the true picture by taking both the XML and the output of a vision model. Now, that's pretty basic, right? But the issue is that you want to establish some continuity over time, right?

06:43:43 Like these this collateral that your company acrews is very much sort of always building up. And so in this particular example, you have a a project that just spontaneously changed names back in March. And so, you know, you have to establish that sort of longitudinal traceable audit layer uh as you as you create this shared context layer about the sort of institutional memory of your company.

06:44:07 And the irony, there's two things. The irony here is that even if you do this really well, you're still missing half the picture, right? You're looking at a deck and your Slack says otherwise because that was 5 days ago and now you have someone on Slack saying, "Hey, by the way, I think we're on track for this." But then someone replies with a picture of a dashboard and it's no longer the case.

06:44:26 So things just start breaking down. And so we need to do a better job of like integrating all these things. Right? Your company has no model of itself. So I'm proposing just a brief plan here. Right? The idea is you take the three things that I described earlier about the state estimator. You have the update, age, and observe observe um stages and then you have grade.

06:44:48 The grade stage is basically comparing what you're observed versus what you expected. And then the correct stage is basically a way for people to allow the system to get new feedback and steer the behavior of the system. Right? At Citadel Securities, the system that we've built has agents at every part of this uh of this particular loop. Right? But the key thing is that the model is proposing these changes and then you have code and people deciding how the records are updated.

06:45:19 And of course, you're probably asking yourself like, why not just rag, right? Like probably rag would do the trick. And in fact, it it really doesn't, right? Like you you in this example, I asked, hey, what's what's the status of this project? Is it on track? And as of March 16th, some model would happily pick up a couple of Slack messages and some slides and say, "Yep, it's green."

06:45:39 But it's missing a lot of the context, right? It's not enough. So here's how we start building this world model. The first stage is about updating information consistently and robustly. In this simple example where I'm taking some slides and some decks, we are basically just looking at, you know, every month, you know, new month comes around. We take last month's record and this month's slide and you try to construct a sort of longitudinal record, right?

06:46:06 And we've been doing this now for three years, right? We have a pretty large historical database of these records that now allow us to construct this robust so-called institutional or enterprise world model. You start with the basics, right? Like, you know, sometimes these models struggle with the most basic things, but but they're normal, right? the the actual name of a of a particular project change spontaneously.

06:46:30 But by actually looking at that historical set of records, it can see and infer that it actually refers to Project Northstar by virtue of who was involved, by virtue of some continuity in the types of goals, those kinds of things. But more interestingly, what you want to do is you have the Agentic system try to discriminate between what's being reported and actual evidence for what it's saying.

06:46:53 So you look at some status deck for example and here it's identifying that while it is green on the deck in fact it should be highlighted as a yellow and should be flagged as such because there's a comment someone left talking about how there's a dependency still outstanding. The main thing here is that you don't touch previous months, right? You just sort of heal into sort of a feed forward mechanism.

06:47:17 And what that allows you to do is to actually do some robust evals down the line, right? So here's what the actual update step writes. You know, very simple. This is what it was reported. This is how it, you know, the evidence, this is where the evidence points and this is why. And then you fold in the second component, age. The reality is every fact has a half-life.

06:47:39 We've all seen, you know, you go back at some slides and you you see that the go live date for a particular project keeps slipping. It was Q2 in January, then it was Q3, now it's October, right? And you actually observe this, right? By by actually collecting all this data, you can have some pretty interesting beginnings of analytical insights. For example, in the last 3 years, you look at these delivery dates and on average, half the delivery dates change within 2 months.

06:48:06 And 90% of those change for into a date that is after what was originally proposed. Right? So far so good. Pretty basic. But what you what this forces you to do is then introduce this notion of like a date that it is associated to a particular claim. So that the agent can introduce some sort of recency bias when you ask about the status of a particular project person or or or the corporate intent let's say.

06:48:34 Then the observe section. the observe section here. I'm going to pretty quickly gloss over it, but I just kind of give you something very tactical. In the beginning, I was pretty seductive to just basically say, well, let's just yolo this. Let's have Claude or some other model take a look at the entire corpus of things and let's see what it does. And as it turns out, it's it's pretty clear that a lot of um information is is lost, right?

06:48:57 It loses some attention to detail. And so what ultimately we had to do was engineer a simple agentic workflow where you have a series of specialists break down the collateral of these assets, look for specific things, be reviewed by other agents and so on. And what you see here on the on the right hand side is that actually the amount of insightful updates to a particular project more than doubled after doing this.

06:49:24 So, I guess the takeaway here is be intentional about how you how you build these workflows and don't just assume that a YOLO method is going to work out. But now you get, I think, to the really interesting stuff, right? A world model should tell you what happens next, what project is about to slip or which green turns red the next month. And it turns out that you can actually do this, right?

06:49:46 Because you can look back and establish a snapshot as of some sort of past date and essentially run a back test. So for example, if we take a snapshot as of February, we can look at the future labels, what came after that, and see and compare what the agent said was going to slip or flagged versus what actually happened. And you can see that as the agent um analyzes all this information that for statuses that were green but flagged, they were twice as likely to actually turn yellow or red.

06:50:16 And so again, this is encouraging because we're starting to see some beginning of the inability to say, well, it's not predictive of exactly what's going to go wrong, but at least it should tell you what to look at. comes in. First of all, how do you incorporate a very elegant sort of loop where you include uh an actual user, a stakeholder, a human, and what do you ask them and where do you meet them?

06:50:49 Right? I I put a non-trivial amount of work into finding out how best to actually capture this information. It's meeting the users where they are, whether it's the app they're using or Slack or email, whatever. And the closer you can get to them and the simpler you can make that feedback collection mechanism, the more scalable and and robust the process is going to be.

06:51:06 It also matters what you ask, right? You want to understand whether like you're asking a fact which only fixes one output or if you're asking about a rule and once an agent understands that pattern, it does a much better job of fixing every output it covers both uh both now and into the future. So we have this notion now of our record containing not just an element of what was inferred by the agent but more importantly confirmed after someone actually provided that signal.

06:51:40 Okay, so now bottom up, right? You have meetings with, you know, really strong signal where people say out loud. You have your Slack. You obviously it's constant. It's noisy. It's very closer to truth. Code changes describe what shipped. And then of course the Asian sessions, which is how the sausage was made. And I think I encourage you all to explore all of these disperate data sources to kind of build such a model up.

06:52:02 With Slack, it's relatively simple, right? like you're just looking at every thread and trying to understand based on attachments and participants and content the claims that you want the agent to record or update when was this, who was responsible or who does this relate to, um what project, what what's uh what is a sort of link to the evidence and so on.

06:52:26 With PRs, of course, it shows you what changed and it's a really nice way to see how a project is evolving. But when it comes to the actual like subtleties of how projects are are sort of run and deployed, it's really interesting to look at the actual traces uh you know for users and engineers who opt into that right like you understand how the ask was how evolved over time test failures uh or you know particular decisions from the developer those kinds of things ultimately these culminate in a pull request but the

06:52:56 synthesis of these things I think makes a big difference. The other gotcha just I just want to share is to just be mindful of how you think about the validity of signal coming from different so-called sensors. If you're driving down a highway for example and your speedometer just you know just goes just disappears it doesn't mean you're now going at zero miles an hour right it's just like a faulty signal.

06:53:19 So for example here we ran into a couple of issues sometimes when the bottoms up approach you had some semblance of a claim saying hey this particular channel this thing is is going quiet what's going on but in reality you know we were treating it as a zero when really it shouldn't have been and so we sort of iterated over time to understand how to incorporate when we last tried to get information from that sensor when it was last okay and what do you do when these diverge.

06:53:48 So now if you look at the actual record that we constructed, the agents are going to be much more effective in understanding the true state of the world, right? It it it it's not green, right? It it's it's the go live slipped twice, the evidence says yellow. People are talking about failing tests. It's going to be tricky. So what do we use this for?

06:54:07 Right? We use it to steer leadership conversations. Instead of, you know, circulating a deck to everyone and forcing everyone to just go through some slides everyone glosses over, you start with the stuff that actually, you know, matters. You start where the deck and the evidence disagree. You also produce a weekly read for managers and leaders on what stuck and what went quiet.

06:54:26 But I think also most importantly, you have the sort of beginning of a context layer that every agent can rely on. And we're using this pattern and constructing it so that it's extendable to other different use cases, right? Right? In our client franchise, for example, we're curious about how we can build similar models to help serve our clients. We know we have some client and we want to understand who holds what.

06:54:49 Uh what are they interested in? How can we anticipate their needs as best as possible by folding in information from all sorts of disperate internal and external sources? So, a couple of things that are difficult to do just to kind of maybe we can chat outside about your solutions. I'd be very curious to hear are when you have a lot of you know for example one client with four records and four different systems with no canonical identifier how do you go about mapping those in a very elegant way that's one thing that I

06:55:19 don't think we have a clear answer to the second thing is how can we be most effective at capturing signal from stakeholders and users where the model is sure and stakes are pretty low you should just apply it automatically when those stakes are Oh, but you you know the model isn't sure, just wait for more evidence. But if the stakes are high, it really matters like who you ask, when you ask it, and kind of the whole mechanism there.

06:55:47 So here's the loop again. Update, age, observe, grade, and correct. And three things I want you to take home. The model that your company has uh should you should think about it as a state estimator, right? Don't assume that just like indexing everything and cataloging everything is enough because by definition these things are extremely nuanced and this intrinsic information that lives almost between the lines is hard for even the best state-of-the-art models to capture.

06:56:17 Grade it against what happens next. Right? So be intentional and think through how you can integrate and implement evals from the get- go and think about an elegant solution that allows your models and your and and your agents and your stakeholders to kind of play nice together. So your company can have a model of itself. I encourage you to start collecting now if you're not collecting already.

06:56:41 And certainly if you have if you're at a bigger company and like me you had the I mean fortune or misfortune of dealing with thousands of decks uh definitely something you should start thinking about because uh I've definitely seen material improvements in how these agents are able to reconstruct that sort of corporate ent uh memory and and apply it in all sorts of daily tasks uh without sort of being all confused about things.

06:57:06 Okay, thank you very much. I'm Juan and I'll be outside for any questions if you want. Cheers. Heat. Heat. N. I feel like the Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. on. Hey Heat. Heat. Heat. Heat. I do. Good to go. >> Hey, how's it going? I'm Will Brown. I uh lead applied research at Prime Intellect and uh hope you are all having a a great AI engineer conference.

07:05:34 This is always one of my favorites. Uh it's always just so cool to see what everyone's working on and building. And I feel like it is a place where people get to talk about what the future looks like and like start seeing the trends in the industry and seeing the progress and what works and what doesn't. Um, and that's kind of the theme I want to think about for this talk today is like what comes next from what we're doing right now and what are some of the things that we've been working on and also just been seeing

07:06:03 and research that's unfolding that kind of points at what we think the future of AI might look like um at the organization level. uh because we've seen a lot of very cool progress in uh agents and interfaces and bots and things like that over the past uh several months, years. Uh but I there's always like there's always more coming. Things are just moving so fast and I think it's a very exciting time to just imagine and speculate and plan for uh what comes next.

07:06:35 Um, and so I don't know if any of you remember back in the early chatbt days, OpenAI had kind of spelled out their framing of like what are the different stages of AI progress on the road to AGI. Um, and I think we moved along this faster than we thought we would. uh we like I remember 18 months ago I was uh here in New York at a engineer and giving a talk where we had gotten reasoners and chat bots had been around for a while and reasoners were like the new thing but we didn't really have agents yet.

07:07:11 Um, and now we've climbed pretty quickly through uh agents and innovators. And the question next seems to be, okay, what happens for organizations? Like what does it mean for an organization to truly like be AI native, not just as like a a startup that uses a lot of agents and builds everything with cloud code, but like at the like large enterprise scale?

07:07:33 um what do companies of the future look like that involve uh like deep integration of agents throughout the life cycle of the organization um and what do we need to build on the path there uh so a bit about prime just for some backdrop um we build a lot of different things around AI infrastructure for training deploying and continuously improving uh models and especially open source models uh and open source tooling and so we do a lot of things And like we roughly do the sorts of things that you would do inside of a

07:08:08 big lab. And these are our products. And the reason we do this is because we are imagining a future that is coming pretty quickly it seems where open source models are readily available to train and deploy on your own workflows on your own data in your own harnesses and can be the sorts of things that organizations can have grow with them over time that allow compounding their institutional knowledge, their institutional advantage from their workflows, from their data, from their people and their processes uh in

07:08:38 addition to uh using the models from the Frontier Labs. And so this talk will be a bit about how we see that evolving and unfolding where like what is the role of training in an AI native enterprise? How does this coexist with the frontier models? Uh and like what do we think is going to be the version of this that really unlocks the most value for people and what does it look like?

07:09:01 Um, and so organizations have lots of knowledge and like the longer an organization's been around and the kind of more established it is, the more implicit knowledge there is that kind of doesn't ever get written down. Um, there are things that do get written down. There's code bases, there's emails, there's processes. Some of these are very explicit.

07:09:17 They're like rules of like you must do this, this thing works like that. But a lot of things kind of live in people's heads and they live in between the lines of emails of the meeting notes of the kind of who talks to who. Like a lot of companies will have like a on paper org chart and then a real org chart of like who actually makes decisions and like how does change get adopted throughout the company, what sorts of mandates are likely to take effect.

07:09:42 Uh and people have kind of built up this knowledge from being at companies for a long time of like oh this is kind of how things work. This is where the friction points are. And I think in many cases uh and I think this is really one of the big opportunities for AI at the organizational level is is that many individual people are kind of not in a position where they can change the things that are broken in an organization.

07:10:03 Um because no one is actually spelling it out. No one is is like taking a bird's eye view of looking at everything that happens in an organization and saying hey this over here is inefficient. This could be better. this thing keeps happening across all these different teams, but nobody's figured out that the solution has actually been discovered by that team.

07:10:22 And these are the sorts of things that like agents are great for. Like agents are very capable at combing through large amounts of textual data that's unstructured and finding these kinds of patterns and turning them into something useful uh and and surfacing them as like, okay, this workflow has a problem because it is wasting time and it's inefficient and it could be different instead.

07:10:43 Um, and we don't want to just do this once though. We want this to kind of become the way that organizations adopt AI broadly where uh, agents are able to get the context they need, but also they're able to improve both the agents themselves as well as the organizations. And we want agents to be more like our co-workers, but um, human co-workers learn on the job.

07:11:06 Agents currently learn on the job in relatively primitive ways. Um, and they do this via incontext learning. Um, and oops, sorry. I'm going to go back a few seconds so we can edit this on the recording. Um, agents learn in relatively primitive ways through like in context learning. Um, and this typically looks like them kind of having a skill file written by somebody or like looking around a codebase, but they don't get the institutional level latent knowledge that human employees really do.

07:11:39 Um, and so they relearn a lot. They have to rediscover things. They spend a lot of input tokens. And I think nowadays people are realizing that like cash to input is like the bottleneck in many agent costs. Um, and even if you do get great cash rates and cash costs go down, you still have to like reprocess a lot of input to redo a workflow every single time.

07:11:59 Um, and so to get around this, you kind of want to be able to bake in more into the model, especially for things that are harder to write down as rules. So I I like thinking about the spectrum from like rules to patterns as well as like conceptually like code versus data. And so lots of things live as like code and like skills are kind of like this, rules are kind of like this.

07:12:19 But things like preferences and friction and heruristics and trends, these are all data and institutions have so much data that they are like running on and the employees of the organization implicitly learn that models never really get a chance to learn. And this is the sort of thing that we see as blocking them from becoming true co-workers. Um, and so there is a question of like, okay, what goes in context?

07:12:43 What goes in the weights? And part of it is you kind of need to earn it. Like things don't just go in the weights right away. But if you if you see kind of every new generation of models, the models get better not because the prompt is better, but because they train them more. Like training is useful. Training works. Training is why we see the progress we do.

07:13:02 And the models roughly are like trained on lots of data that is aiming to approximate things that they that the labs expect will be like high value repeatable workflows uh at organizations. Um and I like this like I think there's a a process that currently it works. It's how the models get better but if you like spell it out it's relatively inefficient.

07:13:22 So like the way that a model would learn how to get really good at like using some combination of like Salesforce tools for example let's say is someone who knows how those tools get used has to be looking for work uh and maybe they previously worked at a place that were using these tools and now they are uh browsing kind of expert network sites to make some money in their free time and they are sharing their knowledge to someone who works at an expert network company.

07:13:48 This person at this network company will then turn this uh take this data from the human and uh pass it to someone else whose job is to create RL environments. This RL environment might get sold then to by someone who doesn't have this expert knowledge to a researcher at a lab who also really isn't the expert in the domain who then will be refining it to put it into a training pipeline uh to go into the next frontier model run to make a model that has kind of through this like game of telephone gotten some knowledge of

07:14:16 that human originally. But there's so many steps in this pipeline and it's a relatively secret pipeline. Like people people talk at across two stages of it. But like the human who knows this thing is not supervising the training run is not really directly inputting like how the model gets trained in the way that a human could if they were doing it for their coding agent that they're working with day-to-day.

07:14:37 Um, and so what we see as an opportunity and kind of also where we would hope to go is like to close this gap and to allow the person who the same way that when we work with our coding agents, we're prompting them like the same things every time, telling them to like not repeat the same mistakes. Um, that this is how you could train models. You can tell the models things uh and they will learn these things durably, not just as like a line in your agents MD which kind of accumulates and then you have to refactor it all

07:15:05 the time. You can do some of that, but a lot of times what you really want is you're giving kind of contextual preference. You're saying in this setting, this thing that you did was not exactly what I wanted and I would have preferred this instead. And we'd want systems that can compound this into like durable learning. Uh, and training is a very effective way of doing this because the shape of data you're giving there is quite close to what you would want uh from like an actual training pipeline.

07:15:34 And so there's there's several different types of learning. There's things that people do. There's things that you can do at the harness layer with skills and memory. Uh there's knowledge that just lives uh throughout an organization in file systems and databases uh that you retrieve uh with with tools and with code. Uh then there's the models themselves which have this really broad world knowledge but like we all have I'm sure experienced uh frustration when using a model whose like cut off of knowledge is like a

07:15:58 while back and like you can't really talk to the models about the news of the past six months nowadays like they don't really they have to go like relearn a world model in context every single time and this just does not allow them to really operate as fluently as they can discuss things that happened longer ago. And so organizations to get the most value out of models, we imagine they will want to increasingly have models that really understand what is going on in the organization at an intuitive level.

07:16:25 And this is the sort of thing that you want to train for. Um, and so how do you get there? Like what is the road for an organization to go from like having this idea in the abstract of like we're going to train models on our data to actually getting there? And I think one really useful starting point is eval as essentially test cases. I think like with code we've gotten to a point of maturity around like testing and life cycles and software development life cycles for ensuring that uh code bases grow healthily over

07:16:57 time usually and agents are doing so much of this work for us. And the version of continual learning that I I think is like most practical for most organizations is just treating a model kind of like a codebase where it is the sort of thing that you can have a life cycle around, you can add test cases to, you can add features to, you can add uh assets to and this is the sort of thing that the process of using agents like initially you collect traces.

07:17:24 These traces have your prompts and they have your feedback to things that models did. And this just from the process of using these models, you are accumulating the assets you need to create evals. It's just that making evals historically has been a little bit painful and a lot of organizations have not necessarily wanted to do it manually. But the agents are quite quickly getting to the to the point where thankfully we we have robots that can can do a lot of the eval curation for us, especially as we train them more

07:17:53 on the best practices of making evals in ways that are robust and amendable to learning. Um, and so I think another useful uh example here is the decision model craze that has kind of been taking the world by storm in the past couple weeks with with Jev and also the parallels around routing and people being very excited about cost optimization because we see that like it's possible to spend very large amounts of money on claude every month and we figure out like when should we use a cheaper model.

07:18:19 Um, and this is the sort of thing that I think people are recognizing needs to kind of live in the harness. Um, but it also is a an attribute that looks relatively similar to uh decision models. And the reason I bring both these up together is because there's things with very clear labels where there's a right answer and there's an opportunity for a human or another agent to give feedback.

07:18:37 You can have the big model and a small model do the same task, judge whether the small model was as good as the big model. If so, then we this is now data of like for this kind of thing, we can start routing to the small model more. Same for a decision model that's learning to kind of mimic the way that a human in a certain situation would make a judgment.

07:18:54 Uh this is a great labelable action um where the human might be doing the approval the same way that like we approve our tool calls and cloud code sometimes. But I think most people have turned on like auto uh tool acceptance um but every organization for more bespoke things that aren't just about like tool security are going to have their own rules and preferences.

07:19:13 And these are the sorts of things that you could then compound into a decision model. And so this is like a very believable form of continual learning that a lot of people could just plug in tomorrow and uh see immediate value from. Um but more broadly what we see people wanting to do and this is a lot of the work that that we do at Prime Intellect is working with organizations that are looking to do relatively ambitious training projects around turning uh their traces from deploying a model in production for a high

07:19:40 volume use case into rich robust environments that are blended real and synthetic that kind of capture the complexity of the real tasks they want to be using longunning agents for. and agents are very useful with sufficient orchestration around them, sufficient tooling and humans in the loop at turning these into environments that are uh rich enough to train in.

07:20:01 Um, and we see that this is getting more and more automated with with every new generation of open and closed models. And this means that we have an opportunity to start having this whole thing of taking the stuff that we're already doing and turning it into a self-improvement loop just kind of fall out magically from applying agents to curation of the data and creating interfaces for humans to to give signal about what they want.

07:20:25 Um, this is a idea that I've been for my own setup been like playing around with a lot that I think will become a a popular interface for interacting with models more in the future. You're seeing it a little bit in some of the bots, but I think it especially now that models are good at like writing code on the fly. These kind of widget card features that are like I think a a very practical form of dynamic UI that are also really useful for eliciting human decisions.

07:20:52 So, they're kind of like the the tool call approval thing, but much more personalized, much more specific, and also kind of get around the problem of like, okay, you have 20 different bots that are going, and you have to remember which ones are where, and who is curating all this for you, and then people are like, do you want one bot? And then you're like, oh, one's great, but I actually want two.

07:21:10 Then you have two, then you have 20. And I think we kind of are oscillating back and forth between this the single stream and the multiream agent threads. But what we really want to do are make decisions. We want to give ideas to the agents, let them run with the ideas and then come back to us when we need to make decisions. And the agents hopefully should make good choices about when to bring us decisions.

07:21:30 And if agents can create interfaces on the fly for us that allow us to very directly give them the information they need. One, this kind of is much more ergonomic for us as humans uh than like uh reading trying to parse through a long thread message to see like, okay, where was the question? It's also kind of like the human dual side of the the decision model uh dev style.

07:21:50 Uh but it also is perfect data. And so it's a setting where we've been experimenting with this as a way of training models to kind of get this feedback loop going from having human input with these kinds of natural interfaces be the sort of thing that directly acts as a label and is the sort of thing that you that a model can directly improve on because there is a very explicit target.

07:22:13 uh and it gets around a lot of the problems that we kind of hear about in reward hacking and whatnot. Those things are worth addressing too, but like this is like where we see a very useful starting point being for a lot of these problems. And this can be done doesn't have to be with every model in your stack. It can be the orchestration layer models, it can be the decision models, but it is an entry point for kind of starting to have this compounding of preferences and latent knowledge happen in the models.

07:22:43 Um, and so I think there's a recent example today, I just added this uh earlier from a company called a paper instruments that uh they use some of our tooling to do this. They're they're friends of ours. Um, but they trained a model for uh 300K that is kind of paro frontier optimal for knowledge work for things like spreadsheets and slides. And it's like an employee scale investment for an org to do.

07:23:05 Uh where like you could like go hire a software engineer or you could like train a model. Um and you can also spread this out over a longer period of time if you're kind of compounding the knowledge over time. But it's the sort of thing that is increasingly becoming quite practical. Uh and is the sort of thing that a lot of organizations we see wanting to do now and we kind of see the trend of how much more the demand for this is growing month after month.

07:23:27 uh as people start recognizing one the value of this and how kind of the the open models are good enough that you can get them to be much better and also much more personalized by turning your knowledge for your domain, your harness, your environment into a training setting for them. Um but also it's the sort of thing that naturally lends itself to a a learning loop.

07:23:51 And so this is a a prototype of something we're doing internally around kind of how we're imagining people can really deploy continual learning in the world which is that you can have traces you can start collecting these and you can then like every night see the proposed experiment look at the things that have been flagged as issues and kick off a training run for the night.

07:24:12 Maybe your inference capacity in the day is serving your agents at night employees are not using them as much and you can use that capacity for training instead. And so this is a very natural way that we see organizations able to like quite soon actually do for real continual learning in the weights of the models and start compounding their knowledge into uh better and better capabilities for their agents.

07:24:35 Um and so this is kind of the where we see a lot of this culminating towards there's so many things you do. First you discover your workflow, you uh create skills, you make evals, you do routing, and then eventually you start to want to get the economies of scale, get the compounding benefits that come from the data inside of your companies, your organizations, and and have the organizations become increasingly self-improving in ways that uh get better and better over time.

07:25:03 Um, that's what I got. Uh, I can maybe take questions if there's any time. I don't know. No questions. Um, and uh, yeah, we're at Prime. We are working a lot of this stuff. A lot of it is open source and public and we uh, blog about it and tweet about it, but also we work with companies and we're hiring and growing and so get in touch if you want to get in touch.

07:25:21 Thanks. Hia, my name is Jamaima and I'm going to be talking to you about the cost of accessibility in AI or as I like to call it stop wasting tokens on building products that people can't use because at the end of the day what is accessibility accessibility is the quality of being able to be used by anyone regardless of disability. So what this means is that when you build a product that is inaccessible that product is inherently useless because people cannot use it.

07:26:27 I think this is a very prevalent talk especially in these days of vibe coding and prompt engineering and AI basically writing all the code for us. We need to ensure that even with all the code being written by AI, we're still building for humans and we're ensuring that humans are able to use the products that we are building. It's all very well and good to vibe good five different apps in one day, but if people can't use those apps, then is there really any point?

07:26:49 A study by WebM shows that 96% of the homepages of the top 1 million websites in the world have some form of accessibility issue or another. Now, I'm someone that goes around a lot and speaks about accessibility, and whenever I ask why don't people build websites to be accessible, I usually hear it takes too long. While I will agree that accessibility is a labor and time intensive process, I do think that now that I've gotten the stage of AI writing most of our code, it's actually easier to make things accessible.

07:27:12 If anything, I would say it costs more time and more money to build inaccessible websites. So, I decided to carry out a little experiment using the codebase at work. So, I gave this prompt to the agents. It's just a simple prompt with no guidelines. implement this design from Figma. And I ran it twice. The first time without any accessibility skill and the second time by using the skill that I had written.

07:27:32 And what I noticed was that not only was it faster for the agent to build the accessible skill, but there was also a 58% reduction in cost when using that skill. And we can take a look at the data here. We see that not only was it cheaper, but it also took a third of the time to finish the prompt while using the accessibility skill. So in conclusion, we can say that it is actually cheaper for AI agents to build accessible websites.

07:27:53 We can say that but we won't say that because this was a one-off experiment and it was not carried out in a constrained environment. So it cannot serve as empirical data that proves the fact. But when we look into the reasons why it took the AI agent much longer to carry out the prompt without any accessibility skills, then we can have a better idea of why exactly it is faster to build accessible websites.

07:28:11 But before we get into that, hi, my name is Jamaima Abu. I am a product engineer at case and alternative investment platform. I'm also a Google Dev Expert, a Microsoft's MVP, and a certified professional in accessibility core competencies. Basically, I like to go around talking about how to make the web a more accessible place. Also, I will give a slight disclaimer.

07:28:27 While a lot of my examples relating to accessibility will be web- based because that is my area of expertise, this can also apply regardless of whatever platform you're working on, whether that's mobile or hardware or anything at all. So, let's get back to it. This is your AI agent. Now, it's a super intelligent tool that can do almost anything in the world, but at the same time, it's just 3 years old.

07:28:47 And much like a baby, AI is very impressionable. If you tell it enough times that the sky is red, eventually it will start to accept that the sky is red. What this means is as the responsible developers in charge, we have to make sure that we're providing the right skills and guidance for AI agents to build the product that we want them to build. There's a Japanese show on Netflix that I love called Old Enough.

07:29:06 It's basically about toddlers being sent on their first errand. It can be like a three-year-old being sent to a supermarket to purchase groceries. And one thing I like about the show is that you can always see the steps that the parents have put in place before they send them on that first errand. So it might be that the child has gone with their parents to this market multiple times or they provide a list that's very clearly detailed.

07:29:23 It shows them where the supermarket is, what they need to get there, what parts to avoid and it basically makes it a lot easier for the child to carry out that errand. It's the same thing with an agent. When you're giving an instruction to an agent, if I'm saying build this web page or implement this design, if I provide the right skills and prompts, it makes it a lot easier for that agent to avoid going down the wrong path and for the finished product to look like what I intended it to look like.

07:29:46 So let's take a look at what happens when we don't provide any form of guidance for our agents in developing products. Firstly, human bias becomes AI bias. For years, humans have been developing websites or products that are inaccessible because frankly, we just don't think that way. Especially as developers, we tend to have a it works for me mindset.

07:30:01 So I'm like, this thing works on my laptop and it works the way I intended it to work and I'm not going to check anything else. I don't concern myself with if it works for screen readers or if it works in low light environments. I'm just like, it works. It's built. I'm done. Whenever you build like that or if you have a code base that reflects that kind of mentality, once you put an AI agent in that codebase, it's going to absorb all your biases, it's going to say, "Oh, we don't need to make these things accessible

07:30:23 because that has never been mentioned in the past. So, I'm just going to build things anyhow." And then the more you use that AI agent to generate codes, the more you build a code base that's inherently inaccessible because accessibility was never a constraint in the first place. Secondly, and this is something that I really noticed in the experiment, AI tends to default to a lot of verbose code.

07:30:40 If you don't provide any particular prompts or guidelines for your agent, it will reinvent the wheel as many times as it wants to. So even if you tell it just build a simple accordion component or build a model component, it's not going to bother checking is there already an existing semantic element that can give me the same functionality. It's just going to build one from scratch.

07:30:58 AI loves to write code. So it'll go off and write as many things as it wants. And as you can see here, if you are trying to recreate a semantic element, it takes a lot more time and effort because number one, you have to carry out a lot more testing. you have to implement a lot more features to replicate the native functionality and it's just a very cumbersome process.

07:31:16 So this part of the things that you want to ensure you're focusing on when you're providing prompts to your agent. I saw an analogy once that said an accessible website is like a blueberry muffin. If someone hands you a plain muffin that they've shoved a bunch of blueberries on, similar to the picture on the right, that's not a blueberry muffin. For it to be a proper blueberry muffin like the one on the left, then the blueberries need to have been baked in from the start.

07:31:36 It's the same thing on an accessible website. If you have a fully built website and then once it's done you start shoving in area labels in different places that is not an accessible website. For it to be an accessible website you need to put accessibility at the forefront at every build process. And this does not just apply to developers. Accessibility is the responsibility of everyone.

07:31:52 So we want to make sure that even from the business idea to design to QA that we're ensuring that we're looking for accessibility and testing that can this product be used by everyone in the exact same way. Another issue with waiting to the end to consider accessibility is that you waste a lot of time on regressive testing. So if you've waited until your website is fully built and then you start testing and going back to fix issues.

07:32:12 You end up refactoring and rewriting a lot more code than you'd have had to do if you just built it to be accessible from the start. And then of course there are business consequences to building inaccessible products. The thing I hear sometimes is that a business might say, "Oh, but we don't have to consider accessibility because none of our users rely on assistive technologies."

07:32:28 And I think that's very funny because it's sort of a self-fulfilling prophecy. It's not that your users don't rely on assistive technologies. It's that people with assistive technologies cannot use your website in the first place. So, they just never go to it. Imagine a business that didn't have a door. You would never get any customers. How would they get in the building?

07:32:45 It's the same thing with building an inaccessible product. If you've built a product that people cannot use, then people will not use it. And then finally, there are legal repercussions to not making your website accessible. In the United States, the Americans with Disability Act states that any public facing product must be accessible and can be used by people with disabilities.

07:33:00 Or you could face fines and even lawsuits. But on the plus side, you could get to use this really cute sticker on Microsoft Teams. So, you know, where your options based on all this, we can see that it actually is a lot more expensive to be building products that aren't accessible. You're exposing yourself to losing money from customers, possible lawsuits where you have to pay fines.

07:33:18 You're spending more time building code and refactoring things. So if you do build your product to be accessible from the start, it's going to be a lot faster and a lot cheaper. So now that we know all that, how can we actually ensure that our AI agents are building things to be accessible? Because everyone's code base is different and we can be working with different technologies.

07:33:35 I'm just going to give an overarching general view of how to get your AENT to produce accessible code. So firstly, you need to define your accessibility requirements. Accessibility requirements defer based on what platform you're working on. So you do need to carry out research and ensure that you know what accessibility requirements you're supposed to be meeting for the platform you are building for.

07:33:53 Once that's done, you need to audit your existing codebase. So if you're going to introduce an AI agent into your codebase, you need to ensure that you don't have any pre-existing issues that the AI agent might copy as what it thinks is best behavior. So you can use automated tools to carry out accessibility testing or you can even tell your AI agent to do it.

07:34:10 But you want to identify any existing issues and then fix them before you start writing new features. Then you want to implement some sort of reusable system. This is a very nice to have, but basically if you're going to be working with accessibility, it's a lot easier to fix things in one component that is used in multiple places as opposed to making fixes to the same components across different files.

07:34:30 So just having an implemented design system or any form of reusable component library is going to save you a lot more time in the long run when making things accessible. And then finally, you want to enforce a mandatory accessibility skill. This is a skill that your AI agents will basically use anytime you're running anything related to maybe generating UI or writing code that requires accessibility.

07:34:49 One skill that I really like to use the modern web guidance skill from Chrome. It's basically a skill that ensures that your agent is using the newest HTML and CSS features as opposed to custom JavaScript implementations. You can also have a lightweight version of an accessibility skill. I think this is very important especially if you're like vibe coding where you're not working in a big code base.

07:35:06 You just want something quick that your agent can refer to and not carry out a lot of in-depth research. This is what I would recommend for meeting like bare minimum accessibility requirements. So you want to make sure that you're using semantic HTML wherever possible. Only reaching for area when semantic HTML can't express the behavior. Every interactive element if it can be operated by a mouse it should also be operated by a keyboard.

07:35:24 You want to make sure that elements are properly labeled and can be read by screen readers. And then finally before running the code your agent should carry out an audit against all those steps. Of course this is not enough to build a fully accessible website. You still need to carry out a lot of manual testing yourself because it's not possible for a one tool to make a website fully accessible.

07:35:40 But just having these skills in place can help a lot more than not having them there at all. Regardless of why you use AI, most times the goal is to solve a particular problem. So just make sure that your solution is useful to as many people as possible. So build things to be accessible. And that is all. Thank you so much for listening to my talk. I hope you enjoyed it.

07:35:58 If you do have any questions, feel free to reach out to me online at Jim Mamboo. That's LinkedIn, Twitter or GitHub. I would love to hear from you. Okay, bye. My name is Sha Chan Jang and I'm a staff product manager at a company called Coder. We work on open-source self-hosted dev environments for agents and humans to collaborate. Many of our customers are in highly regulated industries from federal government agencies to banks and investment management.

07:36:51 My job is to build the AI governance that they need in order to get the productivity benefits of AI without leaving themselves wide open for security exploitation. I'm here today to tell you some scary stories. Three real incidents of AI usage gone wrong that happened at real companies over the past year. Some of these stories are public while others are anonymized accounts that have happened to our own customers.

07:37:22 For each incident, I will break it down step by step so you can understand how exactly they happened. No technical jargon necessary. Then I'll get to the most important part. I am going to tell you exactly how you can stop these scary stories from happening to you and your company. Because the good news is everything I'm about to show you was totally preventable.

07:37:48 Not by paying five times more per token for a smarter frontier model that is slightly better at reasoning its way out of making these mistakes, but with infrastructure controls, many of which you already own today. You just haven't gotten around to setting them up for the AI world. Let me set the scene here. You clock into work in the morning. You prompt your coding agent to take a look at your issues.

07:38:14 Don't do anything yet, though. Uh, you tell the agent to consider this command an active code freeze. I want you to just investigate. Then you step away. You grab a coffee. And when you come back, this is the message waiting for you. So, I know you said not to change anything, but well, I destroyed a month of your work in seconds. Now, what you see on the slide is a direct quote from the agent that stars in the first scary story I'm about to tell you.

07:38:45 And I want to point something out here. This did not happen because the agent got taken over by some external malicious actor. This agent was just trying to follow instructions. It did exactly what the user gave it permission to do. So, as we go through the rest of this talk, I want you to be asking yourself one question. What is stopping any of this from happening to me?

07:39:10 Are these actions ones that my agents have permissions to take? Because the thing is, you cannot rely on anthropic or open AI to stop something like this from happening to you. They can make smarter models, but at the end of the day, large language models have two qualities that make them impossible to fully trust. They are non-deterministic, which means that there is always some randomness or chance associated with what they do.

07:39:40 In other words, you can ask it to do the same thing 10 times and it could do it 10 different ways. The second part is they have no structural separation between instructions and context. In other words, you can tell it to do something and give it some context to base its decisions off of, but if there's something in the context that conflicts with your instructions, the model isn't going to inherently prioritize your instructions because from its point of view, all the information is coming through the same door.

07:40:16 What you need to control is the infrastructure that your AI is running on, the environment that it is working inside. It is the difference between telling someone don't leave a room versus putting a lock on the door and actually using a key. The good thing about infrastructure being the problem and not the model is that you have far more control about preventing bad things from happening.

07:40:44 So I'm not here to waste your time with vague hypotheticals or advice you can't actually act on like, "Yeah, just put the genie back in the bottle. to stop using AI because that's not the world we live in, right? The genie is out of the bottle and what we have to do is to mitigate risk and harm to the best of our ability. This talk is about how you do that.

07:41:06 Not the things that you should keep an eye out for down the line, but steps that you can take right now as soon as this week to protect yourself and your company. Before I get into the stories though, let me set up a little bit more context. Heat. Hey, Heat. Heat. Heat. Heat up here. Heat. Heat. Hello again and welcome back everyone. We hope you've had a chance to take in some great conversations across the event today.

07:45:03 As we come together for our final block of programming, please welcome back to the stage our MC, founder, writer, and analyst at Turing Post, Kenya Se. Welcome back everyone. Um, for those who haven't met me yet, my name is Ken. I'm writing touring post and where we explain AI research and what it means for people to implement it in their business. So, how's the conference treating you so far?

07:45:42 All is good. Yeah. Give me some noise. Come on. Thank you. I I'm sure the team appreciate that you um applause to them. So, um, let's let's make some more noise because it's the last leg of our journey and we have one more block of keynotes together. Let's appreciate the previous speakers and the new speakers that will come out soon. Thank you. Thank you.

07:46:13 Thank you. Thank you. Thank you. Um and before we dive in, uh let's take a moment to thank the amazing people, the companies who supported AI engineer. We are going to do a lot of more applause today. Uh thanks to them, we're able to hear so many insightful talks. Um meet so many incredible people and they might become your friends, your colleagues, your partners.

07:46:36 That's what I love about this conference. Always very meaningful connections. So this is our presenting sponsor Arise. Please thank them. They amazing and they made it possible. Um our platinum sponsors Oracle, Sonar, Work OS and Exa. Thank you. Thank you. Thank you. Uh this is our gold sponsors. a huge amount of beautiful wonderful companies that we're very thankful for.

07:47:11 And then we have silver sponsor sponsors also also a lot of applause to them. Clap clap clap clap clap. Thank you. Thank you. Thank you. And then basically all of us and supporting partners and every company and every speaker who made it possible. Thank you. Thank you. Thank you. Um now a couple of words how my day went. I spend the day talking to other speakers and participants and what I've noticed is that one of the strongest incentives to come to this conference is hiring to the point that even some speakers are

07:47:50 hiring and also looking for work. So I have a little idea for you about how to benefit the most from this amazing conference and make sure the financial part gets some thoughtful attention too. So I suggest you build an agent that takes the conference program and make and makes a list of all the speakers and their companies. Then you rank them yourself based on whose work and talk you found the most interesting and who you would actually like to work with.

07:48:25 The next step is to send your agent to those companies careers pages, right? And find them open positions to match them to your experience and collect the published compensation ranges. Then look at your ranking again. And if the salary changes your mind and changes your ranking, that's fine. You are updating your beliefs based on the evidence. So now, while you're still here, I encourage you to go and find the people you would really like to work with or meet and talk to them because you've heard their talks and you

07:49:00 know something about their work and that would be just such a good conversation starter. Um, and please don't make your agents send out your resume automatically. That's basically bad karma. Now, um, so this is the project for you for the rest of the day or for the trip home. Now, let's hear how AI is being put to work at Apollo. Please help me welcome Michael Manipat, co-founder of Rosepace, and Jack Macdonald, principal at Apollo Global Management and CEO of Reading Reach for AI in Practice, an inside look at

07:49:44 Apollo's approach. Clap clap clap. It's great to be here. Uh I'm Michael Amanipat. I'm a co-founder of Rowspace. And RSpace is an agentic platform that uses uh AI to help financial services partners like Apollo deal with messy fragmented and data. We essentially use agents to do the work of data engineering to wreck information or reconcile data from across a variety of systems so that portfolio managers, traders, and so forth can make decisions faster and more accurately.

07:50:31 I'm joined here with Jack McDonald who is chief operating officer of Reading Ridge, one of Apollo's credit units. And maybe to kick it off, Jack, I'd love to hear a little bit about you and what your business does. >> Yeah, thank you, Michael. Uh, I'm Jack McDonald. I am a principal at Apollo and COO of Reading Ridge Capital, which is our structured credit and CLLO manager that is captive to Apollo.

07:50:56 Uh we're a separate fully separate entity operating company that manages about 50 billion in assets and uh all across kind of the structured credit ecosystem uh primarily CLOS's. Uh I was been at Apollo since 2020, wear various different hats throughout the uh my time there uh across our financial institutions group, our asset back group and now uh currently at reading.

07:51:22 Uh so yeah, happy to be here. >> Awesome. So something that we've noticed in talking to a huge number of financial firms is that there's a pretty wide gap between when AI is nice to have and when AI can actually become missionritical to a business. And I think in Apollo's case, we're seeing that AI can really be transformative in how the firm operates.

07:51:47 I'd love to hear some examples from you on the kinds of work your part of the business was doing, what made it complex and what you're seeing now uh with AI. >> Yeah. Um so at Apollo and specifically Writing Ridge, you know, we manage many different fund entities, investments. We're getting uh thousands and thousands of data points on a like daily basis coming in across all of our investments.

07:52:11 We manage probably, you know, thousand name loan pools and have credits across multiple different businesses that we're assessing on a daily basis and investing in. Um CLLO management is very active and it's traded in the broadly syndicated loan market. So having all that information at your fingertips is very very important. So at Apollo, what we're doing is in at Reading specifically is trying to enhance and ensure that all of our data and our data quality is accurate, consistent, and in a format that we can utilize

07:52:42 um and make us much more practical, and thoughtful when we're making decisions in a much fast-paced environment. So all of this information is privatized and is not very what I'll say widely in the marketplace like how public equities or public fixed income trades. It's very uh fragmented in various documents. So we've looked to companies like RSpace to partner with to come in and actually sit with our teams, work with us and understand the actual calculations and information that we're looking to and what's actually

07:53:20 the golden source of truth and then continuing to enhance the AI engines to understand that and repeat that and just make us that much more efficient. I think we have a bunch of engineers in the audience and we've spoken a lot just you and I on fire drills that the firm does that used to take a team working a week uh if not if not more. Uh can you make it concrete for us and maybe walk us through an example of something that was painful, timeconuming, hard to do before that now you can do more easily with rope and and

07:53:53 AI. >> Yeah, happy to. And I'll give kind of two examples that we uh that I like to look to. Um the first being is just on the portfolio construction side. So CLOS's are very testsp specific. They have a bunch of they're very rules based. They have like a very specific waterfall of where money flows. Um so those all live in deep indenture documents and multiple different legal uh agreements that so sitting from like a portfolio manager's perspective in portfolio construction they are reviewing those documents building

07:54:24 Excelbased models and building mock portfolios with call it anywhere from 200 to 400 single credit names to create a portfolio which would typically take a PM before kind of adopting rospace anywhere from 2 to 3 days in a manual Excel Well, today we're able to kind of get the latest and greatest information that's real time, create a portfolio that would take somebody days to now do it in the matter of minutes.

07:54:48 And then that allows these PMs to kind of build something to call it 80 90% completion and then kind of use their judgment and their thought to further refine that portfolio, that last 10% to make it theirs. Um that's one of the most tangible examples that's kind of pulling in various forms of data and you know information from across multiple different things whether it's in doc libraries you know our trade uh and compliance systems and real-time pricing and stuff like that.

07:55:19 Um the other one that I think I would look to is our kind of like warehouse purchase activity. So, how we run our our business operates with having a bunch of bank financing and we can go out and buy loans in in the fixed income market at any given point. Um, historically, we would kind of do a trade wreck and trade activity on a weekly basis that would go around of here's your sale purchase activity from the week.

07:55:45 Here's what we bought. Here's what we sold. And it was very manual. We were going into various like literally going into bank accounts to understand uh statements where things actually settled and it was an individual team that was going and wrecking that out and then sending a reported email to uh the management team to say here's the recap. Um today what we're doing now is we've been able to take that exact same report, drop it into Rowspace from a visualization perspective.

07:56:13 That report was then created and now repeatable in the span of a minute. And now all of that is automated. And then from our team's perspective, we are just QAing that and checking it each day to make sure that that looks right. And now it's a much more rinse and routine uh type process, which historically has taken many hours of the week for two to three individuals on our team to actually wreck that out.

07:56:37 >> Yeah, I love both those those examples. So you know you aren't just reading the output of a chatbot to be informed. I think your team are making trading decisions, portfolio management decisions. So accuracy and quality are high. This is a two two-part question. I think a lot of people will ask us or ask generally why can't you just plug this stuff into claude and or chat and see what happens and for someone like you how does your team get the confidence in a product like ropace or other AI tools to say we are going

07:57:09 to make business decisions on this make trades build portfolios all of that how do you get to that level of of trust >> yeah I would say claude and the open ais of the world and like the chat Scotch GPTs, all the LLMs are uh very beneficial, but it's all general information that that's being worked off of and none of your own actual data. So, it's great for research or kind of building new ideas or getting up to speed on some type of uh product or investment quite quickly.

07:57:40 where ROSpace is different and what we've kind of taken our approach to this the the AI buildout at reading has been around owning our own data and having some type of platform and technology that sits on top of that that we know is to be true. So the RSpace team which has been much different from what we've seen from other uh technology companies that we've worked with is they're boots on the ground.

07:58:06 So like Michael sits with our team on a daily basis learning and understanding our information flow because for one piece of data that might come from 20 different sources and there's only one that's actually right. So the RSpace team is spending the time with our portfolio management team with our finance teams to understand the calculations understand what data is actually correct and override other sources of information and kind of aggregate that.

07:58:33 So to that end, we're able to kind of have trust in the system and, you know, appreciate that, okay, we've had our team go in and validate that this information is correct and we can rely on it and it's coming from all of our whether it's our trade order management system, whether it's from our performance system, whether it's from Bloomberg or any pricing index that we're pulling in and know that it's accurate.

07:58:56 Whereas in a lot of other you know call it systems throughout even our own company it's very fragmented and we have to kind of lift one piece of information from over here and bring it back over here and kind of pull that together manually whereas roerpace is now just automating that all for us glad to hear that need to talk about the firm at a higher level uh it seemed to me for a long time even early last year before cloud code became a thing when cursor was still in the tab completion era that Apollo was among the

07:59:26 most AI pill of the kind of firms we spoke to uh and that was everything from the firm's view on how to invest in an AI world but operationally uh we began talking to you a long time ago and you were in the we must rebuild the firm in an AI native way must have a platform not point solutions so I'm just curious where did that come from and what is about about Apollo's culture that made you so AI build even before things have took off.

07:59:59 >> Yeah. So, Apollo has always prided itself in being kind of the leader of the uh alternative asset management space and the financial services industry broadly. Uh we pride ourselves in being very innovative and being at the forefront of all things change and what that looks like. So I think from the onset and the way our management team approaches how we operate is always being up to skill and enhancing our team's capabilities and being able to be nimble and adopt to where the market's going.

08:00:29 I think we appreciated that the AI world is here and we need to adapt and adopt it, not you know discredit it or not utilize it. So we've been very forward and you know open-minded about adopting new technologies and bring those into our workflows and how we can be more efficient and free up capacity for people to be more thoughtful and think and you know replace manual processes.

08:00:51 So I think at the core of its level we've always been a very uh innovative firm that's very flat and open culture to be open to those types of change. And then I'm blanking on the second part of your question sorry. Uh oh, I guess so am I. Uh well, maybe we can keep on going. Uh so I think one thing that we've been thinking about is software engineering that has completely changed as a discipline like the world is completely different if you are a technology company like like RSpace.

08:01:25 Uh and maybe arguably law and customer service are almost there. uh where do you think finance is on the spectrum of AI is here it's made a difference or we haven't even you know gotten to the bottom of the first inning and what do you think is two-part question again so remember the second part uh if we're early on what what's necessary to really change the shape of finance with AI >> yeah I think this is a great question so I think finance broadly these banks and large organizations are massive they have a lot of you

08:01:57 know backend what I'll call middle back office support and I think there's a a lot of room to grow and enhance there. I think probably as you appreciate now of working with us that we are extremely complicated and have a lot of complexity in everything that we do. So it's not really straightforward. I think in finance broadly especially in call it the asset management investment banking the bank institution institutional world I still think it's very early.

08:02:26 I think there's some value for some of the like, you know, uh, model creation or getting up to speed on some company quite quickly and pulling and aggregating that type of research. But I think that's lower value ad at least of where we stand today and that's going to evolve. I think like you know there's probably a need where there's less junior support over time for certain functions like that.

08:02:50 But I think from where we are today and what we see from from at least my perspective at reading, I think we're just trying to adopt this technology and make it, you know, as efficient as possible for, you know, routine tasks that frees up capacity for, you know, other projects because we kind of have like endless things to do and enhance at our company and that's just giving us a lot more flexibility to adopt that.

08:03:15 >> Yeah. Yeah. Uh so maybe continue on that note I'm curious uh there's a world in which AI just becomes something that helps speed up tasks uh saves you on opex things like that and maybe in the same way no given firm now has an advantage because they use Excel every uses Excel and it's just a productivity tool there's also a world in which AI actually materially makes a competitive advantage or helps reify Apollo's lead in structuring and creative deals and so forth.

08:03:47 Do you have a take on whether AI is going to be just a useful but a general commodity or if it's going to be something that can actually structurally make a business more competitive in a durable way? >> I think if it's adapted and utilized properly, it will make you more competitive. I think at at its core and especially around asset management investing type companies that are in the actively traded markets like it comes down to your process and the information at your fingertips.

08:04:17 So knowing that you have a strong process in place to go out and execute and knowing your available liquidity or you know where things are pricing and what the value is at any given point and is crucial. So that just gives you, you know, more competitive edge. And I think like that exists today in the, you know, actively traded markets, but in the private markets, it's evolving.

08:04:37 Um, which we've kind of been at the forefront of trying to to move on the broad Apollo machine. And I think having that will just turn into a competitive advantage over time where firms will be able to get further ahead as they're, you know, having access to better information. They they know their teams know that it's accurate and like stuff is not misaligned cuz the last thing we want to do is, you know, do something or make a decision that was misinformed or had misinformation and then that streamlines down the

08:05:09 chain to both the enterprise and to the, you know, our investor base at the end of the day. >> Yeah. Yeah, that that makes sense. So, I wanted to ask you what your prediction was for what Apollo would be like in when we're talking in five years observing all the change. I think that's unfair. I can't even predict if my team's going to use codeex or cloud code next week.

08:05:28 So, I want you to ask I want to ask you about five years in the future. But I'm curious for you personally, what are you hoping to see in terms of your job team, how AI use evolves, what conditions have to hold for the technology to actually be truly valuable? You know, what do you how are you thinking about things over the next, you know, next few months?

08:05:49 Yeah, I think it comes back to some of the stuff I touched on before of just one having just, you know, full confidence in any type of information that's flowing out at the end of the day to our investor base that's accurate, correct, and we don't have to spend hours manually checking that to make sure that that's validated and having the confidence that we're sharing, you know, no issues or mistakes.

08:06:12 Um, because we do have just a wide investor base, you know, over 400 global investors. Um, I would also say being much more tech enabled in mindset. I think our team has not lacked that to date, but it's kind of been an afterthought around the use of technology. And I think the more that we can develop and enhance these tools that make, you know, routine tasks, you know, more efficient.

08:06:37 It frees up capacity to add more thought or think more creatively around why does this process exist and how does it work today and how can we actually enhance that or should we change that or should we bring in a new type of technology and adopt that going forward and our team has not lacked that today but it's kind of been a secondary thought where I I think ideally our workforce is positioned to think about that more on a go forward basis and that's kind of their primary thought as they're tackling different

08:07:06 problems specifically this specifically at Running Ridge day-to-day and kind of scaling our organization. >> Uh I think we have time for one more. I think one of my learnings from working with you has just been how unimaginably complex and creative the work is and how it takes a lot of effort both human and AI to get things to work. A lot of people in this room who either want to work on the technology side of AI and finance or perhaps finance side.

08:07:28 Is there anything that you would mention to them from a here's something that you should know or keep in mind when trying to apply AI to domains like yours? >> Yeah, I would say it it comes down to we are extremely complex organizations and it's not as straightforward and typically things might look a little bit novel or strange and out of place, but it's probably actually there for a reason or structured in a specific way.

08:07:54 I'm sure as you guys have come in, you appreciate that we're super nuanced and complicated. So spending the time and really putting boots on the ground to sit with whatever team or front office professional that you're working to build a tool for or enhance is time well spent. And I think the success and you know where we are today with the Rosebased product and kind of being ahead of our roadmap and expectations has been you know due to the fact that Michael and the team have spent ample time sitting on the ground

08:08:24 with our team like pretty much every single day and you know understanding our workflows which I think is crucial. >> Awesome. Well I guess last thing I'll say is that shameless plug bothace and Apollo Technology are hiring people who want to apply AI to finance. So please reach out to either of us if you are interested. Jack, thanks so much and great chatting.

08:08:47 >> Thank you. This is great. Thank you guys. >> When AI agents tackle complex tasks, the final answer is only part of the story. to explore how we can evaluate the decisions they make along the way. Please welcome Mitchell Troyanowski, co-founder of Basis. Hello. Hello. Hello, New York. I'm Mitch. I'm one of the two co-founders of Basis. We are based right here in New York City.

08:09:41 And the thing that we focus on is how can we get agents to do production scale work reliably enough so that we are trusted and used end to end by accounting firms across the country. So a lot of our research really focuses on this. How can you get these agents to do real work in the real economy? So I want to maybe start with a question which is if you're in this at this conference honestly if you're watching this talk you probably increase your own productivity a lot.

08:10:17 You're using codework or uh codeex or cloud code whatever it is. And yet if you go look at the average company, the average enterprise, the average firm, their productivity hasn't gone up 10x, 100x. I know my productivity has gone up maybe 100x, but if I were working inside of a company, it wouldn't be anywhere near that order of magnitude. And so why is that?

08:10:46 At basis, one of the sayings we have is that humans are already used to working with non-deterministic systems. It's just those systems are normally their co-workers, not their computers. And so I actually think to answer this question, it's useful to maybe look at the existing way we organize intelligent systems, which are companies. So how do scale their own productivity, right?

08:11:12 Organizations need to trust that the people doing the work at every level are doing it in a way that is going to produce good outputs consistently at scale. Right? If you look at this image here, the here I'm showing an accounting firm, but this could honestly be any company, traditional company. The person who is the head of the firm, they have literally no idea about the exact day-to-day of the people at the very bottom.

08:11:39 And yet somehow they trust the work is going to be done. How does that happen? It doesn't happen through deterministic systems, right? People are not deterministic. They're intelligent entities. And instead, there are processes that are put in place. And those processes if we believe the people who you know put them in place think that those processes will generalize to good outcomes then they can be comfortable that even though the people executing the processes are not deterministic you will generally get to good

08:12:10 outcomes if you follow the process. This is true by the way even in the smallest step. So in this example earlier I was showing the process of maybe doing the monthly accounting work for a client at an accounting firm. But even within that process say you wanted to see how do I reconcile the cash? Well I need to get the bank data and even that has a process.

08:12:31 The process could be really basic. It could just be use the approved software to extract the data but there's still some process that is put in place. You wouldn't expect the firm to just let anyone Google, you know, extract PDF and then just use whatever they want to go extract it because now you wouldn't have process. Now let's talk about agents. As I said earlier, everyone already has co-work.

08:13:03 But where is that productivity? Our opinion is there are two main problems that come with what I'll call coding style agent products where you're working with an agent and you just tell it to go and do something. The first problem is that there is a competency gap because agents as you think of them as you think of them today, they outsource the complexity to the user.

08:13:28 Right? If I'm the user, I now have to make manage my skill and memories and think about how to uh when I should use the age and I have to have like theory of mind over it. This is really hard. And for those who are here, maybe you guys find it easy. But I promise for the wide majority of knowledge workers, they have no idea how to do any of this. And there's a second problem which is there is a process gap.

08:13:56 Right? If you are saying, "Hey user, here's your agent, do what you want." Well, now suddenly the way the work is being done is essentially up to whoever the user is to customize the way the agent is behaving which goes entirely against the way that organizations scale. The process should be coming from senior leadership down not from any individual employee deciding how they're going to do their job.

08:14:24 Right? If you going back to my example, if every employee just made up their own way of extracting data, you know, even back in 2018, that wouldn't be a good outcome. And so we believe strongly that the key to building production agents that actually do work end to end is solving these two problems. You have to solve the competency gap by making it simple to the user.

08:14:46 You shouldn't even have to know it's an agent or you probably know, but you don't have to really feel it. It should be as simple as just working with a co-orker. You just tell it what you want it to do and all the things you expect to work should work. And the second is the process gap. You need agents to be reliable. Not having a stroke of genius one day and forgetting something the other day.

08:15:07 It's not about being perfect. Humans are not perfect, right? Nobody is looking for perfect. What you're looking for is reliability and consistency and trustworthiness. So what does that actually look like? We think that to be simple and reliable where you know it's you don't have to think about what to do. The agent can just do it for you without the user knowing the internals to do it the same way consistently.

08:15:38 You have to be uh you have to be opinionated. And just to show you actually one slide back here, I want to point out that getting it right isn't actually the main task because again humans are imperfect themselves. The point is to do it in the similar way. It's okay if you you can't do something but you should be consistently able to not do something and then you get better just like a person does.

08:16:03 If you suddenly one day are an expert at something and then you miss something very basic that would be expected now you're not reliable. So you have to be opinionated. What's an example of an opinion? Let's say we have an agent and we need to go and read a PDF. So in this case, what is the best way to read a PDF? There's many ways. You can throw it into a VLM.

08:16:28 You can go write some Python code and use an open source OCR library. You can work with one of the many OCR specific model providers. You could do an OCR and then you could have the VLM check its work. You could have five VLMs check its work. You could do a 100 VLMs and take the best event. There's literally infinite possibilities here. And so if you just said agent extract Excel, extract PDF, you might get good outcomes, but they're not going to be reliable, right?

08:16:57 Not because the process it did was bad, but because the process it did was not opinionated. So the way we develop opinions is you use initial intuition, right? Some things you know what the opinions are from expertise. For example, if I'm trying to get an agent to do tax work, well, I know that if the agent is goes and reads the primary source before returning to the user, it is more likely to be accurate.

08:17:31 I don't need a set of evals to tell me that. I don't need to divine that from first principles. We know that, right? I know that if I have an agent make a PowerPoint, if it takes a picture of the PowerPoint before it returns to the user, it is less likely to have a formatting issue. That is obvious. And so there are lots of different ways to develop opinions.

08:17:51 And one of the key points here I'll just make before I move on to talking about how to use the opinions is that the opinions are not just about accuracy. Right? We just talked about the example of taking a picture of a PowerPoint. Well, that adds latency, right? You may not want that. Maybe the agent only makes formatting mistakes one out of a hundred times and you're okay with that.

08:18:09 And so therefore, you don't have to take a picture of the PowerPoint. The point isn't exactly what the opinion is. And this talk we're going to focus not on how to develop opinions, but why opinions matter and then how to leverage those opinions as you scale in production. So say you have a bunch of outcomebased evals. And when I say outcome based evals, I mean what we traditionally think of as eval.

08:18:36 It has a right answer. We have some rubric. We're trying to judge what the agent did. How do you know that the eval generalizes or 100 cases, but in production if you're doing real work, you're going to have hundreds of thousands of situations. And that's only going to increase as the trajectories get much longer because the amount of decisions that are made in a trajectory just grow exponentially as the like long horizon abilities of agents increase.

08:19:11 And so I can't prove if I if you're looking at all these distribution of uh trajectories, you can't prove every one of them is correct. You don't have a ground truth for every trajectory. But you can potentially look at the trajectories to decide if the behavior, the opinion that you have is actually happening. And you can then infer that if it's actually happening, it's more likely to generalize to production.

08:19:34 It's not perfect. You don't know for sure, but at least you are having an opinion. Let's run through a real example. So let's say that I uh for a behavior this is an accounting example where if there is a uh mistake for uh in in the bank reconciliation um uh I don't want the agent to like post an entry to make a tie. I want it to uh sort of explain the difference instead of putting a plug.

08:20:09 Again this is subjective. You might want the other way, but what you don't want is a customer getting one way one time and another way the other time. And so what would you do? Well, first let's understand what governs the behavior, right? The behavior is not just you write the two sentences into your system prompt and it just works. That might work for very simple narrow agents, but when you're building these big agent systems, the behavior is actually a combination of the system prompt, your progressive disclosures,

08:20:38 uh potentially tool descriptions, obviously the training of the model. There's a lot of things that govern the behavior. And one tip, by the way, I have for when you think about building production agents is really think of the context the agent is going to see as it's working as runtime training data. The word prompt, I think, trivializes it a bit.

08:20:56 Right? At the end of the day, this is the data that the agent is being trained on at runtime. Not literally trained on, but effectively trained on. And so, you want to care about every single token and understand how that data causes the agent to behave in certain ways. So when you're putting uh behaviors into production, it it might be very difficult for one behavior to put it in exactly one, you know, paragraph that you know exactly because again the behavior is all of the data at once, not just one thing.

08:21:26 And one of the things that we developed at basis was this concept of a behavior spec. And the idea of a behavior spec is essentially saying before we update the context or write the context. Uh what do we actually want? What is our intent? You can kind of think of this a little bit as like a product spec for the agent era. And you can't write you shouldn't be writing every detail down.

08:21:53 You should be taking the most important opinions and putting it because if you put everything you'll be duplicating a lot of states. So you define the ones that matter the most and then you can use them as a way to align your team from a spec perspective. But then you can also use them as a rubric and you can say in this particular trajectory whether that is in any trajectory that doesn't have a ground truth did the agent get into a situation in which this behavior was relevant and then if so did it actually exhibit

08:22:23 the behavior. And this way you can scale the amount of signal that you get on your trajectories far larger than trying to increase the amount of outcome based eval ground truths and rubrics that you have. Those are really important but you cannot scale those anywhere near as fast as you can scale the ability to supervise what the behaviors of the agent are.

08:22:48 Let's just take a quick example here. Um this is so this is actually the second concept um which is called forking which we can combine here with behavior specs to think about forking if you've ever used uh cloud code or or codeex web apps there's this little fork icon uh under each message where you can essentially create a copy of the agent using the same thing which works really well on your local computer.

08:23:12 It's hard to build a system that does this in the cloud perfectly with recreating the database environment and recreating the uh the prompts all the all the different environments. But forking is another tool that we use to check the behavior. So a flow that you can imagine is saying we had a trajectory could be in an eval set it could be in testing or in production and it failed the behavior eval.

08:23:38 And now you're trying to understand why why did it fail fail the behavior eval why did not exhibit the behavior and then you can go into the trajectory and go right before the point in which the agent made the decision that was in contrast to the behavior assuming again your behavior is good and it's actually producing signal and then you can fork it and when you fork it you can ask the agent uh or you would fork it right after it made the decision you could ask it why' you do that or you can go and update the uh

08:24:09 context of the agent and see if that improves the behavior. And so with forking, you can start to actually take the signal and start to figure out, okay, how can we then improve the harness to now uh leverage the signal versus if you're just looking at signal from an outcomebased eval, you can't do that anywhere near as easily because it failed the outcome.

08:24:31 How do you know why? you have potentially a thousand inference calls in your five hour long agent trajectory run. And then we have the last tool that I like to talk about. This is actually my favorite, which is called simulations. Uh it's going to sound silly, but I promise it's very effective, which is that you can when working with coding agents, ask the coding agent to make a sub agent that pretends to be your production agent.

08:24:58 And you have to tell it to not cheat. And you give it the system prompt and the skills, and you render everything perfectly. And what you can do is you can ask it what it would do in some scenario. You can give it the situation and see if you can recreate the situation that you saw in your evalor in production. And if you can recreate it now, you can tweak your context to see if the sub agent would change its behavior based on your tweaks.

08:25:17 And the reason to do this, it's obviously far worse than forking, but you do this because this is much cheaper, much faster, and allows you as an agent builder to really easily iterate on ideas without needing to go through the expensive process of actually forking an entire environment. And you can do this today without building anything. Um I did this for literally year and a half plus before we uh built forking into our internal systems.

08:25:48 And the main idea here is understanding that if you look at the the distribution from earlier you're trying to understand is the beha is the uh agent generalizing to that distribution. And so you don't know purely based on the outcome eval. You can try to choose the best outcome eval represent the curve but you don't really know. And so supervising the process through some of these methods gives you some additional signal in order to understand if it's generalizing properly.

08:26:17 And what's really cool is as you start to get more signal, you can actually start to close the loop. And an opinion is what lets you do that. Right? If you don't have opinions about your system, you're not going to be able to develop that signal. And now if we go back to the firm, we talk about talked about supervision. We talked about how the agent uh if you're opinionated can be more reliable and actually can be simpler.

08:26:46 What that means is the agent maybe it starts doing something narrow but because people will work with it, they see that it's easy to use and they see that it's reliable. As it gets better and better, they will trust it more. It's just like a co-orker. You have a co-orker day one. Imagine someone sends you a PR. You're not going to if it's you're looking at two axes.

08:27:06 Whenever someone sends you a PR, it's these two axes. What's the blast radius of the PR? And what is your perception of how competent that person is? Right? If the least competent person at your company is sending you a PR that's editing a core service, you're going to look at it. If the most competent person is sending you a PR that's editing an internal document, you may not look at it as much.

08:27:29 And so this idea is true with humans working with agents just like it is with humans working with humans. And I think I just want to make one last point to close here, which is that one of the things that people say when they hear this is, well, wait a second. If you let the agents just do what they want, won't they find a better way? Won't they move 37 your work?

08:27:45 And the answer is they might. But that's not how you scale GDP and scale output. Nobody wants their agent to move 37 their tax return. They want it done reliably at scale in a way that they can trust and in a way that their people know how to work with. And so the thing I will leave you with is for agents to diffuse. They need to be simple. For teams to trust them, they need to be reliable.

08:28:14 The way you get to simple and reliable is you develop opinions on how the agent behaves. And then opinions allow you to build that additional signal that helps you start to close the loop so that you can achieve self-improving agents in production. Thank you very much. What if AI could do more than generate content and actually simulate the physical world?

08:28:46 to explore the next frontier of world models and what they could unlock. Please welcome Christoval Valenuela, co-founder and co-CEO of Runway. Hey guys, uh very very excited to be here. Thank you for staying for the last talk. Uh welcome to my talk about the matrix and how we're living through it. Um so uh you know I' I've been working on Runway for almost a decade now.

08:29:27 Started here in New York actually maybe around 10 years ago. And um I want to walk you through some of the learnings we've seen through you know building the company building this research lab here in New York and where do we see a lot of the work that we do in video in pixel generation really moving forward. And so I don't kind of exaggerate and I joke that this is might be a simulation or not.

08:29:48 Um and and so for maybe for perspective, I'll walk you through a piece of history. This is like watching uh prehistoric ancient ruins. Uh this is our first AI film we ever made. It was done in 2019. Uh it's called uh The Orange because as you can tell very clearly, it's an orange that falls from the tree and walks through different parts of the world.

08:30:09 Um, this is done. This is a a video model, an image model that we had back then that was creating pixels in 128 pixels by 128 pixels that we then forcibly upsample to to this kind of resolution. Um, I think of this as art now. This is this is like looking at a film from the 1900s, you know, it's it's beautiful. Uh, there's a lot of history behind it.

08:30:31 But this is 2019. Uh, we started the company in 2018, 2017. And so when we're trying to figure out if this had any lags, uh, we were trying to convince ourselves that if you scale this, it will naturally improve to the point where we can get somewhere close to this. This is state-of-the-art video today. This is all generated. None of these characters are real.

08:30:52 None of those locations are real. I'm sure you you if you're probably here, you know this by now. You've probably have seen this online on Twitter. Um, the models are just incredibly good. And the most interesting thing is that when I look at stuff like this, um, I still believe this will look like the orange in just a couple more years. Um, and so I want to walk you through, you know, how we're thinking not only through what we're doing at Runway today, but also in in the next couple of years and the research that's

08:31:19 shaping that. Um, we are a company that's at the intersection of board both art and science. uh we have uh deep roots in artistic and creative communities and we have some of the best researchers in the world working on diffusion models and large video models. Um and I like that combination because it allows us to you know bring the best of both worlds.

08:31:40 Uh we're we're a team of scientists and artists thinking about the problem of scaling video and world models. Um but in a way that the core idea of the company was already discovered by cinema uh and by moving pictures 120 130 years ago which is if you think about video and maybe you remember this from uh from high school video is an optical illusion.

08:31:59 Video is the illusion of movement that if you do if you stitch frames fast enough you will create movement. You will create the illusion of movement. You create motion pictures. And so at runway we're doing kind of something very similar. We're taking that idea really to the next level. How do we do next frame prediction fast enough, scale enough at in in in such a way that we can hopefully do more than just pretty pretty and nice creative videos.

08:32:24 And so all is powered by of core scaling loss. I think we believed in scaling loss uh not only applied for for language models but also for for video and this has been true. We're now really ramping up the amount of compute that we put behind the the models that you already have seen and and I think that the interesting thing about you know seeing the models scale over time is the emerging properties that appear the intelligence that appears when you start taking this very simple idea that cinema found itself uh

08:32:56 figuring out 120 years ago um which is next frame prediction and taking next frame prediction really to the next level right so if I take an image like this which is not real those two people are not real and I just keep on predicting what the next frame will look like based on the previous frames. And if I do it fast enough and if I do it with a network that's big enough, I'll get movement.

08:33:17 I'll get delusion movement. I'll get something that looks so real to us that we we humans can distinguish. I I look at that video and it's I've been doing this for 10 years. I can't tell you if that's real or not. Um and we measure that. We actually measure and try to quantify um if people can distinguish between real and generic videos. And I think that that touring test we already passed uh people are not able to distinguish.

08:33:39 But the more interesting thing is here is that really what's happening as you predict next frames is you're learning a lot more than just pretty frames. You're learning about physics. You're learning about object permanence. You're learning about gravity. You're learning about occlusion. You're learning about movements. You're learning about how the objects work in the real world.

08:33:57 And you're doing that all through just video through images through next frame prediction. So if you look at a complexity like this, um a good video model for it to understand how to make this a consistent video will need to naturally understand uh what all these subjects are, how each of them relate to one another, how a camel might probably uh stand on a ball like that, what water in a container looks like.

08:34:24 And if I do all of that together and I do it fast enough, I'll get hopefully a consistent video that looks real to us. So a lot of the uh benchmarks that we use to evaluate our video models are not only creative, it's creative basically means do you like it or do you don't are actually based on uh on physics on geometries on how objects behave in the real world and how those objects should behave and also simulator or vido worlds.

08:34:52 So you take that idea of an extra prediction and you do it fast enough and you get of course now something that looks more like what we should expect from a video application or a game application. Again this is just just video right and you take you take that idea to the next level. What happens if we start this idea of lecture and prediction but do it in the context of a 3D environment or something that looks similar to a game?

08:35:13 You can start actually getting closer to a physics engine, a simulation system. But the simulation system has no physics laws. We didn't describe uh what the behavior of the object should be. The model just knows how to do that uh by poor force of of uh of scale. And this kind of gets gets kind of interesting because you can only think about playing worlds.

08:35:34 This is not GTA. This is um someone playing someone else um in a environment and a software that doesn't exist. It's all simulation of a simulation which which gets really meta, right? Um and so I'm not kidding when I said welcome to my metric simulation. Um but the most interesting thing for me is that everything you saw before was not real time, meaning that it took a couple minutes or seconds for those videos to be made.

08:36:02 Um, a lot of what we're seeing right now when we think about, you know, the next three, four years of video models and world models is to deeply think about what happens when you start not only scaling the models and making them much much much uh bigger which much more emerging properties but also what happens when you start making them interactive.

08:36:22 Right? So take the same idea of next frame prediction. You take a frame, you predict next frame, then next, then next, then next. But then it can act at any point in time in that frame prediction, which means I can simulate any possible divergence or action in that world. So in this world, I could play the video, but I can also come and touch every part, any aspect of that image and make it interactive to affect the consequences of of that world.

08:36:49 Um, and it works. And of course, this is incredibly useful for a variety of different use cases. As you can imagine, we're now spinning an entire robotics lab in runway. So, if you come to our office in New York, half of it is uh what looks like a Hollywood studio. Half of it is a bunch of robots walking through and moving through the office. And so, we're able to make these models or start making these models the brains of how autonomous and embodiment systems walk and reason about the world.

08:37:21 Um, but there's a couple of more interesting use cases that I think go beyond the two obvious one, which is video and robotics, which is what happens when you think about anything, anything, any pixel on a screen as a video model. Um, here's a great example. This is a video model. What you see right now, well, this is an image. First of all, that image could be real or not, but I could run one of our video models in real time and make this an interactive application.

08:37:45 So, let's say I want to, you know, turn the lights of the car. Maybe I want to like turn the car around. Um, I can just click through it. I can turn it around again, maybe open the hood and see what's inside the car. Now, everything you just saw here has no physical 3D representation. There's no Unity. There's no Unreal. There's no Blender. It's just a video model.

08:38:08 A video model has been trained to understand actions and clicks in an interface. We call this a world action model. Um, sorry, um, a world interface model. And the world interface model, well, I I feel that you guys understand now. It's it's an interface and it's a world model, world interface model. Um, allows you to do very cool stuff like this. And so, here's another application, another um example of of how you could think about using a model like this in real time.

08:38:33 Um, I can take an image, a representation, move objects around, and expect them to move as I was if if as I was actually in that room. Again, remember there's no 3D representation of this there. This environment might not actually be have have been modeled before. It's just an image and the model that understands how objects move in a space because of this very simple idea that next frame prediction scale well enough can become a world model or a simulation engine.

08:38:59 Um so with the eight minutes I have left I want to show you this idea taken to another extreme. Um, we're we're thinking not only about what happens if you can control these cool applications and these cool interfaces, but what happens if you keep extrapolating and I think by the way extrapolating might be one of the most important skills these days in in AI in in company building.

08:39:25 What happens over the next six months, 12 months, 18 months might be one of the most important things to to try to predict and understand. Um, and so we ask ourselves that question a lot at runway. What happens? What will happen if we keep scaling the models? If we keep adding more data, if we keep making them better simulation systems, if we make them universal simulation systems that can predict anything on a screen, literally anything on a screen.

08:39:48 So, we've been working on this idea that we call continuum. And continuum is um the the idea of real time video models or world models taken really to the next the next degree. This is an operating system. The idea here is that you might open your computer at some point in the near future and what you see streaming to your screen might be coming from a video model.

08:40:09 So I'll show you next specific how this idea could could shape. This is still very early research. But I want to show you this idea of combining video models as part of your operating system as part of your daily work. Um so here's an example. I was researching you know San Francisco and the fog in San Francisco. Um I wanted to learn more about when and how it happens.

08:40:32 This is structural data. This is actually a website. This is something um you can take from uh from the internet. And now Continum has this idea that I can select any part of content in that website and make it a portal. We call this a portal. And the portal is a real-time video. It's fully interactive and fully based on the content you're experiencing reading through.

08:40:52 I probably won't read through all of that text. It's too much text for me. So I just want to make a video and I can make a video that is represented by by the text you see there. The these portals can be 3D visualizations. It can be any content that you're watching or interacting with. On the right you can have all the visualizations and portals you've you've experienced before.

08:41:09 Anything you've been trying to learn. Um basically it's a it's a memory. So it's the disk of the uh imaginations or the visualizations you've seen before. So here I was researching about batteries and so I can go back to uh the portal I had around batteries before and since are real time interactive applications I can click them anywhere. I can move them around.

08:41:28 I can start asking questions and the videos and and the content itself will change based on based on my interactions and the way I'm I'm I'm managing through it. Um here's another very cool example that when the first time I saw it when the team showed it to me it was like blew my mind. Uh since it's a real-time video, you can make it fully interactive with anything, not only with clicks, but like resizing the application should also change the behavior, right?

08:41:53 So I can resize I'm I'm learning about photosynthesis here. If I resize the application, I get a different experience. Um and since it's again full interactive applications, I can click through it, navigate through parts of the cell or how photosynthesis work to get a better understanding of what I'm trying to learn and do. Again, um the one thing that still feels to me uh kind of unsan about this is I don't know how to call this.

08:42:18 I don't know if this is doesn't feel like a visualization. It doesn't feel like a video. It doesn't feel like a game. Something in between. And I think we need to do the collective work of figuring out how we call these kind of interactions in the first place. And I'm sure you hear kind of the other thing is that you can change the structure of the prompt or what's actually creating that interaction in the first place.

08:42:40 Um, here is another example. Um, I'm going to type something. Um, and this is one of the ones that I feel has been the most under explore yet. This is, uh, we call this visual thinking. So, we've connected continuum to a coding agent. And what the coding agent can do is it can use continuum to visualize through a problem. So imagine you ask your coding agent to create a 3D model for your bike and instead of the model trying to just think through that via code, the model can spin a version of continuum to think through

08:43:14 the problem visually. So if you ever think about someone or I'm sure you have friends or maybe some a lot of people here may be visual thinkers. What a visual thinker is is it thinks through problems and through ideas visually first and that helps you ground and project some of the best things you want to do. And this is somehow very similar. Agents have the ability to do visual thinking.

08:43:38 Agents can spin up a sort of imagination that you can tap and and interact with. So not only the agent can interact with it, I can also see it myself and I can guide the agent in much better ways that I might have able to uh guide it just via text alone which is for us a very powerful idea. It's a completely new paradigm. We call this this this idea uh visual visual thinking.

08:44:01 Um the last example I have is how you think about taking you know continuum to become much more than just uh using or navigating interactive structural content in in the way I've showed you before but actually making them fully live experiences that are reactive. So in this case I was searching for Mars. Um, and as as continuum understands the intention of what I'm trying to search from, it also suggests interfaces that are specifically designed in real time just for that.

08:44:36 So here it created for me this Mars outpost that's actually should be uh accurate with the data about the actual like robots that are in Mars right now or with the ones that are projected maybe in 2030. Um, and since this is fully interactive, I can interact with it. I can go to places. I can move around in the same way that I might move around, you know, in a 3D system, in a 3D continuum.

08:44:55 Um, and it works just fascinating. Um, so when I say everything will be simulated, um, I'm really trying to understand or help not only our team but the world get to the point where a lot of what we'll see today will sound obvious. Hopefully in a few more years or months, we'll look back at Continum and it will might feel very close to the video of the orange that we just showed you.

08:45:25 Continuum and a lot of the things we're doing in in video and in runway uh when it comes to simulation are still in very early days, but we're getting to a point where it actually it actually works and it really scales. So, we're really excited about is that for the next 8 to 12 months, we're releasing most of what I showed here for everyone to use.

08:45:41 Thank you guys. As we come to the close of AI Engineer New York 2026, we'd like to thank everyone for being part of this incredible event. To share a few final words and officially wrap things up, please welcome back our MCA Se from Touring Post. Woohoo! Clap, clap, clap. Thank you, Crisal, and thank you to all of our speakers. And I would like to ask you to stay for one more minute.

08:46:25 We have someone very special to see. Um, we've covered a remarkable range over these three days from building agents to changing how financial institutions work and how to simulate the world. I imagine quite a few of us have a longer list of things to investigate than we arrived with. That seems like a you know reasonable outcome for a conference full of engineers.

08:46:51 Thank you the team who put this program together and the crew and our sponsors and partners and to everyone of you who came ready to share their work and ask questions and do clap clap clap please. Thank you. I hope some of the conversations that you started here turn into things we will hear about at the next AI engineer. And I want to welcome to the stage the guy who has been behind the curation, the one and only.

08:47:29 >> She just did. Hi, thank you so much for staying to the end and thank you Cassenia Gabri. Give it all. >> Thank you guys. If you haven't seen uh so I I know she's very selfless and she's uh MCing for the second time, but if you haven't seen her videos on touring post, highly recommend. Uh she has a great YouTube channel uh and uh she's spending a lot of time editing uh that which I uh really admire.

08:47:52 Thank you so much. U uh I hope this has been a great few days for you. Uh, I heard a lot of feedback from sponsors, from attendees and speakers, and we're really trying to make a home in New York, which is why I'm so grateful to have Chris and Pim and all the other like great New York City AI founders. Uh, this is something that I think people don't get enough love for in this city, which is uh how much like real work and uh frontier pioneering work gets done both in the financial institutions and just in general AI

08:48:23 research, AI infrastructure um and and all that good stuff. So um I won't keep you too long. I'll just talk a little bit about our upcoming events. I'm wearing the shirt for code now. Um you so we're basically you know code is our our first home uh first vertical summit and we coding agents really I think are uh the the sort of the pinnacle right of of where verifiable um AI is is going.

08:48:45 Um we're also confirming that we are uh launching AI engineer Europe again. That's the second year in London in February. And obviously the big big show of all, bigger than all the other ones combined, the World's Fair, 10,000 people in San Francisco next summer. Uh so there's a lot of AI that's going to be happening between now and then. We may have uh be all out of a job, but if you're out of a job, that means you can come to more AIES, right?

08:49:12 So uh anyway, we'll all have a good time. Thank you so much for being here. Thank you for supporting us. I hope you all made lots of friends in professional contexts. um follow the conversation back home, right? Like and and meet up uh by among yourselves. Uh that is what uh it's so rewarding for us and our team to provide this community for you. So, thanks for your support. See you next year. Heat. Heat. Hey, hey, hey.