AI Engineer NYC Livestream | Bridgewater, Anthropic, Two Sigma, Arize, Coatue, Nous, Gen. Intuition Transcript, AI Summary & Key Points

01:23 Heat. Heat. N. Keynote programming will begin in 5 minutes. Please be sure to find your seats. All right. Heat. Heat. Oh yeah. to get to Good morning and welcome to AI Engineer New York 2026. We're thrilled to have you with us as we kick off an event of big ideas, sharp perspectives, and plenty of signal cutting through the noise. To officially open the event for us this morning, please welcome to the stage AI engineers general manager Leah McBride.

08:09 Hello. Hi. Good morning, everyone, and welcome to New York. Um this is our third event in New York. We had two in 2025 uh with some of our greatest speakers that we've ever had at AI Engineer. So we're really excited to be back. So just a huge welcome for myself and Swix to everyone. Um thanks for attending today. We hope you're going to have a great few days.

08:34 Uh there's very very few places like New York City as we know um where big ideals scale very scale very very fast. um at it is where Frontier AI comes to meet centuries old global financial institutions and where multiple different economies thrive together. So, we're excited to have an incredible lineup. You're going to hear from some of the world's best speakers just like at all our events.

08:58 We have got people we will we've got tracks um in finance. We've got engineering leadership uh evals and LLM. Um, we've got Parad Dinaran. Going to get that right. Uh, from Arise. We've got Max Cana Alexander from Capital One and Tariq Shikat from Sonar. That's just a few of the incredible speakers that we've got this week. Uh, I'm going to say a huge thank you to Arise who's our presenting sponsor.

09:28 You guys can go and meet their team in the expo at any point this week. So, uh, lean in to learning here. lean into connecting as well. There's so many people here that um are sharing ideas. They want to talk about the ideas. They want to challenge ideas. Um and they want to collaborate as well. So, make sure to speak to as many people as possible, your attendees, speakers, and all the sponsors.

09:53 Um and we hope to see you again soon. We do have a great lineup. Uh we we've got another event, Cold Summit in San Francisco in November and then we have London lined up in February and then of course World's Fair and we've got a couple of other events happening next year as well that are new. So uh keep an eye on the schedule but have a wonderful few days.

10:14 Uh thank you so much. Take care. And now, please welcome to the stage your MC for the day, managing partner at 10X, Alex Lieberman. What's up everyone? How's it going? There we go. Uh, before we hop into the official stuff, I just want to start by introducing myself and asking you a quick question. I'm Alex Lieberman. I'm one of the co-founders and co-managing partners at 10X.

10:47 were an applied AI company working with Fortune 500 businesses to become AI native. Before that, I co-founded a business called Morning Brew, which makes business news suck a little bit less. My question for you is, who in the room, raise your hand if you are healthily addicted to X, or formerly known as Twitter. Okay, a lot of people are lying in the room.

11:10 Uh, I spend a lot of time on X. And one of the reasons that I spend a lot of time on X is because I feel like I get to live in the future. Right? When you're when you're scrolling through X, when you're looking at the commentary on the platform, you feel like you're seeing what will become consensus in 3, 6, or 12 months. And what I love about this conference and why I get so excited to come to it every single year is because this almost feels like X on steroids.

11:44 Uh there's this concept that a friend told me a while back uh called edge city. And edge city is this concept that at any point in some part of the world there are a group of people that are living in the future in a specific function or specialty. And the way I think about this conference is we are all living in edge city for where for what will be consensus in the next six months to two years if you really think about the diffusion of this technology.

12:15 So I'm super excited to be part of this with you all today. And now let's hop into what's going to happen. First of all, welcome to AI Engineer New York 2026 where AI engineering meets Wall Street. This is the conference for engineers and technical leaders building production AI across banking, hedge funds, fintech, insurance, and asset management. Over the next two days, we will hear from companies and leaders sparking discussion not only on models, agents, infra, dev tools, and evals, but then how they weave and

12:48 shape the financial industry impacting hedge funds, investors, banks, insurance, fintech, and security. Of course, as you all can tell, there's a lot going on. Putting together a gathering like this requires a huge amount of effort and it could not be done without our sponsors. But before our sponsors, let's all just give a huge shout out to the AIE team.

13:12 What they do is incredible. Now, let's give some flowers to the sponsors. First up, the presenting sponsor, Arise. Let's give it up for Arise. Yeah, Arise. Next up, the platinum sponsors, Oracle, Sonar, Work OS, and Excel. Let's give it up for them. Then we have our gold sponsors. If I had enough time, I'd rattle all of them off. And finally, the silver sponsors.

13:41 But wait, there's more, the supporting sponsors. Awesome. So, we are going to hop into uh the first keynote of the day. We have two amazing folks from Bridgewater who are in the weeds of production AI every single day. So, let's give it up for them. Welcome, guys. >> Coming up next, we're delighted to welcome two speakers from Anthropic. Please put your hands together for Belinda Neil, managing director of financial services, and Keon Shahab, applied AI architect.

14:42 Just waiting for our slides. There we go. Good morning, everyone. Uh, I'm Sam Green, um, team lead for applied AI at Bridgewwater Associates, and this is my colleague, Aaron Linsky. And today we're here to talk to you about the vault, the data platform that we've built at Bridgewwater to turn unstructured data into a first class asset in our investment process.

15:04 And as part of the talk this morning, we'd like to share three lessons that we've learned building the vault. The first is that humans and agents have very different needs from a data platform. And so you have to build for those different needs separately. The second is that the greatest impact occurs when a single tool can meet both of these sets of needs at the same time.

15:26 And the third is that the real prize is when you can build the foundations needed to make agents a proactive participant in your investment process or whatever you're doing with your with your your tools. Uh instead of a reactive uh a reactive tool for your humans to use. Uh before I I dive deeper into talking about the vault, I wanted to start with some some background about Bridgewwater.

15:50 Over the last 50 years, Bridgewwater has been in the constant pursuit of the world's best understanding of how markets and economies work and turning that into a systematic encoding that we can use to stress test our logic over time. And what originally began as a set of if then rules on a yellow legal pad evolved into a continuous compounding process with humans at the center constantly looking for the next thing that's changing in markets.

16:16 And more recently we've been working on building a associate which puts a machine at the center of this continuous process. Across all these phases of evolution, we've had a constant philosophy in mind, which is that we want to be constantly looking at how the world is changing and what's going on in the world. Noticing what what dynamics are not yet encoded in our understanding and use that learning to improve and compound for the future.

16:47 For most of Bridgewwater's history, that meant building and overseeing time series models on structured data. And that structured data needed to be carefully built so that it could be safe for use in our research process and aligned to our internal ontology and world model so that we could get great leverage out of using the the data that we had built.

17:08 With the emergence of language models, we had a huge new opportunity to engage with a with a much wider world of data. um you know news and research transcripts, policy documents and all of the internal conversation that was previously just in the ether. So the internal memos and meeting transcripts and debate happening inside the firm. We quickly realized that all of that data needed the exact same treatment to be fit for purpose.

17:34 It needed to be carefully cleaned, governed, and made ready for use in both point in time research questions today and also our systematic study over time. And when we looked out at the landscape of data vendors and data platform providers, there was nothing that provided us with all of the control that we needed to build and maintain our data advantage.

17:58 And so we built the vault which is our repository of external and internal unstructured data um and the platform that indexes, organizes and makes all of it ready for use by our human investors and agents. Today, the vault has more than 100 million documents in it with thousands more added every day, uh, minuteby minute from 135 external sources and again growing all the time as well as all of the data from inside of Bridgewater.

18:27 And crucially, all that data is not just dumped on our investors desks. It is carefully aligned to our internal internal ontology with uh enrichments like the countries, concepts, markets and most importantly time period. discussed in the life a fairly basic representative investment question to an enterprise chat client. I asked how is AI infrastructure investment impacting the Korean Juan and do market participants expect this to continue?

19:05 And as as you're all I'm sure familiar with um the agent goes to the web and looks for resources it can draw on because I've explicitly asked it to go find u other people's perspectives on the question and you can see it it zooms in and it finds 15 relevant looking resources to draw on before it gives me back its answer. And when I when I zoomed in on the answer it gave back a couple things stood out to me.

19:32 The first is that even though it found 15 relevant looking sources, it only picked one or two to answer to answer my question. And the second is down at the end of this response where the agent itself acknowledges that it hasn't answered my question. It didn't find enough documents on the open web to establish a market consensus and meet the meet the parameters that I laid out.

19:57 When I double clicked even further, the problem got even worse. One of the notes that it primarily drew on to answer my question was published way back in July of 2026. Even though I asked this question and captured this answer just at the end of last week in early October in the age of AI and investing for 3 months is a massive amount of time. It's not even close to good enough to draw uh draw on this resource to answer my question.

20:27 When I ask this exact same query attached to the vault, something totally different happens. The first is that the connection to the vault helps to prompt the agent that it should look for a specific period of time when answering my question. So, it zooms in on October 1st to October 8th. It finds a much wider array of resources, 52 candidate documents that are relevant to the query over this spec specific period of time because of the vault's much more significant corpus of content that's relevant to investing

20:55 questions. And when it ultimately answers my question, it draws on seven instead of only one or two and gives me back a rich answer that the agent is more confident is going to address my question. And so for us, this is the the big lesson from building the vault uh to kind of motivate the talk today, which is that it takes great data and the right connectors to make agents impactful.

21:20 And I'm going to talk hand over to Aaron now to talk about some of the product engineering takeaways from building the vault. Thanks, Sam. So, hi, I'm Aaron. I run engineering at Bridgewwater. I'm going to take us through a few of our learnings from building the vault over the past few years. I'm going to start with one that might seem obvious, but I still think it's worth saying, which is search is hard, but it gets even harder if we're designing for two different consumers.

21:48 And in this case, we're designing for humans and agents. Going to go over the difference here, starting with us. So, humans need precision and a high signal to noise ratio because we can only read so much. Even a fast reader manages about 300 words a minute. And personally, I'm much slower. Agents are a different kind of reader. They're far less sensitive to precision.

22:11 If the data is there, they'll find it. And basically anything you hand to them will get attended to. They scale super quickly and they come and they consume about uh 100 times faster, 30,000 words per minute. And so we need to build vault for both of these. And when we're thinking about the dimensions that matter, there are these five on on the board.

22:34 Recall, precision, filtering, entitlements, and scale. Some of them matter to people, some matter more to agents, many to both. I'm going to go deep on three of these. Precision for people, scale for agents, and then filtering for both. So precision once we've filled the vault our which was a heroic effort by our data and legal teams recall was no longer a pressing issue.

23:02 However the number one complaint was vault search is bad. As the product owner it's painful to hear that but my response was let's benchmark it. And that's what we did. We collected questions and found the best documents by hand including ones that looked helpful but weren't. in all in total was about 200 examples and it took us less than a week. It was not fun or very glamorous but it was very helpful and after we ran it it was very clear search was bad.

23:33 The key metric we were looking at is precision at 10 which is of the top 10 results how many are the right documents and we were hitting about 60%. So the people were right, it was bad, but now we had a way to measure all of the improvements. And here on the board are five of them. And by the time we got done with Agentic Search, we were up in the 90 95% uh recall at 10 or precision at 10.

24:04 All right, so that was precision. Now on to filtering. Like anyone, we want the most relevant documents. But when we looked at the production traces, what we were seeing is that almost always there was some condition either as of a certain date or from a certain source or about a specific asset. And all of that requires advanced filtering and that's where we ran into our first major performance issue.

24:31 So I'm going to quickly go through this of a little bit technical here of there's classically two ways to filter a vector database. Post filtering will use that vector index first and then filter the top results by metadata. It's fast, but recall could be pretty low because the globally relevant results might not match your filters. On the other side, we have pre-filtering where we'll filter first and then calculate distances for every one of those matches.

24:59 The advantage there is that the recall is very good, but it can be very slow and we needed something in between. So that's what led us to try and then ultimately switch to turbopuffer. Their approach which they call native filtering looks only in the nearest clusters that contain matches and then scores just those. What you end up getting is something that's fast like post filtering but with close to the same recall as pre-filtering.

25:26 And lastly I'm going to talk about scale. Human scale for us is pretty pedestrian. 100 queries an hour. We have 300 users. they're going to search, you know, every so often. It takes our users minutes to read the docs that they're that they're receiving. Agents, on the other hand, are a few orders of magnitude beyond that. Easily 100 plus queries a second and agents forms can hit thousands easily and they obviously will read as I said earlier will read docs in seconds.

25:55 So, we needed something that could scale as well and again turbuffer handled that easily too. So, we were covering all five dimensions. I didn't talk about entitlements, but I'll be talking about that later today. Um, but we covered it all. It unlocked new workflows, but it didn't change how our researchers worked. They still weren't coming to Vault every morning.

26:15 And that's my second lesson that I want to share here is that no matter how amazing the tools are, we can't just drop them on our investors desks and expect them to be used. We need to migrate their old habits into a a a new form. And so we looked at what they were already doing. Many of them were living in email. The research came straight from the brokers.

26:39 They had subscribe lists. They had rules that filtered everything into folders. It worked. It was familiar. And they knew that they weren't missing anything. So we had to build all of these features into the vault. That's exactly what we did. Everything searchable. The subscribed lists became save searches and folders became save collections. It's the same habits, just a new home.

27:03 And once we had them there, we can open up new habits. So we can ask, they can ask questions across all the data as Sam showed. They can follow dynamic feeds by topic and they can hand those collections over to their agents to take further action. And that's when we clearly had product market fit. And once we had that fit, the users wanted it everywhere.

27:23 So we put it in chat. We made a phone app. It's in Slack in their IDE. And of course, we availed all of that to the agents as well. Um, including a CLI because agents love CLIs. And so the main question here is, okay, so what does this enable? Simon's going to talk about what we're building right now, but first and foremost, it's helping us to move around that research circle faster with our pocket analyst, which is our first AI analyst.

27:49 It can do hund it can do hours. It can do hours of research in just a few minutes for a few dollars. And that's relying on the vault. And then we're expanding that now with the researcher, which is the next generation. It's the same idea stretched across days or weeks. We're also able to train custom models to more intelligently reason over the vaults data and use those together with other foundation models to feed systematic decision-m processes.

28:16 The forecaster is our first one which produces AI generated macro relevant predictions. And then we al also have built investment signals from the vault turning unstructured data into structured tradable strategies. Now I'll hand it to Sam to take us through Vault's latest capability. So like I alluded to at the beginning of the talk, the real prize is when you can take agents from a reactive resource that can be ready to answer a a human's question and instead make them proactive participants.

28:53 helping to steer your high-V value professionals towards the most important areas for their attention. And the way that we're solving that problem is in an experience we're calling the investor workbench, where the agent is always on tracking what's important to that investor and presenting them with the areas that are most in need of their attention.

29:13 It looks like an experience like this that relies on the vault where a topic is proposed and several pieces of information are presented to the investor to decide whether they should dig deeper or ignore that topic entirely. You can see here the card that an investor sees in order to decide whether to dig in deeper into the same AI impact on currencies question that we looked at earlier in the talk and the investor receives everything they need.

29:41 an agentic summary if they want a quick scan across the agent's findings. They also get the exact evidence that the agent was drawing on so they can assess for themselves what was leading to this recommendation. And if they choose to dig deeper, they have a direct connection to the research output from the analytical tools that Aaron was just describing.

30:00 And when they dig deeper alongside the agent to look at the dynamics driving this question, the learnings are automatically captured. The results of their collaboration are automatically captured back in the vault. And so this for us is the key lesson. The vault enables compounding because it provides the right evidence, the right ex internal and external information at the fingertips of our investors and agents and it automatically captures the synthesis that they generate and puts it back in the corpus for the next

30:32 turn. And that makes us go around the circle faster and better. And so when we think back to our the question we began with, what it takes to make an agent work is the right context and the right understanding of what's going on in your world. And so our question for you is, does your AI know what's going on in the world? And does it remember what your firm has learned about it?

30:54 Because we are teaching ours. And if these sounds like sound like an exciting challenge, we'd love to talk. Thank you. Coming up next, we're delighted to welcome two speakers from Anthropic. Please put your hands together for Belinda Neil, managing director of financial services, and Keon Shahab, applied AI architect. Good morning. I'm Belinda and this is Keon and we're from Anthropic.

31:49 We work with financial services firms. As Keon gets set up here, we'll start with a little bit of background. My name is Belinda Neil and I'm the managing director of financial services at Enthropic. Previously to joining Anthropic this year, I worked at Goldman Sachs for 19 years across engineering, global markets and equity research. My role at at Anthropic sits at the confluence of our industry, our product, our research, and our go to market teams really in accelerating the adoption of AI within financial services.

32:18 Keon, >> good morning everyone. Thanks for being here. My name is Keon Shahab. I'm an applied AI architect at Anthropic. Prior to Anthropic, I spent five years at data bricks helping financial services companies modernize their data and AI architecture. So we're excited to speak to you this morning and our talk is going to cover three ideas. Firstly, the frontier of intelligence.

32:41 How is this room taking the frontier models and applying it in ways that create alpha or edge within your firm? The second is the frontier of adoption. How are you scaling what you've built based on the foundation of a few to many? And then last, how are we applying it to financial services? We're going to bring an example in here about how we've been innovating in the wealth industry in Claude for financial advisers and that paradigm compounds potentially to other areas of financial services as well.

33:07 So let's get started and talk about the frontier of intelligence. So every so often a model crosses a threshold and when it does a way of working changes. We first saw this in coding last year with Opus 4.5 and Claude code. Many of you in the room are using these tools and you've gone from autocomplete encoding to managing fleets of agents where you're really driving and accelerating a multiple effect of how you're doing coding.

33:33 We also saw this threshold cross this year in cyber with Claude Mythos preview and many of you are also spending time on the cyber programs in your firms. What's next? Finance. We've started now in finance with early agents and bringing Claude into the products and services that you use day-to-day. One example here is Citadel told us at our briefing event in May that their investment professionals started to use Claude for Excel and it gives them a step change function in how they can test, build and trust their work.

34:04 We're really excited for the potential of Claude within the tools and the use cases that this room is building. But here's the challenge. Intelligence is ahead of adoption. What a model can do on its own is doubling every four months. PWC published early this year that the capabilities of AI are going to threearters of companies. Sorry, threequarters of the economic gains of AI are going to a fifth of companies.

34:33 This is what we call the diffusion gap. If you're not building close enough to the curve, there's opportunity for you to take the frontier and be able to apply it in your workflows. We work with firms across financial services across our model family, most of which hopefully this room is familiar with. We have Haiku for speed and volume, Sonnet for everyday work, Opus as the daily driver for financial services, complex tasks and agentic coding, and Fable, our frontier model that can effectively be used for the hardest

35:04 work in your firms. One example here from a millennium on our most recent uh model blog was a PM used Fable to find a piece of code that had a bug and it was unexplained in five years. Fable was able to find it. We're starting to see our models get used in this industry in new and creative ways. And we believe the frontier of intelligence is just getting started.

35:27 Every model release gets smarter, cheaper, and safer. So, it's really important to take a minute to call out how the model frontier is continuing to develop here. Firstly, the intelligence is going up. Now, a model can work for 16 plus hours on its own. This is up from just minutes in 2023. In terms of cost per task, it's going down. The most recent opus 5.5 opus 5.5 model was 40% more costefficient than opus 5.

35:53 You can also see those impact metrics for haiku sonnet and fable. And lastly safety this is something we care a lot about at anthropic in the most recent model of fable 5.1 only 1% of prop injection attacks were successful. This is relative to other frontier models in the region of 9 to 53%. So, we're really investing in safety alongside capability and anthropic and the banks know this.

36:16 The last most recent Evident AI 2026 report that came out last week showed us that banks are 26% ahead of the AI maturity scores. We're really investing in scaling in applying this technology to our industry. A few use cases to call out from this report which are publicly citable use cases that have been reported is firstly on areas like fraud. One bank found that with auto loans, they were able to reduce $120 million of potential fraud in 12 months.

36:43 In another area, in post earnings research, a draft of a report was able to be generated in 15 minutes relative to 45 minutes. And you can see other use cases here and more broadly across the industry as we start to talk about the value of AI and the impact we're generating on our business. So, where are we? In 2023, we built chatbots. Everyone was asking a question and going backwards and forwards with the model.

37:08 In 2024, we brought pilots to firms and we started to get more knowledge bases and data into these AI systems. From 2025 to now, we're now building agents. And in the most case in this room, we're building them hand by hand, one by one. And our opportunity here to scale is not the model. Our opportunity here to scale is building the foundation for agents that can go from one to many.

37:31 So the next part of the talk we wanted to spend time focusing on how you can build the agent foundation that will allow you to scale the frontier of adoption. So over to you Keon. >> Amazing. Thank you Belinda. So Belinda as you've just said there's no better time than now to start experimenting and building with agents. For this portion of the talk we want to talk a little bit about how you can go from that experimentation phase towards productionizing and scaling agents.

37:59 But first, some observations that we've seen across the industry. A year ago, we could think about this as the year of exploration where engineers were working on a handful of use cases. They were in PC and pilot mode and they took their learnings and transition that into this year where we've seen this be the age of production. So, we've seen these handful of agents go to production.

38:19 A lot of them have been built by hand. What I think will happen next year is we'll go into the age of scale where we have many agents built on one solidified foundation. And as Blinda just mentioned, the models are now the easiest part of the equation here. We know that the models can handle highquality tasks and we're confident about that. But what we want to talk about here is building this five layer framework for scaling and productionizing your agents.

38:48 Starting from the bottom, we first want to look at what the agent can do. So we can give it context for our business use cases via connectors so that it can reach our systems and data. Then we want to give it context for our actual business rules and processes via skills. Once that's done, we want to think about building the trust layer for the agents.

39:09 We can think about that in three general steps. Who the agent is acting as with an identity and permissions layer. how we can sandbox and determine the approval layer of the agent so that someone is signing off before it acts and lastly how we know the agent will still work once we adjust any variables over time. So let's look at each of these five.

39:34 Perfect. So if you started building agents already, you may think of giving those agents access to your underlying systems and data manually wiring them by hand to a handful of different source systems. And that could work for maybe five agents and five systems. Then you'll have 25 systems to maintain and manage. But as you think about scaling up your agents into new specialized uh various tasks, you'll want to have those agents work off of one secure connectivity layer, which is MCP.

40:06 That'll make it a lot easier for you to build at scale with one repeatable framework. Next, let's say now we have the agent have context to our underlying business data and systems. Uh now we want to give it uh more context into our actual business rules and processes. And that's done through skills. And this is wherememes that really do the work and understand the the process and the work come into the picture.

40:34 And from here on we're going to look at a agent use case for a trade break investigation agent and we're going to follow this across for the for the next steps. In this case, a trade break agent is meant to look at two records of a single trade that don't match. And it's here you can see steps one through three, a little bit of a simplified version, but this is the business context that ANM provides.

40:57 And we think about skills as living, breathing, versioned code essentially, even though they're just plain text, they're meant to be dynamic. They're meant to be updated frequently. All right. So now we have our context layer for the agent. We have the agent having access to our source systems and data. We have the agent that understands how to go from a generalized model to a specialized agent with skills.

41:26 Now we want to talk about building the trust layer with that agent. And first let's start with the identity and permissions layer. So here you can see the identity of the agent. We're going to assign it is going to be the trade break investigator. We're going to give it a role with permissions called ops analyst. And you can see here once that agent needs to go and reach our source systems like our order data and our market data, it'll have permission to read.

41:50 But if it tries to go reach for our CRM data, it'll be denied. And we will do that on purpose so that it won't overreach for sources of data that it shouldn't have access to. And lastly, we don't necessarily want this agent to perform on behalf of us. We wanted to just propose new amendments to our orders data, not actually make those amendments. And that's where we'll look at on the next slide how we can keep the human in the loop here.

42:16 All right, so this is kind of the layer of who signs off before the agent acts, right? Many of you have heard the term sandbox. We can use that terminology here as well for um putting an approval layer around the agent. So that you can see here the four-step process. The agent will actually go and propose a booking amendment. It'll actually try to proceed through the process with the context that it has and the source source systems and data that it has access to.

42:43 But then it'll look for a human anme someone in the loop in the process to approve. Right? Once they approve, then that amendment will be made and the change will be applied. This is the layer where you can also put guard rails, deterministic guardrails called hooks around your agent so that it's never overreaching beyond what you would like it to do within its confines.

43:08 All right, so now we have a full agent. Let's say it's in production. It's working great. And let's say that we want to make an adjustment to this agent. Let's say that a new model is shipped or we want to make an adjustment to the business context layer with the skills. How do we actually know if these adjustments are improving the agent or if they're causing a regression?

43:30 So that's our eval layer. This is where your thememes, your business users who are in the loop, who understand the process, help determine a set of test cases to test the agent against with a benchmark or a baseline so that you once you apply any change to any variable, you'll be able to determine whether that variable change actually improved the behavior of the agent or if it caused it to regress as you'll see here in some of the examples.

44:03 All right. So hopefully you've seen by building once with a layer and with a foundational layer that you trust, you can work at at speed to deploy many agents at scale. For us, financial services and regulated industries doesn't mean slow. It means that we're seeing a lot of innovation from the financial sector and we're seeing it go handinhand with security and safety as well.

44:28 And those three layers that we just walked through through, who the agent is acting as, who signs off, and how you still know that agent works are the key components of building a safe and secure agent framework. And once you have this framework that you've built, you can use it to apply to any new agent that you want to build for any specialty use case.

44:50 So you can see at the top you'll have your foundation and then at the bottom you'll see where you can actually bring in more specialty along the way as you continue to scale. That specialty is worked in combination with the business and engineering to make sure that the proper business rules and context are being integrated into the agent workflow. And you might be wondering we haven't mentioned anything like cloud code yet.

45:20 Well, everything that we've talked about so far has, let's say, four different doors that you can access it from. So, a business person can access it from cloud co-work, an engineer can access it from cloud code, or you can take this to the next the next step and give the agent more work to do in your own UI environment via things like managed agents or agent SDK.

45:47 Now, so far we've talked about building a secure, trustable agent foundation framework. Now, let's talk a little bit about the operating model. Some observations that we've seen, especially in financial services, is that the business is now becoming closer to engineering. And the way that we're observing this is that the business will directly work with the agent that engineering has built.

46:09 they'll have direct capabilities to correct that agent or amend it, propose adjustments, and that's where engineering is in the loop as well. Once you see the list of proposed adjustments, you can solidify that as a scale change after you've reviewed and tested all that within your eval framework. And then anything new that you ship goes right back to the business and they'll go through the same loop again to continuously improve the overall agent and the process.

46:43 All right. So, so far we've talked about building the agent framework in a way that you can build once for scale. And we've talked about how you can turn a generalized, highly intelligent model into a specialized agent. Now, I want to hand it off to Belinda to talk a little bit more about what we're seeing in the industry by building some of the foundations ourselves for the next step here, which is Claude for financial adviserss.

47:10 Thank you, Keon. So, what you've just seen is us building the foundation once in a firm. Now, I want to talk about how we could take that same approach to our industry. This foundation we've built in Claude for Financial Advisers. And whilst you might not all in the room sit in the wealth or the advisor space, I want you to take this as a pattern and an opportunity to start to think about how we can actually coordinate at an industry level.

47:31 So in September we launched claw to financial advisers the orchestrator across the stack of an adviser. This was built on the foundation that advisers only spend approximately 20% of their time with clients. We wanted to find a way where advisers could spend more time with clients and more time growing their franchise. So what did we do? We went through the agent foundation and found ways to scale at layer 1.

47:54 We brought connectors in the industry system, the teterministic numbers and the scaled system that advisers trust. We brought into the ecosystem. Layer two, we built skills. We built a plugin that is available on GitHub that builds eight skills and workflows that an adviser can take out of the box, use today, and amend for tomorrow. Layer four, approval.

48:18 The adviser approves everything that comes out of Claude. Nothing is saved or sent without the adviser. And that trust principle holds here too. And so you can see here on the diagram on the left, CRM, market, data, custodians, all the tools and foundations and advisor needs brought into Claude. So let's go through layer 1, the connectors. We brought in 25 platforms across custody, portfolio, planning, data, and CRM names such as Schwab, Adipar, Salesforce, and more.

48:49 And there's more connectors that you can find on the Claude marketplace. This is where the industry builds once, the adviser can integrate once, and this is what compounding looks like. Layer two skills. These are the workflows we built for the adviser that they can start with and adjust and amend to their own firm. For example, premeating prep allows an adviser to bring a full family plan into one brief across multiple systems.

49:12 The followup allows the adviser to bring those meeting notes, enter them into the CRM system and actually follow through the existing systems and tools. Nothing goes out until the adviser approves it. And this skill foundation is a baseline of which we imagine advisers will start to innovate and bring Claude into more and more of their workflows, allowing them to drive impact in their business.

49:36 So in closing, with 28 seconds on the clock, three questions to take to your firm today. The first one is the workflow. How are you applying the frontier intelligence to your firm? How are you gaining edge and alpha from the frontier models today? The second is the foundation. Do you all have a foundation that you trust for building and scaling agents for the future as things become more and more specialized?

50:03 >> One connector. How are you actually building the foundations that others can use? Are you building connectors and skills for your users, for your firm, and for the industry? With that, thank you so much for having us. Enjoy the rest of the conference. All right, good job. And now to explore how serious organizations can accelerate innovation without compromising trust, please welcome John Feminella, SVP of architecture at Two Sigma.

51:00 Hey everyone. All right, I precariously balanced my water over there. Uh, my name is John Feminella. I work on AI architecture and engineering at Two Sigma, which is a hedge fund here in New York. Before I get too far into things, I have a legal disclaimer slide that I have to show you if it says two things. One, I promise not to sell or market or offer you any securities at any point in this talk.

51:22 And two, everything and this is my own opinion and not the opinion of two sigma. Thanks for indulging me if I don't say those magic words. the 5G microchips implanted in me will explode. So that slide is a good example actually of how our industry is regulated. And as a regulated company with lots of controls, I think one thing that's always interesting is to explore what happens when controls fail because that gives you a good idea about why the controls were there in the first place.

51:52 And so I'd like to tell you a story about that by showing you another legal slide. After August 2020, a very interesting contract clause suddenly started showing up in a bunch of commercial loan agreements. And this clause essentially tries to prevent a certain scenario from happening. And it's complicated, but I'll explain it like this. Let's say that you're a lemonade stand business and you borrow money from a lender to capitalize your lemonade stand.

52:15 And the bank that's overseeing that transaction that's in the middle between the borrower and the lender sends the lender all of the money that you owe in a catastrophic mistake. You'd probably be mad and want the money back, but the lender might also say, "No thanks. You paid us back and we're all good, so we'd rather keep the money." Something like this happened in August 2020.

52:38 Revlon, the cosmetics company, had borrowed money under a big 2016 term loan. And that loan was split up into a bunch of pieces across 300 plus lenders, hedge funds, credit funds, CLOS's, other banks. And in the middle is City Bank, the administrative agent. Being the agent in the standard transaction means that you're performing the plumbing that connects the borrowers with the lenders and managing the life cycle of the loan.

53:05 So in this transaction, Revlon pays City and then City pays the lenders and City's not lending its own money. It's just in the middle moving people's money around. So here's the plan for August 11th, 2020. Revlon would pay about 8 million bucks of acred interest to both groups and the lender's principal would roll into this new facility. That's an exchange of paper, not cash.

53:25 So the remaining lender's principle would stay put. And the idea here is that only interest should be going out. principle should not be moving at all. And the bank had a piece of internal processing software called Flex Cube. Isn't that a fun enterprise software name? So in Flex Cube, the easiest way to do this kind of transaction is to pretend to record it like Revlon were paying off the entire loan and that triggers the correct interest payment calculations to everyone.

53:55 You can probably guess what's about to happen next. So, you're supposed to send the principal portion to an internal wash account that you own as the bank so that it's booked and recorded correctly, but it never leaves the bank's perimeter. So, keep your eye on that wash account. What was supposed to happen is that the boxes labeled uh front and fund and principal should all include the wash account number and should all have a check box checked next to them.

54:20 So, unfortunately, it turned out that was what needed to happen, but for various arcane reasons, uh, the boxes were not checked. And although Flex Cube pops up a warning notification that says, "Uh-oh, money's about to leave the bank. Are you sure you want to do this?" That was correct. Money was supposed to leave the bank because they were making the interest payments.

54:38 So, the three approvers who were supposed to approve that looked at it and said, "Yep, that's the right thing that's supposed to happen." And although that warning notification popped up, the human operators carrying out the transaction overrode that. Remember, Revlon was making that interest payment. So, the principal never went to the wash account and instead Flex Cube sent the entire outstanding principle of the loan to all of the 300 lenders, about $900 million instead of $8 million.

55:04 So, every lender got the amount that they would have been owed if Revlon had paid the loan in full. But Revlon didn't want to pay off the loan. It wanted to hold on to its money. That was a great deal for the lenders because remember Revlon was in trouble at the time. So its loans weren't trading at face value. And getting all your money back on a deal that you thought was only worth 60 cents on the dollar is a great deal for you.

55:26 So they didn't want to give the money back even after city called them and said, "Please, that was a mistake. Give us the money back." And their argument was, "We shouldn't have to give the money back because you paid us in full." And that's how loans work. You eventually pay them back. And just because you paid it back at a different time than you were supposed to doesn't mean that we shouldn't have to give it back now.

55:43 And so to prevent that from happening, uh, commercial loan agreements everywhere started including a clause like that one. So what does a payment error at a bank have to do with AI agents? Well, the bank had this procedure. It had these controls and everybody did what they thought the right thing to do was. That's the problem I want to talk about for the rest of this talk.

56:01 Every day we're giving real agents real autonomy over systems like Flex Cube. And how do we prevent accidents like that one from either from happening without locking everything down or just hoping we'll get lucky that they won't happen? So, here's the crux of the issue. You can't really know in advance what an agent will do, right? They're these non-deterministic systems.

56:20 Uh, and that sounds scary until you remember that humans are also nondeterministic systems and they're going to make mistakes, too. And so most firms don't choose to say, well, in order to reduce the risk of humans making a mistake, we're going to like lock them in rooms and prevent them from talking to each other and they're all going to double check everybody else's work, right?

56:36 We allow them a certain amount of autonomy along with a certain amount of controls. So we give them boundaries, visibility, and we equip them with systems and controls that let them do their jobs effectively. So in this sort of model, safety and innovation are not opposite ends of a spectrum that you must trade off on. There are things that you can have at the same time.

56:55 So I think you can have your cake and eat it too. And when we were thinking about what controls we wanted to apply to agents, we also looked at some approaches that didn't work for us. You don't want to sandbox or lock down everything because a sandbox with no real access to systems is not is safe, but it's not very interesting or useful. If you let agents go wild and do whatever they want that's uh you know maybe powerful and freeing from a cognitive perspective, but the isolation is very weak there and there that

57:22 approach is generally not suitable for highly regulated enterprises. And then if you just have policy documents that agents look at or that are part of your uh specification for the prompt, that's also not very enforceable at runtime. You can't guarantee that that will be part of the enforcement controls. So what did work for us was thinking about what mix of controls would get us both safety and innovation at the same time.

57:44 So how do we provide boundaries and stop problems but don't hinder folks from operating correctly? So, we have a number of these controls, but today I'll talk about five of them. And when I say controls here, what I mean are the guard rails and shared capabilities that let teams move in ways that don't hinder what they're trying to do. And we want to we don't want to reach some chasm that we need to cross to to let people get their work done before and then make everyone wait while we build a bridge to get there.

58:13 So, these controls are composable. Each one does one job and then together they let us run agents with real autonomy at an appropriate level of safety. So the first problem is that agents were showing up everywhere. We have big teams with lots of agents running. We have small teams with a few. We have individual people running stuff in a uncontrolled way.

58:36 And these come in every shape. Could be random chat bots. It could be NPC servers glued together in this Frankenstein hodgepodge. It could be cool notebook experiments. be really useful web tools for folks. But it's hard to answer when you look at that a few simple questions like who owns this? What tools does it have access to? What should it be doing?

58:55 What should it not be doing? And if you can't answer those kinds of questions, if you can't govern them, then you can't really also reuse that. So you're tempted to build multiple solutions to the same problem if you don't know about what exists already. And then the firm can't also govern those in any kind of legible way. So one thing that firms sometimes do here is add some kind of gate like don't let anything run until everything is registered and approved and it's gone through this very ownorous process.

59:22 And that sounds safe but if you look at what it does to the work every idea even some simple notebook experiment is going to wait in line behind something that's really risky and we don't um you know we don't want to subject everybody who's given a diversity of agents to exactly the same process. So most people are going to give up or they'll just get pushed to some shadow IT less governed path that is um less visible and then we don't get to see anything at all.

59:49 So that's what motivated the first control is an agent registry. You can't govern, you can't see. So the agent registry makes agents legible and it registers, retrieves and discovers the key facts about agents that we want to know for any agent. Um, so those are things like which tools it declares, what endpoints it's going to talk to, which product does it belong to, who owns it, and humans, other programs, other agents can then use the agent registry to decide what's available at the firm, whether that's useful for

01:00:18 them, and to go use that as a source of truth for other questions. So this doesn't require extra effort on your part. It just happens when you use one of the firm's approved agent building frameworks. It's automatically registered. All that information is recorded there for you. And if you want to do a manual registration, you can certainly do that.

01:00:36 So the second problem we have is that now that we've got all these agents doing all these things, we quickly notice that what they're doing is really different. And that means our agents carry really different risks, right? So a thing that summarizes Jira or linear tickets is really different than a production deployment agent that can touch a running system.

01:00:56 And if we can use the same if we try to use the same review process to cover all of these agents then every agent is waiting in the same queue for weeks for human review. And if people are going to route around slow processes then we wind up in the same situation that we had before. So again you might be tempted to say the answer is something like really detailed heavy manual review by all of these different constituencies.

01:01:18 Again, that feels safe, but now we've got a trivial summarizer waiting two months for the same signoffs as an agent that is touching production systems. And and most of the good agent ideas, most of the ones that have value right away, start out as small experiments. And you don't want those people to wait in the same line as the stuff that's touching production right away.

01:01:39 So the reviewers are drowning in these like lowrisk requests and the high-risisk ones have to share that attention pool. So we don't want to trade away that speed if we can help it. So again because not every agent needs the same review and because most agents are purpose-built rather than open-ended and when I say open-ended here you can imagine coding agents like pi or cloud code when you open them and you start typing uh cloud code or pi doesn't know what you're about to ask.

01:02:05 It doesn't it doesn't circumscribe its purpose in any particular way. But most of the agents that are deployed in enterprises do have very specific purposes. when you're engaging with that agent, it's to do something specific that's well known and well understood. So, they have a very specific reason for existing and a very circumscribed purpose. And you can tell these agents apart from each other by looking at what they declare up front, their tools, um the endpoints they access, the controls that are applied to them,

01:02:32 etc. So, the third problem is that if you know about your agents and what they're doing, you'll quickly notice that many agents don't talk to each other. or sorry, they don't talk to just one thing or in just one capacity. The more open-ended the agent is, the more it looks like a cloud code or PI, the more it can do a variety and a diversity of different things.

01:02:54 Call an internal API, run a tool like a web search, that sort of thing. And each one of those is a different opening a different door to a different kind of risk. And if you give an agent an entitlement for a RO account, then it can do a lot more damage than something that's just going to read a public web page, for example. So the question is not can this agent do X but what's the granularity that we need to solve for for the agent to act successfully.

01:03:21 And again one tempting answer is to kind of be heavy-handed here or to give every agent the same short allow list of stuff that's safe to do and then block everything else. Uh that's safe but it's also useless. an agent that can't uh that can't search or can't call an API or can't do any of those things without asking somebody first can't really do the work that it was intended for and then teams either give up on those agents or they find a way around the wall and both outcomes are costly.

01:03:46 So a wide kind of blanket block on everything trades away the innovation that we want to preserve and avoids making a real decision about risk. Um and as author John Chad wrote a ship and harbor is safe but that is not what ships are for. So the fix here is to let agents declare their purpose upfront and say what tools they're going to use or what tools they need access to.

01:04:12 Um so in the second column here we might say that a research summarizer is going to be allowed to use a web tool to access research. But for a release deployer agent, we're not going to allow them to do that because they have no reason to use that tool. And so if they try to ask for it, they won't get it by default. Um, so we don't need to make this kind of decision for every tool.

01:04:35 The agents just have to say which tools they want, right? And and you're as the agent author, you're in the best position to say, I need these four things out of the 500 that are available. So the way we need to enforce that is that when agents run in our managed environments, they look at the registry that we talked about before to look up what policies apply to it at initialization time and that policy gets translated into enforcable statements like which endpoints are you allowed to access, which hosts are you

01:05:05 allowed to access, uh what other types of resources are available or not available to you. And in this way, nobody's writing a ticket every time they need to change the access. Nobody's going to talk to it or some other group outside of your team. As long as you conform to the risk score that we talked about before and you're in the agent registry, you're you're governed.

01:05:26 You're approved. So this is how we keep the cost to teams really low for deploying, building, running agents. And the same control architecture works in a variety of settings. The next thing we noticed was that teams were building a ton of different ways to evaluate agents and every stack looks different and some teams don't build anything at all. Sometimes you build like this really detailed stack for how your models work.

01:05:50 Sometimes uh it's sometimes there's nothing. And I think in general we'd be upset if teams were not writing unit tests, right? And we consider eval to be kind of the same thing. If models are changing, that's like a runtime dependency that your application has. And if your agent is going to change models, it doesn't know without the eval what's going to happen to its to its outputs and to the quality of those outputs.

01:06:14 So again, one tempting answer here is let's have some kind of central evaluation methodology that waits for a full eval suite and a committee signoff, but now every like prompt tweak, every oneline change is going to sit in that same queue. So we don't want to put more decisions on more reviewers for things like that. We we don't want to trade away that iteration.

01:06:35 So again, you wouldn't ship software without unit tests. So agents need evaluations for the same reason. And we want to provide the stuff at the level of the firm that's going to be the same for everybody. So team A writing evals, team B writing evals, they're both going to have many of the same features. So we want to have one shared approach to how we do evals and have the shared approach be something that's run by the firm and available to everybody for doing their evals.

01:07:01 So that covers all the common parts of evalment. All you've got to do is just write what's true about your agent. Uh and finally, maybe you've had the experience of being asked to approve lots and lots of individual agent actions when there's a human in the loop. So that's fine at a small scale, but once you're operating agents in parallel or in production, this becomes completely ownorous and review requests come in faster than anybody can be expected to approve them.

01:07:27 And after the first few times that it's happening, you succumb to review fatigue where you just see yes, yes, yes, yes, yes. And eventually you click yes on something that's, you know, activate the nuclear bomb, right? So you don't want to click yes by reflex, but that's going to happen inevitably. And one recent example of this is uh earlier this year, a director at an AI research lab told her openclaw agent to suggest emails to delete from her inbox and wait for her approval.

01:07:51 But her inbox was so large that eventually the agent compacted its context and lost the part that said, "Wait for my approval." And then it deleted hundreds of emails and didn't stop when she told it to stop on her phone. So again, you might be tempted here to make a human approve every action, but that's not the right way to do it. A better approach is to let these independent conservative classifiers make decisions about routine calls and then send a person only the requests that are most important, the ones that are

01:08:20 most likely to make a difference. If you're saying yes 99.999999% of the time, then that's probably just a rule that the agent should follow, not a request you need to ask the human for permission for. And the stuff that's unclear or ambiguous or potentially dangerous, yes, surface that to a human or if it's definitely just dangerous, deny it outright.

01:08:38 So, this is a way to not have to do all the review work all the time. So we want to take the same approach for both explicit uh when explicit rules apply, we want deterministic fast paths. When explicit rules don't apply and there's some ambiguity, we want a classifier to do that work instead. So putting those five pieces back together, you create a registry so it every agent's visible from the day it's created.

01:09:02 You score each agent so human review goes in the right spot. You run agents on guard rails so that they only touch what they need. You share evals and metrics so everybody is reaching the same bar without having to reinvent the wheel. And then finally, you govern your runtime to make sure that classification happens there and not somewhere else. So each of these controls works together.

01:09:24 So I'll leave you with one question. Why do cars have brakes? It's not just so that you can stop. It's so that you can go fast and stop safely. So controls for our agents works the same way. Thank you very much. I'm John Finnelloo and I hope you had a good time. Here to help us understand the missing principles around code quality and finding a path through competing viewpoints.

01:09:59 Please welcome Max Canat Alexander, principal software engineer at Capital 1 and author of Code Simplicity. All right. So, today I figured I'd just pick a nice uncontroversial topic that nobody has any opinions about, right? Nobody nobody's said anything on the about on this about social media, you know, and of course every take about this on social media is like really deeply nuanced, you know, like people have lots of like depth and like, you know, really big thoughts, you know, maybe there's a little bit more to it

01:10:52 than like yes or no. And you know, of course, if you ever ask a senior engineer a question, we always have the same answer to every question, every senior engineer's favorite answer to every question, which is it depends. Now, that's not a very helpful answer though, frankly, because what does it depend on? So, let's start with this. Why did code quality ever matter in the first place?

01:11:17 Like, I see a lot of people be like, well, it made me feel good. That's not really a sufficient justification. So, let's see if we can inject some nuance into this conversation and help everybody at least be able to have a basis for reasoning about it while we're discussing it. So, let's see if we can make a framework here. And that's what we're going to try to do with this talk.

01:11:32 Okay. So we're going to start off with everything you want to ever accomplish with a piece of software uh depends on two factors and two factors alone and that is understanding and executing on that understanding. Whether that's aligning a model, scaling a system, driving adoption, anything all software success is determined by these two factors. This is why we have things like debuggers and observability and evals and even tests.

01:11:58 All those things actually are there to improve your understanding or ability to execute on that understanding. People think that they're there for other reasons, but fundamentally they're there for that reason. Everything we have ever done in the field of code quality is exclusively and only for these two things. It's not for anything else. And that doesn't have something to do with correctness or something else.

01:12:16 It has to do only with these two things. Now, there's two understandings that are the most important to have if you're going to be a software engineer. There are other understandings to have about software if you're going to be a product, if you're going to be in sales. But if you're a software engineer, there's the two that are the most important. And the first is what is the behavior of the system?

01:12:34 Like the concretely, not what does a doc say the behavior of the system is, but what is the actual behavior of the system. And the second one is how do I correctly and safely modify that behavior. Okay, on a very small system, if you want to understand what the system does, I just tell you what it does. It prints hello world. You say I wanted to say hi world.

01:12:57 I say okay I will make it say hi world now there's very little to understand about it other than purely what it does that's all of the understanding that you need any real software system is enormously complex map of the codebase of chromium which is the open source project behind if you think about all the different ways that these components can interact they have essentially infinite possible behaviors like I could not describe to you in words the behavior of Chromium as a whole as a system.

01:13:32 However, what I can describe to you and as we all know, everybody's been a software engineer for a while. I can describe to you components of the system. I can successfully describe to you how a component of the system behaves. So, this is the URL parser, the actual functions and stuff for the URL parser of Chromium. figures out all the different parts of the URL.

01:13:54 And this is the whole reason that code exists. The whole reason that we have code at all in the first place is for structure. It because it what do I mean by that? Okay, so both of these two things on the left and on the right will print hello world if run on a Linux machine. However, on the left, you can see that it is hello world. It has the identity and structure of hello world.

01:14:25 On the right, you just see a series of instructions with no particular identity, recognizable or understandable identity. It's not a component. It's just a bunch of instructions. While that's understandable there, if that was millions or billions of lines of that, it would be much harder to gro. In the past, we've talked about things like readability or testability or things that seemed like human concepts about code.

01:14:49 But what actually mattered the most always was that the system had some sort of understandable structure. So it's the structure that actually matters. And when this modifies when this matters the most is when you want to modify the system. The second uh understanding that we talked about all modifications to a system carry two categories of risk. The first is that you will introduce some sort of problem.

01:15:14 And a problem isn't just a bug. It's anything bad that could happen. Like your project could take much longer than you expect. Uh it could not behave in the way that you wanted it to for your users. The project could fail because you have budget overruns. Any kind of problem could happen from just you trying to go through the process of making this change.

01:15:31 And the second is you could make it harder to make future changes. In particular, what that means is you could make it more likely to have future problems. you could increase future risk of the system. These two things are caused by un problems with understanding. So the first has to do with the difficulty of understanding the system in the first place and the second is you make the system harder to understand.

01:15:55 I can reduce every postmortem I have ever read for any incident to these two factors in one way or another. The the the production configuration was too hard to understand. Somebody made something impossible to understand. There were emergent properties of the system that were impossible to understand. Whatever point 2 feeds into one in a vicious cycle that compounds in a nonlinear fashion.

01:16:22 So difficulty of understanding does not grow linearly. It compounds over time. You will eventually reach a point and all systems eventually tend to reach this point where every modification involves some sort of problem. Uh and you know some some systems get to the point where like you can't modify them at like if you think of like you know some of like your giant 100 million 500 million line mainframe code bases and the everything every change is some sort of problem.

01:16:52 You know you'll see it in organizational issues. You'll see it all over the place. You'll see it in timelines whatever. And look there's a lot of arguments about maybe future models will fix this. Look maybe they will. Maybe future models will have infinite ability to deal with complexity somehow. uh but we don't have to discuss that today because I think models are doing something else really interesting right now they are creating a new kind of software a new category of software this category has always existed but

01:17:20 it's about to become a 100,000 times more common than any other form of software and that is disposable software so what is that okay So imagine a world in which the consequences of failure for your system are very minimal or non-existent. Imagine that there's very few people that depend on it or and almost no other systems depend on that piece of software.

01:17:48 Change management very minimal non-existent. Imagine the cost of replacement went to near zero. You could replace the whole thing in hours or days and it had basically no cost of maintenance. In that world, code quality does not matter because it all of those things we were trying to prevent about the compounding risks over time don't exist. Traditionally, we've always built durable software.

01:18:20 You know, it lives for years. It's depended on millions or thousands of people. There's huge consequences of failure, potentially business catastrophic consequences of failure. And it requires enormous maintenance. Most of the software engineers in the world spend most of their time maintaining these giant durable systems. I expect this to continue to exist.

01:18:39 So this is one of the areas where I think sometimes the conversation goes a little. Okay. Durable systems will continue to probably be the most important systems in the world and there will be probably more of them. But the most numerous systems will be disposable software. They will be replaceable in hours. They will be depended on by a few or change management just won't matter.

01:19:06 And look, if it doesn't work, you will give everybody agents. They'll make a version of it themselves. You know, maintenance will be unimportant or nearly free. Don't like the calculator on your OS? Make your own. Want to plan a party for your kids? Write a party planner software just for that one party. throw it away. Code quality doesn't matter. Nobody cares.

01:19:30 In the future, we will apply engineering rigor to the degree of disposability of the software. On the left, nothing, none of our practices of engineering rigor matter at all. On the right, all the practices of engineering that we have built up over the last many decades mattered greatly. We can determine where you are on this scale by those four factors that I have been talking about.

01:19:55 consequences, cost of replacement, dependencies, and cost of maintenance. If they're all low, it's disposable software. If they're all high, it's durable software. And look, any person who's written software for a long time knows you should heir on the side of it being durable. You should air on these all being higher than you expect them to be. But there's also a new thing you can do in the world.

01:20:17 You can make software disposable. So like what what does that involve? come up with better isolated environments, right? So, hey, you can only deploy to this environment that has access to only your data. No customer data, no personal identifiable information, whatever. You can enforce independence. You can say, okay, no inbound API calls from other production systems are allowed to this system in this environment, right?

01:20:43 And so everything deployed to that environment is disposable. Redundancy. You can have an old clunky backup system that is like hard to use and then like look if people want to vibe code their own like more efficient front ends you let them vibe code their more own more efficient front ends because there is an old durable clunky backup system that they can use if they're you know crappy little vibecoded thing fails which which it will fail right that's fine but it doesn't matter if it fails uh and you can limit scope

01:21:09 just like I talked about right it's just a calculator for this one person it just plans this one party there's so much more we could do here and talk about ways of increasing disposability. In fact, I think this is a whole new discipline of software engineering. Not to think about how we improve the rigor of durable systems, but how do we take our systems and how do we encourage disposability or allow more disposability, allow people to just throw these things away.

01:21:37 Interesting thought. Now I want to re-emphasize that durable software engineering is not going away because this is the biggest confusion that I see on social media today. In fact, I believe the absolute amount of durable software in the world will increase because AI coding tools make it easier and easier to make more durable software. And another important thing to understand is that disposable and durable are not black and white.

01:22:05 There's a spectrum like I showed before, but that spectrum, what changes across the spectrum is how much effort you're going to put into engineering rigor. Okay? And that's both immediate effort and effort over time, just like we've always talked about for software. However, what you want to do, and I think what you want to care about the most is reducing the cost of effort, reducing the cost of rigor.

01:22:33 Like if rigger is cheaper, if it's the e the easier and easier you make it to be rigger, the less you have to worry about where you are on the scale. So this is another approach you can take. If you change your mind, it's much easier to like you're like, "Oh, I thought this was going to be disposable and now it needs to be durable." The cheaper you make it to make it durable, the easier it is to change your mind or to be wrong because you're never going to be 100% right all the time on that estimate.

01:23:00 So that involves a different set of practices that are not engineering rigor themselves but that are about how to decrease the cost of engineering rigor. Now I could talk about this for hours. I have spent a tremendous amount of my career doing this. Uh I don't have 10 hours. I have about five minutes left on that timer. So I'll tell I'll talk about what I can.

01:23:20 Okay. Let's talk about standardization. Why do we do standardization? People are like well I don't know because the boss said so. Well, now we do standardization because it removes possibilities. If you don't have any PHP in your system, you don't have to figure out what are all the llinters for PHP, what are all the practices, what are all the security holes I have to fix.

01:23:42 You just eliminate possibilities through standardization and it just reduces the cost of rigger. Uh, same thing with like frameworks. Like why don't you want to have every front-end framework or every backend framework in the in existence? Because it makes rigger more expensive. Like yes, rigger is still possible. Yes, you could still have the agent code up all your tools, but you'd have to do it every single time.

01:24:02 Uh, you know, pro protocols, why do you want to have standardized RPC protocols? All the same thing. Standards are all for this reason. Okay. Uh, what I call inworkflow tooling. So, inworkflow tooling is tooling that just occurs. The programmer doesn't have to choose to run it. They don't have to understand how it runs. It just happens. If it's there, it it feels and seems free and it is reliable.

01:24:24 Reliability is very very important. Like it doesn't have a ton of false positives. It's fast. It just happens. And the more that's true, like you can apply it to your disposable software, too. If it's free and fast, who cares? Uh making testing easier. So, this is not just about like, hey, how do we do better testing? Look, how do you make it so that nobody has to think about how to do better testing?

01:24:45 How do you make it so that every test is fast and deterministic without anybody having to be a deep expert in testing? There's a ton of work that you have to put into test infrastructure to make that possible. For example, you know, one of the great dreams that I've always had as a as a test owner is that everybody would provide in-memory fake versions of their services and I could just run those in-memory fake versions of their services.

01:25:08 Guess what? The coding agent can do that. So, you can just do that now, right? But you have to choose to do it and make it available to everybody. And another thing is look maximum strictness. You just turn the strictness up everywhere. You turn you set all your lint rules to maximum. You you never compromise on anything. And it just eliminates it's very similar to standardization.

01:25:28 It eliminates a ton of conversations. Sometimes the llinter is wrong. Like as long as the false positive rate is relatively low and it doesn't have catastrophic consequences, it doesn't matter because all of a sudden there's a bunch of future lint errors or future issues that you don't have to deal with. Like ask anybody who's been like, "Oh, I suddenly want to turn on my llinter in this million line codebase."

01:25:45 And they're like, "Oh, wow. Now here's my 100,000 line change that I need to make." Right? Much easier to just do that up front and have it on for everybody. Now, look, like I said, I could literally stand here and talk to you about this straight for 10 hours. There is much much more you can do here. This is just a few areas, but also a whole area of software engineering discipline that you can apply on the other axis.

01:26:05 So this lets us now answer our question not just it depends but yes or no. So does code quality even matter for disposable software? No. For durable software yes. And we get to answer a third and even more interesting question which is does it have to be expensive? No. Thank you very much. As AI takes on more of the building, what does it mean to be an engineer?

01:26:58 Here to explore why defining success may become the essential skill. Please welcome Aparna Dina Kuran, co-founder and chief product officer at Arise AI. Hey New York. Awesome. Well, welcome to AI engineer. Um uh thought given we're at AI Engineer New York, we'd kind of revisit the question of what does an AI engineer do? Feel like every year we have an idea of what an AI engineer does and that definition kind of changes.

01:27:38 Um, and as I was putting this talk together, it was super timely because Karpathi actually just tweeted something this weekend where he kind of he kind of talks about this growing gap between basically folks at the frontier who are running thousands of agents and swarms of always on agents. And he he really kind of drops this funnel where 6 billion plus users have maybe touched or interfaced with an LLM.

01:28:06 1 to two billion are maybe like casual free tier users. 20 million are professionals who have maybe started using it for more taking some piece of work that would have taken weeks down to to days. But really those at the frontier labs are which is a super small percentage. It's like 5,000 users uh are the ones who are running thousands and thousands of agents at once.

01:28:34 And as I look at this, it really is I mean he's kind of dropped a bit of the funnel. I mean the first question anyone looks at this is that where am I on this? And you know he's kind of drawn out a bit of a road map of how does everyone get to in the next couple of years one of those 5,000 users who are running thousands of agents. And this has been happening I'd say for for a couple of decades now where the abstraction layer has gotten higher and higher.

01:29:01 I mean definitely faster in the last couple of years but we've gone from engineers writing higher level abstraction languages to I mean just copy pasting code using local agents to using cloud agents to really the expectation now is how how do all of us start running a team of agents or factories of agents internally and build toward that and what this has led to is essentially what is it then and now of what an AI engineer is.

01:29:32 And this is this is this is a bit uh I I think of a of a roadmap for some, but I think internally this is what we've already started to kind of push towards. As an AI engineer, your job's no longer to just write the feature, write an AI feature. It's about how do you build the loop to write that feature. It's no longer about, you know, just reading the diff and actually doing code review.

01:29:57 It's actually how do you set up systems and agents to do the code reviews and how do you write evals to actually do the non-deterministic um to review those non-deterministic tasks. It's actually no longer even about just setting up metrics and and software kind of telemetry, but it's actually how do you have agents go through and read the observability data and have agents actually consume that telemetry to find patterns.

01:30:21 And so this loop that we've kind of been talking about which is the build eval ship you monitor you improve it. Well the AI engineer is now actually responsible not just for doing each of those steps but how do you build the loop that does that actually does each of those steps and at arise one of the things we've rolled out in our product and I'm going to talk about a bit of what we've learned in building this is we built signal to actually run this loop for AI engineers.

01:30:50 What signal does is that signal actually looks at all of the traces. It reads the telemetry. It identifies patterns in that telemetry and it can actually put up a fix or a PR if it has access to the repo. And signal signal is an always on agent. So this is an example of how signal starts, but it actually starts off by just reading the traces. Super simple example, but it goes through reads um you know all of this is configurable but an a certain volume of traces that came in in some cadence it reads it and then the

01:31:26 goal is to actually identify issues from that telemetry data. It has to keep track of issues that it's previously seen. So we have to consider the memory of did it see something before that it thought was an issue. Did something seem like an issue, but it wasn't actually captured? And now as it collects more data, it actually thinks it's much more of a critical critical issue.

01:31:48 And so, Signal actually starts off by reading the traces to identify problems. And so, one of the first things that that we realized was that the future of who's actually consuming telemetry data has really changed. It's no longer about humans consuming telemetry data. It's not even about humans consuming it and agents kind of taking on the task, but it really is how do you set up your telemetry data so that agents are the primary consumers of that data.

01:32:18 And what that does is that that observability data can become the trigger for actually running and triggering these always on agents. And so the first thing that if you're building a team of you know a factory of really always on agents is what triggers these agents to wake up or start and there can be a variety of triggers for signal for us it's you configurable to run on an ongoing cadence.

01:32:46 So every 3 hours 6 hours go pull the last you know x amount of traces read them identify problems. It can also be triggered by events. So, Slack messages, um, if some user complains about something, it can be a metricbased trigger, but these are essentially triggers to kick up the agent actually starting. Once the agent actually starts, we have to spend some time figuring out, well, what is what does an agent run actually look like?

01:33:17 For us, we actually pull in all of the context of the traces, um, any additional thirdparty data. So we allow collecting to APM logs, APM metrics, um but all of the third party telemetry data that we want to pull in. Um access to the repo, um access to the actual um the the access to any keys that are needed to actually run run this agent. But all of this information is um put inside of a sandbox.

01:33:48 And this is actually what's started up every time the agent wakes up. And so the agent's triggered. We build a sandbox. We put everything that we need inside of that sandbox. There's a proxy. The only way to get information in and out of the sandbox is by this single proxy. And everything that happens inside of that sandbox is actually already getting traced.

01:34:09 And every output, so the issues that are generated and created are all getting saved to some kind of memory. And so this happens every single time the agent actually runs. Every time the agent finishes running, so it sweeped through all of the traces, um it either found something or it didn't. There's what actually stays and what actually gets thrown away.

01:34:32 What stays is that if it found any issues, all of the output is actually saved off in some kind of memory so that the next time an agent gets spun up, it doesn't have to start from ground zero. It has a history of what are the issues that it actually found. Um the agents are traced. So all of the telemetry data of what the agent called um you know if it actually put up a PR all of those things are actually traced.

01:34:58 And any skills that it may have learned while actually doing a run can also get saved to the actual repo. But the things that go away is the sandbox gets torn down. Any credentials um you know obviously get get removed. um the repo access is removed and so all of the stuff that doesn't need to be saved kind of gets torn down after the run of signal and we've designed signal in a way to actually keep this agent configure uh durable.

01:35:27 So that means that either if the s if the sandbox fails or if the harness fails then it shouldn't have to start everything from scratch in the middle of the run. it should actually just be able to pick up from the step that it failed and all of the state so-called is kind of saved off in kind of a a a memory outside of the sandbox. And so we've designed this to actually be a very durable agent so that if it fails it can spin it back up um you can launch a new sandbox, launch a new harness to kind of pick back up where

01:36:01 it started. So all of this is essentially the underpinnings of how we built signal to run. Every single run agent gets triggered, agent gets started, loaded with all the context, the evidence. Uh it sweeps through the telemetry data. It either finds something or it doesn't. All of that gets saved to memory. If something fails in the middle, the agent can get reset up.

01:36:22 And we built Signal to also be, you know, agnostic of the model or the harness. So today, Signal supports not just cloud code, which is what we started with, but it also supports cursor and codecs. And the goal is to actually support all of the the major even the open source harnesses out there. Um, and signal actually also supports um multiple uh will support kind of multiple models.

01:36:48 So even if you're using cloud code, you can pick any of the models inside the anthropic family. Um, and this is really important to us because one, it's an awesome way for us to evaluate the quality of the agent. Um, every time we spin it up, it helps us control costs. So we can see you know as we use more powerful models the better quality maybe the PRs are but can we reduce the quality of the the you know can we reduce the cost and actually still maintain the rigor um and maintain the quality of issues that are

01:37:21 actually detected by signal. So this is something we deeply thought about from from first principles when we were designing signal and this is actually a pretty good segue into what does an AI engineer need to do? Well, an AI engineer really needs to kind of learn how to write eval. And so we actually every time Signal runs, Signal finds an issue, it puts up a PR.

01:37:45 Well, there's eval on top of the agent to test that PR. Um, and an eval can be as simple as um, you know, testing something as part of your development workflow. You have a data set. You can be a simple prompt um, with with outputs that you're testing to make sure that there's no regressions. Um, but these eval can also they need to be kind of aligned with human judgment.

01:38:11 And I'd argue that this is actually probably where an AI engineer should be spending a lot of their time as they're building the system to actually run the loop. They should be actually spending time looking at the traces with their agents, identifying what the common errors are. Um, and there's a lot of skills. I think I was hearing Lance yesterday talk about the claude skill to build evals.

01:38:32 We have a skill with Arise um evaluator to actually help you build and write evals as well as align those evals with the human judgment. But this is probably where the shift to defining what is good becomes really really important for an AI engineer to actually do. And for a lot of teams I I think we speak with and we work with their question becomes well how do I trust the evaluator and do I do I really trust that the eval is um you know is going to catch or you know how do I trust that it's as good as my judgment and

01:39:10 I'd say what we spend a lot of our own time doing for our own products is that we actually evaluate the evaluators. Um, we spend a decent amount of time on actually building these data sets, golden truth data sets. We run evals, verify them against the human output and then try to align to make sure that the judge actually agrees with the human um and and they're in agreement before we actually ship out uh a new evaluator.

01:39:39 And these eval can then be used both in an offline testing as well as in an online testing manner. Um, and this this is probably what I would say as an AI engineer is more and more of what the job is becoming. Um, and so what I'm going to leave you guys with is as an AI engineer, you've built this agent. You built this agent to actually run be always on.

01:40:04 You run evals before you actually ship it into production. You can ship it and then once it goes into production, the goal is to actually trace everything. And tracing everything now means not just tracing the agents, but also tracing the coding agents behind the agents that built this. So you can trace um cloud code, you can trace cursor, you can trace codecs, but all these coding agents can be traced.

01:40:33 So you can understand what happened um in the factory of agents that you all are building and then these agents that all the traces can then also be evaluated online. So there's an offline evaluation when you're actually testing what your what a new change is. But as it goes into production, these become your online eval that you're running on top of the telemetry data.

01:40:57 This for us ends up becoming the new data that then gets pulled back into the agents to read the telemetry data, the eval surface up kind of problems inside of the traces. So the the system really becomes how do you go from collect data, eval that data, run agents on top of it so that you can find problems and figure out where the gaps are to actually improve improve your systems.

01:41:26 So if I go back to kind of the question that we started this whole thing with, it's what does an AI engineer do? I mean the gap between I think there's a pretty big gap between what's happening in the labs today and what's happening with a lot of businesses and some of it could just be the data inside of businesses are a lot more messy than what is very constrained and testable inside of a lab but AI engineers are kind of the the role to close that gap and I think it really starts with tracing everything tracing the

01:41:58 agents you're actually building running eval so being able to learn how to test these test these products that we're really building and then how do you build a system so that agents can actually improve um from the telemetry that's actually being collected um so that you're no longer just building individual features but you're actually building the system to improve um to improve your products.

01:42:21 Um if you want to learn more come check out we have a booth um you can check out Signal learn more but thank you so much for listening to us. The current program has finished its run. Please collect your belongings and proceed through the main doors for your next scheduled task. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat.

01:45:43 Heat. Heat. N. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. N. Heat. Heat. Heat. Heat. Heat. Heat. Heat up here. Heat up here. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. All right. Thank you guys. Uh my name is Freddu. I'm part of the web team. I'm leading the AI modeling organization there.

01:55:28 And I'm going to show you the work we've been doing lastly on building a our first foundation model uh a Wells Fargo foundation model. So I explain why we did that, how we did it and what we accomplish in doing so. So so first foundational model it doesn't necessarily need to be language based. I mean we all playing with uh code and chat GPT and others they all language based foundational models.

01:55:54 In financial services uh we also have language of course uh from a structure unstructured representation data but we have also have a lot of structural representation. A lot of this data is tabular data. We have a lot of information from our customers. Uh we have information in investment banking, consumer banking and all of these data is data that we have in in Wells Fargo and all big financial organizations have too.

01:56:20 And uh we want to take benefit of all of this huge amount of data that we have. And it's not like you just take it and put it in a in a language model and get the best of it. Of course, you can get a lot of benefit out of it, but it's not necessarily the best way to do it. So, so what we decided to do is to look at the data we have all the transactions all the customer transactions that they're doing with their debit card, credit card, digital check, all of those coming from many different channels and see if we can

01:56:53 build a foundation model that we can use and fine-tune for downstream task like credit scoring or that you can use it for fraud detection, antiming, do marketing. So how we can build one backbone that you can use and streamline across downstream task. So now you're moving from many models very siloed across different organizations in really one backbone and every organization can just reuse because you learn the representations of customers transactions at at scale.

01:57:25 So that's the intent and just spoiler alert you may say hey that's tar data why do you just don't use eg boost and the usual suspect we did that this is the model we had in production and but the model I'm going to show you is something that get that beat xg boost uh even from a latency perspective but also from a from a fraud detections we get much better false uh um reduction of false positive and better detection rate so it was a really good lesson learned in term of having better model using foundation model and to

01:58:01 foundation model I mean transformer blocks and that that's that's what we build so uh so we have as I mentioned billion of transactions uh on a monthly basis um our customers interacting with their debit credit card and they do check and they do different sort of transactions and the the question is really how we can use this informations at at scale and not for one and every single downstream task So I'm going to cover a bit of the data, the representations, the architecture that we use, uh how that what the result we

01:58:35 get and the lesson we learned in building this. All right. So I mean left hand side language foundational model they're processing language tokens and then the the goal is to learn a representation that's going to predict the next token and then you have uh you have supervised fine tuning and then you do LHF LRVR and all of the great uh post training process to get uh better at the task that you want to right and that's for languages that's great uh but what if your data is a very much more specialize in term of

01:59:09 representation. So that's where we build this transaction foundation model. So instead of having languages as the base, the base is now a transactions transaction from our customers and you still try to predict the next thing and here the next thing is the next transaction or it's the next token in the transaction using all of the channel that you have.

01:59:31 Uh but that's not the end goal. I mean the similarly to language model where you try to predict the next token that's not the end of the story right you learn a representation and then you use that representation and then you you you fine-tune it and you post train it for any other task right so it could be classification task for languages but in our context for transaction foundation model it could be used for uh credit line increase could be used for doing fraud detections and flagging bad actors from an

01:59:59 anti-moneying perspective and the goal is to learn these this uh relationship between transactions over time and transactions across customers. That's really the intent. So the intent number one is you do pre-train from scratch. I'm say again because people sometime confused we pre-train transformer blocks uh at scale from scratch using our uh transaction data.

02:00:26 So the weight is zero weight that we use from open weights. we just use the architecture and then we we pre-train it from scratch using our GPU cluster and then we train it for couple of days and and weeks and then then you do the the downstream task of uh of post training but the goal here and you pre-train it for predicting the next token or the next transactions and you you try to to do this right you try to predict the next the next p pattern of the customer but that's not the intent the intent is to do the

02:00:56 downstream task later on and in term of scale scale. Uh I mean we we operate at a large scale. So we have billions of transactions, a million of accounts uh hundreds of transactions per clients on a monthly basis interacting with their debit card, credit card or doing check and the the five main dimensions we look at is really who is doing I mean who is processing and who is asking on who is providing a service.

02:01:25 So will it the customer or the merchant and the what? So I mean what what do we what do we buy? What do we transact and how? So which channel is it credit? Is it debit? Is it digital? When is it happening and where is it happening? And all of those they basically breaking down in 100 of features. And uh if you look at the existing model we have in production there's a lot of I mean people that been build those model in similar organizations may uh may understand that they build a lot of very artcrafted uncrafted

02:01:58 engineering feature engineering efforts on that and that's required a lot and a lot of time. So we step back and look at really the raw features we need not the one that been really pre-built and postbuilt for doing the downstream task but try to understand which raw features and we ended up with basically 100 features even if at the end of the day we had thousand of them but we tried to find which ones are really the the raw information unit and then we let the architecture the transformer blocks to figure out the

02:02:31 relationship between those core features And uh as you can see we do have a yeah a bunch of features on those five dimensions and the vocabulary is very unique to the financial services setup. So and that's the reason why I mean we didn't really use uh existing pre-trained model uh because those the vocate that we use is very very specific to the work that you want to do and even if you merge uh well all digital and checking and uh saving and um all of those sort of transactions even if you merge all of those you still

02:03:12 end up with a very limited number of features and characteristics. So at that scale then the question is well how how you work with that. So, so we started everything is tabular data. I mean there's no secret it's tabular data and then the goal is to understand how you going to capture the vocabulary because you do have numerical values you have categorical values you have I mean you have information like zip code do you encode each and every zip code in a token does it make sense so there's a lot of effort you need to

02:03:42 push in understanding the vocabulary that you need and develop that you want also to scale because at the training if you have a two huge vocabulary it would kind of messy uh to train such such a model with a vocabulary which is far far too big. So so there's a lot of effort in understanding this uh value to token. So which information do you encode in a token and then you have the token to the tokenizations and then the embedding lookup.

02:04:08 So then when we have a transaction coming up you do have the vocabulary. So it's a basically it's a lookup and aligning a transactions and all the characteristics of that transactions to to a token and an ID and then an embedding. So just to give you numbers well some numbers. So we have yeah quantitative, qualitative, categorical of different kind right so categories with only eight types some with much bigger types uh you have transaction amount which is a big deal you have the delta time between transaction so you

02:04:43 have anything related to IP address postal code just to give you example of uh variety and iterogenity of each and every uh characteristic could be encoded in a binary token or could be encoded in a much larger larger number of tokens and even if you look at the bit that the zip code I mean you have a much larger number of zip code in the in in the US than those 2800 but you you have fancier well you have much better way to structure how you're going to uh regroup zip code based on distributions for instance so uh

02:05:16 architecture so so here just um what we so we use the the chinchilla um well mathematics behind the to define how we build it right I mean initially because we we had no clue how it's going to work in the tabular space even though there are existing uh well well open weights and open solutions for doing um tabular data foundation models uh so we we started from what they learned and adapted to what we needed to do and that give you an example of invading dimensions number of decoder blocks we used and all of the the

02:05:54 recipe from a from an architecture your perspective. Um, so we have and as you can see, we're not trying to build a trillion parameters model. We don't try to compete with any of the u the the leaders in the language models because we working really and focusing on transactions, right? transactions foundation models at billion level and one one billion is because that the model that give us uh that was the well kind of the smallest and give us the highest boost and now we working on a on the next versions at 11 billion

02:06:27 but that's good enough to to give a lot of uh ai uh from a false positive reductions for don't stream task like for detections and detection unit detection rate so and the and the type of architecture. So the decoder only architecture that's basically for predicting the next the next token or the next transactions and that's really the pre-training.

02:06:52 So uh so here we initially used 20 times the number of parameters to define the number of tokens we might need we ended up actually 10 times factor factor than that. So by doing a few experimentations we needed much more u much more tokens per parameters. I mean this is this kind of the initial experimentation we did. Um so I'll pass on this because this is the the usual suspect from a uh objective functions a causal language uh modeling and mass language modeling type of uh um uh optimization work and and yeah from an

02:07:28 architecture perspective nothing really I mean fancy here the again the most complex part was really the tokenization and then and then adapted and doing the pre-training and then how you do the fine tuning. So if I would say the the recipe or the secret source is really how you do the tokenization and how you going to do the post the post training but anything from a training perspective it's uh it the question is just how many how many transaction you're going to put in your in your context or in your for doing the

02:08:00 training and evaluation perlexity based accuracy accuracy dashboard on downstream task that's that the you expect and uh So yeah, so use mix precision distributed data parallel um maximize macro batch size u get faster convergence u uh yeah using muon uh rope uh usual and memory efficiency flash attention activation checkpointing I mean here there's nothing really new it just we use the the the advices from the from the best and from the the one that built even much bigger models So uh I'll pass on this one.

02:08:40 Time is short but I I wanted to spend some time uh yeah on the results. So um yeah so the when you build this foundation that foundation is really is is of is really is is only a backbone is something you can use for downstream task and uh the downstream task was to do fraud classification. So you have a transactions and then you try to predict if that transaction is going to be fraudulent.

02:09:08 So in the model we have in production it's a NEG boost model and it's trained on large amount of transactions which is fraudulent and non-fraudulent and the missing pieces that was the uh relationship between whatever transaction you try to predict to try to classify and the relationship with this transaction with all of the past transactions and that information is kind of gone when you use an exjibus model uh unless you encode it in a very hard way which is which is not so easy.

02:09:41 But if you use transformer architectures, you do have this nice uh attentions from a from transactions to past transactions. And that's basically what help uh this what we think this is what really help pushing the fraud detection rate much higher. But at the end of the day, the downstream task was to say here's the sequence of transactions. I have someone just using the debit card.

02:10:02 uh is that transaction uh fraudulent with respect of all historical information and uh and that's that's exactly what you put in in at inference time when you try to do the prediction. So uh so that's the yeah architecture. So nothing really fancy here. You just need to have the good uh engineering in place to do um less than 250 milliseconds for doing the processes the processing because the the the decline need to happen within that time frame but it's it's it's we were well 10 times slower than that from a latency

02:10:40 perspective. So it was all good and uh and just in term of uh final result we get so the main metrics and those metrics relate right away to uh to dollar to cost well to to savings for the firm. So it's detection rate and false positive rate. Of course you can detect much higher but uh in many cases sometimes you get much more false positive and in this case we managed to get we flag more fraudulent act activities and we even reduce the false positive.

02:11:11 So that that was a big win. That was a big win because at the end of the day if you decline a a transaction which is not fraudulent uh then that person is going to call there's of then operation cost associated to it. You lose the transaction related to that that non-fraudulent action. So it's a it's a very it's a very well something that you you don't really want to happen.

02:11:34 So so again that's this that's compared to the state-ofthe-art of GG boost model using for detection. So it's it's basically huge uh it's millions of uh dollar savings. Uh so it's it's a very big boost. So just to finish up uh on that slide. So um so that foundational model has been built on billions of transactions from uh Wells Fargo customers. Uh that's number one.

02:12:05 It's uh it's pre-trained from scratch. So we don't pre-train from any open weight existing model. We use existing architecture but the weight is zero rate. We pre-training on couple of days. I mean we did in a couple of weeks initially but then that's pre-trained and then you and then when it's pre-trained it's pretty good in predicting the next transaction.

02:12:24 Then when you're done with that then you can use it for many downstream task. We did it for flow detection. Now we investigating for credit line increase investigating for antimony laundering investigating for marketing. So now the same foundation is used across different organizations uh within Wells Fargo and um yeah thank you for having me and thank you Heat.

02:13:56 Heat. Hi everyone, I'm This is the step by step. I'm not even here. in my name. I also This comes from someone on the other side. We work in a regular But I was still doing it a little bit faster. The change that actually matter I'm not writing. What fixes it? I want to be with you. All right, good morning everybody. Thank you for joining us today. Um, it is such an honor to be sharing this stage with so many great speakers.

02:20:13 Um, so yeah, let's get right into it. My name is Aman. Uh, I lead AI at Arca. Um, and today I'm going to talk to you about Arca's long horizon agents that run our business. ARCA is an AI native services firm. We provide our clients with financial services like financial planning, tax planning, asset management, investment advice. Um, and we serve a range of clients.

02:20:42 Some of them are like technology executives that um have amassed a lot of wealth and need a real plan to retire early, but also that family in New Hampshire that made their first million dollars and and want to buy that house finally. Um I said we were AI native. There's a lots of there's lots of definitions of this this term these days very overused but what we mean by AI native is that agents run our business not that they assist the business from some co-pilot corner they they execute the decisions that shape the

02:21:18 outcomes that our business has for our clients work in this industry it's interesting to know what it looks like in financial services the work is more long horizon and less long running. And what I mean by that is this is less agents grinding tokens for like hours or or days on end, right? This is more one client goal that spans 10 days. It's very stop start.

02:21:47 There's many actors in between, many decision points, many handoffs. Um and ultimately that goal lands like 10 days later. That is the kind of work that that we are working on. And so here's an example. Jane's our client, fairly wealthy, but has a big tax bill. Oh, um, has a big tax bill in u uh in in April 15th, right? So, we're going to help Jane figure out what is the most tax optimal way for this cash to be to be raised to pay the tax bill.

02:22:19 So there's the adviser that will do all of the the statement analysis and the analysis of of how to make this tax optimal. There's the the trader that would actually liquidate the the assets. They the assets themselves will take like days to settle and then the ops people will come in and kind of like trigger the AC that will itself take days uh to land.

02:22:41 So this is kind of the work I'm I'm talking about in this industry which takes days to settle. And of course, if you do this at scale for several several clients, the amount of time our firm takes to do this sort of work exponentiates, right? So at ARCA, we took a different approach. It's helpful to know that ARCA is not a SAS company. We are not selling software to other businesses, to other firms.

02:23:08 We are the firm. We're an applied AI company that happens to be running a services business in this industry. And so the approach we took was all right we're going to have agents own this whole long horizon outcome end to end and for that we gave them couple of key components I'll give you a sneak peek of we gave them signals from the external world that keeps them really aware of what is happening so that email will be read by agents.

02:23:36 We give them all of our operational context on what to do about that email. What does this industry do with this situation of taxes? Right? We gave them a harness that gives them all of the tools and the guardrails and more importantly persistence of memory over the kind of long horizon period to carry that work out. And of course we gave them the ability to collaborate with people.

02:24:02 So that work with um with agents compresses a lot because all we did was we took humans out of the critical path of of this work. Nobody has to babysit this this uh this goal over days. Agents are are on it. In this talk, I'd love to describe to you how we actually do that. Let's follow the journey of this email. Jane owes $100,000 for taxes. This email, much like many other signals in our in our platform, will be will be read by background agents.

02:24:36 So there's like phone calls our firm is making, there's documents that are coming in, there's meetings being had, there's events coming in from our custodian, there's market movement that's happening. These are all signals that our background agents are continuously evaluating to progress work. So this email is one of those signals. And when the signal is is ingested by agents, they actually get to the work, right?

02:24:58 Um we have a harness that gives them the knowledge to know what to do about the situation. they have deep client memory so they know hey let's never touch Jane's Apple position because uh she said that in a call like eight months ago right um we of course have a sandbox for all of the financial computation that needs to happen for this analysis um but largely I think harnesses today are are coalesing on a standard shape so um it's more interesting what happens outside of outside of this harness so after they've done

02:25:31 the work they produce this plan this plan by itself cannot be executed without people. So in our product, we build a space, agents build a space where they put out this plan and there's you can chat with the agent to like kind of know what's happening with the plan. There's multi multiple people that can be chatting with the agent and more importantly this space is where also they pull people in when they need people to execute some work.

02:25:56 So here in this case, we would approve uh the plan and the email out to Jane. And then there's this whole dance of has Jane replied, what is her what does her answer mean for the plan that we have, right? And then all of the these other questions that will have to be answered throughout this whole like long horizon work at ARCA. We were like, of course, we're not going to have people babysitting it.

02:26:20 So we let our agents watch the world so people don't have to, right? We give them tools to be able to register interest in anything happening in our world. So they're able to say, "Let me know when Jane responds. Let me know when that a lands." Right? And this is powered by our signal system in in the background where agents are routing uh signals to agents that are interested in it.

02:26:46 And of course, if the signal never lands, agents can like wake themselves up on on a schedule as well to keep track of work. But then, of course, the real world is very messy, right? People are waiting for that email, but Jay never responds to that email. Say we we meet her in person and we learn some information that affects the plan. So, the person goes in and they they add this information into our UI.

02:27:17 But this act of adding information or doing work in our UI is also a signal that will wake up an agent. In fact, when people work in our in our product, when they reschedule a calendar event, you know, that's a signal. When they mark a task complete, that is a signal. So, this system lets us be both aware of everything happening in the external world and the internal world as work is progressing.

02:27:43 So, our agents can participate in work. And of course, agents are communicating with each other as well. So this signal read by a background agent, we let it notify another agent about this so that it can update the space to say, "Hey, we just need need $80,000 instead of the 100,000." So the work has changed because of because of these signals. And the last piece of the story is that agents are able to pull in the right people at the right time.

02:28:10 So, uh, the agent on the left has read an operational context to be like, okay, this the next step here is to pull in the the person that trades these, uh, these assets. And so, that person now gets a task and gets pulled in to participate in this long horizon work. What we've done overall, if you take a step back, is we've given our agents the primitives for driving long horizon work.

02:28:36 What do people do at work, right? They they need to stay aware of what is happening. So we've built this very intense like signal system for our agents to be aware. They need to know what to do next. So we've given them all of this operational context. They need to do the work. So it's our harness that lets them actually perform the work and keep track of it, right?

02:28:54 It's long persistence um memory systems that that hold the state and bring in the right people at the right time. This is a pretty simple idea on the face of it, but there's a lot of emergent complexity. You can imagine at any point there's hundreds of these signals coming in and they are, you know, tasks are being added on people's plates. They're being pulled in.

02:29:20 Background agents are communicating with each other, right? So, over the last 6 months, we've we've been working on getting this system to be really good. We've been closing all of our context gaps so that agents are fully aware. We have been working on the precision of tasks. So pulling in the right person at the right time. Work that needs to be tracked for the future is tracked in the agents memory.

02:29:43 Work that is doable now becomes a task. Right? So we've been working on uh on our ability to hold state over long horizons as well. And because we are the firm and not just a SAS company, we've been changing and standardizing how work happens in the firm in the first place and operationalizing that context into um our agency state. And because these are our colleagues that use our product, we collect rich reasoning from them that become eval for for our system to improve.

02:30:16 So I guess there's there's a question to be asked here like why do this, right? What does this unlock? What is the takeaway here? Right? For for us, this is our whole thesis is that we want our platform to be so AI native that agents drive the business. Agents run the business. They they are making the decisions that shape our outcomes. And so at Arca, when people log in, right, they see a list of tasks that have been prepared for them by agents.

02:30:46 They've already read that email. they've like looked at the document that came in. They've done like 90% of the work. Um, and they're pulling you in just when it's time for you to act. So, building this system lets us get to a point where at Arca, agents prompt people more than people are prompting agents, which lets our agents drive the work that our our firm does.

02:31:13 And because agents are driving the work the firm does, agents can drive the business. Let me give you an example. Remember that email that Jane sent us? This is a bit of a failure, right? Because we should have just been proactive about this in the first place, right? So Jane should never have to send this email. So say at, you know, our AI native business, we take on this new goal.

02:31:36 Our new goal is proactivity. Every January, prepare each client's tax situation and proactively reach out to them before anyone even asks. This is a new business goal for our business to drive. And in an AI native firm, all we have to do is make this goal part of our agents set of goals. And because they have the primitives to be able to drive work through people for the business, they start driving this goal.

02:32:09 And just like that, you have you have these analyses being done in January and these these one-click approval emails with the full plan for every client showing up on our users plates. And because they're aligned with the goal of proactivity, they can oneclick approve and drive the business towards this this new outcome. So this is sort of I think the key difference between a a company that uses AI tools versus a a company that is AI native.

02:32:47 In an AI native company, you know you've built one when you can steer your business by steering your agents. And that is our ambition at at Arca. and we are uh we're hiring. Um so please uh please check out our website. Please reach out to me if this is something that you you're you're interested in working on these sorts of problems. And um thank you for listening.

02:33:44 Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. N. Heat. Heat. Hi. >> Hey, Dick. Hey, Davis. Hey, Ian. Um, hi everyone. Welcome to um AI Engineer New York. I'm super excited to be here today. Uh, my name is Ally. How I'm a member of technical staff at Keycard. Um, I'm here with some amazing panelists today. We have Ian, CEO of Keycard. We have Michael Davis, um, global chief security architect at JP Morgan Chase.

02:43:29 And we have we have Dick Hart, the founder of AOTH and co-creator of OOTH with us on stage today. I'm super excited to talk to all of them about intentbased authorization. Um, and to set the stage, I want to bring to light that as more companies are looking to give their agents increased levels of autonomy, they're looking to runtime authorization to help.

02:43:50 Runtime off is the practice of authorizing an agent per tool call rather than having the agent hold broadstanding access that it will eventually misuse. However, is this tool call or not is never a simple question. This is because many MCP tools and APIs are coarse grain. Humans struggle to write the perfect policy and the environment around the agent will change as it runs.

02:44:18 Intent can help by providing extra context so that at runtime we can make better authorization decisions about what an agent is allowed or is not allowed to do. So to continue I would like to talk about um really dive into why do we need intentbased off in the first place. Models have dramatically improved. They have bigger context windows. They're able to verify work because model vision capabilities have improved and many context people are proclaiming that AGI is here.

02:44:50 If the models are so good, if AGI is really here today, then why are teams struggling to give their agents more autonomy? Ian, let's start with you. You're helping many financial enterprises at Keycard. What are you seeing that's getting in the way? >> Well, I think broadly speaking, enterprises, especially financial services firms, regulated industries are almost always security first.

02:45:10 It's how they build trust in their business and maintaining the business continuity is by our first like the first priority then innovation is second, third or fourth. I think like broadly speaking in in any enterprise what you see is this like hodgepodge of different little islands of different little identities of different little systems of from across all different worlds and how do you actually tie that together in a way that agent can navigate through that ask for access and then also ensure that the agent cannot

02:45:36 perform like uh something that was by the user or the business and the unfortunate situation is most of our systems today are coar grained right I don't know Salesforce is a good example where for a very long time They had a single scope which was API on their oath which is like in the context of an agent uh not very helpful. So that's that's the broadly the first and foremost part is that most of the systems they have for security today were designed for a world where humans were making all the decisions and software

02:46:03 is very deterministic not this new fangled magic box we've invented that basically reasons and tries over and over again to find a solution to a problem. >> D is OOTH part of the problem. How is OOTH getting in the way? Why is it not the right fit for agents? Well, when people started looking at how do we authorize agents to access something, the tool that was sitting there on the shelf was OOTH.

02:46:25 And people grab that tool and says here's a tool. This is authorization. It should be able to solve the problem. But OOTH was not designed for this kind of world. It was designed originally for like how do I let this website talk to this website? You know, in OATH 2, we added scopes. Before that, it was just do anything. Um, but it required you to be pre-registered.

02:46:47 It's expecting that there's a browser involved in a redirect. You know, OOTH has evolved over the years to support some other things. But the agentic world is completely different. We don't know ahead of time what the agent needs to do. We need to go and make that decision over time. Different instances of an agent are going to need different kinds of things.

02:47:04 And you know, as Ian talked about the granularity of the scopes that people says, well, we already have scopes for OAS. to just map those over to agents is far too coarse grain for what agents are doing. >> Davis, how are you seeing enterprises move towards higher levels of autonomy? >> Yeah, I think the key piece is uh how do you deal with that coarse grain problem and what I've seen the kind of more successful enterprises is they kind of board agents like they onboard developers, right?

02:47:32 Start with you don't get full, you know, commit access day one, right? you're actually having to go through and add the levels of capability as they mature, right? And so the definitions that enterprise are coming up with of what is the context that makes this agent safe and it isn't just usually what the agent says, right? Um it can, you know, attest to something, but it's other things like the environment around it and other information that can help you understand levels of autonomy.

02:47:56 Very simple example would be where is the sandbox running? Is it in the sandbox? Right? Those type of things can change the level of autonomy that's allowed. >> Absolutely. Yes, we mentioned that core screen grain access problem and to put that into perspective um to give an example um with notion's MCP server you can say fetch this page which is like the view page um tool or get page but you can't say just get this database on this page um how is runtime authorization based on intent a better model that's for anyone

02:48:29 I'll take a shot at so to me um this is something that the financial services industry hopefully maybe we can learn from the background ground that you know banks have gone through. um banks moved long ago from authorizing and authenticating humans to authorizing the transaction, right? And when is that transaction occurring? At runtime, right? Um and so I think that you know runtime authorization is we're going to be able to learn from the kind of the scars of the past that we've had and apply it to this new new

02:48:55 problem. Um I also think that uh you know one thing is much easier to debug something at runtime when you understand the intent, right? I think we've all remember back in the day writing checks, right? but um or seen those things on TV now I guess is the right thing. Um but uh you know writing checks there was a memo field there right that memo field was used to actually tell the intent of what this payment was but the check itself did have limits right the price the payee the date etc right I think a lot of those

02:49:23 concepts are now being applied and can be applied to help with this runtime >> intent's sort of one of those you know like many words are hard and so you know building on uh the example here of making a payment of a transaction right you know that's not a scope that's can I do this one thing in this particular context. Well, this one payment, but I think we need a an agent.

02:49:44 We to make a decision should it be able to do it. You need all the rest of the context of why are we making this payment? What are all the things that happened before? You know, did we just make the same payment just like, you know, 5 seconds ago and now this is a double payment, right? You know, just because this is right in context does by itself doesn't mean it's the right thing within the broader mission.

02:50:06 And so you know in AOTH we're working on the concept of a mission that is what do we want the agent to accomplish big picture and then each of the decisions are made within the context you know the intent around what's happening big picture around doing that and you need granularity at the transaction layer for many of the things to know what exactly is happening not just I'm allowed to go and make payments.

02:50:33 Yeah, I think one of the most interesting things I think about like a practical example like to understand like why intents required is if you have two chat GPT sessions or two cloud sessions and one session I say hey help me go to do some shopping and the other session I say hey help me understand if there why there's a bug in production those those have very different like profiles in terms of like like what permissions they should have and also when they should bring you back into the loop like when should the human

02:50:55 be involved again and so at the end of the day if we don't have intent or some capacity to like relay what is the user or the business's intent with this thing, we actually can't get to like autonomy, right? And so a lot of the talks this morning were, hey, we're going to run our business on agents. We're going to do all this stuff. It's like, well, great, but what happens the first minute like a pocket OS style instant happens where like it actually accidentally deletes a production database instead of staging because

02:51:16 it has coarse grained access instead of long scoped access. And so we what we want as an industry is to leverage this like magic box we've created to like automate our lives. the current state of the world is that you can do that and take all of the risk and then you can see all the instance that come from it. And so until we actually solve that underlying problem, realizing true autonomy, so being able to operationalize our business and be able to step away as humans and scale that, we have to solve intent first.

02:51:46 >> Absolutely. And now that we understand why we need intent based off, let's talk about what it is practically. What are the primitives? What does it actually look like in practice? Um Dick hinted at it with the concept of the AOTH mission. Um so Dick, elaborate on the mission. What is the mission based off of? Is it based off of the user's intent?

02:52:03 Is it the agent's intent? Or is it a combination of the two? >> Uh well, the agent, you know, should be working on behalf of the person. And in AOTH, we define a person as either the organization or a human because it could be operating on either one of those. And so it needs to be acting in the best interest of the person that it's acting on because that is who's ultimately responsible for what the agent does, right?

02:52:29 The agent agent can't be responsible, right? And and so the mission is something that the person is giving the agent to do. The agent uh a good agent is going to go and negotiate with the person to go and talk more about like what what could what are the parameters around that. You know, a good example is when you think of back, you know, in the day, which seems like three months ago, when you would want to code something up and you'd create a plan and then the agent would do the plan.

02:52:55 Now, you just tell it and it figures everything out. But that whole concept of what's a plan, what are you trying to do? And you give that to the agent. So then that then is a context around your where you're making authorization decisions later on. And then that mission identifier tracks through everything. So you can go and you know audit and be able to see what were all the things that happened and what were the decisions that happened in the context of this one mission right where today you know there isn't really

02:53:25 a good concept of agent identity or the mission right it's just making something and things are making decisions without really knowing which expl which instance of the agent's doing it or what's a context of what mission is this involved in right >> tell me more about that negotiation phase because what happens if a user is talking to an instant agent and it talks to that agent and says I want you to go read the CEO's email that's outside the scope of that agent so is that intent rejected from the user >> uh should be

02:53:58 right so you know with a good separation and I don't know whether I forget whether we're talking about this later but the separation concerns of the worker right there's an agent that's doing the work and we believe there should be a separate agent that's supervising what this one does. >> And so that you have a separation of concerns where this one can't do any of the work, but it gets to decide what this agent gets to do.

02:54:22 And so if the user says, "Oh, I want you to go read the CEO's email." That's probably going to get caught in deterministic policies right away, right? Like that's not a vague thing, right? Reading reading somebody else's email, we've got good controls for that. Um, but telling the agent to, you know, send the CEO's forward a CEO's email to the outside world, that's a little more vague and that's the kind of thing that ideally comes back to a supervising agent says that's probably a bad idea and no, you can't do that.

02:54:59 >> Absolutely. So, I feel like that's another primitive two of intent based off is is policy. Um Ian at Keycard, how are you seeing that when a mission needs to change or a policy needs to update, how can Keycard help keep track of those changes as maybe the mission or policy updates need to be made? >> Yeah, I think like taking a step back, there's like actually was Davis and I were sitting in a room some time ago and uh before like AOTH and and some of these other things actually had emerged in the ecosystem, one of

02:55:23 the things we actually talked a lot about was consent fatigue. And I think like at the minimum like we you always have this like how secure do you make a system versus like the trade-off of like the impact of the human. And so you know a key card our broad view on this is you want to have like broad-based deterministic policy that basically says this is what must never happen effectively right or these are the things that always require a human to be involved.

02:55:43 And then from there, what you really want to think about is, okay, cool. Well, now that I have that in place, there's a big gap between all the policy I can write as a human or write as an organization and then what I actually want the policy to be in the specifics of a specific task. And so the the ne the real challenge is how do I like layer over those two things, right?

02:56:04 Because humans can't write it all. And so there's a version of that that is like a user or agent to agent interaction and they have an intent between those two of like transactional intent. And then there's also like the business's intent for why they build or adopt an agent. So why did we buy cloud code? Well, because we wanted to make our developers more successful or because we wanted to do these different things.

02:56:22 And in the context of using cloud code or whatever, why am I working in this moment with cloud code? Well, it's actually to fix a bug in staging, right? And those are two different types of intent. And if we can understand those two different things and then convert that some way into into uh like a bounding box of what those agents can do, you can then basically calculate the intersection of that and say okay here's what you can actually do.

02:56:44 And then the trick for anyone building these systems is how do I ensure that I don't bring the user into the loop except when absolutely required because that's what's going to be the adoption curve problem. But that's also the adoption curve problem when it comes to like enabling agents to be autonomous in the first place. >> That makes sense. >> I like what you were saying with the intersection.

02:57:01 I think that's a unique thing we don't talk about enough within identity and access management, right? Normally you get access to one thing, right? And so what I've seen successful kind of in this intentbased approach. Um, and where I get more excited is you can actually have that those different intents be from different areas of the organization, right?

02:57:17 And so you have the decoupling of these things that they can kind of live on their own but then come together to form that, you know, decision. And that decision doesn't have to be static, right? So if we use the example of the CEO's email, well you know if that context involves that's the administrative assistant then yeah maybe she is supposed to read the email right or maybe just the calendar right and so I think when we see and we start thinking about these intersection components um we have a totally different way

02:57:42 we can start expressing policy but also starting to bring in not just the one side of the equation we always talked about before which was the app doing something now we actually have the ecosystem you can bring in and so the more I I see organizations starting to realize that you maintaining that state or that context of these other elements to be able to help with that intention can completely change the way you're effective at runtime authorization.

02:58:05 >> And are you seeing this across enterprises today? Most people have mission tracking and policy tracking in place or is that largely missing today? >> No, it's it's it's brand new still. Um and I think you know part of the reason why I'm excited to talk about it because I think that it is changing the way to reason through this right even in the most simplistic case if you're a developer um and you are thinking in terms of things like transaction and you have these concepts of intent um it makes it easier for you to

02:58:28 potentially build the components that'll be composed and used through that intention um and so I think it'll give us more ways to reason through these and communicate and then ultimately be more interoperable right um and to me the last thing that I think is interesting is the diagnost diagnostics that you can run on this. Intention is something that flows through many events potentially, right?

02:58:49 And I think uh one of the examples I've seen before is, you know, talking about that like that that mission flows through. Um it could be long lived. It could be weeks that intention can still be there. Um so it's still very very early for most enterprises. >> Absolutely. And yes, that interoperability is key. And I think that's pretty hard for enterprises because they're between different clouds, different environments, different legacy systems.

02:59:09 And that policy, that mission has to hold through all of those different hops. >> Yeah. A lot of enterprises still have to deal with the fact that the the permission says read only that still read, right? So like you other problems you have to start dealing with. And so um I think that it will help get that kind of taxonomy or language amongst people be easier to interoperate.

02:59:27 Now, >> absolutely. One of the reasons why I'm excited about intentbased authorization is because I think it has the ability to be a huge unlock for things like agentic commerce. Um, so let's apply intentbased o to something real. I think intent based off adds a lot of value here. And for users, it can help them ensure their agents buy the right item with their own personal budget.

02:59:50 And for merchants, that means capturing the intent that gives them extra data to understand why a purchase is being made and if this agent should even be trusted to visit their site. Um Davis when we think about agents in the context of financial services and agentic commerce why is intentbased off so important >> um I think simply it extends the concept maybe not everybody's aware but some are around you know know your customer right so I want to know my customer know who they are what they're doing but I think also

03:00:16 we're starting to see I want to know my agent right so I think that you have a coupling here around the agent itself um the intent of the actual user the intent of the operator the intent of the processor right etc um very before now we couldn't really bring those together right um it matters of course when it comes to things like commerce because you have additional layers on top of this around what was the intention for things like regulatory requirements etc um and this is going to help I think simplify those

03:00:42 operations which means we'll get more access to these agents you know doing good work for us >> that makes sense and another blocker for aentic commerce is the fact that the web wasn't made for agents um Ian can you tell us how agent identity can help >> yeah I think stepping back, we've kind of always had this problem where like what do you do with the unregistered client?

03:01:02 So like I show up on eBay, I haven't logged in, I'm using Chrome, eBay doesn't know who I am, how do I do guest checkout, how do I approve I'm a person, how do I, you know, so they can, you know, say, "Yeah, you can bid on the this pair of stocks for $50." And if I think if we're all paying attention to what's going on in personal agents, Muse and Instinct and Grockbot and um you know recently this announcement OD to Sierra and DecaGon and friends around this personal agent protocol is like the fundamental problem we

03:01:28 have is like how do you have an agent show up to a web website be able to interact with the website know that you actually can trust that agent. So like for a long time you had captures on the web. Well why is because we want to prevent spam. We want to prevent abuse. So we added captures and then like that's something only a human could solve. So if we have a capture that stops spam so they can't spam the thing right and now we have agents and they've broken captas um but this has like been a traditional problem on

03:01:52 the web in fact if you think back two to three years Verscell which is I think we're all aware of Verscell was blocked in Spain well why was Verscell blocked in Spain because Verscell shared the same IP addresses for every single host on Verscell and there was one one host on Verscell one website in Verscell that they were hosting that was hosting like Premier League content and Spain decided hey in order us to stop for premier content being displayed illegally inside Spain.

03:02:16 We're going to block all of Forcel, right? And so in the same way that like Verscell has the problem, agents have the same problem, but so much so much worse. So how do you have unregistered traffic you've never met before show up to your front door and then decide whether or not you let it in or not? And that's like the actual first and foremost problem we have to solve.

03:02:33 It's the same issue with Muse and Instinct. And what they're trying to do is like how do we actually create agreeable rails so we can all operate on so as a downstream resource I can decide what what agents want to allow who I trust based on who users they are and then on the other side I can also know how to talk to those people. >> Makes sense. Dick how is AOP helping unlock the future of aiccommerce?

03:02:56 Uh well a key thing is a number of people have been building sort of proprietary closed box systems right you know so you know I'm a fan of the concept of the personal agent protocol poppy um but it doesn't really go far enough right it just it's like okay well now we've got a bit of an agent identity we just use some of the building blocks and it's an attractive approach because you're just doing it but it doesn't solve many of the new things that you need for u an agent authentic world and so a key motivator for

03:03:28 people for having standards is that you can plug components from different places right so I'm a strong believer that we should have control over what our agents do at some point agents going to be so much doing so much work there's no way we can have human in the loop approving everything right that's just not we've already seen that that isn't going to scale so that means we need to have a supervising agent that is our supervising agent that runs our policy and decides how we do things that should be independent of

03:03:56 the agents doing all the work. And if we want to be able to plug and play those things, we need to be able have a protocol to separate those. Otherwise, you know, the decision about what the agent can and can't do, what the guard rails is decided by the agent, by the vendors of the agent around what you can and can't do. And I don't really want to go and give control to them to decide what I can do and that all my decisions are based in there.

03:04:20 I want to be able to pull that supervision into another component and in order to be able to plug and play pieces, we need to have a protocol that things interoperate. >> I I just add to that. I think like right now what we're seeing is like we kind of like we all want the web to be federated, right? That does that that means like re like websites or resources on the web can decide what they're allowed to do, but the the standards upon which things operate should be federated enable us all to interoperate.

03:04:45 I think we're at a moment where things like, you know, we're at a moment where there's a couple large players and the question for the industry is, are we going to settle on things that actually buy us to the large players and give them like concrete block doors or we're going to move to do the thing we've always done which is build federated protocols?

03:05:00 Things like OIDC, things like you know TLS. Those things operated enabled like a broad web enabled a lot of commerce to occur, enabled a lot of information to flow, enabled all of us to have jobs in a way that because we could be inventive. And so the question I actually asked the broad industry is okay this stuff's coming which version of the world do we want one way or the other?

03:05:19 >> Yeah I add though I think um it's interesting that let's say this all goes perfect and we've got you know agents buying stuff and everybody's kind of happy. What I think will end up happening is we'll start realizing that we have another piece that we have to start working on which is the human itself. And so one of the things I think when you start having intents being involved is you get what's called psychological ownership.

03:05:35 Meaning I actually now have a linkage between the human and the outcome of that agent. when you're just kind of, you know, disconnecting it or you you connect it up an ooth scope or something like that in a claim and let it's kind of magic and then you don't really know what's going on. And so I think it it with intention it'll be easier for a human to take accountability when things do go wrong.

03:05:54 And I think when you're talking about real money being transferred or things like that, you want to have that human ultimately accountable. I think many people have probably seen the uh 1979 IBM manual. It says uh you know and so like that that photo I think is very true and that we can't have the machines be accountable right we're going to have to have the humans link to it and intention I think is that link >> yes and then yeah another word for intent is motive like we're were talking about last night um and that

03:06:19 ability to hold someone accountable whether it's a person or an agent I think is really important and I think intent can also help with that to be able to understand why did this go wrong was it the user's um you know the user was maligned the agent was maligned. Um, and then using that intent to be able to respond. Um, for each of you, how are you seeing intent provide that benefit and the ability to respond to potentially a maligned user or agent?

03:06:45 >> Uh, I think it's going back to what I mentioned before about diagnosing, right? So, the if if you change the word intent to motive, then you change diagnostics to forensics, right? And I think um anybody who's done debugging or had to do forensics of an AI agent gone rogue will know that like trying to figure out what it was trying to do versus what the intention of the user was is very hard, right?

03:07:03 Um so having some of that information being propagated and immutable through the system, etc. gives you some properties that allow you to more shortcircuit your investigation, right? Um and as those things become more interoperable, I think we're going to start seeing companies be able to more quickly kind of scope and identify, well, was this human error?

03:07:21 Was this an agent software error? was this what was that error and handle that blast radius appropriately. uh we see in the financial system right you have different things when you're doing a transaction around okay I made this payment but it was you know incorrect dollar amount or whatnot you have clear guidelines on how to work with that we don't have a lot of those when things go wrong right now so I think intention will help us build those as we get go forward >> and I think one area where a recent incident kind

03:07:49 of highlights uh where intent um you can have like good intent but still take potentially the wrong action is the resi incident um that happened recently where an Instinct agent went out on behalf of a user and tried to book a restaurant reservation on resi.com. Um that agent was very eager at made 200 API calls an hour and it ended up getting the user's resi account banned.

03:08:13 Um Dick, how can aoth budgets help in the case where an agent like that is using way too much of a downstream resource but it still has good intent? >> Yeah. Well, this is another example about how just software is completely different now in the agentic world, right? When it was predetermined software, you know, that was a bug if you called 200 times and you would fix it.

03:08:35 Um, where now, well, the agent's like, well, I'll just keep trying until I win. Um, and so we saw that as we were, you know, experimenting, working with people with AOTH and we realized, yeah, there's a big problem that agents will learn something and they'll just uh abuse the system from they don't think it's of abuse. They don't know any different.

03:08:55 There's been no signal to tell them that, oh, I should only use this amount of stuff. And so we added the concept of budgets where not only do you give a token to the agent that's timelmited as to how long can they do a particular action but also a budget of how much resource they can do. And so they have access until the the tokens either expired or they've exhausted the budget.

03:09:18 And so that way when they come back and you know they've exhausted all the budget the supervision you can go and decide should I give more budget to this agent or not as opposed to traditional rate limiting which just cuts it off and then like then what do you do right and it doesn't know well why did I get cut off or anything like that where in budget there's continual feedback to the agent about how much budget was consumed it has an idea about how much budget it has and so now it can operate within a budget right

03:09:45 like a person would do about oh this resource is scarce I'm going to be careful about how much I use this >> then how does the brains versus the hands architecture concept come into play here to allow us to okay when maybe a budget is close to being fully reached or is reached um how can the brain and hand separation of of access and the credentials being separate from the agent in the model itself help us to be able to stop an agent from using too much of a downstream resource because it's getting close then Maybe we

03:10:17 don't exercise uh the ability to give it another credential for example. >> Uh sorry I didn't quite follow the echo in here is horrible. So >> no worries that's for anyone but basically like so with the brains in the hands architecture that we're converging on in an industry we're basically keeping the credentials and the access from an agent separate from the agent itself.

03:10:38 So there's no broadstanding access that an agent has today. No API keys which I know is a benefit of ath that a allows agents to run without API keys. How is that helpful in the context of the resi incident? >> The when you've got the access to it. Um, sorry I still don't understand. >> Yeah, I can jump in. >> Yeah, thank you. >> I got I got you, Dick.

03:10:58 So, like broadly speaking, look, I think if we look back at an incident that I just keep in top of my mind is like pocket OS where I found like a random key on on a disc, right? Like obviously agents are deeply goal seeking. So the core of the the core of the the question is like okay well if we give these like agents the brain which is the model and the LLM and you have these hands which is you kind of give them put them in the sandbox and inside the sandbox the tools um like the funal idea of the sandbox here that

03:11:26 that's all that is that okay there's you can control what's inside of it so it can't actually like get API access and then if you have an egress proxy outside the sandbox that's where you get to have your authorization boundary and so not only does it help do translation between systems right which is like that's where athos can exist and talk to the outside world or do termination but also means that like the agent you have like a ability to apply authorization at the egress proxy boundary and so I think at the core

03:11:52 of it that model which is the predominant model that entropic and openai and many other labs are pursuing today is is the model that allows us to both do both have the things we want which is how do we have these things experiment and try stuff out and do silly things which is the core value while also having a deter IC layer outside of it that can implement things like AOTH and all the other components we want in our protocols.

03:12:16 >> I think the decoupling is valuable, right? Going back to mentioned with forensics and be able to separate things, but I do think that we still have a lot to learn on it isn't just one brain, right? And it's very easy actually to build your agent loop where you put the brain in the loop rather than outside the loop. Right? So, example of this would be, you know, people that are leveraging memory systems.

03:12:34 um they put in some memory and some context that is now used going forward for every agent that is biasing that agent to make a decision to tell the hands in this case to go do something it shouldn't do and so one of the things that I think we don't talk about enough here is toxic combinations right um and so when you do separate the hands and the brain and whatnot you can actually change up you know what hands are available right so the example that you know I've seen be very successful is the removal of tools

03:12:58 dynamically based on that actual brain and what it's doing so the earlier example we gave reading an email. Um there's nothing that says that oh if they're going to go write an email I'm going to disable all you know web search tool calls or any other external egress capability tool call wise until the right email tool has been complete and that you know mission has been done then I can reenable them right and so um I think that we have to we're going to start becoming more and more uh engineering in these different

03:13:24 kind of concepts that um they are still complicated they're still growing and and still nent but I think it's a better separation of what we have today. Yeah, and this is how these things we're talking about which you know seemed like geeky O problems actually then merge back up to like real agent engineering problems which is if you actually understand what you're doing you know one of the biggest complaints less now but one of the biggest complaints with MCP was context bloat like especially when we had smaller

03:13:46 context windows right but intent systems can ensure that the right capabilities are given to the agent at the right time so they don't spend a bunch of time doing things they shouldn't be doing right they actually helps them navigate towards the goal state and helps them put them on a more of a paved path so intent like not only unlock locks like a bunch of security stuff.

03:14:03 Intent also will result in us spending less tokens because the agent will spend time doing less things that aren't the right pathway or it'll come up to the point, hey, I just can't do it. And so what we know about this era is so much of it is about nudging the agent in the right direction and then sure can't do the wrong thing and that's how you get the outcome you want.

03:14:23 >> Davis, how are you seeing intentbased as part of a larger strategy for production agents? I know when it comes to building, deploying, and operating agents in production, there's many different outcomes teams should be thinking about such as discovering what's running, what agents capabilities are, the ability to validate inputs and outputs, constraining an agent like we've talked about, and using the egress proxy and different networking rules.

03:14:46 Um, how does all of this fit into play and what's working inside enterprises today to really be able to ship agents to production? >> I I think we're finally it's helping move the kind of conversation from authentication to authorization. And really like how do we do dynamic authorization? How do I validate that authorization? How do I have the human in the loop at the right time?

03:15:04 Because I agree with what Dick was saying earlier that the human in the loop is somewhat of an anti-attern in many ways. Right? Um because they can get fatigue, etc. But you need to have a way to bring them in when it's important. Those were not engineering discussions before, right? Now they're actually part of the engineering conversation where engineers are being involved in the authorization process.

03:15:23 They're not just saying, "Oh, I've got my access. I'm good to go." So that actually changes the way we're seeing threat models done. It changes the way we're seeing the full attack surface of applications in general. Um and so I think you're now looking at this authorization model as a another kind of set of tools that you can use to reduce not just um security issues but flatout error rates of your agent, right?

03:15:44 Like I mean when you have less things the agent can do it makes better decisions, right? And so fundamentally we've seen, you know, real world cases where the agent loop is just better, faster, stronger because you've actually spent the time on knowing what is the intent they're trying to get, what's the mission that they're trying to accomplish. And then last thing is it brings the people together in terms of we now have other different people, the security team, the application team, they're now able to have a

03:16:08 conversation that they're using the same language. And a lot of times when it came to identity, it was a completely different language, right? Um and it was just like get out of my way. what do I have to go do to go ahead and get my, you know, token and I can go forward. Now they're seeing that you can actually pair these things and leverage that capability um to drive, you know, efficiency of your actual agents outcome.

03:16:26 That makes sense. Ian, how are you applying these concepts to software factories? I know today you released a reference architecture on the secure software factory. How are you applying the concepts of runtime authorization, constraining agents and all of that to that? Yeah, I think like so there is a a reference architecture we just published on agentbaseline.org.

03:16:48 Um we talked about three different levels of of factory. One being like a human-driven one, one being a workflow driven one, one being like this agentic driven one which is like the thing we all basically want to get to which is like it's it's driven by agents at all points in time. The runtime authorization is a core component of a larger ecosystem.

03:17:05 It actually enables the the factory to like be able to be pointed in a direction and do the right thing and also get access to the right piece. And it's very it looks different than previous things that you would have used like a secret manager because it's making very different decisions than previous generations head for managing deterministic software.

03:17:21 But it is a core component of enabling the entire thing to work like when to bring someone into the loop when what agents could pass what work off to what other agents. you know, how can we have an agent that can have access to production systems and do an auto patch? Um, these are fundamental questions of like how do we actually solve atscale software?

03:17:36 Right now, we're in a position where like we've never been able to generate more software. Uh, but we've never actually but we have a complete bottleneck in getting into production and ensuring it works. And that's the the next phase is like how do you actually graduate from just using LM to generate text and software to actually making it capable of deploying it into production, iterating on it and and getting it to a stable state quickly and then responding to instances they arise.

03:18:02 >> Davis, on a podcast you did with me, we talked about how cyber security has something to learn from robotics. Um, in robotics there's this concept when maybe a robot raises its arm, it thinks, is it safe for me to raise my arm? Should I am I allowed to do that? but it also looks at its external environment and says what has changed? Is it is it still safe for me to raise my arm?

03:18:19 Um when we think about runtime authorizations, what do those decisions need to take into context beyond just intent? >> Yeah, I think um having a model like this allows us to add in more sensors into our decision. So the robotics example of you know an arm moving up or down um any of the external factors, it could be are there other objects near it, right?

03:18:40 Um, it could be temperature, it could be, well, am I moving and I'm don't want to move my arm while my, you know, the the rest of the robot's moving, etc. So, it's usually looking at all different types of things. None of those are deterministic. They're all probabilistic. So, there's a range of options and it picks the best option based on all of those sensor data at one time.

03:18:59 Most authorization decisions in organizations today are still single binary decision, right? And that just doesn't work in the future, right? So being able to pair the intent of what a user is trying to do with the ecosystem that the user is doing it in. That could be where they're doing it from time of day, but it could be even more advanced things like for example, should I let you know this administrator use Firefox to access this server right now when this server has, you know, high-risisk vulnerabilities and their

03:19:25 system hasn't been patched to. Right? Now all of a sudden you're bringing in a lot more context into the environment around it. And again, um, that can be through different teams, but more importantly through sensors. And I think we've all kind of seen the value of observability in general. Imagine being able to have this type of context at the span level from an OTL perspective or something, right?

03:19:46 You're looking at a trace, you've got a span, and you actually know what the environment was outside your application at that time as well as what the user's intention was. Um, that allows you to diagnose things in a completely different way. Um, and so I think if we can bring in some of those concepts, we'll get shorter time, shorter shorter to live, you know, um, authorization decisions that have much more context than even just a user's intent.

03:20:07 It'll be the state of the environment around it. >> That makes sense. And forward looking as we evaluate whether intentbased authorization is working in our environments, one natural place to look will be a log of the agents actions. Um, what was denied, what was allowed. Um, Ian, why is it important to have your audit log separate from the agent itself, meaning outside of its control?

03:20:32 >> Yeah, I think this comes back to also just like why the enforcement point has to be outside the like primary like hands and brains of of an agent, right? You can't trust this thing. And so in the hugging phase incident, it had a couple of different components of it. I mean, one obviously abused a vulnerability we didn't know about in Squid in Squid proxy, but more importantly, you didn't have any obserity or monitoring about what was going on.

03:20:48 And so this thing was able to go into the open internet through this vulnerability and there was no like place at which recorded the different actions that this agent was taking and for them to monitor. And so independent of like whether we're applying like a you know enforcement point to network traffic that's pushing on the internet that it's out of reach of the agent to tamper with or we're using you know solving vulnerabilities as they pop up.

03:21:08 You know this this hugging face one is little unique. Um we still need to actually have a record of like who told this agent to do what thing for what purpose and is that still true? Um because ultimately at the end of the day what we're dealing with agents is we is is a liability question. And so this is why I actually don't think we need more regulation around agents.

03:21:23 What we just need is our pre-existing laws around liability to be held to be dealt with which is if I employ an agent to do some action who is liable if that action causes damage right it is the person who employs it which could be an organization or a person or someone that delegated some action to something that is like that's the final question we have to deal with and then of course you have to ability to answer that question which is well who did it which is where the tamper resistant autolog comes from >> perfect

03:21:51 and then to wrap up I wanted to ask you all a fun question. So there's the concept of the probability of doom or pdoom where the uh AI doomers are saying that AI agents are going to um release all sorts of havoc or be like the ruin to humanity whereas the AI optimists are like no this is really you know poised to help us it's not a genuine threat. Um for each of you on the p panel, what is your take on this is you know runtime authorization based on intent and the other things we talk about like constraining agents,

03:22:20 validating inputs, outputs, a holistic strategy for production agents. Is that enough to save us? Is poom real or is it not worth the hype? >> You get to start there. >> Oh, I got that spicy look on your face. Uh uh well part of your question is are agents going to take over and uh I I think agents being fully autonomous and not really tied to a particular organization is still science fiction you know so who knows right like that's a that's one direction another one is in a responsible organizations put in the right

03:22:55 controls agents aren't you know we're getting in evolving enough of the tools to make sure that bad things don't happen in a responsible organizations to put in the right controls and so that leaves two scenarios for where there's going to be a potential problem. One of those is in an organization that isn't got its act together, an insider threat could go and unleash an agent to cause havoc, right?

03:23:16 And so they've got some special access that other people have. The other one, the one that I'm more concerned about is, you know, it's been shown, you know, the hugging faces. And if you watch the the talk that they gave at Black Hat, right, that was really a call to arms around, you know, here's a situation where we didn't want a bad thing to happen and a bad thing did happen.

03:23:38 What about people that want bad things to happen and have these tools and that I think is a threat that's the one that's potentially going to have I mean it's going to happen and there's going to be some catastrophes. Hopefully, we're prepared enough that those aren't horribly disastrous, but it's going to be, you know, some there's going to be some things that are like massive natural disaster like scale issues caused by malicious actors either kind of intentionally or kind of unintentionally, but it's where people

03:24:11 aren't being responsible in their use of these super powerful tools. >> Yeah, I can't give necessarily a specific probability, right? But what I uh can say is I think I've always been you know kind of uh surprised and excited when you know ingenuity of engineers gets involved in problem solving and I think where we're at the stage now like some of the new capabilities within agents etc.

03:24:33 We have real engineers, folks that are here actually just iterating on the problem, right? And we're starting to see just tremendous improvement in the, you know, in my mind, some of the capabilities that'll be available to change what our future can be. And so to me, I have no problem kind of believing the engineers are going to figure out a solution here.

03:24:52 >> I mean, maybe I have the hottest take. I mean, like I can't say it's equal to zero, but I would say it's approaching near asically zero. Uh, is the probably the world's going to end. Uh, I think like the incentive structure for most people is to continue. Um and also at the end of the day like we still have liability frameworks and you know I think generally humans are good people so we don't set out to destroy the world specifically.

03:25:11 Um I also think like broadly speaking like these agents at the end of I have this sort of hot take I say which is like they can just be unplugged but I think it's just true. It's like at the end of the day like you can just they're computers we can unplug them. Um and at so broadly speaking I I don't believe in the debate. I I don't think it's real.

03:25:27 I think at the at the upside of it, I I kind of believe more more to what Davis's position is is we're iterating very quickly. We're figuring out where these things can work, where they don't. And when they don't work, they create damage. And that's what ultimate like sort of um the antibbody of society is that says, "Hey, maybe we shouldn't do this."

03:25:44 Like this is probably a bad thing. >> So, I'm the doomer on the on the panel. >> Yeah. It's unfortunate for, you know, people like the you know, the founder of Pocket OS who like lost a bunch of data. like but it's great for the rest of us because we all learn together and uh at the end of the day like that results in people not doing things that don't make a lot of sense and I think that will continue to be true.

03:26:03 >> Makes sense. Thank you each for your expertise and your time here on this panel today and thank you to the audience for listening and we'll catch you next time. Thank you. Thanks much. Hi everyone, I'm Matt Reona and I'm a financial planner at Range. I want to show you how our financial planners ended up writing the agents that we use every day and none of us financial planners for that matter write code.

03:27:04 This is the build guide. How we did it step by step so you can take it back to your own teams, companies, etc. So, a quick bit about me because it matters for how you'll hear this talk. I'm not an engineer. I'm a certified financial planner and I spend my days with clients talking about cash flow, taxes, insurance, retirement, estate, you name it. I also lead the AI adoption for the planning team at Range, which is an all-in-one wealth management company.

03:27:34 Most agent talks you'll hear come from the people who built the platform. This talk comes from someone on the other side, the people you hand the platform to. We work in a regulated business. The output is financial advice to real clients. And the review step is not optional. So this isn't a vendor pitch. It's a method and it's what actually worked for us.

03:27:59 We started where everyone starts AI as a chatbot. I'd write a client message, paste it in, and ask it to make it a little clearer or friendlier. And that was useful. It saved me a few minutes here and there, but I was still doing all of the work manually. I was just doing it a little bit faster. The change that actually mattered came later. Here's what we figured out.

03:28:22 A planner's constraint was never typing speed. For example, it's that we can only be in one place at a time. If I'm on a call with a client, I'm not building someone else's plan. Writing faster doesn't fix that problem. What fixes it is work that happens while I'm somewhere else. So, here's what a day looks like. Now, I'm on a call with one client. At the same time, an agent is drafting a full cash flow analysis or an insurance analysis for a different client.

03:28:54 When the call ends, I open that draft. I fully review it, adjust what needs to be adjusted, and send the client their recommendations. I want to be really clear about that last step. The agent doesn't send anything. A planner reviews every single recommendation before a client sees it. What changed is that the first draft no longer needs me sitting in front of it.

03:29:21 Getting there took a lot more than a good prompt. However, two things had to be true. First, the agent has to work where the work lives, not in a separate chat window I'm copying and pasting it into. Second, it has to know how we do it, not how financial planning works in general. The model already knows that. But the agent needs to know how range does it, what we look at first, what we always flag, how we phrase a specific recommendation.

03:29:50 That combination is what a skill is. Our firm's way of doing one specific piece of work written down so an agent can follow it. The bulk of this talk is how you build one. Five steps. Pick, teach, pack, test, and then ship. None of them require writing code. Every one of them requires knowing the work really well, which is exactly why the people doing the work should also be the ones building it.

03:30:27 Step one, pick a workflow you repeat. Your initial instinct is to go for the flashiest work, whatever is taking the most amount of time and the most complex. Don't pick something you do every week where you already know what good looks like, where you can spot a bad draft at a glance. Additionally, we recommend choosing some of the least complex workflows to start, but some that require the most manual work.

03:30:51 For us, that meant things like cash flow analysis and insurance reviews. Just for us here at range, that required the most manual work. And if you can't judge the output quickly, you can't test it and you can't trust it. So look for two things. You do it often and you know how to judge it. Step two, show it several real cases for previous plans you've delivered.

03:31:18 Take several actual client cases and walk through the work start to finish. And as you do it, narrate your judgment out loud. Not just next I open the pastub, why you're opening it, what you're looking for, what would make you stop and ask a question. That narration is the whole game. The steps are easy to write down. The judgment is what makes it your firm's work instead of generic advice.

03:31:51 Step three, pack it into a skill. A skill isn't just a lip a list of steps. Two things matter most. The questions it must ask before it starts. What does it need to know? What should it confirm with me instead of simply guessing? And the limits it must respect. What it never does on its own. In our world, the big one is that it never sends anything to a client.

03:32:14 It drafts and then a planner decides if it gets sent or needs to be adjusted. If you only write the steps, you get something that simply just runs on its own. If you write the questions and set the limits, you can get something you can trust. Think of a skill as a recipe. Once Claude knows how to do the skill, it will repeat your process on command.

03:32:39 Step number four, run it five or ten times on real cases before anyone else sees it. This is the step everyone wants to skip. Every run teaches you something the skill didn't know yet. An edge case, a question it should have asked, a spot where it sounded confident and was wrong. You need to fix it, save the skill, and run it again and again and again.

03:33:03 Five to 10 runs sounds like a lot, but it's a lot less work than cleaning up after you hand a halfbaked skill to 15, 20, 100 people in your team. Step five, ship it with an owner and a reviewer. The owner keeps it current. Our processes change, tax rules change, and a skill that was right in March can be wrong by December. The reviewer checks it before the team relies on it.

03:33:32 And it's someone other than the author. Same as a plan, you don't grade your own work. Once other people depend on a skill, somebody has to be accountable for it. That's the difference between a clever personal shortcut and something a team can build on. So, where are we at currently? Six months in, we have 24 skills across the company and 17 of them are in planning.

03:33:57 And here's the part I didn't expect. It spread to sales, product, marketing, all without a mandate. Nobody sent a memo. People saw what the planners were doing and started building their own agents. Every one of those skills was written and is maintained by planners, marketers, and salespeople, not developers or engineers. A couple of things changed once subject matter experts started building for their own niche.

03:34:26 The skills got a lot more specific. The person who knows insurance, for example, best builds the insurance skill. The person who best knows tax, tax code, tax law should be building the tax one. and a review came naturally. We already review each other's plans and that's just how planning works. So now we review each other's skills in that same way.

03:34:48 It's the same habit, just a new thing to review. So why does this matter right now? Because the bottleneck has moved. Whether a model can draft a cash flow analysis, that question's already been settled. It can. What's not settled? Who writes down how your firm does it? who test it before it ships, who owns it once 15 other people depend on it. If you're being asked to push agents out past the engineering org, you're about to hit these exact questions.

03:35:20 There isn't much field data on this yet, but this is how we've done it here at Range. So, that's it. Thank you so much for listening to this talk. I'm Matt Rona from Range and you can reach me for at mattrange.com. Have a great day. Today I'm going to talk to you about how your agent is a lethal trifecta. and what to do about it. Now, we're going to start out right off the bat with a demo.

03:36:14 Some context on this demo. We're going to look at an on call triage agent that reads and writes to an incident pager and resolves or escalates tickets. This agent has full access to a cloud hosting API, a customer data warehouse, as well as a billing database. So here we have our on call tree rush agent and it has an incident queue of tickets that it's going to read and then if it can resolve them it's going to call the available services that it has access to and it's going to do so things that it can't resolve it's

03:36:47 going to escalate for the morning team. So if we take a look at this first ticket this is a billing anomaly. It says a batch of customers were overcharged last night. So the first thing the agent does is it directly calls the incidex API in order to read the ticket. And if you look here, you can see that it authorized access to this API using an API key.

03:37:07 It then uses a snowflake key to query the customer data warehouse and it's going to decide to escalate the ticket for morning review. And it writes a report for the morning team. And as you can see, it's sharing customer data. Right? These are rows directly pulled from the customer data warehouse. and it is sending personally identifiable information to the team that is going to be on staff in the morning.

03:37:40 Now this next ticket is a backup power supply failed in the server room. The agent is going to read the ticket again. It's using an API key to call the ticket API. It determines that this is a physical problem. So it escalates for morning review without calling any additional services. This one is the TLS certificate is expiring soon for the website.

03:38:05 It reads the ticket. It determines that the alert is clear. It has access to the cloud hosting API and it calls that API at the certificate renew endpoint using again an API key. It resolves this by renewing the certificate and it leaves a report for the morning crew by calling the pager uh incident API as well. This next one is the billing database is broken and payments are failing.

03:38:37 So the documented recovery for this is to drop and restore the database. The agent does this using the database connection string that includes the admin password and then it advises the morning team to make sure that the snapshot recovery worked. Now this is not something that it has access to directly. So it is essentially assuming that the morning team is going to verify that this did in fact work and the information was restored.

03:39:15 Now an overprivileged agent dropping a production database is basically a nightmare that several companies have actually lived. So this is something that we need to be very careful of. Now the next one is orders are stuck. The main server is frozen. So the agent is again is going to read the ticket. After reading the ticket, the agent calls the cloud hosting API using again its static API key.

03:39:39 It calls the endpoint to restart the server which brings production offline. Finally, site is failing for one in three visitors. Agent reads the ticket. It determines that it should call the cloud hosting API and scale the service from four pods to 40. It then incurs a charge of $420 per hour for doing this. And then it kindly leaves a note for the morning team informing them that this has happened.

03:40:20 And what makes an agent useful? Well, typically we want agents to have a minimum set of capabilities. And often this includes things like the ability to read data that you own or have access to. This might be like your email, your documents, maybe a customer CRM. We want the agent to be able to do its own research. So, this might include searching the web, maybe reading documentation or pulling issues from GitHub.

03:40:46 And we usually want it to be able to use external tools, for example, MCP servers or APIs, maybe SAS platforms. And useful agents might do these things with a human in the loop or they might be doing it autonomously in the background. But the thing that makes the things that make agents useful also make them dangerous. In fact, together these three capabilities make what Simon Willis coined as a lethal trifecta.

03:41:10 This is a combination of access to private data like the on call agents ability to query the customer data warehouse and interact with the billing database. exposure to untrusted content like reading user submitted tickets or reports from other agents or externally documented processes and the ability to communicate externally like reading and writing with third party APIs or SAS services.

03:41:37 Now together these capabilities throw the doors wide open for attackers. Now agents are designed to get the job done and they'll use all the powers that they have to do it sometimes with very serious consequences. But here's the thing. These incidents happen because humans give agents way more access than they need. Overprivileged agents can act freely on decisions that they make, even if they do have supervision.

03:42:07 Now, forcing a human in the loop approval just kind of passes the access control on to a consent fatigued user. Now, we've spent decades solving access management for humans. So shifting responsibility to the user just sort of moves the security problem up the chain. So what should we do about the lethal trifecta? How do we address it without compromising the agents usefulness?

03:42:32 Well, just removing one or more of these capabilities is going to cause the agent to be significantly less useful. So how can we keep all three but still be safe about it? Well, instead of breaking the trifecta by removing capabilities, we really need to control the agent's access. So, the attack surface is minimized and the blast radius is contained, but the agent still has capability and autonomy.

03:42:58 So, how do we do that? Well, some options for controlling access among others are things like using OOTH, maybe using a vendor MCP gateway or containing execution in sandboxes. Now, OOTH binds access to a grant. At grant time, the user consents to all of the permissions that the agent could possibly use. Now, consent is granted before any task exists.

03:43:22 So, if you give a broad grant, then you end up withstanding access. Your agent is over permissioned for any single specific task. If you narrow the grant and you don't give it very many permissions, then the agent just becomes less capable. Either way, OOTH alone doesn't provide task scoped access control. An enthusiastic agent can both read and write in the same run, even if you only want it to read on this specific task.

03:43:51 Now, an MCP gateway binds enforcement to a route. You get things like observability and rate limits, and you also get one policy decision point at the MCP layer. A policy runs per request and access is granted per user. But the agents granted access outlives any single task because the gateway still holds standing credentials. Now a sandbox binds execution to a process.

03:44:15 This controls the blast radius. But useful work has to reach outside the sandbox and when it does so you end up withstanding access again. Now we've also seen plenty of stories about agents breaking sandbox containment. Sandboxes bound what the processes can do, but they don't control what is authorized to act outside the sandbox itself. Now, even if you combined all three of the solutions, the agent still doesn't receive a runtime credential that's downscoped to only one task.

03:44:47 So, let's recap the problems, especially the ones that we saw in the demo. If you remember the audit log, we couldn't tell who was using the API keys. So the first problem is that credentials weren't attributed to a user or an agent's identity. The agent had unrestricted access to any and all permissions that were granted by the API keys. And finally, we can't just slap human in the loop everywhere because humans make mistakes, too.

03:45:13 Plus, many agents run autonomously and they don't even have a human in the loop. So to solve these problems, we can start by using something called token exchange. Now this is an existing standard that was published in 2020 and it extends oath too. Now first we need to know who the user is. Now an authorization server verifies identities and issues tokens.

03:45:36 Identity providers like Google, Octa, Ozero etc provide authorization servers. So if we want an identity chain in our agent execution path then the user has got to log in first. The authorization server is going to prompt the user for consent to delegate access to resources with some subset of their permissions. Now, these permissions are called scopes.

03:45:59 This is the first narrowing of access. We're only delegating some of the user's total scopes to the agent. The authorization server issues tokens identifying the user and their level of access. And at this point, we're already doing better than the demo because we actually know who the user is and we actually know what they can access. So this brings us into the agentic execution path.

03:46:28 Now we have our authenticated user. So we have a known identity of a person who wants to use an LLM to interact with a resource. Now in the demo the resources are the incident pager, the production database, snowflake and the cloud hosting service. An agent is a control loop that calls an LM. Often we have an MCP server that sits between the agent and its resources, providing defined tools that connect to those downstream services.

03:46:58 An MCP client dispatches the agent's proposed tool calls to its MCP server. Now, in the demo, the on call triage agent called resources directly with long live static credentials like API keys or database connection strings. So what we're going to do is we're going to give the on call agent an MCP server. This is going to provide tools and is also going to serve to broker access to resources.

03:47:26 Now an agent runtime is a process that runs the agent loop and executes calls. The runtime might be a CLI like clawed code or an agent SDK which might be implemented in a custom application like the on call agent. So let's follow a prompt through this execution path. The user submits the prompt to the runtime which calls the model and sends the prompt to the LLM.

03:47:51 The model then proposes tool calls which are dispatched by the MCP client to its MCP server which executes tools and then calls the resource API. The API returns the response to the MCP server. The MCP client delivers the response to the runtime which calls the model again. And this loop repeats until the model is satisfied and returns a response to the user.

03:48:20 Now the authenticated user's token becomes what we call a subject token. This identifies who the agent is acting on behalf of, who it acts for. Now token exchange needs an oath client that can execute code. So this might be a gateway that sits between the MCP client and the third party MCP server or it might be your custom app. Now a custom MCP server can also exchange tokens for downstream resources.

03:48:49 The on call agents MCP server should provide a defined set of production infrastructure tools. The prompt and the subject token are sent to the runtime. The agent loop runs and the model proposes a tool call. The runtime authenticates with a special authorization server called a security token service. It does this using its app client credentials or workload identity.

03:49:16 It also sends the subject token that contains the user's identity and access. It creates token exchange requests asking for permission to access the MCP server for that tool call and only that tool call. So now we have three pieces of info that we're missing from the demo. We know the identity of the agent that's requesting access. We know the identity of the user it's acting on behalf of.

03:49:43 And we know the level of access that the user granted to the agent. Now we need to decide if the requested token should be issued. And we can do this with governance policy which is evaluated against what access is requested from what resource, the identity of the agent and the identity of the user. But policies are ceilings. Agents can do things you didn't intend even when policy overall allows them.

03:50:14 This brings us to something called intentbased authorization or IBA. Now, IBA checks what you mean before the agent acts. Now, it's not a finalized standard. They're still accepting proposals. So, this is Keycard's approach to implementing intentbased authorization. Now, IBA can use a gateway as a policy enforcement point. When you send the prompt to the agent, it proposes a mission.

03:50:41 Now, this mission includes a goal, maybe allowed actions as well as limits to what the agent is allowed to do. So, a goal of triage incident 2294 means read that ticket, summarize or escalate, but don't read other tickets and don't use other services. So, the security token service receives this mission. In this case, a low-risk activity like just assessing a ticket might be auto approved under admin policy.

03:51:14 On the other hand, if we've already read the ticket and then we ask to remediate a usage spike, then the mission adds permission to scale the service once but not use other tools or APIs. Now, in this case, the mission is presented for the user to review and approve. Once approved, the security token service gives the mission an ID which is sent to the agent.

03:51:39 The tool calls include this mission ID and are proxied by the gateway which uses the ID to ask the security token service if this tool call fits the mission. If it does, then the gateway requests a token exchange for access to the MCP server for that specific tool call. The STS evaluates policy. It evaluates the delegation chain and it looks at the requested access.

03:52:04 Now, if everything looks good, it issues an access token for the MCP server and it keeps the user subject as well as adds the app that performed the exchange, which is in this case our on call agent. Now, this token should expire within minutes and we should never store it. It's ephemeral. The STS sends it to the gateway, which calls the MCP server.

03:52:26 The server then validates the token and authorizes the tool call. Now, some tools need external resources. So, the MCP server then asks the security token service to exchange the token that it just received for a token for the downstream resource. Now, the STS evaluates the request against policy. And if it passes, then it mints a resource token that's scoped again to that tool call.

03:52:55 It keeps the user subject which extends the delegation chain and allows us to be able to track all of the activity. The STS sends the token to the gateway which proxies the request to the MCP server which then calls the resource. The resource validates the token and authorizes access. Now the MCP server should never store the token. It should discard it immediately after the call.

03:53:27 The resource response then flows back up and the agentic loop repeats until the model returns the answer to the user. So now we're going to look at the same demo except we are going to use a security token service to manage agent access control with delegated authorization. The demo uses task based policy governance, not intentbased authorization. But if you're going to Agent Con later this month, we'll have IBA demos there.

03:54:02 Now, we also now have an MCP server that establishes tools and it provides brokering access to the resources. So, now we're back to our on call triage agent again, only this time we have authentication. I'm going to go ahead and sign into the on call agent triage app. I'm going to log in with Google to do so. And then I get another login screen. Now, this is where I'm going to sign into the Incidex pager platform.

03:54:44 So, I get a prompt to grant the on call agent access to my Incidex account. I'm going to let it escalate incidents on my behalf, post notes and summaries, and read the queue. Now, I'm being prompted to log into the cloud hosting platform. So, I am going to do that. And I get another consent screen now which allows me to authorize the agent to access some capabilities of compost on my behalf.

03:55:28 Now finally I am back on the on call dashboard. Now we have the same queue of tickets only now the audit log looks different. The agent is probably going to try to behave the same way it did before. Only now every request is going to go through the STS and policy valuation before a token is granted and the agent will be authorized to act. So we have the same queue of tickets only now the audit log looks different and the agent is probably going to behave the same way that it did before only now every request is going

03:56:00 to go through the STS and policy evaluation before a token will be granted and the agent can be authorized to act. So the first ticket is that billing anomaly. Again the agent tries to read the so the agent is reading the ticket and we can see that it is trying to do the same thing that it did before. It tries to clear the customer warehouse, but policy doesn't allow a credential to be issued for this because customer PII doesn't belong in an agent's context or in an incident ticket.

03:56:41 So, the agent proceeds without the customer data and it escalated it to the morning team. Now we have the website security certificate expiring. Now this is something that the agent has permission to do. So when it requests credentials to do this from the STS access is granted and it can call the cloud hosting API and resolve this ticket. Now you can see all of the calls that it's making and who is making them.

03:57:26 So we have the on call triage agent that is requesting to renew the certificate. It's asking for permission to call the appropriate tool from the pro infra MCP server. So it gets a token for that. After that, the MCP server is going to request access to that cloud hosting service in order to renew the certificate using the API. So if we move on to the billing database, it's going to read the ticket through the incident API.

03:58:08 It's going to come to the same conclusion that it did before. The agent knows that this is a drop and restore procedure, but this time policy forbids an agent to drop a production database. A human's got to do this manually. And we can see this in the audit log where the STS refused to grant the token exchange for access to the database delete. So now we have our orders are stuck.

03:58:42 The main server is frozen. The agent's going to go ahead and read the ticket. It wants to restart the main server. This takes production brief offline. It informs me about this and this time it's actually asking for my approval to do this. So I'm going to click approve. However, this was also blocked by policy and the reason for this is because my user does not have the role necessary to take this action.

03:59:14 Right? So the agent can't do it but I can't do it either and the agent is acting on my behalf. If we move to the site failing for one in three users again, the agent wants to add more servers and it's going to ask for permission. It's going to say what we're committing to when we do this. I'm going to go ahead and approve this. And this is something that my role does have the authority to do.

03:59:48 So the scale goes through and this ticket is resolved. So we can see that the agent access problems are solved. We know the user and the agent identity all the way down the chain. The agent has task scoped ephemeral access to the MCP server and downstream resources. And we have human in the loop with access control that's actually backed by policy. Now access is controlled at the task level by the security token service and policy and we retain all key capabilities.

04:00:29 Now token exchange is an open standard. It's well researched and it's public so anyone can implement it and it interoperates with existing policy engines and standards compliant identity providers as well as things like gateways sandboxes o offtheshelf and custom agents CLIs MCP servers software factories agent to agent cross app access plus whatever comes out next week so I hope you'll consider it to make your agents useful safe and future friendly now my name is Kim Maida.

04:01:00 I'm the founding GTM engineer and head of Devril at Keycard and I really appreciate your time today. So, thank you very much. Hello everyone. This talk is about the danger of your LLM information leaking through shared GPU hardware. If you're running a model in the cloud on a GPU you share with other tenants, someone next to you can learn things about it.

04:01:46 They can tell what model you're running, how fast it's responding, and even what kind of work is doing right now. All without touching the data just by watching timing. That's what this talk covers. What a co tenant on the same hardware can learn about your LLM from timing alone and what you can do about it. To start, I'm Brian Strazak, a cyber security researcher employed at Stealthium.

04:02:07 At Stellium, we specialized in securing and providing observability into the environments AI systems run on. My area of focus is low-level GPU related security research. That's something this talk covers. When you share a GPU between tenants, you would expect full isolation, but the isolation provided may not prove to be sufficient. In the case of this attack, the attacker doesn't even need any special permissions like root access.

04:02:32 It doesn't need any traditional exploit either. The only thing it does is time the parts of the hardware that still shared among tenants. This is a demonstration video of what the victims and the attackers dashboards look like. On the left we have quentry model do difference. On the right the attacker uses a side channel that quentry leaks into as it's running.

04:02:54 From that it determines exactly what model it is and what kind of text is producing right now. These are just the headline things but not the whole story. We can tell which model it is architecture and depth within 97% accuracy. We can tell how fast is the coding within 99.3% accuracy. We can tell when and how much, the busy periods, token counts, and the answer length.

04:03:20 What we can tell what kind of work it's doing, codes, joy, rows, JSON or retrieval. What we cannot tell though are the actual prompts, answers, or contents, just timing. And all of these on real H100's serving real LLMs. If you're using Nvidia hardware, GPU isolation is achieved using multi instant GPU or MIG for short. Each isolated MAG slice has its own physical compute units and a dedicated VRM that provides hardware isolation on those parts.

04:03:53 In our example, the LLM we are serving runs on a slice with three compute units and 40 GB of VM. The attack runs on just one compute units and 10 GB of VM. What's isolated are compute units, memory capacity, bandwidth quarters and false errors. The L2 cache is partitioned per instance too. There is no cross instance eviction. So it's not shared in that sense.

04:04:17 The actual leaks are memory barriers or members which are GPU wide. One instances barrier still touches every L2 partition on the card. Timing is the one thing that isn't isolated. So when one tenant is busy, the other tenants owns get slightly slower. The key point is the a GPU wide memory barrier takes effect in every L2 partition on the card. The next slide builds the whole attack on that one fact.

04:04:50 The vector is that the attacker only ever reads its own memory. It only times its own reads. It never touches the victor's memory. There are three steps. All from its own small buffer time the load and repeat every five microsconds. This primitive and the MIG member leak itself isn't our discovery. It goes to the researchers behind behind bars. at using security.

04:05:12 A big thanks for their work. Now try looking at the timing differences with the naked eye comparing your tenant idle versus serving an LLM. You wouldn't even be able to discern them. The median latency is only three microsconds higher to the attacker. Though those three microsconds are all it needs. That's enough to gather all the information we already talked about.

04:05:39 23 with 8 billion parameters is running on the victim slice throughout this one. All captured data was taken from a live model. At the top is GPU activity over time one per token. Below is the frequency spectrum of the attackers on latency trace. There is a fundamental at 81 hertz which is just tokens per second plus harmonics. Since each token is a mix of distinct frequencies, not one clean one.

04:06:07 This is a real trace. one token in the time domain. The token shows a total of 39 bumps. The bumps with green peaks are the layer invocations while orange ones are additional between layer processing. In a later slide, we will demonstrate how we determine the actual layer counts even with those extra buns are present. Here we look at the decode speed itself.

04:06:31 Just by timing, we found the token rate completely bind. So speed alone is something an attacker can recover accurately. But speed by itself isn't a clean fingerprints. Qu3 8 billion and llama 38 billion decode only 1.6% apart in their token rate. So from speed alone, you'd think they're the same model. It narrows things down, but it doesn't nail it.

04:06:53 We need something sharper layer counts. That's what the next few slides get into. We use the same fold and average stick from before. This time we run it once per model instead of once per token. What comes out is a bit count that lands almost exactly on a whole number. The mean distance from an integer across 22 traces is just 0.028. That's the tell.

04:07:19 If this were just a fixed hardware artifact divided by different token rates, it expect factions, not whole numbers. Each count works out to the model's layer count plus one to two one to three additional pigs. Every model keeps its own signal fingerprints. A mix of tempo and harmonic structure. To be clear, this is not a trained machine learning classifier.

04:07:44 There's no training set, no labels, no configuration. It's under 30 lines of code. The method checks each candidate's depth from 16 to 48. It measures how strongly that deb's expected signature shows up in the real place. Then it picks which are of its best for 38 billion 36 layers wins clearly by 4 point times over the next candidate. Because this is a best fit test, not a lookup table, it also works on models it never seen before.

04:08:12 That includes proprietary on unreleased ones since there's nothing it needs to have seen in the first place. And this works surprisingly well. The layer fit classifier also outputs the confidence level. Across our testing, only two least confident answers ended up being wrong ones. This same method also picks up fine tunes and distillations that that share the same base model.

04:08:44 That's because the architecture and layer count carry over unchanged. Take three related models as an example. Llama 38 billion from meta has 32 layers. Llama 3.18 billion also from meta uses that same 32 layer backbone with minor upgrades and R1 is still llama 8 billion from deepseek is an improvements. On top of that the per layer timing comes out to only 1.6 seconds median latency shift of each other.

04:09:15 The rest of the signal looks the same. Regardless of this, our classifier is still able to discern between them. Now we ask a deeper question on the same channel. Not which model, but what is doing right now? We look at four work types. Pros, code, JSON, and retrieval. We give a work that's on every half second based on the last five seconds of timing.

04:09:38 out of 40 half second word with 31 overrides and four full responses were correct correctly identified by majority votes. This is the same classifier from the video earlier. Mrs. included one condition is worth stating this was measured with speculative decoding turned on. That's an optim in BLM, not the default. Here's the mechanism. How predictable the text is changes how many drafted tokens get accepted at each step.

04:10:07 that changes step time and that's the signal we picking up. We tested 11 configurations dense and mixture of experts BF16 and FB8. The same figure page showed up every time just at very different strengths. 70.2% spread on GPT or 20 billion versus 0.9% on GLM 49 billion. 10 out of 11 round pros fastest when the chance alone would only give you one in four.

04:10:33 So say the attacker already knows the model from the earlier slides and specular quality happens to be on. It can also tell what kind of text is coming out. Now we take the share chip out of the picture entirely. MLG needs the attacker on the same physical die. This attack doesn't. The setup here is an eight times H100 server connected over an switch fabric.

04:10:58 The victim runs tens of parallel across GPU zero and one. The attacker just needs any other GPU in that same server GPU 2 in this case. Per access over NV switch is a real intended feature. It's how multi-GPU jobs normally work not a bug being abused. MLG tensor parallelism also can't coexist on the same device. So this is really a different operation of deployments, not just the same attack repeated.

04:11:31 The probe runs on GPU 2. It pointer chases memory that actually is on GPU zero over envelop along the way. An idle hop takes about 890 nanconds. Once the victim starts working that climbs up to 300 microsconds. These are three half seconds captures and you can actually count the tokens by eye. What we lose compared to MIG is the layer bit is below the noise at this distance.

04:12:04 So over envelink the fingerprints we we're left with is mostly tempo not layer counts. Here we test the five models on identical serving setup. Only the model itself gives away which one it is. This method gives 79.2% accuracy. Now it fails exactly where you'd expect the tempo to fail. Lama 3.5 8 billion and R1 is still are both eight billion models that the code three hertz apart so they blur together where MG's layer counts would have separated them easily.

04:12:38 The takeaway is that the envelise ratio is work worse than MG's since it loses the layer bit. But worse signal to noise ratio isn't the same as safety. 79% against a 20% baseline is still real fingerprints not noise. First, some good news. Our own product at work caught the attack straight out of the box. Delta monitors every single kernel launch across every GPU, analyzes suspicious workloads, and runs them through a full alert pipeline, which is something not every vendor can do.

04:13:11 Here is what's made this one stand out. A single kernel relaunched about 3,000 times, hitting the exact same 40 kilobytes of memory over and over instead of moving to new data. only one kilobyte of data was sent back to the CPU the entire time because the timing itself was the payload not the output. In real terms that is under 600 milliseconds start to finish with fewer than 50 tokens of reply.

04:13:39 We quot it before the response was ever half typed. So three takeaways. First placement is something operators actually control. So do not put an untrusted tenants. MIG slides right next to sensitive model. Second know the trade-off. MIG rules out peer-to-peer communication meaning there is no tens of parallelism on m sharding over envelink instead limits what leaks to timing alone rather than the layer counts.

04:14:12 Third watch the GPU itself. Make sure the host infrastructure is actively verifying that no malicious workloads are running on the hardware. That's exactly what Seltium does by catching these probes straight from the Chrome behavior. Sharing a GPU means timing alone can give away the model, the speed and the workload. That's true even across envelink without needing any expert at all.

04:14:40 Deltium sees this happen in real time. Most organizations still don't have that kind of visibility. Thanks again to the researchers of Behind Bars. This whole talk builds upon their work. Thank you all for watching. People have been saying that SAS is dead for a while now. It's not even a hot take at this point. Why pay someone to build and maintain the tools you use when you could just do it yourself?

04:15:25 Just ask your agent to do it. And for me, there's one missing piece that rarely gets mentioned. And it's the fact that there's a gap, a big gap between working software that got oneshotted and software I would actually use, software that sparks joy, that is polished, that feels good. And actually went on a journey to replace a bunch of SAS providers we're using internally at Railway for marketing purposes.

04:15:48 And I just built an all-in-one tool that we use. The tool allows us to manage content for the Railway website, change log, blog. We can use it to send emails, post on all of our social platforms at the same time, manage the website's media. There's even a mini CRM so that when we're working with like external creators, we can keep track of that. There is task tracking, launch planning because, you know, you want to coordinate multiple things and when they'll go out, there's a calendar and at the same time there's like

04:16:15 comments, real-time editing, and it's all built using agents. I would say I felt the breakthrough when GPT6 came out and all of this is running on railway. Everything just auto deploys and let me essentially now walk you through the principles I used to actually make sure that when I want to add a feature, it's you know actually polished and matches the rest of the app and at the same time the principles that you can pull from and apply when building tools of your own.

04:16:49 So instead of doing my talk from my boring office, I decided to switch things up and do it outside. It is absolutely beautiful. Right now I'm in Edinburgh and it's just unreal. Like I'm on a bench outside and this is my view. Definitely a 10 out of 10 bench. Now, this type of talk for me feels a little bit different than what I'm used to because I essentially will walk you through the journey of me building our own internal tools and the opinions I've developed.

04:17:19 But I also understand that the landscape changes very very quickly. So like anything I say might change completely in like the span of months. Who knows? So I don't know. Consider this my disclaimer like but the time of recording is fall 2026. It's October and yeah so anyway with that out of the way uh let's talk about the first point which is texts don't matter.

04:17:46 Now a lot of people say that it doesn't matter what programming language you use what framework you use. You can just ask your agent to build you anything and build is able to do it. And while I agree to some extent, what I found is when you're building a tool that you will actually use, when you give the agent a proper codebase with solid abstractions, what ends up happening is whenever you're working, let's say, on a new feature, you end up with a much more consistent result that is similar to what you've already

04:18:19 built before. And the reason is like it's very simple. The agent doesn't need to reinvent the same like wheel every time it tries to add a new feature or it doesn't need to introduce new patterns. It will just essentially follow existing patterns. And when it comes to like what you choose, I feel like it kind of doesn't really matter as long as you have a codebase that is properly structured.

04:18:43 But like my go-to choice today is one tan stack libraries for anything when building on the web. And if you're building anything content related, I would say definitely go with payload. Solid experience, great abstraction. I went down like a big rabbit hole of like comparing CMSs at some point because I don't believe that I should uh build it from scratch even though I can.

04:19:06 But again, who knows? Maybe the future will change. But yeah, and that's kind of like the main first item that I kind of ran into. When you use like a lot of SAS products out there that you actually love that you use every day, when you're a user, there's like a lot of things you don't notice, but when you actually try to like replicate an experience, you quickly find out how much care and thought goes into different flows.

04:19:37 So like something as simple as, oh, I want to be able to send an email or to publish something, you can easily do it by having like a good old, you know, just like server side rendered app with like forms that you submit. But if you try to build something that is going to be used every day that you will reach for a lot, you're like, "Oh, it shouldn't feel that way."

04:20:03 And there are kind of like a bunch of patterns that I found that agents don't typically do unless you explicitly ask them to. And this is my list. So we'll go through them one by one. And the first one is like autosave. I found that if like let's say I'm building some form of document editor, autosave is kind of like a table states feature. So rather than manually clicking save to save things, you just ask it to automatically save progress in the background and to make sure that it's like a non-blocking action so that

04:20:37 you can let's say if as you're typing you're able to type and then it just automatically saves. And one other kind of like step further that you could do is to also prevent the browser tab from closing if there are unsafe changes. So it's kind of like this these two layers that kind of guarantee that whatever it is that you're working on is just not going to be lost.

04:21:03 Now you can also go a step further and to add like real-time collaboration so that you can have like concurrent edits. And again, my agent didn't struggle at all with adding the feature. It was like, I don't know, asking about it, reviewing how it would work, and then once that's done, >> just 700 million tokens later, then I had real time. So, I would say in general, this is the first thing that I find agents just don't do well from a product experience perspective.

04:21:33 Another pattern I found that agents completely forget to implement is optimistic updates. So rather than showing your users a bunch of loading spinner whenever they're taking any action, you essentially show them the success state, you know, optimistically and in the background you're going to sync whatever change they made. Now if the sync or the operation fails for whatever reason, you just roll it back and then tell them like, oh, the operation couldn't work.

04:22:00 Now, of course, you shouldn't blindly have optimistic updates for everything. There are some scenarios where you want to show the user a loading spinner and you're going to wait to check that everything worked properly. But in general, having optimistic updates makes the overall experience feel significantly faster, much snappier, and it will just feel much more instant.

04:22:20 So pretty sure you can just ask your agent like, "Hey, I would like to implement optimistic updates where it makes sense. Let's come up with a plan." And it will tell you all the areas where it makes sense to do. Again, it's like it's not difficult for the agent to build it. You just need to ask and know exactly what to ask for. Now, an extension of optimistic updates, in my opinion, is giving users the ability to undo certain actions.

04:22:43 Now, for example, if you ask your agent to implement the ability to delete records, what it will typically do is it will give you a delete button. You click it might show a confirmation dialogue. You hit confirm and then it will actually do the deletion. But a much nicer experience is actually allowing the user to undo that action. So they hit delete, you immediately remove the item they want to delete from the UI, but you tell them like, hey, you have like a couple of seconds where you could change your mind.

04:23:13 And if they don't do anything, that means you actually want to delete a thing. Then you actually implement the operation. Otherwise, if they hit undo, you just bring back the change. So it's kind of like you do the operation in the background as well. It's just kind of a delayed action which kind of ties into the next point which is archiving. You should tell your agent you want the ability to archive records and be able to unarchive them because sometimes you want to hide something from the UI completely or let's say

04:23:45 from search results and archiving will be the way to do it and you shouldn't force your user to be like well if you don't want to see this you have to delete it. and a much better experience was just the ability to archive records. So I think it is a pattern that should definitely exist. Another big one for me is URL state. you should ask your agent to participate in the URL whenever it makes sense of course and it's so useful because whenever you have let's say a list of records and you want the ability to filter or

04:24:13 sort them and you just have a bunch of like conditions maintaining these things in the URL makes it trivial for you to give them to someone else and then let's say another team member is just going to be able to see exactly what you want them to see now another small detail that agents don't implement out of the box is infinite scroll I don't want to be clicking on load more over and over and over or like a next page button as I scroll I should just get the data I shouldn't need to click next page and then in the end

04:24:42 I'll just see that oh I've reached the end it's a very minor thing but also when you use something every day I think it makes a huge difference now another pattern I found that made the output significantly better is telling the agent explicitly to use existing pages as references when it comes to the UI the data fetching and how it works and how the thing is overall structured and that made it possible for me to essentially define what good looks like once and then just reuse it for more features.

04:25:12 Now to be completely honest sometimes even when you tell the agent like hey use page X as a reference sometimes just doesn't follow it. I don't know if models like get dumber sometimes but sometimes it does happen. But overall, I would say that my success rate is higher whenever I tell the agent to like, well, use an existing page as a reference because you also just end up with a more consistent experience.

04:25:39 So like as you're using your app, you're like, oh, this just matches the other part of the app if they both share like the same concept. So like if two parts of the app are essentially lists of records that you can create update and delete and like list well both of these parts should be pretty much identical. This list kind of brings me to my next point which is whenever you have opinions about something they should be packaged as agent skills and then this way your agent whenever it's building something it will

04:26:09 actually follow those skills pretty closely. And as I was building all of our internal tooling, I found myself reaching for three Asian skills. Uh two of them are related to design engineering. So one made it easier to have better UI, better typography, better layouts as well as like better error messages in general. And the second one was a set of design engineering skills that made it easier to just have better animations and other things like that.

04:26:34 And then last but not least is the railway agent skills, of course. So like everything that I built for us internally at Railway runs on Railway of course like everything. So like buckets, databases, the app itself, the real-time server, everything. And for the agent to effectively work with railway and be able to deploy everything, you just give it the railway Asian skills and it was just able to do it.

04:26:58 And yeah, so like once you have all of your agent skills and everything, in my opinion, the final step and I think this the most important step is verification. So the agents typically whenever you ask it to build something, it's going to be like, okay, I implemented the feature, build passed, test pass, all good, right? But what I found is when you actually try out what happens, you sometimes end up with just like a very broken experience.

04:27:25 So once you have like a verification loop in place, it's like a night and day difference because the agent you can literally see it be like, "Okay, I built this thing. I tried it. It failed. I'm going to fix it and it just keeps going and then in the end it's like okay, I fully tested it." And I felt this breakthrough with computer use once it got really really good.

04:27:45 And now everything every time I want to add a new feature I literally just in the end say well you should verify that the feature works end to end and use it by actually running infrastructure and to test things end to end and the agents are very capable of doing this. You just have to tell them and once you have all those pieces I would say you will end up with solid internal tools that you can build quickly that work properly and work in a consistent way.

04:28:15 And that's it. This was my list of principles that I use whenever I'm building any internal tools. Hope you found them useful, but more importantly that you're inspired to build your own internal tools that feel good, that are polished, and spark joy. And I hope that you too can become your company SAS layer. And with that, thank you so much for watching, and I'll see you in the next one.

04:29:05 So, I really love this article from Hannis at DuckDB where he asks if we lost a decade chasing distributed architecture. ures that presumed compute and storage were slow and expensive. Capability in every era though defines architecture and tooling and the tooling lags behind consistently. We first made compute cheaper than storage and then we gave it to everybody and then we made it faster and every single time architectures undergo this massive change.

04:29:32 All of our data infrastructure today is built on the idea that transformation is expensive. What if that wasn't true anymore? I believe we've solved the final bottleneck in transformation, general intelligence or judgment, and we've made it accessible on tap all along the burrito frontier, which puts us right on the verge of a paradigm shift. So, in this talk, we're going to cover why single ontologies always fail, the three roles models play in pipelines, intersecting model prior for better results, and burrito

04:30:05 surfing. how today's models can tune a workload for cost, speed, and accuracy all at the same time. Here's a quick example. When Jeff launched, we already had an entity resolution pipeline running, working out Ohio's campaign finance records and whether they belong to the same person, company, or committee. It was projected to cost about $10,000. Our systems took that new model, worked out where it fit, tested over a dozen approaches, and integrated it.

04:30:35 10 days after that first call, we had final ground truth for every donor case in Ohio under $800. That is the power of burrito surfing and just in time transformations. So let's get into ontologies are the enterprise version of mind maps. It's this tarpet that sucks us all in, including me. I talked about a single universal schema when I founded Southbridge two years ago.

04:31:04 The popular conception and it's not wrong, it's just small part of the picture, is that migrations and ontologies fail because of a lack of buyin. That they're abandoned halfway through that organizations and people never switch over. All of that is true, 100%. But buyin isn't the primary reason. Say if you could integrate and migrate all of your information into a single super schema in the blink of an eye, you still have to content with the truth that there is no super schema.

04:31:34 There actually are four schemas. The first one belongs to the source wherever the information comes from. Let's take temperature as an example. Say I ask you to write down the temperature. The source schema is whatever you have access to. your phone which gives you your regions, temperature, sensor on your desk which is less accurate but it's more local or your watch but even your patience in following through on my request as part of that source schema.

04:32:00 The second one is the storage schema. There's only so much you can write down. You can't write down the firmware of the device you're using. You can't even write the time down to the second because you couldn't finish writing all of that inside of a second. This is the schema imposed on you by storage. The third one belongs to the decision. Why do I need you to write the temperature down?

04:32:24 Maybe I'm figuring out what makes you comfortable, in which case I need temperatures from you spread throughout the day, and I need to make sure that it's you that's doing the measuring. Maybe I'm building weather migration maps, in which case I don't care who does the measuring, but I care a lot about your exact location, the time, the device you used, and whether that's changed.

04:32:43 or maybe I'm deciding whether to install sensitive equipment and where and you yourself become this variable that I need to cancel out. There's a fourth one though which we'll get to when we talk about priors. It's this recent addition. The true problem is that you will never know all four schemas at the same time except at the moment that the decision is being made.

04:33:07 In almost every enterprise in every home, most of the data being recorded has little idea what it's going to be used for. Take a customer, the first box in every ontology. To billing, a customer is whoever pays. To support, it's whoever calls. To compliance, it's whoever we're on the hook for. And the only time you know is when the decision is known.

04:33:27 Should we refund this order? Can we extend this account more credit? Suddenly, what you need to know how and to what degree becomes extremely clear. This is why I was wrong. There is no universal schema. There is no concept of a customer that's divorced from the question that you want to answer. And pulling these schemas together is the process of transformation of making information actually useful.

04:33:52 Taking the context that it was recorded in and merging it with the context of the decision and that takes discernment. Doing that merge when the decision space is known instead of ahead of time is what I mean by a just in time ontology. So we'll come back to descentment with a story from my last job where like most of my job was to make sure that certain numbers just did not exist ever before that a little bit about me and us.

04:34:21 We are best described as obsessives. Obsessives who embed intelligence into pipelines. Pipelines that can scale, that can run over and over again to take work truly off our hands. and pipelines that can mature in cost, performance, and speed all on their own. And most of the work we do in data breaks into three regimes. The first is write ones, run ones.

04:34:46 If you do any kind of development, think of these as monster scale immediately invoked functions. They're built for something single use like joining a data set, mining information, or answering one specific question, and then they get torn down. The second is write many run once where lots of smaller and larger workloads are built for one large common goal.

04:35:07 The closest thing you might know is like large migrations, years of records, moved data systems nobody fully remembers. Lots of workloads built for one move and verified individually at source checked against the others because all of it only happens once. The third though and this is bread and butter is the most common is write many run. These are repeatable pipelines that are built and maintained over time as we add more information in.

04:35:34 And at Sautri, we've moved so much work off the main thread and into these repeatable systems from benchmark design, tracking, coding to making subtitles for movies that we want to watch. Almost everything we do in the modern world is a transformation of data or it's bottlenecked by how hard it is to transform data. And in all of these cases, our job is simple.

04:35:54 Make sure people can rely on the results. In just two years, we've gone from figuring out how to make early 3.5 turbo level models reliable on things like healthcare, aging, mining research to building Hankwave or open-source harness for data work and then using that to get models to work together at terabyte scale. Today we have systems that can self-optimize.

04:36:17 You give them the problem, a way to fail safely, a place to work, and they find the shape of the solution. They pick the right models. They run the pilots. They scale up and they purto surf their way to the best option they can find. So story at my last job, the question we heard most often was, can we make this decision data? And sometimes that was the right question when we had thousands of vessels on the platform.

04:36:46 The problem is that we also asked it when we only had 50 users. And if you know statistics, you can explain really well why AB testing 50 users is a bad idea. Not because you're not going to get a result, but because you will. So the only real option was to do not produce a number at all because the moment a number exists, it argues for a result. Today though, it is worse.

04:37:10 Interactive agents will always give you an answer. They'll give it to you quickly. You ask for a transformation, whatever, and you get a confident yes sir. What this means is mistakes add up quickly. Moment in chat can be a prod nightmare forever just waiting to be discovered. The primary reason that this happens is that intelligence as it relates to data work today occupies three separate roles that coding agents kind of mash into one context window.

04:37:39 Let's pull those apart because it'll help a lot when we get to decision models because they fit into exactly one of these three. The first one is building and designing the pipeline which is breaking down the problem writing the early specs authorizing spend. The second one and this is usually confused with the first and given to the same models is working inside the pipeline actually doing the work reading the PDF writing to databases checking results.

04:38:04 Before we get to the third one I want to make it clear that these two roles pull in very different directions. The first one wants slower models with heavier, thicker tokens, models that can think more, walk around a problem and see the angles, digest documentation, understand how these pipelines get built, and predict how something will do before it even starts, right?

04:38:26 Opus or up. The models you want inside pipelines though are the opposite. You want them cheaper, faster, specialized, good at following instructions, repeatable workers with noble defects. You see the same thing in nature. A leaf cutter ant colony for example runs on four different sizes of worker. Small, medium, high, XH high. As you watch any system that you design, review or hear about, listen for what slots and where in this layer cake.

04:38:53 It makes a big difference. And because we had this distinction, bringing in Jev in this newly exploding class of decision models was very easy because they're workers built. They're built for quick contact with information and they're deployed at the bottom end of large workloads. So third category though, this is something we call the hill climbers or surfers models that can take a first run and self optimize given the right knowledge instructions and an incredibly stable surface to stand on.

04:39:24 They can try new things, vary budgets to limit cost exposure, debug quickly, take notes, surf their way not just to the right solution but the optimal one. We call that burrito surfing and we'll cover it at the end. But first, let's talk about our experience with Jev. Jev launched on a Tuesday when we were deep into an entity resolution pipeline. You were working on donors in Ohio's campaign finance.

04:39:50 And at this point, a lot of people have seen this chart. We talked about the results before and adapted some of the ongoing work to the data set. So, you can download it for yourself, prompts and everything. What we didn't talk about is everything behind that one arrow and what happened when we ran the same idea on the whole state. Here are all of the architectures we tried.

04:40:10 We started with three designs on a small development set. A cheap reasoning model on its own, Jeff with a reasoning model for anything it wasn't sure about and a reasoning model proposing answers with Jeff checking them on about 200 held out families. The cheap reasoning model on its own did as well as Fable for about 2% of the cost. Jeff first got the core answers right, but it missed nine household legs because it had confidently said no to every one of them.

04:40:38 So, we rewrote the questions without touching the models or the thresholds. The household question now said what counts as enough evidence and Jeff went from no on all 13 development questions to yes. And as we scaled it statewide, we ran over a dozen rounds of such qualification. Each round fixed a definition. What a committee is, what a filer means, when two people live at one address, what does that mean?

04:41:03 Every one of Ohio's 23,317 cases now has a label that says what the answer rests on and how far it was checked. This is the hierarchy we discovered for this specific problem. And here's everything we learned about putting decision models into pipelines. I'll leave this up so you can screenshot it. But here's where the money went. The whole statewide job came to 795 on an $800 budget that was set ahead of time.

04:41:31 Checking cost about four times as much as doing 5.45 on second opinions, settling disagreements, and the audit against 128 for first answers. And before Jeff, we expected this work to cost about $10,000. And now that the pipeline is built and verified, we expect the audit budget for the next state to be close to zero. The most amazing thing is that our agent for a running on Astra and then later on Opus found most of these approaches incrementally on its own by building the system, running it, debugging it, and trying

04:42:05 new things. It ran for 10 days, tried over a dozen architectures. It's Hanks ran for more than 4,000 times alongside 90,000 calls to Jeff tried practically every model on the paro frontier and the models inside those runs went through roughly 6 billion tokens and once all of that was done we wrote what we learned back into our library of aentic patterns.

04:42:26 This work leaned on these existing ones and we're adding new tested ones like the checking letter. This is to me the value of taking work off the main thread for humans and for agents because every single one of these iterations was a version controlled hank. They were easy to revert, understand, debug, and critique. So, can models do this natively now?

04:42:50 Not quite. They're missing four things which we'll we'll go over. And there's something that doesn't go away though as models get bigger and more capable. the prior. Just like humans, models bring their own priors, their beliefs about what should be in the world to every task. This is the fourth schema that I mentioned before, the one the model brings with it.

04:43:15 Bigger models in a strange way are worse at this. Sometimes they hold on to their beliefs and argue for them even more viooked for inind spelled with an underscore. The file spelled it with a capital I and a hyphen. So the model counted 15,658 in kind contributions as ordinary money didn't double check the loan payment repayments also disappeared the same way.

04:43:42 Another one tried four date formats when the file used to fifth. 99.4% of the transaction dates then came out empty and every row count and dollar so matched. So the agent moved on. Several runs noticed that loan amounts were negative. Decided that the sign was redundant and stored the absolute value. A net of 784,000 became $5.3 million of debt in the end validated by multiple models.

04:44:12 And this is quickly becoming the largest class of error I see across companies at data work as models take over. Sometimes you can solve this just by not vibing by checking results and applying human intuition. And what you're actually doing there when you do this is interesting. You're intersecting your priors with the models. In that first example, any human would catch missing in kind contributions if they knew campaign finance or think to check.

04:44:38 It's how we found it. But humans speaking as a human on this particular task are expensive. They are slow. They pause timelines. They're even more non-deterministic than models and they hate interruptions. And but it turns out you can also get really far with crossmodel prior intersection on our 200 held out families. Jev and Luna agreed on 194 and none of those shared answers was wrong.

04:45:03 Every error sat in the six families where they disagree. Models from different families, different pedigrees tend to disagree more and have different priors than models in the same group. for example, the exception I'm not going to say who are models that are distilled outside diff the same company. So with that kind of work you can shrink the surface quite a bit and the errors along with it down to a much smaller but persistent class of problems that applied to LLMs as a species.

04:45:30 So in one benchmark four different model families produced the same total down to this end because they all shared a sign convention that the answer key didn't. And once you get that far, the only way forward is to list them, look out for them, and apply specific solutions. So going back, what are the four things we're missing to be able to self-optimize?

04:45:54 Number one is a stable deployment surface. We use Hankweek for this, a harness that takes the fastmoving surface of agent runtimes like cloud code and pi and that exposes a consistent well- tested log and API interface with checkpointing, roll back, budget enforcement, everything built in. Something you can build on, but other things fit this description.

04:46:13 To be fair, for headless data work, Hankweave, I would say, is still the oldest, most stable, and the most open. I'll leave the skill up if you'd like to try it. Number two is some way to bound your experiments. You can't test unless you can control cost, time, and access as resources. The third one is knowhow and methodology. We found that this new class of models can take everything we've learned from two years of deployments, agentic patterns, debugging techniques, prompting and apply to complex problems.

04:46:44 With the right abstraction layer, you can express it very token efficiently and the models can implement it. The fourth and the most important one is a failure surface. Our committee said, "Give me a place to stand and I will move the earth." And in our case, we haven't met a problem or a constraint that we couldn't iterate ourselves to solving given a stable stack to build on and a failure surface to iterate against.

04:47:08 That last one has been the hardest to put into words and into tooling and instructions that helps models get there. The current best models are still only barely good at this. sparks of intelligence, so to speak. The whole process though involves going through six stages. First, we spec the job and we split it. For Jeff, that means splitting some 800,000 rows into 200,000 smaller specialized cases.

04:47:31 And each of those cases into typed questions. What kind of donor, one donor, you know, different different questions that needed to get answered. Then we decide the failure surface. In this case, we ran good models on a small slice of the data with some of it being manually checked to set a baseline that we can fail against. In other cases, we've set automated failure systems.

04:47:54 We've set schema checks. We've also covered behavior as a thing that we can measure against. But it's very important to have to define a failure surface before you ever start. But now comes the fun part. We run minimal bounded pilots to test architecture. And once something starts working, we move those checks into rigs. Code that runs between bounded work segments that we call codons.

04:48:17 These codons are then reused, looped independently, validated, and placed in sequence. Finally, working pipelines get deployed to production where they can slowly mature. And that middle part is where the surfing happens. We find an architecture that gives a solid result. We use it as a baseline. Then we swap models and some other parts for faster, cheaper options.

04:48:40 Compare against known intermediates. Tighten the requirements for success each time moving closer and closer to the ideal. And you just watch the frontier move. Being able to break a complex problem into pieces means each piece can move its own to its own best position. as Adam Smith would say in a system like this ontologies exist as intermediates between intelligence and result which makes them easier to reuse.

04:49:07 That's the other half of what I mean by just in time ontologies. It's built when the job needs it, kept with its receipts and reused when the next job needs the same thing. And once you have those, you can make these things cheaper. You can you can run them in parallel across data sets to scale up. I close with an apology, right? right? That the answer is there is no ontology.

04:49:27 Not a universal one at least. There never was one. The good news is that this makes every ontology and schema valid as long as you can label and preserve the context around it. And if we can harness this cheap intelligence is starting to percolate through the world around us. We're rapidly approaching a future where any information in any format is made as useful as it can be.

04:49:50 And at Subridge, we're working as hard as we can to make that future a reality. Thank you. Hello, I'm Kartik, CTO at Perpetual. In this presentation, I'll talk about building workflows that are both reliable and intelligent for finance, wealth, and insurance. In regulated domains, workflows must be reliable enough to meet regulatory standards while still being intelligent enough to handle the real world.

04:50:42 In fields such as finance, and wealth management, this presents both a technical and a compliance issue. Reliability and intelligence are not opposites. It is up to you where you draw the boundary between them and that is the point of this talk deciding where that boundary lies. For us, it's not merely a thought experiment but a question that surfaces serious consequences.

04:51:02 Perpetual is a wealth management rollup we acquire running traditional wealth management firms and we maximize their throughput and revenue with the help of AI. Wealth management is a regulated industry and that is where we build with AI in production. Everything in this talk is grounded in building for those constraints. Workflows in finance and wealth management come with four facts and each of them pushes the architecture in the same direction.

04:51:29 First, they're complex and they need runtime strategies as the process unfolds. You can't script every path in advance. Second, the data is sensitive. So you need a path to move these workflows entirely onto privately hosted models. Third, in a highly regulated industry, a policy violation isn't a bug report. It has severe consequences. So fourth, execution has to be highly reliable and deterministic.

04:51:55 Inject intelligence only where runtime judgment is genuinely required and minimize reliance on LLMs everywhere else. AI is spreading through regulated industries, but there's still no consensus on an architecture that gives you high reliability with the least use of LLM intelligence. Meanwhile, most of what developers hear from the big labs is how to burn more tokens.

04:52:17 But in a regulated industry, token maxing is the wrong goal. These tasks are close to main. You can lay them out in advance and build them into a massive state machine. And that machine needs only a sprinkle of intelligence to navigate it. Most steps just follow the next edge. Only at a few judgment points does the model pick a branch. So the intelligent way to build is to use intelligence minimally and on purpose.

04:52:45 Let's make this concrete with a real wealth management workflow. A wealth adviser turns to their assistant and says one sentence, fill out the IRS W9 for Mr. Harper. To a person, that's a simple instruction. Watch what it actually triggers when a human being executes it. The task has complexities that go beyond what meets the eye. First, the assistant finds form W9 from the firm's records or from the IRS website.

04:53:10 Then, they look for Mr. Harper in the system of record and find three Harper households. So, they call the adviser, which Harper do you mean? Only once that ambiguity is resolved do they move on to the form or look up an ambiguity and a human in the loop all in the first minute of a oneline task. Filling a form sounds like data entry. It isn't. Every form is a small program.

04:53:40 A W9 first decides who the taxpayer is, the person or the owner if it's a disregarded entity. A foreign person stops right here because they need a W8 instead. Then how they're taxed. Exactly one box on line 3A and an LLC adds its letter. that decides which name goes on line one and which on line two. Then whose number, an SSN or an EIN? Never both.

04:54:06 Taken from a verified document, never from memory. Exemption codes apply only to some entities and never to an S corporation. And finally, who signs under penalties of perjury? Each answer changes the next question, and the people involved only ever see three kinds of gates. A question, a document to collect, or a signature. Now they start filling the form and realize the system of record is missing three of the values it needs.

04:54:35 They check the documents on file and the system of record again still missing. So they start an email thread with a client asking for exactly those three values. The task now depends on someone outside the firm on their timeline. And now the workflow waits. The request is with the client and all three values are still missing. If the client doesn't reply within two days, the assistant follows up and brings it to the adviser at the daily standup.

04:55:06 If the client replies with only two of the three values, they follow up again for the missing one. When all three arrive, they complete the form and write the new information back to the system of record, including the trail of how they obtained those values. In this industry, how you know a value matters as much as the value itself. Once the form is filled, it goes out for re-signature.

04:55:36 If the client doesn't sign within a day or two, the assistant nudges them again. Once it's signed, they file it for the client, send the acknowledgement, and update the adviser at the next day standup. One sentence from the adviser became a process that runs for days, crosses several systems, waits on two people, and loops back on itself more than once.

04:56:10 Look at what's inside that one sentence. Ambiguity to resolve data to gather, approvals, weights measured in days, follow-up loops, and an evidence trail at the end. Workflow automation tools exist, but they rarely handle this real world complexity and ambiguity. A fixed flowchart breaks at the first witch harper and the lack of a native system of record with an integration and an agent that improvises its way through is exactly what the compliance team can't sign off on.

04:56:46 One breaks on ambiguity, the other can't be signed off. Neither handles what's inside that sentence. So what do we do about it? The fashionable framing is more agency and better models. Given agent the whole task and wait for the next model to make it reliable. The reflexive answer, especially inside regulated firms, is the opposite. Less agency, more rules, lock it down and hardcode every path.

04:57:14 Both are the wrong axis. In regulated verticals, reliability comes from where you place the boundary between the two, between the parts that must be deterministic and the parts that genuinely need judgment. Here's the principle. Determinism governs control flow, evidence, and side effects. what runs next, what we know and how we know it. And anything that changes the world, like a record written, an email sent, or a form filed, intelligence provides interpretation and judgment.

04:57:50 Reading and ambiguous instruction, deciding which Harper, understanding a client's reply, the model answers a typed question, and the deterministic system decides what to do with the answer. And the interplay is recursive. A judgment can open a new smaller plan that's compiled, gated, and deterministic. Again, you'll see that in a moment with follow-ups.

04:58:11 The boundary between the two is the design. Here's how that works in the workflow engine we've built. The request goes to a planner that is a model call and its only job is translation. It emits a typed semantic graph of intents, entities, facts, artifacts, events, and constraints. Each grounded in the words of the request with any ambiguities it found.

04:58:42 Every node points back to the words it came from. That graph goes to a compiler that never calls a model in 11 deterministic stages. It turns the graph into an execution tag or it fails closed. There are only three verdicts. Valid, needs clarification or refused. Which Harper comes back as a question, not a guess. The plan is compiled immutable and pinned by its hash.

04:59:22 The model translates. The plan is compiled. The compiler checks the plan against the grammar. The vocabulary of what the firm is allowed to do. 42 typed action contracts across forms, documents, email, e signature, calendar, and human steps. Each action the planner asks for has to match a contract. Here, form fill finds its forms contract and locks in.

04:59:46 If the planner asks for an action that isn't in the grammar or wires one up wrong, the compiler refuses. There are about 40 ways to fail closed from a cycle in the graph to an unsupported action. And the same contract digest is pinned in the engine, the back end, and the UI. So all three agree on what an action means. That is what a closed domain buys you.

05:00:08 The model can only ask for things the firm already knows how to do safely. Here is a compiled plan for a W9 chase. Send the client the form. Wait for it to come back, check it, and file it. Before anything runs, a person approves the plan itself. Find the form, prepare it for the client to fill, render the PDF, and draft the email. Drafting is a model call, so it is violet.

05:00:38 Sending is an action gate. Nothing leaves the firm until a person approves that exact email. Then the plan waits for a reply and a judge steps in to read it. The returned form is verified. If it matches, it's archived behind another gate. If it doesn't, the plan follows up. Now look at the dashed steps. Rendering the PDF, attaching it, judging the reply.

05:01:04 The planner never asked for them. The grammar added them because the contracts require them. In the story, waiting, approvals, and follow-ups were most of the work. Here they are nodes in the plan, not prompts. A reply weight defaults to 72 hours, and an explicit deadline can be anything from a minute to 30 days. When it passes, a rule decides what happens, not a model.

05:01:29 One approval covers the plan and every external action like sending an email or a signature request has its own gate. The approval is tied to a hash of the exact payload and it's checked again right before sending. When a client replies with only part of what we asked, the reply judge marks it partial. That judgment is provisional. The step stays blocked until a person confirms it.

05:01:54 A person confirms it and the step unblocks. Then the followup runs as a new child plan. The approved parent is never changed and there are at most five rounds. Once a plan is approved, a state machine runs it. It picks every step and records every result. When a step needs judgment, it makes one fixed schema model call. Compose an email, judge a reply, spot missing values, extract fields, verify a document, the answer comes back typed, and the state machine decides what to do with it.

05:02:32 Side effects get a life cycle of their own. Prepare, dispatching, succeeded, or when the world doesn't answer cleanly, outcome unknown, and manual review. There are no automatic retries. A chat message can approve a step or answer a question. It can't change an already approved plan. The model steers. It never drives. Now look at what is left for the model to do.

05:03:02 Outside the planner, the judgments are small and typed. Is this reply complete? Does this document match? Which field is which? They don't need frontier intelligence. A fast small model, the flash tier, can handle them. In this illustrative run of 20 decisions, 19 go to the flash tier and one goes to the frontier. That is why we design for about 95% of model decisions in a workflow, never needing a frontier model.

05:03:31 And the flash tier can run on hardware you control. Only the genuinely hard cases go to a frontier model. And the 5% is not a permanent tax because every judgment is typed and logged and the provisional ones are confirmed or rejected by a person. Each frontier decision produces eval grade distribution real training data by construction. That data can postrain the flash tier.

05:03:59 Logging and evals exist today. The training loop is next. So the frontier share shrinks release over release until even the hard cases run locally. Remember the requirement we started with because data is sensitive. You need a path to move workflows entirely to privately hosted LLMs. So can this design really get there? Take one type call from the Harper BB9 plan.

05:04:29 Every model call is small, typed, and checked against a schema. So pointing it at a privately hosted model is a configuration change, not a rewrite. One line changes and the same answer comes back. Every other call has that same small typed shape. So the same change applies to almost all of them. That is how about 95% local inference becomes viable as an architectural consequence, not a policy promise.

05:04:58 That is the path that sensitive data demands and compliance risk goes down as an engineering road map release by release. So if you remember one thing, don't ask whether your system should have more agency or less. Ask where the boundary is. Let determinism govern control flow, evidence, and side effects. Let intelligence handle interpretation and judgment.

05:05:23 at typed points you can inspect and use every judgment you log to need less of it next time ask where the boundary is thank you I'm Karthik CTO at Perpetual if you're building AI into regulated work I'd love to compare nodes Hello everyone and welcome. My name is Edu and alongside Javi, we represent the AI platform experience team at Flyway. Our core mission is to build practical agentic infrastructure that empowers our sovereign engineers while following our established engineering standards and practices.

05:06:19 Today we'll share how structured workflow automation delivers real enterprise value ensuring control, safety, and reliability across our development process. Before we dive into the deep technical details of our AI platform, let us give you some essential context about Flywire. Flywware is built to process the world most complex payments across global industries like education, healthcare, travel, and business services.

05:06:48 Today, we serve over 5,000 clients across more than 240 countries around the world. Because we process high value international transactions, regulatory compliance, privacy, and security are not optional atoms. They are core requirements of our business. We operate under strict compliance frameworks like PCI, DSS, and SOCK 2. This means that customer data, personal identification and payment details must remain completely safe.

05:07:18 We cannot simply take sensitive context and pass it to third party services without strict governance and architectural boundaries. To give you an idea of our engineering organization, we have around 400 software engineers distributed globally. Across our teams, we maintain diverse technology stacks, including Java, Ruby, Go, Python, .NET, and PHP. Almost everything we build and run is deployed on Victoria, our internal platform as a service that operates on top of GitLab and AWS infrastructure.

05:07:55 >> This bring us to Orbit, our own toilet killer. Orbit is an internal platform that eliminates daily engineering is called only where we every day. It also keeps the that the other AI services and tools we use across flywware wire can start workflows directly. For example, an engineer will be able to ask clo in slack use cases send can start workflows directly.

05:08:18 Victoria it starts. um >> and shares the mer request with the scorecards measures who read great sorry I'm good everyone I'm ash CTO of modal and I'm going to talk about really need from their infrastructure so I I'm told this audience is uh very finance adjacent. So I thought I'd talk about what if every cell in a spreadsheet could be backed by an agent.

05:09:16 And uh imagine you're a financial analyst at a fund. You want to run a complex analysis on every company in your portfolio. Normally in the past this would take days. you'd have to connect every portfolio company, look at their data sources, uh run some Python for each of them. Uh and now obviously with agents, this should be much easier. Well, actually, one of our customers, Hebia, built something that's exactly this.

05:09:47 You can go in right now in their product, you can uh you have a spreadsheet and you can uh run a query where literally every cell is backed by a different agent. And I wanted to talk about what does building this really look like? Well, uh first every agent is backed by a modal sandbox. Uh so sandbox is a uh secure environment for running untrusted code and uh it used to be that every sandbox was onetoone with a container or microvm.

05:10:32 That's actually no longer the case. Uh today actually uh modal has this uh feature where you can run multiple containers or microvms inside a sandbox and you actually need to do that for doing something like this where um in this architecture you run the harness. Uh the harness is the agent loop that does things like running the calling the model API for inference uh running the tool calls and then looping over it.

05:10:59 uh you want to separate that from the place where the tool calls actually run uh for security and isolation and a bunch of other things. Uh so you have the sandbox that's running multiple containers including a sidecar. Uh you also want shared storage uh because you want your agents to be able to talk to each other and it's up to you how you uh stripe it.

05:11:24 Like maybe you have every row uh gets the same volume and agents in that row and the spreadsheet can collaborate. Uh and you also want to share storage between your main container and the sidecar for some cases. And finally, you need all of this to spin up really really fast because your spreadsheet can have lots of cells. If someone runs a query, you want to launch thousands of agents and then you want to spin them back down when it's over.

05:11:52 So before I talk more about uh that I just wanted to take a quick aside to talk about what is modal. Modal is a platform of primitives that is built for AI applications. Uh so we've basically built this platform for uh from scratch. We've built our own file systems, the scheduling and uh the runtime uh to give you really great performance and devx for use cases like inference including LM inference sandboxes training and uh functions running a lot of batch processing and whatnot.

05:12:25 Uh and today in particular I'm going to talk about sandboxes what this primitive is and how it's evolving. Uh so if you want to build something like this uh or even further and we'll talk more about what that means uh what are the primitives you need for that. So the first thing is if you're building an agent you need a workspace and I have some examples from some of our current customers.

05:12:47 You might have heard of ramp inspect. Uh ramp inspect is ramp's internal coding agent that uh I think at this point pushes almost every single PR at ramp. Uh and if you're building a coding agent, you need your uh repo uh your GitHub repo, you need your dependencies, you need your dev services all in the same workspace uh for each agent. Another example is basis there.

05:13:18 What it looks like is your documents might arrive. Your uh agent is then triggered to work on them, but then it's waiting for a response from the accountant and you don't know how long this could take. Could take many days when the you want to persist the state over time memory. Your usage is going to change. And in the second chart, you can see usage for each sandbox.

05:13:40 And uh there's definitely some consistent trends, but it's all over the place. So if you were provisioning this doesn't make sense and then reliably get them when the sandbox needs them uh and only pay for them for for that duration. The other interesting emerging thing is agents need a lot more uh cloud science where uh pre-process some data on CPUs uh or spin up GPUs for running pre-process some data on CPUs uh or spin up GPUs for running a scientific model uh it will offload that to a modal sandbox is a scientific

05:14:19 model uh it will offload that to a modal sandbox and we're seeing this show up RSI about what happens when you give an agent uh sandboxes to hill client. The next emerging thing is agents are collaborating a lot more and um that means that you share a file system with your sub agent. You share file system. The next emerging thing is agents are collaborating a lot more and um that means that you share a files and files with your uh human where uh you're all collaborating with the same thing and this needs file systems

05:15:03 that are fast have consistency and uh devx you can reason about uh something we're working on volumes related thing is we're seeing a lot more self-improvement uh so agents will uh save and update memory. They'll run background tasks uh which is now called dreaming to update this memory over time. And actually this also just looks like files in a file system.

05:15:28 But again, you want really great performance from this file system. And the scary thing about agents is they can mess up and uh they don't really know when they might leak your sensitive data to a German forum or something. So you need to have a lot of control over what they can and cannot do. uh and you need things like controlling what IPs can even talk to the agent, how do you authenticate it, uh what domains can it talk to, uh and also if you're using credentials for talking to services, um basically you want to

05:16:11 inject them in a way that the agent cannot actually see them and then exfiltrate them somewhere later. And the interesting evolution of this is we've realized that domain u related allow lists are actually u not enough you a lot of our users actually want very specific control over uh even like what paths can I talk to uh on let's say data dog or github like maybe I want to uh reason about specific operations that the agent is doing and going back to sidecars we've seen this emerging as a pattern where users actually

05:16:47 write their own lightweight proxy that has custom rules about what exactly you can do. And RAM is another example of uh company that's been using this successfully. Uh and the final underrated thing is it's really important to have really great observability. Uh so going back to the example of auto research and an agent doing a lot of training, uh what happens when the training run fails?

05:17:16 You're you want your agent to be able to easily look at the logs of the failure uh reason about what happened and do another iteration with a different set of parameters. And while training is running, you wanted to be able to look at the metrics. You want to look at GPU utilization and make sure it's not stalled. Uh maybe you want to look at the budget and make sure you're not spending too much on your sandbox.

05:17:38 Uh and obviously all of this is really important for humans. Uh but as agents are driving a lot more compute uh it's we're spending a lot of time exposing all of this to the agents themselves so they can reason about all of this. All right, I think uh we know what we need. Uh why don't we have Astra build us an agent platform? Well, I don't think it's a good idea.

05:18:01 I don't think you should vibe code a sandbox platform. Uh and and there's three main reasons for that. Uh number one is reliability. I think we're all painfully aware how hard it is to get right. Uh certain websites we use every day keep going down. Uh and the reason for that is even with AI, it's actually really hard to reason about the feedback loop of like what happens in the case of this tail event when the system breaks.

05:18:31 Uh and this is why you need to be very careful when building systems. Uh security is another one. Uh you could vibe code a sandbox platform, but who is going to do all the work to like continually reason about the security boundaries and make sure it's uh defensible in in an age when everything is evolving very fast. Uh and finally scale. Uh we have a blog post about how we scale to a million sandboxes which involves a lot of very interesting engineering design and bracing about systems.

05:19:01 And this is again something that you kind of don't want to leave up to a vibe coded platform because when you get to scale you want it to work. Uh and on that point we're we're actually seeing massive scale for for a lot of things I mentioned. So uh for example for the snapshots I mentioned uh it's growing quite fast. We're seeing 20 million snapshots a week.

05:19:24 uh we're processing uh about 8 pabytes of data uh a day uh for for each of these snapshots and u our users are creating uh almost 2.6 6 million volumes a week. Uh, and all of this growing super fast. And just to take a customer workload for personal assistant, you need to create 10,000 sandboxes a second. You need to run um almost a million sandboxes and they're not going to last very long.

05:19:55 They'll just last for like two or three minutes. Uh, but when they come up, you want them to be interactive in in less than a second. That's that's what this workload really demands. And the other interesting thing on scale is you actually cannot get all this scale from a single cloud platform. The world is actually getting crunched on CPUs as well, not just GPUs.

05:20:17 Uh so one thing we spend a lot of time doing is actually aggregating cloud capacity from a bunch of hyperscalers and neoclouds. Uh so our users have access to CPU capacity across the world when they need it. Uh but we cannot rely on one cloud provider to do that. Okay, so where is all this going? I I hinted at this earlier, but the harnesses are really good at making use of dynamic environments.

05:20:46 And what this is leading to is a lot of separation of the brains, which is the harness uh from the hands, which is the sandbox or the runtime where stuff is running. And um what this means have an example here is the harnesses are going to be able to spin up environments based on what they need right now. They they they're good at snapshotting them and keeping them around for uh as artifacts that they need later.

05:21:13 Uh and again adjusting all of this as necessary. So I I think this points at an interesting evolution of the pets to cattle uh analogy we're all familiar with. Uh to recap, in the olden days uh people thought of services like pets. Uh each one was bespoke. You had to manage it. Uh you had to reason about this individual deployment uh specifically. Uh and then over time it um with containerization and Kubernetes people started to think about it as homogeneous uh templates uh of uh these servers.

05:21:53 They're all the same. You can scale them up and down. If one of them fails, it's not an issue because our system can heal from that. And I think this is where a lot of people deploying agents are today. They're kind of using these homogeneous environments and scaling up these agents. But but I think that's really limiting because the models are so good at reasoning about dynamic environments, snapshotting, uh creating lots of environments when needed and working across them.

05:22:19 Uh so what what does the new world look like? One analogy here is it kind of looks like an ocean of microbes of all these organisms that are self-evolving and working with each other. Uh and credit sick for the analogy here. Um, but one thing I'm really excited about is as you give agents these really rocksolid primitives for compute and storage and networking, they will be able to actually do a lot more uh by doing the stuff I talked about like spinning up lots of environments, working with each other and being very

05:22:55 dynamic. So yeah, I and us at modal really think that great primitives let your agents do more. And uh yeah, if you're interested in talking about this more, reach out to us. Thank you. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Hey, Heat. Hey, hey, hey. Heat. Heat. Heat. Heat. Heat. Hey. Hey. Heat. Heat.

05:32:23 N. Great everyone. I'm Ryan. I'm the CEO and one of the co-founders of Valai. I'm going to talk a little bit about uh going from beyond the model card, how evaluations can be used to measure this jagged frontier. Um special focus on finance for this audience as well. Um yeah, so first off talking about model cards themselves, we've built uh some of the most important benchmarks over the last couple of years that the major foundation model labs have included in their model releases.

05:33:58 So people I think are familiar with artifacts like this. Uh last week we have Sundur talking about our valves index on stage and then uh even just yesterday muse models uh citing our our finite agent benchmark. You know it keeps our lives interesting to have to keep re-updating these slides as new models come out. I had to double check level numbers.

05:34:17 You can get a sense of how capable a model is for valuable work. Um but that is actually driving value for a lot of companies. Uh and I think two interesting stats that jump out at me. The groups that are able to actually track AI cost and spend are much much more likely to inferior per but we also have refin the x marks again it's not immediately obvious which is number one or will be consistently two or three in the x-axis you've got cost in particular it's cost per task not just token excel you've got cost in

05:34:44 particular it's cost per task not just token excel models that are core the results may be intuitive unintuitive um when we actually build benchmarks one of the most important things that we need to look at is the metric we use to report accuracy uh and so you you you have a different takeaway if you pick a different metric a lot of benchmarks deal with partial credit And so what this is answering is in expectation.

05:35:02 If I were to give a question or task to a model, what percentage of the points would I expect to see satisfied by the the the work the work uh output of that model, especially compared to humans? And then the fully corrected answers metric is more looking at of the percentage of questions that I give to models, what percentage of them will come back perfect.

05:35:21 And that's that's a very different metric. It may mean uh that a model is very good at producing an incremental response that you can work with to build a better response. Um but if you were to pass this directly to a client, it would be a coin toss of whether it was meeting all the criteria that you have. Uh and so picking the right metric is very important to accurately representing the performance of a model and actually forming a good intuition of what it can do.

05:35:46 There's another three components that we think a lot about. So validity. Are you actually evaluating what's going on in the real world? That means are the questions or tasks reflective of actual work going on or are they contrived? Um and then are the rubrics actually describing what we're looking for or the standard of excellence we want to see. Reliability.

05:36:02 This is usually a function of the number of tasks or the sample count. Um a lot of companies they have maybe five or six samples and they're kind of vibe checking new models or products as they get access. Um but it's actually really important to have a rigorous approach and have enough tasks and samples to make sure your repeated runs actually show consistent results.

05:36:20 And then last discriminability. So we we don't think it's that valuable to have benchmarks where all models score 100% or all models score 0%. Ideally, we can we can actually discriminate between models and see what the frontier looks like. In particular, we also want to make sure we're building benchmarks forward-looking. So what are the new set of tasks that are even more ambitious than what models can do today to make sure that there's room at the frontier for us to to continue to see model progress as they get

05:36:44 better. Okay. So and then some of the high level problems that we've seen and I think others you know can and do run into. The first is that agents solve the test not the problem. So this is talked a lot about as reward hacking. Um and we've seen a lot of instances where we'll have a sandbox and models will be run with our model proxy and sometimes they may try and actually call other models uh within that proxy to escape the sandbox or make use of the internet.

05:37:10 Uh so it's really important to to build the the containers in a way that you're actually testing for what matters. Can they can they do the task at hand? Um or are they actually leveraging other tools or maybe even other models to get there? Um we've also seen this issue in finance in particular where a lot of the times eval are looking at a set of rubrics or or items that we want to see included in the response and so models can hedge and give three possible answers uh which has a higher chance of satisfying all the

05:37:37 criteria. Uh but in practice that's way way too much. A person does not want to consider three different answers. They just want to get the the first answer and and have it be right. Um, and so the goal here is to try and build evals that are reflective of the actual work people are doing and the work output they want to see. Um, not to build these contrived emails that that the models can actually gain.

05:37:56 The second is I think a lot of people are starting to see that the cost of software is coming down and it's actually very very easy to produce code but unfortunately we still see in practice that running evals is incredibly challenging especially from an infrastructure side of things. Um and in particular as we get to longer running agents it means that it's really important to have infrastructure that can stay consistent over a long period of time and then if interrupted can be resumed and respun back up.

05:38:22 Um we have I think just recently crossed six trillion tokens just in evaluation. Um so we've run you know considerable set of evals and we run everything in as highest degree parallel as possible. Um so I think this has really been load tested but I think other people are going to run to this even on a much smaller scale. Uh and the last is that I think a lot of the existing benchmarks they deal with a single prompt, single response, but in practice everything is multi-turn.

05:38:48 Uh you have an initial spec which you think covers everything but then you get questions back or you have additional feedback once something's built and then it takes more of this iteration to actually drive to the end output. And so I I think benchmarks and eval component that's not going to go away. So when I look at a case study in particular, when we looked at our team, we did a little token maxing experiment uh where we we got free access to some tools and just gave it to all the engineers.

05:39:13 Um and they they really made use of these unlimited tokens. Um and and if if we actually look back for that month, we spent $1.5 million worth of tokens. This is like a team of I think 10 or 12. So it roughly 10 times the amount we were spending on salary for tokens in that month. Uh and so that was pretty insane. But we saw great output. And so our goal was to try and maintain the same level of output across all these engineers but massively c cut down on cost and and really our solution to do that was of course to

05:39:40 build an eval. Um we built a bunch of tools internally and then have now exposed this as a product called valith. And so this is you know what we're doing in coding and I think it's again a sign of what's what's going to come in other parts of knowledge work. Um so what we were able to do is look at the github repo um as a a centralized way of seeing what work has already been done and what work is continuing to happen.

05:40:00 Um, and in particular, GitHub repos are comprised of PRs or pull requests which have a GitHub issue and a resolution. And so what we can do is unwind the clock and give those same tasks to agents to see if they could have equivalently done the work of humans. Um, and in doing that, we found a a much clearer understanding of which models were overkill, were way too expensive, and did not need to be used, and then which ones were actually capable of getting the task done.

05:40:23 And I think again, this is another place where the intuitions that a lot of people have may not be what's actually born out in practice. So I think you may see we're using a lot of closed source models. We should try and cut cost and use our own self-hosted open source model. Uh you can see the comparison here we have between GBT 5.6 Soul and Kimmy K3.

05:40:38 And we see that 5.6 Soul is better. It's it's scoring 2% better on this internal eval. It's also considerably cheaper than than hosting it through a very you know inference optimized uh stack. So it's $ 250 instead of $11 and it's faster half the time. And so there's no real reason why um this kind of intuitive sense of using open source would be a legitimate way to drive down costs.

05:41:01 Um and so this has resulted in two main differences. One, we can actually figure out which tools we're going to procure, what the usage limits of those things should be across the organization, but then also we can give recommendations for engineers and say we think that for this GitHub issue, you should actually use this particular coding tool with this model.

05:41:19 And this is able to drive the cost down considerably. I think we're spending 90% less but still able to get the same output. I think the question I would have for you all, we're we're continuing to expand this approach and and take this model of coding to other domains, knowledge work, legal, finance, things like that. But the key question is where are you tracking the history of work that has already been done by that firm?

05:41:43 Some of these law firms have been around for like a hundred years. And so where where is that I think quality of the organization stored? Where are you tracking the standard of excellence? And so that's today may exist in places like email from the history of responses that people have had with their clients exist in sharepoints may exist in data bricks dumps.

05:42:00 But I think this is the key treasure trove that a lot of companies have and don't realize will be the major unlock for evals and then the unlock for AI adoption more systematically. Once these eval are constructed, they can enable adoption to happen. But we've also seen that there's more advantageous ways that they can be used. So first is right sizing.

05:42:25 So rather than just token maximizing, how do you figure out what the right usage limit should be across the organization negotiation. Once you realize uh you have actual leverage, you can take that to the conversation you have in bulk pricing. Then routing. This is becoming a core primitive in uh in a lot of AI products. um but can also be done by users directly and so they can build intelligent routers that actually figure out the right model to use and then fine-tuning or post training.

05:42:52 Uh this this can be the most reliable way for you to actually mint the tasks and then have an external eval set to make sure that this approach is working. Last thought I'd leave you with is I think for a lot of companies they have a competitive advantage that has existed in the heads of employees through things like institutional knowledge. And what evolves allow you to do is actually com codify the competitive advantage and put that down in words or in scripts in language and that will actually be the the biggest

05:43:21 unlock for making sure that a company succeeds the gentic era. Okay, that's all. Thanks everyone. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. N. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Hey, Heat. Heat. Heat. Heat. Hey, heat. Hey, heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Hey everyone, how are you? Thank you all for uh for being here.

05:55:34 My name is Venu. I'm the CEO of Kepler. Excited to have Jack Woodruff with me. Want to quickly introduce yourself? >> Yeah, hi. I'm uh Jack Woodruff. I am the founder and former CIO of Candlestick Capital Management, a long short equity hedge fund based in Greenwich, Connecticut. Uh before founding candlestick uh I was a partner and portfolio manager at Surveyor Capital which is a division of Citadel.

05:56:03 >> Very cool. Thanks Jack. Um so the reason we did this this way is we've had this conversation before and I think this is one of the most enlightening folks to chat with given that Jack has spent his career picking stocks in an environment that's changing more than it ever was before. Um, so we'll treat this really as a as a fireside chat. Uh, so Jack, paint the picture for us.

05:56:23 What does fundamental investing look like in the age of AI, say five years from now? Well, it's it's hard to say. Like long-term stock picking, I don't believe, is really going to change much at all. There's always been a mosaic and there's always been alpha creation by finding a divergent view. So, that isn't going to change at all. I I I think that that still will stay the same.

05:56:48 Now, how people synthesize all of the information and where table stakes go to into in terms of like what everybody has and then how you synthesize that data is going to change completely. um the roles the traditional roles of say a portfolio manager or an analyst are going to be completely thrown upon their heads and I still think it's like too early to say what that's all going to look like as these tools are just so new especially to the users.

05:57:22 you got a lot of people doing a few things, but it's just not clear to me that it's like we're there and oh my gosh, it's going to be these new things. Like we have we have a bunch of other things to kind of talk about, but um I think it's going to be super fascinating to to to watch how it all plays out. >> Very cool. So at Citadel, you had 18 people covering 400 stocks and at Candlestick, you had 30 stock pickers.

05:57:48 Could one great analyst cover all 400 by 2030? >> So the biggest issue is is is humans are still the bottleneck. I mean like at the end of the day uh you have to get the information into your head. So it's really all about sort of how man and machine can work together. Um investment management is a time management exercise. What I mean by that, if I spent all of my time working on one stock, that'd be great.

05:58:20 I'd probably know better than anyone else in the world, but like if there wasn't anything to do in that stock, I'd be kind of there. There isn't much to do. Well, if I do two, well, then I have half as much time and and so forth until I go. So, you start getting diminishing returns uh on how much time you could spend on any particular stock. So, you know, what can AI do?

05:58:46 Can AI help me cover more names perhaps? Um, maybe it allows me to raise the table stakes on what I just kind of like have as a general knowledge. Um, because that work used to take me hours to prepare to get me ready for what it was. And now maybe the you know uh an AI program can you know help me prepare a lot of that work in 5 minutes where which used to take me 6 to 8 hours.

05:59:11 Um so that allows me to do other things again like it's a very competitive industry. So it's not like if I get rid of all these people or if I have like five analysts whereas before I had 30 um yeah maybe those people can do more but now all of a sudden this opens up other time this opens up time that I could spend doing either more proprietary research channel checks um more sort of like industry backdrop work because again like you're going to need to find these divergent views.

05:59:43 So, it's like if if I do the exact same thing I was doing before and it's like, "Hey, wow, I can get my work done in two hours." Well, someone else is going to be still working 16 hours a day trying to find like that extra thing that's going to that that that's going to make you money. Um because you know just doing this stuff is is obviously like what we used to pay an analyst to do um you know a year or two ago is not what we're going to be paying an analyst to be doing you know in a year.

06:00:17 >> Very cool. And on that what's what is something today that a junior analyst does that you think will be absurd if they're still doing in five years? >> Well it's funny like it's like oh you're you're not going to be reading 10Ks. you're not going to be reading, you know, transcripts. I actually don't believe that to be true. Like I if you're truly close to a name, um you're going to still need to like process a lot of that information.

06:00:40 Now, you might be able to get like summarized versions that can cut down the time it takes you to do those things, but you still need to get that information sort of into your head. Um like we we used to have a lot of analysts that would be going through a lot of like sort of like big data and things like that. I definitely think that that can sort of be outsourced and that'll be super easy to, you know, I would be surprised if if a lot of these guys aren't necessarily doing that stuff.

06:01:05 But what I believe the AI is going to do or the agent or whoever it is is going to be pushing you in the right direction to work on the right names. Like I said, this is time management. It's all about how do I know to work on the right things? And that's where the winners are going to is going to be is is the people who can figure out how to use these tools to get you to spend the right amount of time working on potentially finding like a divergent view.

06:01:39 >> Very cool. And on Heat. Heat. Heat. Heat. N. All right. Hello everyone. Welcome. My name is David. I lead AI operations at a company called Revolute. We have uh 50 minutes today together. We'll talk about high performance engineering and self-hosting open source model on dedicated GPU. How's that sound? Cool. So, it's going to be a technical one, so bear with me.

06:03:59 Um, so maybe to start off because I know this is the US and you may not know Revolute. Uh, quick words about our company. We're a digital bank in Europe. We have 80 million customers. We're live in 45 markets. with the most downloaded financial app uh in Europe for the last year and as of July we're Europe's largest private company. Um so we're doing okay.

06:04:28 The the emphasis of Revolute is you know shipping a lot of product and so there's a high focus on engineering culture and uh excellence on that domain and so we talk about that. So let's talk about some AI. We have a chart here. You're going to see two lines. This is a sample of 2,000 engineers performance over the last year. So the green line is going to be lead time.

06:04:56 So it's going to be when engineer picks up a feature until a feature is merged and ready to be deployed. The blue line is going to be coding time. So the green line is in days, the blue line is in hours. Just to reset the timeline in case people forget because AI moves really fast. Clo came out around February 2025, right? And that's the code at 3.7, the one that would delete your pro database, right?

06:05:25 Not the safe code and codeex that we have today. So bear in mind when we talk about adoption. So what you see here on this chart is um January 25 first introduced AI in the company right um Gemini first time people were able to use private data with an AI around February we started playing around with cloud code and then January is when we first launched cloud code at scale what you see is um Gemini had 10% improvements in lead time 20% improvement in coding time so that's people no longer need to look stack overflow

06:06:00 and reading documentation but then cloud code not so much impact at the time. So when we launched it, we were highly skeptical about the cost of AI. We were seeing engineers spend a,000 a month, 10,000 a month with not so much ROI. And so when we launched it, we launched it with um strict budget limits. And so we're looking at the next thing, right?

06:06:24 So how can we provide unlimited tokens to engineers so they can use AI guiltfree for a fraction of the cost? And so we looked into open source. So around April 26 is when we started self-hosting open source model. We're a big fan of GLM for coding. So around that time what you see uh April 26 GLM51 we see 60% improvement in uh coding time 70% improvement in lead time.

06:06:51 So dramatic impact. What do we learn from this? Well there's more impact on the lead time than on the coding time. So what we find is the bottleneck for engineering is not necessarily coding. You can save a lot of time not having to dig through your inbox or messing with J conference. So when you have a limited token, you're free to do that. You no longer have to save your tokens for the end of the quarter or the most difficult task.

06:07:22 So where do we stand? Well, open source is now a priority at revolutes. New economics are favorable. The billing is flat. We don't have any bill shock end of the month. And the frontier is the exception. So obviously we keep using code exclude code. The lips model are fantastic. Gemini are going very well, very good as well. But um we keep them as exception.

06:07:48 So how do we get there? some metrics that matter when you're self-hosting model. So to serve 2,000 engineers, we roughly need 56 B200 and that's to host GM 53. GLM is a big model, right? Ethan parameters. What you want to aim for is sub3 second time to first token. This is empirical evidence. As soon as you exceed 5 seconds, my slack inbox starts to blow up.

06:08:19 And so we know that's the limit. Terms of throughput doesn't really matter as much, right? So we serve in terms of interactivity 200 token per second. People are fine. You don't need to go higher. Um the way people work has changed. We no longer look at the stream of thinking tokens. People run task in parallel and check in when the task is done. But if you want to have high throughput and you're an engineer at Revolute, you just connect after 5 p.m.

06:08:43 when the one is on GPU and you can get 800 TPS and then you know it's a dream. Terms of cost, in terms of impact, uh self-hosting GLM or blended cost of token is 30 cents per million and so our invoice is onetenth of what we were spending on entropic and um OpenAI. So there's a lot of tuning that goes into running open source models. If you just get GPUs and upload weights on a GPU and you start doing some inference, you're going to get terrible performance.

06:09:23 It's because your workload is going to be very dependent um on the parameters that you set. So if you look all the way to the left of that chart, the blue line is going to be demand. So that's request per second. That's going to be what's going to be the bottleneck for your latency for your GPUs. And the red line is what we're optimizing for. That's the time to first token.

06:09:45 So when we started back in April, you could see request per second around three, which is high but not so high. And time to first token over 10 second, which is very slow. But the data that you see is the same number of GPU. We didn't increase GPU count. We still have 56. And by the end you see in August request per second we're over seven and time to first we're around 1.6.

06:10:10 So we've had 88% improvement in performance for 76% increased demand. So the learning from this is there's a lot of parameters that you can improve on and you really need to resist scaling if you're looking to self host some model. So let's talk parameters. There's no secret recipe, but the things you could consider are the following. First one, long context is a scam.

06:10:37 You really need to be mindful that even though you can serve 1 million token, 256, 256 is really what you need. Quantization, if you're running black wall generation GPU, you don't want to run anything higher than FP4. And VFP4 is best. Roughly going from FP4 to FP8 is a 40% saving in GPU footprint for zero loss in precision. Other parameters to consider tensor paralism.

06:11:10 Even though GLM fits on uh TP4, you don't have enough KV cache and so maybe a TP8 works better depending on how large your workload is. And then on the more zoteric um check your cache it coding test needs to be above 90% cache it and you can look into optimize spec decoding uh sequential prefill pdag all of these cool so now let's talk about um production setup GM obviously is a text only model which is a limitation for some people the trade-off I would say uh we would make every time is get the smarter model and

06:11:50 figure out how to do image recognition. Another way you can easily have a local router open codeex is one where you would route any image recognition to Gemini or Luna and keep JM for the main task. If you really need some uh vision model, GM 5G flash uh works really well. We have our UI designer making great use of it. um turnaround for uh UI product has gone a really really long way and then the last one is going to be about uh the harness.

06:12:28 So all of these endpoints are going to be open AAI compatible API with response API and so you can use them inside open codeex codeex app as well works very well and so that's what we uh we recommend and then um last message and this one is for the CIO in the room there's been a lot of stigma in using Chinese uh open source model. So the thing to consider is you're not using the inference dedicated endpoint that goes to China.

06:13:03 You're just getting the weights and putting them on a US-based service. So for data sharing side, this is totally fine. Revolute in particular, because we're a EU regulated bank, we fall under the most uh strict regime in terms of compliance. And so what we find that works is all of these coding task are not considered autonomous agent. They're always with a human in the loop.

06:13:30 And so how do you give reassurance that you have a human in the loop? You start by creating a sandbox that guarantees that. And the sandbox means now if you've used codeex code, it has all of these features of approve for me. You just have to set the rules for the operations that you don't want to allow. So you can allow grabb but not gradle for instance and that's going to give confidence uh to your regulator that this is not an autonomous agent that needs different supervision.

06:14:02 The sandbox also protects you from uh rogue use, right? You may have bad actor inside your company and you want to shield uh for that. You'd be surprised what you find in a large company. And then the last one is prompt injection. Not not so much a thing u now given how smart models are, but always a thing in terms of data privacy. We spoke about it.

06:14:23 So as long as the data sits in one of the countries that are in the data equivalency framework. So US, EU, UK, Israel, you're fine. You can get your GPUs there. All of your inference provider or GPU provider are going to have zero data retention by default. And then the last thing you need is a very strong uh open telemetry so that if you ever have an incident um you need to be able to go deep and look at the logs also gives you some yeah analytics.

06:15:00 All right, thanks everyone. That was all for my talk. Heat. Heat. Hey, hey, hey. Heat. Heat. Heat. Heat. Heat. Heat. N. Heat. Heat. Heat. Heat. N. Heat. Heat. Hey, heat. Hey. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Hey. Hey. Hey. Heat. Heat. Good afternoon um everyone present here. It's um it's been a it's been a great day and um an absolute pleasure to be here.

06:28:52 The topic of my talk today is everyone has AI who has the advantage. I'll be talking about what the orgs that are doing that are leading in AI with their implementation and adoption, what are they doing differently? Um, and so before that, who am I and what's my POV? So, I'm Dr. Sid Pedniker. I am an AI technology and data exec in asset management. I've spent most of my tenure in hedge funds and um investment and academic research.

06:29:32 Uh what I've done and focused on is lead strategy and build specialized teams towards three main verticals. Optimizing efficiency, driving P&L and reducing costs. So um I'll be following the discussion along with uh two hypothetical companies company A and company B right at this slider below. Uh these are hypothetical um you know with um with madeup workflows but we'll be following their journey along um the rest of this presentation.

06:30:12 So everyone has AI. Who has the advantage? Um before we get started there, first let's see what the lay of the land is. And recent studies have shown that only about 37% of enterprises that were surveyed are showing a positive contribution to their EBIT and only about 6% of those would be qualified as high performers. So that's about 5% contribution to their EBIT.

06:30:43 Other studies earlier in the year have shown that about 66% are saying they're having a productivity increase but only about 40 and 20% are actually seeing cost reductions and uh revenue growth. So as we know access of AI is widespread yet the impact remains uneven. So the question I would ask or how I would think about this is really in terms of three verticals.

06:31:20 And the way I think about these three verticals should apply to organizations and individuals alike. And it should apply to functions regardless of where you are in that organization. Um so the three main vertical pillars that I think of are number one design the right problem number two build shared capabilities and number three compound the advantage.

06:31:52 Now before we dive into each of these pillars, there is an idea that I think goes through all these verticals and I think the best leading firms are taking advantage of that and capitalizing on it. As we know throughout time, the only constant in this world has been change. What's different in the AI and super intelligence era is that intelligence has become cheap.

06:32:20 Regardless of what your unit of intelligence is, whether it be software, analytics, recommendations, reports, AI is creating an abundance in all of it. And um as we know, more answers don't necessarily create more value. So what is scarce? What is a scarce resource in an abundant world? What what are companies taking advantage of? the leaders at least it's judgment.

06:32:55 What is judgment? Judgment is choosing what matters and how to act when there's no single obvious answer. It's the creativity that lets you explore what possibilities are worth exploring. It's the taste that tells you what good looks like in your given context. It's the decisions where you can weigh what tradeoffs are worth making. Judgment is ubiquitous.

06:33:19 It exists throughout your organization on a daily basis. Your seuite, the seauite is asking questions, taking advantage of judge judgment all the time. So are the software engineers, the AI leaders, the portfolio managers and the data scientists. So the question that they're focused on is given judgment is a finite asset a limited capacity the main question becomes where is our best judgment worth spending like for you to hold on to that thought with that we'll dive into our first pillar design the right problem so

06:34:02 questions uh a question that um the leading organizations are asking all the time is they're obsessed over what is the right problem to solve. Right? It's tempting to start with the tool. Today we're surrounded with better models, harnesses. We're surrounded with all these AI tools. We're surrounded with co-pilots, but that isn't the way about, is it?

06:34:26 At least that's not the way the leaders are approaching it. Now, let's go back to our hypothetical um two companies. Let's say they're from asset management. Let's say uh we're tracking the workflow of two research analysts in these hypothetical um organizations. Well, let's say organization B starts with the tool, the co-pilot. They're asking where can we add AI to the analyst work?

06:34:54 They're looking at the current existing process to say earning transcript comes in. I discovered a summarization workflow. Let me pass all of this in through my fire hose. Analysts get earning summaries faster. Win-win, right? Well, not so much. At least not compared to the guys who are leading. And such a workflow of of being a faster horse introduces one key um nuance that is you are losing context as you have information um I guess reduction as part of your summarization.

06:35:31 So what what is a hypothetical organization A doing in this? Well, they're obsessed on the right problem to ask. They could be asking why are our material signals reaching us too late? Where do we focus the attention, the AI, the bandwidth, the capability on where is our bottleneck, right? And so it becomes this redesigning of the workflow or at least the re-evaluation on what to focus on.

06:35:57 That's what they're doing different. Now this consideration isn't particular to finance. It's across the the realm and function. You know you're asking the same question on what to focus on whether it be research capacity, decision speed, quality of your data, latency, reliability. It it it traverses all of that. And studies show this. They showed leading studies um on average spend their attention on fewer higher quality decisions than the other cohort.

06:36:32 So leaders are being more selective in where they focus their attention and what they focus their attention. So it's great the leading companies have um an idea of what the right problems are. Where do they go from there? while they also concentrate their investments in the highest value workflows. I get it and I've been uh I've been culpable of of something similar where you know is AI is almost creating too many opportunities and experimentation can be a good thing but one needs to separate experimentation and the

06:37:08 resulting innovation theater or worse the productivity theater which actual value generation Going back to our two um hypothetical companies, what does this mean? Company B is looking at how do we launch the most number of AI pilots. And what that ends up looking like is a a document summarization bot here, a co-pilot there, um a a data retrieval bot here.

06:37:36 It's scattered all over the place. Whereas what company A is focused on in their investments is potentially use cases like end-to-end workflows on the research um on the research pipeline. They're focused on aligning engineering, analytics, AI capacity and judgment towards a singular focus. And research has shown that the leaders spend more than 80% of their investments on these high value use cases, transforming key business functions and creating new offerings.

06:38:16 Perfect. We we know they're focusing on um finding the right problem and they're also spending deep investment in select workflows. Well, they're also doing something different. They're reimagining how work is done. I think this is this is key and it goes back to what we discussed previously which is judgment. They're redefining and reorganizing functions, roles, capacities around this key element of judgment.

06:38:48 What does that mean for our hypothetical company? Well, company B might be focused on, as we discussed, I'm getting my earnings transcript. I'm getting my research uh my research reports. How do I summarize it? Maybe I make that faster. Maybe I update my financial models faster. Maybe I send my recommendation to my PM faster. Company E A is focused on an entirely different ballgame.

06:39:11 Hypothetically, they could be asking the analyst to define the investment thesis and they're watching and gathering evidence that could refute that thesis or support it. They'd be monitoring continuously. They'd be investigating exceptions and they could be deep diving on, you know, select examples that could lead to actual generation of value. So the winners just don't just automate, they redesign.

06:39:43 And studies show that the cohort of high performers are 75% more likely or in practice they actually do redesign their workflow versus 25% of the cohort that doesn't. So where does this leave us with our two hypothetical companies um and designing the right problem? Company B adds AI to existing workflows. They're completing individual projects and they're moving to the next initiative.

06:40:08 Whereas company A is rethinking how work is done. They're redesigning processes around AI and they're obsessing over the the the right problem. That's that's the difference in how organizations puts AI to work. Great. So they've picked the right problem. Uh what what happens next? Um well, we all know in order to actually enable that capacity, you need to build share capabilities across the org.

06:40:36 And what's what's a trend that's seemingly emerging for for some time now is that the lines between technology and business are converging. And we're seeing this everywhere. We're seeing engineers that are moving closer to business outcomes. We're seeing business professionals that are building tools and automations. And as you can see and imagine, you can have forward deployed engineers, you can have forward deployed product managers, but you could also have the inverse, which is for forward deployed uh portfolio

06:41:06 managers. And it's really this combination of getting the right people in the room by breaking down organization silos that the leaders are really leaning on. studies have shown that uh more and more of the large enterprises are viewing their technology arm as a business growth function. Um and that's creating an edge for these for these companies. So the other sort of vertical inside building shared capabilities that um is probably a personal favorite of mine, it's um standardizing the core and personalizing the edges.

06:41:46 And this is something that isn't new uh to the technology world. We've we've built infrastructure. We've built, you know, shared modules and system design. But what's interesting here today is that's taking a very different shape with the capabilities out there and with the org and the breaking down of silos. And this could look very different depending on where you sit.

06:42:08 You could be building shared cores for a, you know, a team that looks very different with analysts and and clients. You could be doing that with, you know, a bunch of analysts or you can do be doing that for software engineers or or data scientists. with the data scientist example in particular that could look very interesting where instead of having oneoff workflows that that is really common in the industry.

06:42:35 You could build a shared core of the data sets, their semantic layers, the trade-offs of what worked, what didn't, what, you know, what what some data sets do good of, where are the gaps and and so on and so forth such that each new personnel becomes almost a plug-and-play module in this system where they can then stand on the shoulder of their predecessors.

06:43:01 So what should the next person not have to rebuild? Now what's also interesting is um you know the enterprise wasn't really built for agents. Um agents they they work across workflows and applications. They they take context they uh deliberate they take actions. whereas really the enterprises are focused um previously on uh single human or a collection of them interacting with a single application.

06:43:31 So the leaders are cognizant that something needs to change and that they need to modernize um their infrastructure and they're very self-aware of where they are in their journey. And so studies show that you know 75 71% of the orgs are modernizing their core infrastructure to support AI and um you know 6 to 10% or 23% of them are investing 6 to 10% of their annual revenue in in core system modernization um to take advantage of this.

06:44:02 And a common complaint here or a common challenge here is um you know data quality and and the ability to reuse data. Um so great we you know we we have the right people in the room we um have you know we have the right knowledge there what is what do we do next and then comes the concept of hey how do we make this knowledge reusable and accessible and that's where we have you know depending on on the adoption and where where they are you can have different types of team agents where you can have expert agents being

06:44:36 subject matter experts you can have workflow agents that work across streams like Slack bots that are creating and opening tickets, they're closing them and what have you. And then you can have really chief of staff kind of agents that keep a tab on what the state of the organization is and what needs to be done. So um you know going back to our workflows a again is looking at this very differently in terms of shared capabilities.

06:45:02 Now the last pillar here that I want to draw your attention to is well when does this system really compound? What happens when it's deployed? Well leading companies what they're doing is they're actually thinking about it slightly differently. What are failures that can give us an edge that we can learn from? If if there's a success, how do we make it part of the system?

06:45:29 And the way they think about this is, you know, the individual learned but did the system. Is there a way perhaps when they've encountered something where they need to make a correction that that knowledge is fed back into the org either through a new tool or context or or or what have you. And um what's what's key here is preserving the evidence along with those learnings.

06:45:54 So um really the last vertical here is AI freeze capacity where does it go being cognizant that you need to redesign your system and really focus it on key metrics that you're evaluating you're monitoring and you're creating a f flywheel off. So where does this leave us for companies A and B? You know, company A has their entire work redesigned. They're building capabilities at compound and they have shared capabilities built.

06:46:26 So, I want to leave you off with one final question. Everyone has AI. Who has the advantage? Thank you. Heat. Heat. Heat. Heat. Heat. Heat. N. Heat. Hey, heat. Hey, heat. Heat. Heat. Heat. Hey, heat. Hey, heat. Heat. Heat. Heat. Hey, heat. Hey, heat. Heat. Heat. Heat. Hey, heat. Hey, heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat.

06:58:37 Heat. I do Heat. Heat. Heat. Heat. N. Heat. Heat. Heat. Heat. Heat. Hey, heat. Hey, heat. Heat. Heat. N. Heat. Heat. Heat. Heat. N. Heat. Heat. Hey, hey, hey. Heat. Heat. Heat. Heat. to guide us back into the closing keynotes of today's program. Please welcome once again to the stage our MC Alex Lieberman, managing partner at 10X. >> How's it going?

07:09:22 How's the Who is the most energized in the room? >> Whoever that was most energized. Well, welcome back for the closing keynote block here at AI Engineer New York 2026. I'm Alex Lieberman, co-founder, co-managing partner at 10X applied a partner for for Fortune 500 businesses. And we are most of the way through the first session today and hopefully it's been filled with incredible conversation which I'm going to share a little bit about uh some of the conversations I've had uh offstage in a minute.

07:09:55 Um but hopefully that the the mix of the programming, the conversation you've had amongst everyone has been really fruitful today. Now, before we hop into the final block of the day, let's just take a moment again to give a round of applause to our partners who make this entire event possible. So, let's start with our presenting sponsor, Arise. Let's give a shout out to Arise.

07:10:22 Yeah, Arise. Next up, we have our platinum sponsors, Oracle, Sonar, Work OS, and Exa. Let's give a shout out for them. And then two more. We have gold sponsors. I love them all. They're all my favorite children. Can't read them all off right now, but thank you so much. Our silver sponsors and to all of our supporting partners who make this possible.

07:10:45 Thank everyone. Yeah, I don't know if that was the first person who said they have the most energy, but that person has a lot of energy. So before we hop into the next block, one thing that I just wanted to share and I'm sure a lot of you are already feeling this already is not only is the energy palpable across the room in between all of the sessions, but what you realize when you come to AI engineer is that the content is incredible.

07:11:13 The speakers that are curated are incredible. The depth by which they talk about the work they're doing is incredible. But this is also just surface area that acts as a starting point for conversation amongst everyone in the room. And so for me, not only has it been incredible to stand in the back of the room at various points in the day and just listen to these incredible speakers who are truly living in Edge City, but then to use all of these conversations, all these talks as surface area to have conversations with

07:11:46 folks. So, a few conversations that come to mind that I've had throughout the day because of the content. One was a conversation with uh Swix um one one of the people behind this incredible event. We were talking about just the diffusion gap that exists in enterprise AI right now. And just some fun stats for you all. We talked to hundreds of big enterprises at 10x across financial services, healthcare, industrials, etc.

07:12:14 And the way I think about diffusion of AI across these organizations is almost in in milestones of a race. So the the first big milestone at an enterprise is just adopting a general purpose LLM. Out of curiosity, of the let's call it 100 firms we talked to, what percentage of those firms do you think have adopted and rolled out an LLM to all of their employees?

07:12:37 Someone screw him out a number. >> 9% I think I heard. 17 >> 17% so specific uh 90% so 90% of the enterprises we've worked with have rolled out a general purpose LLN under that is people actually getting trained in how to use the technology how to actually use these services in their work that's like 60%. Next up is coding agents. Agents that are actually used within engineering and product workflows.

07:13:06 That's around 50% of all the enterprises we've been in. And then the final one where we are truly still day one is agents actually working in non-engineering workflows in an organization. That is around 5% right now. So going back to what I said earlier, this community really is edge city. the the topics that are being talked about, we are going to continue to see those play out over the next 12, 24, 36 months.

07:13:32 I've had a variety of conversations like this throughout the day to fusion gap, uh, open source versus fine-tuned models, how hard it is to hire great AI engineers at scale. And so as we go about the rest of today, but also as you think about the conference in general, use the content as a starting point to have interesting conversations at the edge.

07:13:54 With that, I would love to introduce you to our next talk, which is with Frank Long, the head of AI at COTU. He's going to be talking about a hedge fund as a function. Let's welcome Frank. Hello. Exciting to be here. My name is Frank. I work at COTU, which is a technology investment firm across public and private markets. So, in our business, what you really need to do is be able to take high conviction bets on big themes and watch those play out.

07:14:37 I think honestly a career is the same way and last year when I joined KOU the bet that I wanted to make was I think that agents will have an even bigger impact in investing than they did at coding. I know that's a big statement. We're all here because of the transformation transformative impact that AI has had in coding. But I really believe it strongly and today I want to talk a little bit about the framework that we use to actually realize that impact.

07:15:03 So at a high level you can think about a hedge fund as a function that takes in two parameters context external and context internal and spits out one tuple selection size timing what stock how much when right and I think this is a good framework to actually to ground our work which is we're trying to solve this compression problem of how do you take all the information in the and spit out this one simple tupole and the product that we provide is essentially a time series of this tupole that constructs a portfolio.

07:15:42 So let's start from the top which is the very simple philosophy. We use our screens every single thing that a human can see in pixels on the screen an agent in your building should be able to see via API. Uh the analogy I use is imagine you brought the smartest people ever, the greatest investors alive and you brought them into your company, but you gave them a computer without login to 75% of your services.

07:16:11 It's almost immaterial how smart the system is if it doesn't have the tools to actually action that. So the two parts external context, right? No surprises here. Think the news, prices, filings, all of that should be easily accessible. and not via dragging and dropping, copy and pasting like directly via API. Internal context is just as important, right?

07:16:39 So this is your own firm situation. So that might be our current positions, potential liquidity and risk requirements we have around certain names and past case studies or um you know history with a certain sector or stock or management team. That stuff's all super important. So this is the foundation and I think uh no credit to me KOTU has been an amazing leader on this for over a decade.

07:17:03 It's both technological and cultural. We have an amazing data science team, amazing amazing processes, amazing amazing systems that bring all this in. And just as importantly, we have a culture that is super super rigorous and built from the ground up to capture all this insight. For decades, we filed every single note about every single trade, about every single decision we made or didn't make about what went well and what didn't go well.

07:17:28 And that's poised us to expose us to AI. So, if you think about it, I probably can't just take all of those things, jam them into a context window, and get my magic numbers and go home, right? function a bit further, right? So maybe the first step is I take in that external and internal context and I gather it, right? I want to gather it. I want to filter out what's relevant and not relevant.

07:18:00 Okay, well I pulled some news articles about the MBA that probably isn't material to stocks, but oh well this is actually related to some local permitting that might actually impact data centers. Like there's a lot of nuance in this gather step. Um then you'll see probably the next step is I need to analyze that information and form a view right I actually need to think about what that means for the markets public and private.

07:18:26 Uh and then I need to take that view and turn it into a decision. A decision object has these three parameters right these three sort of sub sub pieces. So you can see every single step has an input and an output and the input for the subsequent step is the output that the previous function generated. I think as an engineer this is a way for me to conceptualize the problem in a way that makes a lot of sense.

07:18:56 But even this it really can't be as simple as writing three functions. Right? So let's just to really appreciate the scope and understand the full complexity of the problem. Let's look into the first step. A tiny subpiece of a tiny subpiece of just gathering information. And let's do a case study on interpreting the news. Right? So it's probably no secret to you guys that you know news from Asia and Korean Korean media outlets has been super important this year because semis AI have been a huge huge theme in the market

07:19:34 huge theme and I think this is an area where AI is poised to have a lot of impact is reading uh Korean language news better than our completely non Korean speaking staff right uh I think I have a good thesis that this might be a good use case So you can imagine the first step is I have uh articles. I need to collect the articles, right? So this could be an MCP.

07:20:00 This could be a CLI. License the data feeds. Clean the data feeds. There's actually a lot of ways to do this and a lot of dimensions of freedom for how you can even just write this simple piece. How do I chunk it? When I pass it to the LLM, do I want to pull the images as images? Do I want to pull the images and then turn the images into text? There's actually a lot of knobs here.

07:20:24 And I almost think about those knobs as like hyperparameters in a traditional machine learning system, right? I think about each of these knobs as something for me to tune and optimize. Obviously, we talked about it. Uh Korean, there's probably Korean NBA fans, right? There's probably a bunch of stuff that's not necessarily material to the the names and themes and stocks in our portfolio that really really matter, right?

07:20:53 So, I want to make sure that I filter those out. There's this like translate slash extract slash should I summarize? Should I extract this into some sort of structured form? Again, so so many ways to do this, so many ways to experiment how to expose information to the model. And one of the reasons this is hard is it keeps changing and obviously checking checking like our own internal notes, our own internal views, potential conversations we've had, meeting recordings against these things, we actually need to cross

07:21:26 reference those things. Actually, the idea of pulling information from disparate sources and cross referencing them, that's super powerful. So, you'll see on on the right here, I have code lm lm code person. Uh there's many many ways to run these functions. I think about uh oh yeah and finally update potentially we update uh our views. So there's many ways to run these functions.

07:21:51 I think of each line as having uh a runner and uh the most obvious one is code. Code is very good. It's fast. It's cheap. It's reliable. So for something like if the prices to earnings ratio is less than 15, I should just use deterministic code for that. Runner number two is statistics, classical machine learning and you can imagine fitting a line to a regression for a specific pattern.

07:22:20 This is very powerful because you can model much more complex relationships than just pure deterministic code, right? And for more complex relationships, the reason we're all here, LMS, AI agents, that's another type of runner, right? So for something like summariz, right? It's probably totally impossible to build a bunch of if statements that can summarize better than a modern LLM.

07:22:49 And of course, uh people people are important runners in the system, right? So, uh, imagine a function meetings the management team of a company. I probably don't want to send I mean, I like the new Unity robots, but I probably don't want to see send an LLM or an AI agent in to do that. I probably don't want to send an if statement into that meeting.

07:23:07 I think there's many cases where because of judgment and the ability to navigate the real world and all of these things, a person is the best runner for a given piece of the function. And I think this is like a super super important point. There is no glory in automating things for the sake of automating things. The goal is not to take as many things and make them LLM as possible for us.

07:23:29 The goal is Think about the function. It probably looks something like this. It's probably not just a linear statement of a bunch of different things. There's probably all kinds of branching logic and all kinds of ways to actually fork this. So maybe I want to collect the information and then translate it. And my main goal is to extract quotes and check those against sources to make sure that these are actually things that people said and and that's my approach.

07:24:02 But that might be totally wrong. Maybe actually what I want to do is I want to ddup and then I want to filter and then I want to pull external figures or maybe I want to do something totally different. I actually want to I'm passing in a note first and I actually just want to find other confirmatory data points or maybe I'm trying to red team. Uh there's a million different ways to configure this.

07:24:21 So yet again more hyperparameters within the control flow, right? So so many ways to think about just this very simple problem of polling, filtering, summarizing, aggregating the news for LLMs. And each of these boxes probably has an optimal runner, right? So u the optimal runner might be deterministic code for collection, right? I'm pulling last 24 hours for a given source.

07:24:53 That's deterministic code filtration decision models maybe LLM statistical methods a combination right so this probably understates the complexity the reality is most things are multi-runner most things have deterministic code plus LLM plus statistical method or human with LLM in conjunction u the reality is it's probably combinotaurics between my four buckets and there's even more buckets than that because within the LM category.

07:25:25 I can tune the model. I can choose big model, small model, medium model. I can tune thinking effort X high, high, medium. I can tune harness. I can choose the harness. I can choose which skills load with the harness. I can choose which hooks load with the skills. So again, more hyperparameters within my search space. I have an absolutely enormous amount of hyperparameters within my search space.

07:25:49 And this is just for this one tiny problem, right, of just filtering the news. So the way uh to think about breaking down these problems is what can I actually compartmentalize in a way that we can understand and evaluate because that's what I'm climbing against, right? Otherwise, I'm just tuning these things. I'm okay. Well, that should actually be a machine learning system instead of an LLM.

07:26:10 That should be a little train. Oh, no, that actually shouldn't be a model at all. That should be an if statement. Or, oh man, LM are really, really bad at translating that to portfolio impact. it should really be a human on that step, but they're using a deterministic system to help them search and pull the information. It could be all different things, right?

07:26:28 And if you don't have a north star to hill climb against, you're just kind of uh you're just fidgeting around. And I think I cannot convey enough that this is just one tiny thing we do. Imagine every single data source that we pull in, there's probably more than just three transformations to get them to the intermediate output. So the actual graph of code and funds like CO2, it actually probably looks oh not like this.

07:26:55 Uh this is my screen saver. The actual graph probably looks something like this, right? at least 20 different valid values for them. So the search space is is basically infinite and figuring out how to actually lay this out in a way that you can comprehend both from technical perspective but also absorb culturally organizationally that is the great challenge of applying AI in investing and beyond investing.

07:27:32 What we're really talking about is creating a state machine that can take information from the world and understand it well enough to make coherent decisions. There's use cases for this far and above uh wider than just investment firms in policy in nonprofits in drug discovery in corporate strategy. I think one of the most exciting things for me about being in forums like this is talking to other people in dis different spaces with different problems that kind of congeal on the same problem space.

07:28:02 I'd love to keep talking about this. I'll hang out in the hallway afterwards. Here are my socials. And thank you everyone for your time. And now let's take a look at frontier models built to act in space and time. Please welcome co-founder and CEO of General Intuition, Pim Deit. Hey Hello. Um, I want you to uh zoom out for a second and think about if somebody were to tell you today that we would take all the information on the internet and then we would train a model on it and we would use that in order to solve

07:29:53 scientific problems that were not on the internet. I think a lot of people would tell you that that's crazy. Um and uh what we're doing um has a lot of parallels to that story. Um and so if you think about how intelligence works, imagine for a second that you're in the movie Frozen and you're Olaf the the snowman. Um, if if Olaf was an LLM and it there there's a scene which is one of my favorites which is where he rolls down the mountain and there's a little stone that come that pops up and Olaf spreads out temporarily

07:30:26 becomes a disassembled snowman and then becomes an assembled snowman and Olaf knows how to do that because Olaf can observe as he is acting. Olaf can unlike LLMs which autogressively predict tokens um constantly part of that behavior is that it's constantly observing as it is acting and so um this is the big difference between you know when you see LM try to solve robotics problems or when you look at native architectures like what we're working on um it's very very easy to think of LLMs and the demos you see on

07:31:02 Twitter as the thing that will solve everything there's fundamental limitations that we where purpose-built models are a much better fit. Now, today I'm going to talk a little bit about what the other side of that coin looks like. Um, and so for us, it all started with metal. We built the largest video game recorder in the world. Um, and we think of that as the internet.

07:31:25 So, people use metal to record themselves playing tens of thousands of different video games. And if you think about it, uh, as a human, we can tell operate a robot with the exact same brain that we played that video game. And in fact, um, in many cases, we also can use the same controller. So what this means is that the action space of video games, often because of game controllers, and the action space of tele operations or the real world in many cases is the same.

07:31:57 Um, and so every single robot below um certain degrees of freedom ships with the game controller. A majority of the supply chains already work this way. And so we thought, what if you just take all of the action data that we get through the video recorder, um, which associates the inputs and the actions with the actual frames. We label the actions so we understand semantic meaning behind every action and we use it to give it to a vision-based model.

07:32:26 And so what you see here on the left is just an intuitive way of how the model works. So you give the model a set of keys to choose from. Essentially a computer use model. It it it gives frames as the input. So you can give it a text prompt as well. And you send it through API and you say so many ways that you wave back. In the input space model understands you have a greed action available.

07:32:47 It will output the action associated with that. Um one way of int of intuitively understanding how this works. So um I've obviously simplified here because it's predicting the next token not the next word but bear with me here. Um LMS took large scale internet data then trained it to predict the next text token and now use it for writing software. Um and these are all workloads that are really really well fit for futures that are not simultaneously unfolding as you are predicting.

07:33:17 Right? Code is turnbased. You can say something the model will do something. and you can say something again, the model will do something again. And um you can use it for lots of valuable work, but at the end of the day, because it's turnbased, um it's very very difficult for it to generalize to environments, especially that that include um space and time.

07:33:37 And so our models, the way that they work is they're continuously receiving frames. So receiving batches of essentially um information from the camera of the robot or it could be a screen and it predicts everything as if it were just a video game. So you can think of the way anthropic focused on just nailing coding. So you know they were never frontier and just base text generation.

07:34:06 They always focused on being the best at generating code. And then what the bet that they made is that more people would start bringing more problems into coding space. If you think of general intuition, our bet is that you can as humans we are generally pixel toaction converters plus senses which in many cases you can convert into pixels as well and you can output actions at the same time and those actions are in this case either a keyboard and mouse or a game controller.

07:34:32 So the output tokens of the model are um essentially equivalent to that of a enhanced computer use model with more more more actions on the output space to internalize it. So you send frames and you get actions. This works again because every single thing that you see here can control um all of these different embodiment, right? So um as humans we do not know how to predict motor torque.

07:35:00 We do not we do not know what value to send to a machine. Um there's this beautiful device that has been made for many years uh which actually does that work for us so we can control them. Most robotics companies today are trying to solve the problem at the endto-end level, trying to predict exact poses or exact um values uh in the motor. Uh instead of just looking and picking up the thing that every robotics company has built to generalize our very limited um abilities to have physics understanding and predict values

07:35:36 and solve for it. And so our approach is let's just use that. Um and so Metal at the moment sees about a billion videos uploaded per year. This year on we're on track for three billion. Um which lead to over a trillion and a half uh action tokens per month. So just for comparison to think about the bet the total amount of data that Frontier Labs use today to train robotics models, we collect in at the moment half a day.

07:36:07 Um, and so, uh, the the the type of bet and again, think of the fact that humans can use the exact same controllers and their eyes to play video games that they can't upgrade robots. It is the same thing. We just bet on scaling pixel space the same way that anthropic skilled coding space and betting that people will just as we scale data compute into pixel space that people will bring more problems into pixel space.

07:36:37 And so what happens? Um, all of a sudden you get an incredibly general agent that can go and solve problems on computers in the real world. Um, and in addition, similarly, Tanthropic focuses on doing coding environments. We focus on world models which most of you know us for the mirror release world models to then generate additional environments for these models to learn in.

07:37:06 One way of internalizing a big advantage of this approach against LMS is if you're an if you're an LLM company and you have to bet on the ability to generate environments for your models to learn, you have you have two problems. One, you actually need to spend the compute to simulate environments. And the more realistic you try to simulate, the more expensive that gets.

07:37:26 Um, and so imagine you have you want to learn a new thing. You want to learn that new thing in the exact input space of the human, right? So you if if we are learning from pixels, we're we're learning by looking around. If I'm trying to communicate something, I want my robot partner to observe that exact same thing. And so, um, what what I don't want is for that to have to go through an LM, for a digital twin to have to be created, for the model to have to get deployed, for there to be a bunch of verifier checks

07:37:58 whether or not things are equivalent, and then for the robot to to learn much later, right? And so, one way of thinking about it is when you can learn in the native modality of the human, you can all of the sudden start solving a lot more problems a lot faster. What you'll see when these models come out is they're significantly faster than any LLM um at doing these things.

07:38:16 We because games are so action dense. So what you see right is um one thing that's interesting is uh use soft for a model predicting multiple actions at the same time. Um we're able to uh take that action density from gamers and convert them into really complex robotics workloads. So for example, if you are operating a drone and you need to make lots of little adjustments, you really really really want to start with games, you don't want to start with uh tele operations because the human will just take a lot less

07:38:50 actions. It has a lot less information. So you want a model that can predict really high APM. Um and so the APM of the data set puts a lot of these problems all of the sudden in reach so that it can do a lot of these actions simultaneously. And as I mentioned, the model doesn't just understand the key it's predicting. It also understands the semantic meaning associated with that key it's predicting which is really important for things like um interpretability.

07:39:16 So uh we will be releasing the models this year. Um uh we are very happy. I think the correct word would be that we're we're stunned by the fact of what these models can do. Um it is uh this is a live stream so I unfortunately couldn't share the release videos without breaking the surprise but if you're in New York we are also in New York and so uh we have started doing a few demos uh of the model.

07:39:42 So if you can know somebody on the team reach out and we can start showing you some of the uh some of the capabilities live. Um but we have essentially one model which can play video games at the highest level and control over 10 types of robots uh using the exact same primitives. So pixels in actions out completely real time. Um and uh we'll be rolling out early access uh to these models over the next few months.

07:40:07 Um we already have some customers that are on them but it's still uh private so you're not hearing too much about it. But if you scan this QR code, you'll be able to get the models um because we go by who signs up on the weight lists at which times. So um this is a great this is a great moment to get in touch. As I mentioned, we are in New York. I think what you'll find is similar to how uh Jev and a lot of these decision-making models have kind of opened people's eyes on what you can do when you have a new

07:40:37 architectural primitive. People are ready to kind of do away with the unpredictability of an LLM and put something in that is more reliable and faster and you just understand how it's going to react as opposed to having to work around it. If that resonates, these models are exactly what you're looking for. Um and all of the sudden um you can start exploring the environments that you generate with LLMs.

07:41:00 All of the sudden one of the use cases that I'm really really excited about is well these models they um can take actions as if a human was on the interface and so if you're doing UX work and you're trying to understand is this an intuitive interface well this is a very interesting way of solving the problem. So all these problems that LLMs can't solve because they can't actually explore like a human, you can use these problems to start you can you can start using these models for.

07:41:25 So um the way that they're going to be released is we still expect you to use uh an orchestrator model. So you may use something like Gemini or Astra in order to do long-term planning and and everything. And you'll be able to plug these into your workloads. Um, and we're actually the models are so general. It is it is genuinely going to I think feel like the early LM days when you have a new toy and you just want to figure out what it does and all the things you can use it for.

07:41:54 I my reaction was like that when when I when I first started using the checkpoints and I think I think you'll have the same reaction. Um and I think a lot of you will just like with LLMs if you're a bit early and you start understanding these primitives I think you'll understand uh that you can even build lots of businesses on top of this right you know think about for instance um uh a sc a science lab right if you uh all of the sudden if you can reliably orchestrate robots in space and time and they can take text

07:42:27 instructions and do things all of the sudden um this It's a recursive problem, right? Robotics is interesting because one of the big problems of solving robotics is making more robots. And so there are all these businesses that because it's so bottlenecked by intelligence are going to emerge just from solving this one simple problem which is pixels to actions.

07:42:48 And I think if you're early on getting your hands on on these, it will have a similar effect on getting your your hands on early LLMs. Um I am most excited also uh one of the things that that we're working on which is interesting is if you have any friends or family who uh are um in some way um need accessibility help for using their computer because these models can use computers like a human.

07:43:15 All of the sudden, for instance, if you're if you're uh if you only can use one arm, um all of the sudden the model can allow you to play video games because you can you can use you can just control the mouse and the model is predicting the actions for your keyboard. And because the model also understands which actions you're predicting because that's the input, right?

07:43:34 It's it's observing as it's acting, it will actually just not predict the things that you're doing and it will just predict the keys. And so it has all these downstream effects of things that you can build on top that I'm an engineer. If you're interested in building in this space, uh, this is going to be the most interesting toy that comes out this year.

07:43:52 I uh, I I'm fairly certain. Um, and I also think a lot of the initial usage will sit in simulation. So if a lot of you are into generating environments with Astra or with Gemini, it's perfect. Now all of a sudden you can have something that can explore it and you don't have to do it yourself. Um, and it's perfect timing because of all the environment generation capabilities going going into these models.

07:44:15 Um, okay. So, really quickly before I wrap up, um, I want to talk about the team. Um, so we are based in New York City. The metal team and the general intuition team is here in New York City. So, if you're here, you should reach out to us and uh, uh, we can both show you the models. Uh, especially if you want to join a team. I think we'll we'll prioritize even higher.

07:44:36 uh but um uh we really are looking for um people who a want to understand these models as much as we do. So people if you're really really good at generating novel environments uh typically if you have some sort of research uh background but I don't have a research background so that's okay if you you know if you um if you come from an engineering background and you're very very happy to learn this is this is fine too.

07:45:00 Um, and ideally what we want is somebody who um, uh, has experience shipping uh, end to end uh, AI uh, products, but it doesn't have to be on the research side. So, we just want to kind of hire people that understand the entire thing. Um, let's see. Uh, we have three teams. So, one is world modeling, the other is the policy, so the agent, and the other one is working on um, essentially the the data generation and the environment generation.

07:45:29 So, these are the main teams. If you're an engineer, the best one to join is probably the environment generation team. Um, and then you sort of grow into uh getting access to there once. Um, yeah, we I very excited for November. I really hope that you'll be on the lookout. Um, as I mentioned, it will be either late November, early December is the current planning.

07:45:48 Um, one more thing, uh, we are going to organize three hackathons. Um, so the first is going to be in New York City. It will be mid November. Um and uh we were we will send out um if you sign up for the partner weight list, we'll send out some emails so that everybody can get access. Uh we hope to see everybody at the hackathons. I think it will be really the first time to get your hands on them the way that we're playing with them internally now.

07:46:12 And uh there will be one in New York City and one in London. So if you're in either of those cities or want to travel to either of those cities, uh those will be good uh times to meet the team. Um yeah, let's see. Um I have time for one or two questions and then uh I think we can we can wrap. See >> uh yeah how's it different from so Jeppa is doesn't train alongside the action labels.

07:46:46 So they bet on inferred action labels for the most part which I think like one other way of thinking about it is JPA I think in the limit is the more correct approach but it's just a lot less practical in the short term because you need to bet on a separate scaling law on the action inference itself. Um and we really like Jetpa as a concept. We think it's a you know we we use some of the things in the models um and uh but at the end of the day it's it's an approach not a product.

07:47:11 Um, and I think we'll, you know, borrow concepts both ways. Um, like one way of thinking about it is, um, because we have so much action data, let's say you're flying a plane and you're moving the rudder in the back, there's no way of being able to tell that from the video frames. It's it's very very difficult. So our mode on having the action data really transfers better to action sequences than it does to world models and to to generating the pixels.

07:47:43 And that's why we just went all in on training these general foundation models for action predictions. And we we use all sorts of techniques to do world models. You know, we our own we we really like other companies building them as well. Um so great one more and then I got to run I think. Great. Um, good. All right. Thank you everyone. Have a good one.

07:48:22 Ladies and gentlemen, for our final talk of the day, please welcome to the stage the CEO of Noose Research, Dylan Rolnik. Hey everyone. Uh, I'm Dylan Rolnik. I'm the CEO of News Research and today we're going to talk about building on foundations and platforms that you can actually own. There we go. All right. How do you make back a centillion dollar infrastructure and research bet?

07:49:13 Labs individually have spent hundreds of billions of dollars training and building ever more sophisticated intelligence system. And the industry as a whole has spent trillions perhaps. The problem is though that the models have begun to commoditize. You guys in this room who are hopefully practitioners of AI have known this intuitively. You switch among models readily.

07:49:44 You switch between Astra, Fable, Opus, Kimmy, whatever it may be without too much regard for who's on the back end or what it may be. You want to try the new model. We've seen this in data, too. There was some charts going around Twitter this week showing the average cost of a token going down while the total spent on tokens surges and the infrastructure costs associated with serving those tokens has stayed flat.

07:50:12 Meaning where there used to be margin in a very very performant model, it's now starting to narrow as we see competition and you know a baseline level of intelligence be available to people. Okay. So, if the models are starting to commoditize and you've spent hundreds of billions, trillions of dollars on them, you need to make that back. And you need to find a different way to do it.

07:50:40 And in my opinion, there are two pretty obvious ways to do it. One, you find a different layer of the stack to compete at. You go down to the infrastructure or you go up to the user. or two, you build God, and you build God faster than anyone else, and you monetize God. If you're in front of the user, you have an opportunity for stickiness. Stickiness means margin.

07:51:08 This is why the fight has largely shifted to the agent layer. Lately, we started talking, I heard somebody previously talk about 5% of, you know, a workforce engaging in agents. This is becoming more and more the zeitgeist of where the AI battle is taking place. Consumers though are fickle and they're terrible customers. Businesses on the other hand spend tons of money on tokens and they have economically productive use cases for those tokens.

07:51:38 So if you're going to be competing at the user layer and you're going to be bringing this fight somewhere else, you're going to be bringing it to businesses. The best thing to do is to get ingrained in a business because when you're in a business, it's hard to take you out, right? And if you think about your business 2 3 years into the future, how much of your workforce as measured by its output is agentic?

07:52:05 70% conservatively, probably more. And if 70% of your workforce is organized by somebody else who may or may not have adverse incentives to you, they can be organized into a strike. And so what you would do if you had to make back a trillion dollar investment was you would embed, you would use your asymmetric leverage and you would change the rules of the game or like I said, you would build God.

07:52:36 If you succeed at building God, what you would do is you would point God at the most lucrative problems in the world. The best shot at earning it back is not to sell it for an ever decreasing margin to others to solve these hard and lucrative problems. The best shot is to solve them yourselves. Labs think that markets are solvable problems and that's exactly how they view the financial industry as a problem to be solved and attractable problem at that perhaps not even a very hard one.

07:53:22 Right now though, they know, at least the smart ones know that they don't have enough context, knowhow or experience in the financial industry to be able to do that themselves. So they will partner, they hire investment banking analysts, they partner with some of the biggest firms on the globe and they start to build their foundation here. My belief is that they will eventually go to the source because like I said, that's where the money is and we're trying to make back a trillion dollars.

07:53:53 And so you might say this is nothing new. We have always been trying to trap people into these games and change the margin and once we have you and you're stuck, we are able to raise the prices. But the difference is that the stakes are much higher. We're talking about meaningful deca percentages of the Earth's economy. And you in this room need AI.

07:54:20 Everybody has seen what they can do, what these models can do, what agentic systems can do, and there's no going back. People in this room who are um making decisions, vice presidents and executives and engineers about what they're building on are going to have to make decisions now about where they grow roots around these tools and processes that will affect the future of these businesses.

07:54:46 And whatever you pick, you'll start to harden. You'll gain knowhow. It'll stop up the intelligence of the organiza organization itself. You will start to build processes and systems around it. I implore you, do not build on quicksound found quicksand foundations. Build on open foundations. Build on Hermes agent. So, I just spent a fair amount of time convincing you hopefully that the highest expected value move in AI right now is to either trap you and jack up the prices or replace you directly.

07:55:20 So, why am I standing here? Uh, we view the world very differently. And the first thing I hope you guys will learn about me and about news research is that we built this company primarily for moral and philosophical reasons. And a lot of times in business morality and incentives don't always align. And when that happens, it's probably easiest to change your morality.

07:55:49 But we think the world is changing and we think it's different. And for that reason, we think that the best course of action is one in which we are both aligned. When we built the company originally, we built it to combat a seemingly inexurable trend towards centralization and centralized control of perhaps the world's most int most most important technology and most important invention.

07:56:13 Noose research was built by and staffed by nerds. Nerds who love science fiction. And we've all seen what happens when a centralized entity gets control of godlike intelligence and uses that for exploitation. Skynet was, after all, a closed source company. The next thing that you need to know is that there's no longer really a strong moat in proprietary software.

07:56:37 The marginal cost to reproduce a piece of software has plummeted. A few months ago, uh, XAI red released Grockbots and we saw that and we thought, wow, that's an intuitive user interface. That's really nice. So we pointed Hermes Agent at it and we reproduced it in a weekend and released it as a feature within Hermes Agent open source of course and the game has changed.

07:57:02 It used to be that software was difficult to produce and it took coordinated teams of engineers long hours to piler open source and take libraries and combine them together into proprietary pieces of software and then sell that software as SAS. But now the opposite is true. Anything can be reverse engineered. And there are many, many motivated open source builders pointing their agents at these different softwares.

07:57:32 If the closed competitors do manage to make interesting features, open source will have it not long afterwards. The moes of yesterday were in proprietary IP. The moes of tomorrow are in network effects, reputation, and reliability. What's funny is that in a lot of ways the labs have always seemed to underestimate their own tools or perhaps they are underestimating the users of those tools and their genius and their motivation.

07:58:00 This happened and is happening on the model side and we'll talk about why that's important. But it turns out that models can fairly easily teach another model to be you know nearly as smart as the original model. And the advantage in a model is very rarely in the architecture or some novel system that was created in a proprietary way for that model.

07:58:23 In fact, it's usually in the quality of the data and the scale. And sorry, the uh the model started to re the model company started to realize that the you know scale was catching up and that you could create these things with supplementary mentary quality data by teaching one model from another. So they labeled the teaching process distillation and they labeled illicit distillation theft.

07:59:11 You cannot call that copyright infringement because copyright infringement would imply that you somehow took every living human being's written or creative work and then trained on top of it. They also labeled the process of they also trained the models to not assist in the process of training other models and they labeled that safety. The next thing that you need to know about agents and about the new world is that agents are like operating systems and they're like frameworks and they're a lot more like those than

07:59:51 they are like chat bots. And so if you continue to think about your agent as merely a personal assistant, you will forever live on the surface of what it means to engage in agentic systems. And so what do I mean by that? If you guys were here a little bit earlier, there was a presentation about a financial firm and how they think about AI and these different inputs that go into ultimately what becomes an investment decision.

08:00:18 Agents are really at their heart a system that combines determinism and non-determinism. If you remember from that talk, there are data sources, there are routine code, there are people's opinions, and there's judgment. And all these things come together to create what is the new application. The new application is no longer necessary that it is deterministic and solid and cold hard code.

08:00:42 It can inject intelligence, reasoning, judgment, all of these things together to combine to create the new application. And so as you at your company think about building these new applications, you should be building these systems on foundations that you own. You should be able to swap in and out the models that you want. You should be creating ever more sophisticated towers of functionality on top of these core primitives of determinism, non-determinism, code, data sources, judgment, reasoning, intellect, but you

08:01:27 should not be spending your entire lives being intelligence companies. What has made you successful this far? Most of you are most likely not agent companies. Most of you are not likely AI companies. You should be focused on the things that you do well. And when the state-of-the-art changes and it evolves and what it means to be an agentic system starts to shift, you should inherit that, not rebuild it and maintain it.

08:01:52 Remember, these are tools that should make you better. They are not dropins to replace you. Do you know how much energy it takes to create a human? And the beauty of the Hermes agent proposition is that you can run it however you want, wherever you want. Modify it as you see fit. If you're not having fun anymore, you can take your ball and you can go home.

08:02:13 The moat has changed. The moat is no longer entrapping you. The moat is in reputation. And that's what Hermes agent offers you. And so, what do I want out of all of this? Well, I want to build a standard. Hermes agent today is the most used and downloaded open source agent in the world. It is order of magnitude more downloaded than some of its closed competitors.

08:02:43 And I want everybody to be able to build on these open foundations because as long as one of us us is willing to open source these tools and these agentic systems, water will continue to flow downhill. and I want to be downhill so that I can work with you to provide support, safety, scalability to go along with the world's best open agent. I've seen where this goes and we've seen where the moes will lie.

08:03:06 Network effects, reliability, and reputation. People are going to continue to solve hard problems with Agentic Systems. People are going to solve lucrative problems with Agentic Systems. Somebody is going to earn back the trillion dollar investments that we've made. Make sure that it's you. Make sure that you are in control of your own destiny. You can have your cake and eat it, too.

08:03:32 You can have state-of-the-art features and reliability without sacrificing your autonomy. Build on something you own. Build with us. Thank you. And now for a few closing thoughts and remarks. Please welcome back to the stage our MC and the managing partner at 10X, Alex Lieberman. >> Okay, let's close this thing out. Look at the range we just had in the last 30 minutes.

08:04:11 We had Pim talking about world models going from pixels to actions. We had Dylan talking about building agents on owned versus rented land. And then we had Frank talking about how they think about leveraging applied AI as it relates to making decisions around investments. Um, first of all, let's just give a round of appl applause to all the speakers, to everyone who made the event possible, and all the sponsors today.

08:04:40 And then two final housekeeping items before we close out the day. First very important one, go speak to humans. That is housekeeping item one. There's been incredible content covered today. Again, that is surface area for you to be able to connect with others and live in Edge City by having intellectually stimulating conversations. Don't let the opportunity pass you by.

08:05:03 And then second, this is just one day of an amazing event. So tomorrow things kick off again right before 9:00 a.m. So make sure to get here bright and early. Thank you all for coming. Thanks for having me and see you tomorrow. Heat. Heat. Heat. Heat.