An AI Future Without the Lock-In — Remy Guercio, Tailscale Transcript, AI Summary & Key Points

00:01 [music] Well, hi folks. Uh, thanks for joining me here for the last talk on this stage uh of the uh AI engineer world's fair this year. So, um, yeah, thank you for joining me. My name is Remy. I am on the strategic projects team at Tailscale. Uh and more specifically I actually work on our AI gateway aperture and today I want to talk a little bit about you know what it means or you know what it's going to take uh to have an AI future without the lockin.

00:42 I think a lot of the locking that we've seen uh over the last 12 months and yeah so if we take a look real quick at the you know the last 12 months you know I think it can really be you know described quite a bit by uh the phrase token maxing and I I don't mean that in the kind of you know negative sense that you might kind of hear outside of the you know kind of AI enthusiast circle.

01:03 I mean that in the sort of you know it's been a year of trying to figure out you know what we can do with million token context windows uh with the ability to get context from anywhere right not just you know MCP being everywhere but you know also having coding tools right that are great at finding context using the CLI bringing that context into the model uh bringing it you know bringing it into you know everything that you need and and and going from there right it's it's been pretty cool right I mean we've gotten

01:34 It's really made truly agentic coding possible having all of this right. So yeah, it's I mean it's it's quite cool and you know we're going to see uh not just agentic coding but you know other agentic use cases continue to take off uh as we as we do this. So um you know token maxing is cool but right it has it has problems right there's obvious problems right it's very expensive right it's it's it's an extremely expensive endeavor uh you know I think the last 12 months are you know have maybe been marked by hey we have

02:05 you know unlimited AI budget uh and that's changed here in the last few months uh in terms of you know what it what it means to you know what it means to use AI um it also I mean you know it's expensive like just generally if you're doing it, you know, right, but it also it does incentivize what I would call like wasted tokens by default. I mean, I I work on an AI gateway.

02:26 We use our AI gateway. I get to see all of the usage from the AI gateway. You would not believe the number of sessions that are hundreds or thousands of messages long where nobody clears the context. They just kind of keep going. I mean, it's helpful sometimes, but you know, it's it's it's pretty surprising. you will have you know tens of millions of tokens of a session uh where it's the cash is basically maxed out maxed out the entire time uh it gets really expensive um and the other thing is right so you know this

02:56 has been an exciting time right people are doing new things building new things uh but this has led to you know in many cases very vertically integrated tool chains uh cloud code is obviously you know a big one codeex right everybody's kind of building up their you know for all the way from the model you know all the way all the way up to the top uh you know building their uh you know building their agents, building their tool chains.

03:15 So um all right there's a lot of problems there and so you know my question right in terms of this is uh you know how do we address these problems in particular how do we address the lock in and you know how do we actually improve uh how we use AI such that you know we can use it for things that aren't just you know kind of essentially frontier task or where we're just kind of wasting wasting money wasting tokens uh you know that sort of thing.

03:38 Um, but first I want to talk about how not to address the problem. Uh, and this is a super super common thing. So, uh, over the last 6 months, we've been talking to I've literally been talking to hundreds and hundreds of folks about their AI spend, their AI usage, right? That's the kind of benefit of working on an AI gateway. Folks come to you, they're like, I don't know what's going on.

04:00 I know we're using a lot of AI. I get, you know, uh, I get an anthropic bill every day. Uh, I get a codeex bill every day. Um uh so you know I it's not just that you know I see it all the time. So you know what do I do? They usually come with one kind of solution in mind uh and that is oh we're just going to like uh everybody seems to be using cloud code right now.

04:19 So we're just going to like consolidate on a single vendor maybe two vendors right maybe two vendors. So that's uh they're like I need to get a handle on this. How do I get a handle on this? How I get you know oh okay yeah I will say everybody's going to use this right we're just going to use this thing. um that is not not a great idea. uh you know I might seem obvious sitting here it's maybe not a great idea but you know when you're in that panic zone of we've spent a lot of money and we don't know how exactly uh and

04:49 we you know finance is talking to you it is being like what are you doing uh um you know like we said to you know use AI but maybe not use that much AI uh and you know it it's a pretty natural response to be okay I'm going to consolidate everything down right okay I'm gonna have a control on it I'm gonna have you know I'm gonna have control on this uh control of this Um and you know I think this is kind of peeking at what we're going to see the next 12 months and what people are actually trying to do uh at least at the

05:13 you know kind of frontier is uh what I would like to call ROI maxing right we're not going to be token maxing we're going to be ROI maxing uh and you know it's it's really just a necessity right it's not uh um and I don't mean that when I say ROI maxing I don't just mean cutting costs right I mean actually proving some sort of value uh to the thing that you're doing uh and not only that but also kind of opening things up, right?

05:39 So if the only thing you have is like two frontier models and their entire stacked ecosystem, that's a certain cost level where you you it just makes things certain things untenable, right? You can't actually expand AI usage into the company if that's the only way you can use AI. Um and so you know when we think about ROI maxing, we first need to kind of think about like what are the components of an internal AI deployment.

06:02 I'm not talking about like a hey, we've built this new AI feature in our app. I mean, you know, internal internal to your organization. And so, you know, there's four things, right? There's the LLM or LLMs, right? There's that layer. Uh there's sort of the data connector layer. I mean, this is MCP for a lot of people, but in reality, it's, you know, CLI or APIs or really just any way that you get data could be reading a file, right, into your uh into your agentic system.

06:23 Uh there's the interface. I think people kind of forget about this sometimes, but you know, it's started with the chat interface. We got two, you know, we kind of got a uh we got, you know, idees with chat. We got, you know, kind of a whiplash back from two into, you know, I'm going to do it on my go to my phone. Uh, you know, to, oh, now the, you know, the hot thing is, you know, claude tag and like I'm going to use, you know, I'm going to use we're just going to interface with agents via Slack, right?

06:50 So, uh, um, you know, it's another core component and it is a component that you can break out. Uh and then the last one is sandboxes just the environment in which the agent runs right so those are your four kind of uh you know four components really uh and to be ROI maxing uh means to be maxing your you know every opportunity that you have uh for choice to make a choice at one of those layers right you don't have to necessarily have a unique thing at you know oh I've made my bespoke special choice at every one of

07:20 those layers but it's really important to be able to make a choice at any given layer that you need at any given time. Uh, and it might be different for different teams or different folks, right? It's, you know, it can be pretty different across the organization. I don't totally imagine like an entire engineering team just coding via, you know, slack thread.

07:39 Might be fun, but I don't totally imagine that, right? Uh, um, but, you know, the entire marketing team might be working entirely uh, you know, in a slack thread, asking data questions, doing other things like that. So it's really important to you know to be able to uh kind of separate these things and right so I I work at tail scale right I work on an AI gateway uh you know maybe no surprise that I'm here to talk a little bit about why an AI gateway uh will help you do that can help you do that and I don't just mean

08:02 necessarily using our AI gateway I think it's nice but you know just any AI gateway is the um you know is important for this uh so our g AI gateway is uh called aperture and um what it does uh is right it's maybe No surprise here, right? But it let you it sits in between your agents and that interface of that agent uh and you know all of your endpoints uh as well both like LLM endpoints and data you know kind of MCP endpoints API endpoints anything that you have uh and sits in the middle and let you both control and

08:33 monitor what's happening from a that can be from a like security perspective that can be from a cost perspective that can be from a which we do a lot I want to learn about what somebody is doing or what an agent is doing uh perspective. It's actually really cool when you have logs uh and you can tell your agent to hey uh I it maybe it ran out. It kind of just kind of you know oh it it didn't reach an answer.

08:58 It just kind of spiraled out of control. Uh maybe happens less now. Definitely used to happen quite a bit. Happens with you know maybe cheaper models more often too. Uh they're usually not great at stopping themselves. Uh but they are great at analyzing where they went wrong after the fact. And so when you have something like an AI gateway, it's really easy to point it back at what you were doing and say, "What happened?

09:16 Why did this not why did this not, you know, work the way I was expecting it to?" Uh, and so, you know, if you've used an AI gateway, how many people like have an AI gateway, have deployed an AI gateway, use an AI? Yeah. So, if you've used one before, you're probably used to, okay, so I've put all my keys on this gateway. Now, I need to go create synthetic keys for everything that I have, right?

09:37 So, it's like, okay, I've traded some keys for a bunch of other keys. uh we get to do something a little special. So Aperture is built on tail scale and if you're not familiar with tail scale, tail scale is an identity based mesh network. It's built on wire guard. Uh and what you get to do with tailscale is uh every connection that is made over a tail.

10:00 So it's a tail scale network uh you know you install the tail net agent everywhere that you need. It can be containers, laptops, GPU servers, doesn't matter. um you install it everywhere and every connection that's now made over that tail net uh comes with the identity of the machine or the user on the other side. So that can be a tag, right? So that can be like hey this is a machine ID.

10:20 This is a PR review bot, right? Uh and then it would get a separate node ID underneath. This is the instantiation of that PR review bot, right? So you can do that or it can be the user, right? So it's just me using cloud code on my laptop. Uh that can come with it as well. uh in addition to if you're using something like Entra or Octa uh we can actually sync all the groups in uh so you can see the group that that person is associated with.

10:41 So something on the other side is able to see all of that. We have this really cool library. It's called TSNet. It's an open source library. You can use it to build these sorts of apps. That is what we use to build aperture. That is what is actually happening here. So it is able to see the uh it is able to see the identity of that user or that agent on the other side of the thing on the other side of the connection.

11:02 And so you know again just reminder components of a you know an internal you know AI deployment uh right so there's the LLM side of things there's the data side of things the interface side of things and the actual sandboxes uh sandbox and environment in which things run uh side of things. So, um, when it comes to using Aperture, uh, you know, as far as LLMs go, this is pretty similar, I think, to, you know, what you might expect from other other AI gateways, except now, you know, here I have a, you know, a configured

11:31 gateway. Uh, it's got anthropic, open eye, versel, you know, it's got even anthropic oath, uh, you know, bedrock codex, everything, uh, that you can imagine. I put all those keys on that gateway and then now when I want to actually connect to that LLM provider as an agent or a user there's no I don't have there's if I had this expanded there's there's no extra screen for create these API keys create these identities create this once I connect to the once I actually connect to the gateway it knows who I am it knows what

12:00 I am and it's like oh you have access to this models this provider right you can use GLM 5.2 uh you get this budget with GLM 5.2 too, right? You get unlimited budget with a cheap model and you get an extremely narrow budget with a Frontier model, right? Those sorts of things. Uh there's no extra setting here. It's just add your add your API key and go.

12:18 So again, we've taken all the LLM layer and we've put it into a you know, put it into one place that you can then, you know, kind of expand out within your organization. Again, doesn't matter if you're trying to switch models or providers, you can do that. Uh the next thing is the data side of things. So, uh, in addition to, you know, being an an LLM gateway, and this is actually why I said AI gateway at the beginning.

12:40 So, I know that seems kind of nebulous, uh, but I'm actually I very specifically chose that term because we're both an LLM gateway and an MCP gateway and we even do off like, uh, injection for just standard APIs and reexpose those as MCPs, too. So, uh, you know, kind of intentionally choosing the broader term here. Um, but with this, as you can see here, uh, in Aperture as an admin, you can go in and configure whatever MCP servers you want.

13:07 We have kind of built-in ones where we've, you know, checked them out, make sure they're easy to easy to work with, all those sort of things. If you've ever deployed an MCP or used MCPS within your organization, you'll know that, uh, there's a standard, but everybody follows a different version of that standard. Uh, you know, you can tell exactly when an MCP server was built by exactly how they decided to do off.

13:27 Uh yeah, it's it's a great uh you know kind of archaeology lesson uh when you go through when you go through setting up MCP servers, but we try to make that a very simple process here, right? We've kind of gone through this. Uh so you as an admin can say, "Hey, I've configured these. These are the ones that are allowed." You can use that same identity information to say, "Hey, the marketing team, you know, they get access to Google Calendar and Gmail, but the engineering team, they'll also get access to GitHub, right?

13:50 uh you can even say things like oh the engineering you know this side of the engineering team gets access right access to GitHub or readonly access to GitHub you can do all all sorts of things here and what's really nice and again talking about decoupling is all of the authentication happens here so if you have folks that want to use cursor or they want to use cloud code or they want to use codeex or they want to use some special agent framework that they're you know some new hot thing that they've you know they've

14:18 discovered or built or whatever uh all they have to do is hit that endpoint and if they're on the tail net everything's authenticated as them or as whatever the agent or as whatever you know the tag device is. Uh there's no like oh I have to go reth into everything and then reauth into everything and then reth into this new tool. Uh we make the switching cost right of like actually trying out tools and trying different workflows and different experiences a lot a lot easier with this sort of thing.

14:43 It's all those little points of friction uh that lead to use cases where like you know people just don't try new use cases. They don't try to uh you know oh maybe I should experiment maybe I can make this cheaper maybe I can make this faster. Um you know next thing talking about the interface so uh with Aperture we actually have a built-in chat UI surprise uh you know everybody seems to have a chat UI we have a built built-in chat UI um however it is built on top of APIs that you can also leverage yourself.

15:11 Uh so our whole goal with Aperture is to actually provide you with the same sort of you know tools and interfaces and or tools to build interfaces that we can. So whether or not you want to build a Slackbot or you want to build uh chat UI or you want to build you know some new thing that nobody's thought of some mobile app kind of thing you know it's voice only interface right uh you can you can do that.

15:31 The whole point is you're going to be able to do that with Aperture. We want to support all of those different individual uh use cases. But again you can see it has connectors. Those are all again set up through Aperture itself. They're not set up in the chat UI. You're just getting them uh by way by way of Aperture. Uh and then the last thing, so this is uh you know, so we talked about interfaces, you know, a little bit.

15:54 It's, you know, it's again, it's very important uh that you be able to kind of use all of that. Uh actually one other thing that's coming very soon here is um uh if you've ever I don't know if anybody's ever tried to set up like open web UI or you know or like Libra like or any of the like kind of like any of the like oh I'm going to set up like a hey this like you know this interface or this interface uh one big thing is you always lose identity information and then suddenly all of the you know everything's coming

16:18 from the UI right everything's it's like oh everything's just coming from the system uh we're actually going to be adding a full like pass through O uh so you'll be able to actually see all of the like user identities and machine identities if for some reason they're using whatever interface that you have. Uh you should be able to set it up so that you can pass through the O and you still get all of the same visibility that you're kind of expecting from a from an AI gateway as well.

16:39 Um and then yeah the last thing is right this is actually the I think the area in which um folks are doing pretty good. There's actually a lot of choice already people are trying to do this. Uh I think that's because uh the definition of a sandbox I'm sure I could poll everybody here and would have a different everybody would have a different definition of exactly what a sandbox is uh or what they need from a sandbox or what they want from a sandbox.

17:02 Uh and so you know there's we're kind of in that let a thousand flowers bloom phase of you know there's a sandbox provider for every sort of whim and need and you know and want out there. Um but uh you know and folks are doing a pretty good job of integrating them but again it's a really important thing. We're actually going to be adding kind of an orchestration layer as well for sandboxes to a gateway.

17:22 uh again trying to use that term AI gateway very loosely intentionally very loosely because in order to again kind of make the best choices for all of the different endpoints that you're you know environments LLMs data connectors all that sort of thing uh you really do need uh to kind of have a place to sort of see them all uh and then a place that brings them all together such that you can then use any other one so right kind of a almost a universal translation layer in a way um but yeah so that's that's mostly that

17:50 and you know like I said is a reminder here uh is you know ROI maxing really is all about maximizing choice uh to prevent lock in right so you know even if you don't use Aperture I do highly recommend you start considering an AI gateway you look at an AI gateway it's good just beyond like actual you know oh you know I don't have to you know uh have three different providers configured or switch between them this way you know uh it's good beyond that right it's great for just you know understanding what agents are doing

18:19 even if it's just you as an individual we have so many people who or just as an individual they have their usage behind an AI gateway just to have an idea as to what their agents are doing because you know if you're running uh very commonly uh is people are running something like openclaw with on tail scale and it's on their tail net and they and it's just running in the background and they're like oh my god it did something I'm not sure what it did it did something it was not supposed to do with something like an AI

18:45 gateway you can actually go back and you can see and you can look and you can actually diagnose where it went wrong what happened Um so you know it's it's not just lock in right it's actually just experimentation and understanding as a whole uh is you know is very important to use something uh like a like an AI or LLM gateway. So um that's it uh for my talk just short and sweet uh thank you very much for attending I this is the the last talk in here of the day so uh I hope everybody's had a really great conference I

19:14 know I love the AI engineer world fair uh and I hope you do too. So thank you