New ways of working with Codex: context via plugins/Appshots/computer history, fast iteration with Ultrafast and annotations, Codex Cloud V2 remote environments, shipping via ChatGPT Sites, code review, and delegation through Voice, Dots and parallel sub-agents.
Appshots is a screenshot-capture tool that embeds text in captures so application context, such as Slack threads and documents, can be passed directly to Codex.
ChatGPT Space is a collaborative rich-text workspace where agents and teammates co-edit plans, runbooks, and other documents using assistant instructions. It was presented as part of connected agent workflows involving context gathering, shared knowledge, and collaboration.
Codex CLI is a terminal harness for AI coding agents. It includes an agents view for switching between sessions, voice mode, and remote-control functionality that can expose a development machine for connected workflows.
Codex Cloud is a managed environment service for software repositories. It clones repositories into managed virtual machines, automatically configures dependencies and secrets, supports sharing environments across teams, and connects to private resources through OIDC or Tailscale.
Codex computer use is an AI-agent capability that lets Codex operate macOS in the background or through a Chrome extension. It can retrieve information and test applications such as the iOS Simulator; the video also demonstrates connected workflows in which Codex edits footage in DaVinci Resolve and returns a cut for review.
Codex remote control is a feature that connects a phone or another device to a machine running the Codex desktop app or command-line interface, allowing the machine to be controlled remotely through a ChatGPT subscription.
Codex Voice is a voice mode for Codex that acts as a hands-free control plane. It is used to manage threads, check status, retrieve data, and drive the application through spoken interaction.
Cutroom is an internal video app in which footage can be uploaded, commented on, and handed to a remote Codex agent. The agent edits the footage in DaVinci Resolve and returns a cut for review.
DaVinci Resolve is a professional video editing, color correction, visual effects, and audio post-production application developed by Blackmagic Design. It combines nonlinear editing, the DaVinci color grading tools, Fusion visual effects, and Fairlight audio tools in a single program. The software is available in a feature-limited free version and a paid Studio edition.
Google's Agents API defines and packages reusable AI agents with a base model, instructions, and a preconfigured environment. Agents run in managed cloud sandboxes rather than on the user's laptop; the environments can be shared with users or teams and accessed through an API call.
Tailscale is a networking service for creating private WireGuard-based networks between a user's devices and servers. It can provide remote access over SSH and route traffic through an authorized device configured as an exit node, such as a local Mac, so external websites see that device's network address rather than the server's data-center address. The project includes the open-source `tailscaled` daemon and `tailscale` command-line tool, with the daemon running on Linux, Windows, macOS, and to varying degrees on FreeBSD and OpenBSD. The repository also supplies code used by Tailscale's mobile applications; platform-specific GUI wrappers and hosted-service components are separate. Packages are provided for several distributions, and the project uses WireGuard with authentication features including 2FA, OAuth, and SSO.
Searchable transcript of New Ways of Working with Codex — Dominik Kundel & Charlie Guo, OpenAI — AI Engineer (58:03). Search for a phrase, then click its timestamp to jump straight to that moment in the video.
Captions sourced from the original video on YouTube, published by AI Engineer. The video, its captions and all related intellectual property remain the property of their respective owners; AINotes claims no ownership. Provided for research, accessibility and search — see the Transcript Notice and Copyright Policy.
00:12 All right. Thanks everybody so much for coming. I am Charlie. And we're both on the developer experience team at OpenAI. And today we wanted to talk to you guys about some new ways of working with Codex, right? I think it's been a little while since we've had an AIE. We've talked about some things in the past, but wanted to catch you up on all the new things we've shipped kind of over the last four or five months, I think, right?
00:36 But before we get there, you know, it is worth asking the question, is Codex dead? I think you if you are terminally online, you may have seen some hot takes around this question. And I think the answer is not only is Codex not dead, I think Codex is now available, you know, in more ways than ever, in more places than ever, with more features than ever, right?
01:02 And I would say that it's it's, you know, not not just that it's growing, but that it is evolving. I think one of the big, you know, changes of this year is that Codex kind of merged with ChatGPT work, right? And in doing so, you know, we tens of millions of more people have been able to leverage the harness that is Codex, kind of start working with long-running agents, right?
01:24 And really kind of feel the AGI in the work that they do every day. You can see here too the the date where we actually, you know, combined Codex and ChatGPT and just like what that did to to the number of folks using it. It's evolving into the cloud. I think, you know, not a lot of people know this, but the very first version of Codex was a cloud native, you know, agent that ran in its own environment.
01:47 It checked out your repo. It did all of its work remotely. And we've been, I think, slowly getting back to that place, right? it comes to making better tools for you to to remotely control Codex on your machines, right? As well as increasingly spin up more powerful cloud environments to keep doing that kind of work. And it's available across, you know, more services.
02:10 And we're going to talk about, you know, some of these in more depth today, but Codex as a harness is now what powers, you know, ChatGPT work, the CLI at ChatGPT and Slack, Dots, our new agents API, a lot of these things that you kind of know and love, it's available in a lot more places now. Like I said, back in July, you know, we had the IIE World's Fair and I was here talking about things that, you know, you could do with Codex.
02:40 In that in that workshop, we talked about here's kind of everything we've shipped between January and July, right? Since I'm back, I wanted to update this slide. Here's everything we've shipped between July and November, right? And I had to cull this list in order to make it fit on the screen because there were too many things to put in here. I think the team has been shipping at an unbelievable rate.
03:02 And, you know, we're going to cover like a lot of these features, right? But I think at a high level, I want to talk to you about kind of some of the themes that we're going to touch on, right? Obviously, we want to to talk about context and new and improved ways to bring the right context into your agent. We want to talk about iteration, how to keep working with long-running agents and and get to the right outcome because not everything is a cool one-shot demo that you post on Twitter, right?
03:27 We want to talk about working with Codex everywhere, both locally and in the cloud and remotely, how to ship, how to get from an idea to production and some of the new tools we have to make that happen, the the new things that we're seeing with delegation, working with sub-agents, working with Dots and across these different form factors, and then a quick speed round of tips, right?
03:48 To help get the most out of Codex for everyday work. So with that, I'll turn it over to Dom. Cool. Awesome. So the first thing we wanted to talk about is context, because obviously, um, the biggest thing for you to be able to get anything out of a coding agent is to have it have the right context in order to, um, understand not just what is it trying to do, but what is the goal that we're trying to fulfill.
04:11 And so, um, you know, the last two years we talked everything everyone talked about context engineering and like trying to perfectly combine all of the different things that have to go into a context. But I think we're increasingly getting into this world of what I call like vague prompting, uh, which is like you really want to be able to like converse with your coding agent the same way that you would do with a colleague or a friend who understands and has all of the context already.
04:35 Um, so you don't have to continuously dictate everything because like reality is nobody wants to work with a colleague like that who you have to like explicitly dictate every little thing at the point to which you can just do it yourself. Um, and the great thing is Codex is great at that context gathering. Um, when we originally built Codex, um, we had the problem that OpenAI builds on a giant mono repo, um, which means that like if you want to have, uh, be successful with working in that mono repo, Codex needs to
05:11 be really good and our models need to be really good at like navigating these giant repositories and finding like the proverbial like needle in the haystack. Um, and this context gathering capability spans beyond just, uh, you know, code. So on the one hand, you want to be able to make sure that all of the standards that, um, apply to your repository or in your repository I think we all do this at this point, whether it's through agents.md files or linters and other tools, like realistically, if you have a really
05:42 skilled engineer and you drop them in your codebase, they should be able to contribute without making a mistake. The same goes for agents. So if it's just things like, uh, coding standards, you should always put them in the actual repository. However, a lot of context lives outside of that repository. And this is where plugins come in. Um so plugins are uh bundles between uh MCP servers, um skills, and any additional resources that might be relevant um for that particular use case.
06:14 And so on the one hand, we have a plugin marketplace that we launched that has a wide series of plugins and we keep adding to them. You can submit yourself any plugins that you build, um, and they use remote MCPs for the majority of them to not just have a connection to, like, your one machine. But because, as Charlie said, the open source Codex harness powers all of these different surfaces at this point, you can use that same plugin connection across all of these surfaces.
06:45 So if you connect once, say on ChatGPT on the web, your like Slack plugin Codex on your dev box has access to those Slack messages the same way that like any other, um, uh, device that is, or like any other, um, interface that uses the Codex harness, like the @ChatGPT, uh, Slack tag for example, has access to those same resources. And so that means we can share that context across all of these surfaces.
07:17 Um you can also build these plugins and just share them within your company or um connect them or or publish an open source marketplace to GitHub so that other people can connect to it. And with that, you get to a much more natural conversation. This was like a message I sent to my Codex a while ago um where if you look at it, you might get a bit of context out of what I'm asking it to do, um but the reality is there's a lot of openness, right?
07:41 Like I didn't say like where in Slack I got tagged. I I'm like didn't say which Katya I was referring to, but my Codex has all of that context because it's connected to all of the different ways that I'm uh talking, right? So it has access to my Google Google Drive so it can access the spreadsheet that was um shared in that Slack message. It has access to Slack, obviously, to be able to pull that conversation, but it also has it also has access to other things that we need to build new features or similar things.
08:19 The other thing that allows you to do is like take your work way further, right? So we had a this was in June, but I still it's still one of my favorite examples where we had to update our documentation site on Sunday at midnight. I don't know about you all, but like I don't want to have to like watch that. So I told my Codex to go and I prepped with it the PR.
08:43 I was good. I asked it to continue to try to get like an approval on time and then keep an eye out on the Slack channel in case there's any last minute changes to this launch and otherwise launch it on time 20 minutes ahead of time. That's how long the deployment took at that time and then make sure it's live. And so 4 minutes after midnight on that day, it was like, hey, it 's live.
09:06 We're all good. And I didn't really have to get involved there. The funnier thing though was on Saturday, the day before, I didn't notice this until afterwards where my I got tagged about like the changes that had that were being done and where the preview URL was. And my Codex replied for me. And my colleague was like, looks good. So I didn't have to deal with anything there.
09:33 It just had all the contact. They could keep up with it and solve this because I empowered it to like be able to like shepherd that launch ahead, right? It didn't go and like have a conversation about something else, but it was able to like push this forward. One of the interesting things though with plugins is that we recently launched plugin extensions and plugin extensions gives you another way of dealing with context.
09:55 So so far we talked about having plugins as a way to give your agent context. But there's also a lot of different ways that you want to better interface with said context in the overall ChatGPT ecosystem. And so with plugin extensions, you can extend pretty much every part of the desktop app as well as ChatGPT online and on mobile. So you can add sidebar entries.
10:25 You can add a side panel. You can create a full-on full page view, but you can also change things like the file viewer and composer. So if you have things inside of your company that are very specific to your workflow and your ways of working, you can go and build a plugin that makes that more accessible for both you and your agent. And so let me give you a quick example of this.
10:49 So this is the desktop app. And one of the things that I built, so you can see we have all these different pages here. A lot of them are actually plugins that we built ourselves and are just like shipping as part of Codex. So we build on the same primitives. So you can see the sites plugin here, code review which Charlie is going to talk about, security cloud.
11:14 A lot of these things are plugins themselves. But I built a little plugin called design system. And so this is like now directly in the desktop app as a way for me to explore our company's design system. I can look at what kind of components are available and then work with them. And this is incredibly helpful because like I don't know about you, but like I I sometimes end up like having like a tap open without design system just to like get some inspiration of like what I'm trying to prompt the model to do, what
11:41 I want these things to look like. And so I can have this embedded directly in here. And then I can go in and I can change, you know, like maybe I want this sort of like drop down in my project. And I asked Codex, all of this was built with one prompt from Codex. I was like, here's our design system, build me a plugin that does this. And I have an add to composer button here.
12:03 And now I injected this context here that has, you can see here we switch to actions drop down. It has all of the context now that I don't have to type out. It Codex doesn't have to go and find it anymore. It's injected in there um all in a very like native way. The other thing that I can do with this though is I can modify the composer. Um so I can add uh mention the design system here and then it says tap to search.
12:29 So if I tap into it, now I see all of my um design system components again. And this can be anything, right? So if you if you install the Figma plugin for example, you can get autocomplete on um like the Figma files that you have or uh similarly you can like extend it to whatever internally you're using. Um so that's an example of extending context through plugins.
12:57 Um the other thing that I'm a big fan of is app shots. How many of you have used an app shot before? Okay, just a few. Highly underrated feature. Um if you're using Mac uh and you keep taking screenshots and sending them to your Codex, you should switch to app shots. So if you have the desktop app already installed, you can try this right now. Press command command um so both command keys left and right or if you're on Windows, it's alt on both sides and it will take a screenshot, but it will also take it might
13:32 ask you the first time for like some permissions that you have to grant. But it's going to take both a screenshot, but it's also going to take um additional context that is outside of that viewable port because if you're for example working on a website, right? Or you're looking at a website, there's a lot of things that are often outside of that screenshot or same on Slack or if you want to like screenshot a conversation, there's like more beyond the fold.
13:52 And so an app shot is going to take the accessibility tree information that is on that page and send that as well. So you can go in an app shot. Let me quickly show this to you. I did this earlier. Um um here. Um so this is it looks like a screenshot, but we can click here on view text. And you can see there's a lot of additional information, right?
14:18 Like variables, numbers, anything that is like part of that accessibility tree. And so it's going to give you way richer information to your Codex that you don't have to cover. The other part is computer use. So if there's no plugin you and you still need Codex to be able to access information, you can use computer use for this. On macOS, this can work entirely in the background.
14:41 So you don't it doesn't take over your your computer and you have to like sit away and like it's going to handle it. It actually has its own cursor so it can interact with it. This is great for just pulling information, but also testing applications. For example, in this case it's like interacting with the iOS simulator. The other part of this is you can install the Chrome extension.
15:04 So if you don't want to have computer use for your entire desktop, but you wanted to have contacts over your websites, you can then use the Chrome extension to pull in that context through that. And that works on Windows, Linux, and macOS all in the background. So this is great when like you want to continue to work on multiple tabs in your browser without Codex taking it over, but Codex having that same logged in context.
15:38 The other thing and this is unfortunately macOS only at the moment is computer history. This is one of those things that like you have to turn on and just wait and it just magically works. It's not the thing that like you turn on and you instantly see a benefit. But computer history basically keeps track of what things you are doing and learns based on that how you use certain tools, for example, because that is again important context.
16:01 So for example, like through computer use my Codex learned that whenever I work on a blog post, I draft those in a Google Doc first so that I can get them reviewed. If I changed that habit, I started actually drafting them now in like chat GPT space, which we'll talk about in a second. It learns about that context because it knows what apps I'm interacting with.
16:20 Um and then it can use that as memories inside uh Codex to be able to steer its behavior, use that as context. It can also do things like help you with what was that document that I was looking at earlier and it can actually go and pull that up for you. Um the way you turn it on is you just go into the computer history part in the settings, you turn it on but and then it starts creating you this history where you can see the things that it's been doing.
16:49 It uses uh for that a lot of the like event streams of your of your Mac rather than screenshots, which makes us way more token efficient, but also faster. Um and you can see it sort of like building that history of like what are the different apps that you are using. I know now though you're thinking what about safety. Um you have a lot of different controls for you to control the privacy aspect of this.
17:15 So on the one hand, you can pause computer history whenever you want to. Um you can also go retroactively and clear the most recent um history if you're like, oh shoot, I just started like looking at this document. I'm supposed to turn off computer history for this. Like let me go and clear the last like minute. Um and then you can exclude specific websites or apps entirely.
17:38 So like private mode um website stuff is already uh excluded, but if there's like specific websites that you don't want to be have as part of your computer history, you can. The last thing I want to talk about for context is um I think we all deal with this, but like context is not this one thing that you throw into your repo, right? Like your context is normally spread across multiple different surfaces, whether it's the PRD that your product manager wrote in a Google Doc or Notion, um it's like your
18:08 architecture diagrams that you've been like slowly moving into your GitHub repository, you know, sales sends you a sends you a Google Sheet with like a bunch of customer feedback, and you're trying to juggle all of these different things. Uh, it makes incredibly hard both for Codex to continuously stitch these things together. Um, but it also makes it hard for you to collaborate with others and Codex on these challenges.
18:31 Um, a good example of this is, you know, an a plan for a larger feature. You might ask Codex to write one for you, and then uh one of the things I think a lot of us have done is like go and like check it into the repo so that I can see it in a pull request um and review it there. This is a giant pain to like iterate on and get feedback on. So a lot of people start like copying and pasting the plan into a Google Doc or something so you can send it to their team, and then you like copy paste it back or you have to
19:05 like link your Google Doc. Um, but for that reason, for this like collaboration between agents and people, we added uh we launched ChatGPT Space. And so from here, I can just like ask Codex to go and turn the plan into a page, and it's going to go and create a page in ChatGPT Space uh that is fully editable in this rich text format. Um, and I can go from here and tag my ChatGPT.
19:32 So if I have like questions like, you know, do we have already some database structure that we should fit in here that was like missing? Uh, ChatGPT can work through that. But I can also go and tag Charlie, for example, and uh ask him for a question. If he wants to, he can tag in ChatGPT, he can tag in his dot. Um, and we can all kind of collaborate on this, and it's incredibly fascinating to see, you know, like ChatGPT having like a cursor pop up and start editing things, and you can even add rich functionality
19:59 like visualizations built in HTML objects or deal with one of the most painful things I think in this world, which is updating uh legacy information because how many times have you like written a document on like this is like your your runbook, for example, for on call? Like how many times do you write that and like, ah damn it, I forgot like update that we recently changed the command here.
20:22 Um, you can actually ask uh your agent to do that for you by giving assistant instructions that are both dealing with when an agent is going to modify a page in space or you if you turn on auto update uh for it to like have instructions on how to go and update these on a regular cadence. Um, so you can think of this as like agents.md for your individual pages.
20:44 It's incredibly helpful. The next thing I want to talk about is iteration. Um, so far we talked about context and sort of what instructions you have to give your agent to be able to work on a goal. Um, but there's a lot of times where we don't actually have all the context, right? The most classic example of this is you're trying to build a new feature or web app and like you don't have a design from your designer or something.
21:12 You're trying to kind of just like figure out which direction do I want to go, what fields write and iterate on that. One of the things that is incredibly helpful for that is Ultra Fast. Um, so we launched Ultra Fast for both uh GPT-6 Soul and GPT-6 uh GPT-6.1 Soul and GPT-6 Astra um in October at Dev Day and it's eight times faster um at just six times the cost.
21:30 But I think I want to I want to say one important thing. You don't always need Ultra Fast nor should you always use Ultra Fast. Um, I know it's incredibly tempting to just be like I want everything fast. Just give like we're living in an economy where like everything should be fast. Give it to me fast. The problem with this is that um a lot of times even if you use Ultra Fast, you're not going to materially see changes fast if for example your task requires a lot of validation and build time.
22:08 If you're building an iOS app or like you know a video game or something and like half your time or even longer is like spend on like running compile uh compilers or you know like running end-to-end tests and validations, it might not actually like you might not actually notice that much the speed up. What I love using it for though is to really be back in the locked in and in the zone.
22:25 Like, one of the things that I started to miss over time from like going from like, you know, writing code myself to having agents do a lot of the work is sort of like I kick off the task and I walk away and there isn't this sort of like creative process like where I'm collaborating and like trying out different ideas and quickly iterate on an idea.
22:49 And so this is where UltraFast is really good. Um the other thing before I jump into a demo that I want to show you is annotations. So annotations, if you've never used them in the desktop app, you have the in app browser and you can click the button at the top left to go and annotate different parts of the site, which means that you can give very specific targeted feedback.
23:12 But you can also use that little settings button to go and tweak things in the page live to get a better feeling for something. So let me show you both of these quickly in action. Um so here I have a little um site that we were building for a demo and I'm running a dev server. I think this is one of the first things that like if you're using UltraFast, if you don't have a hot reloading dev server, you're probably not actually benefiting from using UltraFast for these use cases.
23:43 So have a have a dev server running um and then if you're using UltraFast, just turn it down. Just go reasoning on like light anything above it um can work, but you're going to see like not sort of that magic moment that you get from UltraFast. So sometimes I turn turn it like if I know that something is like fairly complex or there's like nuance, I turn it to high, but most of the time I'm on light.
24:10 And so from here I can go in and I say like remove the all good music great company um header with the button. Takes me longer to write these things than um it takes to run for some of these things. Um the first task sometimes takes a second because it starts reading a couple of the files like orientate itself and again this is sort of where you're running into like the limitations of um you know like your file system faster than anything else.
24:43 Um make the other header full screen. And so what I often do when I'm doing these sort of things where I'm like iterating on UIs I often even just like started giving it very small tasks um and uh scroll through it. So like in this case actually let's just do horizontally full and uh rest normal. Um so like I can iterate on these things. I sometimes literally just like sit back on my chair and I'm just like using voice dictation.
25:17 I'm just like okay do this do this. Um add some background animation, make the LP um look more like real vinyl and spin. And you can see here all these changes like this like you know restoring took like five seconds. This took 10 seconds. And so like you can really quickly iterate on these things and get to a world that you're actually sort of like envisioning with um your stuff especially if you're unsure what you actually want to get to.
25:48 Like I think it would be fun if we can make it scratchable with the cursor. You know um it's an LP. Um but like right now I'm relying on like fairly simple instructions um but you there's a way for you to like speed up some of these things even more. Like here for example make all of these um more readable. Like I can give like very targeted uh feedback and like sometimes you see me like trying to queue up a bunch of these messages very quickly.
26:20 Um and it's hard for me to keep up because Astra is like kicking them out or in this case Sol is kicking them out so quickly. We can see here it's scratchable now. Um to give you a very concrete example, um where I recently used this was leading up to DevDay, um the night before the launch, um I was looking at some of our docs for ChatGPT space.
26:46 And I wanted to make this page that talks about the features a bit more engaging. Um and so all of these little features here, they're all HTML, they're not videos. Um and you can see here um I basically just started queuing up things and I was like touching my way to the solution that I wanted cuz I wasn't quite sure what I wanted. Um and so like every time I would do a turn, I'll get more clarity on like what I'm actually trying to get towards.
27:22 Uh the last thing on annotations is um if we go back here. Right now you can see every time I'm like hovering something, it tries to do this based on the DOM, um which is the thing that is powering every HTML site and gives it the structure. Um but there's moments where that's not actually what we want, right? Um and this is where we add in an annotations API.
27:40 You can just ask Codex to do it. It will know what what I'm talking about, but it allows you to select what is actually selectable on these pages. So for example, if you're working on a design system, maybe you want all of the individual components to be selectable, but you don't want it to have like the frame of your story book a story book selectable or you want to have different settings.
28:05 For example, for the um chromatic Game Boy launch that we had yesterday, we um for the mod retro plugin, we ended up um using the annotations API so that you can like mark up things inside the emulated game, but you can't like mark out the like actual emulator. All right. So we talked about context, we talked about iteration, but really one of the other magical parts, right, is long-running tasks.
28:31 We've went from last year being excited about, you know, seven hours of tasks running. Um, looks like I guess there's no more animation. Uh, we went from like seven uh hours of tasks running to like now being able to have tasks that run for hours or days. Um, and as a result, a lot of us, I know you've done it too, started buying like Mac Minis or like dusting off that old laptop and shoving it under your desk as like an additional dev box just so that you can like keep your laptop closed um or you start renting
29:08 random virtual private servers online just so you build your fleet of compute. This might be uh fun if you're like doing vibe coding, but like it's not really like sustainable if you're doing this on a regular basis for like a business, for example, because you have to deal with like setting up all these devices, deal with the updates, but you also have to like continuously scale them as you're running into like issues around, you know, saturating the amount of compute and tasks that you can run there or running
29:36 into disk space issues, etc. Um, uh and we really want you to be able to use Codex everywhere, right? Like Codex should just power wherever you want to work. And so that's why we put a lot of effort into like allowing you to go and close that laptop and still be happy with uh the tasks that are running. And one big part of that is revamping Codex Cloud.
30:01 So as Charlie said, we actually launched Codex Cloud last year as one of the first big offers around like May last year. So it's been a while. Um, but it took a while for models and harnesses to kind of like equalize into something where we can build a really engaging experience. And so at Dev Days here, we went and revamped Codex uh Cloud to be a new V2.
30:19 Um, I'm going to show you a couple of the things um on how to how you can use it. So um right now, so far we've been running all these tasks on my computer. Um, some of you might have gone into settings and um gone into connections and you connected remotely your other devices, right? So if you've never done this, you can connect um your phone, for example, to this Mac by like just connecting here and it connects through your ChatGPT subscription.
30:56 You can control other devices that have this installed and so you can like network them all together or if you either are using API keys for your authentication or um you anyways have all of the devices on the same network, you could also use SSH to connect to things. Um, but with Codeux Cloud, we can now go and um spin up new environments that uh inside directly from the uh composer.
31:20 So here you can see I already have a couple of environments and we'll take a look in a second of what an environment actually is. And these can be shared within a company if you're if you're working in a company or you can just create your own um and you start off by picking the GitHub repositories that you want to have access to. This is one of the first changes that we did is you can now select multiple repositories at the same time.
31:50 And from here, Codeux goes and sets up this environment for you. So an environment on the one hand, you can think about it, it's a it's a VM that uh clones your repository or in this case multiple repositories, but then Codeux also goes in and figures out what are the things that are necessary to actually successfully work on this repository, right?
32:07 Like what are the um you know API keys that I might need for this? What are the um uh dependencies that need to be installed? What what are the commands that I actually need to run to spin up a web uh a dev server or verify the tests running, etc. And so previously you had to do all of this manually and now Codeux is handling this for you. You can still modify all these things, but by default Codeux is just going to kick off a task, and it's going to start figuring out all of these nuances um for your environment,
32:45 and um call out different things. I'm going to while this is running, um I'm just going to show you an example here. Um so in this case, I kicked off the same thing, and you saw me pick two repositories. One was called Blossom Music, one was called Blossom Music uh mobile. And um if you've done any mobile development or if you've tried to build an app, you know that you cannot actually build an iOS app on a Linux machine.
33:06 And right now, all of these Linux VMs that we host, we're working on letting you be able to bring your own environments so that like you can have your own hardware that still benefits from codex cloud, but like you can use your own your own machines. So if you have some cloud max, you could use that. But in this case, you can see that it actually noticed that there was an iOS app as part of these repositories that you're cloning, and it called out that like it can't actually do anything like um you know spin up
33:33 the app, but it's still going to keep it around so that it can reference it. And which that's actually what I wanted it to be able to do so that we can still get an idea of like, oh, there was this new feature that we implemented on the mobile app, go and implement it on the web player as well. Um and then you can continuously give feedback or iterate on it.
33:53 If we go back to the other one, um it's beginning to work here on a install script. So this is what I was talking about on like if figuring out what are the dependencies that are necessary to go and run um this task. Um how do we actually spin up things? And all these things are important because the way we change this is that um these environments as they're being set up, you snapshotting and continuous updating to make them really fast.
34:27 Um so on the one hand, that means that uh we can quickly like speed them back up. If you're sending a new task and it's been like a couple of hours you haven't touched it, it should still feel very instant. Um on the other hand, it means that we can in the background on a regular cadence, like make sure all the dependencies are up to date, make sure that like the latest changes are being pulled um from let's say main.
34:48 Um, and once you have your environment, we go back here. Um, once you have your environment, you can go and modify this. Where's the button? Let's go to a different one. Um, here we go. You can edit that environment. Um, and you can control things like do you want to have web access? Are there any like secrets that you want to store? You can share these with your team or you can have personal ones.
35:17 So like they can be attached to your environment or you can say, hey, this is actually something where people have to put in their own key. Um, the other thing that I wanted to show here though is sometimes you have the situation where you have resources that are not accessible to the public internet, but you need cloud you need your environment to have access to them.
35:37 This is normally where cloud environments break for people. Um, in this case, we added support for both OIDC, if your cloud provider supports that, or Tailscale. So you can um set up Tailscale, you get an API key, you configure it, and from there um you can expose like you can expose HTTP services that your code your Codex cloud environment can still access through that Tailscale connection.
36:02 So you don't have to make them publicly accessible, um just accessible within your Tailscale network. Um, let's jump back to the slides. The last thing I want to call out because this is a common misconception, so I showed you how you can set up Codex remote as well, meaning on one of your own devices, if you want to be able to go and access it um access that device, for example, on your on your phone, you normally have to install the desktop app.
36:35 You can actually use the Codex CLI to do the same thing. So if you have the Codex CLI installed, for example, on a dev box, um then you're running on some cloud host, you can go and run Codex remote control start, and it spins up the app server, which is the Codex harness that is powering all these things so that you can then connect to it um and run it on your phone without having to install like the desktop app.
37:01 And with that, I'm going to hand it off to Charlie. Cool. Um so an incredible amount of stuff you can do with Codex, right? Whether locally or in the cloud, I want to talk about some new things that we've added to help you just get things out into the world and actually, you know, ship them. Um to start with, I want to talk about sites. Uh who is like used or made a ChatGPT site?
37:21 Cool. Um so I think this is one of actually the most powerful features of ChatGPT and Codex that folks don't know that much about necessarily. Um incredibly powerful uh both to like quickly iterate on content and to like share it with the world, right? Um as one example, uh this is a screenshot from an internal site that I use pretty much every day.
37:43 Uh Dom and I are on the developer experience team, and part of that is um making, you know, supporting launches, maybe that's writing documentation, giving talks, but also making launch videos. And after we do the filming, um we we use this site that we've built internally called Cut Room. Uh and what we do is essentially upload that footage or, you know, the videographer uploads that footage, um and then we have this UI where we can like look through the footage, leave comments, upload, you know, screen
38:08 recordings or B-roll. Um, and then when we submit these and we, we tell, you know, the website that this is ready for review, it hands all of this off to, like, a Codex agent running remotely that; that agent has access to Da Vinci Resolve, uh, to a bunch of, like, video editing skills, um, and it just like, autonomously go, autonomously goes and creates a new cut and then uploads it back and notifies us that, "Hey, we're ready to review the next version of this," right?
38:33 And we are like most of the way through I think like just uh automating out the video editing portion of a lot of our launch materials. Um I believe we had a number of like GPT-6 Astra, uh not the main video, but like a number of the sort of secondary videos that were just fully edited by Astra in this workflow. Um, think part of the reason that people underestimate sites a little bit is because they assume that it's, you know, basically a way to just public static content or host static content.
38:58 Um, and there's actually a lot more kind of under the hood that's that's happening, right? Um, you get like a full database service through D1, right? You get a full file and object storage service through R2. Um, you get through sign in with ChatGPT, uh which I'll go into a little bit, um this like full authentication and user management layer.
39:14 Um, and then, you know, because it's running with a lot of Codex harness tools, you get access to plugins and data. Um, obviously, you know, you can get analytics back out, you can store secrets, right? So it's a way to build like actually fully fleshed out web applications, not just hosting static content. Um, one of the I think the the best parts that I really appreciate about this is the sign in with ChatGPT feature.
39:39 Um, if you're a developer that's made your own like microsites, you have probably, you know, like a hundred times tried to just roll your own auth or use like a Ruby on Rails or a Django plugin to just try to like, you know, install user auth. But then very quickly you get into some like murkiness around, you know, what fields should be available, how do I think about permissions, teams, all this stuff.
39:57 Um, with sites you just get sign in with ChatGPT out of the box, and this just gives you a way to like immediately, you know, gate features to certain users um or just like hide the app entirely. Uh often we'll build, you know, different versions of sites for different audiences, and we can kind of direct who has access to what information and what features.
40:15 Um, and more recently, you know, we added the ability to use plugins to bring all of that goodness that comes with plugins into ChatGPT sites, right? So um you know, you can build applications that revolve around connecting people's Google Calendar. Um, we're actually working on an iteration of Cut Room where we can like book our filming space just by automatically like hooking up Google Calendar and letting people book times on a shared calendar.
40:39 Um, you can use Linear to track, you know, what uh tasks are open or like create tasks right and assign them to to the team. Um, there's a lot that you can start to remix when you have you know, sites and plugins together. Oh, and actually speaking of remixing, um, you can use remixing to like let other people uh, you know, create copies of your site and like tweak it as as necessary for their use case.
40:59 And I think when you put all these things together, um, you know, there's a lot of different ways that you can use sites, some that aren't immediately obvious, right? I think a big use case for me has been working with internal data and finding more interesting ways to present it, right? Um, I think across the company we have seen a lot of different use cases like this.
41:18 For example, you know, I think the Comms team did a launch recap where the entire theme was like eight-bit video game, right? And like they presented it in this way to make it more engaging. The engineering teams have done like recaps of our architecture merging like told through the lens of like scholastic, you know, book volumes, right? Um, the social team has done things where they've reviewed like social content that went out, but they built it as like this iPhone viewer, right?
41:42 And like you were able to scroll through and actually see analytics of the content as you were going. Um, and me for me personally, this is like way more interesting to to consume and look at than like a Google Doc or a PowerPoint um, that just kind of like you know, sits once and and is like static, right? Um, kind of on that last note, you know, like things that interesting things you can do with sites, um, just kind of on a whim, uh, I wanted to test out Astra's computer use capabilities.
42:06 So, if you've heard of this concept of wiki racing, um, this is an idea you know, for me this was back in high school you would pick two dramatically different Wikipedia pages, uh, and two people would try to race to see, like, how quickly they could navigate from one to the other. Um, and so I built a ChatGPT site, um, with an API key in the background that was using, you know, the agency API and computer use, uh, to try to, like, get Astra to race me in this case, from the Frida Kahlo Wikipedia page all the way
42:35 to the Rubik's Cube Wikipedia page, right? Um, and so I'm on the right here in the orange box and you can see um, I am like desperately trying to find a way to get to Rubik's Cube. I don't know. Unfortunately, I realized at this point in the race that like it was probably made um, by somebody from Hungary. And because Astra has picked up on that already, I didn't know that.
42:57 So I was trying to navigate to Christmas and then go into like a toys subcategory and figure out if I could get there, right? And again, like this, you know, not necessarily like driving shareholder value with this, but like this was just a really cool experiment for me to test the capabilities of the model and to just like make something that I wanted to share with with the rest of the team.
43:17 I don't know how much time we have, but like spoiler alert, Astra is going to beat me in this one. And I think it like in play testing, it is like pretty close to 50/50 on whether I'm able to beat the model, which is a little bit concerning. Cool. Another part of shipping, right, is obviously not just shipping your own code to production, but like reviewing code that other people have perhaps vibe coded and then thrown on your plate for a review, right?
43:44 And to help with this, we we have a new code review tab in Codex that let me see if I can find it. It's over here on the left, right, or you can find it kind of in here. And it pulls up, you know, automatically will pull up when you connect GitHub pull requests that that you've been either created or you've been assigned to. And this is a way to just like review the code quickly without having to leave the application.
44:10 You know, you can see in this case like the person who authored this or I guess this account authored this took screenshots so that they could actually verify that, you know, everything worked. Took mobile screenshots. If you want to dive into, you know, the the actual code, you can see all the diffs here. You know, obviously there's links back to GitHub or to the URL.
44:31 But one of my favorite features is just this like single button click, you know, review with Codex, right? And this is going to create a separate thread. We can get back to it, you know, in our main view, but it's just going to automatically kick off a review of the pull request. It has like a pre-made prompt which we can go in and customize. We can give it, you know, custom instructions, uh if we wanted to focus on certain things.
44:53 Um but that's going to go ahead and, you know, just start using like a sub-agent to to review um and then give us the results back in this chat. Um we will check back in on that in a little bit. Cool. And I think the last big section here um that I want to cover is about delegation, right? Because um you know, when we started the year, we were just kind of poking at sub-agents.
45:19 Um but increasingly there are like more ways to think about delegating within Codex, across Codex. Um I think starting with Codex Voice, right? Uh and this is a feature that we we shipped a few months ago. Um and I think there's a lot of misconceptions around it where most people I see that use voice uh with coding essentially treat it as like just a different medium on top of their normal work, right?
45:39 So like you might start a new thread and just sort of say, instead of typing, I'm just going to like dictate um out, you know, what I want and just going to go back and forth that way. And voice mode in Codex um can do that if that's what you want to do. Um but it's actually more powerful than that, right? And I like to use it as kind of this control plane across everything that is happening for me in the Codex app.
45:58 Um nearly anything that you can do as a human user in the Codex app voice mode can just do on your behalf, right? And so um I don't think about it as saying like here's the one lane of work that I'm doing right now with voice. I think about it as saying how do I manage my threads? You know, how do I check in on stuff? How do I pull data from other apps on my computer with computer use?
46:18 How do I create new threads in the application just using my voice, right? Um and hopefully uh we will see a demo that works here. I'm going to give uh Codex a quick call. Hey, can you hear me? Yes, loud and clear. We can't hear you. Testing? Oh, I think it's on the venue. Are you able to turn the sound on for the laptop? It is on. All right, we're going to uh laptop audio real quick.
47:05 Headphone port? That thing? your Mac back and Yeah, we'll do built-in. Can you say something? Yeah, hi. I'm here. Can you hear me now? Cool. Yeah. I just want to do some some work real quick. Can you uh do me a favor? Can you check on the poll review request that we kicked off? Um and then would love to know the latest from Matt in Slack. Um oh and then if it's possible we're giving a presentation to AIE.
47:31 Can you in a new thread pull up those slides and let's do a design critique? Um and then when you're done go ahead and pin that slide review thread uh and we'll we'll take a look at your notes. All right. I'll pull those together and keep it tight. I'm going to mute myself, but um you know in the in the background here Codex is uh you know working across those chats.
48:00 It's using all of the features we've been talking about today, right? The ability to drive computer use. Which Matt is Slack chat? Is it Matt Sullivan, Matt Weaver, Matthew Poschiasc, or Matthew Cordier? Also Thanks, Codex. Um so that's going to go and take a quick look, right? Um we we're running out of time so I don't know that we can wait for that all to finish.
48:27 Um but again like you know you you don't need to and you don't need to necessarily have the voice talk back either, right? I think one of the things that that I like to do is basically just like have voice mode on and then just have it control things as I go, not necessarily to to like give me notes back, but um to just act as a driver across applications, right?
48:47 Especially um like Dom was saying if you've got Ultra Fast on uh you can just sort of dictate to it and just like have you know changes come in and get delegated in the background. You can also use it to start to visualize things on your computer for you and just say hey pull this up, you know, move this around. You start to kind of feel a little bit like Iron Man, like working with Jarvis, right?
49:04 Just like having it control everything on your computer. Um next up is Dots, right? And like Dots is a relatively new thing and I think a relatively new way of delegating to AI, right? Or to an agent. Um and I think if I'm honest, right? Like as a Codex power user, Dots feel a little bit awkward to work with, right? And I think because they're a new paradigm and because they're different, they're good at different things than the main Codex is.
49:29 So I think there's three kind of ways that I think about using Dots that's a little bit different from Codex that that Codex is maybe is a little bit trickier to do. The first is using Dots as a safety net, right? These are some real examples of like things that Dom's Dot and my Dot have like flagged for us, right? It grabbed that like there was a bug in a in a PR that we were going to ship.
49:49 It told Dom Dom that like there was a merge conflict. And in one case, it told you like, hey, Charlie has this other PR and you were in this channel that he's not in and you should go tell him that like this thing he 's working on is out of date. It let me know today that like my my speaker hold overlapped with a one-on-one that I had to go cancel.
50:11 And I think it told you one point there was like a fire in the Oakland or Berkeley Hills, right? And it said, hey, like, you know, I know you live in the East Bay, this might affect you on your commute home, right? And like these are things that just your normal Codex are not going to do for you. Another way to use Dots is like as a delegation sink, right?
50:28 So like, I use Dots a ton in Slack, and because we have it connected there and I, m just constantly forwarding messages to it in Slack and just telling it, "Go look into this, go create a to-do item for this," like treating it like a true, you know, virtual coworker or direct report as I send it stuff throughout the course of the day. And I think the third way that Dots are a little bit different is, you know, we're we're aiming to make them truly everywhere that you work, right?
50:53 Obviously in the ChatGPT apps and in Slack and Teams, but also like on the phone, right? The ability to call your Dot, the ability to text your Dot like iMessage or or RCS, right? We wanted to be in as many places as possible. And the last way to think about delegation here, right, is, you know, inside of your own products. We have the agents API, which is currently in beta, and it's a way to take all of these goodies in the Codex harness, right, that power you know, the desktop app that power dots and build them
51:23 into your own application. So if you want the ability to create, you know, hundreds, thousands of ephemeral sessions with the same access to sandboxes and plugins and everything else, we can do that now through the agents API. Cool. We got a few minutes left, so I'm going to turn it back over to Dom for some quick speed round tips. Cool. So a couple of different things that we were like we felt like we wanted to share with you all, but didn't fit into a sort of these things.
51:51 The first thing like turn it down. Like you don't have to like I know it's very enticing to just slide the slider all the way to the right, especially if there's like pretty effects. But the reality is for most of the things that you're doing, you know, high reasoning is often a good starting point. Also as new model generations come out, it's a good idea that like if you were like stuck on like extra high before and a new model comes out, like dial it down slightly to a high and see if that fares as well for you.
52:22 Like these these models become increasingly better, but you're not necessarily getting more out of it if you just crank it to the highest tier, especially as there can be diminishing returns or things like where it's overthinking problems. And if you are really sort of like struggling with like what model do I pick, what reasoning effort do I pick, one of the things you can try is like asking your agent for help.
52:50 So, for example, you can have use Sol on medium for the majority of the things that you're doing and then tell it to, like, spin up Astra sub agents on high if it needs help with, like, something that it can't figure out, or spin up a bunch of Luna sub agents on high for, like, more tedious work um uh, that it can, it can delegate more easily. Um there is going to be I'm going to share with you in a second like the full prompt for this um but you can just set up a little skill for this to like tell it to do these
53:23 things on Soul Medium. Uh fair warning this doesn't always mean it's always better or always cheaper. Models still have a hard time to perfectly figure out what is the right use case for for which things but it takes at least a little bit that pressure away from you of like having to figure out what is the right combination. Um another interesting way of sub agents working together is sort of through this like pair um uh of sub agents.
53:50 So like, there's tasks where you might um be able to work on different things in parallel, but there are still somewhat, somewhat interdependent on it. And like, rather than you having to like go and first split things up ahead of time um, and then like do or like have one, one agent create like a giant plan and everything perfect, and then paralyze implementation.
54:13 You can spin up these pairs of agents. So a good use case is let's say you you're working on a new app and there is a web app and a mobile app for it. You might have one agent work on the web app which uh involves both the front end and back end logic and then the mobile app work uh another is assigned to another agent. In those cases you want to have them still collaborate because maybe some back end changes are relevant for the mobile app or the mobile app made some decisions that require back end changes.
54:40 And so you can have it spin up a new um uh new uh set of peer sub agents essentially that coordinate with each other. This is a bit more um tricky in the sense that like you need to have Codex figure out like where are the communication barriers between these but then the models learn learned how to communicate with each other so they can like uh keep each other up to date.
55:04 So if you take a screenshot of that and you ask your agent to like find some find some good use cases for this setup, it will probably find some. Um Charlie briefly covered this, but I think not enough people talk about it. You can have your Codex spin up other chats as well. So, rather than sub agents which are tied to a main session, these can be completely independent.
55:26 You can go steer them and you can have sort of a thread of threads where like you spin up all of your different agents or you can have your dot for example spin up threads for you. Um if you are a big fan of the CLI, we recently updated that. So, you have now an agents view in there as well. So, you can just press the left arrow key and you can switch between those uh sessions.
55:46 It's also helpful if you just like have to SSH into your dev box and you quickly want to switch between the tasks that are running there. Um similarly, we added voice mode to the CLI as well. So, you if you love the voice mode and all the things that Charlie was showing you, you can do the same thing there. Um not enough people know this, um but you can just set up more you can set up shortcuts for pretty much anything in the app which makes it much easier for you to like really tweak your um use cases.
56:14 But you can also set up global dictation and uh a global voice hotkey so that like you don't have to switch into the app if you want to either start voice mode or if you want to use dictation in other apps without having to install a third party app. The last thing um is go and apply for daybreak access. So, daybreak is our like cyber security um program.
56:38 If you apply for daybreak blue which any individual can do, you don't have to be part of a company. You just need a hardware uh key like a YubiKey and we have a discount for that. Um you can apply for that and then you can use it for um securing your own applications. So, definitely do that. Um the last thing, don't get too attached to how you're using things.
57:01 Um things are constantly changing, new models come out, new capabilities. And every time a new model comes out, consider like abandoning the things that you've been doing so far. Try out if it works better without it, and then bring back the things where you're still running into issues. With that, scan that QR code if you want to have a chance to like win a Codex Micro, which is a cool keyboard that helps you really speed up your Codex workflows.
57:24 Um but also that link has all of the details that we covered as well as additional resources. And because we're out of time, um we're going to be right outside this room. Um if you want to ask us any questions, we're happy to chat with you. And uh with that, thank you so much for joining us.