← All transcripts

Why We Rebuilt Microsoft Office From Scratch for AI Agents - Yijia Liang, Compound AI Transcript, AI Summary & Key Points

AI Engineer · 11 days ago · Science & Technology · 15:45 · EN

Watch on YouTube

AI Summary

Compound rebuilt Microsoft Office from scratch — spending 1.5 trillion tokens and over 8 million lines of code run by looping agents — to build autonomous AI analysts for institutional investors. Personal agents took off because of reliable background execution, proactivity and integrations, but finance breaks those: workflows run through Outlook, data rooms hold 15,000+ files, and managing directors don't want rogue agents burning tokens. Compound's approach is multiplayer agents with built-in email inboxes that join deal-team threads, a headless Office engine (Excel, PowerPoint, Word in Rust) that runs work in parallel and recalcultes 20 MB Excel files 4x faster, and agent swarms that read data rooms with grep instead of vector DB indexes, grounding every finding in citations.

Key Points

  • Personal agents took off in the last 3 months for three reasons: reliable background agents, proactivity (acting without being asked), and integrations with email, Drive and Calendar.
  • Those three factors break in finance: the buy-side analog of booking a restaurant is going from a data room to full IC models, slides and memos — a 4-week process to compress to days; managing directors don't want rogue agents burning tokens; and the relevant context is data rooms of 15,000 files and SharePoint/Box stores with petabytes of a firm's 40-year history.
  • Every Compound agent comes with a built-in inbox, because institutional finance workflows run through Outlook with email chains of 20 or more messages that the agent should join.
  • Agent email design is non-trivial: it requires authenticating read/write access for everyone on the thread under NDAs, and baking in latency so multiple people emailing in quick succession don't trigger a spam of acknowledgements.
  • Compound evaluated desktop Excel/PowerPoint/Word plugins, computer use on virtual machines, and off-the-shelf engines like OpenPyXL — none scaled — so it rebuilt Office from scratch, running agent loops with 1.5 trillion input-output tokens and over 8 million lines of code against Office as an oracle.
  • Excel: started from the open-source Iron Calc, 800,000 lines of Rust later produced a replica more performant than Excel; the same looping method covers PowerPoint and Word, with pixel-perfect Rust renders so files look identical when downloaded to Office.
  • A headless engine beats desktop tools: desktop Excel is bound to one CPU, while headless runs tasks in parallel and recalculates 20 MB Excel files 4x faster.
  • Owning the editor enables comment-driven drafting: the talk's slide deck was built in Compound by dropping comments on a first draft; a team of five can rip through a 100-page deck in an afternoon instead of weeks of revision.

AI in practice

Used for

Agents

  • Compound multiplayer analyst agents — Act as real teammates on deal teams for institutional investors — join email threads, draft IC models, slides, and memos from data rooms. 2 held 02:29

Tools & resources

7 items

CNo. 0016
AIAINotes.us AI product

ChatGPT

chatgpt.com

ChatGPT is a general-purpose AI assistant that can generate movie ideas and expand them into story and storyboard prompts. It can also produce software projects from natural-language prompts, such as a Python dating application with user profiles and swiping logic, and help explain electronics and hardware-development concepts. It is also cited as a general-purpose AI capability that consumer products can package into more personal, character-based experiences.

Mentioned in
119 videos
Kind
AI
CNo. 7126
AIAINotes.us AI product

Compound

getcompound.ai

Compound is an AI platform for buy-side finance teams that provides long-running, proactive deal-agent tasks. Its agents analyze data rooms, filings, and uploaded documents, then produce financial models, memos, and PowerPoint presentations with citations tracing figures to source files, pages, lines, model cells, or filing passages. Agents run tasks asynchronously and in parallel, including across thousands of files, with multiple agents and subagents able to work concurrently. Compound V2 provides multiplayer agents that teammates can share and direct through comments on workbooks and decks, along with browser-based Excel, PowerPoint, and Word editors built for agent-driven work, including formula-compatible spreadsheets and versioned changes. It supports PDFs, Excel files, CSV and TSV data, Word documents, PowerPoint files, images, and other common formats, and orchestrates models from multiple providers for different tasks. The service states that customer data is not used for training by default, is encrypted with AES-256 at rest and in transit, can be deployed in a VPC, and is SOC 2 Type II compliant.

Mentioned in
3 videos
Kind
AI
INo. 7127
AIAINotes.us Tool

Iron Calc

In the AINotes directory

Iron Calc is an open-source spreadsheet engine. It was used as the starting codebase for agents that iterated on a headless spreadsheet implementation to recreate Excel-style functionality.

Mentioned in
1 video
Kind
Other
MNo. 2114
AIAINotes.us Tool

Microsoft Outlook

outlook.com

Microsoft Outlook is an email and calendar client developed by Microsoft for consumer and enterprise use. It is available as a desktop application for Windows and macOS, mobile apps for iOS and Android, and a web app at outlook.office.com. Outlook connects to Exchange Server and Microsoft 365 accounts and supports IMAP/POP/SMTP, calendar scheduling, contact management, rules and add-ins; features include Focused Inbox, Teams and meeting integration, and support for PST/OST data files and enterprise deployment policies. It is distributed as part of Microsoft 365 subscriptions, as a standalone product, and as the free Outlook.com web service for consumers, including use for supplier confirmations and procurement updates.

Mentioned in
4 videos
Kind
Other
MNo. 0077
AIAINotes.us Tool

Microsoft SharePoint

microsoft.com

SharePoint is a web-based collaboration and content management platform developed by Microsoft. It provides intranet portals, document management, file storage, team sites and integration with Microsoft 365; it is available as a cloud service (SharePoint Online) and as an on-premises product (SharePoint Server).

Mentioned in
3 videos
Kind
Other
ONo. 7128
AIAINotes.us Tool

openpyxl

In the AINotes directory

openpyxl is a Python library for working with Excel files. The video describes it as an off-the-shelf Excel tool that was evaluated and rejected because it does not render spreadsheets for formatting checks.

Mentioned in
1 video
Kind
Other
RNo. 0706
AIAINotes.us Tool

Rust

Open source · rust-lang/rust

Rust is a systems programming language and toolchain developed by the Rust project. Its compiler, standard library, documentation, package manager and build tool Cargo, formatter rustfmt, linter Clippy, and editor support are maintained in the main repository. The language uses a rich type system and ownership model to enforce memory and thread safety at compile time, while targeting fast, memory-efficient software for critical services, embedded devices, and integration with other languages. Rust is primarily distributed under the MIT and Apache 2.0 licenses, with portions under BSD-like licenses.

Rust
Stars
★ 121,571
Forks
17,368

Links mentioned

🔒 Full analysis locked

Unlock more videos and the full analysis

A credit unlocks one video's full analysis for good — the build steps, the tools and how each was used, the methods behind every use case. Pro opens the whole library instead, and raises how many videos you can analyse a day.

Unlock full analysis — free

Transcript

Searchable transcript of Why We Rebuilt Microsoft Office From Scratch for AI Agents - Yijia Liang, Compound AI — AI Engineer (15:45). Search for a phrase, then click its timestamp to jump straight to that moment in the video.

Captions sourced from the original video on YouTube, published by AI Engineer. The video, its captions and all related intellectual property remain the property of their respective owners; AINotes claims no ownership. Provided for research, accessibility and search — see the Transcript Notice and Copyright Policy.

00:01 [music] I was previously an analyst in a previous life. I was a vice president at Coatue for about 3 years specializing in late stage private software investments. Spent way too much time in Excel, PowerPoint, and late nights doing that type of work and so decided to split and start building tools to replace my old job. Quick show of hands, how many people in here have used Muser Instinct in the last day?

00:36 All right, I expected more hands, but uh maybe people are too shy to raise them. And how many people know an analyst in finance that fully delegates their work to AI, like fully hands-off running autonomously? All right, one. But a lot fewer. I think if you think about the question, why did personal agents take off within the last 3 months? Internally, we came up with three main answers.

01:00 The first, reliable background agents. You can delegate a task, book a restaurant, book a flight, and it'll do it reliably now. Computer use agents have gotten sophisticated. The models themselves have gotten better. The second is productivity. It'll do things without you asking. It's part of that magical experience when Instinct wakes you up saying it's your wife's anniversary, go buy some flowers.

01:21 Or better yet, I already bought the flowers for you. And finally, integrations. It's connected to the systems of context that matter most for you. Email, Drive, Calendar. But all the factors that make personal agents work so well doesn't really translate to institutional finance today. If we think about background tasks something as simple as booking a restaurant reservation the analog equivalent in buy-side finance might be: "How do you go from a data room to a fully prepared set of IC models, slides, and memos,

01:57 taking a process that would normally take 4 weeks, multiple rounds of iteration, and review, and compressing that to a few days?" If we think about proactivity, your managing director doesn't really want agent rogue running around doing a bunch of things and burning tokens without them providing and soliciting input. And then finally, if you think about the context that matters, it's not just email, it's not just calendar, it could be a data room with 15,000 files.

02:24 It could be SharePoint box with petabytes of unstructured data a firm has collected in its 40-year history. So, how do you deal with these problems? So, the purpose of today's talk is to discuss a bit more about how we at Compound have approached the problem of solving long-running agents in finance. On one hand, it's creating a real teammate that collaborates with you in the deal team.

02:45 So, that's multiplayer embedding that into the app. On the other hand, it's solving for how you can you run long tasks in parallel in the background, reliably execute them. And so, for that, we actually rebuilt Office from scratch. And then finally, it's how do you manage the context of everything, everything we just discussed, petabytes of data in SharePoint in box.

03:05 And we'll talk a little bit about how we've designed a solution for that. So, the first question we have to ask, I think, is why does multiplayer actually matter for finance? If you look at the anatomy on a deal team, right? The type of tasks that matter for long-running type of workflows for example, going from a data room into all the sorts of materials that make it its way in front of an investment committee it doesn't work for the single-player, siloed workflows.

03:33 Today, for people who have deployed Claude or ChatGPT in their institutions, it's all single player. People working in a plugin, people working in the web app, producing outputs, but invariably those people need to collaborate with their team. And what does that look like? It means downloading the file, uploading it into email, uploading it to Teams, asking for feedback.

03:55 I think the right way to approach this, and this is the way we've thought about at Compound, is how do you get every person who matters on a deal onto the same thread? How can they all talk to the same agent so that one agent can share the context of everyone who matters for any particular work stream? When we thought about how do we actually bring multiplayer into a familiar context, we looked again towards personal agents.

04:16 Why did they work so well? Well, you can easily text one in iMessage or you can text one in WhatsApp. So for agents at Compound, every single one you create comes with a built-in inbox. Because ultimately at the end of the day for institutional finance, most workflows run through Outlook. You'll have chains of emails 20 or more running. You want an agent to be part of every single one of those conversations.

04:40 Designing email for agents is actually non-trivial. It's not just as simple as hooking it up with an email inbox. It requires A, authenticating, making sure everyone part of that thread has read and write access to that agent. Ultimately at these financial firms, privacy is incredibly important. You're always under NDA no matter what deal you look at.

04:59 And so you want to make sure everyone on that thread can actually talk to the agent. But then beyond authentication, you also have to think about how do you design a real human-like experience? Because after all, we're trying to create a real teammate. If multiple people email in the same thread in quick succession, you don't want multiple acknowledgements spamming that group.

05:16 You actually want to bake in some more latency to make it feel more human. And so there are other design decisions when thinking about how to create real email agents for enterprise finance. But multiplayer aside, we realized even after you solicit the input of everyone who matters, if you wanted to scale up the number of tasks and run them all in parallel, existing off-the-shelf solutions simply don't cut it.

05:39 We evaluated a lot. The desktop plugins of Excel, PowerPoint, and Word. We evaluated computer use using virtual machines to try to scale up the number of agents that could work in Excel, PowerPoint, and Word natively. And we also evaluated a ton of off-the-shelf engines, whether those are open source packages like Open PyXl or commercial engines that are available for purchase.

06:01 None of them really cut it for what we were trying to look at. And with coding agents dramatically improving and bring the cost of software down, we decided to just build our own office from scratch. And like a great AI engineer would, we ran agents in many loops, spent 1.5 trillion input-output tokens and over 8 million lines of code to rewrite office from scratch.

06:23 And I'll walk through a little bit of how we approached it. But it's as simple as pointing agents at an objective function, which is recreate office in full fidelity, and then letting it run. So, take Excel as an example. We started with an open-source solution called Iron Calc. We decided, "Okay, how can you recreate this perfectly for Excel, use office as an oracle, and just run loops?"

06:45 Benchmarking it on real files from both real-world corpus that we grabbed online, and also ones that internally we created. And 800,000 lines of Rust later, we had a performant much more performant compared to Excel, one-for-one replica of the product. With this, we're actually able to do that for not only Excel, but also PowerPoint and Word. Right?

07:07 So, we ran loops across every single one of these products, and we have faithful recreations. You can try it for yourself today at the compound.ai. And it's the same method, loop agents against office as a judge. Using Rust renders to make sure that the rendering of these products was pixel perfect. So, what you see in Compound is what you see when you download to office.

07:29 We needed one-for-one parity because at the end of the day, these firms only trust what they can download and they what they can send. Now, let's talk about performance. Why is a headless engine so much better compared to using a VM or using desktop Excel? Well, for one, desktop Excel or PowerPoint or Word is bound to that CPU. With headless engines, you can do tasks in parallel much, much faster.

07:52 You can build out a complex model in a much shorter amount of time. You can recalculate 20 MB Excels 4x faster. So, every single piece of work of an analyst would expect an agent to do, cuz again, their expectation is quality of the human at much lower speed and much lower cost. You have to have a headless engine to do this. If you try to scale this up with Excel and PowerPoint, I think you'd be going down the wrong path.

08:21 But, it's not just about performance. Owning the editor actually allows you to create a much richer product experience that allows you to work and collaborate with agents asynchronously much better than you can with other products. So, this entire slide deck you're seeing today, I built in Compound just by dropping in comments. I had it create a first draft, and then as you can see here on the UI, I dropped in comments for every single piece of formatting.

08:44 From my consulting days, from my Coatue days, all the learnings I had on what makes a great slide, I imparted with the agent. This can also work for multiplayer. You can have multiple people part of the same team using the same multiplayer infrastructure we set up, dropping in comments. So, for a team of five, you can rip through a 100-page deck in an afternoon rather than sitting in a room revising it for multiple weeks.

09:08 Finally, with respect to building your own engine, it also allows us to run iterative checking loops much more as well. So, if you have a task that, for example, is writing a complex model, it might take a few hours. But, how do you How can we make that faster? How can we make it more accurate? If you're using, for example, an OpenPyXL code library, there's no render you can use to check formatting.

09:29 You're just guessing and hoping that the model writes the right thing, and maybe you do some parsing to see a cell has a particular color or has a particular font. If you try to use Microsoft's own interface, you're going to be limited by what Microsoft provides you, which is admittedly very little. So, with an own engine, you actually own the right, render, check, and fix loop.

09:49 So, you can rapidly iterate on the output for the user. So, the last thing I sort of want to discuss today beyond multiplayer and building our own office is how to deal with the context that these firms have. For firms on the buy side, they deal with situations where cross is both breadth and depth of data. You may have situations where you have a data room of over 15,000 files, and then files in that data room might be well over 20 megabytes.

10:18 People in finance complain all the time, "Hey, I have an Excel model that's 30 megabytes. It takes 5 minutes to recalculate because we have 2 million formulas and five sensitivity tables that need to recalculate every single time." So, how do you deal with this? Well, for breadth, I think again we take a look at a page out of the coding book and apply agent swarms to the large number of files.

10:39 No longer do you have to index everything in a vector DB unless it's for it's for a very specific use case. It's actually much preferable to use coding agents to read files either through grep or through some other methodology, and then have a lead agent slice the contents of the file directory, and then have swarms of agents come back to you with insights.

10:58 And then you iterate that in a loop until you're satisfied with the output required. Furthermore, through this process you're actually able to ground every single finding inside citations in the source document, so that you know exactly what you show on the screen is linked to some number that the analyst can trust. If we think about the depth of data, right?

11:19 Take that 30 megabyte workbook. How can you actually process this more accurately? In today's world, if you were to just use ChatGPT or Claude off the shelf to process that 30 megabyte model, it would write code, it would use open-source packages like openpyxl. It would be expensive and slow because it's printing out every single piece of context about that model in code for the model to read.

11:42 But because we've developed our own engine for Excel, we're actually not only more accurate, but a lot faster and cheaper because the engine processes it and then lays it out in a very easy-to-read and easy-to-process format for the model. This means we can expose a much richer set of APIs and a much richer set of tools for the agent to work over. And that allows us to process these models in a much faster manner.

12:08 So, if we sort of take a step back and think about, "Okay, what is all of this context built for?" At the end of the day, what we want to do is prevent people from spending time finding files, from reminding the model this is how to do things. Instead, they should be focused on importing their judgment on the deal process. They should be focused on tailoring, training agents, and deploying them on every part of the deal flow.

12:30 And so, it goes from the bottom level, connected sources, off-the-shelf, you know, connectors to SharePoint or Outlook, to the cited findings grounded in the sources that matter. And then finally, embedding in the preferences, company knowledge that matters. How do you do it, model? How should it look? Which differs by every single firm. So, that for humans, they spend most of their time imparting judgment.

12:51 They spend most of their time thinking about is this actually good deal? Not did all the numbers tie? Does the model look right? Will my VP yell at me because it looks different way? So, Oh, cursor needs to update. If we take a step back and think about the type of tasks that currently are on autopilot um versus what are on co-pilot, I would say there's already several finance tasks you can do pretty well with off-the-shelf tools.

13:22 Spreading comps, as the ChatGPT head of finance mentioned earlier, they have now their own products for this. But I think there are a lot of longer running tasks that are not capable with existing solutions. How do you draft a 50-page IC presentation off the cuff? How do you get agents to build out a model exactly the way you want it for a private equity professional or a hedge fund professional?

13:45 I think those are tasks that with off-the-shelf tools still exist as co-as sort of within the co-pilot realm. People want to be embedded deeply in that process without letting go. But with some of the tools we've built today, I think we're able to expand the aperture for which autopilot can take over in finance firms. So, rather than just using tools embedded in Excel as plugins, hand-holding along the way, you can start delegating.

14:11 You can start becoming more ambitious about what AI is capable of. If I think about where we are today in the world of finance, it very much reminds me of when Opus 4.5 launched last December, where people stopped using code as a simple co-pilot to generate the next file or the next line of code, but they started delegating far more ambitious tasks.

14:34 They started realizing, maybe I can just truly vibe it. If I have sufficient checks, if I have sufficient tests, maybe I can just trust all the code it writes. I think with some of the tooling we've built today, along with what we're planning in the future, these are the type of problems you have to solve before you get to that point for finance. How can analysts, how can the entire deal team become more ambitious with AI?

14:57 How can they just drop in a data room and decide, okay, I'm going to trust every single one of the outputs, and then let's do that tenfold so we can see more deals, so we can expand the aperture of what matters. So, to conclude, enterprise agents are certainly coming to finance. I strongly believe we're at an inflection point in the adoption curve, and that people are slowly becoming more ambitious, and they're moving away from co-pilots to auto-pilots.

15:21 And so, this was sort of how we've approached it here at Compound. If you work at a firm and you're interested in deploying our agents to drive more of your deal work, please reach out. Or if you're engineer and you're curious about talking about our approach and sharing ideas, please either reach out online or I'll be around if you want to find me. Thanks so much.