← All transcripts

The Future of Web Apps: Will AI Agents Generate UIs on the Fly? Transcript, AI Summary & Key Points

InfoQ · 9 days ago · Science & Technology · 47:07 · EN

Watch on YouTube

Answer

Yes — the expectation is that AI agents will generate UIs on the fly, via standards like A2UI where an API sends the agent a JSON schema and the agent assembles a matching widget from components; how much of this becomes viable depends on investment in open-source protocols and standards.

AI Summary

Rachel Lee Nabors, former React core team member, argues that small language models running on-device beat cloud LLM wrappers for most everyday tasks — for cost, energy, latency and privacy — and walks through how to pick and evaluate them with eval suites and golden datasets. MIMA (mima.social), a local-first client that aggregates social networks into one interface with local models for summarization and moderation, is the working example. The conversation then projects the future of app architecture: personal OS-level agents will become the main consumption interface, platforms like Amazon will fight to keep their ad-bearing surface area (as in Amazon's suit against Perplexity over the Comet agentic browser), dynamic UI standards like A2UI will let agents generate interfaces from JSON schemas on the fly, and B2B companies that invest in open standards will keep the interoperability moat while B2C businesses are reduced to APIs plus fulfillment. Engineers should shift from producing UI by hand toward computer science fundamentals, database and ML understanding, and collaborative work with agents — the boot-camp era of learning to just build things is over, since anyone can create things with a 30-minute YouTube video.

Key Points

  • MIMA (mima.social) is a local-first client that pulls social connections from X, LinkedIn, BlueSky and Mastodon into one place, forms a local graph of connections, and lets the user reply and repost across networks — with no servers holding personal data.
  • LLM wrappers send user data to someone else's server, cost more with every call, and risk PII leaking; running a multimodal model for inference uses roughly four times the energy needed for most of the work it could do with smaller models.
  • Small models now handle summarization and tool-choice tasks; JEPA is highlighted for picking the right tool or a simple yes/no answer.
  • Llama 3.2B with post-processing matches Claude Sonnet at summarizing conversation threads locally — a decision made via evals, not reputation; Gemma 4 would have increased latency and produced worse summaries.
  • Clients should moderate incoming content themselves rather than relying on platform owners like Facebook and LinkedIn, who treat moderation as a cost center and cheap out.
  • All local models in MIMA are opt-in: a user turns on a feature and downloads the additional model; nothing is installed by default.
  • Evals are test-driven development for AI: assemble real threads, note expected summaries, prototype with big models (Astra, Claude Opus 5.5), then line up tiny candidates (Qwen, Gemma, Llama 3.2B), run each model/input coupling roughly three times, and judge accuracy versus latency yourself or with a large model as a judge.
  • Golden datasets — including fuzzed conversations for a toxicity detector — become part of CI/CD, so a new model release just gets dropped into the eval suite to see if it beats the current one on speed versus accuracy.

AI in practice

Agents

  • null — Prediction: OS-level personal agents will become the primary surface for consuming content and buying things; an API can send the assistant an A2UI JSON schema and the assistant renders a widget on the fly using its own design system and componentized UI. 2 held

Business ideas

A client app that aggregates feeds from every social network the user belongs to (X, LinkedIn, Bluesky, Mastodon), forms a local social graph of connections and posts, allows cross-posting and replying from one place, and uses small language models running on-device for summarization and moderation. No servers hold user data, eliminating both privacy exposure and API/LLM round-trip costs.

For
People who must maintain presence on several networks (e.g. tech workers) but cannot check each app daily; initially engineers on modern, capable hardware.
Solves
Each social network is a walled bubble that demands all your attention, so you only hear from one bubble and miss whole communities. Existing AI assistants and clients can't cross-pollinate the personal data locked in each network.
  • MIMA (mima.social) — the creator's own local-first social aggregator with on-device models, privacy-first and local-first.
🔒  Build steps and tools for 1 idea. Unlock

Tools & resources

11 items

ANo. 6403
AIAINotes.us AI product

Arize Phoenix

phoenix.arize.com

Arize Phoenix is an open-source AI observability and evaluation platform developed by Arize AI for tracing, evaluating, and debugging AI agents and LLM applications. It collects traces of LLM requests and the surrounding application execution, displays them in a trace UI, and supports evaluation storage, datasets, experiments, annotations, and result comparisons. Phoenix can use OpenInference telemetry to record execution paths across multiple application steps, helping identify failures in retrieval, tool execution, or model responses; it also supports code evaluations and LLM-as-a-judge evaluations. It can run locally or be self-hosted in another environment.

Mentioned in
3 videos
Kind
AI
DNo. 4586
AIAINotes.us Tool

DuckDB

duckdb.org

DuckDB is an in-process SQL OLAP database management system for analytical workloads. It uses a PostgreSQL-inspired SQL dialect, can run from edge devices to servers, and reads and writes formats including CSV, JSON, Parquet, Iceberg, and Vortex both locally and in object storage. Extensions add functions and format support, while native clients provide APIs for Python, Go, Java, Node.js, C, C++, R, Rust, and ODBC. DuckDB is open source and MIT-licensed, with governance by the independent DuckDB Foundation.

Mentioned in
3 videos
Kind
Other
ENo. 3333
AIAINotes.us Tool

Engineering Culture Podcast by InfoQ

infoq.com/podcasts/#engineering_culture

An InfoQ podcast series about intentional corporate culture and software management in engineering organizations. Hosted by Shane Hastie, InfoQ's Lead Editor for Culture & Methods, it features conversations with senior software leaders about building engineering teams and systems, organizational practices, hiring, leadership, and lessons from decisions they would handle differently. The podcast is intended for software architects and senior developers and is distributed through services including Apple Podcasts, YouTube, SoundCloud, Spotify, Overcast, and an RSS feed.

Mentioned in
2 videos
Kind
Other
GNo. 6533
AIAINotes.us Tool

Generally AI

infoq.com/generally-ai-podcast/

Generally AI is an InfoQ podcast about artificial intelligence, machine learning, and data engineering. Hosted by Anthony Alford and Roland Meertens, it presents conversations and discussions on topics including large language models, generative AI, algorithms, programming, and AI's relationship with the physical world. InfoQ maintains an episode archive and RSS feed for the podcast.

Mentioned in
1 video
Kind
Other
INo. 6530
AIAINotes.us Tool

InfoQ Online Certification Programs

certification.qconferences.com/

InfoQ Online Certification Programs is a set of five-week professional-development cohorts for senior software engineers, architects, and technical leaders, covering software architecture, AI engineering, AI security and privacy, AI-assisted engineering, and high-performing teams. InfoQ adds and schedules cohorts on its online program site. Each week, participants watch a QCon talk, join a four-hour live session with a facilitator and a confidential peer group, and apply a framework from the talk to a decision from their own work. The program includes time-boxed homework, a capstone technical article presented in the fifth week, and the possibility of publication on InfoQ. Certification requires consistent attendance and active participation in the live sessions, along with successful completion of the group capstone project; sessions are not recorded to preserve confidentiality.

Mentioned in
1 video
Kind
Other
MNo. 6523
AIAINotes.us Tool

Mima

mima.social

Mima is a local-first Mac app that combines Bluesky, X, LinkedIn, and Mastodon into a chronological feed, universal inbox, and composer for cross-network posting, reposting, and replies. It retrieves content directly from users’ social accounts without Mima servers storing users’ content or messages, keeps credentials on the user’s computer, and supports mentions, direct messages, pins, and reminders. Mima works without AI. Its optional AI features include an on-device small language model, with the built-in Llama 3.2 3B model operating locally and offline for tasks such as thread summarization and content moderation. Users can instead provide their own OpenAI, Anthropic, or Google API key, in which case content is shared with that provider. AI-generated content is displayed in a distinct style, and the app is free for Apple Silicon Macs.

Mentioned in
2 videos
Kind
Other
PNo. 0772
AIAINotes.us AI product

Phoenix

Open source · Arize-ai/phoenix

Phoenix is an open-source AI observability platform developed by Arize for experimenting with, evaluating, and troubleshooting LLM applications and agents. It traces application runtime with OpenTelemetry-based instrumentation, supports response and retrieval evaluations, and provides versioned datasets, experiment tracking, prompt management, and a playground for comparing models, adjusting parameters, and replaying traced calls. Phoenix also includes PXI, an AI engineering agent for debugging traces and iterating on prompts, plus a remote MCP server for querying traces, datasets, and experiments from MCP clients. It can run locally, in containers, through Kubernetes and Helm, or in the cloud, and is distributed through the `arize-phoenix` Python package, Conda, Docker images, and related TypeScript and Python packages.

Mentioned in
6 videos
Kind
AI
QNo. 4579
AIAINotes.us Tool

QCon San Francisco 2026

qconsf.com/

QCon San Francisco 2026 is an in-person software engineering conference and training event for senior engineers, architects, and technical leaders, held November 16–20, 2026, in San Francisco. The conference covers applied AI and machine learning, agent orchestration, evaluation and guardrails, API design, architecture, distributed systems, platform engineering, observability and resilience, developer experience, and engineering organizations. Its program is organized into 12 curated tracks with more than 60 presentations from senior practitioners describing systems and practices used in production. Optional half- and full-day training sessions run November 19–20 and cover topics including building AI agents and selecting among agent-development SDKs.

Mentioned in
2 videos
Kind
Other
RNo. 0513
AIAINotes.us Tool

React

Open source · facebook/react

React is a JavaScript library for building web and native user interfaces. It uses declarative views that update the appropriate components when application data changes, and lets developers compose encapsulated, state-managing components into larger interfaces. React code can use JSX, an HTML-like syntax for defining interface elements, although JSX is optional. The library is designed for gradual adoption, can render on the server with Node, and supports mobile applications through React Native. Its core repository is developed openly on GitHub under the MIT license.

Mentioned in
12 videos
Kind
Other
SNo. 6529
AIAINotes.us Tool

Software Architects' Newsletter

infoq.com/software-architects-newsletter

The Software Architects' Newsletter is a monthly InfoQ newsletter covering the frameworks, platforms, tools, technologies, techniques, and architectural practices relevant to software architects and aspiring architects. It is curated by InfoQ and domain experts, and provides actionable takeaways, insights from software practitioners, and lessons about current architecture work. Issues are delivered once a month and past issues are available on the newsletter page.

Mentioned in
1 video
Kind
Other
TNo. 6532
AIAINotes.us Tool

The InfoQ Podcast

infoq.com/podcasts/

The InfoQ Podcast is a weekly InfoQ podcast series featuring conversations with senior software leaders, architects, and technical team leaders about building software systems and teams, including lessons about what they would do differently. Its episodes cover software architecture, AI, engineering practices, DevOps, cybersecurity, and related topics, and are published with interview transcripts through the InfoQ podcast site.

Mentioned in
1 video
Kind
Other

Links mentioned

🔒 Full analysis locked

Unlock more videos and the full analysis

A credit unlocks one video's full analysis for good — the build steps, the tools and how each was used, the methods behind every use case. Pro opens the whole library instead, and raises how many videos you can analyse a day.

Unlock full analysis — free

Transcript

Searchable transcript of The Future of Web Apps: Will AI Agents Generate UIs on the Fly? — InfoQ (47:07). Search for a phrase, then click its timestamp to jump straight to that moment in the video.

Captions sourced from the original video on YouTube, published by InfoQ. The video, its captions and all related intellectual property remain the property of their respective owners; AINotes claims no ownership. Provided for research, accessibility and search — see the Transcript Notice and Copyright Policy.

00:00 How much authority should an AI agent have? At Cucon AI New York, learn how senior practitioners set permissions and evaluate agents in production. Use their experience to decide whether your own agents need boundaries and checks before release. Join us December 15th and 16th, 2026 at the Western Jersey City Newport. Explore the program at new york.con.ai.

00:29 AI. >> Hello everybody. I'm Olympop and I have in front of me Rachel Neighbors. They are quite a creative person. They did a bunch of comics I think right and also they were involved with React and React Native frameworks. And because I don't know how to put it in a better light, Rachel Lee, can you please introduce yourself? Thank you so much for the introduction.

00:59 Yeah, you may remember me from my pandemic babies, the react.dev and react native.dev portals. I worked on the React team for the pandemic. I've worked for Fang on browsers for many years and with the W3C on web standards. But before all of that, I used to make comic books for teenage girls. I was one of the the first web cartoonists that had their own personal website and mailing list, etc.

01:25 So, I take my inspiration from Japanese comics and anime to this day. You can still find me illustrating my own slides and creating mascots for the different open source projects I work on. >> Nice. Thank you for the idea that you put so far. So, probably my kids will read some of the comics. Today we're going to talk about another endeavor that you have and that's local I don't know aggregator.

01:55 It would be probably the the best interface for the social networks that we use. >> Oh yeah. I used to be really big on developer advocacy. You know when I moved to London I was a developer experience consultant for lots of startups and AI in the Bay Area. But I've always wanted to get back into engineering. And finally, I found a project that inspired me to return to building again.

02:22 And it's a Mima. That's m I ma. You can find it at mima.social. It is the client I've always wanted for managing all of my social and news connections. Like I don't know about you, but you can't do business in tech without being on X and LinkedIn and probably also Blue Sky and Masttodon depending on where your friends are. If you got friends in data, you probably need to be on Blue Sky.

02:53 If you got friends in open source, you got to be on Masttodon. I don't have time to check all of those different apps every day. And every one of them is like, "Oh my god, you got to spend all your time here on X. Don't What are you doing posting links to other websites? How dare you? They are very protective of their their little circles, their their bubbles.

03:14 Problem is, you stay in one of those bubbles. You only hear from people in that bubble. If you only listen to X, you'd think that everyone's using coding agents, but there are so many grads coming out these days that they've never touched Claude for coding. So, there's a whole world that you don't experience. I built this to make it possible for me to exist and to see things coming in from any social network I wanted to connect to.

03:41 So I can repost something from X to Blue Sky. It forms a local graph of all of my social connections, everything I've been saying, and it uses native web APIs under the hood. So there's there's nothing particularly magical happening there. It's it's the browser I always wanted because for me all of the big leaps I've made forward in my life in terms of career understanding etc.

04:04 they all came through these social networks but over the years social networks have become harder and harder to access from outside the bubbles. So, I started building Nema to solve my own problem, which was I just wanted to see what everyone was saying in one place and be able to reply from one place. And it's turned into its own little local app. It's got little local models on it to make the experience better.

04:31 It's uh it's all privacy first and local first, which was kind of on purpose, kind of by accident. The truth is, I just don't want to have servers with your personal information on it. The last thing I want is to have various countries coming and being like, "Hey, we need all the DMs from user XYZ." And me being like, "What? Oh, no." So, I I have no interest in owning other people's information.

04:56 So, that's how it ended up starting. And yeah, I'm I'm actually going to be in San Francisco doing a little pop-up soft launch for MIMA next week. It's really exciting. >> Okay. Good luck with that. We are just curious about the technical details because you mentioned that you are part of the React crew and did a bunch of stuff. So I'm curious about what were were the the choices you've you've made while building it because you did mention also models and rather than having an L in front of the large language model you

05:27 have an S. So guide us through through the decision the technical decision you've you've made. I have been following along with the evolution of LLM since like GPT3. One of the things I've noticed especially in hackathon scenes because you know developer experience consulting is like my bread and butter. So I go to a lot of hackathons. I I see what people are building and I noticed you know a lot of what people started to build as soon as LLMs became you know API accessible was a wrapper.

06:01 It's an app that takes everything the user gives it, sends it over to someone else's server somewhere where they do god knows what with that information, probably train your competitor's model, and then they send you back a response. First, this is incredibly wasteful. I did a a series of articles on web.dev dev with Google about like right sizing the model and did a lot of research on like the cost the the energy cost of running inference on a multimodal model it's like four times higher than necessary to do most of

06:38 the work that you're you could do with it so it's energy intensive that means that cost gets passed to you uh in terms of the API cost now you've got linear costs going up with every time you call that and also It brings in all these wonderful privacy concerns. You know, is that PII? We've had so many instances of private personal information showing up in public because somebody sent a prompt to a model and it uh it got passed on to a search engine.

07:09 So, there are reasons why you don't want to just round trip with a big inference. Luckily for us, the technology has moved forward so fast, so well that small language models now do pretty much all the same things that people were trying to send off to big inference back in the day. Small models are perfectly fine for summarizing things. They we've got Jev, which is great for picking the right tool or offering a simple yes or no.

07:35 Jev just completely blows the lid off of everything. So there's a lot we can do with a little model that we don't need to round trip to Gemini or to Claude to get done. And I think it's the first thing we reach for. You know, the idea is like, can we prove that it can be done with a big model that's fine, but you probably don't need to be shipping to production the round trip or the high API costs that you are.

08:02 >> Okay. Thank you. So, you have multiple models on uh in Mima or you have one that has multiple hats? >> Well, that is a great question. What's the right model for the job? Mima does have multiple different little models inside her. The one that I got started with was Llama 3.2b. I was using Claude sonnet to summarize for instance conversation threads when I first started building her.

08:29 But very quickly I started feeling uncomfortable with sending my private messages to Claude. So I actually set up some evals and with post-processing I was able to get llama 3.2b to perform as well as claude sonnet at summarizing locally the conversations that I was having. Evals are really your best friend here. They let you actually rate and judge the inputs and outputs.

08:55 Like if I had just listened to my friends, I would have chosen Gemma 4 because Gemma 4 was really hot right then. But it turns out Gemma 4 would have actually like substantially increased the latency on these summarizations and it wouldn't have provided as good summarizations as Llama 3.2b did. So, I used Phoenix, which is a a little eval library that you can you can set up there to run tests against real information and used a large model as a judge to pick the best model that could do the job.

09:29 It turned out, of course, it was Llama 3.2b, which is one of Facebook's models that they've trained on user data. So, it makes sense that it would be good at summarizing conversations between humans. However, that model might not be the best at doing things like identifying when people are being toxic or if images, for instance, contain uh alarming things in them.

09:54 I'm a big believer that our clients should be able to moderate for us rather than relying on the platform to do that because platform owners, you know, Facebook, LinkedIn, etc., they always don't really invest in moderation. It's a it's a cost center for them, so they cheap out on it. And that means that sor lord of the flies, the kids are running running the school room in there.

10:17 So it's better if your client can do things like, "Oh, this is an incoming message from somebody this person doesn't follow. I'm going to check to see if it's abusive content. You know, you you post a picture one night, you wake up, there's 40 comments on it. How many of them are that's a great picture versus how many of them are like oh how dare you take this picture of a dog and you never know.

10:40 But the idea is that you can use models that are specifically trained to check for different kinds of things to help you write better or to check to see if incoming content is abusive or unfriendly. And MIMA is mostly geared around moderation right now. you know, making sure your trigger warnings aren't being tripped up, etc. And for that, summarization is not necessarily the best option.

11:09 There are smaller, lighter models that can do a better job of that. And all of them are optin. If a person turns on a feature that needs another little model, you know, they have the option to turn that on, download an additional model. It's never installed by default. I think this is something that companies who should know better, who have a lot of experience building products for humans have missed from back in the day when, you know, 15 years ago we were all reinventing the wheel and learning how to build apps and

11:41 progressively enhance the user experience. There is this idea that the user should be able to opt in to enhance their experience. But if it's going to cost extra, you know, extra storage space, etc., you need to ask their permission. And of course, you could always offer them a cloud solution as well, but for the reasons I already stated, I'm not really providing one of those at this time.

12:06 It's not like, well, would you like me to process all of your incoming tweets on my servers or would you like to download this model? you could do that, but for me it's just going to be would you like to download the model? >> That sounds closer to home and also I I like the fact that the the information doesn't leave your machine. So that's quite useful from from my perspective uh and probably others as well and the moderation it's it's highly important.

12:34 It is. I think one of the things we're going to see happening with AI assistants and personal agents in the near future is we've been building them thinking about like, you know, HubSpot integrations and how to do business with them so far, but once they hit the consumer market, they're missing something important, which is your personal context. Yeah, you can connect them to your email, but how are you going to connect them to your LinkedIn to help you find your next job?

12:59 How are you going to connect them to your Facebook to keep up with the kids? How are you going to connect them to all of these walled gardens out there? Each one of which wants to keep your data in their databases. And remember that the the power of that data is only really unlocked when it's crosscontaminated, which is a huge security risk. I know, but there's also so much to be said for if we could just have access to the graph, you know, our own social graph across multiple different networks, our own posts across

13:33 multiple different databases. Our personal agents would become much more powerful for having that context. Not even necessarily to post, but they would just know more about us and be better able to help us as individuals. Well, I like that you're looking only at the positive part and I all I'm all in supporting that. You did mention evals and that tweaking your model for that particular use case.

14:02 It gives you the option of having the same output as a large language model. Well, outcome I think it's more more appropriate. Maybe a couple of hints hints on that especially given that you discussed about something that doesn't have deterministic form. You spoke about text summarization, right? That's a bit polymorphic. Probably a summary for you to be different than a summary for me.

14:28 How did you attack that problem? What would be the tips that you can give me if I intend to do something similar? >> So I am a big fan of Hamal Hussein. I don't know if you're familiar with his work, but he is like a the demigod of evals, evals, evils, evals. And I come from test-driven development background, but when I saw evals, I was like, "Oh, this is test-driven development, but for AI, got it."

14:55 And Hamal is a big proponent that you don't just put together an eval, you know, say, "Hey, Claude, which one of these outputs is better?" Hamal is a big proponent of you walk through each one of the the sets, you know. So, the idea is you take a couple of threads from whatever your source is. In my case, we'll say it's X and maybe some LinkedIn threads.

15:21 And I know what the threads say because they're mine and I have an opinion. They're real. First off, they're real threads with real people. They're not fake. They're not something that Claw generated to hypothetically uh summarize. First, you get them together. I know what they're going to say. I have an idea of what the output is. Note that down somewhere.

15:42 Note down what I think the summary should be. And then you run those through really big models. You get your Astra to summarize them. You get your Claude Opus 5.5 to summarize them. And you see, yeah, those look about right. Those are good. That's the bar, right? You're also recording other metrics like latency and making sure that it's actually able to output the right format for whatever it is you're using at the time.

16:08 Now, the next step is now that you've prototyped big, you want to start with the tiniest model you possibly can and a couple of others. In this case, the tiniest I had was like, oh, it was like Quen 3 point something or itty bitty. Quen is really bad at this. So, I was like, "All right, we'll see. We'll see y'all." But Quen was also really fast for being terrible at the job.

16:35 So, you know, you line them up in a row, you get your Quen, you get your Gemma, you get your Llama 3.2B. You know, you're you're going to have to do a little research to pick the ones that seem like they are, you know, top of the line for you. And you run an eval suite with all of them. And you run it multiple times over the same thing. Because it's non-deterministic, you'll get different summaries each time you run it.

16:56 So, usually like three times with a model and an input coupling will give you a couple of things. What you do then is you actually go through and you can read it yourself and be like, "Yeah, that summary is about right. No, that's not right." You get to actually judge them yourself. Or you could just say, "Hey, Astra," and throw it your keys. I want you to see if this lines up with what you expected as a summary.

17:18 That's the the faster way. But I think Kamal would be like, "No, you need to judge them for yourself first and go through those steps." But then once you have it set up, you can actually go and you can compare like on a a metric, how many times did it pick out the different topics that were brought up in that thread, did it sense all three topics? Did you count all three topics that were there?

17:42 Um, was it able to point to the place in the conversation where each topic was brought up? How long did it take? And it's usually on an accuracy and speed. Uh you're making a trade in one direction or the other. And you will get to make the decision how important it is for things to be accurate or fast. But that's how it is. Once you set that up, you start building your golden data set.

18:07 In this case, I add fuzzed conversations to this data set to start looking for things like, you know, if I wanted to build a toxicity detector using Llama 3.2b, to be I'd have a golden data set of mean things people have said to me and others on Twitter. Not that I have that. Not that I'm actually using things that people have said to me on Twitter to test models at catching mean things people say.

18:30 But you build this golden data set becomes a part of your traditional uh CI/CD runs and that is how you move forward. Now, when there's a new model that comes out, just really balling and you're like, "Oh my god, is Gemma 5 it? Is it going to be faster, better, etc." Now, you just pop that sucker into your Evals and you can decide for yourself. It'll show up on that uh speed versus accuracy binary, and you can decide whether you're ready to swap in a new model or if you want to stick it out with the old one, or you

19:05 need to make some changes to your prompter input pre-processing. To summarize your answer is it depends right like all all proper engineers leaving joking aside it's um you choose your poison and see what's more uh what's more important because you mentioned the local first if you push on that one of the points of the local first idea is don't have any spinners if having the fast things and accurate things then you can probably have a balance and choose the right spots for you And then once you have a golden set, then

19:42 you can move on with that. >> Interesting thing, surprising stuff. You mentioned no spinners for local first. And this is one of those surprising things. I didn't realize that even though the model is running locally, it can be slower than a round trip to a model out there. Gemma, for instance, Gemma 4B running locally on a pretty beefy Mac. That's another thing.

20:04 the metal that's available on the device might mean it's slower. An older machine isn't going to do as well running a local model as like something with an M5 chip. So, keep that in mind. But I thought local first would mean low latency. Hands down. It was not true. Smaller models can take longer than foreign ones to to do the same amount of work because they're they're underpowered.

20:29 It's like the difference between having like a jet engine and like a a little two-stroke scooter engine over there. Yeah, you can power one from the gas station. That doesn't mean it's going to take you there faster. >> Well, and probably there is the other thing that just thinking about it can be problematic when running local first is that you don't have always the same setup.

20:52 If on the same machine things are happening a bit differently. I don't know you are not running on a Mac you're running on a Windows machine and you forgot to restart it in the morning the user experience will be probably degraded or even if the data started gathering because I would expect that putting data locally the meta data that comes with all the social networks to be challenging to take care of and then probably also the I suppose you'll have something locally like a database and on On top of this and a small

21:26 language model, you have a database that you're querying and if that increases so it depends that you have to respond to >> there are I have a feeling eventually MIMA is going to be one of those things where you can buy the cloud solution and you get a VM somewhere that's running an operating system and it has infinite space to grow your database. I have a feeling that would one day be the solution.

21:50 But for now, getting it to run locally on a big beefy Mac, my initial consumer is, I assume, is a person like me who has a modern MacBook Pro, etc., the testing for making sure that local models perform well, really requires you to know your audience. You know, if you have existing data, this is why it's much better if you're working with an existing project than if you're building one from scratch like some people I know is that you have all this user data.

22:20 What is it that people are using? You could then ship local models to folks that you know are going to have a more capable machine for handling that. You know, the latest operating system that requires certain chipset to run is a pretty good indicator, for instance. And you can also do a little bit of detection to see if they've got the right amount of horsepower to run the model.

22:44 And if they don't, you can fall back to a cloud provider in that instance. Another thing, just because your local model, your your small model can't be run locally doesn't mean that it can't be run in the cloud. you still can pay incredibly like just the hosting costs on an open-source small language model in the cloud is still going to be much cheaper to run than you know running up your your chat GPT or your cloud API bills.

23:12 So keep in mind just because it can't run locally on the machine doesn't mean it can't be run cheaply in the cloud. Just thinking about well this particular case how much of your archiving data do you actually need in the real time because as you mentioned the way the the scenario that you're using is what do you usually do what your friends did what happened in the news scrolling through it finding ideas so most often than not you just focus on things that happened in the current day minus x and pretty much that's it.

23:48 So I am trying to solve a similar problem and then that's what we realized that actually 30 days of data is more than enough and rather than using a beefy database we can go with something small in our case it's DB use that and after that just offload everything in park files to them calendar sharding and yes at the given point of time you actually can go with cheap solutions for S3 buckets in the cloud and when needed you just go on that point and surprising enough uh another idea that just popped while listening to

24:25 you is the degrading aspect of a of a system that comes mostly from the distributed part so from microservices if something is done I don't know just for the sake of it Spotify maybe the search is not working now because that micros service behind the clouds is down now well that's something that we can understand also if we treasure our local first data there are a couple of important decisions that we have to make okay I want to have it locally and that means that some things will work slower or faster depending on

25:00 what they have and I think it's quite quite important um I had a debate a couple of years back when the green software came to being and everybody was discussing about how important it is to adapt and so on so forth and what I was challenging at that point is the fact that at points as consumers, not as engineers, we behave chaotically than when thinking about something.

25:24 For instance, a lot of the friends around me are listening to music while coding on YouTube, right? And the thing is in it's in a different app, but that's still streaming 4K video that is very expensive. And because on my machine it's very cheap to decode it, it doesn't bother me. But actually behind the scenes wherever that coal is being burnt or because that's most of the cases I was surprised to hear that in Virginia most of the electricity was produced through coal burning and fossil fuel.

26:00 It's actually a problem. So bringing the problem on our machines and as you mentioned also the target audience are probably engineers that are more accustom and hopefully they'll understand it. >> The target audience is for now. We're always the first ones to adopt new things but I truly believe more efficient ondevice AI is coming for everybody. And I love what you said about how it brings the challenges of like oh I don't want to use this one cuz it drains my battery there.

26:28 that helps humans make better decisions. That is localizing the cost right onto their machine. But you could also say as a business owner, you're externalizing the cost of running the the inference and running the algorithms onto someone else's device. That's not your cost anymore. And personally, like I love edge computing, like literally on the the the user's device because it really shows where economizations need to be made.

26:57 And you were talking about what do you do with really old information? Well, MIMA is a new project, so we haven't necessarily crossed those bridges yet, but every 30 days it's just, you know, content older than 30 days is released into the wild. It's goodbye. It's garbage collected. >> Yeah. Green pastures. >> But you do need some persistent memory about the user.

27:21 >> Yeah. I remember when I first had the idea for MIMA, I thought I was building an agent. And a friend of mine, Kristoff Henlleman in Germany, who's a PhD, and he was like, "No, you don't need an agent for that. You don't need that. You don't. You just need this common algorithm and that common algorithm." I'm like, "So, I'm basically designing a database with an interface on top of it."

27:42 And he's like, "Yeah." And it's true. It's true. Most everything in this world is just a database with an API and a UI layer. and agents themselves usually boil down to it's a database with some non-deterministic LLM accessing it to reformat its contents for a human and once you see that you can't really unsee it so the persistent layer is really just there for remembering the human's preferences and their relationships with the people around them which can be really nicely distilled down like you can keep that

28:18 information very tightly packed in text files. So that's the part that actually does need to be retained. What's this person's relationship with this other person? Which articles do they like to read over and over? But it's not too hard to store that locally. >> I like how you put it. So it's more or less energy. So it's like content that we digest and uh the rest of it is junk and it needs to be flushed down the sinkhole or whatever.

28:45 Yeah, that's quite good. that evolves. Well, these this context means something else. The metadata for particular users. You you store it, you have it, and that's it. But again, that's a problem for a different moment. The challenge of having things locally, it means that you might lose it. So, probably that's something that you'd like to have in the cloud or something like that.

29:07 But that's a whole different local first conversation. >> I think it's kind of like keeping old magazines. There's some people who are going to keep every magazine and every newspaper they get. They they won't let them go. But most of us could probably recycle them every month. >> Yeah, agree. But this opens the door to a different question. How will apps look in the future because now we have the ability with models leaving aside the S or the DL up front uh have the ability of just building something for you.

29:40 Now we have the ability to just give everybody the chance of building the the app that they wanted. There is an ongoing debate build by vibe. These days >> build by vibe. Love it. >> It's something that we have to bear in mind. So you you did mention that you were part of uh those teams that put together probably some of the most important uh frameworks for building web and even local applications.

30:13 How do you see it? Will the future still have frameworks, applications, web, mobile, whatever you want in the normal sense or we'll just have meshes? This is probably the most important part of the conversation we're going to have here. And it may sound crazy to some of your audience members. They might think that can ever happen. I have often felt that way myself and I've watched it happen.

30:40 Well, we've seen more apps released in the last year than ever before, but app consumption hasn't gone up. In fact, people typically don't adopt new apps like their core apps that they're using or apps they installed 4 years ago. Apps are not growing. Websites. What are websites used for nowadays? They are used as landing pages to download apps and they are used for dashboards for B2B companies that don't want to build a and manage a bunch of apps.

31:09 They're the lowest common denominator for doing business. How will apps be influenced by personal agents when they come around? And they will. Personal agents are the app that kills all software for consumption. When it comes to consuming content, buying things, it will most likely come through an operating system level agent. That's because any operating system owner, you know, your Google's, your Microsofts, even Apple will want to and benefit by completely owning your interface with the rest of the world.

31:47 This has been at the core of Microsoft's battle with browsers over the years and it is continuing to become the core of now Google's investment in Gemini and AI assistance. Every big vendor wants to own your experience. This is why Amazon sued Comet. Well, they sued Perplexity for their their agentic browser Comet that allowed people to place orders.

32:13 They didn't want to give up that surface area cuz that surface area is ads. That surface area is upsells to their credit card. That surface area is the one place where they can get you. And to have them just being boiled down to a set of APIs that a UI calls where they're just like a price point where someone's making a choice on where is the best place to get cheap beer.

32:36 That's bad for their bottom line. They don't want that. So, it will be an ugly fight, but it will ultimately be an impossible to win fight. We already see this people, you know, using Claude to do their searches for them, just looking at the the AI summaries on Google search. They don't care. They don't want to visit your website. They don't want to download your app.

32:57 They never wanted to do those things. They just needed an answer or a thing. They needed to snap their fingers and have it done. That's all we've been building for. So that's what the future for that looks like. And if you are curious about like well how are they going to generate all the interfaces for all this information? Check out things like A2 UI.

33:18 This is a standard by which your API can send the AI assistant a JSON schema of what information you're going to give them. And that assistant can look at it and use its design system to make an a widget on the fly that meets that schema's needs. needs an image, needs a next button, it needs price, you know. Yeah. Yeah. And just stitch that together using components.

33:43 Using components that react did such a great job of normalizing. And part of that's possible because so much work was done to create sort of this componentized ecosystem where you can just zip UI together on the fly. And we can trust language models to do that right now to a good extent. Notice I said something there that something's been standardized.

34:06 When things are standardized, that's when it's possible for an agent to come and make things possible. Stocks are standardized, you know, like the symbol, you know, whether it's going up or down. These are these are things we've all agreed on. There's a agreed upon way to represent them. There's an agreed upon set of information you need to work with them.

34:28 That's a standard information concept. Now think about things like your software as a service, your Quickbooks, your Shopifies, you know, these are the dashboard screens that people work with, Salesforce. The problem with these is that sure, anybody can vibe code a dashboard and, you know, a bunch of data structures and databases underneath. You know, you can create your own in-house Salesforce.

34:53 Yeah. You're not going to be able to hire anybody who's going to know how to use it immediately. you're not going to be able to use it with any colleagues or vendors at other companies. You're not going to be able to be like, "Oh, yeah. Let's team up together and share our our user bases." No, because you're now using you're off-roading. You've built your own standards.

35:11 You've built your own data shapes. You you're using a whole another set of information and it is not going to be able to work with anything else, which makes it less powerful. One of the reasons QuickBooks is powerful is because every accountant can sign in and use QuickBooks to help you with your mess. And of course, you could say, "Well, can't agents do that?"

35:31 Not quite yet. There's still a lot of stuff that needs humans. Humans still need to lay hands on things. And often the most important human that can lay hands on things isn't in your company yet. And so that is where I think the investment in open-source protocols, open- source standards, how much people invest in them will dictate how much of this can be vied or handled via anic interface.

35:57 If you're like, "Oh yeah, our company uses the open accounting protocol, too." It'll be much easier for your corporate agent and your accountant agent to do business with each other if they both speak the same language. So that's how I see this evolving on a B TOC level and the the B2B level. B TOC we're all going to be reduced to APIs. APIs and the databases that we own.

36:23 You know competition is going to be on fulfillment. On the other end of those APIs B2B the interoperability layer is is the moat right now. It's the moat. >> Yeah. So standards are important. Usually standards are brought behind companies and that put an acronym on it and then that's it. >> Yeah. >> Well, obviously if a standard is not good, it's just about time to for having another standard that replaces so on so forth.

36:53 So we we know that. But the question is we as consumers what should we transform oursel into because we all saw something on a dashboard that looked good but we don't have the proper naming. So probably as you mentioned a U vicotus language of different things for different perspective it would be quite quite useful maybe the the right question is how are we preparing for this kind of future what are the tools it will be useful to have tomorrow >> I think the era of the boot camp is justifiably at an end I think it'll

37:27 be very easy to kind of like hey join our boot camp and you can learn how to chat GPT apps But very quickly people will learn there is no future in becoming a developer in a person who can just create things because anyone can create things. You don't need to go to school for that. You need to watch 30 minutes of a YouTube video. Question is how do you build things at scale?

37:51 How do you build things that don't turn to crap as soon as they go out into the road? The complexity of the things that we are able to build now is now also infinite. So no one on the React team fully understood how React was built by the time I got there. It had been through multiple key maintainers and revisions. And each member was an expert in like a different part of it, but openly admitted, "Oh, I don't think I know how the entire thing works."

38:19 We now have systems that can not only understand how the entire thing works, but they can make something five times as complex before the weekend. How is a human supposed to build with that? This is where I think computer science core fundamentals really understanding databases for instance this is something I spent a lot of time on early in my career and I heard from many developers you don't need to study data structures you don't need to understand databases that much you're just working in UI right and I was like

38:49 but it's interesting and I want to learn more about it that's the attitude you need to have because the conversations you're going to have with agents are going to be about making trades going be about the complexity of the system and reducing the complexity and building the longevity and understanding how you're going to like change the rails while you're crashing down through the mountain on the engine itself.

39:14 Those aren't things that you really can learn without years and years of experience these days. The trick is going to be learning those things early in our careers, which will probably happen because of many hard knocks working in the system. But I think refocusing from just make UI by hand or you know a to-do app. Uh refocusing from the more grunt work and into the more theoretical work I think will be the first step and learning how to work alongside the agent as a collaborator as opposed to just asking it for output

39:56 and turning around. We do see that happening with young folks these days too. you know, they're just adding vibe slop PRs and, you know, failing reviews. The real trick isn't so much as just pumping out PRs. The trick is pumping out PRs that ratchet up their quality with time and building the systems that can do that. And that requires thinking under having a good understanding of computer science, but also having an understanding of machine learning and LLMs and how they are actually working under the hood.

40:26 They're not magic black boxes. They're math. Once you understand how they're working, you can go, "Ah, you know what? If we just saw it off the top of this Jeff model, we could completely replace that model over there and that would reduce the latency." That's the kind of conversation that needs to happen now. So, a return to basics, a return to the Well, it's exactly against the obvious things happening these days.

40:50 Everybody considers that we don't need to know. Well, that was true also when the internet appeared. We don't have to know everything because you have access to it. But if you don't know what to uh look for, what to search for, it's impossible. And then this is more on the social skills side, softer side, it's probably critical thinking, curiosity, continuous learning, and then the social ability of interacting whether with the human, with a fellow human or with a fellow agent to be able to continuously move forward.

41:23 >> Agreed. And one of the biggest challenges is that people think their agents know everything and they don't. There are new standards and new technologies being released all the time that your model hasn't been trained on. You are the wetw wear that gets to go out there, talk to other people who are building big things. Come back and be like, "All right, I got an idea.

41:41 Let's try this spec, this format, this thing." That's your job. You're the busy bee pollinating in the garden. We've come from an engineering culture that made heroes of the code machine who could disappear into a room and just pump out perfect PR after perfect PR. And we would tear each other apart in code reviews. And it put a lot of emphasis on being perfect and being right because you only get one chance.

42:07 And if you're not, then you're dragging the team back. We're moving to a place where code's free. Part of it's going to be wrong. The real question is, are you able to reason about it? Are you able to ratchet up the quality? Are you able to build systems that are flexible enough to pivot and weave with what the users need? And that is a more collaborative space and less of an antagonistic space moving forward.

42:31 And I think those collaborative skills as you said are going to be real assets. >> We spend a lot of time into the mechanicals of things. For instance, both of us mentioned that we are traveling in the upcoming period. I was just thinking about being in a car driving and then obviously the comfort of a car is important but if you'll be able to teleport like in Star Trek it would be more important why are we going there what would be the things on the other side and that's my feeling we as an industry we are not as old

43:00 as other industries we are just having a couple of decades but we did spend a lot of time into we call it craft but it was more like crap of how we we build off and we just focus on the mechanics of how to build the things. Now we can look at the social aspect of that point. Why are we building it? How are we building it as a group and the outcome that we we have and I think that's that's important.

43:26 Another idea that was flying and I had a couple of conversation on it. The easy technology problems are done as you mentioned how to do the buttons and so on so forth. It's so easy to make the things now. It's important why are we doing it and all the important things that actually would help us as wetware on this uh on this planet. >> I got to be honest, I hear a lot about people building these software factories.

43:52 And I feel like agents are really good at building stuff that's been built before. If you just want to build something like Netflix but for dogs, you know, an agent's going to be able to build that without a lot of human input. If you're trying to build something that hasn't been built before, maybe because it wasn't possible previously, or maybe it's because it just wasn't worth anyone's time previously, maybe you want to build a browser that brings all of your social networks into one really great interface.

44:21 Now's a good time to look at that. But you're going to have to put your hands on the code. You're going to actually be working with an agent. You're going to be having conversations and making trades. You're going to need to know your databases. you're going to need to know the different kinds of, for instance, encryption formats. So, the exciting stuff is probably not building another fitness app.

44:42 It's probably that one crazy idea you never thought you could have time to work on, and it's worth looking at. Now, >> one of the other demigods of the AI you did mention and that was in line with uh what I I tried and what to believe AI can provide that extra 20 30 40% of your IQ if you decide to use it because you can just say do that for me and that's it and you're happy with what you got or you can just say okay teach me what you did wax on wax off as uh somebody would say and that's the important thing I I For me,

45:21 what it means in all the doors that were open and probably for the most of the people around me is we now don't have the need of doing a lot of research because previously as you mentioned we have tons of databases. Which one would you pick? You have to understand it. Do your research understand and then if you go down the rabbit hole you're gone and you're just debating that and you forget about the purpose.

45:42 But now you don't have any reason of not to have it because you have all the points that you actually need. It's just about what you're trying to do. As you mentioned, it's easy to do a lot of stuff. Now, it's about having them. >> It's true. >> It's a conversation that probably can go forever, but we have to be also worry about our time. >> We have to go get packed.

46:06 >> Yeah. Yeah. Yeah. Exactly. Even though we are going two different direction. You're going west and I'm going east. There is a lot that we could talk about with MCP, WebM, MCP, MCP apps, but you're just going to have to have me come back. Folks, if you want to keep up with my writings about the future of the web and these topics, you can find me at nearest neighbors.com.

46:25 That's neighbors nab o rs. I've got some courses coming out on LinkedIn Learning around MCP. If you are a LinkedIn Learning user, you can catch me there. And uh yeah, check out mima.social if you've got a chance. Cheers. Thank you, Rachel. >> Thank you. It's been a pleasure.