← All transcripts

Scott Jenson on Evolving Desktop OS, Local-First, & Agentic UX Transcript, AI Summary & Key Points

InfoQ · 2 hours ago · Science & Technology · 40:06 · EN

AI Summary

Desktop operating systems have largely stopped evolving, while mobile systems prioritize content and simplicity at the expense of productivity and flexible file handling. Local-first software offers a balance between cloud capabilities and local control by keeping data available, preserving privacy, and allowing work to continue during connectivity failures. Desktop interfaces could become more useful by remembering context through spatial organization, richer clipboard history, project-specific working-memory areas, and activity timelines. LLMs should be integrated into applications and workflows rather than exposed only as generic chatbots. Local models and agentic coding can shift power away from large providers and help UX designers iterate more quickly, but they introduce ethical, privacy, quality, and safety risks. The broader goal is to recognize shortcomings in current systems and explore better solutions instead of accepting the status quo.

Key Points

  • Desktop operating systems have essentially stopped evolving in fundamental ways for about 20 years.
  • Apple's desktop improvements over roughly the last 15 years have mostly tightened integration with its ecosystem rather than changing the desktop itself.
  • Mobile won consumers but did not win productivity; tablets are strongly suited to content, gaming, and social media but are less effective for broad work.
  • Local-first software addresses both cloud privacy concerns and the practical problem of losing access to work during connectivity failures or cloud outages.
  • Mobile applications often hide files inside application silos, making it difficult to edit a file in another application or find downloaded files.
  • Direct manipulation is intentionally stateless: actions such as cutting, pasting, opening, and closing do not preserve much history or context.
  • A more supportive operating system could provide working memory through spatial window organization, accessible clipboard and project context, and a visual activity history.
  • Virtual desktops can be powerful but may be too rigid for users whose workspaces are organic and messy.

🔒 6 more in the full analysis

From this video

4 products

DeepSeek Gemini Google Sheets Tiger Data

AI in practice

Used for

How
Used a model running locally for coding.
Outcome
The result was described as pretty terrible.
With
DeepSeek
How
Switched from a poor local-model coding attempt to an open coding agent using DeepSeek.
Outcome
The result was described as shockingly good, although not perfect and still subject to ethical issues.
With
Gemini
How
Asked Gemini about pivot tables, then used it in Google Sheets to build the pivot table.
Outcome
Gemini explained what a pivot table was and built one in Google Sheets. It also provided a mini-documentary-style explanation of why pivot tables exist and how to use them.
Replaces
Manually figuring out how to organize the data and manually creating the pivot table.

🔒 2 more in the full analysis

Business ideas

Build a desktop-oriented, local-first productivity environment that keeps user data available locally, continues working without cloud connectivity, and remembers enough activity and context to help users resume work.

For
People who work across multiple files, applications, devices, or projects and need continuity, offline access, privacy, and easier retrieval of past work.
Solves
Cloud-dependent software can expose users' data to a provider and stop working during connectivity or service outages. Conventional desktop interactions are also stateless, while mobile applications hide files inside individual apps and make cross-application editing difficult.
  • Spatial window manager: spreads activity across a wide visual area so users can find things quickly without relying on rigid virtual desktops.

🔒 The rest of this idea is in the full plan.

Get credits →

Build software that embeds an LLM into a user's actual workflow, combining conversational guidance with conventional interface controls and direct manipulation of the work product. The LLM should help users learn, choose among alternatives, and perform actions inside the application rather than merely returning chat responses.

For
Professionals who understand their work but may be unfamiliar with a particular software tool, such as an experienced accountant moving to a new system.
Solves
Users can know what they want to accomplish while not knowing how a new application performs it. Generic chatbots provide explanations but often stop short of carrying out the task, while adding more chatbot skills does not necessarily provide an effective workflow.
  • An experienced accountant moving to a different software tool: knows what to ask for but may not know how to perform the task in the new interface.

🔒 The rest of this idea is in the full plan.

Get credits →

🔒 Full plans for 2 ideas.

Get credits →

Links mentioned

🔒 Full analysis locked

Unlock more videos and the full analysis

Buy credits to process more videos. Each run includes the full analysis, not just the summary — and you get access to the locked analysis across the library.

Get credits →

Transcript

Searchable transcript of Scott Jenson on Evolving Desktop OS, Local-First, & Agentic UX — InfoQ (40:06). Search for a phrase, then click its timestamp to jump straight to that moment in the video.

Captions sourced from the original video on YouTube, published by InfoQ. The video, its captions and all related intellectual property remain the property of their respective owners; AINotes claims no ownership. Provided for research, accessibility and search — see the Transcript Notice and Copyright Policy.

00:00 Postgress shouldn't force you to choose between transactions and analytics. Tiger data creators of time scale DB adds hybrid row and columnar storage 95% compression and continuous aggregates. So both run in one Postgress database. Try it free at tigerdata.com/go/trial. Hello everybody. Hey, I am Olympop editor and today I have in front of me Scott Jensen.

00:29 For those of you, if the name doesn't say nothing, he's probably one of the UX designers that has a lot of impact on your life or the things that you're doing if you're using a MacBook or if you're using an Android phone because you worked on a lot of this stuff. Thank you, Scott, for talking to us. >> Great. I'm happy to be here. Yeah, I basically started off at Apple.

00:48 I worked on the Macintosh system 7. Um, I then worked over and moved over to the the Newton, which was a a big handheld at the time. Got a lot of hype. Then moved over to do a lot of desktop work. I ended up working at Google and worked on the first version of Google mobile maps. Went over and worked on the Chrome team. Had a project called the physical web and then did quite a bit of consulting.

01:14 I was at Frog Design for a number of years as creative director and worked on some big projects for say HP and Disney and went back to Google at the end of my career. About two years ago, I left Google. I'm now kind of working on my own. I liked one of your talks and the way how you framed the current work that that you're doing. You mentioned that you're repenting for your sins in the last period and that's why a lot of your work is currently in open source.

01:39 Thank you for taking the energy and time during this period and doing that much exploration and that's why we are together because you did have I think almost a year ago it wasn't at Ubuntu I think conference where you spoke about the past present and future maybe but more on the present and future of what UX for desktop means that got a lot of traction.

02:05 I think it was last time when I checked it was half a million views and then I saw the presentation that you had local first not that much fraction because it's more current uh than the other one but what you had different there you had a couple of ideas that you shared actually you put your fingers or your ideas creative ideas into practice and you provided a couple of ideas how UX can look for desktop and to better organize your work but let's not get ahead of oursel and take a step back.

02:35 Let's look a little bit as you mentioned you grew through the phases of what this top means. Let's look at how the UX in the desktop space evolved and maybe touch a bit if you think it's useful on the differences between mobile and desktop. Because even though mobile is around with us for some time now, we are still using desktop and there is no sense of something that will come and replace them that people try to replace them with the tablets, but that's nowhere near into into that room.

03:09 >> Well, I think you you covered a lot of the kind of the key hits I've been talking about over the last couple years. The first one is just that the desktop has basically stopped evolving like for the last 20 years. Apple has introduced 12ish features in the last say 15 years but they're all about making the ecosystem better. You know universal clipboard or the phone being used in the desktop or airdrop or things like that.

03:35 So the vast majority of the improvement with Macintosh has been to tighten its control around its ecosystem and the desktop itself has not fundamentally changed. And that transitions to what you were just mentioning about mobile that a lot of people feel like all the hotness, all the excitement is in mobile. But the point I made in my talk is that mobile won consumers, right?

03:56 It didn't win productivity. So back around 2015, all these companies tried multiple companies at Google, they tried this. They called it tablet Tuesdays and they was like, "Everybody bring a tablet to work and do all of your work on the tablet." And that was a disaster. It's not that tablets are bad. It's just that tablets are hyperfocused on things like content and gaming and you know social media and so mobile and tablets are very good at a very narrow set of things and so a lot of my work has been trying to say

04:28 let's not throw desktop out. What is it about desktop that is so powerful and then how can we inject some of that into mobile? It's not either or. It's about what's good. And so I've wanted to both improve mobile, but in the talk I gave at Iuntu was I also wanted to improve desktop. And my general thesis is that we don't seem to be that excited about improving things.

04:55 We just kind of accept the status quo. And I gave a couple of ideas on that. This thing where you're saying yes, we have tablet and it's useful for some stuff. We have mobile and it's useful for some stuff, but desktop is still king in a couple of categories and that's very useful. That sounds a lot like cloud where you have everything across the board and then you regardless of what you're using you'll have those things.

05:19 But given that our discussion started around local first and what that tries, I had a similar conversation with Adam Wiggins which is involved both with local first but he also was one of the pioneers of the cloud with Heroku and bringing things between developers and one of the conclusions of our conversation at that point was we have a tendency as technologist to go from one extreme to the other.

05:45 We moved from desktop, we moved all the way to the cloud and now we realizing that that abstraction is too much and we miss some of the points and now we have to find a sweet spot and that doesn't have to be something set in stone but it will be probably something like uh uh gradient. You're moving left and right depending on what's the scenario or different types of approaches.

06:08 What's your take on it? How do you feel about it? what would be the the things that we can take from I don't know maybe mobile maybe desktop and what will be the points that we need again you touched on many points there I think the big issue with the cloud well there's there's lots of things to talk about but the the two big ones is privacy the idea is that with the cloud all of your data is with one particular provider and so this is a well-known issue people talked about I'm not the first one to mention this And

06:41 that's why local first gets a little bit worried that you're giving all your data to Google or all your data to Microsoft. And so there's that aspect of it. But the other one is just simply pragmatic, which is to say we live in a world where you don't always have connectivity, right? And so how do you continue to work on something if it's in the cloud because you're cut off and whenever there's an AWS outage, the world seems to always rediscover this fragility of it.

07:04 And so there's a both a practical and a philosophical reason, I think, for talking about it. But like you said, we don't want to throw the baby out with the bathwater. It's not like cloud is inherently wrong. I think local first is just trying to build it so it's robust so I can keep all of my data. You can't take it away from me and if the cloud goes away, I can continue to work on it.

07:28 And that feels like a very powerful thing that cuts through all of local first. >> Yeah. And I think it's uh well it's more heavyweight than what we're discussing but distribution of of load is something quite important and that goes around in pretty much all aspects of technology in life. I'm I'm just thinking now I'm based in Romania. So that means that we have a a huge neighboring country called Ukraine that has a problem for the last couple of years and that means that if the things are not distributed they have a

07:58 huge problem with bombardment just taking out electricity and I think that's something important also for us just distributing stuff as you mentioned across we have different copies and there are other problems to that but that's a whole different conversation as you mentioned so now we have more or less two different trends if you want. We have the Linux- based family of systems and I think you know about all the open source distributions or the ones that got forked from them more or less like Mac OS back in its

08:29 infancy and then we have Windows. What do you think? Where does the creativity coming from? Because they have a tendency of copying each other. It's just like a game of tag. Where is the creativity coming from at this point? Yeah, that's an interesting question because as a UX designer, what I would say was that Apple was always trying to kind of initially just really solve deeper problems.

08:55 Where did I worked at Apple for a number for eight years and it was just like what did people need, right? And what do we want to do? And I think all tech companies have changed recently. So I don't want to single out any one particular one, but Apple and frankly Google have been what do we need? Right? It's that's it's why I felt like I had to leave Google.

09:15 It was all about the money. And when I was at Google, we really were a bunch of boy scouts and we just wanted to do what was right and the money would follow. It's not like we didn't want money. It was just more like if we did the right thing, it would take care of itself. And it's become so clearly about lock in. I mean what Google has done with Android and Chrome and what Apple has done with Safari and locking out browsers.

09:38 This is not for users. This is not a user empowerment strategy. This is locking you into our ecosystem. So we get our 15% of the revenue. And so yeah, I think there's a natural frustration with us, for lack of a better word, latestage capitalism that is showing up in these companies. And I refuse this framing like, well, you got to eat somehow, which just gives up any attempt at ethics or longer term thinking.

10:07 And I believe that there is a win-win here. You know, you can make money and you can think longer term. And in fact, as a UX designer, you're frequently making this play to companies, which is to say, yes, if we do this UX work, it's going to cost a little bit more money, but you're going to get brand loyalty. you're going to get less errors, you're going to have less call backs.

10:29 So, this idea about investing in the future to get bigger returns is kind of at the heart of social democracy. It's it's a it's a longer term vision that means that we can do the right thing and possibly make money. >> Yeah, I I agree with you and while listening to you, an idea was just spinning in my head. It's all nice and cozy thinking about having partially the data that you normally need in your pocket on your phone or tablet or whatever.

11:00 But the tendency of the standards and I know that it's that's a topic that you touched upon is that the images for instance they are so huge and once you put them on iCloud they have a tendency of providing very small thumbnails on your phone but you still need the cloud. So even though I have a what I consider something that is very generous, it's I 250 GB of memory of space on my phone, it's always full and whenever I want to check a a photo, it's like okay, your connectivity is not that great.

11:34 So please rejoin the Wi-Fi. So I think that's one of the challenges into probably a degrading way of looking at it. I remember back in the day 10 years ago we developed an application and we had a degrading level of quality. For instance, if uh I was on my mobile phone, the photo was up to a given resolution. And then if I was on a more powerful device like a tablet or even a a MacBook, I was providing different uh degrees.

12:03 And then that allowed us to zoom in or zoom out depending on the resources that we had. And as you mentioned, there is an ecosystem and everybody tries to lock you in. And I do have to admit that the Apple ecosystem is very well integrated. Everybody else is ears light years away. But that's my problem as you mentioned if I need or a very simple scenario for me it happens that I need to send emails from my mobile phone I download something and I cannot find the file anymore >> right >> and then it's a lot easier to

12:33 just go to my MacBook but I think these are the points that you had as well and going back to desktop what's the the real problem or how you call it the curse of direct manipulation I think you you referred to it. What are the building blocks that you were starting when we're looking at desktop? >> Yeah. Well, the previous point you made about the downloading a file and then not being able to find it on your phone.

13:01 That is a a fundamental problem of mobile because what mobile did was it hid effectively files within each application. So if I want to edit a file in another application on mobile, it's nearly impossible, right? The only way I can do it is to share it to the other application, which often makes a copy. And there is a file manager, but most apps don't use it.

13:21 In fact, I have a whole blog series about this and how we can have the best of both worlds. We can have the simplicity of hiding files uh within applications, but we can expose them as files. That's a a separate conversation about just evolving the mobile system to be more file friendly, but keeping this whole self-contained model that makes them so much simpler.

13:44 But now going back to your curs of direct manipulation, I I use that kind of with my Allen K quote that perspective is worth, you know, 80 IQ points. And so by understanding a little bit more about what the desktop has historically done, it gave us perspective to figure out what we wanted to do to improve it. And so the idea there was that direct manipulation is stateless.

14:07 You cut, you paste, you type, you open, you close. And they're all these atomic elements that let you manipulate a document. It's it's all good. No one's complaining. But it's by design stateless. It doesn't remember what you did. It has no history of what you've done. And so I used that as my perspective to say, what would happen if we had an operating system that actually remembered a little bit more of what you did?

14:31 And then I had three different prototypes that kind of showed, you know, what that was like because that would allow effectively a little bit more stuff to happen and give some support because to me the whole purpose of an operating system is to support the user doing work. And drip manipulation is one level of support. But then the next one is this ability to help me maintain my context so I can come back from lunch and I can keep working.

14:59 >> Yeah. So taking a step back, it's something that just circled in different environments. People are saying, okay, data is the new oil and stuff like that. So that's kind of true also for us because in the end, if you get a laptop and doesn't have anything on it, it's just a machine that is very powerful but doesn't have the ability. Once you have an operating system on it, you have a collection of uh applications that allow you to do basic stuff.

15:24 But once you start using it for you, you have files and all those aggregated. But if you'll open the downloads folder for instance, that's crazy. It will be impossible. And what you you mentioned and what you underline now as well is it should be an extension of ourself, our brains and have also a working memory, some kind of metadata that will allow you more more context.

15:50 and the isness the facility through which we are mentioning to do that it will allow you to make things a lot easier and then just resume at different points and then avoid the different types of um system of um that you need to just do by hand. >> Yeah, exactly. >> You had also a couple of suggestions. You didn't just put the finger on the wound, but you had a couple of proposals.

16:15 Three of them if I remember correctly. What do you think at this point would make our lives as users of desktop easier? What would be the options that we have on the table? >> Well, yeah, I I have a whole talk I can give on that one if you want. Um, but no, the >> No, no, no. But the the the the three prototypes that I showed at the local first conference was about this idea of effectively working memory.

16:40 The first one was about trying to not be blinkered by the small screens on mobile or frankly even laptops. And so I imagine we had a bigger screen and what could we do with that? And so I had effectively a window manager that allowed you to spread things out so that you had a a wide range. And that gets into human perception and how it's very easy for the human eye to kind of look left and right on the horizon and see things in the distance.

17:09 And so the idea was to have all of the activity that you're doing kind of spread out in a very helpful way that gives you the ability to find things quickly. I have a lot of issues with people in the Linux community because they are very smart and they're very organized and that creates a certain type of very rigid software. Uh and the example I gave is virtual desktops.

17:31 Virtual desktops are very powerful but they only really work if you're really smart and really organized. And I think a lot of the world is very organic and messy. We all have desktops where things are piled on it. And so I kind of wanted to have a windowing system that was kind of the anti- Linux approach which was just more organic. So that was the first one was a window manager that kind of spread everything out.

17:50 But the other one was to say that we've all been in the situation where you cut and paste something and it's lost because you cut something else on top of it. And yes, all operating systems now have a clipboard history, but we don't really use it. They're kind of a pain. It's like, oh, it's the seventh thing down. Or, you know, you could in theory go into one field and cut every single field out and go to another field and paste them all back in again.

18:15 But that's nearly impossible. It's tedious. It's just so hard to cover. So, the idea was to say, how could we elevate this whole copy paste history model? So, I created a system where a little bit like Obsidian, we had a little side drawer and you could put all of your clipboard history in there or the files that you're working on or sub things. It was meant to be effectively a little mini context window that allowed you to kind of store all your data for that particular project and you could open and close it like a

18:45 file, but it acted like a folder. And so, it was meant to be a way for you to have working memory of that stuff. And then the final one was more of a history that was totally stolen from a classic paper in the '9s called lif streams. And the idea was to say, well, what have I been doing the whole day? Just give me a ribbon of everything I've been doing, but provide some clever visualizations because once you give someone some scaffolding, a memory scaffolding, it's like, all right, that's what I did right before I

19:16 talked to an Olymp or this is what I did before lunch or this is what I did. And it gives you a little bit of a structure and you can find things effectively by adjacency. And so these aren't the only three things that we could be talking about, but it is an attempt at me to articulate and give examples of how we can have an operating system go well beyond what it's been doing today.

19:37 And I hope it got people excited. >> Okay. So we are looking about the working um memory and how the things are being done and different perspectives on top of that. One of them as you mentioned was spatial and that allows you to organize it in a different way but not in the way with the virtual desktops we have because my feeling is that always when I'm looking at virtual desktops it feels that I have you know my my skiuit on and I have so many pockets but I I it seems that I'm not ever able to find the car keys or

20:10 anything else especially when I'm wearing the the ski gloves. That's one and then that's allows you one important aspect that for me it was really impressive was the fact that even the the smaller files are useful. So meaning that you can see the data there and you can just use them or for instance you had the example of a music player and then that's depending on the way how it's zooming in or zooming out it just changes shape and uh is still useful.

20:37 The other one is in terms of a timeline of what we actually worked and then allows you to look at the things differently or the clipboard being organized in an associative manner where you have different things that are grouped in the way that it's better. What do you think other things that would be useful just as guidelines like best practices? You mentioned you have more.

20:57 So if there is anything that needs to be brought to light. Well, my overall thesis is the desktop and frankly mobile have gotten frozen in time and all I want us to do is to try a little harder, you know, to just explore a little bit. So, I've written on my blog I have the future of files and the present of files that talks about how we can extend mobile.

21:23 I've done work in the past uh with Google. I did a project called Eloquent which is also on my blog that talks about how we could significantly improve text editing on mobile and then of course these three concepts I just did on the desktop. So all I want us to do, my my my my primary goal here is to say there's so much we could be exploring and the world is not set in stone.

21:47 But so often people think that I want to blow everything up and and there's a crowd of people that says it's taken us 30 years to get to this point. Don't blow it up. And I'm like I don't want to blow it up. I mean things are always changing and we're not going to use the same desktop 50 years from now. Something's going to change. And right now, the way things are looking is we're just going to let Apple hand us the keys to the future because they're going to define what we want to do.

22:11 And they clearly aren't earning that right. And frankly, neither is Google. Now, I know I'm becoming a little bit more of a radical here, but I mean, my point is to say, let's appreciate that there's more to do. And so, I gave you three examples. I know there's a gajillion more. And so I don't have anything specific other than those three, but what I am doing is I'm actually trying to turn my prototypes into real code because that also helps.

22:40 So the last one I gave you, the the live streams that shows, I've now got it written as a Chrome extension and as a Swift extension and it's capturing desktop events and it's capturing Chrome events and it's putting it together into a single thing. And I'm going to be giving another talk in September at KDE and I'm hoping to have it done in time for that and then I'll open source that of course so everyone can play with it.

23:04 But and it raises by the way all sorts of privacy issues. But the goal right now is to just try and to see if it's powerful and if it's helpful and then if it is then we have to then have further conversations because there is this tension between a system that remembers and does things for you is also a privacy risk. How do you navigate that and how do you make sure that that's done in a way that's protective of the user and so there's a deeper conversation to have there.

23:34 >> So we're looking forward to your work. Good luck with getting it in time in September. there is not that much time until this. So, best of luck with that and I'll try to add another point based also on the conversations we had and also on your talks and that's standards because now there are a couple of ecosystems and um I'm just thinking now about Apple.

23:54 I would have stayed away from that. But you did mention earlier airdrop and my feeling is that airdrop is such a great technology especially when you have people in the proximity and you don't want to go all the way through the cloud and send them on different type of messaging apps but that works only in the Apple ecosystem >> that recently changed right now Android basically has figured that out and now it works >> and that's where I wanted to get to is the fact that there is a lot of effort on the European Union

24:22 side to just open that up to create a standard based on it AirDrop started it and it seems that the points regulation is useful as well. Do you think standards is also an important aspect to just have this bridge across different technology gaps? Is it something to worth to keep in mind? >> Oh absolutely. But I whenever you say the word standards, every tech bro in the room will stand up and yell, "You're killing creativity," you know, and there you can.

24:49 And so I want to make sure we acknowledge this is a challenging issue. But like say for example getting Apple to get rid of the lightning connector to go to USBC. That would have never happened if it hadn't been for the EU. And so that was I think very good pressure. And again, it shows the fact that from an e-waste point of view, it's the right thing to do to have one universal connector.

25:11 But it was not in Apple's best interest. It was in the world's best interest. I wouldn't call that a standards conversation. I would just call that kind of a compliance conversation. >> Y >> and so I do think that there is a bigger role for governments to force companies to think longer term. And I'm gonna go back to my radical roots here about latestage capitalism.

25:36 It's not that these companies are necessarily evil as much as people like to say they are. I like to say that they are amoral. They're not immoral. They just don't care. They literally only care about the very very short-term effects, which is why we have global warming and and other things as well. So to me, the way of framing it is what's the longer term goal?

25:58 Because there's money involved. It costs money to think longer term and so how do we do that in a way that is fair to these companies but frankly is also fair to us and because I think the vast majority of the decision-m has been only what's fair to the companies it's been extremely one-sided so I do believe it's time for a little bit of push back I would normally refrain from having these kind of comments especially in a time bounded context but you mentioned radical twice already so I have to mention my radical

26:29 perspective as well. You're totally right that uh companies tend to look only at their interest but that interest is fueled by our comfort because we have a tendency and you can see that all across as humans we have okay it's a lot easier to bookmark a page but rather than bookmarking it and being organized and that gets us to the initial point we use Google search engine and that's obviously something that goes all across the internet it's uh burning fuel and so on so forth.

27:01 important now with LLMs that's even more so >> agreed >> ethical aspects environmental aspects and so on so forth so if we would be more concerned about the future because the general population will be the most affected by climate change and all the other problems the companies will have to stand up to their standards and you can see those points mainly across the last couple of months when there is this sovereignity movement in Europe and where we just realized that we do have things as well and I'm happy to see that

27:35 we still have people that are aligned across the pound in terms of what are the European the western values and what do you think to just make sure that greed is not the only thing that is fueling technology and everything else. >> No, I completely agree and there's multiple forces involved. So we talked just a few minutes ago, we talked about the role of governments to kind of force these things, but also there's the role of technology, right?

27:56 Like for example, what's going on right now with LLMs and the big data centers and so forth. There is likely going to be a technological force that's outside of the governments, which is to say we're going to over the next 5 years have machines that are significantly more powerful and we're going to have open-source local models that can run locally.

28:15 And this mirrors what's happened throughout the history of technology whether you're going server side or or you know we went from mainframes to laptop mini computers and from minicomp computers we went to kind of cloud when we went back to phones and so this idea of going cloud and local and cloud and local has repeated itself throughout history and I think we'll see the exact same cycle with LLMs.

28:35 Just yesterday I was trying to do some coding using a local model and um it was pretty terrible but I had to try right but I did switch to an open coding agent and I used DeepSseek and um it was shockingly good now deepseeek has its own issues. I'm not trying to say it's perfect, but I mean I'm saying is we can see the vector right and so as we get a more ethically trained model which still I don't think that mark has been hit and there they all have ethical issues right now but the point is if we get an open source

29:11 model that we can run locally that provides I think a fantastically interesting opportunity to switch power away from these big guys too. So there's lots moving that's interesting. Thank you Scott for moving the direction towards LLMs because I know your opinion and I align with that because they are points immoral and they have a they are a terrible disaster for the green part but they are very useful at this point and they will be able to accelerate it and now you as you open Pandora's box I have to ask the guys from

29:41 Google tried to make everything a web page with Chrome OS well that didn't take off now the guys from open AI want to make a small box with just one button and that's it. Is the discussion that we just had useful in the contest that everything will be a a chat window and everything else will be just typing in. Okay, how much time do you have? I the short answer is hell no.

30:10 And I think as a UX designer, I mean I actually again I I try to write my thoughts down. So on on my blog I've got a post called Boring. You talk about LLMs are boring. Boring is good. And I think that there is a lot of ways in which we can use LLMs that are smaller and easier, but also we need to understand that intelligence is is inherently social.

30:36 It's a construct that you and I agree on as to what is intelligent. And so much of what we do that is work is actually part of our cultural construct. It's not a mathematical concept of intelligence. It's something that we share as a society. And the same thing applies to LLMs and work. So much of the work is more nuanced. It's got context. And so I do believe that we're going to get more LLM based support, but it will move towards the problem and slapping a chatbot and everything is going to become the first resort of

31:06 losers. I mean it's just it's yes you can do it and there is some value to it but I do believe that having the LM built into your problem it's more work but it will allow you to do far more interesting work. So I've written about I don't want to go into the details of the specifics but but I do believe that we will get past this chatbot phase as more and more apps are written but to be clear it's not as easy right it's cheap to throw a chatbot on >> okay in my evil moment when I was working for a company where dollar

31:39 was king or well euro was king I was trying to frame the offer of the company during the takeoff of the LM and what I was pushing for was augmented user experience augmented interfaces. So at points as you mentioned in our previous conversation where I don't know you want to find the right temperature balance with the voice interface that can be quite problematic when you can be a knob like just going left and right a bit more a bit less it could be frustrating and that's the feeling also with LLM at points but what I

32:13 think an LL can do is provide the mechanism at a given point of time where you want to fast onboarding let's Say you have an accountant 20 years of experience. He worked his whole life into a given tool, a given software. He's used only with that. He knows what to do. He moves >> next day to to another one and he doesn't know what to do. But he knows exactly what he he should ask.

32:38 So that from my perspective is a good way of looking at it. It can be voice. It can be a couple of buttons. It can be a chat window. And that's a good way of interacting with an LLM that will provide you the exact tables and charts that you would imagine. And that would be a lot faster than low code from my perspective. >> Oh yeah. I mean, just the other day I had a whole bunch of data and I just said, I want to organize this but not quite sure.

33:04 And he goes, "Oh, you should probably use a pivot table." And I'm like, "What's a pivot table?" Because I'm not a big spreadsheet guy. And and it explained what it what what it was. And now it still was chatbased. And so what I would challenge these systems to do is to then move to the next step which is to say okay I'll actually input the pivot table for you.

33:25 And in fact actually I take it back. I went to Gemini and I asked about it as a pivot table and so I went into Google Sheets and I take it back. It built the pivot table for me. And so that's what I mean by going into the app and and doing more of that. So yeah, I think that there are lots of ways to do that. And I find myself really leaning into the education side of things, which is to say once I learned that, I said, "Okay, now tell me about pivot tables.

33:52 Why are they there and how do they use?" And it gave me a little mini documentary. Now to be fair, this is entirely a chatbased interaction. So, I don't want to say chatbased is entirely useless, but I'm trying to draw the distinction between learning and doing. And I think we're going to be doing more doing as these things get more mature. >> Yeah.

34:16 Well, it's being refined. I think last year I had a conversation with the guys from IDEO and they had a lot of research into the AI space and they did multiple things bringing together like QR codes, LLMs, dots on pictures and all that stuff. and they try different mechanism in different aspects and I think it's a a whole creative evolution but probably we'll see what comes next as long as we remain creative as you mentioned we just try to push oursel into finding the right balance and the right tool before we go I I

34:44 want to make sure that we don't go down this naive path of just adding more tools to the the AI because what I just said implied that I just want to have more skills and then I could talk to it and it'll just do things for me whereas the example I gave in My boring post was about a company that I don't want to encourage them. I'm not trying to recommend them but they had this is called napkin AI where you would talk to it but they would produce effectively an intermediate artifact and a lot of the options were built

35:14 into the user interface. So in other words it gave you a whole bunch of choices and each one had alternatives that you could explore and it was a classic UX. It's like do you want this one or this one? this one. And so you could have talked about it, but my point is a lot of the AI's options revealed itself into classic UX elements. And so that's quite separate from throwing more skills at a chatbot approach, which is I think the only hammer that these systems can swing is to just to throw more skills into the chatbot.

35:46 So that's the the distinction I want to make. Yeah, most probably just just listening to you what what I think I will deem useful especially in ideation phases as you mentioned is the ability of switching dynamically between different environments like we started this conversation but if I will tell you about Scott I want this kind of frame and with that that and that probably it would be a lot easier to just take a white paper just draw on it and then just add colors and so on so forth so probably that would be

36:14 something a more morphic ability of doing it but we'll have to see how things are evolving moving well. Yeah. And I think it's one of the reasons why UX designers are getting so excited about using coding agents because it means that we can see the results of the coding immediately. And so when I'm working on a more creative idea like this timeline, I've done the research.

36:37 I have the UX. I know what I want, but there's a lot of details as to how it should be presented to the user in a very nuanced way that just requires a lot of iteration. And what effectively Agentic Coding allows a UX designer to do is to explore their prototypes significantly faster. So yeah, I'm using a chatbot to write code, but then I'm I'm playing with the code to say what am I learning and what's different and I know multiple designers who are extremely excited by this new switch.

37:08 It doesn't replace programmers either, by the way. It just allows the UX designer to play at a much much higher speed than they were before. And it's I think it actually produces significantly higher quality just because you can try so much and you can fail so much that it gives you that opportunity. So there's there's a lot of power there. >> What I feel is that before I was limited by the knowledge of using particular tools.

37:33 For instance, if I was deploying something to the cloud, I used I don't know GCP or Asia or whatever >> because I knew it and that was the one. But there are other ones out there now. It's just an abstraction point. And even if it's not ideal and that's the same thing with with UI, I don't know, I used React. Now I have the ability of using other frameworks easily that I like how they look or even just go and say I like the way how those things are looking.

38:01 Make me something similar but tweak it in that way and that is a lot easier to get to the point that you want. Now about the quality that's a whole different perspective and that's where things need to be improved. I'm just thinking about the issues both on Open AI or or Antropic where they had agents that just went through the guardrails and just created havoc on the internet.

38:22 >> Oh yeah. >> So let's hope for a beautiful future. I guess the closing point I'd like to make is that my mindset is not that these are the answers. It's more that these are the problems and I just want us to see the problems. So often when I talk about the fact that the desktop is dead, I get people to kind of go, but it's a standard, you know, why do you want to blow things up, Scott?

38:49 And I'm like, no. Can we see that we have so much more that we have to do? So that's all I want to focus on is to see the problems, get excited by them, and to explore new solutions. >> Yeah. So, it's about the evolution and we should not comply with the status quo because it's something that we got used to. We have to get out on the comfort zone and just move out and make things better.

39:13 >> It's surprising how this feels so conservative or the world has gotten so conservative. >> Yeah, that's sad. I remember at some point I was reading an article about the movies and music becoming so boring. All of them are just copycats of the other ones because somebody just look into the numerical part. Just took out the soul, took the creative out and everything became money.

39:33 We should take a step back, look at the problem, improve it and then hope for a better future. >> Exactly. >> Thank you, Scott. Have a great day. >> Thank you. Enjoyed it. Bye-bye.