Teach Your AI Agent to Build Products Everyone Can Use - Jemima Abu, CAIS Transcript, AI Summary & Key Points

00:01 [music] >> Now, I'm going to ask you to do an exercise for me. This is not going to be an interactive session. This is just the one thing I want you to do for me. I want you to take out your mobile phone or your device or whatever laptop you use. And then I'm not hearing people taking out their mobile phones. I can't see you. So, I need to actually hear if you're taking out your mobile phone.

00:30 And what I want you to do is close your eyes. And with your eyes closed, I want you to unlock your phone and go to the notes app and write in hello world. That is what I want you to do for me. Okay. Oh, is that too much? Okay, you know what? You know what? Just unlock your phone. With your eyes closed, write in your lock password. Let's try that. Okay?

00:50 And while people trickled in, we'll see if that works. I'm going to give you 30 seconds to do that. No, right? Exactly. Now, what happened while you were trying to do that? You had no idea what you were doing on your screen, did you? You guys, if you're nodding, once again, I cannot see. So, I'm going to need some auditory help. So, you had no idea what you were doing on your screen, yes or no?

01:17 Oh, we have some yeses. Okay. For people that had the yeses, was it more like muscle memory? So, you're able to swipe up and you remember where your keys are and you're able to use it. Exactly. Now, imagine if this was your first time using that phone and you had no idea where any of those keys were. Would you have been able to carry out the same action?

01:32 No, exactly. And that is what accessibility is. Accessibility is the practice of building products that people can use regardless of disability. Because while you may be lucky enough to be sighted, there are people that for them having their eyes shut is a permanent state. If we are building products that those people cannot use, or if you're building products that people that have motor disabilities, cognitive disabilities, auditory disabilities, and they cannot use that product, that makes it what?

01:56 A useless product. And we don't want to build that. So, let's talk about how we're going to build more useful products. But before we get into that, let's look at some stats. So, 96% of the top 1 million websites online have accessibility issues. What this basically means is that for 96% of websites, [clears throat] they cannot be fully used by people with disabilities.

02:14 Now, why is that? I go around and I talk about accessibility from time to time, so I have the opportunity to talk to a lot of developers. And when I ask them, "Why don't we spend more time making things accessible?" I usually hear it takes too long. So, it's like, "Oh, my manager is not going to sign off on me spending time to make things accessible."

02:30 Or the client just doesn't care about accessibility. Or even, "It's not a priority at my company to make things accessible." But I had a hypothesis that it actually takes more time and it costs more money to build products that are inaccessible. So, because I am a woman in STEM, I had to carry out a little experiment. So, in um I basically prompted my agent to implement this design from Figma.

02:51 The first time was without any accessibility guidance, so I just let it do whatever it wanted to do. And the second time I gave it a skill that I had written specifically for accessibility. And what I noticed that was not only was it faster when it was using the accessibility skill, but there was also a 58% reduction in cost because it was using the accessibility skill.

03:10 Let's take a look at some data. So, without the accessibility skill, it took it 15 minutes to run the prompt. With the accessibility skill, it was just 5 minutes. That's a lot of tokens, as you can see. And that's a little less tokens, as you can see over there. And all in all, it cost less. So, what does this tell us? That it is cheaper and faster for AI agents to build accessible agents.

03:30 Actually, it doesn't tell us that because this was a one-off experiment in not a constrained environment. So, it's not empirical data that says it's cheaper and faster, but the point is when you provide skills to agents that tell them, "Do this thing this specific way." it's going to be faster and it's going to save you a lot more money in the long run.

03:48 Hi, I'm Jemima. I'm a product engineer at Case. I'm also a Google Developer Expert, a Microsoft MVP, and a certified professional in accessibility core competencies. I like to go around talking about how to make the web accessible. Basically, I'm a very dangerous combination of a middle child and a formally gifted child, so I'm constantly seeking validation and I crave attention as well.

04:06 So, that brings us here. So, let's get back to the talk. This is your AI agent. Every popular model that you're using right now, ChatGPT, Claude, Copilot, they were all built 3 years ago. And of course, they're all very smart. The model models are building very quickly, but at the same time, they're just a baby. There's a Japanese show that I really love called Old Enough.

04:26 It's basically about children being sent on their first errand. And so, it can be like a 2-year-old girl, and they ask her to go to the supermarket. And one thing I really like about the show is that whenever the parents send that child on the errand, they always give them very specific guidelines. So, it's like maybe the child has gone to the supermarket with her mother multiple times, or they'll have a helpful little map that says, "This is how to get to the supermarket.

04:46 Don't go into the trees. Don't get lost. And this is the shopping list that you need." This is the exact same thing you want to have in mind when you're prompting your agent. You want to make sure that your agent is not getting lost in where you don't want it to go. So, if you're building products, you don't want your agent to get lost generating things with a lot of divs, for example.

05:03 You want it to follow the prompts that you're giving it, and you want to make sure that the finished result is exactly what you're looking for. So, let's talk a little bit about the cost of inaccessible code. So, what exactly do we lose when we let our agents generate code that people can't use? Firstly, human bias becomes AI bias. One thing that we can see from the stats that 96% of the websites in the world have some form of accessibility issue or the other is that as humans, as developers, we just don't care that

05:30 much about accessibility. And the fact is people tend to have a very it works for me mindset, you know? So, if I build something and I test it on my laptop, and I'm like, "It works for me. That's great. I'm going to ship it." I'm not going to think, "Oh, what about somebody that's using a screen reader? What about somebody that's using a mouse instead of a key or is using a keyboard instead of a mouse?"

05:49 These things don't necessarily occur to us because we have not been taught that these things should occur to us. And now, the problem is, much like our AI baby, we're teaching it the exact same things. AI looks at code that humans have generated and then it says, "Okay, this is what good code looks like or this is what code should look like." And in turn, AI keeps generating code that isn't accessible.

06:05 Humans keep producing more of that code, and we end up in this very dangerous loop where if we get to the stage where AI is writing most of the code for us, we're effectively shutting out 16% of the world's population who has some form of disability or the other. Secondly, AI likes to write a lot of code. I don't know if anybody has noticed that, but if you tell it to write something and it can do it in five lines, it would rather do it in 500 lines cuz, you know, the more tokens you use, the more money they make.

06:34 So, that's something to keep in mind. So, when you are prompting your agent, you want to make sure that you're telling it to default to the best way of writing things. When it comes to building products for the web or building products in general, there is a popular way to do it, but it's not always the right way to do it. So, for example, let's take an accordion component.

06:51 If I was developing an accordion component with JavaScript, I could write a custom code that uses a lot of JavaScript to develop the accordion component, or I could use the HTML details element, which is the exact same thing for an accordion component, and it's way less code. It's accessible by default, and it's browser tested because it was built by the people that built browsers.

07:09 So, because the details element is not as popular as a custom JavaScript implementation, AI is going to write a custom JavaScript implementation. You as a developer, it's your job to tell AI, "Don't go for what's popular, go for what is best." And that is why part of your skill should also be looking out for this. And then, when it comes to accessibility, it's a lot harder to bake in accessibility on a fully built site.

07:32 So, on the left hand of the screen, we have a blueberry muffin, which is what you would expect a blueberry to look like. On the right hand of the screen, we have Imagine somebody handed you a plain muffin, and they had shoved a bunch of blueberries on top. And they're like, "Here you go, a blueberry muffin." You would be like, "Get that trash away from me.

07:47 What is wrong with you?" That is exactly what happens when somebody builds a complete website and then makes it accessible at the end. You cannot shove a bunch of area labels into a product and call it an accessible website. Much like a blueberry muffin, you need to bake it in from the start. Thirdly, people lose customers when you don't make things accessible.

08:06 So, one thing I've noticed from going to a lot of um conferences and speaking on accessibility is that people that have disabilities, they just go to use products that work for them. So, for example, someone um at a conference I was at recently, somebody was talking about a banking app they were using. I can't remember which one it was, and I don't want to besmirch the poor bank's name if I got it wrong.

08:23 So, bank A. And they were basically saying how bank A wasn't accessible. So, they ended up moving over to bank B just because they could not use the product of bank A. So, it does not mean that bank A did not have better features, it was a bank as a whole, but it just meant the user was not able to use their product. So, they went somewhere that could cater to their product.

08:41 So, when you build things to be accessible, you allow people that can use those products use it. And one thing that I always hear, which is so funny to me, is that a business will be like, "Oh, but none of our users rely on assistive technologies." My dude, none of your users rely on assistive technologies cuz they cannot use your products. When they get there and they cannot navigate the product, they leave the site.

08:58 Ergo, you have no users. It's like, imagine a business that didn't have a door. I was like, "Oh, nobody comes into our store." Yeah, why do you think that is? They can't get in the building. Oh my god, it's crazy. So, yes, make sure your things are accessible and stop assuming that your users don't use assistive technologies cuz you wouldn't know, they're not there in the first place.

09:18 And then finally, there are legal repercussions to not building your website to be accessible. In the US specifically, they have the Americans with Disabilities Act that states that any public-facing website must be accessible. And one thing I've learned about Americans is that y'all love to sue because there are so many lawsuits related to inaccessible websites.

09:35 So, make sure you're building your websites to be accessible, or you can use this really fun sticker from Microsoft Teams. So, weigh your options. All right, that being said, the fact is we've gotten to a stage where we're not really writing our own code anymore, right? The agent is doing most of it for us. So, how do we ensure that we're getting our agent to write code that is accessible?

09:59 Firstly, you want to define your accessibility requirements. Every company or every organization has different requirements depending on what field you are in. So, somebody who's a web app is going to have different accessibility requirements than somebody building a mobile app. So, it's up to you as a developer or as the PM or as whoever, even if you're vibe coding, you should know what the accessibility standards are for that app and make sure you're keeping that in mind when you're prompting your agent.

10:23 Secondly, you want to audit your existing code base. If you introduce Have you ever noticed that like if you bring somebody into a room that's messy, let's say you have a roommate, like a roommate's moving in. And then when they're moving they notice that there's a bunch of dishes in the sink. That roommate will never do dishes. Because I'm going to be like, "When I moved in there were dishes in the sink.

10:40 Why would I start doing them now?" Or maybe you all have had better roommates than me. But anyway, if you're introducing your agents into a code base that has bad code or has poorly written code, that agent is going to take that kind of code as the standard. It's going to be like, "Why should I do this when you didn't?" So, make sure you're auditing your existing code base to look out for any accessibility faults in the first place.

11:00 And it's also really easy to do this now because you can tell your agent to do it as well. You can set up an agent that carries out audits on your code base to look for any accessibility issues and then fixes them. Thirdly, you want to implement a design system or some sort of reusable system so that if you're fixing issues in one place, they are going to be fixed across multiple places.

11:18 So, this is just a good standard practice in web development or if you're working in any kind of app development as well, just having a simple library where you can refer to one component and be like, "Oh, this thing has an issue, but if I fix it here, it fixes it across 150 pages." That's a really good practice to have. And then finally, once that is done, you want to make it mandatory for your agents to follow these guidelines.

11:38 So, every single time your agent runs, tell it to make sure that you are building these things in a way that people can use them and not just in the way that I think they should be used. Because it's easy to forget about accessibility when you have 3 hours to the deadline and you told your manager you finished it last week and then you opened the doc and you're like, "Oh I didn't add permissions for that."

11:55 You know, it's easy to forget about these things. So, if your agent has it in mind, it's more likely for it to actually get done. This is a lightweight version of an accessibility skill that you can use. So, this I'll just give a disclaimer, a lot of my prompts are geared towards web web because that is what I do, but like I said, depending on what field you're working on, you can adapt your own prompts as needed.

12:14 This is the light way version of what I use. It's basically telling your agents, "Make sure you're using semantic HTML, only reaching for area, every interactive element must work by keyboard alone." These are basically just best practices or standard practices for web accessibility. Another skill that I really like is the modern web guidance skill.

12:30 This is by a Chrome or Google. I don't know I don't know why they have different names, those people. And basically, it is a skill that tells your agents the newest features to reach for, so kind of the example I gave with the accordion. This skill just tells your agents, "Oh, if you want to build an accordion, you can use details. If you want to build a model, you can use popover."

12:47 Things like that. So, it's a really good skill. And then at the end of the day, I do have to give a disclaimer, which is that just by implementing all these prompts and using all these things, it does not mean that your website is going or whatever product you're building is going to be 100% accessible, but it does mean that it can help. Because at the end of the day, you want to make sure that you are keeping these things in mind because the web is for everyone, so everyone should be able to use it.

13:11 With whatever products you're building, make sure you're building accessibly so everybody can use it. Thank you. >> [applause] >> I don't know about you, I can't see. So, if you have [laughter] any questions, you can be like, "Hey, I have a question." I'll be like, "Yes, please, go ahead." >> Can you talk about your biggest frustration in accessibility?

13:39 >> Can I what? >> Your biggest frustration in accessibility. >> Is that with people making things accessible or when I'm trying to make things accessible? >> What you What you see people >> Yeah, I think one thing is with people making things accessible, they tend to go one of either way, which is that they either don't care at all and then they do things like making a div a button or not putting like keyboard handlers for certain things.

13:59 And their excuse is what I said about how they're like, "Oh, none of our users use assistive technologies." Or sometimes they tend to go overboard and then actually no, I just remembered. I hate those accessibility overlays. I don't know if you've ever gone on the websites and you see like something in the corner. It's like usually looks like an icon of a stick person and then you click it and then it says those things are terrible.

14:17 They're like the blueberry muffin analogy. You're just adding blueberries to your plain muffin. So, those two things I would say I really hate. So, not caring enough about accessibility, or just throwing an overlay on it and acting like that's the fix-all. >> Thank you. >> No worries, Lance.