Why Risk Should Determine Your AI Architecture Transcript, AI Summary & Key Points

00:00 I was in a conversation recently with a public sector leader. They told me AI is so easy to build now. We can just have an AI build an AI and they're not wrong. A lot of things are easy to built badly because there's easy AI and there is risk appropriate AI. And those two, they are not the same thing. Most teams approach this backwards. They build the system then ask who should oversee it?

00:29 And when governance fails, when someone needs an explanation, when the model drifts, when a person or a reputation is harmed, they realize the architecture was never designed to support the oversight they claimed to want. This is the relationship that matters. Risk should inform requirements. Requirements should inform architecture, not the other way around.

00:57 Let me show you why this matters with something real. My favorite definition of the word data is that it is an artifact of the human experience. Imagine you're walking down the street and you find a piece of paper that looks like it's been ripped out of a spiral notebook. That's an example of data, right? There's a data up at the top, maybe some personal notes.

01:22 You walk a little bit further. And you find the notebook it was ripped out of. You can see what was written before and you can see was was written after. Now you have context. You know exactly where that page came from. Data plus context equals information. Then you turn the book over and you see your sister's name on the cover. You know it's her diary.

01:49 You understand exactly who wrote it. Data plus context plus relationships. Equals knowledge. We as humans, since the dawn of humankind, we have imparted wisdom from generation to generation through storytelling. This combines data, context, relationships through stories. Data does not equal information. Information does not equally knowledge. And most AI systems are operating somewhere in that first category, pretending.

02:28 That they're the third. Now I'm gonna prove to you that oftentimes just having the data is not enough. What I want you to do, I want ya to take out your cell phones and go to the bookshelf in your house or your apartment, take a picture of it, and upload it to your favorite general purpose large language model. Ask it to guess what your job is. My bookshelf, I've got books about AI and data science, I've gotta lots of books on science fiction, I have an entire row on cryptography.

02:58 History, I like to paint and I like to weld so I have books about painting and welding and oh yeah, I think I have some Calvin and Hobbes in there. The AI decided I work for the NSA. I guess it over-indexed on the cryptography books. I do not work for NSA and if I did, I would not confirm it to a chatbot or ever talk about it on one of these videos.

03:19 The AI, it saw the data. It saw the books. But it couldn't understand why certain books were there. And it assumed all of those books are mine, that is the gap between pattern recognition and knowledge. And when your AI use case depends on closing that gap, on explaining why a decision was made and not just what it is, a purely probabilistic system that only sees the data without context and relationships may not be enough.

03:55 When an auditable explanation is required, that is not a model issue, that is an architecture issue. Most teams, they have AI principles, fairness, transparency, explainability. They're on the website, they're in the framework document, they may even have their own font. A principle is a statement of intent. You cannot build a system from statements of intents.

04:21 You cannot audit one, you cannot put one in a contract. What principles need is operationalization, specific functional and non-functional requirements that are risk calibrated. This is language that a builder can use to implement, to know what to build. Language that a buyer can contract for. Language a governance board can own and review. Let me show you how by giving you an example with just one principle, explainability.

04:55 But ultimately, you would do this with all of your principles. I'm going to start with baseline. Baseline explainability. Imagine you're interfacing with Netflix and it recommends a TV show for you to watch. And you're wondering, why? Why is it recommending this show? Baseline Explainability might be for the AI to just say, because of your prior viewing history.

05:17 Baseline is fine when the consequence is 90 wasted minutes. An enhanced version of that same scenario. Is that Netflix might say in response, people who watch the same five shows you rewatched at 2 a.m. Also watch this. Yeah, it can show your sources, maybe some data lineage in provenance, but you still can't show the reasoning, you still cant show your work.

05:44 Vigilant explainability, this is for your highest risk use cases. Imagine you've got a nurse in the field. That is using AI to figure out how to triage patients, right? Those outputs, those explanations, they're gonna need data lineage, data provenance, test, retest, reliability, and a traceable explanation tied to the recommendation from this patient in this moment.

06:13 That is a fundamentally different architectural burden. Not every use case carries that same level of risk. Not every principle applies to every single system. The level of rigor, whether you choose baseline or enhanced or vigilant, the level of rigor you need depends entirely on the stakes. So when someone tells you AI is easy to build right now, the question is not whether they can build it.

06:42 The question is, can they build it in the way this use case actually requires, given the risk? We don't argue about build cost. Do we ask? Do you have the skills to build this AI in this way, given the risks, to build it for failure, to red team it, to structure accountability before the deployment, not after? The teams that govern AI well understand one thing really clearly.

07:11 The risk level that you assign to a system determines the architecture that you have to build. It determines the team with the skills. You need to build it right. It determines the skills you need for the people who are ultimately gonna be governing this AI and even defining the systems around this AI. If the risk is low, probabilistic AI with baseline explainability might be totally fine.

07:39 But if the risk is high and people need a decision that is traceable, contestable and defensible, your architecture must carry that kind of burden. For high stakes use cases, right, clinical decisions, benefits determinations, public safety, none of this is aspirational. This is what the requirements demand. The question is not whether you can build AI this afternoon.

08:06 The question, is whether you can still defend what you build and how you built it six months later. Risk should determine your AI architecture, not budget. Not urgency, not what's easy to build, risk.