Knowing When Not to Use AI: AI Agents vs Rules vs ML Transcript, AI Summary & Key Points

00:00 It's incredibly easy to fall into  the trap of using AI for everything. Many teams are starting to integrate AI agents  all across their systems. And why wouldn't they? AI agent systems are undeniably powerful. There's  no doubt about it. But we need to stay strategic about where and whether we want to inject AI  into our systems. AI is not magic and it's not the best solution for everything.

00:24 Just like  any engineering decision, choosing to use AI or not and at what scale involves trade-offs  across accuracy, cost, complexity, and risk. Most problems in software and data systems can  be solved in one of these four ways. Humans for humanity and judgment. Rules or codebased  solution for deterministic logic. machine learning for statistical pattern recognition and generative  AI systems for flexible reasoning and generation.

01:06 These aren't some ladder that you progress through  from human to agents. In fact, they are distinct options each with a very specific role. The goal  is to pick the right system for the right problem. Let's start with humans. I may have a little  bias here, but humans are great at making complex judgments. People can handle ambiguity and  edge cases very well, making complex decisions with incomplete or conflicting information.

01:34 Particularly, they can also apply context, ethics, and accountability in ways that machines  cannot. Humans should drive tasks when there are high stakes decisions, situations  requiring ownership or liability, and cases of significant ambiguity. Some examples  of these are hiring decisions, medical diagnosis, legal interpretation, and large-scale strategy  or financial decisions.

02:01 If something goes wrong, accountability must be with a real person. Now  for the trade-offs. Human judgment is high quality but expensive, slow, and very hard to scale. If  you think about hiring a candidate for a job, you could screen resumes automatically, but  the final decision involves more nuance such as team fit and communication skills, which  are not something you want fully automated.

02:33 On the other hand, you would never rely on humans  to manually review every single transaction on a credit card network. That's millions per second.  So let's look into some of those cases where a code or rules-based deterministic system might be  beneficial. In particular, a traditional software solution is good at executing clear, stable logic  and producing consistent exact outputs at scale.

03:01 Rule-based systems are ideal to use for  tasks when logic is known and explicit. For example, if X do Y. Requirements  are stable and errors are unacceptable. Some examples of these are payment processing,  input validation, data transformations, and most importantly, security and access control. Code  is fast, cheap, reliable, and very interpretable.

03:31 It's generally straightforward to test, debug, and  scale. If I want to process a credit card payment, for example, I want a code system to check if  balance is sufficient and approve or decline. This should be as fast as possible  deterministic and have no room for error. Using AI in this case would introduce unnecessary  risk. Same for validating an email address format.

03:56 You don't need a model. You need a simple rule.  But where fail the rules fail and begin to break down is for cases where conditions are constantly  changing and logic becomes just too complex. Take fraud detection for example. A static  rule like flag any transactions over $1,000 quickly becomes useless against fraudulent  attackers whose behaviors evolve.

04:17 That's where machine learning and statistics come in. Machine  learning excels at learning patterns in structured data and making probabilistic predictions. Machine  learning is ideal to use when patterns exist but aren't very obvious, rules are too complex  to define manually and when you need predictions. Some common areas of use are fraud detection,  customer churn prediction, demand forecasting, and recommendation systems.

04:52 Here's an example  in practice. If I want a system that forecasts product demand, a human might rely on intuition  and bias and a rule or code solutions may be too simplistic to pick up on the nuances of demand  trends. This is where machine learning shines. Able to learn complex trends and patterns in  data, resulting in more accurate predictions and improved model outputs, making it the  ideal system for this particular task.

05:24 Machine learning models scale very well  and are generally predictable. However, they do require more monitoring and ongoing  maintenance as over time there can be model drift and low flexibility outside the initially  defined problem scope. Machine learning models do also have varying degrees of explanability, but  if you needed to write a report on what it thinks the societal causes of these demand trends are,  you're out of luck.

05:47 While the task of analyzing and predicting trends over massive structured data  may not be ideal for your typical LLM, they are good at working with unstructured data and tasks  that involve reasoning, synthesis, and language. Generative AI systems like LLMs and agents  are best to use when inputs are unstructured like text, documents, and other media, tasks  require interpretation or transformation.

06:25 Flexibility matters more than precision and  some amount of error is tolerated. Some common examples are retrieval, augmented generation  for question answering over document sets, large document summarization, code generation, and  multi-step workflows with agents. Take a customer support system for example. Instead of hard-coded  responses to user questions, an AI agent can understand intent, pull relevant knowledge,  and generate a unique and helpful reply.

07:01 Trying to do so with rules alone would be nearly  impossible. Generative AI systems are incredibly flexible and time to success is very fast with  LLMs being such good generalists at tasks. However, the tradeoff is non-determinism  in answers leads to difficulty in testing, correctness not being guaranteed across runs, and  they tend to be significantly more costly at scale than these other tech solutions.

07:28 That is really  the key to knowing whether or not to build agents and LLMs for your tasks. How much non-determinism  and uncertainty are you willing to tolerate? For example, if I want an AI agent  to analyze my spending for the year, I can't just feed in thousands of transactions and  expect it to create a non-h hallucinated answer or even just get the calculations correct.

07:54 Instead, I might want a calculator tool that uses code and math for determinism and traditional  machine learning to identify trends over time. And of course, I might want an LLM to still  generate a nice, clean natural language report at the end. Well, these hybrid systems tend to be  the most successful using all of these types of solutions in conjunction properly balancing the  correct tool for the task.

08:26 When someone says to make everything agents, this is not literal.  All of the best agents and modern solutions use some combination of all of these types of  systems. It's important to identify and think out where each of these are most appropriate to  apply. Here's an actionable heruristic. Use humans when you need judgment, accountability,  and ethics or have high-risk workflows.

09:03 Use rules or traditional code when you need  clearly defined logic as if, if X happens, do Y. And that logic won't change  often. Use traditional machine learning when you want to find patterns and  predictions in past data. Use generative AI when you need to interpret, generate, or reason  over complex inputs. And flexibility matters more than precision.

09:34 In general, what's most important  is matching the technology to your use case. and choosing the right kind of intelligence for  your tasks. Most failures to get to production aren't going to come from bad models, but rather  from choosing the wrong system in the first place. The future of technology isn't about replacing  humans or putting AI everywhere.

09:54 Great systems are built by making disciplined choices about how  decisions get made across humans, rules, models, and machines. The most successful AI systems  likely won't be the ones using the most AI. It will be the ones who consistently choose  the right kind of intelligence for the problem. Because the most important AI skill  is knowing when not to use it.