Top 5 AI Building Blocks Every Engineer Should Know Transcript

00:00 Most AI products today, from ChatGPT to Claude Cole, or even Waymo's self-driving cars, are built from the small set of core components. Today, we'll walk through five building blocks you will see again and again in your real AI systems. One, large language models. The LLM is what made the early ChatGPT experience possible. It is a neural network based on the transformer architecture that can predict the next word given a sequence of words.

00:26 Frontier LLM training has three stages: pre-training, SFT, and RL. In pre-training, the model reads trillions of words from the internet and learns grammar, facts, and the other basics. In SFT, it learns to actually answer questions by learning from curated question and answer examples. Finally, in the RL phase, the model practices. It produces multiple responses and prioritizes the answers preferred by humans.

00:55 LLMs are the most important building block in AI systems today, as they are the brain of the entire system. They can process the context, understand what it means, and decide what should be done. Two, embedding models. Embedding models are also neural network but trained with a different objective. Instead of predicting the next word, they map an input into a high-dimensional space called the embedding space.

01:20 These models are trained on millions of pairs of data such that if two samples are related, they end up nearby in the embedding space. This property is important because it can enable search and retrieval. At runtime, we can map a user's query into the same embedding space and find other data similar to it. Most search systems are powered by embedding models.

01:43 For example, in Google search, when you search for cheap flights to Europe, pages about affordable airfare still show up because your search and the pages are mapped into the same embedding space. Three, retrieval-augmented generation. LLMs knowledgeable and can answer questions based on what they learn from their training data. But they cannot answer specific questions about your internal documents.

02:10 RAG fixes that gap by combining an LLM with retrieval. It first create a knowledge store from your internal docs using an embedding model. Then at runtime, instead of directly sending the user query to the LLM, it embeds the query and retrieves parts of the internal docs that are relevant to this query. Then the user's query and the retrieved content all go to the LLM for a response.

02:34 Products like Notebook LLM work this way. You upload your documents and then ask questions. The system processes your docs and answers come only from those files with citations to the passages used. RAG is a very effective and cheap way to give a model knowledge it was never trained on and to build custom systems that work well on your data. Four, agent harness.

02:57 LLMs can answer most questions, but if a question needs up-to-date information like sports events this weekend, they cannot answer alone. Many queries require web search or running code. An agent harness is a system we build around the LLM that handles those things and gives more agency to the LLM. It safely executes functions, call external tools, and returns the results to the LLM.

03:23 It keeps repeating this until the LLM has a final answer. Along the way, it manages the model's limited context window, keeps memory between sessions, and decides which actions need human approval. The harness is what make powerful agents like Codex, Claude Co, Anthropic Co work, and ChatGPT possible. Their main value is indeed the system built around the LLM that handles everything.

03:52 Five, world models. LLMs can predict the next word, but they probably don't understand physics or how objects interact. They cannot produce the next state in environment. World models are a different class of neural networks trained with a different objective, to predict what happens next in an environment, usually in the form of video. During training, they learn how objects move and interact by watching enormous amounts of video and predicting upcoming frames.

04:20 The neural network takes the current state, say the last few frames, >> [music] >> and optionally an action, say moving right, and it predicts the next few frames. World models have lots of important applications. DeepMind's Genie 3 generates interactive 3D worlds you can walk through in real time, powered by their world model. Nvidia's Cosmo models let robots and autonomous vehicles train in simulation before touching the real world.

04:47 Many AI systems are built on top of these building blocks. Coding agents are LLMs plus an agent harness. Robots can be powered by world models and LLMs. Most recommendation systems, like YouTube's and TikToks, are powered by embedding models. An internal support bot is likely a rack system. Understanding them and having them in your tool set helps you build your AI application more easily.