AI & Data Science Periodic Tables: How They Work Together Transcript, AI Summary & Key Points

Data science has been around for decades. And today AI is having its moment. But everything happening in AI, it all sits on top of all the work that data science had been doing the whole time. Right. So an AI embedding, well, what is that? I mean, it's just a series of numbers until the data behind it has been cleaned or a large language model is only as good as its training data set.

Right. Now these things rely on good data science, and lately the influence runs the other way too. Now data science they're using AI to label the data and to generate synthetic data. Now, the training sets, they all depend on all of this label and synthetic data that the models sit on top of. Right, so we can really think of this as actually a bit of a loop.

Data feeds the models and lately the models are feeding back into how that data gets... prepared. Now, one thing that's in common between data science and AI is there's just a lot of jargon on both sides. Yeah. No kidding, right? Which is why we each try to organize data science and AI into periodic tables. So how about a quick recap of what the data science periodic table looks like.

Here's the data science periodic table. Now I should point out, there is no official periodic table for data science nor AI. Martin and I came up with these as a learning aid, but the structure is like the periodic table, just like chemistry. So each cell is an element with a name, like extract, transform, and load, which is a two-letter symbol that we call ET.

Now the rows track the maturity of data. So row one is raw data. So it's stuff straight from the source. The extract, transform, load, and the data cleansing, it lives right here. Now, row two is the prepared data where we have data and jest and data and coding. Row three is where the actual modeling now happens. So that's model data, and this is where clustering, aggregation, they're all in this row together.

And now at the bottom is called validated insight where data has been governed and audited. So data governance is in this row. And across the groups is here where we have columns. And the first one is called acquisition. Then we have preparation, modeling, generation, and then evaluation. And then below it, we even have what's called a quantum addendum.

And these are five elements that aren't part of the classical data science, but show up when quantum techniques get involved. We have quantum encoding and quantum metrics are among them. So that's the data science side. Now over to you, Martin. Thanks Aaron, and here is the AI periodic table. And unlike Aaron, I wasn't smart enough to come up with a quantum addendum for mine.

Anyways, this has got five groups across the top and then four rows down the side. So I'll start with the groups. We've got reactive as one of the groups, so these are things that take input and they act on it, like agents, for example, that's kind of the obvious one. Column two is retrieval. So this is essentially the memory side of AI. So we've got things like embeddings here, which is how we encode information into numbers and where we store those numbers, such as in the vector database.

Column three, that's orchestration. So this how the pieces coordinate, things like RAG. And then column four is validation, which relates to anything keeping the system honest, like guardrails and red teaming. And then the final one is models. So LLMs of course, but we've also got things like thinking models that reason before the answer. Then for the rows, we've got primitives, which are the atomic building blocks.

There's only three in this table, prompt, embed, and LLM. Then we've all got the composition as well. So things like function calling. This is the stuff that runs in production. And that lives here in deployment. So under deployment, we've got things like a framework, for example. And then we have just kind of anything shiny and new, still gaining traction.

That's grouped under emerging, like for example, multi-agent systems. So that's the AI side. But now let's put both tables to work. So periodic- tables predict reactions. And a reaction is a real working system built from elements on these tables. Now let's work through an example. What shall we go with, Martin? Well, how about a favorite of every enterprise AI adopter?

Document Q&A, chat with your documents. Right, so let's say a finance analyst at a large company types into a chat, how do the value of generative AI tokens relate to the investment we're making in digital and human workers? Now the concern here is left to its own devices and the AI model might hallucinate a response. We need to make sure that the answer is grounded in the actual company documents and preferably with citations so we can be sure about that.

And it cannot leak anything outside of the analyst's permissions. Well, I think we can build this with the 12 elements that we have from both of our tables. So it'd be six on my side and six on your side. All right, well, you go first, because I think you're going to tell me that good data science is where this starts. Well, indeed I am. So let's start with element one, which is Et.

Now this stands for Extract Transform and Load. So if we have HR and Generative AI policies, they're all scattered around the place. It could be in SharePoint, Confluence, file shares, maybe even some obscure wikis. But Et will pull them all into one place. And Et happens how often? Like once? Well, I'm getting to that, and... That's where the next element comes in.

So we'll say element two, which is Di. And this is data ingest. So policies get updated weekly. So the Di, it keeps the pipeline with the most up-to-date information. Right, we don't want a model passing off like last year's policies as current. Exactly, and then there's gonna be element number three, and we call this CD, which really means data cleansing.

So PDFs come with letterheads, flitters, watermarks, and maybe even some OCR junk that we just don't want. So CD strips this unnecessary stuff. Yeah, garbage in, garbage out, right? Exactly. And what we're talking about here is with this garbage and CV, we want it out. And this gets us to element number four. Now this is called ST. And this is structured data so that each document gets chunked by a section and tagged with metadata like department, effective date, and maybe even sensitivity.

So now that data is queryable? Well, you know, queryable isn't a real word, but I take that the point is now element number five, right? And this gets us to En for data and coding. So now the categorical metadata department, the sensitivity tier language and so forth, it gets converted into a form that the system can now filter on. Now, this is also where data science and coding starts really to look a lot like the AI embeddings.

Now embeddings are from my table, Aaron, so let's just stick to your table for now. All right, all right. But you have this element called Go. So we say G-O, really. But this is one that's about data governance, right? So permissions and audit trails and the like. And that one's super important, right? Because if Go governance isn't enforced, and the system might kind of just like cheerfully cite a document the analyst isn't clear to actually see.

So data science, really, it's done its job. So over to you for the AI elements. Since you already mentioned it... First one we're going to talk about is embeddings, E, M. So each cleaned chunk needs to become a number or more specifically a vector. And embedding turns words into numbers. Yeah, so this is where similar passages get similar vectors. So that's how the semantic search works.

And then once we've created those numbers, well, we need to store them somewhere for later retrieval. So for that, we're gonna use the element Vx that is for vector database. These vectors are now available for semantic search at runtime. So far, all of these have been tasks that happened before inference. Now, getting the data ready to be found by the AI model is really what's important, but what happens at runtime when a financial analyst submits their first query.

Right, so we've done all this work up front, but now people are sending in queries, and the first thing we're gonna do is go to element Rg, that is RAG or retrieval augmented generation. So a question came in on that chat bot, the question got embedded, then the vector database is queried for the closest matches, and then the top chunks are returned.

The element go or GO decides which chunks the analyst is really allowed to retrieve even in the first place. Exactly. Those chunks the ones that are allowed they go to the next element, which is PR. So basically we're going to take those retrieved chunks and embed them with the analysts question into a prompt template. So this is the grounding that we spoke about.

So chunks of these documents that are relevant to the analyst queries are now embedded into the prompts. The model, it doesn't have to hallucinate an answer. Yeah. Exactly and then that's where we bring in the model in. So LG, the large language model, generates an answer that is indeed grounded in those retrieved chunks and then the final stage is GR.

That's guardrails. It's kind of the final filter on the output. So did the cited claim actually appear in the source? Is there any PII that needs redacting before the analyst sees? That sort of thing. So there's our AI and data science pipeline for the document Question and Answering Use Case. Now this is a linear reaction so data flows in one direction, well and then back the other way just because the lightboard's only so wide, but a question goes in and an answer comes out.

Right, so once the answer is delivered, the pipeline forgets it ever even ran. So how does the system get better over time? Well, as it runs, we collect signals about which queries succeed and which ones fail. Right, not every answer is- necessarily going to be a winner. Maybe retrieval returns irrelevant chunks for certain questions. So we're going to add three more elements to turn our linear reaction into a loop.

So the first one is what we call DR, which stands for data drift, or we call it drift in this case. But DR, it watches the distribution of things going through the pipeline, like query embeddings, clustering differently, or even looks at user feedback rates dropping, and then it fires when those distributions move away from a baseline. Then we go to synthetic elements, which really means synthetic data.

So once the DR flags a drift, the team uses AI to generate this synthetic data Q&A pairs covering the failing patterns. Nice. Synthetic data that's in both of our tables actually. So that's pretty cool. And then we go to Ft, that's fine-tuned. So these synthetic Q&A pairs become training data for the embedding model. Each pair tells it that this question should land near this particular chunk in vector space.

So as the model retrains and our internal vocabulary clusters around the right answer chunks, the improved embedding model goes back into EM. Right? So we've now have completed this little loop here. So what the system learns from running is gonna get fed back to make it better and users like our dear analysts get better answers over time without anyone actually manually udating the system.

So which I think nicely makes the point that we raised at the very start. So as important as AI pipelines are, these real-world AI, it runs on good data science. Yep, and both AI and data science have plenty of jargony terms to explain what they do. So hopefully our periodic tables can help make some sense of those. Martin, how about we talk about the quantum periodic table elements?

Yeah, I think that's an entirely new lightboard video. That's a good point. So check out our dedicated videos on the AI and Data Science periodic tables to learn more about those. Thank you.