The proposed table organizes data maturity in rows, from raw data toward insights, and analytical activities in columns, from data acquisition through evaluation.
ETL is presented as the first element associated with raw data.
It is described as the second step in the prepared-data row.
The numerical representations allow these forms of data to be used in later analytical operations.
The proposed workflow places regression after data cleansing and uses the resulting relationships to support generation of additional data.
The technique is presented as a way to cross-validate models rather than relying on a single fixed training-and-testing split.
It is presented as a way to make the outputs or decision processes of models more interpretable.
Monitoring drift helps identify how the data or the performance of a model changes as time passes.
The transcript connects prior knowledge and probability distributions to the creation of predictions.
Repeatedly resampling the data is presented as a way to assess the uncertainty around results.
The proposed table places structured data in the model-data stage.
The transcript presents this organization as supporting validated insights.
The reduction is described as compressing and simplifying data while retaining the information that matters most in terms of variance.
The combination of models is presented as a way to produce higher-quality insights than relying on a single model type.
The scenarios can use ensemble models to examine alternative results.
The transcript describes aggregation as a way to derive summaries from data.
The transcript links clustering to statistical summaries and describes density-based estimates as one possible related technique.
It is presented as the entry point for the quantum addendum to the data science table.
These are presented as three alternative encoding elements within the quantum section.
The transcript describes the approach as hybrid rather than exclusively quantum.
The states are presented as generated quantum data for exploring quantum systems.
These measures are presented as ways to assess quantum-system predictions.
Searchable transcript of Data Science Periodic Table Explained: ML, ETL, Analytics & Workflow — IBM Technology (08:51). Search for a phrase, then click its timestamp to jump straight to that moment in the video.
Captions sourced from the original video on YouTube, published by IBM Technology. The video, its captions and all related intellectual property remain the property of their respective owners; AINotes claims no ownership. Provided for research, accessibility and search — see the Transcript Notice and Copyright Policy.
Data science is a deep field of applied study, but is it really that deep and confusing? Terms such as these, cross validation, drift, clustering, statistics, principal component analysis, well, they're all parts of the anatomy that makes up the modern day data science. But how do all these data science pieces really fit together? Well, what if we could organize these data scientists elements into a data science periodic table, like how Martin organized an AI periodic table?
So just like chemistry, this could all help us relate these terms together. So welcome to the data science periodic table. Now this periodic table, it's organized into rows and groups. Now, since data science is all about the data, the rows show the maturity of data through the progression of raw data all the way down into insights. Now the groups over here, or the columns, They represent the type of analytical activity from acquiring the data all the way to evaluation side.
Now together, each cell, it shows a specific data science element applied at a particular stage of the analytics lifecycle. Now, a quick disclaimer. So there really is no official data science periodic table like there is in chemistry. Now this is my take on what the structure could look like. But once you understand it, you can decode any data science project.
Any product demo or any vendor pitch. You'll see which elements they're using, how they connect, and maybe even what might be missing. You can even use this to build your own data science system. First, you do know that data science is about the data, right? So that gets us to our first element or Et here. This is for extract, transform, and load. Now this element, what it does is it moves raw data from sources into a centralized system.
Now this could be. A database or a table within that database, or even an unstructured pile of data. Now, as we look at this row called raw data, all of the elements, they are related to the raw or unrefined data. Now, this is the closest that we're actually going to get to the original data. Now, at the group level here, we'll see that the next element is called Di.
Now, This represents data ingest. This element is atomic streaming or batch operators for processing the data. Now, this is the second step of the row two, which we then call the prepared data. Now, what can we do with this prepared data along the groups? Well, the next one that's gonna be over here is for data encoding, which we'll label En. Now this element, it converts the categories, the text, or even the dates into numerical representations.
But before we go on this group, let's finish the top row here. So from this group we're going to go to the next one which is called data cleansing, which I'm going to label that as Cd, right? This type of finishing is further refined by the next element, which is then called Re. And we call this one regression. So this estimates the relationships between variables using these regression techniques.
So now we can also generate additional data after we understand those kinds of relationships with the next element, which is called Sy. And this stands for synthetic data. Now, finally, we're getting to group five, evaluation. And if we look at that element, we're gonna call this one. And this stands for really metrics and evaluation here. And this is the start of the refined data rows.
So let's continue down group five evaluation. So the next element is called Va. And the Va stands for cross validation. Now this is a method for cross validating models or robustness checks by rotating training and testing slices of data. But even if we apply elements of Me and Va, we still need to progress to the next element, which is then called Ex.
And Ex, it represents explainability, which then in turn explains that model behavior or feature importance and predictions. Now, as these can change over time, the element Dr, which means drift, right? This helps us to understand how shifts in the data or model performance, how it differs over time. Now, some of these models are represented by the element Ba here.
And this is a Bayesian model, right? And uncertainty can be modeled by distributions and incorporates prior knowledge for the creation of these kinds of predictions. But now to finish up this row, we then have what's called Bo, bootstrapping. Now, this element, it creates resampled data sets to estimate variability. Or confidence intervals along your data.
So let's go to the beginning of row three, model data. So the very first element here is called St. And this structured data, it organized data into tables, schemas, or graphs for easier use. Now the last row over in this group here is called validated insights. And looking at it, the element is Go. For data governance. Now, it's very important to define rules to ensure that the data quality, security, and compliance all match up.
And this level of organization really helps us to get into the validated insights. Now, as we continue across the groups, here we now have PC. And PC stands for principal component analysis. It helps us really to reduce the dimensionality of the data while maintaining the highest variance. This helps us to compress and simplify data while really keeping and understanding what really matters.
And to produce the high quality insights, the element Es, which means ensemble it has these systems to put different types of models together that could vote on a particular outcome. We can even use these types of models within the next element, which is called Si. And Si means simulation, to create hypothetical scenarios to explore all the different possible outcomes.
Now looking at the next one, so we'll take and add in what's called Ag. This is called aggregation, where we can apply summarization, which really is this type of aggregation methods to find counts, means, or other statistical analysis techniques. Now these types of statistics lead us to clustering or Cl. This is an unsupervised method to find natural groupings or patterns within that data.
In fact, we can use density-based estimates or other generation techniques such as Dg. All right, and we call this distribution generation. There's still a section that's just outside of this table. It's outside the realm of classical computing. It's a quantum addendum. So here we can start with Qa. It's quantum accessible memory. So this element ensures that we can move the quantum or classical data into or out of quantum accessible circuits.
Then we can progress to Qe. And QE is all about quantum encoding. This encodes the classical data into qubits using three elements. So first we have amplitude, basis, or angle encoding. Next, we can look at Qo. And QO is all about quantum modeling here. So it uses a combination of qubits and classical techniques for the implementation of quantum machine learning.
Now, the following one is called Qs . And we wanna do this and use quantum to create synthetic quantum states that can help us test or even run different simulations. But we still need to be able to evaluate these quantum systems. And that's where the next element comes in. So we call this one Qn. And this helps us to measure the quantum prediction, the accuracy, fidelity, or even loss.
And there we have it. With this data science periodic table, data science stops being a jumble of terms and becomes now a structured landscape that you can navigate. Each element now has a context and a purpose, giving you a clear lens to explore, connect, and apply these techniques with confidence.