Thread
The Core Components of ML Architecture
ML solutions are used to solve a wide variety of problems, but in nearly all cases the core components are the same.
Here is a visual representation of these core components 👇
A Thread 🧵👇
ML solutions are used to solve a wide variety of problems, but in nearly all cases the core components are the same.
Here is a visual representation of these core components 👇
A Thread 🧵👇
1. Data Generation:
Every machine learning application lives off data. Usually, it's generated by one of your core business functions.
Every machine learning application lives off data. Usually, it's generated by one of your core business functions.
2. Data Collection
Data is only useful if it's accessible, so it needs to be stored in a consistent structure.
Data is only useful if it's accessible, so it needs to be stored in a consistent structure.
3. Feature Engineering Pipeline:
Algorithms can't make sense of raw data. We have to select, transform, combine and otherwise prepare our data.
Algorithms can't make sense of raw data. We have to select, transform, combine and otherwise prepare our data.
4. Training:
We apply algorithms, and they learn patterns from the data. Then they use these patterns to perform particular tasks.
We apply algorithms, and they learn patterns from the data. Then they use these patterns to perform particular tasks.
5. Evaluation:
We need to carefully test how well our algorithm performs on data it hasn't seen before. This ensures we don't use prediction models that work well on ‘seen’ data, but not in real-world settings.
We need to carefully test how well our algorithm performs on data it hasn't seen before. This ensures we don't use prediction models that work well on ‘seen’ data, but not in real-world settings.
6. Task Orchestration:
Feature engineering, training, and prediction all need to be scheduled on our compute infrastructure. So we need to reliably orchestrate our tasks.
Feature engineering, training, and prediction all need to be scheduled on our compute infrastructure. So we need to reliably orchestrate our tasks.
7. Prediction:
This is the heart of ML architecture. We use the model we’ve trained to perform new tasks and solve new problems.
This is the heart of ML architecture. We use the model we’ve trained to perform new tasks and solve new problems.
8. Infrastructure:
The solution has to live and be served somewhere. This will require setup and maintenance.
The solution has to live and be served somewhere. This will require setup and maintenance.
9. Authentication:
Authentication is required to keep our models secure and makes sure only those who have permission can use them.
Authentication is required to keep our models secure and makes sure only those who have permission can use them.
10. Interaction:
API, UI or a CLI interface to interact with ML Model
API, UI or a CLI interface to interact with ML Model
11. Monitoring:
We need to regularly check our model’s performance. This usually involves periodically generating a report or showing performance history in a dashboard.
We need to regularly check our model’s performance. This usually involves periodically generating a report or showing performance history in a dashboard.
These are 11 core components of ML architecture.
I hope you like this thread. I curated this thread with help of the datarevenue blog. you can read more about it here: datarevenue.com/en-blog/machine-learning-project-architecture
follow me at @itsafiz for more such content.
I hope you like this thread. I curated this thread with help of the datarevenue blog. you can read more about it here: datarevenue.com/en-blog/machine-learning-project-architecture
follow me at @itsafiz for more such content.
Mentions
See All
Jaydeep Karale @_jaydeepkarale
·
Nov 30, 2022
Superb explanation 🚀🚀🚀