Machine Learning: 2 Books in 1: Machine Learning for Beginners, Machine Learning Mathematics. An Introduction Guide to Understand Data Science Through the Business Application

3. 
Data Preparation
After the information has been ingested, a centralized pipeline would be
produced that can evaluate the condition of the data, meaning it
would search for format variations, outliers, patterns, inaccurate,
incomplete or distorted information and correct any abnormalities through
the process. The "feature engineering process" is also included in this stage.
The 3 primary characters of a feature pipeline as depicted in the picture
below are: "extraction, transformation, and selection".

---

Since this tends to be the most complicated component of any machine
learning project, it is essential to introduce appropriate design patterns. In
the context of coding, it implies the use of a factory technique to produce
features based on certain shared abstract function behavior and a strategy
pattern for selecting the correct features at the time of execution can be
considered a logical approach. It is important to take into consideration the
composition and re-usability of the pipeline while structuring the "feature
extractors" and the "transformers".
The selection of functionalities could be attributed to the caller or could be
automated. For instance, a "chi-square statistical test" can be applied to
classify the impact of each function on the concept label, while discarding
the low impact features before starting to train the model. To accomplish
this, some "selector APIs" can be identified. In any case, a unique Id must
be allocated to each feature set to make sure that the features used as model
inputs and for impact scoring are consistent. Overall, it is necessary to
assemble a data preparation pipeline into a set of unalterable
transformations, which could be readily combined. Now the importance of
"testing and high code coverage" will become a critical factor in the success
of the model.