Machine Learning: 2 Books in 1: Machine Learning for Beginners, Machine Learning Mathematics. An Introduction Guide to Understand Data Science Through the Business Application

Data exploration
The data set must be scrubbed to remove any discrepancies and errors
before it can be used to train the Data models. To check the data quality and
gathered information required to process the data before modeling, tools
such as data summarization and visualization should be used. Since this
process is repeated multiple times, an automated utility called "IDEAR",
which is provided by TDSP, can be used for Data visualization and creation
of Data summary reports. With the achievement of satisfactory quality of
the processed data, the inherent data patterns can be observed. This, in turn,
helps in the selection and development of appropriate "predictive model"
for the target. Now you must assess if you have the required amount of data
to start the modeling process, which is iterative and may require you to
identify new data sources to achieve higher relevance and accuracy.
Set up a data pipeline
To supplement the iterative process of data modeling, a standard process for
scoring new data and refreshing the existing data set must be established by

---

setting up a "data pipeline or workflow". The solution architecture of the
data pipeline must be developed by the end of this stage. There are three
types of pipelines that can be used on the basis of the business needs and
constraints of the existing system: "batch-based", "real-time or streaming"
and "hybrid".
Deliverables to be created in this stage
Data quality report – This report must include "data summary",
the relationship between the business requirement and its
attributes and variable ranking among other details. The
"IDEAR" tool supplied with TDSP it's capable of generating data
quality reports on a relational table, CSV file or any other tabular
data set.