Machine Learning: 2 Books in 1: Machine Learning for Beginners, Machine Learning Mathematics. An Introduction Guide to Understand Data Science Through the Business Application

Dimensionality Reduction

---

When you are using dimensionality reduction, you are trimming down data
to remove unwanted features. Simply put, you're scaling down the number
of variables in a dataset.
When we have a lot of variables in our model, then we run the risk of
having dimensionality problems. Dimensionality problems are problems
that are unique to models with large datasets and can affect prediction
accuracy. When we have many variables, we need larger populations and
sample populations in order to create our model. With that many variables,
it’s hard to have enough data to have many possible combinations to create
a well-fitting model.
If we use too many variables, then we can also encounter overfitting.
Overfitting is the main problem which would cause a data scientist to
consider dimensionality reduction.
We must choose data that we don’t need, or that is irrelevant. If we have a
model predicting someone’s income, do we need a variable that tells us
what their favorite color is? Probably not. We can drop it out of our dataset.
Usually, it's not that easy to tell when a variable should be dropped. There
are some tools we can use to determine which variables aren’t as important.
Principle Component Analysis is a method of dimensionality reduction.
We take the old set of variables and convert them into a newer set
somehow. The new sets we’ve created are called principal components.
There is a tradeoff between reducing the number of variables while
maintaining the accuracy of your model.

---

We can also standardize the values of our variables. Make sure they are all
valued in the same relative scale so that you don't inflate the importance of
a variable. For example, if we have variables measured as a probability
between 0 and 1 vs. variables that are measured by whole numbers above
100.