Hands-On Deep Learning for Images with TensorFlow

[ 32 ]
Summary
In this chapter, we learned about the MNIST digits, and how to acquire them; how tensors
are really just multidimensional arrays; how we can encode image data as a tensor; how we
can encode categorical or classification data as a tensor; and then we had a quick review
and a cookbook approach to think about dimensions and tensors to get data prepared for
machine learning.
Now that we've learned how to set up our input and output data for machine learning,
we're going to move on to the next chapter, where we will create a Classical Neural
Network (CNN).

---

3
Classical Neural Network
Now that we've prepared our image data, it's time to take what we've learned and use it to
build a classical, or dense neural network. In this chapter, we will cover the following
topics:
First, we'll look at classical, dense neural networks and their structure.
Then, we'll talk about activation functions and nonlinearity.
When we come to actually classify, we need another piece of math, 
softmax
.
We'll discuss why this matters later in this chapter.
We'll look at training and testing data, as well as 
Dropout
and 
Flatten
, which
are new network components, designed to make the networks work better.
Then, we'll look at how machine learners actually solve.
Finally, we'll learn about the concepts of hyperparameters and grid searches in
order to fine-tune and build the best neural network that we can.
Let's get started.
Comparison between classical dense neural
networks
In this section, we'll be looking at the actual structure of a classical or dense neural network.
We'll start off with a sample neural network structure, and then we'll expand that to build a
visualization of the network that you would need in order to understand the MNIST digits.
Then, finally, we'll learn how the tensor data is actually inserted into a network.

---

Classical Neural Network
Chapter 3
[ 34 ]
Let's start by looking at the structure of a dense neural network. Using the network
package, we will draw a picture of a neural network. The following screenshot shows the
three layers that we are setting up
—
an input layer, an activation layer, and then an output
layer
—
and fully connecting them:
Neural network with three layers
That's what these two loops in the middle are doing. They are putting an edge between
every input and every activation, and then every activation and every output. That's what
defines a dense neural network: the full connectivity between all inputs and all activations,
and all activations and all outputs. As you can see, it generates a picture that is very
densely connected, hence the name!
Now, let's expand this to two dimensions with a 28 x 28 pixel grid (that's the input
network), followed by a 28 x 28 pixel activation network where the learning will take place.
Ultimately, we will be landing in 
10
position classification network where we'll be
predicting the output digits. From the dark interconnecting lines in the following
screenshot, you can see that this is a very dense structure:

---

Classical Neural Network
Chapter 3
[ 35 ]
Two-dimensional network
In fact, it's so dense that it's actually hard to see the edges of the individual lines. These
lines are where the math will be taking place inside of the network. Activation functions,
which will be covered in the next section, are the math that takes place along each one of
these lines. We can see from this that the relationship between the tensors and networks is
relatively straightforward: The two-dimensional grid of inputs (the pixels, in the case of this
image) are where the two-dimensional encoded data that we learned about in the previous
section will be placed. Inside of the network, math operations (typically a dot product
followed by an activation function) are the lines connecting one layer to another.