Data Analytics: Practical Guide to Leveraging the Power of Algorithms, Data Science, Data Mining, Statistics, Big Data, and Predictive Analysis to Improve Business, Work, and Life

Chapter 3: Development of Big Data
While most of our data collection and analysis has only happened in the last
couple of years, the term “big data” has been in our vocabulary since 2005. 
Analysis of data has been around for as long as we could count. 
Accounting in ancient Mesopotamia tracked the increases and decreases of
herds and crops and even then we were trying to find patterns in that data. 
In the 17
th
 century, John Graunt published a book, “Natural and Political
Observations Made upon the Bills of Mortality,” that was the first large-
scale example of data analysis.  It provided insight into the causes of death
at the time, and the book was meant to help stop the Bubonic plague.
Graunt’s book and the way he approached the data was a revolution. 
Statistics, as it is now, was invented at that time, even though we couldn’t
use it fully before the invention of computers.  Data analysis came in in the
20
th
 century when the information age really began.  There were many
examples of early data analysis and collection even in the beginning.  There
was the machine invented Herman Hollerith that could analyze data in
1887; it was used to organize census data.  Roosevelt’s administration used
big data for the first time to keep track of the social security contributions
for millions of Americans.
The first real data processing machine came during World War 2.  The
British intelligence wanted to decipher Nazi codes.  The machine, Colossus,
processed 5,000 characters per second to find the patterns in coded
messages.  The task of deciphering went from weeks to just hours.  This
was a huge victory for technology and a massive improvement for statistical
analysis.
In 1965, the electronic storage of information started, as another idea of the
American government.  The system was put in place to store tax return
claims and fingerprints.  However, the project went unfinished because of
the worries of the American people.  They thought of that as something
similar to “Big Brother,” but the electronic storage of information was
already starting.  It would be impossible to stop the flow of information.
The invention of the Internet was really what sparked the true revolution in
data storage and analysis.  Tim Berners-Lee couldn’t have known what he

---

had really started in the world.  However, it was really in the 90s that his
system was turned into the monster that it is today.  In 1995, the first
supercomputer was made.  The machine was capable of doing in a single
second what a human with a calculator could do in 30,000 years.  This was
the next great stride in data analysis.
In 2005, Roger Mougalas mentioned the term big data as a way of saying
that traditional tools could not deal with the amount of data that was being
collected.  In that year, Hadoop was invented to index the internet.  Today,
this tool is used by companies to go through their own data.
Eric Schmidt said in 2010 at a conference that the amount of information
created between the dawn of the time and 2003 (roughly 10 exabytes) was
equal to the amount of information that had been created in just two days in
2010.  Data had become so ingrained in our everyday lives.  Hundreds of
upstarts are attempting to take on big data.  Thousands of business are using
the data to optimize their business models.  Almost every industry is using
the inferences made by analyzing big data.  That information has become
the most valuable currency in the world; the second most valuable things
are the people that are able to properly use it.
As the world becomes more caught in up in the digital world, it will bring
us closer to each other.  It was also brought more of our lives into the public
eye.  Data collection will become more and more important.  Companies
will be using all of this data to find new ways to sell people products and
services.  There’s no doubt that the government will also be using it to
improve the environment, get votes, and keep the people in check.
Even with big data, the future is still a mystery since it may go either way. 
The future could be changed for the better by big data; the future could also
be hurt by the ways that private corporations are using that data now. 
Having more data out in the open gives more and more power to the
governments.  And one day it may lead to the realization of the people’s
“Big Brother” fears.

---