Algorithms For Dummies

4

Struggling with 

Big Data

---

IN THIS PART . . .

Interact with large datasets.

Work with streamed data to use even larger datasets.

Perform tasks in parallel to perform management and 

analysis tasks faster.

Encode data to reduce redundancies and to keep 

data safe.

Compress and decompress data using the LZW 

algorithm.

---

CHAPTER 12

  Managing Big Data 

     225

IN THIS CHAPTER

 

» Realizing why big data is a driving 

force of our times

 

» Becoming familiar with Moore’s Law 

and its implications

 

» Understanding big data and its 4 Vs

 

» Discovering how to deal with infinite 

streaming data

 

» Leveraging sampling, hashing, and 

sketches for stream data

Managing Big Data

M

ore than a buzzword used by vendors to propose innovative ways to store 

data and analyze it, big data is a reality and a driving force of our times. 

You  may  have  heard  it  mentioned  in  many  specialized  scientific  and 

business publications and even wondered what big data really means. From a 

technical point of view, big data refers to large and complex amounts of computer 

data, so large (as the name implies) and intricate that the data cannot be dealt 

with by making more storage available on your computers or by making new 

computers more powerful and faster in their computations. Big data implies a 

revolution in the way you store and deal with data.

However, this copious and sophisticated store of data didn’t appear suddenly. It 

took time to develop the technology to store this amount of data. In addition, 

it  took  time  to  spread  the  technology  that  generates  and  delivers  data,  namely 

computers, sensors, smart mobile phones, the Internet, and its World Wide Web 

services. This chapter discusses what drives this huge data production.

Even though it took time to build this much data, technology changes in recent 

years have finally helped people realize the potential game changer that having 

huge amounts of data (of any nature) at hand represents. For centuries, humans 

have emphasized the power of the human intellect in determining the causes and 

forces driving the natural world using a few accurate observations (small data). 

Chapter 

12

---

226

 

   

  PART 4