The Information: A History, A Theory, A Flood

A book by Jame Gleick

In 2011 alone, 1.8 zettabytes (or 1.8 trillion gigabytes) of data was created. Currently we create 2.5 quintillion bytes of data per day — so much that 90% of the data in the world today has been created in the last two years alone. This data comes from everywhere: sensors used to gather climate information, satellites keeping track of deep space events, DNA sequencing, posts to social media sites, digital pictures and videos, purchase transaction records, and cell phone GPS signals to name a few. This data is big data.

You can check out a sliver of the data we generate in a single minute in this infograph - but keep in mind that these numbers are from 2011 which means that by now most of them will have increased by a factor of 2. Here's another infograph, one that takes a look at how big data is changing the IT landscape and how it will continue to do so in the future > - annual data generation is expected to increase 4300% by 2020... mindboggling!

The possibilities are limitless. Literally every industry and branch of science will ultimately be affected. Big data can help us to better understand everything from diseases to the environment, it can help alleviate traffic problems and reduce waste, under specific circumstances it even allows us to predict the future. When you take into account developments like Adam, the robot scientist and Eureqa, a software tool that can detect equations and hidden mathematical relationships in your data, the future looks bright indeed. Unfortunately, big data also has a dark side but more on that next sunday.

Adam: http://www.wired.com/2011/04/robot-scientist-language/
Eureqa: http://www.creativemachineslab.com/
Big Data: http://en.wikipedia.org/wiki/Big_data

BBC Horizon: The Age of Big Data
In Los Angeles, a remarkable experiment is underway; the police are trying to predict crime, before it even happens. In London, one trader believes that he has found the secret of making billions with maths and In South Africa, astronomers are attempting to catalogue the entire cosmos. These very different worlds are united by one thing - an extraordinary explosion in data.

Horizon meets the people at the forefront of the data revolution, and reveals the possibilities and the promise of the age of big data.