Information Overload

The amount of digital data that exists in the world is growing at a staggering rate, according to a new report issued by the University of California at Berkeley. The results of the report lend credence to the need for better analytics and data mining tools. "The world's total production of information amounts to about 250 megabytes for each man, woman, and child on earth," Peter Lyman and Hal Varian, senior researchers at Berkeley, write in their report. "It is clear that we are all drowning in a sea of information. The challenge is to learn to swim in that sea, rather than drown in it. Better understanding and better tools are desperately needed if we are to take full advantage of the ever increasing supply of information." The report posits that in total between one and two exabytes--an exabyte is one billion gigabytes--of information are produced per year. An exabyte is so large that all printed documents made in one year comprise about .03 percent of the total amount of data produced, the report says. So how can individuals and enterprises benefit from this vast amount of data instead of drown in it? Mark Schwartz, managing consultant with Accelerated Consulting Group, says that soon, all businesses will be adding business intelligence tools to swim instead of sink. "Companies are drowning in data, but thirsty for knowledge," he says. "The tools exist to allow users to look at data in an aggregate form and see only the exceptions. For example, when sales rise or drop significantly and then take action," Schwartz says. "The tools allow you to drop down in detail from the trend to the individual customer records, and manage the causes for changes in business." Other key findings of the report include the fact that individuals are becoming more and more responsible for the increases in data output. "A century ago the average person could only create and access a small amount of information," Lyman and Varian say. "Now, ordinary people not only have access to huge amounts of data, but are also able to create gigabytes of data themselves and, potentially, publish it to the world via the Internet." The report also notes a growing "dominance of digital content," meaning that a vast majority of data in the world is "born digital" and never makes it to a print form. According to Lyman and Varian, the digital revolution in the film and publishing industries is a major driver of this phenomenon.