Big Data Belongs In A Warehouse Not A Silo

4480 Words Jul 20th, 2015 18 Pages
BIG DATA BELONGS 14
Big Data Belongs in a Warehouse not a Silo
In 2012, it was estimated, that human beings were generating around 2.5 exabytes of data every day and that number is likely even greater today (McAfee & Brynjolfsson, 2012). Twitter processes on average about 5,700 tweets per second (Twitter Inc, 2013). All of this data is stored in numerous ranging traditional database tables and spreadsheets to SMS text messages, PDF files, HTML web pages and more. While the value in capturing and analyzing this data is clear, the solution is not. Traditional data warehouse technologies were not designed for this volume, velocity and variety of data, which is collectively referred to as big data.
Some people believe that the answer to challenges posed by big data lie in a relatively new group of non-relational data storage and management products known collectively as NoSQL. However, NoSQL system development is different from traditional data warehouse development in that it is application driven. This has led some pundits to postulate that NoSQL represents a new paradigm in data warehouse design, where highly specialized data silos will replace the traditionally integrated data warehouse. Therefore it is reasonable to ask, should NoSQL be used to build big data warehouses? If yes, then should integration be discarded in favor of autonomous, application driven data silos?
This paper will answer the above questions and review several technologies available for building big…

More about Big Data Belongs In A Warehouse Not A Silo

Open Document