Comparing Traditional Etl And How A Tool Like Apache Drill

There are certain challenges that are to be faced with this zillion of bytes of data; one of the major is integrating and retrieval of data and incorporating with the enterprise system(Dubois, 2001).

One of the main functions of any business is to be able to use data to leverage a strategic competitive advantage. The use of relational databases is a necessity for contemporary organizations; however, data warehousing has become a strategic priority due to the enormous amounts of data that must be analyzed along with the varying sources from which data comes. Company gathers data by using Web analytics and operational systems, we must design a solution overview that incorporates data warehousing. The executive team needs to be clear about what data warehousing can provide the company.

Data and information management is a huge growth area. But it's not just data management creating new job opportunities, its gathering, analyzing, storing and securing the data as well.

Enterprise view of data is required to maximize the efficiency of the organization as a whole (csu, n.d.). The sharing of data and maintaining the transparency is achieved by integrating the data within the system. Integrating the data is a complex task as there is always certain information that is sensitive and it is not meant to be shared with each and every department or individual working within or outside the organization. Making enterprise data strategy demands administrators to understand that the data is critical and is an asset to the organization. Wayne Eckerson describes enterprise view of data as the enterprise data strategy built by the organization to have a successful growth rate plan. He also points out that only one in 10 organizations have such sort of strategy as most of them don’t put together the required soft skills that are needed to manage change incurred by the strategy and the investment for data management techniques and tools to certify the delivery of high quality and consistent data supported with business initiatives and strategies (Eckerson, 2011). Most of the organizations face a lot of difficulties in maintaining the data quality. The cost of data quality incurred is much higher and it sometimes contributes to forty percent of the problems related to IT in a corporation this problem occurs because of the inaccurate data (steria, 2012). One major issue when creating an enterprise view

With the global marketplace becoming increasingly competitive and the insatiable appetite for business information, the volume of data that must be managed and assimilated is growing at an exponential rate. Global corporations take advantage of the current technologies, and infrastructures, require standard processes, consistent data to enable global consolidation and the ability to transform raw data into business intelligence to support better decision making.

The main goal of this real-time data operation intelligence is to eliminate the data warehousing latencies. By achieving this, the organization can make better decisions hence able to make more rapid decisions.

Organizations need a plan; they need to build roadmaps and swiftly implement the changes. However, before building a plan, solid data are required. The sheer amount of data at our disposal can be paralyzing at times, and organizations need to be able to sift them and find the salient information they require. There are several facts that are currently indisputable for certain categories of knowledge workers, especially technology professionals and engineers.

Data warehousing is one of the hottest industry trends - for good reason. A well-defined and properly implemented data warehouse can be a valuable competitive tool. (Perkins).

There’s no getting around the fact that we are in the data age. Data is coming at us from all directions, and in all sorts of shapes and sizes. As technology leaders, we’re tasked with answering complex questions like: How are we going to process all this information? How can we make it more accessible? And how will we benefit from it?

Presence of big data is a very common phenomenon now days, specially when talking about medium to large size corporation. Manyika et al., in their article (James Manyika, 2011) defined the term big data as “large pools of data that can be captured, communicated, aggregated, stored, and analyzed”. To clarify they suggested that big data refers to data, whose size makes it impossible to be processed by the typical software used for database management. Gartner (Gartner, 2012)defined big data in terms of its characteristics of high volume, high velocity and high variety. By volume, he referred to the size of the data, by velocity he referred to the speed at which the data is created and by variety he referred to the range of types of data.

The data warehouse DBMS market is going through a transformation due to the rise of "big data" and logical data Warehouses. Surprisingly, many establishments entered the data warehouse market in 2012 for the first time, swelling demand for professional services and causing vital changes in vendors' positions.

It helps an organization consolidate data from several sources by separating analysis workload from transactional workload. Additionally, a data warehouse environment includes ETL which is Extraction, Transportation and Loading solution, an OLAP which is Online Analytical Processing Engine, analysis tools and other tools so as to look over the process of gathering data and finally delivering it to business users. The data stored in these warehouses must be stored in a way which is reliable, secure, and easy to process and manage. The need for data warehousing arises as businesses become more complex and start generating and gathering huge amount of data which were difficult to manage in the traditional way.

Data warehouse (DW of DWH) also called enterprise data warehouse (EDW) refers to the system utilized in the analysis and reporting of data. The can be described as the main component making up business intelligence. Normalized data warehousing describes the repositories containing integrated data form several dissimilar sources. It contains information which can be utilized in creating investigative reports for the various users within an organization. Examples of reports that can be retrieved from these repositories include annual and periodic trends of sales within the organization. The data contained in these sources is uploaded form the operational systems and hence can be utilized in making accurate reports regarding the operations. Before the data can be used for reporting purposes it could pass through operational data stores. This reports presents summaries of researches conducted in topics seeking to describe various normalized models of data warehousing. The research covers the topics indicated in the table below

Due to the advancement that has been realized, the amount of data being generated may be overwhelming. Many proposals on data warehouses can solve this problem and handle the complex data. However, one challenge often occurs as a result of the analysis. The terminology used by different companies in OLAP is

A data warehouse (DW) can be acknowledged as one of the most complex information system modules available and it is a system that periodically retrieves and consolidates data from the sources into a dimensional or normalized data store. It is an integrated, subject-oriented, nonvolatile and a time-variant collection of data in support of management’s decisions (Inmon, 1993).