Challenges Developing A Big Data Analytic Capability

Every day, we produce 2.5 quintillion bytes of data. 90% of all data in the world was produced in the past two years. Data has been around forever; we have always gathered information. Paleolithic cavemen recorded their activities by carving them in stone or notching them in sticks. Egyptians used hieroglyphics to record significant events in history. The Library of Alexandria was home to half-a-million scrolls of the ancient world. Less than hundred years ago, we used punch cards to record and store information. As technology continues to evolve, the amount of data we store continues to grow. We’ve come a long way since stone tablets, scrolls, and punch cards. It’s important to understand the concept of big data and the impact is has created. This paper will define the classifications of data, explain the challenges of big data, and describe how big data analytics is being used in today’s data driven world.

Since the 1970’s databases and report generators have been used to aid business decisions. In the 1990’s technology in this area improved. Now technology such as Hadoop has gone another step with the ability to store and process the data within the same system which sparked new buzz about “big data”. Big Data is roughly the collection of large amounts of data – sourced internally or externally - applied as a tool – stored, managed, and analyzed - for an organization to set or meet certain goals.

Big Data is an expansive phrase for data sets so called big, large or complex that they are very difficult to process using traditional data processing applications. Challenges include analysis, capture, curation, search, sharing, storage, transfer, visualization, and information privacy. In common usage, the term big data has largely come to refer simply to the use of predictive analytics. Big data is a set of techniques and technologies that need or require new forms of integration to expose large invisible values from large datasets that are diverse, complex, and of a massive scale. When big data is effectively and efficiently captured, processed, and analyzed, companies

The emergence of big data has provided different avenues for organizations to use data to improve different aspects of their respective operations. Be it customer service, research and development, or market position, Big Data has the potential to be a significant driving force in all these areas. However, there’s still a significant gap between the ability of Big Data to produce insightful analytical information based on real-time data and the ability of organizations to capture and utilize this readily available tool. This is, in part, due to the fact that the systems and processes necessary to fully maximize the usefulness of Big Data is currently lacking in most organizations. This lack of a conducive habitat for Big Data is further magnified in new organizations without any knowledge of Big Data. For organizations that have that have little to no knowledge of Big Data, there must be a thorough assessment of the benefits of big data and how they could improve the organizations overall place in the market. There also needs to be steps taken towards the design of frameworks that will enable the organization to better capture and utilize Big Data.

Business thrive when they have the most accurate, up-to-date, and relevant information at their disposal. This information can be used for a plethora of pertinent markers in small and large businesses, relating to accounting, investments, consumer activity, and much more. Big data is a term used to describe the extremely large amounts of data that floods a business every day. For decades, big data has been a growing field, facing controversy on many levels, but as of late, it has been a major innovator in the challenge of making businesses more sustainable. Big data is often scrutinized for its over-generalization and inability to display meaningful results at times. When applied correctly, data analysis can bring earth-altering information to the table.

Data science is applicable in almost every discipline where a large set of data is generated and consumed. In pure science, specifically astronomy and physics, Big Data analytics has long been used for finding undiscovered planet or unknown atomic particle in the accelerator labs. Other areas like Biology and Chemistry are still developing techniques for the Big Data Analytics. In the study the Big Challenges of Big Data Vivien Marx wrote, “Biologists are joining the big-data club. With the advent of high-throughput genomics, life scientists are starting to grapple with massive data sets, encountering challenges with handling, processing and moving information that were once the domain of astronomers and high-energy physicists” (Marx 255). The uneven level of Big Data analytics usage within the pure sciences signifies the need to produce new talents and the gap that exists today.

Big data is buzzword in every field of business as well as research. Organizations have found its application across various sectors from Sports to Security, from Healthcare to e-Commerce.

As a result of the appearance of big data in our world, conventional data warehousing and data analysis methods no longer have the process power needed. What is Big Data you may ask and why is it such a big deal. NIST defines big data as anywhere “[…] data volume, acquisition velocity, or data representation limits the ability to perform effective analysis using traditional relational approaches […]” (Mell & Cooper, n.d.).

This paper will look at some of the ‘Big Data’ being implemented today. Regardless of ow anyone feel, ‘Big Data’ s a thing that is not going away. This paper will look at Video and Image Data, Audio Data, Textual Data, Managerial Accounting.

Big Data is becoming more meaningful with the ever more powerful data technologies, which enable us to derive insights from the data and help us make decisions. Big Data also creates new courses and professional fields such as the data science and data scientist, which are aimed at analyzing the ever growing volume of data. Some might think this exaggerated because data analysis, after all, not a new invention. However, we might all agree that the progress of digitization associated with the generation of ever larger amounts of data have totally changed the ways we deal with data.

The world is changing with respect to the growth in big data and to the way in which it is used. Growth in big data brings with it many challenges, but it also presents new opportunities. Figure 1, helps understand some of the big data related activities that are taking place in the world with respect to volume of data that is being consumed by these activities over the next 5 years.

Big data is an emerging term which has been noticed since it influences our daily life gradually. The big data is a large concept that is vague because different people look at big data phenomenon from a different perspectives, it is not easy to make a precise definition (Moorthy, et. al., 2015). The definition of big data is a matter of debate, however, a typical reference is to the collection, management, and analysis of massive amounts of data (McNeely & Hahm, 2014).According to George et. al. (2014), big data include Internet clicks, mobile transactions, user-generated content, and social media and content from sensor networks or business transactions, such as sales queries and purchase transactions. These procedures are significant to

In the last few years the global marketplace has seen exponential growth in data volume. Every day people create unstructured large datasets of different types such as GPS coordinates, payment transactions, web data, e-mails or smart meter values that are termed as "big data" \cite{nasscom}.

Big data and Analytics have become very prominent areas of study in recent years. Company revenues exceeding more than one million are found to use some or the other form of business analytics. The techniques, technologies, systems, practices, methodologies and applications help analyze the data in the organizations to make critical decisions.

One piece of information may be insignificant, but billions of data points can illuminate. That’s the underlying promise of big data and analytics, which observers have been calling a revolutionary development for several years now. But it’s difficult to know where a revolution is headed while it’s still unfolding. New research from the McKinsey Global Institute