Big Data And Its Impact On The Improvement And Availability Of Data

What is Big Data? Big Data is the mass collection of user data by mathematical algorithms, databases, data mining, and the use of datasets that were once believed to be static and unusable. Big Data’s history goes way back “…70 years to the first attempts to quantify the growth rate in the volume of data, or what has popularly been known as the “information explosion” (Press, Gil).” Researchers had predicted the massive growth of information and how our ability to collect and store it would need to continue to grow as well.

Organizations are relying on increasing amounts of information from a variety of sources, text, images, audio, video, etc., to analyze, improve and execute their operations. These information sources are very large and complex and include data sets that are structured and unstructured and today’s processing applications are inadequate and expensive. Industries suffering from big data challenges include the financial services, healthcare, retail, and communications, to name a few. These industries have been collecting data for years and with the advent of the ‘Internet of Things’ data is growing exponentially. The challenge is how to make sense of this data and turn it into business value – analytics and predictive analytics.

Big data is a term that describes a large volume of data. This data comes in the form of structured and unstructured data. Structured data is information taken and sorted in rows and columns while unstructured data is pictures, tweets, videos, and location-based data. It is not surprising to see many businesses today utilizing data for financial gain. Businesses are harnessing data and using it to make investment decisions, marketing strategies, fraud reductions, and much more. These businesses and organizations can expect to become more profitable, effective, and efficient, but pushes the limits

Big data is not as new as many people believe it to be. It is actually a concept that has been around for almost a century. It is just the “same old data marketers have always used, and it’s not all that big, and it’s something we should be embracing, not fearing” (Arthur). In 1944, Fremont Rider “predicted that the amount of data in the world would increase exponentially” (Hopp). Rider was right on target with his prediction seventy years ago. Data has grown much greater than he probably could have ever imagined back then.

Today, the data consumption rate is tremendously expanding, the amount of data generated and stored is nearly imperceivable and highly growing. Big data that is nothing but a large volume of unstructured or structured data that runs in and out in to a business on daily basis. This big data is analyzed in order to achieve prominent business growth and improved business strategies [1]. Every year there is at least 40% increase in the amount of data growth on global level, leading to which companies have started adopting new data analytic techniques and tools and also have stepped ahead moving their data towards the cloud for their big data analytic requirements and for better analysis.[3][2] In big data analysis it is not the amount of data that is essential but how efficiently we handle, process and analyze it is the key factor. Big data analysis doesn’t revolve around how much data we occupy, it deals with how well you make use

The term “Big Data” refers to the massive amounts of digital information companies and governments collect about human beings and environment. The amount of data produced or gathered is increasing at a faster rate and is like to double after every two years i.e., from 2500 exabytes in 2012 to 40,000 exabytes in 2020 . Security and privacy issues are describes by the 3 V’s of big data- volume, variety, and velocity.

Due to the increase in new technology, business, communication, device, big scale of data was produced. About 90% data in today’s world was just created in last two years alone, without counting those data that has been created previously. The information retained in those data was a big risk to many organizations as the current technology was managing the data with traditional approach, which consisted of user, a centralized system and relational data base. This style had various drawbacks together along with two key problems: less storage capacity and slow data processing.

Like the traditional data, big data through a series of steps that contain collection, storage and analysis to form a complete system to help both enterprises and individuals produce an optimum strategy or decision and maximize benefits in their stance. As for traditional data system, it is usually not enough accurate in analyze the phenomenon or the situation due to lack of sufficient data that results from the speed of collecting data is relatively low and the process

The term big data came into the picture to refer the big volumes of information’s both the companies and governments are storing. The data may be where we live, where we go, what we buy and what we say etc. all will be recorded and stored forever. More than 90% of data is generated in the past 2 years only and this volume is increasing day by day and doubling for every two years. In this world, the organizations are using the data generated by us and no one knows what they are doing with the collected data. Big data is defined as a lot of structured and unstructured data from different sources, such as E-commerce websites, online transactions, social networks, medical records, internet search indexes, banking and financial services, scientific searches, weblogs, and document searches and so on. Big data also can be described by four V’s: Volume, Velocity, Variety and finally Value.

High-quality data are the precondition for guaranteeing, using big data and analyzing. Big data has a quality that faces many challenges. The characteristics of big data are the three Vs Variety, Velocity, and Volume, as explained in the what is big data section of the paper Variety of data indicates that big data has a different kind of data types, and with this diverse division puts the data into unstructured data or structured data. These data need a much higher data processing capability. Velocity is the data that is being formed at and unbelieve amount of speed and it must be dealt in an organizational and timely manner. Volume is the tremendous volume

Already some success is achieved from big data in some fields like Sloan digital sky survey so it’s mean there is some potential in big data and benefits are also real but still some challenges like scalability, heterogeneity, integration, privacy, security etc. need to be addressed for realizing full potential of big data. One of the major challenge is transformation of unstructured data to structured form for accurate and timely processing. Challenges with big data starts with very first phase of big data analysis pipeline that is data acquisition phase. It’s a challenging task to determine what data to keep, what to discard and how to efficiently store the data. Other challenges are faced in data cleaning, integration and data analysis phase of big data analysis pipeline.

Data gathered in the 80s and 90s, commonly called “traditional data” was measured in gigabytes and was mainly structured, organized and analyzed using SQL; (SQL stands for structured query language, it is a the standard language to communicate with RDBMS) (American National Standards Institute), as opposed to now where data includes huge volume, high velocity and variety, it is now measured in Petabytes (1 Petabyte = 1000000 Gigabyte). New technologies have been developed to analyze the new types of data (semi - structured, unstructured) using Hadoop systems.

Now-a-days the Data usage has increased a lot. Data that the human race has accumulated in the past one decade, far exceeds the data that are available to mankind during the preceding century. They also expect that different stakeholders such as consumers, companies and businesses are likely to exploit the potential of Big Data. Several estimates about the accumulation of data have challenged our earlier imagination. Data scientists are increasingly using data quantities in Peta and Zeta bytes. There is no doubt now that

The emerging data from the everywhere in the world make the birth of big data era. There exist potential values in those data, while the big volume, variety and velocity [2] of the data make it nearly impossible for humans to analyze resources manually so as to find the hidden treasures. Under this circumstance, the concept and technique

In today’s world, the amount of unstructured data collected is humungous. This unstructured data is of no use if it is not properly processed, analyzed and evaluated. Using this data for the betterment of mankind is what most of the largest companies like Google, Facebook, Amazon, Netflix and much more are targeting. Big data is a term for datasets which are so large and complex that traditional database systems such as MS SQL, MySQL, etc., are incapable of handling them. It is not the amount of data that is important, but what organizations do with data that matters the most. Data can be mapped to useful information which can be further utilized for analyzing and drawing insights that lead to better management practices and strategic