Analyzing The Field Of Big Data

Abstract- Big data is a hot research topic in today’s world. Data has become an indispensable part of every economy, industry, organization, business function and individual. With the fast growth now-a-days organizations has filled with the collection of millions of data with large number of combinations. This big data challenges over business problems. Big Data is a new term used to identify the datasets that due to their large size and complexity. Big Data mining is the capability of extracting useful information from these large datasets or streams of data, that due to its volume, variability, and velocity. We address broad issues related to big data and/or big data mining, and point out opportunities which help to reshape the subject area of today’s data mining technology toward solving tomorrow’s bigger challenges emerging in accordance with big data.

There are large amounts of information available in the world through various media outlets, such as the internet, mobile conversations, emails, chats, messages, etc. Nowadays, everything is a big data and with the emergence of the whole new breed of big data software which is distributed all over in the world, whatever data is available can be used for analyzing.

With increasing data the storage of the data must also be increased, which is a problem. So, data is stored or recorded in the form of computer data bases which makes easy to access the right data at any given point of time. To extract the right data from all these present volumes of data, usually certain traditional way of data analysis like regression analysis, cluster analysis, numerical taxonomy, multi-dimensional analysis, time series analysis , estimation outcome analysis and many more are used.

As it has been discussed in prior parts of this paper, big data can originate from multiple sources and therefore requires an intelligent process to acquire and store this raw data.

"Such ‘Data Explosions ' has led to one of the most challenging research issues of the current Information and Communication Technology (ICT) era: how to effectively and optimally manage such large amount of data and identify new ways to analyze large amounts of data for unlocking information. The issue is also known as the ‘Big Data ' problem, which is defined as the practice of collecting complex data sets so large that it becomes difficult to analyze and interpret manually or using on-hand data management applications. From the perspective of real-world applications, the Big Data problem has also become a common phenomenon in domain of science, medicine, engineering, and commerce"

In 2012, the concept of ‘Big Data’ became widely debated issue as we now live in the information and Internet based era where everyday up to 2.5 Exabyte (=1 billion GB) of data were created, and the number is doubling every 40 months (Brynjolfsson & McAfee, 2012). According to a recent research from IBM (2012), 90 percent of the data in the world has been created in the last two years alone, and Internet activity in each second today will generate more data than all the data combined in the

Industry is thriving to capture the flood of data present today. Every sector has millions trillions of data which is increasing with extreme velocity to be handled. Social media has been fueling up data growth exponentially. Although human race has been generating data for thousands of years but recent decade has put forward a new

Big data is certainly one of the biggest buzz phrases in it today. The term ’Big Data’ appeared for first time in 1998 in a Silicon Graphics (SGI) slide deck by John Mashey with the title of ”Big Data and the Next Wave of InfraStress” [9]. -Combined with virtualization and cloud computing, big data is a technological capability that will force data centers to significantly transform and evolve within the next five years. Similar to virtualization, big data infrastructure is unique and can create an architectural upheaval in the way systems, storage, and software infrastructure are connected and managed. Big data is an amalgam of large and varieties of data sets including structured data, semi structured data and unstructured data so it’s beyond the capability of traditional tools to capture, store, process and analysis of big data. It is true that big data have capability of unlocking new sources of development in many fields but at the same time researchers are being confronted challenges with big data. This paper reveals the various challenges faced with big data and opportunities realized with big data. Keywords: Big data, Challenges, Opportunities, Security Issues.

This increased the demand for data scientists across Europe. The larger the volumes of data the greater the complexity of data and analyzing of data. Getting high varieties and volumes of data at greater velocities is the advantage of using Big data.(H. Davenport, 2014). The trend of making information from data has started from decision support followed by execution support, analytical processing, business Intelligence (BI) ended with Big Data. The design Big data architecture and

Understanding and applying the use of big data and data analysis creates advantages for businesses. As such, this paper discusses big data and analysis regarding differences between big data and small data, characterization according to 3 Vs, big data applications, data analysis, types of data structures, obtaining sources, tools and processes used for analysis, and advantages of data analysis.

Abstract— Big data is a significant subject in modern times with the rapid advancement of new technologies for example, smartphones, pc/laptops, game consoles, that all in some way store information. Big companies require a place to not only store all the data that is coming in but to also analyze it for specific purposes and at the fastest speed manageable. There are many different providers out there who provide this service, this paper will talk about one way the company Google handles data using their own special made platform.

With 3.2 billion internet users [1] and 6.4 billion internet connected devices by 2016 [2], unprecedented amount of data is being generated and process daily and increasing every year. The advent of web 2.0 has fueled the growth and creation of new and more complex types of data which creates a natural demand to analyze new data sources in order to gain knowledge. This new data volume and complexity of the data is being called Big Data, famously characterised by Volume, Variety and Velocity; has created data management and processing challenges due to technological limitations, efficiency or cost to store and process in a timely fashion. The large volume and complex data is unable to be handled and/or processed by most current information systems in a timely manner and the traditional data mining and analytics methods developed for a centralized data systems may not be practical for big data.

In 2013 the overall created and copied data volume in the world was 4.4 ZB and it is doubling in size every two years and, by 2020 the digital universe – the data we create and copy annually – will reach 44 ZB, or 44 trillion gigabytes [1]. Under the massive increase of global digital data, Big Data term is mainly used to describe large-scale datasets. Big data is high-volume, high-velocity and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making [2]. Volume of Big Data represents the magnitude of data while variety refers to the heterogeneity of the data. Computational advances create a chance to use various types of structured, semi-structured, and

In the last few years the global marketplace has seen exponential growth in data volume. Every day people create unstructured large datasets of different types such as GPS coordinates, payment transactions, web data, e-mails or smart meter values that are termed as "big data" \cite{nasscom}.

As one of the most popular and widely used data mining techniques, cluster analysis is mainly divided into hierarchical clustering and partitional clustering, which are carried out in a supervised or unsupervised way to separate data into different groups based on similar characteristics. Both the hierarchical and partitional clustering have advantages and drawbacks. Especially, the efficiency and accuracy are the primary challenges that clustering analysis has to face. For example, the most efficient algorithm of hierarchical clustering is complete-linkage clustering in some special cases, the complexity of which is Ο (n2). Therefore, the hierarchical clustering usually leads to too slow efficiency for large data