A Brief Note On Big Data Analytics Techniques

Artificial Intelligence may come in many forms, but for the purpose of this paper, I have adopted the definition from The Columbia Encyclopedia (2008), which states

Big Data is an expansive phrase for data sets so called big, large or complex that they are very difficult to process using traditional data processing applications. Challenges include analysis, capture, curation, search, sharing, storage, transfer, visualization, and information privacy. In common usage, the term big data has largely come to refer simply to the use of predictive analytics. Big data is a set of techniques and technologies that need or require new forms of integration to expose large invisible values from large datasets that are diverse, complex, and of a massive scale. When big data is effectively and efficiently captured, processed, and analyzed, companies

products to customers based on items they have already bought or shown interest in.  This

Data mining: is a process of discovering patterns in large data involving methods at the intersection of artificial intelligence, machine learning, statistics, and database systems with the aim of extracting information and transforming it into an understandable structure for further use.

Data mining is really just the next step in the process of analyzing data. Instead of getting queries on standard or user-specified relationships, data mining goes a step farther by finding meaningful relationships in data. Relationships that were thought to have not existed, or ones that give a more insightful view of the

(Ian, Frank & Hall 2011) The world testifies to the increasing gap between the development of data and our understanding of it. Data mining is defined as the analysis of big observational data sets to establish unsuspected relationships that summarize the data in a novel, understandable and useful way to the data owner (David, Heikki & Padhraic 2001). These relationships and summaries are referred to as models or patterns. Patterns comprise sets of co-occurring attribute values referred to as item sets or more complex patterns that include sequences, which consider explicit precedence relationships that are either positional or temporal, and graphs considering arbitrary relationships between points (Mohammed and Wagner 2014). Other types of additional summarised relationships include rules, clusters, tree structures among others.

Today, the data consumption rate is tremendously expanding, the amount of data generated and stored is nearly imperceivable and highly growing. Big data that is nothing but a large volume of unstructured or structured data that runs in and out in to a business on daily basis. This big data is analyzed in order to achieve prominent business growth and improved business strategies [1]. Every year there is at least 40% increase in the amount of data growth on global level, leading to which companies have started adopting new data analytic techniques and tools and also have stepped ahead moving their data towards the cloud for their big data analytic requirements and for better analysis.[3][2] In big data analysis it is not the amount of data that is essential but how efficiently we handle, process and analyze it is the key factor. Big data analysis doesn’t revolve around how much data we occupy, it deals with how well you make use

Big data is an extensive collection of structured and unstructured data. It is a modern day technology which is applied to store, manage and analyze data that are not possible to manage, store and analyze by using the commonly used software or tools. Since all of our daily tasks are overtaken by the modern technologies and all the businesses and organizations are using internet system to operate, the production of data has increased significantly in past

To address the question of how and what techniques has been used to manages this big amount of data or in the field of Big Data, I review some research papers and review articles in the field of Big Data. This paper provides the synthesis of those papers which I found relevant to this field. This paper will focus on the following things:

There are many fundamental issue areas that need to be addressed in dealing with big data: data acquisition, data storage, data transfer, data management, and data processing. Each of these issues represents a large set of technical research problems and challenges in its own right.

Abstract-The amount of data being generated nowadays has increased tremendously. With these large volumes of data, there is a need for efficient storage and processing of data. The data generated from a variety of sources like Social networking sites, Emails, audio and video files, text files and other various files is unstructured. The traditional database cannot handle this complex and high volumes of data efficiently. The Hadoop framework provides an effective solution for storing large volumes of data and processing the data for analysis. This paper deals about the Traditional Databases, Big data, Data analysis and how effectively Hadoop stores these large volumes of data and processes data for analysis. Furthermore, this paper also focuses on the various technologies used in Hadoop for the purpose of storing the data and for processing the data.

Data mining, an interdisciplinary subfield of computer science, is the computational methodology of finding examples in substantial data sets including routines at the crossing point of manmade brainpower, machine learning,

Big data has received popularity as datasets which are large that cannot be easily managed by the traditional relational databases. Big data deals with high volume, variety and velocity of data. However, if traditional relational database schemas are applied to big data, the large volume of datasets cannot be processed and managed by these traditional techniques. The notable solution to manage and process the data is MapReduce framework. MapReduce is programming model which programming model where a large data sets is divided into small data sets; these small data sets are processed in parallel by nodes(computers). This is called as batch processing, where each worker node applies map function and reduce function to the associated data to

In data mining, inductive learning techniques are used when constructing a model which ensures that trained data model can be applied for future cases.

in such a manner that its expected future performance improves [5]. Machine learning is a type of