Hadoop Security Issues And Recommendations

The big data analytics deals with a large amount of data to work with and also the processing techniques to handle and manage large number of records with many attributes. The combination of big data and computing power with statistical analysis allows the designers to explore new behavioral data throughout the day at various websites. It represents a database that can’t be processed and managed by current data mining techniques due to large size and complexity of data. Big data analytic includes the representation of data in a suitable form and make use of data mining to extract useful information from these large dataset or stream of data. As stated above the big data analytics has recently emerged as a very popular research and practical-oriented framework that implements i) data mining, ii) predictive analysis forecasting, iii) text mining, iv) virtualization, v) optimization, vi) data security, vii) virtualization tools for processing very large data sets. In the implementation of big data applications, new data mining techniques and virtualization are required to be implemented due to the volume, variability, forms and velocity of the data to be processed. A set of machine learning techniques based on statistical analysis and neural networking technology for big data is still evolving but it shows a great potential for solving a big data business problems. Further, a new concept of in-memory database for enhancing the speed for analytic processing is further helping

With the development of internet technology, society has been pushed compulsorily into a ‘big data’ period(Craig and Ludloff,2011).”Big data refers to the massive amounts of data collected over time that are difficult to analyze and handle using common database management tools” (http://www.pcmag.com). Not only the development strategy of the government and enterprises, but also threaten citizens’ personal information security. There are significant issues increases rapidly due to this environment such as transfer , reproduction and abuse of personal information. Therefore, it provides convenience to criminal who wants to steal personal information of other people for benefit reason or other reasons. This essay aims to introduce the risks to personal information on the internet and find some solutions to relieve even solve these problems. Firstly, following article will indicate why personal information is risky on the internet and the situation of information security in recent internet environment. Secondly, significant issues from information security will be shown. Finally, solutions and evaluations need to be come up with so that people can protect their privacy.

Big data describes large scale data sets which are analyzed, using algorithms, to determine and eventually predict patterns in human behavior, interactions and the environment (oed.com). Big data is used by government entities to protect against terrorism as well as commercial entities for marketing. Big data needs ethical standards to prevent violations of four central principles, which are privacy, confidentiality, transparency, and identity (Richards 395). Big data is awareness and empowers those who control it. This revolution in information enables companies to shape consumer identity by influencing every interaction the user has with their service. Due to its predictive and persuasive nature, restrictions are necessary for consumers to

Big Data is an expansive phrase for data sets so called big, large or complex that they are very difficult to process using traditional data processing applications. Challenges include analysis, capture, curation, search, sharing, storage, transfer, visualization, and information privacy. In common usage, the term big data has largely come to refer simply to the use of predictive analytics. Big data is a set of techniques and technologies that need or require new forms of integration to expose large invisible values from large datasets that are diverse, complex, and of a massive scale. When big data is effectively and efficiently captured, processed, and analyzed, companies

The author points out that although there are existing algorithms and tools available to handle Big Data, they are not sufficient as the volume of data is exponentially increasing every day. To show the usefulness of Big Data mining, the author highlighted the work done by United Nations. In order to further enhance the reader’s perspective, the author provided research work of various professionals to educate its readers about the most recent updates in Big Data mining field. The author further describes the controversies surrounding Big Data. The author has first provided the context and exigence by elaborating on why we need new algorithm and tools to explore the Big Data. The author used the strategy of highlighting the logos by mentioning the research work of different industry professionals, workshops conducted on Big Data and was able to appeal to connect to the reader’s ethos. The author also used pathos by urging the budding Big Data researchers to further dig deep into the topic and explore this area

of big data are obvious but we also have to ensure our privacy at the same time. If we ensure privacy in

With the use of more sophisticated pattern analysis and with the use of multiple data sources it is easy to detect the threats in early stages of the project itself. Many organizations are fighting with the remaining issues like private issues with the usage of big data. Data privacy is a liability; thus companies must be on privacy defensive. When compared to security, Privacy should consider as a profit making asset because it results in the selling of unique product to customers which results in making money. We need to maintain balance between data privacy and national security. Visualization, controlling and inspection of the network links and ports are required to ensure security. Thus, there is a need to put the ones in insight the loop holes, conflicts, and the contents ready for attacks with respect to cloud computing, and we need to develop a platform and infrastructure which are less protected in attacks.

A significant paradigm shift is represented by public cloud computing from conventional norms of an organizational data center to a de-parameterized infrastructure which opens gates for potential adversaries to use. Cloud computing should be approached carefully with any emerging information technology area with due consideration to the sensitivity of data. A good planning helps and ensures that the computing environment is secure to the most possible extant and is in compliance with all relevant policies of an organization and makes sure the privacy is maintained.

A significant paradigm shift is represented by public cloud computing from conventional norms of an organizational data center to a de-parameterized infrastructure which opens gates for potential adversaries to use. Cloud computing should be approached carefully with any emerging information technology area with due consideration to the sensitivity of data. A good planning helps and ensures that the computing environment is secure to the most possible extant and is in compliance with all relevant policies of an organization and makes sure the privacy is maintained.

The main purpose of this report is to provide a critical review of the processes and own experiences of Hadoop within the context of the assignment which was given to us. The review concentrates on the discussion and evaluation of the overall steps followed during the progress of the project and the reasons for which we have chosen these particular steps. It also draws attention at the main points that were accomplished, both with respect to individual, and with respect to the group 's perspectives. Finally, it concentrates on the project 's progress in terms of changes for a future implementation.

Using big data analytics brings the capability of delivering vital information for detecting the formation of an attack prior to it striking and penetrating network defenses. By identifying and containing the exfiltration of data by malicious insiders, an organization can build a stronger cyber defense posture. Possessing the knowledge of the differences between acceptable and nefarious behavior is the key to identifying and stopping zero-day exploits. Big data analytics expands the knowledge base for anomaly detections by applying innovative pattern recognition techniques with machine learning algorithms identifying malicious behavior.

Big data is not a new technology, however applied to the rapidly growing and changing field of cyber security, big data companies are finding new and emerging ways to use big data analytics to gain a competitive edge (Emerging Trends in Big Data Analytsis for 2015, 2015). Big data analysis looks for and utilizes the patterns extracted from large volumes of data. (Kantarcioglu, 2015). One of the important uses of big data analytics is in the detection, discovery, and prevention of cyber security threats. Big data analysis is also instrumental in the collection and analysis of the large volumes of data collected from sensors embedded within smart grids (Bharathi, 2015). Many companies and organizations in sectors such as electric

Cloud Computing and Big Data technologies can effectively be used to change the way in which businesses operate. Companies operating in fields such as financial, educational, technological and many more, have adopted the use of these technologies or are exploring the possibilities to become part of this new trend, as the benefits are proven to be a tremendous success. Not only for the increase in profits, but these technologies have a positive impact in many areas of a business. The success of Cloud Computing and Big Data has transformed businesses around the globe and “forced” them to spend billions on cloud computing services. The popularity of these technologies is rapidly growing and they are now accessible not only to giant companies that would spend limitless amount of money in order to have the best technology available, but also to small and medium size companies with a less significant budget.

Due to the rapid growth in the use of Internet and its connected tools, an enormous amount of data are being produced on a daily basis. The concept of big data arrives when we were unable to manage this huge data with traditional methods. Big data is a mechanism of capturing, storing and analyzing the big datasets and also an idea of extracting some value from it. It is very handful while determining the root causes of failures, issues and defects in near-real time, creating coupons and other sales offers according to the customers shopping patterns, detecting any suspicious and fraudulent activities in real-time. As it is very advantageous, it also has some issues. Some of the common issues can be characterized into heterogeneity, complexity, timeless, scalability and privacy. The most important and significant challenge in the big data is to preserve privacy information of the customers, employees, and the organizations. It is very sensitive and includes conceptual, technical as well as legal significance.

Over the years it has become very essential to process large amounts of data with high precision and speed. This large amounts of data that can no more be processed using the Traditional Systems is called Big Data. Hadoop, a Linux based tools framework addresses three main problems faced when processing Big Data which the Traditional Systems cannot. The first problem is the speed of the data flow, the second is the size of the data and the last one is the format of data. Hadoop divides the data and computation into smaller pieces, sends it to different computers, then gathers the results to combine them and sends it to the application. This is done using Map Reduce and HDFS i.e., Hadoop Distributed File System. The data node and the name node part of the architecture fall under HDFS.