Computer Science And The Big Data Management Essay

Our every day life has changed forever, thanks for the ubiquitous smart phones and technology dependent information age. We leave a trail of data while travelling, shopping, driveing, bloggin, and even voting. All of these activities leave a digital signature unique to us, which if analyzed can predict our next move. Similarly, a large set of data is being created each day by businesses, researchers and the World Wide Web. According to an estimate by the government, there are about 1.2 zettabytes (250 billion DVDs) of electronic data generated each year by everything from underground physics experiments and telescopes to retail transactions and Twitter posts (Mervis 22). This data growth has created a new challenge and opportunity. The

Big data is the present most-liked theme of today 's technology. These research goes through all description of techniques and technologies of extracting of the data, storing of data, distribution of data, analyzing of data, managing of data with high velocity and from the structured data and helps in the handling of the extreme data. Big data has the presentation the capacity to improve predictions, saving money and enhancing the decision making process in the fields of the traffic control, weather forecasting, disaster prevention, fraud control, business transaction, education system, health and the national security.

As a result of the appearance of big data in our world, conventional data warehousing and data analysis methods no longer have the process power needed. What is Big Data you may ask and why is it such a big deal. NIST defines big data as anywhere “[…] data volume, acquisition velocity, or data representation limits the ability to perform effective analysis using traditional relational approaches […]” (Mell & Cooper, n.d.).

Databases have been around long before the computer first utilized them, but they became a necessity as the industry has been using them now for over 40 years. Everything is a database now, your inventory, your class schedule, your closet, it is all a possible database. Arora and Gupta state, “Stand alone applications have been replaced with web-based applications, dedicated servers with multiple servers dedicated storage with network storage” (Arora and Gupta 2012). There have been several movements to try and change the database environment, however attempts to replace the relational database with the object-oriented database never came to popularity and relational databases popularity remains high today.

Big data refers to the large amount of data that is impossible to handle by using traditional or conventional methods such as relational databases or it is a technique that is required to handle the large amount of data that is generated with advancements in technology and increase in population. Big data helps to store, retrieve and modify these large data sets. For example, with the advent of smart technology there is rapid increase in use of mobile phones due to which large amount of data is generated every second, so it is impossible to handle by using traditional methods hence to overcome this problem big data concepts were introduced.

The dataset comprises of a single csv file data.csv that contains 3 types of entities namely users, businesses and reviews. Records for each entity are distinguished by the 'type ' column.

Big data is an extensive collection of structured and unstructured data. It is a modern day technology which is applied to store, manage and analyze data that are not possible to manage, store and analyze by using the commonly used software or tools. Since all of our daily tasks are overtaken by the modern technologies and all the businesses and organizations are using internet system to operate, the production of data has increased significantly in past

In todays era, the volume of data we manage has developed to terabytes. As the volume of data continues developing, the sorts of data produced by applications get to be wealthier than some time recently. Subsequently, traditional relational databases are tested to catch, visualize, seek, share, break down, and store data. We find many difficulties in managing big data using traditional data modeling techniques. We still need an advance modeling techniques threw which we can solve problems of managing big data. There are two types of data models in data base system one is relational model and other is non

Big data is needed in several situations during election process, but the main purpose of political data is to provide a citizens’ contact list. Campaigns need information about which citizens participate in specific campaign actions like rallies, petitions, candidate support, donating money, volunteering, or issues in tracking polls.

Abstract— Big data is a significant subject in modern times with the rapid advancement of new technologies for example, smartphones, pc/laptops, game consoles, that all in some way store information. Big companies require a place to not only store all the data that is coming in but to also analyze it for specific purposes and at the fastest speed manageable. There are many different providers out there who provide this service, this paper will talk about one way the company Google handles data using their own special made platform.

It helps an organization consolidate data from several sources by separating analysis workload from transactional workload. Additionally, a data warehouse environment includes ETL which is Extraction, Transportation and Loading solution, an OLAP which is Online Analytical Processing Engine, analysis tools and other tools so as to look over the process of gathering data and finally delivering it to business users. The data stored in these warehouses must be stored in a way which is reliable, secure, and easy to process and manage. The need for data warehousing arises as businesses become more complex and start generating and gathering huge amount of data which were difficult to manage in the traditional way.

The massive amount of data available is where the term “big” data comes from. The processing power of a typical server is not vast enough to store this data; new technologies help to ease this burden. To put into perspective how much data is out there, six billion people today have cell phones that are transmitting data. Most United States company currently store a minimum of one-hundred terabytes. It is estimated that by the year 2020, there will be forty zettabytes of data available. This is three-hundred times the amount available in twelve years ago.

In the infancy of computing data storage was, maybe the primary issue plaguing computer scientists’ due to the fact that there was no reliable and affordable means of storing data Through many years of steady scientific and engineering progress finally a reliable source for data storage was created in the form of a hard disk Then the matter of contention shifted to the absorbent price of this new solution to an age-old problem, but when manufactures began to mass produce hard drive disks and the ease of access to these manufacturing resources, as well as, the facilities required to produce this new futuristic product, a new problem presented itself With the advent of major companies having access to hundreds of thousands or even millions

As there is a rise in data volumes, the manageability of data and storing these huge volumes of data became a cause of concern to most of the organizations. It was during this period when Number of SQL or more popularly NoSQL was introduced, to process these large amounts of data efficiently and effectively. For this purpose, various Data Store categories were developed, based on the different data models. Some of the categories are:

‘Big Data’ is the application of specialized techniques and technologies to process very large sets of data. These data sets are often so large and complex that it becomes difficult to process using on-hand database management tools. There are several techniques which are widely used in implementation of Big Data.