Data Warehousing : Big Data Management Essay

In today’s world, data is being amassed at an unprecedented scale. Large amounts of data generated by and about people and their interactions are being collected, analyzed, and stored for future use. Organizations are able to gain access to a variety of data sources including call logs, text messages, emails, client chats, social media pictures, videos, and posts, RFID, Geographic Information Systems (GIS), and much more. The reception of Big Data is described by boyd and Crawford (2012) as being “seen as a powerful tool to address various societal ills, offering the potential of new insights into areas as diverse as cancer research, terrorism, and climate change” as well as being “seen as

The emergence of new technologies, applications and network systems makes it hard to run the current business models and huge data types, and thus emerged various types of analytic tools like Big Data, which make this work easier by way of proper organization of data. Big Data is all about analyzing different forms of data (Structured, Semi-structured and Un-structured) and it is not about the procedure, creation or consumption of data.

We will discuss in this paper the data warehousing and the online processing of data. We will describe the best ways to manage the data and the difficulties that you could face. Also we will talk about how can we solve or reduce these difficulties.

The dataset comprises of a single csv file data.csv that contains 3 types of entities namely users, businesses and reviews. Records for each entity are distinguished by the 'type ' column.

This is to certify that Mr. AKSHAY DOGRA student of B.Tech. in CSE(Evening) has carried out the work presented in the project of the Term paper entitle "BUSINESS INTELLIGENCE AND DATA WAREHOUSING" as a part of First year programme of Bachelor of Technology in CSE (Evening) from Amity University, Noida, Uttar Pradesh under my supervision.

This article can be regarded as current since it was published in 2013. What is more, the authors of this text both work for the department of business in the universities. They may have specific expertise or knowledge in the field of big data as it is an essential factor in business. Furthermore, Business Intelligence Journal contains a professional data warehouse for business. As a result, this article is also authoritative and reliable. Besides, as a journal article, not only does it follow the usual academic conventions like in-text citations and references, but also its language is impersonal and formal, which seems to be objective. Big data has become a useful tool to help companies make decisions and turn to customer-centred

The world is changing with respect to the growth in big data and to the way in which it is used. Growth in big data brings with it many challenges, but it also presents new opportunities. Figure 1, helps understand some of the big data related activities that are taking place in the world with respect to volume of data that is being consumed by these activities over the next 5 years.

Big data is an emerging term which has been noticed since it influences our daily life gradually. The big data is a large concept that is vague because different people look at big data phenomenon from a different perspectives, it is not easy to make a precise definition (Moorthy, et. al., 2015). The definition of big data is a matter of debate, however, a typical reference is to the collection, management, and analysis of massive amounts of data (McNeely & Hahm, 2014).According to George et. al. (2014), big data include Internet clicks, mobile transactions, user-generated content, and social media and content from sensor networks or business transactions, such as sales queries and purchase transactions. These procedures are significant to

Big data is an extensive collection of structured and unstructured data. It is a modern day technology which is applied to store, manage and analyze data that are not possible to manage, store and analyze by using the commonly used software or tools. Since all of our daily tasks are overtaken by the modern technologies and all the businesses and organizations are using internet system to operate, the production of data has increased significantly in past

The variety, volume and velocity (Lohr, 2012) of data have evolved in the present contemporary times with various companies and business embracing the benefits and advantages leveraged out of Big Data and its plethora of real life applications. This concept have taken a huge leap ahead in time and have find some innovative and creative ways to collect, store and analyze data which his enormous these days. The realm of e business and its applications in this industry are conquered by the means of big data.

Big Data embraces structured, semi-structured and unstructured information. It can be demographic or psychographic information about the customers, their opinions, product reviews etc gathered from variety of sources such as tweets, blogs, other social media content, technical devices like sensors and stream of data from mobile devices. Businesses have started using Big Data to get right information to identify right markets and right customers at the right time in order to make right strategic decisions. The Big Data market is estimated to grow to $32.1 bn by 2015 and $54.1 bn by 2017. According to the report “Be Careful or Big Data Could Bury Your Bank”, the world creates 2.5 quintillion bytes of data daily and last 2 years have contributed to almost 90% of the data which exists today. It empowers institutions to learn more, create more and do more using the data available with them.

Data has always been analyzed within companies and used to help benefit the future of businesses. However, the evolution of how the data stored, combined, analyzed and used to predict the pattern and tendencies of consumers has evolved as technology has seen numerous advancements throughout the past century. In the 1900s databases began as “computer hard disks” and in 1965, after many other discoveries including voice recognition, “the US Government plans the world’s first data center to store 742 million tax returns and 175 million sets of fingerprints on magnetic tape.” The evolution of data and how it evolved into forming large databases continues in 1991 when the internet began to pop up and “digital storage became more cost effective than paper. And with the constant increase of the data supplied digitally, Hadoop was created in 2005 and from that point forward there was “14.7 Exabytes of new information are produced this year" and this number is rapidly increasing with a lot of mobile devices the people in our society have today (Marr). The evolution of the internet and then the expansion of the number of mobile devices society has access to today led data to evolve and companies now need large central Database management systems in order to run an efficient and a successful business.

Big Data is one of the most discussed concepts in the business world today. The concept of Big Data is one that has been increasingly debated over the past few years by many different kinds of business industries. One industry where this discussion is prominent is the accounting profession. Big Data will be defined more in-depth later on, but in shorter terms, Big Data refers to the exponential sum of data available in our world today. This amount of data is beyond the traditional analysis that can be computed by human power. When Big Data is converted and used to its full potential, it could have a material impact on the accounting industry as we know it. Companies today must stay on top of the Big Data trend if they wish to

Data warehousing is a system that holds the data of an organization collected through various channels and the data is processed through various analytical tools to generate reports for the business users. This paper discusses the data warehouse concept along with the origin of the data warehouse and the current trends of data warehousing. Various steps involved in the development of the data warehousing project are discussed in this paper. This paper also lists out the challenges encountered while planning, designing and implementing data warehouse projects and the applications of the data warehousing. This paper concludes by discussing the future developments in the data warehousing.

1.1 In 2008 the number of things on the internet surpassed the number of people utilising it, with an estimate of 50 billion active devices by 2020. The Internet of Things (IOT) consists of any objects such as homes, cars or even walking sticks that are able to register external data and communicate this information, usually wirelessly, it has received via its sensors to other devices. Objects on the IOT will be uniquely detectable and allow users to remotely manipulate them via existing networks which will allow for greater data retrieval that is more accurate and increased efficiency of processes.