preview

How Does Apache Spark Compared To Apache Flink?

Decent Essays
• Week 7 Paper 1 - Knowledge Check Freelancer Manoj Please click on the link above to submit this week's assignment. Answer the following questions. Where required, please provide the complete command line with proper spacing and syntax. Giving examples will earn extra points. 1. Provide a brief history of Spark? 2. How is Spark better than MapReduce? 3. What is a Spark RDD? 4. What is the meaning of a "lazy evaluation" and what are its benefits? 5. What are transformations and actions? Give examples of some transformations and actions. 6. How does Apache Spark compare to Apache Flink? 1. Provide a brief history of Spark? ANS : Spark is cluster framework with an open source software. It was 1st invented by Berkely in…show more content…
Spark RDD follows the concept of coarsely grained operation which means the operation is applied within the dataset for all the elements. By transforming existing RDD or reading data from HDFS RDD’s created. Readable data from stable storage can only create RDD. HDFS and existing transformed data in RDD are a good example of this. Example : firstRDD=spark.textFile("hdfs://...") secondRDD=firstRDD.filter(someFunction); 4. What is the meaning of a "lazy evaluation" and what are its benefits? Ans: Lazy evaluation in Spark means that the execution will not start until an action is triggered. It occurs only when spark transformation occurs i.e spark transformation gives the picture of lazy evaluation. Advantages : i.The user can organize their program into smaller programs which helps better maintaining the code. ii. Time and space complexity issues are addressed as not each and every execution is operated which saves time. The lazy evaluation also works with the infinite data structure. The action is voluntary as it triggers depends on the requirements. It provides performance optimization by reducing the number of queries. 5. What are transformations and actions? Give examples of some transformations and actions. Ans: Transformations: Transformation is the functions which provide new RDD.These RDDs created from the existing RDDs.
    Get Access