UMBC – CSEE Department Data Science Program Spring 2022 DATA 603 – Big Data Platforms Homework #6 –

downloaded from

Scholarfriends

https://scholarfriends.com Downloaded by: writerforacademic@gmail.com | https://scholarfriends.com/singlePaper/310871/umbc-csee-department-data-scienceprogram-spring-2022-data-603-big-data-platforms-homewo Distribution of this document is illegal.

UMBC – CSEE Department Data Science Program Spring 2022

Dr. Waleed Youssef youssef1@umbc.edu

DATA 603 – Big Data Platforms Homework #6 – Spark Programming

Questions: (1) [10 points] Simulate the Aggregate() example in the slides, with initial value of (1,0), What would be the expected results? a. List is [1, 2, 3, 4] b. Partition is 2 c. Initial zeroValue is (1, 0)

This study source was downloaded by 100000858468549 from CourseHero.com on 04-02-2023 13:04:21 GMT -05:00

https://www.coursehero.com/file/140654537/HW6-Spark-Programmingdocx/

(2) [20 points] Write a Spark program that reads your browser history file, then displays the top 5 websites you visited in the last week?

This study source was downloaded by 100000858468549 from CourseHero.com on 04-02-2023 13:04:21 GMT -05:00

https://www.coursehero.com/file/140654537/HW6-Spark-Programmingdocx/

(3) [20 points] Implement a spark program that performs the following: a. Reads the posted text file for a book named “Applied Data Science.txt” b. Read the text file into an RDD, and then perform actions and transformations on the RDD c. Displays the most used 5 words of length greater than 5 characters in the file (ensure you result is not case sensitive, so the word “Data” and “data” are the same and should be counted two of the same word) d. The output should be like this: The most used words in the Applied Data Science textbook are: <<word1>> occurred <<n1>> times <<word2>> occurred <<n2>> times <<word3>> occurred <<n3>> times <<word4>> occurred <<n4>> times <<word5>> occurred <<n5>> times

This study source was downloaded by 100000858468549 from CourseHero.com on 04-02-2023 13:04:21 GMT -05:00

https://www.coursehero.com/file/140654537/HW6-Spark-Programmingdocx/

I have tried a lot, but I didn’t get the output for this question. It was showing that there is java errors but if we observe here if it is the java error I should not get second answer as well but it executed well. So I was confused and there is no time for the assignment.

This study source was downloaded by 100000858468549 from CourseHero.com on 04-02-2023 13:04:21 GMT -05:00

https://www.coursehero.com/file/140654537/HW6-Spark-Programmingdocx/ Powered by TCPDF (www.tcpdf.org)