MapReduce and Design Patterns - Top Ten Pattern Example

In this video we are discussing top 10 pattern example

So this video will be assisted with some one practical demonstration for the easy understanding of this concept

So top 10 pattern example and from the users.xml. So here we are using the XML file that is the users.xml

This task will find top 10 users based on the view count

So depending upon the view count, we'll be calculating or finding out those users who are the top 10

So let us go for one demonstration for the easy understanding of this concept

We are implementing filtering pattern design pattern under this category top 10 users from

users dot XML file. So user dot XML is a huge file. It is having a size of 56.18mb deciding

under this slash input slash user folder and we are going to show you the some first two records

first two rows of these users dot XML under the user's tag. Each

every row has got multiple attributes and fields we're having the attribute like your

ID here so here we are having so many rows will be there containing ID reputation

creation date display name last access date going for location website URL

location about me then you shall go for views we shall go for views and then abhorts

votes account ID so these are the multiple attributes are there under each and every

row thousands of rows are there so let us concentrate on the Java programs what are

the Java files we have written here so here we are supposed to find out top

ten users having got the maximum views so we are going to have our Java class

that is a top 10 MR task where we are extending the inner class is our top 10

mapper extending the mapper under this we are creating one t-tree map object that is a shorted map we're creating tree

map object we're overriding the map method we're overriding the map method

within the map method we're having the hash map object there is our XML parsed

so XML pars is my hash map object which will be instantiated using XML to map

we having one method here so let me show you this particular XML to map will take one XML file as input and returns the hash map object as output so that will be assigned onto this object that is our XML 105 past we having a string type ID and string value which will be taking the respective that attribute content from the XML ID and views so string ID and string view so to member to two member will

were defined under this method so we're having this if ID is equal to is equal to null

and views equal is equal now then return so this part of the code will become unreachable unexecuted otherwise shorted map dot put long parts long dot parts long

so converting this string to long and new text value so value will be converted to this

text and add item into the list so in this particular list the item will be added

here so we are going to check that whether this shorted map dot size if it is greater than 0 greater than 10 rather if the shorted map

dot size is greater than 10 then short at map dot remove short at map dot Forski so remove

element where view is lowest so we are supposed to find out the list of those users

having got the maximum views so top 10 views top 10 views will be there so those

users will be required we have kept this one in the try catch block for the exception

handling now we shall discuss the method which we overriding that is a cleanup so in

the cleanup method for text text text into our shorted map dot values we're going

to write the context dot write null writable dot get as a key and text as a value

so it will be writing the key and the respective value there so we'll be going

for the next reducer class that is a top ten reducer is the name of the class

here also we have defined one tree map objects that is a top 10 list we have defined one tree map object that is a top 10

list we're overriding the deduce method overriding the reduced method under this xml

parts is this there xml parsed for each and every value in the values we're going for the

xml parsed object that is a hash map object to be instantiated then xml parse

dot get views so from the view tag we are taking this so there is a string view is

equal to xml parse dot get views and top list dot put long dot parts long view and new text value so here key will be of the type of long and value will be of the type of text so if top 10

list dot sizes greater than 10 then top 10 remove top 10 this dot remove top top 10 list

list dot first key so you are trying to keep only the topmost 10 most having the most

views so that takes text text so that is a top ten list a descending map

dot values so CTX dot write that is a context dot write now writeable dot get so

there is a key and text is a value we are going to write them and putting them in

the try catch block so let me explain the main function here here the argument

the number of common land argument should be passed as to the first one is my top

ten MR tax that is a class name then input folder name then output folder name

If the number of arguments is not equal to two, then we shall be going for exit

Instantiating the job class object job. So job is equal to get instance and this

We're also going for the mapper class and the reducer class to be assigned. We are checking that the number of reduced duct should be one

And the output key and output value. These two will be initialized at the type of null writable class and text class

We'll be writing them. then file input format dot add input path that will be obtained from arg 0 and set output path

that will be obtained from arg 1 code is a value which will be deciding the successful

completion of the code and that code will be returned so already we have generated the

jar file against this particular project so we are not going to create the jar file

otherwise you know we shall go for right button click and then export and then jar

files so that step we are omitting so let me go for executing our program and let me see

that how the outputs are coming we're supposed to find out those 10 users having got the

maximum number of views so let me go for clear then we are issuing the command

that is our we should type the command as Hadoop jar and then respective path has to be given map reduce underscore design pattern jar files is a path and is a jar file name that is a filtering pattern dot jar top 10 is a package name top 10 MR tasks is

a risk a MR task is the respective class name then you shall give the input path so

slash input slash user and output path that is our say slash output

executing the command I'm not finding the

failed actually I think the name node now in the sleep mode so let me take it out

from the sleep mode so hadu DFS admin minus save mode leave

You shall execute the command once again. Yes, now it has got the successful output

Let me go to the output folder. Under this output folder, the part file has got created

So let me see the content of the part file. content of the part file. So we shall go for the console once again. So, HDFS, DFS minus

cat and then slash output slash part star. So here the list of those users having got top

10 views. So those user details have been retrieved from the XML file and they have got dumped

onto this part file as output. So these are the top 10 users having got the most visits are there views are there so they have got listed

so it is a good practice optional it is a good practice to delete the output folder

i hope that you have got the idea that how we have executed the code we have explained the

code we have explained step by step how it can be done so let me go for the deletion of

the output folder slash output Thanks for watching this video