MapReduce and Design Patterns - Counting with Counters Overview

Counting with counters overview

So, in this particular type of design pattern will be using one utility known as the

counter which will be calculating the global sum on the map site

And this particular design pattern in this implementation will be having only the mapper

tasks and there will be no reducer task. So let us go for more discussion on it

So counting with counters. These patterns use a utility called count. called counters of the MapReduce Framework

So, there is one utility known as the counters that will be used in this design pattern

So, using this utility, the framework can generate a global sum on the map site

So, in the mapper side, the global sum will be obtained. It is a very efficient process to get the sum of a large dataset

So ordinary sum algorithms cannot work here because here we are dealing with the big data

So the data set is used. So, that's why to calculate the sum, we are supposed to use this respective method

So, as an example, we can say that if someone wants to track the login by his employee

to his heavily used public website, he can create this type of counting pattern to do it

a very efficient way. So how many employees have got logged or what is the current logging pattern on a heavily use system if the website owner wants to get that information they can go for this implementation this job is a map job only so it

is having only the map job and no deducer will be used in this case and the

mapper processes each input records and increments the counter value with certain

criteria so mapper will take the key values and for the similar key values

it will just go on increasing the counters accordingly. So, counters of different mapper count the value and send the updates to the task tracker

Here we are not having any reducer, but here we are having tax tracker and then we'll

be having the job tracker also. Then the tax tracker send the progress update to the job tracker and job tracker will produce

the final output. So let us show and discuss it with one proper diagram

So this is the diagram we are having. So, here we are having the counting mapper

So, here we're having the mapper tasks. And then we're having this increment counter D, increment counter B in this way, the mappers

will produce the required outputs. And that will be made available to the tax tracker

So here we are having the tax trackers. And then it is going to the job tracker and job tracker will produce the required outputs

that is counted A, B, C, D in this way. And the job will have the success

To have a better idea, please watch the next video. where you will be discussing on this topic into more details. Thanks for watching this