Explain Apache Hadoop's MapReduce .

-

Apache Hadoop's MapReduce 

  • Apache Hadoop's MapReduce is the most widely used batch data processing. The diagram below explains in detail how Hadoop processes data using MapReduce.


  • MapReduce Hadoop MapReduce is a Java-based system for processing large datasets. It reads data from the HDFS and divides the dataset into smaller pieces. Each piece is then scheduled and distributed for processing among the nodes available in the Hadoop cluster. Each node performs the required computation on the chunk of data and is intermediate results obtained are written back to the HDFS. These intermediate outputs may then be sembled, split, and redistributed for further processing until final results are written back to HDFS.
  • As already discussed above, the MapReduce data processing programming model consists of two different jobs executed by programs: a Map job and a Reduce job. Typically, the Map operation begins by turning a collection of data into another set of data in which individual pieces of the data are broken down into tuples consisting of key-value pairs. One or more Map tasks can then shuffle, sort, and process these key-value pairs. The Reduce task typically takes as input the results of a Map task and merges those data tuples into a smaller collection of tuples.
  • Batch processing, in a nutshell, is a way of waiting and performing everything periodically such as at the end of the day, week, or month. In the enterprise, during the specified period, the cumulative data will be large. So, to handle such big data, distributed computing environment, and the MapReduce technique can play a vital role.

Comments

Post a Comment

Popular posts from this blog

Suppose that a data warehouse consists of the four dimensions; date, spectator, location, and game, and the two measures, count and charge, where charge is the fee that a spectator pays when watching a game on a given date. Spectators may be students, adults, or seniors, with each category having its own charge rate. a) Draw a star schema diagram for the data b) Starting with the base cuboid [date; spectator; location; game], what specific OLAP operations should perform in order to list the total charge paid by student spectators at GM Place in 2004?

-
Image
  b) The specific OLAP operations to be performed are: 1. Roll-up on date from date id to year. 2. Roll-up on spectator from spectator id to status. 3. Roll-up on location from location id to location name. 4. Roll-up on the game from game id to all. 5. Dice with status= “students”, location name= “GM Place”, and year=2004
Read more

Define Business ethics . Explain its significance.

-
Image
Business ethics   Business ethics are moral principles that guide the way a business behaves, wrong Acting in an ethical way involves distinguishing between right and then making the right choice. Business ethics are applied or professional ethics. They may take the form of actions within written and unwritten codes of principles, and values and are determined by an organization's culture. They largely govern the decisions in the organization. However, determining what practices are ethical or not is more difficult since the lines between ethical and unethical can become blurred. The following are some of the notable definitions of business ethics Decenzo & Robbins Ethics commonly refers to a set of rules or principles that defined right and wrong conduct. Ricky W. Griffin Ethics is an individual's personal beliefs about whether a behavior, action o decision is right or wrong. Managerial ethics are the standard of behavior the guides individual managers in their work. Clara...
Read more

Short Note on Security Architecture of E-governance

-
Image
 Security Architecture The security architecture of E-governance is a high-level document that sets the security goals of the e-governance project and describes the procedure that needs to be followed by all the e-governance hierarchies such as users, businesses, operators, etc. An appropriate legal framework is absolutely essential for the systematic and sustained growth of e-governance. Protection of Public Order and Decency  The internet is highly capable of being a saturated and versatile medium at the same time. Its reach is very vast and due to its multimedia capability, its impact can be immediate and profound. So the government has to beware of its potential to create a negative impact on society through the promotion of terrorism, pornography, communalism, violence, etc. Section 67 of the IT Act 2000 of India makes it a punishable offense to “publish or transmit or cause to be published it the electronic form, a material which is lascivious or appeals to the prurient ...
Read more