Explain HDFS(Hadoop File System) architecture and it's goal .

-

 HDFS Architecture

The architecture of a Hadoop File System is shown in figure 7.2. Those components of the architecture are described as below:



Namenode

The namenode is the commodity hardware that contains the GNU/Linux operating system and the namenode software. It is software that can be run on commodity hardware. The system having the namenode acts as the master server and it does the following tasks:

•Manages the file system namespace. 

•Regulates client's access to files.

•It also executes file system operations such as renaming, closing, and opening files and directories.


Datanode

The datanode is a piece of commodity hardware that runs the GNU/Linux operating system as well as the datanode software. A datanode will exist for each node (Commodity hardware/System) in a cluster. These nodes are in charge of their system's data storage.

Datanodes perform read-write operations on the file systems, as per the client's request. 

They also perform operations such as block creation, deletion, and replication according to the instructions of the namenode.


Block

In general, user data is saved in HDFS files. In a file system, the file will be partitioned into one or more segments and/or stored in separate data nodes. Blocks are the name given to these file chunks. In othe terms, a Block is the smallest quantity of data that HDFS can read or write. The default block size is 64MB, although it may be adjusted in the HDFS setup as needed.


Goals of HDFS

Fault detection and recovery: Because HDFS relies on a large number of commodity hardware components, component failure is common. As a result, HDFS should have techniques for rapid and automated failure detection and recovery.

Huge datasets: To manage applications with large datasets, HDFS should have hundreds of nodes per cluster.

Hardware at data: When the computation takes place near the data, a requested job can be completed quickly. It decreases network traffic while increasing throughput, especially when large datasets are involved.

Comments

Post a Comment

Popular posts from this blog

Suppose that a data warehouse for Big-University consists of the following four dimensions: student, course, semester, and instructor, and two measures count and avg_grade. When at the lowest conceptual level (e.g., for a given student, course, semester, and instructor combination), the avg_grade measure stores the actual course grade of the student. At higher conceptual levels, avg_grade stores the average grade for the given combination. a) Draw a snowflake schema diagram for the data warehouse. b) Starting with the base cuboid [student, course, semester, instructor], what specific OLAP operations (e.g., roll-up from semester to year) should one perform in order to list the average grade of CS courses for each BigUniversity student. c) If each dimension has five levels (including all), such as “student < major < status < university < all”, how many cuboids will this cube contain (including the base and apex cuboids)?

-
Image
a) A Snowflake Shema is shown in figure below: b) The specific OLAP operations to be performed: 1. Roll-up on course from course_id to department. 2. Roll-up on a student from student_id to university. 3. Dice on course, student with department ="CS" and university = "biguniversity" 4. Drill-down on a student from the university to student_name. c) N = 4 dimensions The cube will contain 54 = 625 cuboids.
Read more

Describe how cloud computing technology can be applied to support remote ECG monitoring.

-
Image
  Healthcare: ECG (Electrocardiogram) Analysis in cloud computing Healthcare is a field, subject, sector, or territory in which information technology has several uses. These apps are being used to aid businesses in supporting scientists in developing disease prevention solutions. Cloud computing has emerged as an appealing alternative for constructing health monitoring systems as a result of the development of the Internet or to put it another way, as a result of the availability of the internet. The ECG machine, for example, is a health monitoring device that measures the human body's heartbeat and prints the results on graph paper. The electrocardiogram (ECG) measures the electrical activity of the heart's Cardium. A waveform is created as a result of this action, which is repeated throughout time and symbolizes the heartbeat.  The analysis of the shape is used to identify arrhythmias, and it is the most common way of detecting heart diseases. Here the meaning of arrhythmias
Read more

Explain market-Oriented Cloud computing architecture.

-
Image
  Market Oriented Cloud Computing (Mocc) As customers rely on Cloud providers to cover all of their computing needs, they will expect certain Qe from their providers to fulfill their objectives and continue their operations. Cloud providers must examine and satisfy the various QoS standards of each unique consumer as stipulated in unique SLAs. To do this, cloud providers cannot continue to install traditional system-centric resource management architectures that do not give incentives for them to share their resources while still treating all service requests as equal insignificance. Instead, market-oriented resource management is required to maintain the supply and demand for Cloud resources at market equilibrium, provide feedback in the form of economic incentives for both Cloud consumers and providers, and promote QoS-based resource allocation mechanisms that differentiate service requests based on their utility. The diagram depicts a high-level architecture for enabling market-orie
Read more