Scribd Logo
Upload

Welcome to Scribd!

  • 50 views

    Hadoop Assignmnet2: 1. Name Node

    Uploaded by

    sivaabhilash
    The NameNode is the master node that maintains metadata, the DataNodes store data blocks and send heartbeats to the NameNode. The ResourceManager allocates cluster resources and communicates with the NameNode and NodeManagers, which run tasks and send heartbeats to the ResourceManager.

    Copyright:

    © All Rights Reserved
    Download as DOCX, PDF, TXT or read online from Scribd

    Hadoop Assignmnet2: 1. Name Node

    Uploaded by

    sivaabhilash
    50 views4 pages
    The NameNode is the master node that maintains metadata, the DataNodes store data blocks and send heartbeats to the NameNode. The ResourceManager allocates cluster resources and communicates with the NameNode and NodeManagers, which run tasks and send heartbeats to the ResourceManager.

    Original Description:

    Assignmnet2_Hadoop

    Original Title

    Assignmnet2_Hadoop

    Copyright

    © © All Rights Reserved

    Available Formats

    DOCX, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    The NameNode is the master node that maintains metadata, the DataNodes store data blocks and send heartbeats to the NameNode. The ResourceManager allocates cluster resources and communicates with the NameNode and NodeManagers, which run tasks and send heartbeats to the ResourceManager.

    Copyright:

    © All Rights Reserved
    Download as DOCX, PDF, TXT or read online from Scribd
    Download as docx, pdf, or txt
    50 views4 pages

    Hadoop Assignmnet2: 1. Name Node

    Uploaded by

    sivaabhilash
    The NameNode is the master node that maintains metadata, the DataNodes store data blocks and send heartbeats to the NameNode. The ResourceManager allocates cluster resources and communicates with the NameNode and NodeManagers, which run tasks and send heartbeats to the ResourceManager.

    Copyright:

    © All Rights Reserved
    Download as DOCX, PDF, TXT or read online from Scribd
    Download as docx, pdf, or txt
    You are on page 1of 4

    Hadoop

    Assignmnet2
    1. Explain the importance of below 4 demons in job execution with minimum of 5 points

    1. Name node:

    Name node is the master node which collects and maintains the metadata.

    Whenever client wants to store the data it contacts the Name node.

    Name node gives the available list of data nodes to the client to store the data.

    Collects the heart beat and block report from the data nodes.

    Stores the metadata in the fs_image and edit_log files.

    Belongs to Hadoop 1.x

    2. Data node:

    Data node is responsible to store the physical data in its block.

    Sends heartbeat for every 3 seconds to Name node and block report for every 6 hours.

    Every data node in the cluster has a Task Tracker which is responsible to perform the
    actual computation on the data.

    Data node replicates each input split into two copies and stores in different nodes.

    Belongs to Hadoop 1.x

    3. Resource Manager

    Resource manager is the master node in YARN(MR2) which is responsible for the
    resource management of the cluster.
    Resource manager selects the node with best hardware configuration and creates
    application master.

    Resource manager sends the list of available resources to the Application Master so that
    Application master starts the job in the available Data Nodes.

    Whenever Resource Manager gets the application request from client it checks and
    confirms with the Name Node whether the file is available in the metadata or not.

    If Resource Manager finds the name of existing file it confirms with the client by sending
    APP_ID.

    4. Node Manager:

    Node Manager is the slave daemon and sends ping to the Resource Manager.

    Node manager is so similar like a Task Tracker which is responsible to perform the actual
    computations on the data stored in the Data Nodes.

    Node manager will be controlled by Application Master for job executions.

    Once job is started, Node Manager constantly sends the ping to Application Master.

    2.

    import java.util.*;

    class HadoopString

    public static void main(String args[])

    {
    String line = "Hadoop is open source frame work, We are learning Hadoop
    framework with Helpism";

    // To split the given string using split method

    String mylist[] = line.split(" ");

    int count =0;

    // To check how many time a word has repeated

    for (String i: mylist)

    if (i.equals("Hadoop"))

    count = count+1;

    System.out.println("Number of times Hadoop repeated: " + count);

    // To replace ',' with ';' using replace method


    String line2 = line.replace(",",";");

    System.out.println(line2);

    //array list with generic string and store the spitted string inside it.

    ArrayList<String> list = new ArrayList<String>();

    for(int j=0; j<mylist.length; j++)

    list.add(mylist[j]);

    System.out.println("The generic list: ");

    for(String n: list)

    System.out.println(n);

    You might also like