Scribd Logo
Upload

Welcome to Scribd!

  • 114 views

    Strategy & Roadmap For Bigtop & Ambari

    Uploaded by

    Keng Oei Quek
    Bigtop and Ambari help manage an Apache big data services stack on AWS. Bigtop provides packages to easily install services like Hadoop, HBase, Hive and Spark as a cohesive stack. Ambari allows configuration, deployment and monitoring of these services through a web interface. Together, Bigtop and Ambari help control administration of big data environments by simplifying installation with Bigtop and providing management with Ambari.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Strategy & Roadmap For Bigtop & Ambari

    Uploaded by

    Keng Oei Quek
    114 views5 pages
    Bigtop and Ambari help manage an Apache big data services stack on AWS. Bigtop provides packages to easily install services like Hadoop, HBase, Hive and Spark as a cohesive stack. Ambari allows configuration, deployment and monitoring of these services through a web interface. Together, Bigtop and Ambari help control administration of big data environments by simplifying installation with Bigtop and providing management with Ambari.

    Original Description:

    Strategy & Roadmap for Bigtop & Ambari

    Original Title

    Strategy & Roadmap for Bigtop & Ambari

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Bigtop and Ambari help manage an Apache big data services stack on AWS. Bigtop provides packages to easily install services like Hadoop, HBase, Hive and Spark as a cohesive stack. Ambari allows configuration, deployment and monitoring of these services through a web interface. Together, Bigtop and Ambari help control administration of big data environments by simplifying installation with Bigtop and providing management with Ambari.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    114 views5 pages

    Strategy & Roadmap For Bigtop & Ambari

    Uploaded by

    Keng Oei Quek
    Bigtop and Ambari help manage an Apache big data services stack on AWS. Bigtop provides packages to easily install services like Hadoop, HBase, Hive and Spark as a cohesive stack. Ambari allows configuration, deployment and monitoring of these services through a web interface. Together, Bigtop and Ambari help control administration of big data environments by simplifying installation with Bigtop and providing management with Ambari.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 5

    To install and configure the Open Source Big Data Apache services on an AWS VM using Bigtop, follow the

    steps below:

    1. Set up an AWS VM:

    • Launch an EC2 instance with the desired specifications.

    • Configure security groups and access permissions to allow communication between services.

    2. Install and configure Bigtop:

    • Install the Bigtop repository on the AWS VM following the instructions provided by Bigtop.

    • Update the package list and install Bigtop on the VM.

    • Configure the Bigtop repository to use the desired version (which is mentioned on the provided
    requirement file) of Apache Hadoop, Apache Spark, and other services.

    3. Install Ambari:

    • Use the Bigtop package manager to install Ambari.

    • Follow the Ambari documentation to configure and set up the Ambari server.

    4. Install and configure Hadoop (MR, YARN & HDFS):

    • Use the Bigtop package manager to install Hadoop.

    • Configure the core Hadoop components, including MapReduce (MR), Yet Another Resource
    Negotiator (YARN), and Hadoop Distributed File System (HDFS).

    5. Install and configure HBase:

    • Use the Bigtop package manager to install HBase.

    • Configure HBase by specifying the necessary properties, such as ZooKeeper connection details and
    data directories.

    6. Install and configure Hive:

    • Use the Bigtop package manager to install Hive.

    • Configure Hive by setting up the Hive Metastore (HMS) and specifying the necessary properties.

    7. Install and configure Hue:

    • Use the Bigtop package manager to install Hue.

    • Follow the Hue documentation to configure and set up Hue.

    8. Install and configure Oozie:

    • Use the Bigtop package manager to install Oozie.


    • Configure Oozie by setting up the necessary properties, such as the Oozie database and
    authentication settings.

    9. Install and configure Spark:

    • Use the Bigtop package manager to install Spark.

    • Configure Spark by specifying the necessary properties, such as the Spark master URL and
    executor settings.

    10. Install and configure Zookeeper:

    • Use the Bigtop package manager to install ZooKeeper.

    • Configure ZooKeeper by setting up the ZooKeeper data directory and connection details.

    NOTE:

    1. Install OS, then install bigtop and configure it. After that, install all services via BIGTOP repository
    manager. No custom installation, later-on it will be hard to manage.

    2. Take VM snapshots at every stage.

    3. even Hadoop and its distribution will be installed via big top. There are plenty of videos available on
    YouTube pls refer.
    Bigtop and Ambari play important roles in the installation and management of the Open Source Big Data
    Apache services stack.

    1. Bigtop:

    • Bigtop is an open-source project that provides a framework and packaging system for Apache Big
    Data components.

    • It simplifies the installation and configuration process by providing pre-built packages for various
    Apache projects, including Hadoop, HBase, Hive, Spark, ZooKeeper, and others.

    • Bigtop ensures that all the components in the stack are compatible and work together seamlessly.

    • It manages dependencies, configurations, and integration between the different services, allowing
    for easier deployment and maintenance of the entire stack.

    • By using Bigtop, you can leverage its package manager to install, upgrade, and manage the
    different Apache services consistently.

    2. Ambari:

    • Ambari is an open-source management and monitoring platform for Apache Hadoop clusters.

    • It provides a web-based interface to simplify the administration and monitoring of the Apache Big
    Data stack.

    • With Ambari, you can easily deploy and manage the various services, configure their properties,
    and monitor their health and performance.

    • Ambari integrates with the underlying infrastructure to automate the provisioning, configuration,
    and management of the cluster nodes.

    • It also offers features like service alerts, metrics collection, and centralized logging for better
    cluster monitoring and troubleshooting.

    • Ambari supports role-based access control, allowing different users or teams to have specific
    permissions and access levels for managing the services.

    In the context of installing and configuring the Open-Source Big Data Apache services, Bigtop helps by
    providing the packages and framework to easily install and manage the services as a cohesive stack.
    Ambari, on the other hand, acts as a management and monitoring tool that allows you to configure,
    deploy, and monitor the services using its user-friendly web interface.

    Both services together ambari and bigtop help data engineers to take more control on administration of
    big data env.

    MUST VISIT LINKS:

    https://www.youtube.com/results?search_query=installation+hadoop+via++apache+bigtop

    https://www.youtube.com/watch?v=7aC98dZHQkU
    HIGH LEVEL CONCEPTUAL DIAGRAM

    Internet

    CentOS VM

    BIGTOP

    Bigtop Package Manager

    Installed services via BIGTOP Package


    manager and being managed by BIGTOP.

    You might also like