Add a Hadoop Monitor

Auto-discover and monitor your entire Hadoop cluster, view performance metrics including DFS used, YARN metrics, memory and log statistics; and be notified on any failure in your DataNode, NameNode, or YARN. 

• Prerequisites
• Add a Hadoop monitor
• Add a threshold and availability profile
• Licensing

Prerequisites

• Ensure the latest version of the Linux agent (17.5.0 and above) is installed in your NameNode to monitor your entire Hadoop cluster. Site24x7 Hadoop monitoring uses the Linux monitoring agent to collect performance data from your Hadoop cluster.
• Make sure the option Auto discover applications on the server is set to Yes in the Settings page (Admin > Server Monitor > Settings).

Add a Hadoop Monitor

Please follow the steps below:

1. Log in to Site24x7 and go to Server > Hadoop > Hadoop Clusters (+). 
2. Use the toggle button to install the agent as root or non root, and run the agent as root or non root after installation.
   For eg: To install the agent as a root user and run the agent as non root with proxy enabled, execute the following command:
   
   bash -c "$(curl -sL https://staticdownloads.site24x7.com/server/Site24x7InstallScript.sh)" readlink -i -key=<device_key> -proxy=user:password@proxyhost:proxyport -s24x7-agent
3. Copy-paste (click on Copy to Clipboard) and execute the command in the particular NameNode to download and install the Linux agent.

Once the command is successfully executed in your Hadoop NameNode, the Linux agent will be installed and the agent will auto-discover and add a monitor for each NameNode, DataNode, and YARN in the Site24x7 web client.  You will be able to see the monitors under Server > Hadoop > click on the Hadoop cluster name > NameNodes or DataNodes or YARN. 

Add a Threshold and Availability Profile

Once the Hadoop monitor is successfully added to your Site24x7 account, add a threshold and availability profile to help the alarms engine decide if a specific resource has to be declared critical or down. Learn more.

Licensing

Every Hadoop monitor is a basic monitor. Each NameNode, DataNode, and YARN will consume one basic monitor license.

Related Articles

• Performance metrics for Hadoop
• Add a Linux monitor | Performance metrics for Linux
• Add a: Docker | SMART Disk | Plugin
• Server monitoring architecture
• Other OS platforms supported: Windows | FreeBSD | OS X