Written by Team RDPextra - compiled from live keyword-demand data, a read of the current top-ranking Hadoop-on-Ubuntu tutorials, and 20 verified primary sources (Apache Hadoop, Oracle, OpenJDK, Ubuntu, Google Research). Commands tested against Hadoop 3.4.3 on Ubuntu 22.04/24.04 LTS. Research compiled September 2026.

How to Install Hadoop on Ubuntu: Single-Node Setup to Production Scaling

Apache Hadoop is still the reference framework for storing and processing datasets too large for one machine to handle alone.^[1] It splits storage across the Hadoop Distributed File System (HDFS) and processing across MapReduce or YARN-managed jobs, replicating data across nodes so a single disk or server failure doesn't take the dataset down with it.^[2]

This guide installs a working single-node Hadoop cluster on Ubuntu from scratch - the same setup Apache's own documentation recommends for learning and development^[3] - then covers what actually changes when you move from that to a real multi-node production cluster, and when the underlying server hardware (not just the software config) becomes the bottleneck.

Why Hadoop, and Why This Still Matters

• Horizontal scalability. Add nodes to add capacity, rather than replacing hardware.^[1]
• Fault tolerance by design. HDFS replicates each data block (3x by default) across different nodes, so losing one disk or server doesn't lose data.^[2]
• Open source, no per-node licensing. The cost is the infrastructure it runs on, not the software.
• Proven at scale. Hadoop's design descends directly from Google's own published MapReduce and GFS papers on processing web-scale data across commodity clusters.^[4][5]

Hadoop, Java, and Ubuntu Version Compatibility

Version mismatches are the single most common reason a first Hadoop install fails silently or throws obscure class-loading errors. Apache's own compatibility matrix^[6] is stricter than most tutorials imply:

This guide uses Hadoop 3.4.3 with OpenJDK 11 - the combination Apache's own matrix lists as supported and what most production clusters still run today. Hadoop 3.5.0 is newer but forces a JDK 17 upgrade path most existing tooling isn't on yet.^[6]

Prerequisites

• Ubuntu 22.04 or 24.04 LTS^[7], with a user account that has sudo privileges.
• At least 4 GB RAM and 20 GB free disk for a single-node learning setup (production nodes need considerably more - see the sizing section below).
• This walkthrough builds a single-node ("pseudo-distributed") cluster, which is what Apache's own documentation recommends for development and testing.^[3] Production deployments need a multi-node cluster, covered further down.

Step 1: Update Ubuntu

sudo apt update && sudo apt upgrade -y

Step 2: Install OpenJDK 11

Hadoop 3.4.x requires Java - JDK 8 to compile, JDK 11 for runtime.^[6] OpenJDK 11 is the safer default on a fresh Ubuntu install:

sudo apt install openjdk-11-jdk -y
java -version

Step 3: Create a Dedicated Hadoop User and SSH Key

Hadoop's start/stop scripts connect to nodes over SSH, including localhost on a single-node setup - this step is required, not optional, and is one of the most common points a first-time install fails on.^[3] Running Hadoop under its own user also keeps its file permissions separate from your regular account:

sudo adduser hadoop
sudo usermod -aG sudo hadoop
su - hadoop
ssh-keygen -t rsa -P "" -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 0600 ~/.ssh/authorized_keys
ssh localhost

If ssh localhost connects without a password prompt, the key is set up correctly. If it asks for a password, double-check the authorized_keys permissions above - Ubuntu's OpenSSH server rejects overly-permissive key files by default.^[8]

Step 4: Download and Extract Hadoop

wget https://downloads.apache.org/hadoop/common/hadoop-3.4.3/hadoop-3.4.3.tar.gz
tar -xzvf hadoop-3.4.3.tar.gz
sudo mv hadoop-3.4.3 /usr/local/hadoop
sudo chown -R hadoop:hadoop /usr/local/hadoop

Check Apache's official releases page^[9] for the current stable tarball URL before running this - version numbers change.

Step 5: Set Environment Variables

Add the following to ~/.bashrc:

export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
export HADOOP_HOME=/usr/local/hadoop
export HADOOP_INSTALL=$HADOOP_HOME
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin

source ~/.bashrc

Forgetting JAVA_HOME here is the other most common failure point - Hadoop's scripts read it directly and fail with an unhelpful error if it's missing or points to the wrong JDK.

Step 6: Configure Core Hadoop Files

$HADOOP_HOME/etc/hadoop/core-site.xml^[10]:

<configuration>
  <property>
    <name>fs.defaultFS</name>
    <value>hdfs://localhost:9000</value>
  </property>
</configuration>

$HADOOP_HOME/etc/hadoop/hdfs-site.xml^[11] - replication factor 1 is correct for a single-node cluster only; production clusters use 3:

<configuration>
  <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>
</configuration>

Step 7: Format HDFS and Start Services

hdfs namenode -format
start-dfs.sh
start-yarn.sh
jps

jps should list NameNode, DataNode, SecondaryNameNode, ResourceManager, and NodeManager.^[12] If any are missing, check $HADOOP_HOME/logs/ for that daemon's log file - the actual error is almost always in there, not in the terminal output.

Step 8: Verify With a Real MapReduce Job

Hadoop ships example jobs specifically for this.^[13] Running the Pi-estimation example confirms YARN, HDFS, and MapReduce are all actually working together, not just that the daemons started:

hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.4.3.jar pi 16 1000

A completed job with an estimated Pi value printed at the end means the install is working end-to-end.

Single-Node vs. Multi-Node: What Actually Changes

Apache's own cluster-setup documentation^[14] walks through the exact workers file and per-node config changes - the daemons and commands above stay identical, only the topology changes.

Which Setup Fits Your Workload?

Matching Hardware to Hadoop Workloads

Performance Tuning That Actually Matters

• Disk I/O is usually the bottleneck, not CPU. Hadoop's own hardware-provisioning guidance for data-intensive frameworks emphasizes local disk throughput over raw core count^[15] - SSD or NVMe storage measurably reduces HDFS read/write latency versus spinning disks.
• Tune vm.swappiness down. Hadoop daemons perform poorly if the kernel aggressively swaps their memory; lowering vm.swappiness (Linux kernel documentation covers the full parameter set^[17]) keeps JVM heaps resident in RAM.
• Network bandwidth between nodes matters more as the cluster grows. HDFS replication and MapReduce shuffle traffic both move data between nodes constantly^[2] - a slow inter-node link becomes the ceiling on job throughput well before CPU does.
• Keep Ubuntu patched. Ubuntu's own security-notice feed^[18] tracks CVEs against the exact packages a Hadoop node depends on (OpenSSH, OpenJDK, the kernel) - unpatched nodes are a real attack surface, not a theoretical one.

Common Errors and Fixes

• "JAVA_HOME is not set" on startup. Re-check Step 5 - this is almost always a typo'd path or a shell that didn't reload ~/.bashrc.
• "Permission denied (publickey)" on ssh localhost. Re-run the chmod 0600 ~/.ssh/authorized_keys command from Step 3 - Ubuntu's OpenSSH server silently ignores keys with overly permissive file modes.^[8]
• NameNode stuck in safe mode. Normal briefly after formatting; if it persists, run hdfs dfsadmin -safemode leave and check the NameNode log for the underlying cause.
• "Address already in use" on port 9000 or 8088. Another process (often a previous, not-fully-stopped Hadoop instance) is holding the port - run stop-dfs.sh && stop-yarn.sh before starting again.

Frequently Asked Questions

What is the correct Java version for Hadoop on Ubuntu?

For Hadoop 3.3.x-3.4.x, use OpenJDK 8 to compile and OpenJDK 11 for runtime - Apache's compatibility matrix does not support compiling with Java 11.^[6] Hadoop 3.5.0 and newer require JDK 17.

Do I need a multi-node cluster to learn Hadoop?

No. Apache's own documentation recommends starting with a single-node, pseudo-distributed setup for exactly this reason^[3] - it runs every Hadoop daemon on one machine and behaves the same way a cluster does for learning and testing purposes.

Why does Hadoop need passwordless SSH, even on one machine?

Hadoop's start/stop scripts manage daemons by connecting over SSH, including to localhost on a single-node install.^[3] Without a working passwordless key, start-dfs.sh will hang waiting for a password that never gets typed anywhere.

Is Hadoop still relevant, or has Spark replaced it?

They solve overlapping but different problems. Spark is a faster in-memory processing engine and commonly runs on top of HDFS for storage^[19] rather than replacing it - many production stacks run both together.

What hardware bottlenecks a Hadoop cluster first?

Usually disk I/O and inter-node network bandwidth before raw CPU, since HDFS replication and MapReduce's shuffle phase both move large amounts of data between nodes constantly.^[2][15]

Can I run Hadoop on a standard VPS?

For learning and development, yes - that's exactly what this guide sets up. For production datasets with real throughput or fault-tolerance requirements, a dedicated server or multi-node cluster is the appropriate next step, not a bigger VPS.

Conclusion

The steps above produce a genuinely working single-node Hadoop 3.4.3 install on Ubuntu - verified against Apache's own documentation and version-compatibility matrix, not just copied commands. Moving to production means changing the cluster topology (multi-node, 3x replication) far more than it means changing individual commands - and once that happens, the hardware underneath starts to matter: dedicated CPU throughput for MapReduce-heavy clusters, or GPU acceleration for ML workloads reading out of HDFS.

If the quiz above pointed you toward dedicated hardware, RDPextra runs AMD dedicated servers and GPU streaming dedicated servers for exactly these workloads.

References

1. Wikipedia. "Apache Hadoop." en.wikipedia.org. Accessed September 2026.
2. Apache Software Foundation. "HDFS Architecture Guide." hadoop.apache.org. Accessed September 2026.
3. Apache Software Foundation. "Hadoop: Setting up a Single Node Cluster." hadoop.apache.org. Accessed September 2026.
4. Dean, J. & Ghemawat, S. "MapReduce: Simplified Data Processing on Large Clusters." Google Research / OSDI 2004. research.google. Accessed September 2026.
5. Ghemawat, S., Gobioff, H. & Leung, S-T. "The Google File System." Google Research / SOSP 2003. static.googleusercontent.com. Accessed September 2026.
6. Apache Software Foundation Wiki. "Hadoop Java Versions." cwiki.apache.org. Accessed September 2026.
7. Canonical. "Ubuntu Release Cycle." ubuntu.com. Accessed September 2026.
8. Canonical. "OpenSSH Server | Ubuntu Server Documentation." ubuntu.com. Accessed September 2026.
9. Apache Software Foundation. "Apache Hadoop Releases." hadoop.apache.org. Accessed September 2026.
10. Apache Software Foundation. "core-default.xml Configuration Reference." hadoop.apache.org. Accessed September 2026.
11. Apache Software Foundation. "hdfs-default.xml Configuration Reference." hadoop.apache.org. Accessed September 2026.
12. Apache Software Foundation. "HDFS Commands Guide." hadoop.apache.org. Accessed September 2026.
13. Apache Software Foundation. "MapReduce Tutorial." hadoop.apache.org. Accessed September 2026.
14. Apache Software Foundation. "Hadoop Cluster Setup." hadoop.apache.org. Accessed September 2026.
15. Apache Spark. "Hardware Provisioning." spark.apache.org. Accessed September 2026.
16. NVIDIA. "RAPIDS: Open GPU Data Science." developer.nvidia.com. Accessed September 2026.
17. The Linux Kernel Documentation. "Documentation for /proc/sys/vm/ (vm.swappiness et al.)." kernel.org. Accessed September 2026.
18. Canonical. "Ubuntu Security Notices / CVE Tracker." ubuntu.com. Accessed September 2026.
19. Apache Software Foundation. "Apache Hadoop" (project homepage, ecosystem overview including Spark integration). hadoop.apache.org. Accessed September 2026.
20. Apache Software Foundation. "Apache Hadoop YARN." hadoop.apache.org. Accessed September 2026.