HDFS 单节点安装

HDFS 单节点安装指南 (Debian/Linux)

环境准备

sudo apt update
sudo apt install -y ssh openjdk-11-jdk

防火墙配置

sudo ufw allow 9870/tcp  # NameNode
sudo ufw allow 9864/tcp  # DataNode
sudo ufw allow 9000/tcp  # RPC

1.创建专用用户

sudo adduser hadoop
sudo usermod -aG sudo hadoop
su - hadoop

2. 配置SSH免密登录

ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
ssh localhost  # 测试连接

3.安装Hadoop

wget https://downloads.apache.org/hadoop/common/hadoop-3.3.6/hadoop-3.3.6.tar.gz
tar -xzf hadoop-3.3.6.tar.gz
mv hadoop-3.3.6 ~/hadoop

4.环境变量配置

编辑 ~/.bashrc 添加：

export HADOOP_HOME=~/hadoop
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64

生效配置：

source ~/.bashrc

5. 核心配置文件

进入配置目录：

cd ~/hadoop/etc/hadoop

配置 core-site.xml：

<configuration>
  <property>
    <name>fs.defaultFS</name>
    <value>hdfs://0.0.0.0:9000</value>
  </property>
</configuration>

配置 hdfs-site.xml

<configuration>
  <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>
  <property>
    <name>dfs.namenode.name.dir</name>
    <value>file:///home/hadoop/hadoopdata/hdfs/namenode</value>
  </property>
  <property>
    <name>dfs.datanode.data.dir</name>
    <value>file:///home/hadoop/hadoopdata/hdfs/datanode</value>
  </property>
  <property>
    <name>dfs.namenode.http-address</name>
    <value>0.0.0.0:9870</value>
  </property>
</configuration>

配置 hadoop-env.sh

export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64

6. 初始化HDFS

mkdir -p ~/hadoopdata/hdfs/{namenode,datanode}
hdfs namenode -format

7. 启动集群

start-dfs.sh

验证安装

进程检查：

jps

应显示：

Web界面访问：

NameNode UI: http://<服务器IP>:9870

DataNode UI: http://<服务器IP>:9864