---
# System prepended metadata

title: Hadoop前置

---

# Hadoop前置

```
sudo apt-get install ssh
```
```
sudo apt-get install pdsh
```
## openjdk
```
sudo apt-get install default-jre
```
```
sudo apt-get install default-jdk
```
## oracle
```
sudo add-apt-repository ppa:webupd8team/java
```
```
sudo apt-get update
```
###### https://www.oracle.com/java/technologies/javase/javase-jdk8-downloads.html
載要的version
(hadoop 3.3.1支援 java 8)
要建帳號才能載
載完執行:

```
sudo su
tar zxvf jdk-8u301-linux-x64.tar.gz -C ../../../usr/lib/jvm/
cd ../../../
update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/jdk1.8.0_301/bin/java" 1
```
再來用以下選java版本:
(會列出來選數字就好)
```
update-alternatives --config java
exit
java -version
```
VM跳出記憶體要爆了記得改多一點
或用下面把不要的刪掉
```
sudo rm -rf /usr/lib/jvm/*
```

## hadoop
```
sudo tar zxvf hadoop-3.3.1.tar.gz 
```
進到
```
hadoop-3.3.1/etc/hadoop/hadoop-env.sh 
```
加入:
```
# set to the root of your Java installation
export JAVA_HOME=/usr/lib/jvm/jdk1.8.0_301
```
(JAVA_HOME是剛放的JDK資料夾位置)
接著再hadoop-3.3.1位置執行:
```
bin/hadoop
```
若跳出文件就是成功了

### Standalone Operation
```
mkdir input
cp etc/hadoop/*.xml input
bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.1.jar grep input output 'dfs[a-z.]+'
cat output/*
```
![](https://i.imgur.com/yeREga9.png)

去output資料夾會有_SUCCESS檔案
![](https://i.imgur.com/QCPOVAj.png)


### Pseudo-Distributed Operation
etc/hadoop/core-site.xml:
```
<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://localhost:9000</value>
    </property>
</configuration>
```
etc/hadoop/hdfs-site.xml:
```
<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>
```
#### 設定SSH
先執行:
```
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 0600 ~/.ssh/authorized_keys
```
有可能Permission Deny
到/etc/ssh/sshd_config:
```
PermitRootLogin yes
```
restart更新:
```
sudo service sshd restart
```
接著確認ssh:
```
ssh localhost
```
會跳出"welcome to Ubuntu...."

#### 設定hadoop環境變數
執行:
```
bin/hdfs namenode -format
sbin/start-dfs.sh
```
會跳出以下畫面:
![](https://i.imgur.com/2hDz0UT.png)

在 etc/hadoop/hadoop-env.sh 加入:
```
export HDFS_DATANODE_USER=<用戶名>
export HDFS_NAMENODE_USER=<用戶名>
export HDFS_SECONDARYNAMENODE_USER=<用戶名>
export YARN_RESOURCEMANAGER_USER=<用戶名>
export YARN_NODEMANAGER_USER=<用戶名>
```
CMD再次執行:
```
bin/hdfs namenode -format
sbin/start-dfs.sh
```
可用以下指令看有沒有成功:
```
jps
```
會顯示:
![](https://i.imgur.com/DTH1RuV.png)

或去 localhost:9870
![](https://i.imgur.com/WH2Ya9u.png)
