:::warning
It's easier to use FRP, please refer to [FRP+Nginx+Certbot with EC2](https://hackmd.io/CAk-fCCLSpi4Vih-FQ5hxA?view=)
:::
# Reverse SSH Tunneling between EC2 (Amazon Linux) and RPi4
[TOC]
## Introduction
Remote management of a Raspberry Pi 4 (RPi4) without a public IP address can be challenging.
This guide outlines how to securely access your RPi4 from anywhere by setting up a reverse SSH tunnel via an Amazon EC2 instance.
This method enhances security by avoiding direct exposure of the RPi4 to the internet while providing reliable remote access.
By following these instructions, you will establish a secure and flexible connection, leveraging the AWS infrastructure to manage your device remotely.
---
## 1. Settings on EC2
### Security Group Example
- Allow Inbound SSH (port 22) from `0.0.0.0/0`.
- Allow Inbound TCP port `2222` from `0.0.0.0/0`.
- Allow Inbound TCP port `8080` from `0.0.0.0/0`.
- Allow Inbound TCP port `1194` from `0.0.0.0/0` (for OpenVPN).

### `sshd_config`
1. Edit the SSH configuration file:
```sh
sudo vim /etc/ssh/sshd_config
```
2. Enable `GatewayPorts` to allow remote port forwarding:
```
AllowTcpForwarding yes
ClientAliveCountMax 5
ClientAliveInterval 30
GatewayPorts yes
TCPKeepAlive yes
# LogLevel DEBUG
```

(https://hackmd.io/_uploads/ByAVSOFLJx.png)
3. Restart the SSH service:
```sh
sudo systemctl restart sshd
# If want to read log
# sudo journalctl -u sshd | vim -
```
---
## 2. Settings on RPi4
### `.ssh/config`
1. Edit the SSH configuration file:
```sh
vim ~/.ssh/config
```
2. Add the following configuration:
```
Host ec2-for-tunneling
HostName ec2-public-ip
User ec2-user
IdentityFile /home/pi/.ssh/ec2.pem
```
- Replace `ec2-public-ip` with the public IP of your EC2 instance.
---
### Forward EC2:2222 to RPi4:22 (SSH RPi4 via EC2)
This allows you to connect to RPi4 via EC2 using:
```sh
ssh -p 2222 -i <rpi4_identity> pi@ec2-public-ip
```
1. Create a service file:
```sh
sudo vim /etc/systemd/system/autossh-tunnel-2222-22.service
```
2. Add the following content:
```ini
[Unit]
Description=AutoSSH tunnel from EC2:2222 to localhost:22
After=network.target
[Service]
User=pi
ExecStartPre=/bin/bash -c 'STALE_PIDS=$(ss -tnp state CLOSE-WAIT sport = :22 | awk "{print $6}" | cut -d, -f2 | xargs -I{} echo {} | cut -d= -f2 | sort -u); if [ -n "$STALE_PIDS" ]; then sudo kill -9 $STALE_PIDS || true; fi'
# ExecStart=/usr/bin/autossh -M 0 -N -R 2222:localhost:22 -o "ServerAliveInterval 30" -o "ServerAliveCountMax 5" -o "ExitOnForwardFailure=yes" -o "TCPKeepAlive=yes" -o "UserKnownHostsFile=/dev/null" -o "StrictHostKeyChecking=no" -v ec2-for-tunneling
ExecStart=/usr/bin/ssh -N -R 2222:localhost:22 -o "ServerAliveInterval 30" -o "ServerAliveCountMax 5" -o "ExitOnForwardFailure=yes" -o "TCPKeepAlive=yes" -o "UserKnownHostsFile=/dev/null" -o "StrictHostKeyChecking=no" -v ec2-for-tunneling
Restart=always
RestartSec=5
ExecStop=/usr/bin/pkill -f "ssh -N -R 2222:localhost:22"
[Install]
WantedBy=multi-user.target
```
- **User**: ==Remember to modify user if it's not `pi`==
- **ExecStartPre**: Cleans up stale SSH processes in `CLOSE-WAIT` state on EC2 before starting the tunnel. The `|| true` ensures the service starts even if this step fails.
- **ExecStart**: Uses `autossh` with robust retry logic.
- **Restart=always**: Ensures the service restarts after failure.
- **ExecStop**: Kills stale SSH processes when the service stops.
3. Start and enable the service:
```sh
sudo systemctl daemon-reload
sudo systemctl restart autossh-tunnel-2222-22
sudo systemctl enable autossh-tunnel-2222-22
sudo systemctl status autossh-tunnel-2222-22 --no-pager
# If want to read the log
# sudo journalctl -u autossh-tunnel-2222-22 | vim -
```
---
### Forward EC2:{port} to RPi4:{port}
This allows you to access services running on RPi4:{port} via EC2:{port}.
1. Run the following script
Simply edit the ports listed
```sh=
#!/bin/bash
ports=(1194 3306 8080 8081 8484 8585 8586 8587 8596)
service_template='[Unit]
Description=AutoSSH tunnel from EC2:PORT to localhost:PORT
After=network.target
[Service]
User=pi
ExecStartPre=/bin/bash -c '"'"'STALE_PIDS=$(ss -tnp state CLOSE-WAIT sport = :PORT | awk "{print $6}" | cut -d, -f2 | xargs -I{} echo {} | cut -d= -f2 | sort -u); if [ -n "$STALE_PIDS" ]; then sudo kill -9 $STALE_PIDS || true; fi'"'"'
# ExecStart=/usr/bin/autossh -M 0 -N -R PORT:localhost:PORT -o "ServerAliveInterval 30" -o "ServerAliveCountMax 5" -o "ExitOnForwardFailure=yes" -o "TCPKeepAlive=yes" -o "UserKnownHostsFile=/dev/null" -o "StrictHostKeyChecking=no" -v ec2-for-tunneling
ExecStart=/usr/bin/ssh -N -R PORT:localhost:PORT -o "ServerAliveInterval 30" -o "ServerAliveCountMax 5" -o "ExitOnForwardFailure=yes" -o "TCPKeepAlive=yes" -o "UserKnownHostsFile=/dev/null" -o "StrictHostKeyChecking=no" -v ec2-for-tunneling
Restart=always
RestartSec=5
ExecStop=/usr/bin/pkill -f "ssh -N -R PORT:localhost:PORT"
[Install]
WantedBy=multi-user.target'
for port in "${ports[@]}"; do
service_content="${service_template//PORT/$port}"
service_file="/etc/systemd/system/autossh-tunnel-${port}-${port}.service"
echo "$service_content" | sudo tee "$service_file" > /dev/null
sudo systemctl daemon-reload
echo "Service file for port $port has been updated: $service_file"
done
```
2. Start and enable the service:
```sh=
#!/bin/bash
# List of ports
ports=(1194 3306 8080 8081 8484 8585 8586 8587 8596)
# Loop through each port and enable/start/restart the service
for port in "${ports[@]}"; do
service_name="autossh-tunnel-${port}-${port}.service"
# Enable the service (if not already enabled)
sudo systemctl enable "$service_name"
# Restart the service
echo "Restarting service $service_name"
sudo systemctl restart "$service_name"
# Optionally, check the status of the service
echo "Status of $service_name:"
sudo systemctl status "$service_name" --no-pager
echo "Service for port $port has been enabled and managed: $service_name"
done
```
---
### Forward for OpenVPN
:::warning
**WireGuard** only supports UDP, while SSH Tunneling only support TCP.
Therefore it's tricky to use **WireGuard** with SSH Tunneling.
We chose to use **OpenVPN** here.
:::
This allows you to securely access your RPi4 and its local network via OpenVPN.
1. **Install PiVPN on RPi4**:
- Update the system:
```sh
sudo apt update && sudo apt upgrade -y
```
- Install PiVPN:
```sh
curl -L https://install.pivpn.io | bash
```
- Choose **OpenVPN** as the VPN type.
- Select **TCP** as the protocol.
- Use the default port **1194**.
- Set up a static IP for your RPi4 if not already configured.
- Use a DNS name or the public IP of your EC2 instance for the VPN connection.
- Reboot the RPi4:
```sh
sudo reboot
```
2. **Generate a client profile**:
After installation, generate a client profile for your device:
```sh
pivpn add
```
- Provide a name for the client (e.g., `mydevice`).
- The profile will be saved in `/home/pi/ovpns/`.
3. **Transfer the client profile**:
Use `scp` to securely transfer the `.ovpn` file to your local machine:
```sh
scp pi@rpi4-ip:/home/pi/ovpns/mydevice.ovpn ~/
```
Replace `rpi4-ip` with the IP address of your RPi4.
4. **Set up SSH tunneling as a service**:
Create a systemd service to forward traffic from EC2:1194 to RPi4:1194:
```sh
sudo vim /etc/systemd/system/autossh-tunnel-1194-1194.service
```
Add the following content:
```ini
[Unit]
Description=AutoSSH tunnel from EC2:1194 to localhost:1194
After=network.target
[Service]
User=pi
ExecStartPre=/bin/bash -c 'STALE_PIDS=$(ss -tnp state CLOSE-WAIT sport = :1194 | awk "{print $6}" | cut -d, -f2 | xargs -I{} echo {} | cut -d= -f2 | sort -u); if [ -n "$STALE_PIDS" ]; then sudo kill -9 $STALE_PIDS || true; fi'
# ExecStart=/usr/bin/autossh -M 0 -N -R 1194:localhost:1194 -o "ServerAliveInterval 30" -o "ServerAliveCountMax 5" -o "ExitOnForwardFailure=yes" -o "TCPKeepAlive=yes" -o "UserKnownHostsFile=/dev/null" -o "StrictHostKeyChecking=no" -v ec2-for-tunneling
ExecStart=/usr/bin/ssh -N -R 1194:localhost:1194 -o "ServerAliveInterval 30" -o "ServerAliveCountMax 5" -o "ExitOnForwardFailure=yes" -o "TCPKeepAlive=yes" -o "UserKnownHostsFile=/dev/null" -o "StrictHostKeyChecking=no" -v ec2-for-tunneling
Restart=always
RestartSec=5
ExecStop=/usr/bin/pkill -f "ssh -N -R 1194:localhost:1194"
[Install]
WantedBy=multi-user.target
```
- Replace `ec2-for-tunneling` with the hostname defined in your `~/.ssh/config`.
- **ExecStartPre**: Cleans up stale SSH processes in `CLOSE-WAIT` state on EC2 before starting the tunnel. The `|| true` ensures the service starts even if this step fails.
- **ExecStart**: Uses `autossh` with robust retry logic.
- **Restart=always**: Ensures the service restarts after failure.
- **ExecStop**: Kills stale SSH processes when the service stops.
5. **Reload systemd and start the service**:
```sh
sudo systemctl daemon-reload
sudo systemctl restart autossh-tunnel-1194-1194
sudo systemctl enable autossh-tunnel-1194-1194
sudo systemctl status autossh-tunnel-1194-1194 --no-pager
```
6. **Ensure OpenVPN Starts After SSH Tunnel**:
To ensure the `openvpn` service starts **after** the `autossh-tunnel-1194-1194` service, create a custom OpenVPN service file with the appropriate dependencies.
1. **Create a Custom OpenVPN Service File**:
```sh
sudo vim /etc/systemd/system/openvpn-server.service
```
2. **Add the Following Content**:
```ini
[Unit]
Description=AutoSSH tunnel from EC2:1194 to localhost:1194
After=network.target
[Service]
User=pi
ExecStartPre=/bin/bash -c 'STALE_PIDS=$(ss -tnp state CLOSE-WAIT sport = :1194 | awk "{print \$6}" | cut -d, -f2 | xargs -I{} echo {} | cut -d= -f2 | sort -u); if [ -n "$STALE_PIDS" ]; then sudo kill -9 $STALE_PIDS || true; fi'
ExecStart=/usr/bin/autossh -M 0 -N -R 1194:localhost:1194 -o "ServerAliveInterval 60" -o "ServerAliveCountMax 3" -o "ExitOnForwardFailure=yes" -o "TCPKeepAlive=yes" ec2-for-tunneling
Restart=always
RestartSec=5
ExecStop=/usr/bin/pkill -f "ssh -N -R 1194:localhost:1194"
[Install]
WantedBy=multi-user.target
```
- **`After=autossh-tunnel-1194-1194.service`**: Ensures `openvpn-server.service` starts after `autossh-tunnel-1194-1194.service`.
- **`Requires=autossh-tunnel-1194-1194.service`**: Ensures `autossh-tunnel-1194-1194.service` is started before `openvpn-server.service`.
3. **Reload Systemd and Restart Services**:
Reload the systemd daemon to apply the changes:
```sh
sudo systemctl daemon-reload
```
Restart the services to ensure the dependency is respected:
```sh
sudo systemctl restart autossh-tunnel-1194-1194
sudo systemctl restart openvpn-server
```
4. **Enable the Services**:
Enable the services to start on boot:
```sh
sudo systemctl enable autossh-tunnel-1194-1194
sudo systemctl enable openvpn-server
```
5. **Disable the Original OpenVPN Service**:
Disable the original `openvpn.service` to avoid conflicts:
```sh
sudo systemctl disable openvpn
sudo systemctl stop openvpn
```
6. **Verify the Dependency**:
Check the status of the services to ensure they are running and the dependency is respected:
```sh
sudo systemctl status autossh-tunnel-1194-1194
sudo systemctl status openvpn-server
```
7. **Modify the `.ovpn` file**:
Open the `.ovpn` file you transferred earlier and update the `remote` line to use `localhost` instead of the RPi4's IP:
```=
client
dev tun
proto tcp
remote ec2-public-ip 1194
```

8. **Connect to OpenVPN**:
- Import the `.ovpn` file into your OpenVPN client.
- Start the OpenVPN connection using the imported profile.
- Once connected, you will have secure access to your RPi4 and its local network.
---
## 3. Testing the Setup
### Test SSH Access
From your PC, connect to RPi4 via EC2:
```sh
ssh -p 2222 -i <rpi4_identity> pi@ec2-public-ip
```
You can also configure `~/.ssh/config` on your PC for easier access:
```
Host rpi4-ec2-tunneling
HostName ec2-public-ip
Port 2222
User pi
PreferredAuthentications publickey
IdentityFile <rpi4_identity>
```
### Test Port Forwarding
If you have a service running on RPi4:8080, you can access it via:
```
http://ec2-public-ip:8080
```
### Test OpenVPN Access
- Ensure the OpenVPN client is connected.
- Verify access to the RPi4 and its local network.
---
## 4. Troubleshooting
### Common Issues
1. **Connection Refused**:
- Ensure the Security Group allows inbound traffic on your ports (e.g., `2222`, `8080`, `1194`, ...).
- Verify the SSH tunnel is active using `sudo systemctl status autossh-tunnel-*`.
2. **Permission Denied**:
- Ensure the SSH key (`ec2.pem`) has the correct permissions:
```sh
chmod 600 /home/pi/.ssh/ec2.pem
```
3. **GatewayPorts Not Working**:
- Double-check that `GatewayPorts yes` is set in `/etc/ssh/sshd_config` on the EC2 instance.
4. **Stale SSH Processes**:
- If the tunnel fails to reconnect, manually clean up stale SSH processes on the EC2 instance:
```sh
sudo kill -9 $(sudo lsof -t -i :2222)
sudo kill -9 $(sudo lsof -t -i :8080)
sudo kill -9 $(sudo lsof -t -i :1194)
```
5. **Network Unreachable**:
- Ensure the RPi4 has a stable network connection.
- Test the connection to the EC2 instance:
```sh
ping ec2-public-ip
```
---
## 5. Additional Notes
- Use `autossh` to ensure the tunnel remains active even after disconnections.
- Replace `ec2-public-ip` with the actual public IP of your EC2 instance.
- For better security, restrict the source IP in the Security Group instead of using `0.0.0.0/0`.
- Monitor the tunnel status using:
```sh
sudo journalctl -u autossh-tunnel-2222-22 | vim -
sudo journalctl -u autossh-tunnel-8080-8080 | vim -
sudo journalctl -u autossh-tunnel-1194-1194 | vim -
```
By following this guide, you can securely access your RPi4 and its local network from anywhere using SSH tunneling and OpenVPN via an EC2 instance. The SSH tunnels are set up as services to ensure they remain active and automatically reconnect after failures or reboots. The `ExecStartPre` commands now include `|| true` to ignore failures and ensure the services start reliably. The `openvpn-server.service` is configured to start after the `autossh-tunnel-1194-1194` service to ensure proper dependency handling, and the original `openvpn.service` is disabled to avoid conflicts.