# Install Manually
[TOC]
This section describes how to install HAProxy and the DataPlane API manually.
#### Images
##### Ubuntu Images
Source: <https://cloud-images.ubuntu.com/releases/>
###### 18.04 Bionic
ova: <https://cloud-images.ubuntu.com/bionic/current/bionic-server-cloudimg-amd64.ova>
###### 19.10 Eoan
ova: <https://cloud-images.ubuntu.com/releases/eoan/release-20200407/ubuntu-19.10-server-cloudimg-amd64.ova>
##### Debian Images
###### Debian 10.3
Source: <https://www.debian.org/distrib/netinst>
iso: <https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-10.3.0-amd64-netinst.iso>
##### Photon Images
###### Photon 3.0
Source: <https://github.com/vmware/photon/wiki/Downloading-Photon-OS>
ova: <http://dl.bintray.com/vmware/photon/3.0/Rev2/ova/photon-hw11-3.0-9355405.ova>
##### CentOS Images
Source: <https://wiki.centos.org/Download>
###### CentOS 7
iso: <http://mirror.fileplanet.com/centos/7.7.1908/isos/x86_64/CentOS-7-x86_64-Minimal-1908.iso>
###### CentOS 8
iso: <http://mirror.sjc02.svwh.net/centos/8.1.1911/isos/x86_64/CentOS-8.1.1911-x86_64-boot.iso>
#### Notes
1. HAProxy >= 1.9 is required for dataplane api
1. Except Photon 3, other distros default repositories don't have the latest HAProxy(2.1)
1. Ubuntu 19.10 contains 2.0.5
1. dataplaneapi needs to be built manually
1. Photon will soon include an RPM for dataplane api binary [TKG-1372](https://jira.eng.vmware.com/browse/TKG-1372)
#### Default repository
##### Ubuntu
###### For Ubuntu 18.04 Bionic
Install from default repositories
```bash
apt install -y haproxy
```
However, the default repositories usually don't contain the latest releases. To check the current haproxy version in default repository:
```bash
ubuntu@u3:~$ apt show haproxy 2>/dev/null | grep Version
Version: 1.8.8-1ubuntu0.10
```
###### For Ubuntu 19.10 Eoan
Install from default repositories
```bash
apt install -y haproxy
```
However, the default repositories usually don't contain the latest releases. To check the current haproxy version in default repository:
```bash
# apt show haproxy 2>/dev/null | grep Version
Version: 2.0.5-1ubuntu0.3
```
##### For Debian 10.3
Install from default repositories
```bash
apt install -y haproxy
```
However, the default repositories usually don't contain the latest releases. To check the current haproxy version in default repository:
```bash
# apt show haproxy 2>/dev/null | grep Version
Version: 1.8.19-1+deb10u2
```
##### For Photon 3.0
Install from default repositories
```bash
tdnf install haproxy-2.1.0
```
##### For CentOS
###### For CentOS 7
Install from default repositories
```bash
yum install -y haproxy.x86_64
```
However, the default repositories usually don't contain the latest releases. To check the current haproxy version in default repository:
```bash
# yum info haproxy | grep Version
Version : 1.5.18
```
###### For CentOS 8
Install from default repositories
```bash
yum install -y haproxy.x86_64
```
However, the default repositories usually don't contain the latest releases. To check the current haproxy version in default repository:
```bash
# yum info haproxy | grep Version
Version : 1.8.15
```
#### Build from Source
##### Preparation
* Ubuntu
```bash
apt update
apt install -y make gcc perl libpcre3-dev zlib1g-dev libssl-dev libsystemd-dev
# Optional. will be used in validation section
apt install -y jq docker.io
```
* Photon
```bash
tdnf update
# diffutils is for cmp command: /bin/sh: cmp: command not found
# glibc-devel provides the unix headers: unistd.h: No such file or directory
# binutils provides the as assembler, which is required by zlib
tdnf install -y tar git make gcc perl pcre-devel zlib-devel openssl-devel systemd-devel diffutils glibc-devel binutils
# Optional. will be used in validation section
tdnf install -y jq docker
systemctl start docker
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
```
* CentOS
```bash
yum update
yum install -y make git gcc perl pcre-devel.x86_64 zlib-devel.x86_64 openssl-devel.x86_64 systemd-devel.x86_64
# Optional. will be used in validation section
# ref: https://medium.com/@gchandra/install-jq-on-centos-7-459dd650baa3
yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm && yum install -y jq
systemctl start docker
```
##### Build
```bash
HAPROXY_VEERSION="2.1"
HAPROXY_TARBALL="haproxy-2.1.4"
curl -s -L http://www.haproxy.org/download/"${HAPROXY_VEERSION}"/src/"${HAPROXY_TARBALL}".tar.gz | tar xzv
pushd "${HAPROXY_TARBALL}"
make TARGET=linux-glibc USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1 USE_SYSTEMD=1 EXTRA_OBJS="contrib/prometheus-exporter/service-prometheus.o"
make install
cp haproxy /usr/sbin/
popd
groupadd haproxy
useradd -s /usr/sbin/nologin --system -g haproxy --home-dir=/var/lib/haproxy haproxy
mkdir /var/lib/haproxy
chown -R haproxy:haproxy /var/lib/haproxy
curl -s https://raw.githubusercontent.com/horms/haproxy/master/doc/configuration.txt | gzip -c > configuration.txt.gz
mkdir -p /usr/share/doc/haproxy
mv configuration.txt.gz /usr/share/doc/haproxy/configuration.txt.gz
```
#### Configuration
##### systemd
```bash
cat > /lib/systemd/system/haproxy.service <<"EOF"
[Unit]
Description=HAProxy Load Balancer
Documentation=man:haproxy(1)
Documentation=file:/usr/share/doc/haproxy/configuration.txt.gz
# allows us to do millisecond level restarts without triggering alert in Systemd
StartLimitInterval=0
StartLimitBurst=0
After=network.target syslog.service
Wants=syslog.service
[Service]
EnvironmentFile=-/etc/default/haproxy
EnvironmentFile=-/etc/sysconfig/haproxy
Environment="CONFIG=/etc/haproxy/haproxy.cfg" "PIDFILE=/var/run/haproxy.pid" "EXTRAOPTS=-S /var/run/haproxy-master.sock"
ExecStartPre=/usr/sbin/haproxy -f $CONFIG -c -q $EXTRAOPTS
ExecStart=/usr/sbin/haproxy -Ws -f $CONFIG -p $PIDFILE $EXTRAOPTS
ExecReload=/usr/sbin/haproxy -f $CONFIG -c -q $EXTRAOPTS
ExecReload=/bin/kill -USR2 $MAINPID
KillMode=mixed
Restart=always
SuccessExitStatus=143
Type=notify
[Install]
WantedBy=multi-user.target
EOF
```
##### haproxy.cfg
```bash
mkdir -p /etc/haproxy
cat > /etc/haproxy/haproxy.cfg <<"EOF"
global
master-worker
log stdout format raw local0 info
stats socket /run/haproxy.sock user haproxy group haproxy mode 660 level admin expose-fd listeners
stats timeout 30s
chroot /var/lib/haproxy
user haproxy
group haproxy
# Default SSL material locations
ca-base /etc/ssl/certs
crt-base /etc/ssl/private
# Default ciphers to use on SSL-enabled listening sockets.
# For more information, see ciphers(1SSL). This list is from:
# https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
# An alternative list with additional directives can be obtained from
# https://mozilla.github.io/server-side-tls/ssl-config-generator/?server=haproxy
ssl-default-bind-options no-sslv3
ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS
defaults
mode tcp
maxconn 4000
log global
option tcplog
option dontlognull
option tcp-smart-accept
timeout check 5s
timeout connect 9s
timeout client 10s
timeout queue 5m
timeout server 10s
timeout tunnel 1h
timeout client-fin 10s
userlist controller
user client insecure-password cert
EOF
systemctl enable haproxy.service
systemctl start haproxy.service
```
#### Dataplane
##### Build Dataplane
###### Build Dataplane from Source
```bash
OS=linux
ARCH=amd64
GOVERSION="1.14.2"
curl -L https://dl.google.com/go/"go${GOVERSION}.${OS}-${ARCH}".tar.gz | tar xzv -C /usr/local/
export GOROOT=/usr/local/go/
export GOPATH="$HOME/go"
export PATH="$GOPATH/bin:$GOROOT/bin:$PATH"
mkdir -p "$GOPATH"
DATAPLANEAPI_REF=0553265
git clone https://github.com/haproxytech/dataplaneapi.git
pushd dataplaneapi
git checkout -b build-me ${DATAPLANEAPI_REF}
make build
mv build/dataplaneapi /usr/local/bin/
popd
rm -rf dataplaneapi
```
###### Build using Docker
```bash
docker run -it --rm -v $(pwd):/out golang bash -c \
"git clone https://github.com/haproxytech/dataplaneapi.git && \
cd dataplaneapi && \
git checkout -b 0553265 0553265 && \
make && \
mv build/dataplaneapi /out"
mv dataplaneapi /usr/local/bin/
```
##### Self Signed SSL Configuration
```bash
# If you have go installed
go get -u github.com/cloudflare/cfssl/cmd/cfssl
go get -u github.com/cloudflare/cfssl/cmd/cfssljson
# Otherwise
curl -sL https://github.com/cloudflare/cfssl/releases/download/v1.4.1/cfssl_1.4.1_linux_amd64 -o /usr/local/bin/cfssl
curl -sL https://github.com/cloudflare/cfssl/releases/download/v1.4.1/cfssljson_1.4.1_linux_amd64 -o /usr/local/bin/cfssljson
chmod +x /usr/local/bin/cfssl*
cat > ca.json <<EOF
{
"CN": "HAproxy Root CA",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "US",
"ST": "California",
"L": "Palo Alto",
"O": "VMware",
"OU": "Load Balancer API"
}
]
}
EOF
cat > cfssl.json <<EOF
{
"signing": {
"default": {
"expiry": "8760h"
},
"profiles": {
"server": {
"usages": [
"signing",
"digital signing",
"key encipherment",
"server auth"
],
"expiry": "8760h"
},
"client": {
"usages": [
"signing",
"digital signature",
"key encipherment",
"client auth"
],
"expiry": "8760h"
}
}
}
}
EOF
cat > haproxy.json <<EOF
{
"CN": "dataplaneapi",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "US",
"ST": "California",
"L": "Palo Alto",
"O": "VMware",
"OU": "Load Balancer API"
}
],
"hosts": [
"haproxy.lbapi.run.tanzu.vmware.com",
"localhost"
]
}
EOF
cfssl gencert -initca ca.json | cfssljson -bare ca
cfssl gencert -ca ca.pem -ca-key ca-key.pem -config cfssl.json -profile=server haproxy.json | cfssljson -bare server
cfssl gencert -ca ca.pem -ca-key ca-key.pem -config cfssl.json -profile=client haproxy.json | cfssljson -bare client
cp server.pem /etc/haproxy/server.crt
cp server-key.pem /etc/haproxy/server.key
cp ca.pem /etc/haproxy/ca.crt
cat >> /etc/haproxy/haproxy.cfg <<EOF
program api
command dataplaneapi --scheme=https --haproxy-bin=/usr/sbin/haproxy --config-file=/etc/haproxy/haproxy.cfg --reload-cmd="systemctl reload haproxy" --reload-delay=5 --tls-host=0.0.0.0 --tls-port=5556 --tls-ca=/etc/haproxy/ca.crt --tls-certificate=/etc/haproxy/server.crt --tls-key=/etc/haproxy/server.key --userlist=controller
no option start-on-reload
EOF
systemctl reload haproxy
```
#### Validation
##### Make sure HAProxy process is up and running
```bash
# systemctl status haproxy
● haproxy.service - HAProxy Load Balancer
Loaded: loaded (/lib/systemd/system/haproxy.service; disabled; vendor preset: enabled)
Active: active (running) since Tue 2020-04-21 22:37:46 UTC; 9min ago
Docs: man:haproxy(1)
file:/usr/share/doc/haproxy/configuration.txt.gz
Main PID: 4879 (haproxy)
Tasks: 11 (limit: 1152)
CGroup: /system.slice/haproxy.service
├─4879 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid -S /var/run/haproxy-master.sock
├─4881 dataplaneapi --scheme=https --haproxy-bin=/usr/sbin/haproxy --config-file=/etc/haproxy/haproxy.cfg --reload-cmd=/usr/bin/systemctl
└─4882 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid -S /var/run/haproxy-master.sock
Apr 21 22:37:46 u3 systemd[1]: Starting HAProxy Load Balancer...
Apr 21 22:37:46 u3 haproxy[4879]: [NOTICE] 111/223746 (4879) : New program 'api' (4881) forked
Apr 21 22:37:46 u3 haproxy[4879]: [NOTICE] 111/223746 (4879) : New worker #1 (4882) forked
Apr 21 22:37:46 u3 systemd[1]: Started HAProxy Load Balancer.
Apr 21 22:37:47 u3 haproxy[4879]: time="2020-04-21T22:37:47Z" level=info msg="HAProxy Data Plane API v1.2.4 0553265.dev"
Apr 21 22:37:47 u3 haproxy[4879]: time="2020-04-21T22:37:47Z" level=info msg="Build from: https://github.com/haproxytech/dataplaneapi.git"
Apr 21 22:37:47 u3 haproxy[4879]: time="2020-04-21T22:37:47Z" level=info msg="Build date: 2020-04-21T21:55:54"
```
##### Make sure the HAproxy Dataplane is up and running
```bash
# curl -s -u 'client:cert' --cacert ./ca.pem --key ./client-key.pem --cert ./client.pem https://localhost:5556/v2/services/haproxy | jq .
[
{
"description": "Returns an array of all configured sites.",
"title": "Return an array of sites",
"url": "/services/haproxy/sites"
},
{
"description": "Returns a list of HAProxy configuration transactions. Transactions can be filtered by their status.",
"title": "Return list of HAProxy configuration transactions.",
"url": "/services/haproxy/transactions"
},
{
"description": "Returns a list of endpoints to be used for advanced configuration of HAProxy objects.",
"title": "Return list of HAProxy advanced configuration endpoints",
"url": "/services/haproxy/configuration"
},
{
"description": "Returns a list of endpoints to be used for advanced runtime settings of HAProxy objects.",
"title": "Return list of HAProxy advanced runtime endpoints",
"url": "/services/haproxy/runtime"
},
{
"description": "Returns a list of HAProxy reloads.",
"title": "Return list of HAProxy Reloads.",
"url": "/services/haproxy/reloads"
},
{
"description": "Returns a list of HAProxy stats endpoints.",
"title": "Return list of HAProxy stats endpoints",
"url": "/services/haproxy/stats"
}
]
```
##### Make sure HAProxy is functioning
```bash
# Set AnyIP for local table
ip r add local 192.168.35.0/24 dev lo
# Deploy backends
docker run -d --rm --name "haproxy-backend-0" hashicorp/http-echo -listen ":12221" -text "This is backend-0 on port 12221"
docker run -d --rm --name "haproxy-backend-1" hashicorp/http-echo -listen ":12221" -text "This is backend-1 on port 12221"
# docker inspect haproxy-backend-0 | jq -cr '.[0].NetworkSettings.Networks.bridge.IPAddress'
172.17.0.2
# docker inspect haproxy-backend-1 | jq -cr '.[0].NetworkSettings.Networks.bridge.IPAddress'
172.17.0.3
# Update HAProxy config with frontend and backend
# Frontend will be listening on one IP from the Floating IP Range
cat >> /etc/haproxy/haproxy.cfg <<EOF
frontend echo
mode tcp
bind 192.168.35.23:12221 name lb
option tcplog
default_backend echo
backend echo
mode tcp
balance first
default-server inter 10s downinter 10s rise 5 fall 3 slowstart 120s maxconn 1000 maxqueue 256 weight 100
server s0 172.17.0.2:12221 check verify none
server s1 172.17.0.3:12221 check verify none
http-check expect status 200
EOF
systemctl reload haproxy
# Make sure the Floating IP is being served
root@u3:/home/ubuntu# curl 192.168.35.23:12221
This is backend-0 on port 12221
# Optional: If you havee another machine in the same subnet
root@u5:/home/ubuntu# ip r add 192.168.35.23/32 via [IP Address of the machine where HAProxy is deployed]
root@u5:/home/ubuntu# curl 192.168.35.23:12221
This is backend-0 on port 12221
```
##### Make sure HAProxy works with Prometheus
###### Add prometheus in frontend
```bash
cat > /etc/haproxy/haproxy.cfg <<EOF
frontend stats
mode http
bind *:8404
option http-use-htx
http-request use-service prometheus-exporter if { path /metrics }
stats enable
stats uri /stats
stats refresh 10s
EOF
systemctl reload haproxy
```
###### Deploy Prometheus
```bash
cat > prometheus.yml <<EOF
global:
scrape_interval: 15s # By default, scrape targets every 15 seconds.
scrape_configs:
- job_name: 'haproxy'
static_configs:
- targets: ['localhost:8404']
EOF
docker run --network host -p 9090:9090 -d -v $PWD/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus
```
* Go to [ip]:9090 where IP belongs to the machine HAProxy and Prometheus are deployed.
* Create a Graph with an expression like "haproxy_frontend_http_requests_total". You should be able to see something like the following
#### References
* Installation: <https://dev.to/tmidi/getting-started-with-haproxy-install-from-code-source-5c4g>
* includes make Options
* haproxy user/group setup: <https://github.com/docker-library/haproxy/issues/6>
* haproxy configuration documentation: <http://cbonte.github.io/haproxy-dconv/2.1/configuration.html>
* cfssl key generation: <https://medium.com/@rob.blackbourn/how-to-use-cfssl-to-create-self-signed-certificates-d55f76ba5781>
* haproxy with prometheus: <https://www.haproxy.com/blog/haproxy-exposes-a-prometheus-metrics-endpoint/>
Sign in with Wallet
Connect another wallet