**Usado**
3 vms ubuntu 22.04 onde foi instalado um gitaly em cada
1 praefect feito deployment no k8s
1 postgres

Arquitetura complet

**Instalação do GITALY**

Repetir o procedimento nos dois nodes gitaly

sudo apt-get update -y &&
sudo apt-get upgrade -y &&
sudo apt-get install sudo -y

**Build dependencies**

Install the required packages (needed to compile Ruby and native extensions to Ruby gems):

sudo apt-get install -y build-essential zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libre2-dev \
libreadline-dev libncurses5-dev libffi-dev curl openssh-server libxml2-dev libxslt-dev \
libcurl4-openssl-dev libicu-dev libkrb5-dev logrotate rsync python3-docutils pkg-config cmake \
runit-systemd

**Usar o git na versão 2.42.x que vem no repositório do gitaly**

sudo apt-get install -y libcurl4-openssl-dev libexpat1-dev gettext libz-dev libssl-dev libpcre2-dev build-essential git-core

Clone repositório do gitaly e build do git

git clone https://gitlab.com/gitlab-org/gitaly.git -b v16.1.1
cd gitaly
sudo make git GIT_PREFIX=/usr/local

Instalação do GO fora do diretório do gitaly

curl --remote-name --location --progress-bar "https://go.dev/dl/go1.20.8.linux-amd64.tar.gz"
echo 'cc97c28d9c252fbf28f91950d830201aa403836cbed702a05932e63f7f0c7bc4 go1.20.8.linux-amd64.tar.gz' | shasum -a256 -c - && \
sudo tar -C /usr/local -xzf go1.20.8.linux-amd64.tar.gz
sudo ln -sf /usr/local/go/bin/{go,gofmt} /usr/local/bin/
rm go1.20.8.linux-amd64.tar.gz

teste o go com:

/usr/local/bin/go version

dentro do repositório gitaly execute o comando

sudo make

criar os dirs e file para gitlab-shell-secret após montar um disco em /mnt/gitaly

atentar para o user para o systemd adiante ter acesso
sudo mkdir /mnt/gitaly &&
sudo mkdir /mnt/gitaly/repositories &&
sudo mkdir /etc/gitaly/ &&
mkdir /etc/gitaly/gitlab-shell

sudo touch /etc/gitaly/.gitlab_shell_secret

sudo chown marques:marques -R /mnt/gitaly/ &&
sudo chown marques:marques -R /etc/gitaly/

A seguir

cp config.toml.example /etc/gitaly/config.toml

alterado as seguintes sessões:

#mapeado para esse path, o default seria para /home/git...
#socket_path = "/home/marques/gitlab/tmp/sockets/private/gitaly.socket"

#mapeado para o local do binário gitaly
bin_dir = "/usr/local/bin"

#liberado para escutar nesta porta
listen_addr = "0.0.0.0:8075"

# # Optional: export metrics via Prometheus
#a porta default é 9236
prometheus_listen_addr = "0.0.0.0:9236"

[auth]
token = 'abc123secret'
transitioning = false

#mapeado para este path e foi criado esse dir, atentar para o user que pode acessa-lo
[[storage]]
name = "gitaly-0"
path = "/mnt/gitaly/repositories"

#criado esse dir e mapeado o path, caso contrário ele não starta
[gitlab-shell]
# The directory where gitlab-shell is installed
dir = "/etc/gitaly/gitlab-shell"

#criado esse dir e arquivo oculto e mapeado o path, caso contrário ele não starta
[gitlab]
secret_file = "/etc/gitaly/.gitlab_shell_secret"

SystemD do gitaly

nano gitaly.service

copiei o seguinte texto:

[Unit]
Description=Gitaly Git RPC service
After=network.target

[Service]
ExecStart= /usr/local/bin/gitaly serve /etc/gitaly/config.toml
Restart=always
User=user
Group=user

[Install]
WantedBy=multi-user.target

mover o arquivo para systemd

sudo mv gitaly.service /etc/systemd/system

mover o binário do gitaly para pasta de bin

sudo mv _build/bin/gitaly /usr/local/bin/

Inicilizar via systemd

sudo systemctl daemon-reload
sudo systemctl enable gitaly.service
sudo systemctl start gitaly.service
sudo systemctl status gitaly.service

Para depurar erros:

Use journalctl -xe | grep gitaly

Para confirmar portas em uso pelo gitaly:

apt install net-tools -y
journalctl -xe | grep gitaly

**Deployment do praefect no k8s**
<section>
<p>Deployment: <p>

apiVersion: apps/v1
kind: Deployment
metadata:
name: praefect-deployment
spec:
replicas: 1
selector:
matchLabels:
app: praefect
template:
metadata:
labels:
app: praefect
spec:
containers:
- name: praefect
image: registry.gitlab.com/gitlab-org/build/cng/gitaly:16-2-stable
volumeMounts:
- name: praefect-conf
mountPath: /etc/gitaly/config.toml
subPath: config.toml
command: ["/usr/local/bin/praefect"]
args: ["-config", "/etc/gitaly/config.toml"]
env:
- name: TZ
value: "America/Recife"
volumes:
- name: praefect-conf
configMap:
name: praefect-config
---
apiVersion: v1
kind: Service
metadata:
name: praefect-service
spec:
selector:
app: praefect
type: LoadBalancer
ports:
- protocol: TCP
port: 80
targetPort: 8076

<p>ConfigMap: <p>

apiVersion: v1
kind: ConfigMap
metadata:
name: praefect-config
data:
config.toml: |
listen_addr = '0.0.0.0:8076'

# Optional: export metrics via Prometheus
prometheus_listen_addr = '0.0.0.0:5513'

# Optional: when set to true, metrics that query the database will no longer be
# available on the /metrics endpoint but will be available on the /db_metrics endpoint
prometheus_exclude_database_from_default_metrics = true

[failover]
enabled = true

[auth]
token = 'secret'
transitioning = false

[gitaly]
enable = false

[logging]
level = 'info'
format = 'json'

##config feita para se conectar a dois gitaly, atentar para ip e porta deles junto com os respectivos secrets. O nome dos Storages também são importantes
[[virtual_storage]]
name = 'default'
[[virtual_storage.node]]
storage = 'gitaly-0'
address = 'tcp://ip:porta'
token = 'secret'
[[virtual_storage.node]]
storage = 'gitaly-1'
address = 'tcp://ip:porta'
token = 'secret'

[database]
host = 'ip'
port = porta
user = 'user'
password = 'senha'
dbname = 'name'

# [gitlab]
# url = 'https.nome.cloud'
# relative_url_root = ''
<section>

<section>
<p>
Após criar o arquivo de deployment e de configmap
</p>

kubectl apply -f diretório/dos/arquivos/
kubectl get pods
kubectl exec -it nome-do-pod /bin/bash

dentro do pod:
verificar as opções
/usr/local/bin/praefect -config /etc/gitaly/config.toml --help

criar as estrutura no postgres
/usr/local/bin/praefect -config /etc/gitaly/config.toml sql-migrate

Depois confirmar as conexões
/usr/local/bin/praefect -config /etc/gitaly/config.toml dial-nodes #reusltado na imagem 1

Fora do pod
kubect logs nome-do-pod #resultado na imagem 2

imagem 1:

imagem 2:

<section>
Refências:

install gitaly self-compiled: https://docs.gitlab.com/ee/install/installation.html

link config gitaly with praefect: https://www.balticlsc.eu/gitlab/help/administration/gitaly/praefect.md#gitaly

link gitlab charts completo: https://gitlab.com/gitlab-org/charts/gitlab

link arquivos yamls e containers https://gitlab.com/gitlab-org/gitaly

link explicação https://docs.gitlab.com/ee/administration/gitaly/

link rpc https://deinfo.uepg.br/~alunoso/2017/RPC/diferencas.html

link armazenamento: https://docs.gitlab.com/ee/administration/repository_storage_paths.html

link config postgresql https://docs.gitlab.com/charts/charts/globals#configure-postgresql-settings

link para external database https://docs.gitlab.com/charts/advanced/external-db/index.html

caso precise instalar o gitaly cluster: https://www.balticlsc.eu/gitlab/help/administration/gitaly/praefect.md
</section>