QUAY
===

# Stand alone (新的demo lab 失敗)
## Install
[reference](https://access.redhat.com/documentation/en-us/red_hat_quay/3.10/html/proof_of_concept_-_deploying_red_hat_quay/index)

## 登入RHEL 主機
$ ssh -i ~/.ssh/id_rsa quickcluster@10.0.88.74

$ sudo yum install -y podman
$ sudo yum module install -y container-tools
$ sudo podman login registry.redhat.io
Username: rhn-support-kchang
Password: !Qxxxxxxxxxxxx


$ export QUAY=/home/quickcluster
$ mkdir -p $QUAY/postgres-quay
$ setfacl -m u:26:-wx $QUAY/postgres-quay

$ sudo podman run -d --rm --name postgresql-quay \
-e POSTGRESQL_USER=quayuser \
-e POSTGRESQL_PASSWORD=quaypass \
-e POSTGRESQL_DATABASE=quay \
-e POSTGRESQL_ADMIN_PASSWORD=adminpass \
-p 5432:5432 \
-v $QUAY/postgres-quay:/var/lib/pgsql/data:Z \
registry.redhat.io/rhel8/postgresql-13:1-109

Trying to pull registry.redhat.io/rhel8/postgresql-10:1...
Getting image source signatures
Checking if image destination supports signatures
Copying blob 5f7175b4038d done  
Copying blob bea2a0b08f4f done  
Copying blob 7822e944d15c done  
Copying config d90782b9a3 done  
Writing manifest to image destination
Storing signatures
6e4cb9b6d08a0c18099e3459d02b4bc91aba089d3d360ae5a92b6019e32e3f33


$ sudo podman exec -it postgresql-quay /bin/bash -c 'echo "CREATE EXTENSION IF NOT EXISTS pg_trgm" | psql -d quay -U postgres'
CREATE EXTENSION


$ sudo podman run -d --rm --name redis \
-p 6379:6379 \
-e REDIS_PASSWORD=strongpassword \
registry.redhat.io/rhel8/redis-6:1-110


## 我們是run在機器的80 port
$ sudo podman run --rm -it --name quay_config -p 80:8080 -p 443:8443 registry.redhat.io/quay/quay-rhel8:v3.10.3 config secret

到Resource hub上找剛剛登入的機器的FQDN

因為我們run quay 是在80 port於瀏覽器上輸入網址
username `quayconfig`
password `secret`


Database Type: Postgres
Database Server: quay-server.example.com:5432
Username: quayuser
Password: quaypass
Database Name: quay

Redis Hostname: quay-server.example.com
Redis port: 6379 (default)
Redis password: strongpassword


下載完後，在mac的terminal

$ scp -i ~/.ssh/id_rsa ~/Downloads/quay-config.tar.gz [e-mail]:~/

$ chmod -R 777 ~/quay-config.tar.gz
$ mkdir $QUAY/config
$ cp ~/quay-config.tar.gz $QUAY/config
$ cd $QUAY/config
$ tar xvf quay-config.tar.gz


$ mkdir $QUAY/storage
$ setfacl -m u:1001:-wx $QUAY/storage


$ sudo podman run -d --rm -p 80:8080 -p 443:8443  \
--name=quay \
-v $QUAY/config:/conf/stack:Z \
-v $QUAY/storage:/datastorage:Z \
registry.redhat.io/quay/quay-rhel8:v3.10.3

打開瀏覽器，點選建立帳號
username: `quayadmin`
password: `password`

回到terminal，試推Image [registry.redhat.io/redhat-openjdk-18/openjdk18-openshift:1.15-7](https://catalog.redhat.com/software/containers/redhat-openjdk-18/openjdk18-openshift/58ada5701fbe981673cd6b10?container-tabs=gti&gti-tabs=red-hat-login)

$ sudo podman login --tls-verify=false node-0.quaykfctest.lab.upshift.rdu2.redhat.com
Username: quayadmin
Password: 

$ sudo podman pull registry.redhat.io/redhat-openjdk-18/openjdk18-openshift:1.15-7

$ sudo podman tag registry.redhat.io/redhat-openjdk-18/openjdk18-openshift:1.15-7 node-0.quaykfctest.lab.upshift.rdu2.redhat.com/quayadmin/openjdk18-openshift:1.15-7

## 這可能是因為你正在嘗試推送一個已經被簽名的圖像，或者目標存儲庫有特定的限制或配置。
## 使用 --remove-signatures 選項：這個選項告訴 Podman 在推送圖像之前刪除所有現有的簽名。這樣可以避免由於更改層表示形式而使簽名無效的問題。
$ sudo podman push --remove-signatures --tls-verify=false node-0.quaykfctest.lab.upshift.rdu2.redhat.com/quayadmin/openjdk18-openshift:1.15-7

回到Quay的頁面，看repository，確實推送完成

#### Optional-檢查資料庫，判斷確實存在

$ sudo podman exec -it postgresql-quay /bin/bash
bash-4.4$ psql
psql (10.23)
Type "help" for help.

postgres-# \l
                                 List of databases
   Name    |  Owner   | Encoding |  Collate   |   Ctype    |   Access privileges   
-----------+----------+----------+------------+------------+-----------------------
 postgres  | postgres | UTF8     | en_US.utf8 | en_US.utf8 | 
 quay      | quayuser | UTF8     | en_US.utf8 | en_US.utf8 | 
 template0 | postgres | UTF8     | en_US.utf8 | en_US.utf8 | =c/postgres          +
           |          |          |            |            | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.utf8 | en_US.utf8 | =c/postgres          +
           |          |          |            |            | postgres=CTc/postgres
(4 rows)

postgres-# \c quay
You are now connected to database "quay" as user "postgres".

quay-# \dt
                   List of relations
 Schema |            Name            | Type  |  Owner   
--------+----------------------------+-------+----------
 public | accesstoken                | table | quayuser
 public | accesstokenkind            | table | quayuser
 public | alembic_version            | table | quayuser
 public | apprblob                   | table | quayuser
 ...
 ...
 (93 rows)

## Repo 確實存在
quay-# SELECT * FROM repository;
 id | namespace_user_id |        name         | visibility_id | description |             badge_token              | kind_id | trust_enabled | state 
----+-------------------+---------------------+---------------+-------------+--------------------------------------+---------+---------------+-------
  1 |                 1 | openjdk18-openshift |             2 |             | 031351ca-8945-452f-a74c-5f1f9bddf173 |       1 | f             |     0
(1 row)

quay=# SELECT * FROM user;
   user   
----------
 postgres
(1 row)

quay=#

## 停止及重新啟動

#
## 停止
#
## stop postgresql
$ sudo podman stop postgresql-quay

## stop quay
$ sudo podman stop quay
WARN[0010] StopSignal SIGTERM failed to stop container quay in 10 seconds, resorting to SIGKILL 
quay

## stop redis
$ sudo podman stop redis

## 確認所有服務皆停止
$ sudo podman ps -a

#
## 重新啟動
#
export QUAY=/home/quickcluster
## 啟動postgres
$ sudo podman run -d --rm --name postgresql-quay \
-e POSTGRESQL_USER=quayuser \
-e POSTGRESQL_PASSWORD=quaypass \
-e POSTGRESQL_DATABASE=quay \
-e POSTGRESQL_ADMIN_PASSWORD=adminpass \
-p 5432:5432 \
-v $QUAY/postgres-quay:/var/lib/pgsql/data:Z \
registry.redhat.io/rhel8/postgresql-10:1

## 啟動redis
$ sudo podman run -d --rm --name redis \
-p 6379:6379 \
-e REDIS_PASSWORD=strongpassword \
registry.redhat.io/rhel8/redis-5:1

## 啟動Quay
$ sudo podman run -d --rm -p 80:8080 -p 443:8443  \
--name=quay \
-v $QUAY/config:/conf/stack:Z \
-v $QUAY/storage:/datastorage:Z \
registry.redhat.io/quay/quay-rhel8:v3.5.7

$ sudo podman ps

## 升級

$ mkdir /tmp/quay-backup

## 因為Quay 是啟動在 $QUAY 資料夾，所以要將其備份
$ cd $QUAY/config

$ sudo tar cvf /tmp/quay-backup/quay-backup.tar.gz *

### 備份資料庫
#### GPS

$ mkdir /tmp/quay-backup/quay_upgrade
$ chmod -R 777 /tmp/quay-backup/
$ chmod -R 777 /tmp/quay-backup/quay_upgrade

$ sudo podman run --replace -d --name postgresql-quay \
  -e POSTGRESQL_USER=quayuser \
  -e POSTGRESQL_PASSWORD=quaypass \
  -e POSTGRESQL_DATABASE=quay \
  -e POSTGRESQL_ADMIN_PASSWORD=adminpass \
  -e POSTGRESQL_MAX_CONNECTIONS=1000 \
  -e POSTGRESQL_SHARED_BUFFERS=1024MB \
  -p 5432:5432 \
  -v $QUAY/postgres-quay:/var/lib/pgsql/data:Z \
  -v /tmp/quay-backup/quay_upgrade:/dumpdata:Z \
  registry.redhat.io/rhel8/postgresql-10:1
  
$ sudo podman exec -it postgresql-quay /bin/bash

## 備份Quay DB
bash-4.4$ pg_dump quay > /dumpdata/quay.pgdump

## 備份clair DB (Optional)
bash-4.4$ pg_dump clair > /dumpdata/clair.pgdump

bash-4.4$ ls -al
total 256
drwxrwxrwx. 2     1000     1000     25 Aug 19 10:37 .
dr-xr-xr-x. 1 root     root         55 Aug 19 10:10 ..
-rw-r--r--. 1 postgres postgres 261110 Aug 19 10:37 quay.pgdump

## 離開容器，確認資料落在 /tmp/quay-backup/quay_upgrade
## 且檔案大小相同
$ ls -al /tmp/quay-backup/quay_upgrade
總計 256
drwxrwxrwx. 2 quickcluster quickcluster     25  8月 19 06:37 .
drwxrwxrwx. 3 quickcluster quickcluster     52  8月 19 06:05 ..
-rw-r--r--. 1           26           26 261110  8月 19 06:37 quay.pgdump

## 啟動新的postgres
$ sudo podman rm -f postgresql-quay

$ sudo podman run --replace -d --name postgresql-quay \
  -e POSTGRESQL_USER=quayuser \
  -e POSTGRESQL_PASSWORD=quaypass \
  -e POSTGRESQL_DATABASE=quay \
  -e POSTGRESQL_ADMIN_PASSWORD=adminpass \
  -e POSTGRESQL_MAX_CONNECTIONS=1000 \
  -e POSTGRESQL_SHARED_BUFFERS=1024MB \
  -p 5432:5432 \
  -v $QUAY/postgres-quay13:/var/lib/pgsql/data:Z \
  -v /tmp/quay-backup/quay_upgrade:/dumpdata:Z \
  registry.redhat.io/rhel8/postgresql-13:1-109

:::danger
出現如下error

Incompatible data directory.  This container image provides
PostgreSQL '13', but data directory is of
version '10'.

This image supports automatic data directory upgrade from
'12', please _carefully_ consult image documentation
about how to use the '$POSTGRESQL_UPGRADE' startup option.

只好run 新的資料夾

$ mkdir -p $QUAY/postgres-quay13
$ setfacl -m u:26:-wx $QUAY/postgres-quay13
$ sudo podman run --replace -d --name postgresql-quay \
   -e POSTGRESQL_USER=quayuser \
   -e POSTGRESQL_PASSWORD=quaypass \
   -e POSTGRESQL_DATABASE=quay \
   -e POSTGRESQL_ADMIN_PASSWORD=adminpass \
   -e POSTGRESQL_MAX_CONNECTIONS=1000 \
   -e POSTGRESQL_SHARED_BUFFERS=1024MB \
   -p 5432:5432 \
   -v $QUAY/postgres-quay13:/var/lib/pgsql/data:Z \
   -v /tmp/quay-backup/quay_upgrade:/dumpdata:Z \
   registry.redhat.io/rhel8/postgresql-13:1-109

:::

## 官方設定指令
## sudo podman exec -it postgresql-quay /bin/bash -c 'echo "CREATE EXTENSION IF NOT EXISTS pg_trgm" | psql -d quay -U postgres'
# CREATE EXTENSION

## GPS 設定指令
$ sudo podman exec -it postgresql-quay /bin/bash -c 'echo "SELECT * FROM pg_available_extensions" | /usr/bin/psql'

$ sudo podman exec -it postgresql-quay /bin/bash -c 'echo "CREATE EXTENSION IF NOT EXISTS pg_trgm;" | /usr/bin/psql -d quay'

#### 官方
Container 還沒停

## 預設沒有jq，須先安裝
$ sudo dnf install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm -y

$ sudo dnf install jq -y

## 尋找啟動命令
$ sudo podman inspect quay | jq -r '.[0].Config.CreateCommand | .[]' | paste -s -d ' ' -
podman run -d --rm -p 80:8080 -p 443:8443 --name=quay -v /home/quickcluster/config:/conf/stack:Z -v /home/quickcluster/storage:/datastorage:Z registry.redhat.io/quay/quay-rhel8:v3.5.7

## 備份config.yaml
$ sudo podman exec -it quay cat /conf/stack/config.yaml > /tmp/quay-backup/quay-config.yaml

$ grep DB_URI /tmp/quay-backup/quay-config.yaml
DB_URI: postgresql://quayuser:[e-mail]:5432/quay

### 還原資料庫到新版

#### GPS

$ sudo podman exec -it postgresql-quay /bin/bash
$ psql -f /dumpdata/quay.pgdump

執行完後 離開容器，並回到[重新啟動](#停止及重新啟動)的章節
執行新版的Quay

$ sudo podman run -d --rm -p 80:8080 -p 443:8443  \
--name=quay \
-v $QUAY/config:/conf/stack:Z \
-v $QUAY/storage:/datastorage:Z \
registry.redhat.io/quay/quay-rhel8:v3.7.0

#### 官方

$

$ 

### 修改

$ mkdir /tmp/quay-backup/quay_upgrade
$ chmod -R 777 /tmp/quay-backup/
$ chmod -R 777 /tmp/quay-backup/quay_upgrade

## 執行Container 
$ sudo podman run --replace -d --name postgresql-quay \
  -e POSTGRESQL_USER=quayuser \
  -e POSTGRESQL_PASSWORD=quaypass \
  -e POSTGRESQL_DATABASE=quay \
  -e POSTGRESQL_ADMIN_PASSWORD=adminpass \
  -e POSTGRESQL_MAX_CONNECTIONS=1000 \
  -e POSTGRESQL_SHARED_BUFFERS=1024MB \
  -p 5432:5432 \
  -v $QUAY/postgres-quay:/var/lib/pgsql/data:Z \
  -v /tmp/quay-backup/quay_upgrade:/dumpdata:Z \
  registry.redhat.io/rhel8/postgresql-10:1

## 備份資料庫
bash-4.4$ pg_dump -d quay -U quayuser -W -O > /dumpdata/quay-backup.sql



## 還原資料庫
$ mkdir -p $QUAY/postgres-quay13
$ setfacl -m u:26:-wx $QUAY/postgres-quay13
$ sudo podman run --replace -d --name postgresql-quay \
   -e POSTGRESQL_USER=quayuser \
   -e POSTGRESQL_PASSWORD=quaypass \
   -e POSTGRESQL_DATABASE=quay \
   -e POSTGRESQL_ADMIN_PASSWORD=adminpass \
   -e POSTGRESQL_MAX_CONNECTIONS=1000 \
   -e POSTGRESQL_SHARED_BUFFERS=1024MB \
   -p 5432:5432 \
   -v $QUAY/postgres-quay13:/var/lib/pgsql/data:Z \
   -v /tmp/quay-backup/quay_upgrade:/dumpdata:Z \
   registry.redhat.io/rhel8/postgresql-13:1-109

$ sudo podman exec -it postgresql-quay /bin/bash -c 'echo "CREATE EXTENSION IF NOT EXISTS pg_trgm" | psql -d quay -U postgres'

$ sudo podman exec -it postgresql-quay /bin/bash
bash-4.4$ psql "dbname=quay user=quayuser password=quaypass"  -W <  /dumpdata/quay-backup.sql


## (Optional) 驗證資料是否存在
bash-4.4$ psql

[檢查資料庫，判斷確實存在](#Optional-檢查資料庫，判斷確實存在)

# OCP 內部
## 安裝quay operator

## 安裝[miniO](https://hackmd.io/Eo5jB--7RKia6wDW5VdSZw?both#Install-minio)

新增bucket

新增user
Name: `quay`
设置新用户的名称和密码为 quay/openshift4!，并赋予用户 readwrite 权限

## 新增Quay registry

$ oc new-project quay-enterprise
$ cat config.yaml

DISTRIBUTED_STORAGE_CONFIG: 
  default: 
    - RadosGWStorage 
    - access_key: quay
      secret_key: openshift4!
      bucket_name: quay
      hostname: minio-svc.minio-dev.svc
      is_secure: false
      port: 9000
      storage_path: /
DISTRIBUTED_STORAGE_DEFAULT_LOCATIONS: [] 
DISTRIBUTED_STORAGE_PREFERENCE: 

$ oc create secret generic config-bundle-secret -n quay-enterprise --from-file config.yaml=config.yaml
secret/config-bundle-secret created

$ cat quay-reg.yaml

apiVersion: quay.redhat.com/v1
kind: QuayRegistry
metadata:
  name: image-registry
  namespace: quay-enterprise
spec:
  configBundleSecret: config-bundle-secret
  components:
  - kind: objectstorage
    managed: false

$ oc apply -f quay-reg.yaml -n quay-enterprise

# RH demohub