Notes on onecontainer use

Overview

• Blog-post references old versions (f31, pulp-3.4, etc)
• Should mention "assumes you have docker/podman installed for your distribution"
• F33 fails to get container running
• Podman shows multiple registries, remind again that we're only in dockerhub
• Trying admin-pwd/status-check "too soon" can fail with scary errors
  • wait ~60s post-spinup to avoid
• Starting a second container with these instructions kills the current one due to shared volumes (maybe need an advanced-topics page?)
  • see ggainey/pulp_startup/onecontainer/start_all for getting around that

Links

• https://pulpproject.org/pulp-in-one-container/
• https://github.com/ggainey/pulp_startup/tree/main/onecontainer

Notes/how-tos

Windows10

install wsl
install docker
in wsl
sudo apt install pip python3-virtualenv fuse
https://gist.github.com/fedme/442e7d1d7eb7d68e02cfbf6441d42759
vi .bashrc
export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3
export WORKON_HOME=~/.venvs
source ~/.local/bin/virtualenvwrapper.sh
export PIP_VIRTUALENV_BASE=~/.venvs
source .bashrc

mkdir settings pulp_storage pgsql containers
$ echo "CONTENT_ORIGIN='http://hostname:8080'
ANSIBLE_API_HOSTNAME='http://hostname:8080'
ANSIBLE_CONTENT_HOSTNAME='http://hostname:8080/pulp/content'
TOKEN_AUTH_DISABLED=True" >> settings/settings.py

https://www.redhat.com/sysadmin/podman-windows-wsl2
https://docs.microsoft.com/en-us/windows/wsl/install
sudo vi
sudo podman

Pavel's Win10 approach

install WSL and Docker for Windows

• make sure that "Use of WSL" at docker installation time is ticked
• after installation docker ask you for one more package with a link (it is directly m$ site) so install and restart docker
• create init.bat file in project folder (my was C:\Pavel\Projects\Pulp)
  ​​​​mkdir settings pulp_storage pgsql containers
  
  ​​​​(
  ​​​​  echo CONTENT_ORIGIN='http://%COMPUTERNAME%:8080'
  ​​​​  echo ANSIBLE_API_HOSTNAME='http://%COMPUTERNAME%:8080'
  ​​​​  echo ANSIBLE_CONTENT_HOSTNAME='http://%COMPUTERNAME%:8080/pulp/content'
  ​​​​  echo TOKEN_AUTH_DISABLED=True
  ​​​​) > settings\settings.py
  
  ​​​​docker run ^
  ​​​​  --detach ^
  ​​​​  --publish 8080:80 ^
  ​​​​  --name pulp ^
  ​​​​  --volume "%cd%/settings":/etc/pulp ^
  ​​​​  --volume "%cd%/pulp_storage":/var/lib/pulp ^
  ​​​​  --volume "%cd%/pgsql":/var/lib/pgsql ^
  ​​​​  --volume "%cd%/containers":/var/lib/containers ^
  ​​​​  --device /dev/fuse ^
  ​​​​  pulp/pulp
  
  ​​​​REM We need to wait here for pure WSL performance. On my machine with i7 20 secs was ok. 
  ​​​​timeout 60
  
  ​​​​docker exec -it pulp bash -c "pulpcore-manager reset-admin-password"
  

• and run it
• as mentioned above after docker run wait about 60 secs (timeout in script)
• after new password is set up pulp is available at http://localhost:8080/pulp/api/v3/status/
• Personally I used VSCode with "Remote - Containers" addon to attach to Pulp - what means remote devel/debug/terminal.

Notes about pulp-cli

Will open issues for these "soon"

• Would be nice if pulp config create took --profile option and created a new stanza (edit exists, this would be a nice-to-have)
• profiles in .toml are "cli-<profilename". This isn't really very clear, and very confusing if you don't realize what's going on.
• .toml keys have rules - primary being ". means more than you think it does". Results when you break this rule are…non-obvious.

Notes not related to improving-one-container, just so I don't forget them

Getting CLI up-for-dev on a "naked" box (starting from "fedora34" vagrant VM from pulp_installer)

cd
sudo dnf install git
git clone https://github.com/pulp/pulp-cli.git
sudo dnf install python3-virtualenv
pip install virtualenvwrapper
source .local/bin/virtualenvwrapper.sh
mkvirtualenv pulp-cli
workon pulp-cli
cd pulp-cli
pip install -e .

Resetting the container's DB w/out starting up a new container

podman exec -it pulp bash -c "pip install django-extensions"
podman exec -it pulp bash -c "pulpcore-manager reset_db -U postgres -c --noinput"
podman exec -it pulp bash -c "s6-svc -d /var/run/s6/services/pulpcore-content"
podman exec -it pulp bash -c "pulpcore-manager migrate"
podman exec -it pulp bash -c "s6-svc -r /var/run/s6/services/pulpcore-worker@1"
podman exec -it pulp bash -c "s6-svc -r /var/run/s6/services/pulpcore-worker@2"
podman exec -it pulp bash -c 'pulpcore-manager reset-admin-password'
pulp --username admin --password admin rpm remote create --name foo --url "https://example.com/"

Access content-app from "outside" your VM (8000 is 'your machine' port to access, '8080' is the port "inside" of pulp)

vagrant ssh fedora34 -- -L 8000:fedora34.padre-fedora.example.com:8080
In external browser : http://localhost:8000/pulp/content/

Get pulp-ui installed, running, contacted

sudo dnf install -y npm
git clone https://github.com/pulp/pulp-ui.git
cd pulp-ui/pulp_ui/app/
npm install
npm run start:dev >> out.txt &
From outside-the-vagrant-box, set up SSH-tunnel to the UI's port:
vagrant ssh fedora34 -- -L 8081:fedora34.padre-fedora.example.com:9000
In external browser: localhost:8081 : ta-da! Dashboard page!

Detailed errors

From F33

Copying config 0c22088e15 done Writing manifest to image destination Storing signatures Error: OCI runtime error: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: Running hook #1:: error running hook: exit status 1, stdout: , stderr:

No error message, it just fails to start.

Deprecated JSON

Running latest, lots of scary-looking "Deprecated JSON" messages still. I know what they are - but to a new user, this is suboptimal.

Scary database failures

If you try to set admin-db-pwd or get status "too quickly", you can end up with scary-looking errors. First guess is "hitting the container before migrations are finished is Bad, please don't." Waiting (literally) a minute and re-running works fine. Might want to document, along with the symptomatic strings from the list below.

[vagrant@fedora34 onecontainer]$ curl localhost:8080/pulp/api/v3/status/
curl: (56) Recv failure: Connection reset by peer

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 581, in get_or_create
    return self.get(**kwargs), False
  File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 435, in get
    raise self.model.DoesNotExist(
django.contrib.auth.models.DoesNotExist: User matching query does not exist.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.NotNullViolation: null value in column "last_login" violates not-null constraint
DETAIL:  Failing row contains (1, , null, t, admin, , , , t, t, 2021-10-03 15:03:14.407959+00).

2021-10-02 16:10:46.133 UTC [14867] LOG:  starting PostgreSQL 12.7 on x86_64-redhat-linux-gnu, compiled by gcc (GCC) 8.3.1 20191121 (Red Hat 8.3.1-5), 64-bit
2021-10-02 16:10:46.134 UTC [14867] LOG:  listening on IPv6 address "::1", port 5432
2021-10-02 16:10:46.134 UTC [14867] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2021-10-02 16:10:46.184 UTC [14867] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-10-02 16:10:46.235 UTC [14867] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5432"
2021-10-02 16:10:46.289 UTC [14867] LOG:  redirecting log output to logging collector process
2021-10-02 16:10:46.289 UTC [14867] HINT:  Future log output will appear in directory "log".
pulp [None]: pulpcore.app.serializers.base:WARNING: Please provide either 'view_name' or 'view_name_pattern' for DetailRelatedField on _call_with_frames_removed.
Operations to perform:
  Apply all migrations: admin, ansible, auth, certguard, container, contenttypes, core, deb, file, guardian, maven, python, rpm, sessions
Running migrations:
  Applying rpm.0025_remove_orphaned_subrepos...Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.ProgrammingError: column core_repository.retained_versions does not exist
LINE 1: ...tory"."pulp_type", "core_repository"."remote_id", "core_repo...