Host your own on-premise Ansible Galaxy

Brian Bouterse
Senior Principal Software Engineer

bmbouter@redhat.com

slides: https://hackmd.io/@pulp/ansible-containers#/

Agenda

• Setting up an On-Premise Galaxy

  • with UI
  • without UI

• Syncing in collections/roles from galaxy.ansible.com

• Installing collections/roles using CLI

• Uploading collections via ansible CLI

• Organizing content in repositories

Software

Galaxy NG - https://github.com/ansible/galaxy_ng/

• A fork+rewrite of original Galaxy code
• Has a UI
• Developed by the same team that runs galaxy.ansible.com
• Only Collection support
• GPLv2

pulp_ansible - https://github.com/pulp/pulp_ansible

• The backend of galaxy NG
• API and CLI Only, No-UI
• Collection and Role Support
• GPLV2

Deploy Galaxy NG

Pulling down the container

podman pull pulp/pulp-galaxy-ng

Create settings files

mkdir galaxy_ng
cd galaxy_ng
mkdir settings pulp_storage pgsql containers

echo "CONTENT_ORIGIN='http://$(hostname):8080'
ANSIBLE_API_HOSTNAME='http://$(hostname):8080'
ANSIBLE_CONTENT_HOSTNAME='http://$(hostname):8080/pulp/content'"  >> settings/settings.py

Start the container with SELinux

• Running on port 8080
• http only (currently)
• Usable with either podman or docker commands

podman run --detach \
             --publish 8080:80 \
             --name galaxy_ng \
             --volume ./settings:/etc/pulp:Z \
             --volume ./pulp_storage:/var/lib/pulp:Z \
             --volume ./pgsql:/var/lib/pgsql:Z \
             --volume ./containers:/var/lib/containers:Z \
             --device /dev/fuse \
             pulp/pulp-galaxy-ng

Start the container without SELinux

podman run --detach \
             --publish 8080:80 \
             --name galaxy_ng \
             --volume ./settings:/etc/pulp \
             --volume ./pulp_storage:/var/lib/pulp \
             --volume ./pgsql:/var/lib/pgsql \
             --volume ./containers:/var/lib/containers \
             --device /dev/fuse \
             pulp/pulp-galaxy-ng

Load required initial data

DATA_FIXTURE_URL="https://raw.githubusercontent.com/ansible/galaxy_ng/master/dev/automation-hub/initial_data.json"
curl $DATA_FIXTURE_URL | podman exec -i galaxy_ng bash -c "cat > /tmp/initial_data.json"
podman exec galaxy_ng bash -c "/usr/local/bin/pulpcore-manager loaddata /tmp/initial_data.json"

Assign an admin password

podman exec -it galaxy_ng bash -c 'pulpcore-manager reset-admin-password'

Basics

Accessing the UI: http://localhost:8080
Showing the logs: podman logs -f galaxy_ng

Use Galaxy NG

1. Show the Repositories
2. Show the Remotes
3. Sync Content

Two requirements file examples

---
collections:
# With just the collection name
- pulp.pulp_installer
- ansible.posix

---
collections:
# With just the collection name
- amazon.aws

Gotchas

• It does not currently provide dependency resolution
• Every time you sync, it overwrites the previous content
• Don't have a ~/.netrc file or it won't use your token

Configuring the CLI

1. Get the config file from the UI
2. Add the token from the UI
3. Install with a command like: ansible-galaxy collection install -p ./ pulp.pulp_installer

Uploading content from the UI

1. Have a local collection, e.g. https://galaxy.ansible.com/newswangerd/collection_demo
2. Create a "namespace" in the UI
3. Upload a built collection
4. Go through the approval workflow

You can turn off the approval workflow

Add this to settings.py:

galaxy_require_content_approval=False

https://github.com/ansible/galaxy_ng/wiki/Enabling-auto-certification

Users, Groups, Permissions

Uploading content from the CLI

1. Same as from the UI, except use the ansible-galaxy collection publish command

Deploy Pulp_Ansible

Pulling down the container

podman pull pulp/pulp

Create settings files

mkdir pulp
cd pulp
mkdir settings pulp_storage pgsql containers

echo "CONTENT_ORIGIN='http://$(hostname):8080'
ANSIBLE_API_HOSTNAME='http://$(hostname):8080'
ANSIBLE_CONTENT_HOSTNAME='http://$(hostname):8080/pulp/content'
TOKEN_AUTH_DISABLED=True" >> settings/settings.py

Start the container with SELinux

• Running on port 8080
• http only (currently)
• Usable with either podman or docker commands

podman run --detach \
             --publish 8080:80 \
             --name pulp \
             --volume ./settings:/etc/pulp:Z \
             --volume ./pulp_storage:/var/lib/pulp:Z \
             --volume ./pgsql:/var/lib/pgsql:Z \
             --volume ./containers:/var/lib/containers:Z \
             --device /dev/fuse \
             pulp/pulp

Start the container without SELinux

podman run --detach \
             --publish 8080:80 \
             --name pulp \
             --volume ./settings:/etc/pulp \
             --volume ./pulp_storage:/var/lib/pulp \
             --volume ./pgsql:/var/lib/pgsql \
             --volume ./containers:/var/lib/containers \
             --device /dev/fuse \
             pulp/pulp

Assign an admin password

podman exec -it pulp bash -c 'pulpcore-manager reset-admin-password'

Installing pulp-cli

pip install pulp-cli

https://github.com/pulp/pulp-cli

Point pulp-cli at your container

pulp config create
pulp config edit

1. Point to "http://localhost:8080"
2. Add username, password

Basics

Showing the logs: podman logs -f pulp
Status check: pulp status
API Documentation: http://localhost:8080/pulp/api/v3/docs/

Create an Ansible Repository

• pulp ansible repository create --name myrepo
• pulp ansible repository list

Create an Ansible Remote

• pulp ansible remote -t collection create --name basic --url https://galaxy.ansible.com --requirements-file ~/Documents/Presentations/container_demo/basic.yml
• pulp ansible remote list

Create an Ansible Distribution

• pulp ansible distribution create --name mydistro --base-path foo --repository myrepo
• pulp ansible distribution list

View it through the Galaxy API

http://localhost.localdomain:8080/pulp_ansible/galaxy/foo/api/v3/collections/

Sync content using the Remote into the Repo

pulp ansible repository sync --name myrepo --remote basic

Installing using the CLI

1. Get the client URL for this repo from the distribution list: pulp ansible distribution list
2. Install with a command like: ansible-galaxy collection install -s http://localhost.localdomain:8080/pulp_ansible/galaxy/foo/ -p ./ pulp.pulp_installer

Uploading from the CLI (part 1)

• Get the client URL for this repo from the distribution list: pulp ansible distribution list
• Have a tarball, e.g. https://galaxy.ansible.com/newswangerd/collection_demo

Uploading from the CLI (part 2)

• Upload the tarball with ansible-galaxy collection publish -s http://localhost.localdomain:8080/pulp_ansible/galaxy/foo/ newswangerd-collection_demo-1.0.10.tar.gz
• View it's availability in the repository: http://localhost.localdomain:8080/pulp_ansible/galaxy/foo/api/v3/collections/

Creating a remote for Role content

1. First a word about the galaxy Role API
2. pulp ansible remote -t role create --name someroles --url https://galaxy.ansible.com/api/v1/roles/?namespace__name=elastic

Syncing Roles

pulp ansible repository sync --name myrepo --remote someroles

Viewing them: http://localhost.localdomain:8080/pulp_ansible/galaxy/foo/api/v1/roles/

Installing Roles from the CLI

1. Get the client URL for this repo from the distribution list: pulp ansible distribution list
2. Install with a command like: ansible-galaxy role install -s http://localhost.localdomain:8080/pulp_ansible/galaxy/foo/ -p ./ elasticsearch,6.2.4

Configuring the CLI permanently

Use an ansible.cfg like:

[galaxy]
server: http://localhost.localdomain:8080/pulp_ansible/galaxy/foo/

Then ansible-galaxy role install -p ./ elasticsearch,6.2.4

Copying Content between Repositories

Not demo-ing but see these docs: https://docs.pulpproject.org/pulp_ansible/workflows/copy.html

Getting Help

• https://www.redhat.com/mailman/listinfo/pulp-list
• https://www.redhat.com/mailman/listinfo/galaxy-ng
• #pulp on Freenode
• See the docs for filing bugs, changes, etc.

Feedback

Brian Bouterse
bmbouter@redhat.com