# Creating AWX CLI Guide

(note: Some links go to internal Red Had Jenkins jobs and are not accessable to the general community at this point).

## Problem

We want to be able to generate and publish the CLI guide to AWX documentation instead of [controller docsite](https://docs.ansible.com/automation-controller/latest/html/controllercli/index.html).

The current Jenkins job and supporting playbooks are controller-branded.

Also, the tool (awxcli) generates a complete sphinx-based html docsite (stand alone).

## Options

The AWX/Controller CLI guide depends on having an active AWX/Controller instance running to pull the details from.

The current code that generates the CLI guide creates HTML for a standalone docsite. We have the following options to create the guide for AWX:

Option 1. Integrate awxkit files and conf.py into AWX docsite generation.
Option 2 Generate RST and integrate into AWX docsite
Option 3 - Create Standalone docsite using existing resources with branding modifications.

**NOTE** would be beneficial to downstream docs if we can produce some interim output they can bring into their downstream toolchain. Today, downstream publishing supports asciidoc, but moving to DITA. TBD if .md will be supported in the future. Sphinx can generate .xml files in the docutils format so that might be an option to consider (``sphinx-build -b xml source build``).

### 1. Integrate awxkit cli files into AWX docsite

There are two steps here:

1. Static doc files are in another directory - we can use a symlink to get them where they need to be. (Tested and this works).
2. Integrate the actual cli generation (requires a running AWX instance to communicate with).

Can test this locally but how would the RTD docs generation instantiate a running AWX instance?? Possibly with GitHub actions?

----------------------------------------
Ideas below here have been rejected...

### 2. Generate RST and integrate into AWX docsite

Since the existing code is branded and generates html for a standalone site, we cannot use it directly. Instead we could:
1. Modify the code generation to pull out an intermediate step of creating rst files.
2. Modify the AWX docs creation to pull in this additional RST.
3. Publish to the AWX docsite.

**NOTE: We need to keep the existing generation working for the Controller CLI Guide so cannot break that rendering option.**

#### Problem: Cannot get RST out of existing extension

The existing code creates a foo.doctree file, but docutils tools like [xml2rst](https://sourceforge.net/p/docutils/code/HEAD/tree/trunk/sandbox/xml2rst/) can't seem to convert this back to an .rst file (not to mention the tool is python2 based and requires some changes to run).

I've been experimenting with the sphinx extension, [sphinx-contrib.autoprogram](https://github.com/sphinx-contrib/autoprogram/blob/master/sphinxcontrib/autoprogram.py) and the [awxcli custom extenstion](https://github.com/ansible/awx/blob/devel/awxkit/awxkit/cli/sphinx.py) based on this. I can see they build up a docutils document structure within the code, but similarly cannot see how to get this to an RST file.

Seems sphinx directives produce .doctree files that also include the TOC for the whole site so can't use this directly, but perhaps the next step is to try and move some of the awxkit/cli stuff into the /docs/docsite folder and see how far I get.

### 3. Create Standalone docsite

The following are basic steps to create a standalone AWX CLI guide (not part of the AWX docsite). It uses existing resources and rebrands for AWX:

The [static CLI rst files](https://github.com/ansible/awx/tree/devel/awxkit/awxkit/cli/docs/source) already exist in the AWX repo, so we could
1. Copy the playbooks to GitHub actions to generate the HTML with AWX branding.
2. Publish to RTD as awx-cli-guide (standalone site).

#### GitHub Workflow

Drafting out how a new **awx_cli_docs workflow** would go.
1. Run the awx image (with [run_awx_devel](https://github.com/ansible/awx/tree/devel/.github/actions/run_awx_devel) action), which will call [awx_devel_image](https://github.com/ansible/awx/tree/devel/.github/actions/awx_devel_image) action) to generate the awx devel images.
2. Run a new action - ``awx_cli_docs_generate`` action with the following:
`CONTROLLER_HOST=https://awx.example.org CONTROLLER_USERNAME=example CONTROLLER_PASSWORD=secret make clean html`
`CONTROLLER_HOST=https://awx.localhost:8043 CONTROLLER_USERNAME=admin CONTROLLER_PASSWORD=password make clean html`

Where the details come from step 1 on where the awx instance is, password, etc. (seems from the .yaml that username is admin and password is password).

3. Append the [tox.ini](https://github.com/ansible/awx/blob/devel/tox.ini#L33) for docs to combine this HTML with the `spinx-build` HTML.

## Creating the CLI guide locally

You will need a running awx instance. You can either build awx_devel locally or get the image.

Notes on building locally:
If running in a VM, must have 16G memory to build UI etc

docker-compose is already part of the docker install
1. pip install -r requirements/requirements.txt
2. Set up docker for nonroot user https://docs.docker.com/engine/install/linux-postinstall/
3. Ceate devel image https://github.com/ansible/awx/blob/devel/tools/docker-compose/README.md
4. (optional) Build the ui (samccann - can't get this step working)
$ docker exec tools_awx_1 make clean-ui ui-devel
5. create awx user -
$ docker exec -ti tools_awx_1 awx-manage createsuperuser

### Generate local copy of awx cli docsite

You can [build the CLI locally](https://github.com/samccann/awx/tree/devel/awxkit/awxkit/cli) if you have an AWX instance running.

1. Install requirements for awxkit -
pip install sphinx sphinxcontrib-autoprogram
`pip install awxkit` (dunno how to run from source)

2. cd awxkit/awxkit/cli/docs/
3. CONTROLLER_HOST=https://localhost:8043 CONTROLLER_USERNAME=awx CONTROLLER_PASSWORD=awx CONTROLLER_VERIFY_SSL=f make clean html

# Generating the AWX CLI guide locally
Just samccann's scribble-notes on how she got it running locally...

## Creating awx_devel image

If running in a VM, must have 16G memory to build UI etc

- docker-compose is already part of the docker install
1. (may not be required if running the latest container image see step 3) pip install -r requirements/requirements.txt
2. set up docker for nonroot user https://docs.docker.com/engine/install/linux-postinstall/
3. create devel image https://github.com/ansible/awx/blob/devel/tools/docker-compose/README.md
**NOTE: You can create the image locally for awx, or you can just get the latest from https://github.com/ansible/awx/pkgs/container/awx_devel**
4. run awx instance - `make docker-compose`

In a separate terminal:
1. (optional) build the ui
$ docker exec tools_awx_1 make clean-ui ui-devel
2. create awx user -
$ docker exec -ti tools_awx_1 awx-manage createsuperuser

## Generate local copy of awx cli docsite

1. install requirements for awxkit -
pip install sphinx sphinxcontrib-autoprogram
pip install awxkit (dunno how to run from source)

2. cd awxkit/awxkit/cli/docs/
3. CONTROLLER_HOST=https://localhost:8043 CONTROLLER_USERNAME=awx CONTROLLER_PASSWORD=awx CONTROLLER_VERIFY_SSL=f make clean html

## From the cli docs:

The preferred way to install the AWX CLI is through pip:

pip install "git+https://github.com/ansible/awx.git@$VERSION#egg=awxkit&subdirectory=awxkit"

…where $VERSION is the version of AWX you’re running. To see a list of all available releases, visit: https://github.com/ansible/awx/releases

## Common steps on reboot:

sudo systmectl start docker
docker run hello-world (just to verify)
source ~/venvs/awx/bin/activate
make docker-compose

### Further experiments
Going from doctrees to xml:

Create a small unpickle.py script:
import pickle

with open("install.doctree", "rb") as f:
result = pickle.load(f)

print(result)

then python unplickle.py > xx to see the unpickled XML generated during the sphinx docs build.

xml2rst
-------
clone or fork https://github.com/docutils/docutils
follow readme - https://github.com/samccann/docutils/blob/master/sandbox/xml2rst/README

- pip install lxml
- run 2to3 against sandbox/xml2rst and sandbox/docutils_xml
- Add this to ~/.bashrc : export PYTHONPATH=/home/samccann/docutils/sandbox/docutils_xml:
- if needed in VSCode - type F1 Terminal: Select Default Shell and set to bash

At this point - python3 xml3rst.py --help should work

# Anwesha's notes

## Generate the AWX CLI Docs locally

1. Create VM and ssh to new VM
- Create fedora 40 Virtual Machine with Core 2 and 20GB RAM
- Python 3.12.4
- get the ip address of the newly created VM `ip a s`
- add ssh key in the local machine and ssh
- `ssh-copy-id user@ipofvm`
- `ssh -L 8080:127.0.0.1:8000 user@ipofvm`
`ssh -l` helps us to open the browser in the local machine.

2. Start the docker container . Fedora 40 comes with a preinstalled docker

`sudo systemctl start docker`
`sudo systemctl status docker`

3. Install sphinx sphinxcontrib-autoprogram

`python3 -m pip install sphinx sphinxcontrib-autoprogram`
`python3 -m pip install awxkit`
`cd awxkit/awxkit/cli/docs`

4. `CONTROLLER_HOST=https://localhost:8043 CONTROLLER_USERNAME=awx CONTROLLER_PASSWORD=awx CONTROLLER_VERIFY_SSL=f make clean html
`
5. Install the `http.server` module and activate the virtual environment

`python3 -m http.server`
`python3 -m venv .venv`
`source .venv/bin/activate`

6. Get the official [awx_devel image](https://github.com/ansible/awx/pkgs/container/awx_devel) from the ghcr repository

`docker images`
`docker pull ghcr.io/ansible/awx_devel:devel`

7. Go to the awx directory

`cd awx`
8. run the docker compose command

`make docker-compose`

9. Open the build html in the chosen port of the local browser.
`localhost:8080/build/html`

## Steps followed in Generating the AWX CLI docs

1. Create VM and ssh to new VM
- Create fedora 40 Virtual Machine with Core 2 and 20GB RAM
- Python 3.12.4
- get the ip address of the newly created VM `ip a s`
- add ssh key in the local machine and ssh
- `ssh-copy-id user@ipofvm`
- `ssh user@ipofvm`

2. Install the dependencies

	sudo dnf install docker docker-compose ansible openssl 
	sudo systemctl start docker
	sudo dnf install git make gcc python-devel xmlsec1 python3-ldap xmlsec1-openssl openldap-devel
	

3. Clone the awx repo

`git clone https://github.com/ansible/awx.git`

4. Start the Python virtual env

python3 -m venv .venv
source .venv/bin/activate

5. Go to the `awx` dir `cd awx`

6. `make docker-compose`

7. Install dependencies

python3 -m pip install setuptools_scm xmlsec 
sudo dnf install xmlsec1-devel 

8. Export the environment variable
` export CFLAGS="-Wno-error=incompatible-pointer-types"`

9. Install dependecies

sudo dnf install libtool-ltdl-devel libtool-ltdl

10. The docker engine on Fedora CoreOS does not support the “buildx” subcommand which is needed to build container images for multiple architectures. To solve this issue follow the following steps (in your local browser)

1. download buildx from [Releases · docker/buildx · GitHub 304](https://github.com/docker/buildx/releases)
2. Get the the code in the VM from local machine.
`scp Downloads/docker-desktop-x86_64.rpm userid@ipofvm`
3. move the binary in `~/.docker/cli-plugins/docker-buildx` directory

	    mkdir -p ~/.docker/cli-plugins
	    mv ~/buildx-v0.16.2.linux-amd64 ~/.docker/cli-plugins/docker-buildx
	    chmod +x /.docker/cli-plugins/docker-buildx
	    sudo chgrp wheel /var/run/docker.sock

11. Build the doc with the make command
`make docker-compose-build`

## Use the ghcr image

1. Start new vm 
2. Follow https://github.com/ansible/awx/pkgs/container/awx_devel to download the awx image
3. run the image to 

## How is the doc  created?

We have the awxkit this along with sphinx-contrib creates the cli docs. What 

- Install desktop for fe

- What is happening this 
- document the flow of the jenkins jobs.


## Brainstorming Ideas

* If we have a API guide can we create a CLI Guide based on that info?
* AWX and Tower repo connection. (Boiler plate seems to be the same)
* Detailed md page -> Controller AWX Cli doc is generated now (for Ansible Engineering Handbook)

## Steps 

- Detailed md page -> Controller AWX Cli doc is generated now (for Ansible Engineering Handbook) - Jira Issue under the AWX Cli epic
- 
- Fork the tower repo to show Toshio and Sandra.

## Links

1. https://github.com/docker/buildx/issues/262