# Reducing redirects on docs.ansible.com

This document outlines a proposal to reduce the number of server-side redirects on Red Hat managed infrastructure behind `docs.ansible.com`. These server-side redirects exist to prevent 404 errors when documentation pages are moved or when url structures changed when the split to collections happened.

The purpose of reducing the number of redirects for `docs.ansible.com` is two-fold:

- Firstly, consolidating redirects facilitates the migration of the `docs.ansible.com` subdomain to Read the Docs. There is a limit of 100 redirects for each project on Read the Docs. At present there are thousands of server-side redirects, greatly exceeding the limit.
- Secondly, consolidating redirects drastically lowers the maintenance overhead that comes with thousands of server-side redirects. To prevent 404s, these server-side redirects need to be kept up to date as modules and plugins are moved between collections and collections get deprecated from the Ansible package.

## Overview of server-side redirects

Server-side redirect rules are defined in `.htaccess` configuration files in the `ansible/docsite` repository. These files exist in two places:

- The `ansible/` directory contains rules for documentation pages that existed before the move to collections. There are a few rules for specific pages and other catch-all rules as well.
- The root directory contains a single [.htaccess configuration](https://github.com/ansible/docsite/blob/main/.htaccess) file with rules for pages in the `devel` and `latest` versions of the package docs as well as for some pages in the core docs.

### Redirects for old versions

Consider the configuration file for the Ansible 10 release at: https://raw.githubusercontent.com/ansible/docsite/a841d2a2c1df01b783a6d0f4361817645d73d67d/ansible/10/.htaccess

These rules primarily redirect pages that had the `/modules/` or `/plugins/` path in the url to the corresponding `/collections/` path.

For each new version of the Ansible package, some of the `.htaccess` files are moved to a new folder. For example, with the Ansible 11 package, `ansible/10/.htaccess` becomes `ansible/11/.htaccess` with no changes to the rules themselves. Here is a PR where that happened in an older version: https://github.com/ansible/docsite/pull/195

Within the `ansible/` directory there are also some `2.x` folders. Within the `ansible/2.x/` folders are more `.htaccess` files that contain rules to redirect pages in older documentation to the `latest` version of the package docs.

This applies to older documentation that predates the switch to using the `latest` version. In many cases pages from the older documentation was linked in StackOverflow posts and blogs. In turn these older documentation versions were establishing SEO authority and usually returned high in the rankings for Ansible documentation searches. To prevent those pages from 404'ing and degrading the rankings, redirects were put in place. As an example, consider the [ansible/2.6/.htaccess](https://github.com/ansible/docsite/blob/main/ansible/2.6/.htaccess) file that contains the following rule:

RedirectMatch permanent "^/ansible/([^/]+)/user_guide/playbooks_vault.html" "/ansible/latest/vault_guide/index.html"

This causes https://docs.ansible.com/ansible/2.6/user_guide/playbooks_vault.html to redirect to the latest version of the package docs.

It's worth noting that several of these older versions were moved into the [docs archive](https://docs.ansible.com/archive/) in case someone needs access to the content. However the redirects do not apply to the archive, only the original urls.

### Version 2.9

One oddball case are the rules under [`ansible/2.9/.htaccess`](https://github.com/ansible/docsite/blob/685e467b26b5eb4b6425a8c99be1323cd0652976/ansible/2.9/.htaccess). As you might observe, these rules redirect content from a `collections` path to either `plugins` or `modules`, effectively the opposite of all the other redirects. Yay!

The reason for the difference in the `ansible/2.9/.htaccess` rules stems from the version switcher. Some Red Hat customers have a support entitlement to the 2.9 documentation so that version has been available from `docs.ansible.com` for several years. Here is a screenshot of the version switcher:


This version switcher simply replaces the version in the url. Consider the scenario where you are on this page:

https://docs.ansible.com/ansible/latest/collections/community/general/doas_become.html

If you select `2.9` from the version switcher, it updates the url to this:

`https://docs.ansible.com/ansible/2.9/collections/community/general/doas_become.html`

However, version 2.9 predates collections so that will result in a 404. To prevent this, the redirects point in the opposite direction back to the corresponding plugin or module that existed before the collections split. That redirect would take you here:

https://docs.ansible.com/ansible/2.9/plugins/become/doas.html

## Supporting the migration to Read The Docs

To migrate the `docs.ansible.com` subdomain to Read The Docs, we need to use Read The Docs' own redirect system for some cases to retain search engine authority. However, there is a limit of 100 user-defined redirects on Read The Docs per project.

Any redirects on Read The Docs that will support SEO from `docs.ansible.com` should be added to the top-level `Ansible` project and not the package docs or core docs subprojects. Unfortunately it is not possible to recreate the thousands of server-side redirects on Read The Docs because of the limit, unless we had budget of several thousand USD per month for enterprise support from Read The Docs. Even if we had that it would take a very long time to manually recreate all the server-side redirects for all the legacy versions.

## Reducing maintenance burden

Another reason to reduce the number of server-side redirects is to eliminate the maintenance overhead of keeping them up to date. Consider the link in [this blog post on ansible.com](https://www.ansible.com/blog/kubernetes-operators-ansible-deep-dive-part-1):

https://docs.ansible.com/ansible/latest/modules/k8s_module.html

There is a redirect for this in the v10 `.htaccess` configuration on this line: https://github.com/ansible/docsite/blob/a841d2a2c1df01b783a6d0f4361817645d73d67d/ansible/10/.htaccess#L2466

This redirect works by following the "latest" symlink to the "10" docs so that users end up on this page:

https://docs.ansible.com/ansible/latest/collections/community/kubernetes/k8s_module.html

However this page is 404 because the redirect should actually point to this url: https://docs.ansible.com/ansible/latest/collections/kubernetes/core/k8s_module.html

This goes to highlight the maintenance overhead that comes with `.htaccess` configuration files with thousands of lines. These should really be verified and updated between releases, especially as collections become deprecated and get removed from the package.

Here is another example where the [defined redirect](https://github.com/ansible/docsite/blob/a841d2a2c1df01b783a6d0f4361817645d73d67d/ansible/10/.htaccess#L2720) points to a [deprecated module](https://docs.ansible.com/ansible/latest/collections/community/network/index.html#plugin-index) in a collection that will be removed from the package.

## Consolidating redirects

We can consolidate all the server-side redirects in the `/ansible/` subdirectory of the `ansible/docsite` repository into a single `.htaccess` file, for example:

RedirectMatch permanent "^/ansible/(devel|latest)/(plugins|modules)/(.+)\.html$" "https://docs.ansible.com/collections.html"

Rather than muliple, separate redirect rules that take users from specific pages to a directly corresponding target page, this rule applies to all matching urls and takes users to a single page such as this one:

https://ansible--261.org.readthedocs.build/collections.html

This approach consolidates thousands of rules into a single rule. It also avoids the need for maintenance to prevent 404s and SEO degradation.

On the other hand, this approach is not necessarily a better user experience because it requires anyone who lands on the target page to search for the appropriate documentation. However, we can mitigate this by including links to the most popular pages as well as embedding search functionality.

For more information on consolidating redirects, see the following PR in the `ansible/docsite` repository: https://github.com/ansible/docsite/pull/261

After we consolidate redirects, we can recreate them in the `ansible` namespace on Read The Docs. For more information about that effort, see https://hackmd.io/3K5Es_UZQvyUTLWPmdtFxQ?view

## Stub pages

To handle the redirects in the `.htaccess` configuration file at the root of the `ansible/docsite` repository, we can replace those rules with stub pages. Stub pages are RST files that use the `orphan` metadata field so that the page is generated at build time but not included in the toctree. The result is that the stub page does not appear in the navigation or documentation structure, while still being reachable from a direct external link such as a bookmark or reference in a third-party site.

Consider the following redirect that was created because the `playbooks_blocks` page was moved during a restructure effort:

RedirectMatch permanent "^/ansible/(devel|latest)/user_guide/playbooks_blocks.html" "/ansible/$1/playbook_guide/playbooks_blocks.html"

For this particular rule, a stub pages was used before the redirect was created. Here is the source for the stub page: https://raw.githubusercontent.com/ansible/ansible-documentation/refs/heads/devel/docs/docsite/rst/user_guide/playbooks_blocks.rst

Because the redirect does not take effect on Read The Docs, you can actually see the stub page at: https://ansible.readthedocs.io/projects/ansible/latest/user_guide/playbooks_blocks.html

For more information on creating stub pages to replace redirects, see the following issue in the `ansible/ansible-documentation` repository: https://github.com/ansible/ansible-documentation/issues/2147