# Pulp OpenAPI observations

The OpenAPI schema (OAS) generated by the pulp endpoints (/pulp/api/v3/docs/api.yaml etc)

# Observations

## Based on OpenAPI v2 spec (from 2014), v3 was released (2017)

V2 not changed since 2017

OpenAPI related tools are moving to supporting v3 only.

Possible solution: Move to drf built in OpenAPI generation? Would be non trivial.

## Generation of v2 spec is based on drf_yasg which is sort of maintained.

drf_yasg has no plans to support OpenAPI spec v3.

Current versions of django-rest-framework has it’s own OpenAPI schema generation *(albeit, it’s not great...)*. One result of the DRF native adoption of OpenAPI is that DRF has started moving away from supporting generic api description generation tools (ie, it’s ‘coreapi’ and related hooks used by drf_yasg).

**Possible solution:** Move to drf built in OpenAPI generation? Would be non trivial.

## The current drf_yasg v2 based generation has a lot of customizations and workarounds.

See https://github.com/pulp/pulpcore/blob/master/pulpcore/app/openapigenerator.py

A lot of this is driven from use of patterns that don’t really match the expected OpenAPI use cases very well.

**Possible solution:** Not sure there is a good solution. Would likely involve somehow making the api implementation more conventional (whatever that means). 🤷

## The {‘pulp_href’} style endpoint paths are unusual.

{ansible_distribution_href}
{ansible_remote_href}
{ansible_repository_href}
{ansible_repository_href}modify/
{ansible_repository_href}sync/
{ansible_repository_href}versions/
{ansible_repository_version_href}
{ansible_repository_version_href}repair/
{artifact_href}
{collection_href}
{collection_href}versions/
{collection_import_href}
{collection_remote_href}
{collection_version_href}
{namespace_href}
{pulp_export_href}
{pulp_exporter_href}
{pulp_exporter_href}exports/
{pulp_import_href}
{pulp_importer_href}
{pulp_importer_href}imports/
{role_href}
{signing_service_href}
{task_group_href}
{task_href}
{upload_href}
{upload_href}commit/
{user_href}
{worker_href}

This use makes browsing the api spec much less useful than it could be. There are no predictable urls to use, and the overall organization of the API is impossible to understand.

Using a path value like `{worker_href}` is also invalid according to the OpenAPI spec (v2 and v3). Some OpenAPI related tooling doesn’t work at all with these paths while other tooling marks it as invalid (swaggerui shows it as invalid, some lint tools just raise “Property {worker_href} is not valid” errors.

The spec for paths objects:
* (v2) https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md#patterned-fields

> A relative path to an individual endpoint. The field name MUST begin with a slash. The path is appended to the basePath in order to construct the full URL. Path templating is allowed.

* (v3) https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.3.md#patterned-fields

> A relative path to an individual endpoint. The field name MUST begin with a forward slash (/). The path is appended (no relative URL resolution) to the expanded URL from the Server Object's url field in order to construct the full URL. Path templating is allowed. When matching URLs, concrete (non-templated) paths would be matched before their templated counterparts. Templated paths with the same hierarchy but different templated names MUST NOT exist as they are identical. In case of ambiguous matching, it's up to the tooling to decide which one to use.

The v3 spec elaborates on the use of path templating:

> Each template expression in the path MUST correspond to a path parameter that is included in the Path Item itself and/or in each of the Path Item's Operations.

`{some_href}` isn't valid for spec v2 or v3, and isn't a valid path template parameter.

**Potential solution:** For populated pulp systems, enumerate the valid hrefs for at least paths that known at the time of rendering the spec. Ie, if there is an 'ansible/ansible' Repository defined:

{ansible_repository_href} -> (/pulp/api/v3/repositories/ansible/ansible/<pk>/,         
/pulp/api/v3/repositories/ansible/ansible/<repository_pk>/versions/)

## Many endpoints/viewset have no OpenAPI spec generated for them.

For example, any viewsets that use a `get_queryset()` but don’t specifically handle the ‘fake_schema_view’ will log warnings like:

UserWarning: <class 'galaxy_ng.app.api.ui.viewsets.tags.TagsViewSet'> is not compatible with schema generation
UserWarning: <class 'pulp_ansible.app.galaxy.v3.views.CollectionViewSet'> is not compatible with schema generation
UserWarning: <class 'pulp_ansible.app.galaxy.v3.views.CollectionVersionViewSet'> is not compatible with schema generation

**Possible Solution:** Add handing of the `‘fake_schema_view’` to get_queryset() methods. Even better if it can be done in a base class or mixin. See https://drf-yasg.readthedocs.io/en/stable/openapi.html#a-note-on-limitations

**Possible Solution:** Manual markup of endpoints with `@swagger_auto_schema`

## There are lots of name collisions on Viewsets,Serializers, and Models between pulp, pulp_ansible, galaxy_ng.

For example ‘CollectionVersionViewSet’ and ‘CollectionVersionSerializer’.

Python/django doesn't care, but the schema generation does as it uses the base names to try to generate the names for the OpenAPI model schema definitions. For a lot of these cases, it will fail and complain about `‘ref_name’` collisions.

**Possible Solution:** For ModelSerializers, add an explicit `‘ref_name’` attribute to the serializers Meta class. And use an appropriate naming convention to avoid collisions.

**Possible Solution:** Extend use of some of the custom metaclasses that set `ref_name`. Ala https://github.com/pulp/pulpcore/blob/master/pulpcore/app/serializers/base.py#L89

**Possible Solution:** Avoid reusing names for models, viewsets, and serializers.

## For the cases where the ref_name has namespace, the namespaces are confusing and inconsistent.

For ‘schema’ components defined:

* Pulpcore uses no namespace (Task, TaskRead, RepositoryVersion)
* Galaxy_ng mostly uses no namespace (Namespace, CollectionVersion, User)
* Pulp_ansible uses ‘ansible’ namespace (ansible.AnsibleDistribution) but also sometimes (GalaxyRoleRead, GalaxyCollection)
* Galaxy_ng doesn’t distinguish between ‘ui’ and ‘v3’ API objects
* Pulp_ansible doesn’t distinguish between current ‘v3’ api models and community galaxy compat v1/v2 style models (ie, GalaxyRoleRead)

**Possible Solution:** Use consistent naming, likely created automatically via something like the BaseSerializer.

**Possible Solution:**
* pulpcore: pulp.* or no namespace (likely for client api compat)
* Pulp_ansible: use ‘galaxy.v1’, ‘galaxy.v2’, ‘galaxy.v3’ namespaces. Leaving ‘ansible’ also an option.
* Galaxy_ng: use ‘galaxy_ng.v3’ and ‘galaxy_ng.v3.ui’ namespaces

## DRF OpenAPI ‘generateschema’ doesn’t work with pulp

https://pulp.plan.io/issues/6714