# Resolving Abstract Distribution Models in Plugin API

## Problem Statement

Related Problem: Deprecation policy

## Proposal 1 (add new fields in the same Master Model, all done in 3.12)

In 3.12
1. Introduce new fields (with new names, e.g. `_repository_version` instead of `repsository_verison`)
2. Plugin writers migrate immediately in 3.12
3. Shims are used to allow code to use the old names in most instances (a very small number of queries need to be changed)
4. No changes needed to serializers or other constructs

Compatibility break - plugin versions w/o migrations only compatible with pulpcore < 3.12, plugin versions w/ migrations only compatible with pulpcore > 3.12

At some point in the future, we can introduce a new model to replace BaseDistribution

## Proposal 2 (new fields in the same Master Model, still use deprecation policy)

#### In 3.11:

1. Introduce new fields on `BaseDistribution` (with new names, e.g. `_repository_version` instead of `repository_verison`)
2. Deprecate the `PublicationDistribution`, `RepositoryDistribution` to be removed in, e.g. 3.13 (to give some time to port)
3. Pulpcore needs to have content app use these fields assuming their could be on either the BaseDistribution or the DetailModel for some time
* But how do serializers work? It can't know whether the plugin has migrated or not. Would need to dynamically change the source for the serializer field at runtime based on whether certain fields are present - if that is possible. I briefly looked into it and it didn't appear to be. At least, not without doing hacks we would not absolutely not be willing to do.
* We can't use shims if both can be present. getter/setter recursion problem. getattr() would no longer help w/ determining migraton status because existence of getter impedes inspecting the model using getattr(). Would need to dig into model._meta?
* Content App and other places need to add similar hacks.

#### For plugins using `RepositoryDistribution` in as early as compat with 3.11 or as late as compat with 3.13
4. Plugin writers switch their detail distributions to inherit from `BaseDistribution` directly
* (dalley) I don't have strong feelings about this, since they still need to use different serializers w/ same names. I'd be fine with just leaving them empty. issubclass() could be useful. No need to remain abstract models though, could just be normal class.

## Proposal 3 Add new MasterModel to replace BaseDistribution over time

In 3.12:
1. Create a new MasterModel named `Distribution`. This would have these fields:
* All the existing fields from `BaseDistribution` [see here](https://github.com/pulp/pulpcore/blob/a0fce1f1917f1a8b85b836af8f76c27699d0b04f/pulpcore/app/models/publication.py#L283-L287).
* Include all of the fields from the `PublicationDistribution` and `RepositoryVersionDistribution` models also, this includes the: `publication`, `repository`, and `repository_version` fields

2. Create corresponding Serializer and Viewset. These will be similar to the [`BaseDistributionSerializer`](https://github.com/pulp/pulpcore/blob/c8c4e5bba6f4ee054d5a93c984c2d123c179e761/pulpcore/app/serializers/publication.py#L78) and [`BaseDistributionViewset`](https://github.com/pulp/pulpcore/blob/70d711c52f92b5c068d6b0b32ccc6a923e5c1a7d/pulpcore/app/viewsets/publication.py#L109).

3. Handle the pulpcore code that deals with these detail types to ensure a mixutre of detail inheriting from `BaseDistribution` and `Distribution` can cooperate together.

* base path overlap, the [code here](https://github.com/pulp/pulpcore/blob/c8c4e5bba6f4ee054d5a93c984c2d123c179e761/pulpcore/app/serializers/publication.py#L132-L157) needs to be updated for both models to consider base path overlaps from both models.
* name field uniqueness. We need to have uniqueness of the name apply across both models
* The [content app](https://github.com/pulp/pulpcore/blob/master/pulpcore/content/handler.py#L61) needs to be prepared to read/handle data from both `BaseDistribution` and `Distribution`.

4. Deprecate the `BaseDistribution` model, `BaseDistributionSerializer`, `BaseDistributionViewset`, `PublicationDistribution`, and `RepositoryVersionDistribution`.

In 3.12+ and plugin can port onto this using:

1. Swap their inheritance to using `Distribution` instead of `BaseDistribution`

2. Ship a plugin migration which moves the data from the `BaseDistribution` master table to the `Distribution` master table.

## Breaking change (for plugin writers) announcement

### tl;dr

1. Plugins compatible with 3.11 should *not* declare compatability with 3.12
2. Review the PR for your plugin below [0] and include it in your 3.12 plugin compatbility release

### 3.12 Plugin API Breaking Changes

In Pulpcore 3.12, which the release *after* the upcoming 3.11 release, we will be making a breaking change to Pulpcore which will require some intervention by plugin writers. This does not follow the deprecation policy; we tried very hard to look at other options, but after a lot of discussion this is the least disruptive approach.

All plugins currently released should set their compatibility ceiling at <3.12, and all plugin releases after the change will need to set their compatibility floor to >= 3.12.

### What you need to do

Ensure your 3.11 compatability releases don't declare compatability with 3.12. Many plugins, including File, Container, Certguard, and Ansible already have a support ceiling set for 3.12 (or before). Other plugins such as RPM, Debian, Maven, don't have a support ceiling set. These should be added to pre-3.12-compatible releases.

Then, for your 3.12 plugin compatability releases, review the PR for your plugin below [0].

... PRS ...

Please feel free to reach out to me over email or IRC if you would like assistance with anything I've just discussed.

### PRs [0]

### Why?

When plugin writers create a plugin, they are expected to create a Distribution model which subclasses either PublicationDistribution or RepositoryVersionDistribution imported from pulpcore. Unfortunately, these models were originally defined as "abstract" models, which means that the fields they declare are added to any model which inherits from it, but they are not present in any tables managed by pulpcore.

In normal circumstances this might not be a problem, but Pulp core and plugins are separate and versioned independently, and we want to allow updating individual components as much as we reasonably can. This makes changes to those abstract models and the machinery around them brutally difficult and prone to error at best, and practically impossible at worst.

After encountering a need to adjust these models, we've realized we need to fix this situation.

All of the fields currently present on PublicationDistribution and RepositoryVersionDistribution will be moved to BaseDistribution. This is why we must enforce the version constraints. It is very important that pulpcore and all of the plugins be updated and migrated together in the 3.12 transition.

# Reply to pulp-dev on updated plan

We now have a new plan, which should preserve the deprecation policy at the expense of being a little more work in the short term. It's written into the epic ( https://pulp.plan.io/issues/8383 ) with most of the work happening in 3.12. The timeline is being driven by auto-distribute being needed also in 3.12 and this work blocking that.

With the deprecation policy remaining intact, it is recommended for your plugin releases compatible with 3.11 to also declare compatibility with 3.12.

Please share any questions or concerns you have. Thank you to all the developers who collaborated in adjusting this plan.