Multiple Advisories in one repo

# Multiple Advisories in one repo ## Problem Statement * Advisories are "supposed to be" invariant * MANY repos in the RPM ecosystem "adjust" advisories with Wilde Abandonne * updating text (e.g., description) (Everyone) * adding References (Everyone) * adding packages-affected (occasionally Red Hat, others) * **changing** packages-affected (EPEL) * advisories have a version-field that: * is supposed to be updated when the above happens * it often doesn't * is supposed to be some comparable-version-string * it is seen as "random string" In The Wild * advisories have issue and updated-dates * they can't be relied on either * third-party repos "in the wild" can also have multiple advisories with the same id, in the same repo * pulp_rpm wants there to be one Advisory with a given ID in a given repo-version * pulp_rpm tries to warn the user when advisory-changes "look bad" * the heuristics for the above are "best efforts" * but failure fails the entire sync ### Advisory Parts * id * text * references * list of BZs/CVEs/etc * package-list(s) * packages changed for this advisory * EPEL changes this ALL THE TIME * version * if int or semver, can be compared for "newer" * highly unreliable * sometimes None * sometimes Arbitrary String * often does not change in the face of updated advisory * issue_date * timestamp (or datetime) of when the advisory was released into the repo * unreliable, sometimes None, sometimes changes * when present, can be before, same, or **after** update_date * update_date * timestamp (or datetime) of when the advisory was *changed* * unreliable, sometimes None * digest * checksum of all of the above, to uniquely-identify advisory-object ### What does "newer" mean (currently) * version exists and "can be compared" - larger-int/semver wins * "can be compared" - vers are same, or int, or semver-int-str (e.g., 1.2.3) * in all other cases, choose first-vers-compared * usually the vers in previous-repo-version * IF NO VERSION - update_date exists, can be compared - newer date wins * "can be compared" == "can be handed to parse_datetime() or datetime.fromtimestamp()" * if it can't be parsed, same as None * issued_date CURRENTLY IGNORED ## Current State * attempt to have one instance of a given advisory-id per repo-version * handled at finalize_version() * doesn't know you're in mirror-mode * finalize_new_version() doesn't know its calling-context * repo.last_sync_details is only set when you *sync* - so a mirror-sync to create /versions/5/ followed by a copy() to create vresions/6/ leaves /6/ thinking it was a mirror-sync * overloading last_sync_details to say "resulted from a copy/upload" breaks the use of last_sync_details * NOT knowing something happened after the "last sync", **also** breaks last_sync_details' purpose * ~~PROPOSAL: add a "last_operation" to last_sync_details?~~ * sync/copy involves pulp_rpm code and can be modified * upload/add-or-remove happens in pulpcore, alas * **Ignore this proposal** - being able to rely on it requires some architectural/workflow changes in pulpcore, needs a lot of thought/care to do correctly ## Proposal ~~* recognize mirror-mode-sync, take Whatever Is Incoming * knowing "what happened" at finalize_version() is Problematic * last_sync_details tells us last-sync-policy - but that may **not** be why we're creating a new version "now"~~ * ~~in all not-mirror-sync,~~ continue driving for There Can Be Only One * dups in incoming - **merge them** * dups in prev - merge them, compare result to incoming * Implication: * copy/modify-content/upload cases **will** result in single advisory that may NOT be the uploaded/copied one(s) ## Possible Scenarios ### ~~Mirror_complete, mirror_content_only~~ * ~~Always take whatever is incoming, no matter what~~ * WE DO NOT DO THIS CURRENTLY! ### ~~additive/copy~~ #### No overlap between incoming and previous IDs, no dups * No problem! #### No overlap between incoming and previous IDs, duplicate IDs incoming * CURRENT: FAIL * Proposal: **merge** #### Overlap, one incoming, one previous * CURRENT: merge * Proposal: merge #### Overlap, one incoming, multiple previous * CURRENT: take incoming and ignore previous one(s) * Proposal: um. OK? * Proposal: merge previous, continue to "one incoming, one previous" above #### Overlap, multiple incoming, multiple previous * CURRENT: FAIL due to dup-in-incoming * Proposal: **merge** incoming, continue to "one incoming, multiple previous"