# Pulpcore/katello/pulp_deb integration meeting, 2024 archive

## 2024-12-03 1000-1030 GMT-5
### Attendees: ggainey, quba42, mbucher, dalley, pbrochado, sjha, hstct, qjames, mdellweg
### Regrets:
### Agenda:
* Rolling Content Views
* first draft PR: https://github.com/Katello/katello/pull/11240
* SCA impacted how some pulp_deb users were relying on subscriptions to limit content-access
* open topics:
1. handling of `IndexContent` ("fairly finished")
2. sync or async integration of repo-publish (e.g. in repo-sync)
3.
* lots of ATIX/katello discussion ensues
* Pulpcore 3.70 ("Breaking Changes")
* won't happen before 2nd week of Jan
* plugins need to update migrations
* https://discourse.pulpproject.org/t/action-required-plugins-should-rebase-their-migrations/1814/2
* Expect the openapi-generator images to update
* Django DEFAULT_FILE_STORAGE deprecation: https://github.com/pulp/pulpcore/issues/5404
* <3.85 compat
* Mtg review - shall we continue?
* AI: ggainey to schedule nxt 6 months
### Action Items:
* AI: ggainey to schedule mtg-invite for next 6 months
* ggainey to add minutes to https://discourse.pulpproject.org/t/katello-pulp-community-integration-meeting-minutes/39

## 2024-11-12 1000-1030 GMT-5
### Attendees: ggainey, sjha, hstct, quba42, mbucher
### Regrets: iballou
### Agenda:
* Structured APT next steps:
* https://github.com/candlepin/subscription-manager/pull/3454
* ATIX has its own version
* would be good to have the reviewer revisit
* sjha to poke
* quba42 is working on Katello test coverage
* Roll out planning:
* Will land in Katello 4.15 but disabled by default
* need upstream docs on enabling the feature
* somewhere here? https://docs.theforeman.org/nightly/Managing_Content/index-katello.html#Adding_Custom_Deb_Repositories_content-management
* Enabling by default for a "future" Katello release
* Enable during upgrade in some future Katello release (make it mandatory)
* bug-fixes :smile_cat:
* mbucher: https://github.com/Katello/katello/pull/11212/files#r1834844655
* WIP: https://github.com/Katello/katello/pull/11216
* ATIX working on "rolling content views"
* not yet ready for PR
* candlepin/manifest Fun happening?
* investigation in progress
* old- vs new-code? SCA maybe?

### Action Items:
* ggainey to add minutes to https://discourse.pulpproject.org/t/katello-pulp-community-integration-meeting-minutes/39

## 2024-10-01 1000-1030 GMT-5
### Attendees:
### Regrets:
### Agenda:
* Structured-APT PR is ready for a final review
* Tested on an internal production system by quba42
* Just requires small remaining changes
* Prototyping transparent/rolling/simplified content views starting soon
* New release of pulp-deb would be good before Katello upgrades
* Pain-point: updating pulp-deb Pulp CLI compatability each time Katello packages pulp-deb is cumbersome.
* Potential fix: loosen the boundaries for pulp-deb Pulp CLI compatability. This could mean breaking changes get in, but it also means packaging happens more smoothly and people can get to testing it sooner.
* Previous AIs:
* sjha to schedule a meeting with jeremylenz, sjha, and ATIX (quba42, sbernhard) regarding "transparent CVs" https://projects.theforeman.org/issues/32514
*
### Action Items:
* ggainey to add minutes to https://discourse.pulpproject.org/t/katello-pulp-community-integration-meeting-minutes/39

## 2024-09-18 1000-1030 GMT-5
### Attendees: ggainey, sjha, mdellweg, iballou, markusb, quba42
### Regrets:
### Agenda:
* Previous AIs:
* Getting "Structured APT" ready for the initial merge: https://github.com/Katello/katello/pull/11058
* PR is "feature complete", but still being tweaked
* Currently working on the suggestions by jeremylenz
* ATIX has performed extensive testing of the "feature present but disabled" case (ensures the refactor bits don't break anything):
* Export/Import was tested
* Internal robotello tests were run
* Large syncs general usage

* So long as the "disabled" case does not break anything we can merge, even it the feature itself is not perfect yet. Call it "tech preview", "optional or just don't tell people about it at first.
* When does Katello next branch?
* 2 months-ish from "now"
* next pulp-upgrade will be one-month-ish from now
* prob want this merged pre-pulp-upgrade
* Proposal:
1. Finish the suggestions by jeremylenz
2. Have Bernhard or Markus perform a final code review and wait for their ack.
3. Final chance for core Katello review/request for changes
4. Merge.
5. Keep improving the feature in small follow up PRs.
* net: merge state above with "disabled"; improve "enabled" functionality with new (smaller) PRs

* [markusb] default ContentView with limited Repo visibility or ContentView providing library-instance repos.
* There is now a thread for this: https://community.theforeman.org/t/transparent-content-view-for-limiting-repository-access-in-katello/39389
* This had been possible by limiting subscribed products on an ActivationKey-level (SCA removed that possibility).
* _Scenario:_ a registerd host should
1. must have access to latest synced content
2. must only see a subset of Repos
* net: lots of discussion on associated foreman thread, incl a proposal - is katello happy for ATX to work on implementing this proposal?
* AI: sjha would like to have a meetup w/ ATIX/katello/subscription-DE to flatten some issues
* [quba42] Is the following (satellite) issue also planned for upstream? https://issues.redhat.com/browse/SAT-25398
* Related question: Is red hat bugzilla being replaced?
* Yes - Satellite is on Jira now
* not currently high on the backlog to fix
* needs an upstream-redmine for community-to-work-on
* [markusb] Should creation of IncrementalCV with not-available content (e.g. rpm not in any repository) throw an error-message?
* ATIX hit internally
* katello: prob useful to have an error msg
* [quba42] I think this can be closed: https://projects.theforeman.org/issues/32514
* old ask for "Structured Apt"
* [mdellweg] Inquiry RE foreman (?) PRs from ATIX not getting appropriate/timely attention?
* possibly not an issue - PTO Happens :)
* [quba42] Is Katello planning to somehow apply indices separately from the pulpcore migration? https://community.theforeman.org/t/publish-and-promote-slow-with-3-11-4-13/39040/11?u=sajha
* Answer: This was solved via backport patch in packaging: https://github.com/theforeman/pulpcore-packaging/pull/1144#event-14065751962

### Action Items:
* ggainey to add minutes to https://discourse.pulpproject.org/t/katello-pulp-community-integration-meeting-minutes/39
* sjha to schedule a meeting with jeremylenz, sjha, and ATIX (quba42, sbernhard) regarding "transparent CVs" https://projects.theforeman.org/issues/32514

## 2024-08-06 1000-1030 GMT-5
### Attendees: pbrochado, dosas, dralley, ggainey, mdellweg, iballou
### Regrets:
### Agenda:
* Previous AIs:
* Discussion about deprecating supported pulpcore branches.
* example: https://github.com/pulp/plugin_template/blob/main/scripts/update_core_lowerbound.py#L35
* (ggainey add example-scenario here)
* we do semver for API, but *not* for plugin API
* we have the deprecation-policy for plugin-api instead
* we do want to make sure the lowerbounds-declarations are still valid
* we want to maintain the broadest range (of dependency-requirements in the plugins) that we feel confident Actually Work
* core gets rapid y-version-updates
* and we can't support all of them forever
* so we have supported-branches for our downstream stakeholders
* Sooo...removing a branch from supported-branches, should also impact all plugin-branches that currently require that branch
* ie, any plugin-lowerbound that specifies (or is older-than) a now-unsupported branch, can now be updated to match the now-lowest-supported core-branch
* if a supported-plugin-branch "breaks" as a result of a core-branch becomeing unsupported - then the **implication** is, that branch of the plugin is actually **also unsupported**
* AI: [ggainey] can we get ocharino into a product-stakeholder-requires-versions matrix
* AI: [dosas] bring this discussion to the Rest Of ATIX as well

### Action Items:
* ggainey to add minutes to https://discourse.pulpproject.org/t/katello-pulp-community-integration-meeting-minutes/39

## 2024-07-02 1000-1030 GMT-5
### Attendees: quba42, iballou, sjha, dalley, dosas, hstct, ggainey
### Regrets:
### Agenda:
* quba42: Thanks for the review and merge!: https://github.com/Katello/katello/pull/11044
* quba42: Forklift questions/issues:
* `almalinux8-katello-devel-stable` still using defunct CentOS 7 Stream repo files => broken untill next rebuild?
* may have to purge/delete-image and then box-update maybe?
* will still be on centos8-stream tho (which will still be broken?)
* I built `centos9-katello-devel` successfully, but:
* Candlepin is creating `cp_contents` instead of `cp2_content`?! Is this normal?! Never seen this before on a production system. (but things seem to be working?)
* guess: cp2-content is Very Old (maybe?) - and decision was to use cp_contents?
* check in w/ jlenz for clarity
* I can't build `almalinux8-katello-devel` possibly because my host system Ansible is to new? Fails with:

     msg: |-
       The following modules failed to execute: ansible.legacy.setup

* iballou has hit something like this - check for multiple ansible-installs (ie system- vs pip-installed)
* quba42: Structured APT: I have a partially working state I would like to present: https://github.com/Katello/katello/pull/11058
* review/demo/comments/discussion ensue
* maybe get feedback from jlenz/paji/toledo(Chris Roberts) RE candlepin side of this
* work w/ iballou RE getting QE-test suite running against
* katello does rely on candle-in-url to look up Things
* is there a migration involved for this?
* non-deb: no
* deb-content: there is a rake-task to enable structured-app feature
* rake-task still a work-in-progress
* when the feature is selected/rake-task run, rake task does the data update
* run during upgrade?
* open for alternate proposals
* The review-ask is "Is there anything that look Really Bad with this approach?"
* katello upgrading to pulp_rpm/3.26 "now"
* checkin: is this meeting format/frequency/approach still useful?
* consensus: let's keep going!
### Action Items:
* ggainey to add minutes to https://discourse.pulpproject.org/t/katello-pulp-community-integration-meeting-minutes/39

## 2024-06-04 1000-1030 GMT-5
### Attendees: hstct, iballou, ggainey, pbrochado, sjha, dalley, vsedmik, quba42
### Regrets:
### Agenda:
* Previous AIs:
* katello is moving to Jira for reporting!
* pulpcore/3.55 is coming!
* https://github.com/pulp/pulpcore/milestone/10
* In Katello content can be "seen" at `URL/pulp/repos/`, but also at `URL/pulp/content/` and by content type for example at `URL/pulp/deb/` (does not appear to show just deb content).
* Are any of these the "preferred" URL?
* Are there any plans to ever deprecate/remove any of these duplicate locations?
* discussion ensues
* /pulp/content/ is the Correct Way
* /pulp/repos/ exists for *very* backward compat (ie, to Pulp2)
* currently no plans to do so
* see Apache-configs for how these are proxied

        <Location "/pulp/repos">
        RequestHeader unset X-CLIENT-CERT
        RequestHeader set X-CLIENT-CERT "%{SSL_CLIENT_CERT}s" env=SSL_CLIENT_CERT
        ProxyPass unix:///run/pulpcore-content.sock|http://pulpcore-content/pulp/content disablereuse=on timeout=600
        ProxyPassReverse unix:///run/pulpcore-content.sock|http://pulpcore-content/pulp/content
      </Location>

* Slow smart proxy sync in 4.11
* We ran into this issue as well, any ideas how one could reproduce it?
* https://community.theforeman.org/t/very-slow-proxy-sync-after-upgrade-to-foreman-3-9-and-katello-4-11-1/
* has anyone any ideas here?
* see how long update-tasks is taking
* if on user-env - how long do API-queries take between foreman-server and smartproxy
* are we bogging down dynaflow workers?
* katello is investigating update-content-counts issue for katello/4.11
* current workaround is "turn off content-counts-task"
* https://issues.redhat.com/browse/SAT-25503
* Are there plans to "containerize smart proxies" (we heard rumblings somewhere)?
* "plans" is *such* a strong word :)
* maybe ask ehelms?
* (If there is time) Structured APT update/challenges
* progress is happening
* candlepin-content-id saved in root-repository-model when creating a new repo
* with structured-apt, need more candlepin-content-ids per repo-model
* added to katello-repo-model
* current PR makes sure non-apt-content doesn't notice this change
* discussion ensues RE ruby-dev-tools
* Pulp discourse discussion RE bindings generation
* https://discourse.pulpproject.org/t/we-will-stop-publishing-bindings-soon/1240/
* ATIX generates their own for their product
* don't forget, foreman-bday-party is coming!

### Action Items:
* ggainey to add minutes to https://discourse.pulpproject.org/t/katello-pulp-community-integration-meeting-minutes/39

## 2024-05-07 1000-1030 GMT-5
### Attendees: ggainey, dosas, jbundesmann, mbucher, quba42, iballou, sjha, pbrochado, dalley, vsedmik, ssingh
### Regrets:
### Agenda:
* Katello and robottelo
* We are experimenting with robottelo tests
* Since robottelo is owned by SatelliteQE, we wanted to ask (to the extent you can/want to tell us) how Katello and robottelo development interacts?
* Who adds new tests?
* Who fixes them if a Katello change breaks them?
* If we think a test is broken how best to contribute a fix?
* Who are the best contacts for questions?
* See also:
* https://github.com/SatelliteQE/robottelo/pull/14608
* We think this test is passing even though it should be failing: https://github.com/SatelliteQE/robottelo/blob/master/tests/foreman/api/test_repository.py#L521
The actual error we see is `The repository's publication is missing. Please run a 'complete sync' ` which is not the error being tested for, but any error will satisfy the test.
* iballou:
* dev/qe work very closely together
* QE makes the required changes/automation in robotello as features/bugs are being addressed by dev
* long-standing backlog in robotello of "things to fix", that QE is working their way through
* process is being improved, but has more work to do
* bandwidth is *always* an issue (as it is for everyone, everywhere, forever)
* ATIX appreciates having someone to ping just to verify something is actually a bug/problem before opening a PR?
* https://github.com/Katello/katello/pull/10942
* How to add better test coverage here?
* maybe merge this fix, but plan to add some more tests around these odd edge-cases
* iballou:
* good to accept PR as-is
* happy to do a peer-coding-exercise on how to add more tests, or rubber-duck on a PR
* Are there plans to use the Pulp replicate features (for smart proxies or otherwise)?
* some conversations happened at CfgMgt camp
* sjha:
* not on the roadmap yet
* there are some katello-side constraints to be considered
* still considered desireable
* quba42: please remember to ping if/when/as we decide to start implementing
* iballou:
* N-2 syncing is higher prio
* *maybe* can take advantage of replication?
* pulp-glue?
* proxies shipping their own bindings?
* Does Katello have any special plans to handle [this](https://discourse.pulpproject.org/t/starting-the-migration-countdown-with-a-new-release-of-the-pulp-container-plugin/1217)?
* sjha:
* handled in foreman-maintain: https://github.com/theforeman/foreman_maintain/pull/829
* iballou
* for upstream, docs say "do This"
* needs to be indexed in katello
* ggainey
* the goal is to not-require-multiple-upgrades to go from X.Y to X.Y+N
* if you jump from pre-that-change, to post-container-dropped-old-codepaths, then your upgrade will pay the (long) price as part of the migration
* structured-apt-work is still in-progress
* ATIX getting up to speed on "becoming a katello dev" :)
* Fun!
* actions often missing the "why" of actions
* iballou: technical debt is...A Thing :(
* iballou: absolutely up for getting together for a "community design review"
* maybe record and get it up on katello's youtube channel
### Action Items:
* ggainey to add minutes to https://discourse.pulpproject.org/t/katello-pulp-community-integration-meeting-minutes/39

## 2024-04-02 1000-1030 GMT-5
### Attendees: quba42, iballou, dalley, sjha, ggainey
### Regrets:
### Agenda:
* Previous AIs:
* ~~Apparently it is not possible to "request a review" from arbitrary people in GitHub (perhaps this requires merge access? or maybe you can't do it if there already are two reviewers on the PR?), so we did a normal ping instead: https://github.com/Katello/katello/pull/10905/#issuecomment-1980695028~~
* Was merged.
* quba42: Who is the expert on js/UI bugs? https://projects.theforeman.org/issues/37240
* Was merged.
* You can always ping sjha or jeremylenz
* Forklift, lived practice questions (quba42):
* Which boxes do you use for development? (centos8-katello-devel, centos8-katello-devel-stable, centos8-katello-nightly-stable, ...? So many to choose from...)
* spread of who-uses-what on katello
* lots of folks use just the -stable boxes
* -stable has some limitations
* DNS subtleties
* tab-complete doesn't work?
* iballou uses centos8 (soon centos9)
* DON'T FORGET TO UPDATE YOUR BASE IMAGE!
* Do you upgrade your active box, or just vagrant destroy and re-create it?
* "It Depends"
* `bundle exec rails katello:reset`
* How long do you tend to keep your boxes?
* "It Depends"
* I keep randomly getting `An error occurred saving the Repository: uninitialized constant ForemanTasks::TimeoutError` only for things to succeed if I take the same action again?!
* mismatch between "UI expecting sync-task when backend has started an async task"
* sounds like a bug - look for an issue, and open one if not already existing?
* repository-pages are being converted from Angular and prob won't take non-critical issues right now
* Inconsistent KatelloRepositories
* pointing to a pulp-repo-version and a pulp-distribution that aren't in the same repository?!?
* CCV with 2 CVs that are the same repo
* container-repo forbids this!
* can't reproduce on current katello (happens on katello/4.7)
* only see it with debian content
* structured-apt story
* katello-library starts "empty"
* works for rpm, **but not for deb**
* needs some empty-content for publishing to even work
* but we **don't** want that when we copy into CVs
* how can we tell the difference between these two cases?
* is there any way to know using current/existing data?
* do we need to add a new flag to the related foreman-task-invocation?
* at what point would we punt and create a completely-new-action to handle specific usecase?
* next katello uses new pulpcore!
* if there's a new version of pulp_deb coming, there's a small window still to ask for a newer version
* waiting on 3.25.2 of pulp_rpm
* button being pushed

### Action Items:
* ggainey to add minutes to https://discourse.pulpproject.org/t/katello-pulp-community-integration-meeting-minutes/39

## 2024-03-05 1000-1030 GMT-5
### Attendees: ggainey, iballou, sjha, sweta, quba42, hstct
### Regrets:
### Agenda:
* Previous AIs: none?
* quba42: I would like to learn to better understand these posts: https://community.theforeman.org/t/katello-4-11-1-release-process/36915
* how to go from this, to "how long should I wait to rely on the release-rpms being available?"
* hard to give a solid "how much time left"
* ping release-owner/engineer for status if/when/as needed
* quba42: How to add a new RPM?: https://github.com/theforeman/pulpcore-packaging/pull/881
* need help on how to generate the initial .spec file for a new package
* is there a good tool?
* pulp-cli-deb is currently blocked waiting on pulp-glue-deb
* prob should wait till odilon is less-busy to have a conversation
* quba42: ATIX Katello PRs near completion:
* Katello fix ready for final review: https://github.com/Katello/katello/pull/10905
* Looks like final review is already requested, but I add it for completeness: https://github.com/Katello/katello/pull/10900
* one can use the github request-reviewer/re-review tools to get some attention if/when/as needed
* requesting new core/3.48 for new katello
* updating to new pulp_deb as well
* core/3.49 coming out "today", includes https://github.com/pulp/pulpcore/issues/5086
* can we tighten the window between katello-announce and build-accepts?
* there is always going to be some window
* iballou to investigate maybe taking 3.49 instead
* pulp-glue discussion happening in katello-land
* change smart-proxy talk to pulp instead of katello
* would solve the N-2 problem
* discussion continues
* core/3.55 is coming!
* discussion around pulp eventually going to retrieve-when-content-exists
* should there be pressure on plugins to change this behavior to the default?

### Action Items:
* ggainey to add minutes to https://discourse.pulpproject.org/t/katello-pulp-community-integration-meeting-minutes/39

## 2024-02-13 1000-1030 GMT-5
### Attendees: ggainey, mbucher, sja, iballou, quba42, hstct
### Regrets:
### Agenda:
* Previous AIs:
* ~~ggainey to archive 2023 minutes~~
* ~~https://hackmd.io/@pulp/pulp-deb-katello-integration_2023~~
* meeting in person at conference was Really Useful!
* fun with Forklift - lots of fixing has happened!
* discussion between ATIX/katello
* discussion around katello/UI bug in 4.11
* needs an upstream issue opened?
* discussing next-pulp-version for katello/4.13
* prob whatever the latest-Y is at branch-point
* would be good if a pulp_deb-Y right before katello-branch so katello gets latest
* pulpcore discussed "supported branches" this week
* discussion ensues
* how can we get katello reviews happening more quickly on pulp-deb/ATIX Stuff?
* can we get ATIX approve-access to the affected repos?
* sure - there is A Process, let's do it!
* Process: https://theforeman.org/handbook.html#Becomingacommitter
* iballou to bring up w/ katello team
* ATIX needs a candidate to start doing regular katello-reviews?
* work w/ iballou/sja to iron out details
* ATIX customer is "losing publications"?!?
* heavy CVV user
* old CVV "loses" pub?
* ATIX investigating - rake cleanup task?
* how can a repo-version exist, but associated publications are gone?!?
* check if pulp was told to delete?
* katello/4.7 (we think) is what's in use
* might be a related fix, in a newer katello?
* sja: there exists a PR to "freeze" repo-access while a publication is happening

### Action Items:
* ggainey to add minutes to https://discourse.pulpproject.org/t/katello-pulp-community-integration-meeting-minutes/39

## 2024-01-09 1000-1030 GMT-5
### Attendees: bsuttner, quba42, iballou, ggainey, dalley, pbrochado, sweta
### Regrets:
### Agenda:
* Previous AIs:
* ~~ggainey: schedule mtgs for 2024~~
* ~~iballou: summarize discussion RE global-repair into PR~~
* ~~iballou: Poke People to get other katello PRs reviewed~~
* hstct to report back on customer experience once RemoteArtifact indices applied
* no answers yet
* Are we looking into [this](https://community.theforeman.org/t/proxy-sync-10x-slower-with-foreman-3-8-katello-4-10/35887/7)?
* looks like "yes" based on discussion there
* seems to be improving for reporting-user
* appears to be resolved now
* I can't sync Python on Katello 4.7 (`python39-pulp-python-3.7.3`)
* There is a successfull sync task, but no actual content is synced.
* The Pulp remote being used looks exactly like the example in the pulp_python docs.
* I could be missing something incredibly obvious since I have never used pulp_python before.
* might be an old katello bug getting in the way?
* maybe retry on 4.11
* https://community.theforeman.org/t/storing-secure-information/36195
* katello issue
* secure-storage for things like ssh/gpg keys/uname-pwd
* could/should this be a new feature for foreman?
* is there/what is the "standard"/existing way foreman stores this kind of info?
* feedback/discussion should continue on the thread above
* ATIX Katello PRs:
* Structured upload: https://github.com/Katello/katello/pull/10639
* sjha out till end of Jan
* needs a new katello person to do the merge
* Fix deb copy API: https://github.com/Katello/katello/pull/10734
* needs review
* Export/Import for deb: https://github.com/Katello/katello/pull/10749
* needs review/merge
* Requires: https://github.com/theforeman/pulpcore-packaging/pull/819
* merged, available
* Host details tab for deb: https://github.com/Katello/katello/pull/10744
* actively in review/discussion
* Incremental update concurrency: https://github.com/Katello/katello/pull/10781
* already on katello-review-list
* Heal broken publications on sync: https://github.com/Katello/katello/pull/10776
* merged

### Action Items:
* ggainey to add minutes to https://discourse.pulpproject.org/t/katello-pulp-community-integration-meeting-minutes/39
* ggainey to archive 2023 minutes

### Action Items:
* AI: [ggainey] schedule 6 months starting first tues in March
* AI: [ggainey] clean up mtg-doc
* ggainey to add minutes to https://discourse.pulpproject.org/t/katello-pulp-community-integration-meeting-minutes/39