# 2021 notes, pulp_rpm team mtg

## 2021

### December 9, 2021

Pulp 3:

* 3.14.9, 3.16.2 releases
* https://github.com/pulp/pulp_rpm/pull/2191
* https://github.com/pulp/pulp_rpm/pull/2192
* What else should go in?
* https://github.com/pulp/pulp_rpm/pull/2185 probably
* https://github.com/pulp/pulp_rpm/pull/2179
* Dist tree uniqueness constraint
* https://pulp.plan.io/issues/9583
* Should it block rpm 3.17.0?
* Yes - and we should move ACS to 3.18.0 - it can be a short release cycle
* Who has capacity to work on it?
* Tanya or Pavel
* EL9 content support - testing in progress
* "nothing obvious is broken"
* createrepo_c upstream work w/ parsing APIs
* making steady progress, but currently rife with segfaults

Pulp 2:

* FIPS issue in 2.21.5 : https://bugzilla.redhat.com/show_bug.cgi?id=2019563
* ggainey needs to submit a PR w/ patch before shutdown

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

3-month planning (every 1.5 months):

### October 21, 2021

Pulp 3:
* 3.16.0 released yesterday
* comps.xml POC coming next week
* sync-optimization and distributiontrees and mirror-sync, oh my!
* needs to be in 3.16.1 for katello

Pulp 2:

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

3-month planning (every 1.5 months):
### October 14, 2021

No meeting, hackfest

### October 7, 2021

Pulp 3:
* https://pulp.plan.io/issues/9398

Pulp 2:

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

3-month planning (every 1.5 months):

### September 23, 2021

Pulp 3:
* https://pulp.plan.io/issues/9399

Pulp 2:

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

3-month planning (every 1.5 months):

### September 16, 2021

Pulp 3:
* Will try to fix mirrorlist PR - then do a release

Pulp 2:

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

### September 2, 2021

Pulp 3:
* GGainey found a potential fix for 3 of the dependency solving BZs (out of 4 total I believe)

Pulp 2:
* Should we help out with the metadata signing fixes

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

CI status check
* https://github.com/pulp/pulp_rpm/actions?query=workflow%3A%22Pulp+Nightly+CI%2FCD%22

### August 26, 2021

Pulp 3:
* 3.15 compatibility release
* We still need: https://github.com/pulp/pulp_rpm/pull/2081
* Need to debug
* ACS support work should starts this sprint
* Bring up at pulpcore team meeting?
* https://pulp.plan.io/issues/9213?pn=1#note-7
* https://bugzilla.redhat.com/show_bug.cgi?id=1993773
* user reported that it didn't seem like retries worked if total timeout was hit - is this related?

Pulp 2:
*

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

CI status check
* https://github.com/pulp/pulp_rpm/actions?query=workflow%3A%22Pulp+Nightly+CI%2FCD%22

### August 19, 2021

Pulp 3:
* depsolving fun
* (see issues above)
* ggainey's proposal is:
* none of these should be beta-blockers
* note for the beta "don't use dep-solving-on-publish, we know there are problems and are working on them for release"

Pulp 2:
*

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

### August 05, 2021

Pulp 3:
* discussions started with rpm software management about switching to using libdnf for dependency solving over a long-term timeframe, monthly meeting set up

Pulp 2:
* repo-metadata-signing issues from EXD need attention
* #soon

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

CI status check
* https://github.com/pulp/pulp_rpm/actions?query=workflow%3A%22Pulp+Nightly+CI%2FCD%22

### July 29, 2021

Pulp 3:
* Updates on the downstream priorities and process
* DistributionTree metadata issues
* How do clients (e.g. anaconda) treat this metadata
* Ask Dennis?
* Should we re-write repository paths at sync time
* It's reasonable, should possibly wait until we're not trying to support the migration plugin anymore.
* Should we add the "main variant" to the schema
* Potentially
* Variants have a link to the repository, it's just not exposed via metadata

Pulp 2:
*

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

CI status check
* https://github.com/pulp/pulp_rpm/actions?query=workflow%3A%22Pulp+Nightly+CI%2FCD%22

---------------------------------------------------
### July 22, 2021

Pulp 3:
* 3.14.0
* Relative path problem progress
* there is some - PR "#soon"
* need to touch base w/ daviddavis RE orphan-cleanup changes
* Repairing incorrect metadata (9107)
* pulp_rpm 3.13.0+, using python<3.8 affected
* discussed w/ jsherrill RE upstream-users
* get the fix out first, then figure out how to address data-repair issues
* also see https://pulp.plan.io/issues/9128
* Any depsolving updates? No
* from QE: if you think of an end-user visible problem, talk w/ QE about how to build a test to notice it
* do we have any complex-RPM-tests (dependencies, large filelists, etc)?
* discussion about signed-metadata fun

Pulp 2:
*

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

CI status check
* https://github.com/pulp/pulp_rpm/actions?query=workflow%3A"Pulp+Nightly+CI%2FCD"

---------------------------------------------------
### July 15, 2021

Pulp 3:
* Start a discussion with DNF team about how to validate metadata / package signatures, maybe implement it in createrepo_c?
* Relative path problem progress
* there is some - PR "#soon"

Pulp 2:
*

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

CI status check
* https://github.com/pulp/pulp_rpm/actions?query=workflow%3A"Pulp+Nightly+CI%2FCD"

### July 1, 2021

Pulp 3:
* RBAC
* ggainey to gather user-requests from mailing-lists
* dkliban will contact Large User about their use-cases
* ACS
* prob just "a few days" for pulp_rpm support
* bugzillas
* prob many patch-releases for rpm/3.13
* ruby binding issue
* https://bugzilla.redhat.com/show_bug.cgi?id=1964797
* no upstream issue yet

Pulp 2:
* https://github.com/pulp/pulp_rpm/pull/2017 EXD test suite fails because of these changes, expect an issue to be filed
* https://github.com/pulp/pulp_rpm/pull/2031 Testing in-progress, in particular w/ migration plugin

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

CI status check
* https://github.com/pulp/pulp_rpm/actions?query=workflow%3A%22Pulp+Nightly+CI%2FCD%22

### June 24, 2021

Pulp 3:
*

Pulp 2:
*

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

CI status check
* https://github.com/pulp/pulp_rpm/actions?query=workflow%3A%22Pulp+Nightly+CI%2FCD%22

### June 17, 2021

#### What is everyone working on thi ssprint (sprint):

Pulp 3:
* Brainstorm & document RBAC use cases
* RBAC mini-team needs to validate if the current/proposed approach to manage permissions fits RPM plugin use cases

Pulp 2:

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

CI status check
* https://github.com/pulp/pulp_rpm/actions?query=workflow%3A%22Pulp+Nightly+CI%2FCD%22

### June 10, 2021

Pulp 3:
* Retry support?
* According to ipanova, the issues that have been experienced with the CDN are down to the CDN provider, and unlikely to be resolved soon. So Pulp needs to support retries to work around these issues.

Pulp 2:
* https://bugzilla.redhat.com/show_bug.cgi?id=1959412

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

CI status check
* https://github.com/pulp/pulp_rpm/actions?query=workflow%3A%22Pulp+Nightly+CI%2FCD%22

### May 20, 2021

Pulp 3:
* Need to shut down https://pulp-rpm.readthedocs.io/
* it's out of date
* it's more googleable and thus brings users there instead of https://docs.pulpproject.org/pulp_rpm/
* maybe redirect users to the new website

Pulp 2:

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

CI status check
* https://github.com/pulp/pulp_rpm/actions?query=workflow%3A%22Pulp+Nightly+CI%2FCD%22

### May 13, 2021

Pulp 3:
* Fixtures broken on F31+ https://pulp.plan.io/issues/8016
* Need to get off F30. EOL was 5/2020 and dev envs are running F31+
* An alternative would be to skip generating these fixtures
* Would mean that fixtures.pulpproject.org/pulp-fixtures container would no longer carry them
* comps.xml types are mergeable \o/
* https://pulp.plan.io/issues/8728
* release madness this week

Pulp 2:

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

CI status check
* https://github.com/pulp/pulp_rpm/actions?query=workflow%3A%22Pulp+Nightly+CI%2FCD%22

---------------------------------------------------

### May 6, 2021

Pulp 3:
* Breaking change in CentOS Stream
* "modules" metadata changed from gz compression to xz compression
* https://pulp.plan.io/issues/8700
* Definitely need 3.10 backport, 3.9?
* https://pulp.plan.io/issues/8710
* let's not backport until we know that the change was intentional
* ask CentOS Stream community whether this was an intentional change
* should we try to get advance notice of such changes? perhaps mailing list to subscribe to?
* +1
*

Pulp 2:
*

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

CI status check
* https://github.com/pulp/pulp_rpm/actions?query=workflow%3A%22Pulp+Nightly+CI%2FCD%22

---------------------------------------------------

### April 29, 2021

Pulp 3:
* pulp3 to pulp 2 sync issue with appstream repo (so far)
* https://bugzilla.redhat.com/show_bug.cgi?id=1954839

Pulp 2:

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

CI status check
* https://github.com/pulp/pulp_rpm/actions?query=workflow%3A%22Pulp+Nightly+CI%2FCD%22

### April 22, 2021

Pulp 3:
* The BZ is for Pulp 2 but we have the same issue with kickstart repos in Pulp 3
* https://bugzilla.redhat.com/show_bug.cgi?id=1938233
* check Pulp3 behavior w/RHEL7-Z sync
* do both treeinfo/.treeinfo exist in remote?
* do we preserve both if so?
* do we rename the file?
* open redmine for this to support in Pulp3 if needed
* static_context in modular metadata before compatibility release with 3.12/3.13?
* depends on the outcome of a meeting about the need for the basic support of static_context in Pulp2
* repomd.xml might be regenerated on a regular basis for the sake of refreshing timestamps and signatures
* if a repo doesn't change much, this can lead to some syncs being operational because we calculate and check the checksum of repomd.xml
* do we want to adjust the logic for our sync optimizations, to take such behaviour into account?

Pulp 2:

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

CI status check
* https://github.com/pulp/pulp_rpm/actions?query=workflow%3A%22Pulp+Nightly+CI%2FCD%22

### April 8, 2021

Pulp 3:
* No progress yet on advisory conflicts
* AI: ggainey to send out note, schedule mtg to finalize "what we gonna do?!?" RE advisory-merge issues
* Daniel (briefly)investigating memory usage issues being reported by downstream + user
* https://pulp.plan.io/issues/8295 disc usage during sync

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

CI status check
* https://github.com/pulp/pulp_rpm/actions?

### March 18, 2021

Pulp 3:
* BinLi's artifact-checksum-mimatch issue : https://pulp.plan.io/issues/8411
* update w/mailing-list discussion
* migration-of-centos8 problem : https://pulp.plan.io/issues/8337

Pulp 2:

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

CI status check
* https://github.com/pulp/pulp_rpm/actions?query=workflow%3A%22Pulp+Nightly+CI%2FCD%22

### March 11, 2021

Pulp 3:
* errata issue work continues
* see doc for details
* https://hackmd.io/ik50Rb6VTtWf8gB1YpIrUA?both#ttereshc-version
* [8229](https://pulp.plan.io/issues/8229) - EPEL resets advisory updated_date **every day**, so every sync recreates every advisory
* dalley submitted [PR#4190](https://github.com/fedora-infra/bodhi/pull/4190) to bodhi
* 3.9.1 release
* cap pulpcore to <3.12
* 3.10 release

Pulp 2:
* Review https://github.com/pulp/pulp/pull/4020/
* bring up at next pulpcore mtg
* pre-req for pulp2 pulp_rpm PR [1942](https://github.com/pulp/pulp_rpm/pull/1942)

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

CI status check
* https://github.com/pulp/pulp_rpm/actions?query=workflow%3A%22Pulp+Nightly+CI%2FCD%22

### March 4, 2021

Pulp 3:
* errata issue work continues
* comments on https://pulp.plan.io/issues/8229 ?
* Need review of ttereshc's advisory-conflict-resolution proposal
* https://hackmd.io/ik50Rb6VTtWf8gB1YpIrUA?both#ttereshc-version

Pulp 2:
* Review https://github.com/pulp/pulp/pull/4020/
* review has happened
* not ready to merge upstream, waiting on cross-team-conversations
* pre-req for pulp2 pulp_rpm PR [1942](https://github.com/pulp/pulp_rpm/pull/1942)

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

CI status check
* https://github.com/pulp/pulp_rpm/actions?query=workflow%3A%22Pulp+Nightly+CI%2FCD%22
* ggainey updating plugin_template-related changes for CI

### February 25, 2021

Pulp 3:
* Grant working on errata bugs, neeed in ~1 week

Pulp 2:
* Review https://github.com/pulp/pulp/pull/4020/

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

CI status check
* https://github.com/pulp/pulp_rpm/actions?query=workflow%3A%22Pulp+Nightly+CI%2FCD%22

### February 4, 2021

Other:
* Grant Pavel maintainer privileges on the repo

Pulp 3:
* Should we add EOL support to the RPM plugin wishlist? It's in F34.
* https://fedoraproject.org/wiki/Changes/Module_Obsoletes_and_EOL
* Downstream doesn't need it but upstream users might want it
* AI : ttereshc to open ticket

Pulp 2:

Open PRs:
* https://github.com/pulp/pulp_rpm/pulls

Un-triaged bugs:
* https://pulp.plan.io/projects/pulp_rpm/issues?query_id=159

Last week stories and tasks:
* https://bit.ly/3dCTomw

CI status check
* https://github.com/pulp/pulp_rpm/actions?query=workflow%3A%22Pulp+Nightly+CI%2FCD%22

## IRC RBAC DISCUSSIONS
* freenote/#pulp:
* 2020:
~~~
Apr 15 13:37:28 <dkliban> it works :)
Apr 15 13:37:32 <dkliban> but yes, it's not the best
Apr 15 13:37:42 <dkliban> you do have other options such as nfs
Apr 15 13:37:56 <lieter> a virutalenv with pip install -e . should go a long way I guess?
Apr 15 13:38:10 <lieter> with a supervisord config
Apr 15 13:38:34 <dkliban> you can use the pulp_installer (formerly known as ansible-pulp) to do a source install
Apr 15 13:38:45 <dkliban> that will also work for setting up a dev environment
Apr 15 13:39:08 <dkliban> lieter: did you use pulp_installer for installing pulp?
Apr 15 13:39:26 <lieter> yes
Apr 15 13:40:31 <dkliban> that's what we use inside the vagrant box that is spun up using pulplift. you can just run it yourself and point to the local clone of the pulp_rpm repository (since that's the plugin you are interested in working on)
Apr 15 13:51:20 <lieter> ah
Apr 15 13:51:24 <lieter> I'll have a look tomorrow
Apr 15 13:51:33 <lieter> another question, can I add more users to the API?
Apr 15 13:51:55 <dkliban> lieter: not yet, but we are in the planning phase of that now
Apr 15 13:52:00 <lieter> perhaps even with access restriction based on the names of the distributions and publications?
Apr 15 13:52:04 <lieter> ack
Apr 15 13:52:17 <lieter> this is all internal stuff, so I can live with password sharing for a short time
Apr 15 13:52:28 <dkliban> yeah ... we are working on adding RBAC
Apr 15 13:52:34 <lieter> yes please!
Apr 15 13:52:43 <dkliban> everyone wants it
Apr 15 13:53:04 <lieter> I'd like to have multiple independent CI runners to upload their packages etc
Apr 15 13:57:48 <lieter> ah, I now know why the local stuff did not work
Apr 15 13:57:55 <lieter> a lack of libmodulemd and zchunk
Apr 15 14:53:24 <lieter> neat, I have a dev env with supervisord
Apr 15 14:54:44 <dkliban> lieter: that's great
Apr 15 14:58:23 <dkliban> lieter: you want to discuss how to implement this feature?
Apr 15 14:59:20 <lieter> let's do tha ttomorrow, I need to do some other things now
Apr 15 14:59:38 <dkliban> lieter: sounds good. i'll add my idea to the issue itself
Apr 15 14:59:43 <lieter> but it looks like pulp_rpm/app/tasks/publishing.py the 'publish' function would be such a place
Apr 15 14:59:57 <dkliban> lieter: yes, i think so
Apr 15 15:01:06 <dkliban> lieter: i was actually thinking that we would generate it dynamically each time it is requeted
Apr 15 15:05:10 <dkliban> since we need to know the URL of the distribution serving the file
Apr 15 15:05:42 <dkliban> the other option is to specify the URL when creating the publication
Apr 15 15:37:44 <dkliban> lieter: https://pulp.plan.io/issues/5356#note-3

May 07 16:09:16 <phlogistonjohn> I didn't think to check for this before I started testing out pulp (3) but it appears that (podman|docker) push is not supported
May 07 16:09:31 <phlogistonjohn> (for the conteiner plugin of course)
May 07 16:09:40 <phlogistonjohn> Does anyone know if this is planned? I tried searching the plan.io system but found nothing that seemed relevant?
May 07 16:09:48 <phlogistonjohn> I want to cache both rpms and containers at home... I could run a seperate registry but was hoping to have one system to rule them all if you will :-)
May 07 16:14:04 <dkliban> phlogistonjohn: yes this is definitely planned
May 07 16:15:43 <dkliban> phlogistonjohn: https://pulp.plan.io/issues/5027
May 07 16:15:54 <phlogistonjohn> dkliban: ah, good to hear!
May 07 16:16:09 <dkliban> phlogistonjohn: i have actually written code to add the feature, but we are waiting to add it until we have RBAC
May 07 16:16:26 <phlogistonjohn> yeesh. I swear I searched for "push" "podman push" and "docker push" in pulp.plan.io
May 07 16:16:46 <phlogistonjohn> dkliban: ah, that makes a lot of sense :-)
May 07 16:16:48 <dkliban> and RBAC planning is under way. i am hoping we'll be able to have RBAC support in 3 months
May 07 16:17:39 <dkliban> and once we have RBAC, we'll be able to reopen this PR https://github.com/pulp/pulp_docker/pull/372/files
May 07 16:18:12 <dkliban> though in the pulp_container repository (we renamed the pulp 3 implementation of the docker plugin to container)
May 07 16:19:01 <dkliban> phlogistonjohn: the latest feature we added to the container plugin allows you to upload a Containerfile/Dockerfile and have pulp build that container image for you
May 07 16:20:14 <phlogistonjohn> dkliban: thank you. I saw that too. In the meantime I could see if my workflows are amenable to that approach
May 07 16:21:04 <phlogistonjohn> currently I always build locally and either push ot a registry or copy tarballs to my test nodes and load them
May 07 16:22:05 <phlogistonjohn> I need to start working more with a certain large container orchestration system (version 4) that likes to pull everything in as containers and hoped to be able to do both caching of those images and uploading my dev work
May 07 16:22:27 <phlogistonjohn> on a single system
May 07 16:22:50 <phlogistonjohn> I'll keep an eye on things in pulp land.... I do like the way the system is put together in general
May 07 16:23:01 <dkliban> thank you
May 07 16:23:49 <phlogistonjohn> thank you! :-)

May 28 15:05:22 <cognifloyd> Next question:
May 28 15:08:43 <cognifloyd> Once I get the basic pulp set up, I'll be looking at building a pulp 3 plugin for a file-like artifact I have to deal with that has some annoying encryption requirements. ie The artifact should be encrypted in the django-storages backend, and pulp must not have the key to decrypt it. Clients will be given a key to decrypt those artifacts. Has anything like this been done? I think a plain file repo would work, but I'm wondering if pulp
May 28 15:08:43 <cognifloyd> d need special support since these would be encrypted.
May 28 15:26:00 <bmbouter> cognifloyd: we don't have have sizing recommendations unfortunately, but I can give some anecdotal info
May 28 15:26:43 <bmbouter> cpu count should equal the number of pulp workers you start, which allows you to perform N repository operations concurrently
May 28 15:26:56 <bmbouter> so 2 cpus, you can sync 2 repos concurrently
May 28 15:28:02 <bmbouter> RAM tends to hit it's high watermark during sync and then go back down to nominal levels, so for N workers I'd say plan on a gig for each and then maybe 1 gig for postgres as a start
May 28 15:28:18 <bmbouter> so for 2 workers, 3 gigs total (2 for sync use, 1 for postgresql)
May 28 15:28:34 <bmbouter> our dev machines typically have 2-4 G and we never oom
May 28 15:29:12 <bmbouter> for disk it's the size of the repos you want all added together. pulp de-duplicates content so even as you sync those over time they tend not to grow very muh
May 28 15:29:14 <bmbouter> much
May 28 15:29:58 <bmbouter> I'm not sure what centos6/7/8 + el 6/7/8 is these days but maybe 400G?
May 28 15:30:43 <cognifloyd> 400G (ish) for the artifacts or the metadata?
May 28 15:30:58 <bmbouter> in terms of the encryption requirements I think pulp_file would work just fine for you, pulp doesn't need to read/parse the binary data it stores ever, it just needs to calculate the checksums and it can do that on the encrypted data
May 28 15:31:06 <bmbouter> 400G ish for the artifacts
May 28 15:31:14 <bmbouter> the metadata is very small and lives in the db
May 28 15:31:14 <cognifloyd> I'm not concerned about the filesize of the artifacts as I'll have them stored in azure blob storage.
May 28 15:31:19 <bmbouter> oh right
May 28 15:31:21 <bmbouter> you said that
May 28 15:31:59 <cognifloyd> ;)
May 28 15:32:10 <bmbouter> your disk can be small enough to provide working storage during sync prior to blobs being placed on the backend, so maybe 50G would do it all
May 28 15:32:37 <bmbouter> pulp verifies checksum data locally and artifacts download/verify in parallel so 50G is probably more than you'll need but it's a bit hard to predict
May 28 15:32:43 <cognifloyd> ah. ok. Thanks for some starting point rules of thumb. I should be able to adjust from there :)
May 28 15:33:08 <bmbouter> yw, if you can share what you find with use we'd love to hear. also let us know if anything could be better or doesn't work
May 28 15:33:20 <cognifloyd> will do
May 28 15:34:28 <cognifloyd> I really like the pulp 3 architecture with versioned repos (an entire repo metadata rollback sounds awesome). And I hate running Java, so a lot of the other artifact repositories left me with a horrible taste in my mouth. Python is awesome.
May 28 15:38:08 <bmbouter> I agree completely, Python is great!
May 28 15:44:39 <cognifloyd> Plus, no other artifact repository has ansible galaxy-like repo support.
May 28 16:01:13 <cognifloyd> I saw some conversation in ML about RBAC use-cases. Is there any work on implementing that yet?
May 28 16:03:00 <cognifloyd> I'm wondering if pieces of another RBAC system could be repurposed for pulp - ie StackStorm just opensourced their RBAC solution (uhm - the License still says its a EULA, but really it's apache 2.0, by "just" i mean they published it a day or two ago and now they need to fix the copyright and license files)
May 28 16:03:00 <cognifloyd> https://github.com/StackStorm/st2-enterprise-rbac-backend/tree/master/st2rbac_enterprise_backend

Jun 02 16:42:21 <wibbit> When are we liable to see rbac implemented?
Jun 02 16:43:03 <daviddavis> wibbit: bmbouter is working on it. not sure if he has a timeline.
Jun 02 16:44:03 <wibbit> Good oh, I may reach out to him at some point, hoping to have pulp access backed by IPA. We never got around to doing any thing with Pulp2
Jun 02 16:44:40 <daviddavis> that would be great
Jun 02 16:44:41 <bmbouter> wibbit: I'm actually working on it right now :)
Jun 02 16:45:05 <bmbouter> wibbit: please reach out with any use cases or needs at any point
Jun 02 16:45:24 <bmbouter> for your IPA use do you use that for your users or group definitions or both?
Jun 02 16:45:35 <wibbit> bmbouter: When you get to a point when you think you would like to do some more testing, hopefully I'll be in a position to assist.
Jun 02 16:45:47 <bmbouter> very cool I will reach out
Jun 02 16:45:58 <bmbouter> wibbit: what pulp3 plugins do you use today?
Jun 02 16:46:36 <wibbit> bmbouter: none yet, I'm in the process of getting it up and running, however I would expect us to be using RPM and File pretty much off the bat.
Jun 02 16:46:49 <wibbit> Possibly look at debian, as we have a fairly large pi setup.
Jun 02 16:47:21 <wibbit> I had tried that plugin in pulp2 but couldn't get rasbian mirrored
Jun 02 16:49:03 <bmbouter> I use rasbian
Jun 02 16:49:12 <bmbouter> my proof of concept will be for pulp_file
Jun 02 16:49:15 <bmbouter> so that's good
Jun 02 16:49:24 <wibbit> bmbouter: with regards to use case, I'm going to have to spend some time looking at the new approach in pulp3 from pulp2 to be sure.
Jun 02 16:49:50 <daviddavis> dkliban fao89 can we jump on a quick video call to sort things out with the z release?
Jun 02 16:50:03 <wibbit> I can give you a rough idea of how we do "releases" and what we would ideally be able to limit/allow
Jun 02 16:50:29 <bmbouter> wibbit: that would be helpful
Jun 02 16:50:50 <wibbit> bmbouter: We currently have three fairly high level concepts.
Jun 02 16:50:56 <fao89> daviddavis, it would be very helpful
Jun 02 16:51:08 <wibbit> The names don't matter too much, but hopefully they'll make sense.
Jun 02 16:51:09 <dkliban> daviddavis: let's do it
Jun 02 16:51:50 <wibbit> 1) Upstream, to be considered "dirty", it is a verbatim mirror of well, upstream. We have no control as to what ends up in there (I understand that's not strictly true).
Jun 02 16:52:19 <wibbit> 2) Trunk, this is ultimately the source our "release", any thing that is in trunk, makes it into the next release cycle.
Jun 02 16:52:43 <wibbit> 3) A release, this is a fixed point in time, and we only add packages into a release, by going through a notional merge process, with suitable review.
Jun 02 16:53:10 <wibbit> From an RBAC perspective, I'd like to be able to control who can affect our releases, or create repositories.
Jun 02 16:53:42 <wibbit> I'd like to be able to allow users to upload new packages into trunk repositories, ideally restricting which repositories
Jun 02 16:54:24 <wibbit> After a new package has been added, to be able to take any steps required to make that package visible to clients.
Jun 02 16:54:33 <wibbit> And of course the ability to remove faulty packages
Jun 02 16:54:53 <wibbit> As I said, this is very much based on Pulp2 methods.
Jun 02 16:55:06 <bmbouter> wibbit: I believe what I'm planning for the POC will do exactly this
Jun 02 16:55:25 <wibbit> I've got a test pulp3 setup with pulp_rpm, and once we get a update out, I'll rebase to 3.4.x and also get pulp_file
Jun 02 16:55:52 <bmbouter> it doesn't including publications initially but it does limit control around remotes, sync, and who can modify content in a repo (either through upload or sync)
Jun 02 16:57:08 <bmbouter> wibbit: and tell me about the IPA part
Jun 02 16:58:01 <wibbit> Our IPA setup is not AD integrated, we manage both users and groups (both posix and non-posix, some ugly bugs there!).
Jun 02 16:58:35 <wibbit> I would like to control permissions based on groups, individual users would be nice, but I'd drop that like a hot potato if it was a choice between the two
Jun 02 16:59:04 <bmbouter> ok great we plan to facilitate both
Jun 02 16:59:09 <wibbit> Ideally it would support kerberos authentication
Jun 02 16:59:21 <bmbouter> yup I believe pulp can be configured w/ that today
Jun 02 16:59:41 <bmbouter> but the issue is no one does because kerberos authN without meaningful authZ isn't useful
Jun 02 16:59:49 <wibbit> So, pulp currently supports authentication, it's just some what lacking authorisation
Jun 02 16:59:53 <wibbit> about right?
Jun 02 17:00:14 <bmbouter> eactly
Jun 02 17:00:17 <bmbouter> exactly
Jun 02 17:09:22 <wibbit> bmbouter: it's still handy, to limit who can affect change, even if you can limit what change they can affect.

Jun 05 07:29:07 <lieter> hi folks, what's the latest status of a CLI client for pulp3?
Jun 05 07:29:19 <lieter> the last I can find quickly is a blogpost from 2018
Jun 05 07:32:38 <ggainey> lieter: work on a POC is in-process right now - see this thread from pulp-dev@ ; https://www.redhat.com/archives/pulp-dev/2020-May/msg00065.html
Jun 05 07:32:50 <lieter> excellent
Jun 05 07:33:04 <lieter> I have my own small python wrapper around the the API now
Jun 05 07:33:06 <ggainey> you can experiment w/it from here : https://github.com/mdellweg/pulp-cli
Jun 05 07:33:17 <ggainey> feedback very much appreciated!
Jun 05 07:33:25 <lieter> which is specific to my usecase, but soemwhat generic enough
Jun 05 07:34:23 <lieter> I can upload debs, rpms and ansible roles and with one command (that reads a config file) create Repositories and Publications and add those to Distributions... but I can't really query the objects yet
Jun 05 07:34:46 <ggainey> sounds pretty cool :)
Jun 05 07:35:08 <lieter> It's WIP and internal for now
Jun 05 07:35:21 <lieter> but I don't see any specific reason not to publish it
Jun 05 07:35:42 <lieter> but I would like to have an official CLI and just script sround that :)
Jun 05 07:35:54 <ggainey> well, 'default to open' seems to work pretty well :)
Jun 05 07:36:39 <lieter> I've imported it into a clean repo already
Jun 05 07:36:40 <ggainey> mdellweg is out this week, but I know he's looking for feedback (and PRs :) ) on what he's got so far
Jun 05 07:36:45 * ggainey cheerrs wildly
Jun 05 07:36:54 <ggainey> cheers, even
Jun 05 07:37:04 <lieter> but like I said, it is soemwhat specific to my workflow
Jun 05 07:37:28 <ggainey> yeah, that's pretty typical of scripting to the API
Jun 05 07:37:43 <lieter> and will only be really useful for our open source projects (to publish packages) once RBAC is implemented in Pulp3
Jun 05 07:38:23 <lieter> hrm, nice it uses click
Jun 05 07:39:13 <ggainey> yeah - rbac is *also* currently in-process
Jun 05 07:40:37 <lieter> nice
Jun 05 07:41:13 <ggainey> we're working at adding features at a pretty steady clip - should be an interesting summer
Jun 05 07:45:52 <lieter> very good
Jun 05 08:08:13 <dkliban> lieter: are you using pulp to mirror container registries?
Jun 05 08:08:22 <lieter> dkliban: I am not
Jun 05 08:08:38 <lieter> dkliban: I _might_ in the future
Jun 05 08:09:04 <lieter> as we might ship container images that are built (and stored) in GitLab
Jun 05 08:09:21 <dkliban> gotcha ... i was asking because we released 2.0.0 beta 1 of the pulp_container a couple of days ago. and in addition to mirroring registries you can now use it as a regular registry that docker/podman clients can push image to
Jun 05 08:09:39 <lieter> oeh
Jun 05 08:09:56 <lieter> but no, no container images as of yet
Jun 05 08:10:08 <lieter> only RPM, Deb and Ansible
Jun 05 08:10:12 <dkliban> that's cool
Jun 05 08:10:32 <dkliban> pulp_container also lets you upload a Containerfile and have it do the image build for you
Jun 05 08:10:43 <dkliban> jsut for future reference
Jun 05 08:10:59 <lieter> how would that work?
Jun 05 08:11:05 <lieter> where does the context come from?
Jun 05 08:11:45 * lieter is doing container builds with Kaliko via GitLab CI
Jun 05 08:12:19 <dkliban> the context comes from the working directory of the pulpcore-worker doing the build. the REST API takes a dictionary that has artifact href's mapping to relative paths (file names).
Jun 05 08:12:35 <dkliban> the worker copies those artifacts from pulp's artifact storage under those names.
Jun 05 08:12:49 <lieter> hmm, cool concept
Jun 05 08:12:57 <dkliban> and then the Containerfile can reference those files
Jun 05 08:12:57 <lieter> doesn't cover my usecase
Jun 05 08:13:07 <dkliban> lieter: what's your use case?
Jun 05 08:13:08 <lieter> but I can see why it would be neat
Jun 05 08:13:23 <lieter> dkliban: building the images directly from the git repo
Jun 05 08:14:03 <dkliban> lieter: the use case we were tryng to cover is having the pulpcore-worker be completely network isolated and only be allowed to build image from repositories and artifacts already in pulp
Jun 05 08:14:23 <lieter> dkliban: I got that :), but it is not mine
Jun 05 08:14:28 <dkliban> yep
~~~
* 2021:
~~~
Mar 05 10:41:24 <anatolijd> hello there, any idea when RBAC will be implemented in pulp3 ? I want to use it in my organization to host private deb/python repo, and the only thing stopping me is singe user reasAPI access credentials.
Mar 05 10:42:26 <ttereshc> quba42[m], fyi ^ rbac is wanted for debian plugin
Mar 05 10:51:41 <ttereshc> anatolijd, pulp3 is pulpcore + plugins. Pulpcore has RBAC support (90%) and provides APIs for plugins to add RBAC. Currently, I think only pulp_container added RBAC support. As for other plugins, hopefully will be able to add to the majority in the next quarter or so, not clear yet. For pulp_deb ask quba42[m]. We also welcome contributions and will provide any guidance and support needed.
Mar 05 10:52:51 <ttereshc> anatolijd, you feedback is helpful so we can prioritize what folks are asking for
Mar 05 10:53:08 <ttereshc> cc gerrod rchan ^ there is an ask for rbac for python plugin
Mar 05 10:55:29 <rchan> ttereshc: ack
Mar 05 11:02:42 <rchan> anatolijd: gerrod is committed to a few items in the near future. perhaps you can work with dalley or gerrod to capture your specific use case in an issue so you can track it (and we're happy to take contributions)
Mar 05 11:03:53 <anatolijd> thanks! My scenario is to maintain private dev/python repos (as well as keeping local mirrors of official deb/python repo). I want to setup pulp as all-in-one solution and delegate repo administration to separate teams. Will it be possible with RBAC inplemented for deb and python plugin ?
Mar 05 11:05:19 <gerrod> i'll take a look into the complexity of adding it and if it's something simple I could get it into pulp python 3.1.0 and have it released next week, else it'll have to go into 3.2.0.
Mar 05 11:09:09 <anatolijd> sounds great! So great that I feel shy to ask deb/pyton support in pulpcore-client :)
Mar 05 11:14:41 <x9c4> anatolijd, what do you mean by hosting private deb/python repos? Do you need multiple users to provide the repos? Or would it be enough to have one mnanagement account and content guards to handle who can consume the repos?
Mar 05 11:21:13 <gerrod> anatolijd, I don't know if you saw x9c4 message, but he asked " what do you mean by hosting private deb/python repos? Do you need multiple users to provide the repos? Or would it be enough to have one mnanagement account and content guards to handle who can consume the repos?"
Mar 05 11:24:03 <ttereshc> "delegate repo administration to separate teams" makes me think that rbac is needed
Mar 05 11:24:49 <x9c4> Oh right. didn't look at that post.
~~~
###### tags: `RPM`, `Minutes`