Try   HackMD

Pulpcore 2024 archive

December 17, 2024

December 10, 2024

  • Issue publishing builds - we think PyPI is fixed now?
    • CI also no longer depends on checking-existence
  • Flatten migration-squash-issue-list by next week
    • various SMEs to do
    • discussion ensues
  • Anyone interested in joining PulpUI Phase 2 working group?
  • [ttereshc] please bring up to date the Core SME list https://hackmd.io/trURxHl-Qa-QyM1g0Jjb7Q, see the team's slack channel for more context
  • Discussion around This Meeting
    • review CodeQL output more often
    • an hour that goes short often? or 30 min that spins off deep-discussions occasionally?
    • release/mtg-lead rotation discussion

December 3, 2024

  • 3.70 is very soon
    • https://github.com/pulp/pulpcore/milestone/13
    • Think about deleting old migrations in Pulpcore.
      • RE old/pre-3.20 downstreams - it is already the case that they require multi-step updates
        • [AI mdellweg] check with Ian
      • [AI ggainey] to open issues for plugins to update migrations in prep for 3.70 compat release
    • <3.85 is the next upper bound to declare.
      • This will not conflict with us maybe making that actually 4.0.0.
    • Update the bindings container images
    • Openapi 3.1 ?
      • Moving out to the next breaking change release.
    • let's not release 3.70 before 2nd wk in Jan

November 26, 2024

  • RFC: task results

November 12, 2024

  • https://github.com/pulp/pulpcore-selinux

    • does anybody have any selinux expertise?
    • 4 (ignored) PRs waiting
    • escalate to mgt
      • can we get an selinux-expert from Elsewhere, to agree to 'own' this?
      • contingency: Merge And Hope

  • https://github.com/pulp/pulpcore/issues/5931

    • multiple FQDN gateways
    • what happens if we just allow CONTENT_ORIGIN to be empty?
    • can we get an experiment to see how this works?
      • just return relative-urls?
      • AI: bmbouter/services-team will See What Happens
    • can we build "absolute URLs" based on incoming request?
      • maaaaaybe - but, it is (or can be) Very Complicated

  • RFE: multiple URLs on a remote

    • https://issues.redhat.com/browse/SAT-26317
      • not RPM - would be core
        • or, does katello manage multiple-remotes for the repo, and retry-on-failures?
    • does order matter? how is it maintained?
    • how does this interact with RemoteArtifacts?
      • RAs have their own set of…suboptimality, already
    • how does this interact with ACS?
    • Can the remote-repos have diff characteristics? (eg, auth?)
    • net of discussion:
      • several possible avenues
      • all have pitfalls
      • How important is this, Really?
        • it's a really specific edge-case
      • escalate to Satellite PM for prioritization

October 29, 2024

  • Squashing Migrations
    • We will continue un this (maybe in a Plugin first)
    • We will require plugins to rebase their migration dependencies on a not too old pulpcore migration with the next breaking change release.
  • OTEL update, it's brokeness, and what we can do about it
    • auto-instrumentation is broken
    • the problems are near endless
    • we're replacing it with a bit of our own middleware and the SDK
    • will simplify oci-images which required special handling for OTEL auto-inst
  • Add 3.63 to supported branches?

October 22, 2024

  • Merging migrations redux

    • we have A LOT of migrations pre-3.20
    • last supported branch is 3.21
      • released Sep-2022
    • can pick something before 3.21 if we like
    • prob w/ squashing is at upgrade-time
      • makes pre-to-post-squash take two steps
    • would want to squash plugins first, if/when we do this
      • due to plugins dependent on core migrations
      • teach plugins to be dependent on "last core migration in oldest compatible core LB" at next breaking change point?
    • absolutely have to consider downstream implementations
      • need to have conversation with downstream because this affects upgrade instructions
      • downstream already deals with this to handle OS-upgrade-transitions
      • what is (if any) the restriction on multi-release-upgrades for downstream?

  • Look at Pulp 4 planning hackmd written by ggainey & give him feedback https://hackmd.io/@ggainey/pulp4_proposal

  • Pulp4 proposal discussion

    • packaging?
    • AI: [all] any review comments to ggainey by next core-team mtg

October 15, 2024

  • core/3.70 (breaking change) is coming - announce it, decide on next breaking-change? (3.85?)
    • do we even have "breaking" plugin-api changes? can't think of any?
    • PRN changes? - it's just a new field, shouldn't break anything, and is already merged
  • Pulp4 presentation draft - available no later than 22-OCT, ggainey to send draft-link to core-team list as soon as there is something to send

October 8, 2024

  • oci-env failing recently with "pip not found"
    • some problems seem to be resolved by switching the binary from podman to docker
  • Pulp 4 discussion
    • we probably do want to provide a v4 API in parallel with v3 in Pulp 3
    • plan:
      • /v4/ "shows up" in core/3.X line, as tech-preview
      • /v3/ and /v4/ co-exist
      • limited changes (prn/domain/acess-policies cfg-based only/remove-artifact)
      • at "some point", a core/4.0 is released
        • /v4/ is now out of preview
        • /v3/ still exists but is unmaintained
      • at "some point" we release a core/5.0
        • at that point the /v3/ API is removed
        • /v4/ moves to "exists but no longer maintained"
    • AI: [ggainey] agreed to own slides/presentation at PulpCON for this, and demo at some core-team pre-pulpcon

October 1, 2024

September 17, 2024

  • content in repo version set calculation is slow

    • what experiments can we run here to evaluate any PoC against?
    • example:
      • script to populate a repository with "many" entities
      • remove each entity (creating a new version) till empty, then add them all back one at a time
      • at each step, measure "amount of existing content" and "time to answer the question"
      • graph and consider the implications :)
    • another example: as above. but with a "broader" instance (ie, more repos and hence larger table sizes)

  • Telemetry worker

September 10, 2024

  • https://github.com/pulp/plugin_template/pull/910
  • users are asking if there is a place they can contribute scripts they've written https://github.com/pulp/pulp_container/issues/1749#issuecomment-2315236062
    • could just go in a gist from the user? have a place for links?
      • discourse?
      • docs-section?
    • we don't want to assume support for such things
    • consensus:
      • users can post to discourse first
      • have a section 'somewhere' ("Utility Scripts"?) in docs for "list of user script links"
      • we can pull things in to such a section as they show up?
  • pulp_labels work is progressing - who would be interested in commenting on a POC based on discussions from last week?
    • bmbouters, mdellweg, pbrochado already involved
      • need one reviewer - mdellweg
    • Q: All entities? Or just Content?
      • for now - just Content
    • comment: The nature of shared-content makes this…suboptimal. There are still objections to adding to Content - but we're moving forward, even tho there are downsides
      • Reproducible builds may conflict with this design.
    • comment: Run the draft-PR API past COPR before committing
    • comment: need to be able to find all the repo-versions "containing labelled content"?
      • also "all repositories by label"
      • Q-filter ways of doing this?
    • comment: the "search-and-destroy" problem is separate from this feature
  • PulpCon 2024 CfP - anything to add?
    • see draft in planning doc
    • ggainey to post to discourse by 1200 EDT - if you think of any themes before then, add directly to draft in planning doc

August 27

August 20

August 13th

  • High memory use during repository delete - https://github.com/pulp/pulpcore/issues/5048

    • draft PR https://github.com/pulp/pulpcore/pull/5049
    • some table-relationship-edge-cases to be evaluated
    • to get a 90% fix out quickly - maybe bundle up the odd/hypothetical-edge-cases into throwing "this isn't handled" exception, to be caught (if ever) in testing
    • can we get this tested in Staging? - work with services on that
    • want to make sure this delete-work doesn't inadvertently leave around "unreachable" entities post-delete

  • pulp_label_select improvement for replicate - https://github.com/pulp/pulpcore/issues/5087

    • existing Q-filter should be available to selecting-labels already
    • BUT - upstream-replicant needs an API that allows specifying to a q-filter
    • maybe add a new API call to alow downstream to take advantage?
    • Add a q_select parameter to the UpstreamPulp API. This parameter will be used to filter distributions. It will have the same syntax as the q filter everywhere else in the API.

  • https://github.com/pulp/pulpcore/issues/3338#issuecomment-2255898417 COPR would like labels-for-content

    • can we answer this by labelling Content?
      • collides with RBAC "ownership"
      • but is MUCH simpler, both for implementation and "user expectation" of a class of Pulp users (ie, single-user Pulp installs)
    • or do we need to add labels to RepositoryContent?
    • or should it be per-repository-version?
      • fails in the "deleting versions collapses content-added into the remaining repo-version"
    • is this a core-feature, or should it be plugin-specific (ie, in rpm)
      • ansible has "marks"
        • more general, more resource-hungry, not-currently-in-active-use
    • AI: [ggainey] summarize discussion into the above issue

  • issue with post-migrate commands when there aren't any actual migrations

    • do we need to explicitly say "any package upgrade needs to be followed by 'migrate'"?
      • that's kind of the current "assumption"

August 6th

  • Feedback request: Domain RBAC for Services

July 30th

  • RBAC for content - is that a thing?
    • based on repository-permissions
    • should answer dkliban/lzap's needs
    • ostree isn't using SingleArtifactContentSerializer…somewhere? (in import-all)
  • Labels for Content
    • COPR request - user added to issue #3338
    • Q: why is pulp_labels not a Global Thing?
      • initial implementation was generic-relationship on basemodel
      • horribly unperformant
      • refactored to use HStore
    • discussion about Content
      • shared-content causes shared-labels
      • this was before-domains
      • maybe it makes sense to do it now?
    • how about following the pulp_ansible content-mark content-type?
    • what about labels on repositorycontent model?
      • also keeps labels specific to repository
    • We could add pulp_label to repository_version
      • Not conflicting the other solutions, but accompany them.
      • think about how/if this violates our "repoversion is IMMUTABLE" rule - do we even care?
      • would also make dedup/RBAC a moot point - you'd have to have repo-RBAC-access already anyway

July 23rd

  • https://github.com/pulp/pulp-openapi-generator/issues/85
    • we def need to version that repo
    • what versioning-scheme do we need?
      • do we need to make versioned-fixes?
        • probably yes - to be able to fix things for older-supported-releases
      • how do we match a specified core version, to the "correct" openapi version?
      • how does this interact with "plugins can be associated with different versions of pulpcore"?
      • how about using calendar-versioning (ie YYYYMMDD.0)
        • easier to map "when" a thing happened than, say, hash-versioning which requires an extra trip to the GH repo
        • semver doesn't actually give us much here - we're not really in control of how-much-changed in openapi
        • use a .0 even if we're only going to branch/support if we really have to - that way tools already admit to the possibility of a "release"
    • this spotlights (again) that bindings should be created, on-demand, to match the actual installed versions in the specific pulp-instance they are talking to
  • Enabling SHA1 digests on an existing deployment. What should happen?
    • discussion ensues

July 16th

  • discussion around template-fix
    • ggainey to sheperd backports as needed (3.21/22)
  • discussion around dependabot Weirdness
    • commit that broke us has been reverted (for now)
    • mdellweg to get currently-open-pulpcore-PRs recreated
      • all: pay attention to how to do this, and do it for "your" plugin
    • HOWEVER - we need to be aware of potentially being broken "again" at some point
      • "widen" currently shows a Scary Warning when used
      • being explicit about that now, may teach us to ignore Scary Error Messages (which is suboptimal)

July 9th

  • pyPI project ownership
    • current state is…a mess
      • some projects only have one owner
      • some have at least one owner whose pyPI login does NOT have 2FA enabled
      • some projects under pulp-login are not owned by the pulp login (?!?)
    • every project under pulp needs to have pulp as one of its owners
    • need to have at least 3 people, with pyPI logins, that are 2FA enabled, be added as Owner to every single project owned by pulp
    • anyone whose login is not 2FA-protected, needs to be not-an-owner
    • there are 62 projects involved
      • ggainey volunteers to do the tedious work
      • proposed minimum-set of owners: ggainey, mdellweg, dalley?
      • anyone else who wishes can be added - but see above RE 'pyPI login, 2FA turned on'
    • Conclusions:
      • ggainey to investigate pyPI Organizations' suitability for Pulp
      • ggainey to drive the below
      • identify pyPI Ownership Team
        • mdellweg, dalley, one services-mini-team member (bmbouters to take back to team)
      • that team prunes list of projects
      • ggainey updates pyPI ownership of remaining projects
  • https://hackmd.io/@ggainey/task_throttling_thoughts

July 02

  • pyPI Next updates
    • Grant needs to find some folk with 2FA working.
    • mdellweg gonna help with it
  • ggainey's POC for pulp-cli
  • mdellweg - dependabot PRs messing with the dependencies' lower bounds
    • couldn't find any docs to configure this aspect
    • previous dependabot PR's didn't touched the lower bounds
    • let's ignore this update and wait for a week

June 25

  • Ruby bindings issue? https://github.com/pulp/pulp_rpm/issues/3639

    • Repository gets created, but then it throws an uninitialized constant error from Ruby
      • Has anyone seen this?
      • Can it be transferred?
      • pulp is doing The Right Thing, but something in the response is confusing the bindings-code
    • what's changed?
      • what version of openapi-gen? (looks like "no")
      • what version of pulpcore are the bindings generated against?
        • target to investigate: is it the "new puilp_rpm/old-pulpcore" mismatch causing us grief?
      • may be only rpm-plugin-version that changed
    • someone needs to sit down w/ iballou and pair-program our way through debugging
    • see potential problems w/ "us" generating bindings here
      • as of "now" it is Way Easier to self-gen than when that thread was opened

  • discussion around Artifact upload in console.redhat.com

    • currently, there is a bug where any-user can upload a naked Artifact
    • "who can see naked artifacts" - only the admin (actually any user can, same behavior as upload)
    • what needs to happen to allow a user to upload into-a-repository
      • you need to have upload-creator-role to allow chunked uploads
    • there was an issue with ostree-uploads - but it's fixed

  • multiple-files-uploaded to create one-repo-version discussion

    • proposal: upload into a temp-repo, then modify target-repo with a single modify (or rpm/copy) command
      • "modify" can require a large list of specified content-units
      • "copy" (available plugin-specific, e.g. rpm) lets one specify "everything from Repo-A into Repo-B"
    • proposal: add orphaned artifacts/content to repo
      • "naked" content can currently only be seen by an admin
      • content is immutable and shared, so no one user can "own"
      • how do you resolve permissions/protection on content in order to preserve "ownership" of content?

  • discussion around saving task diagnostics

    • lots of collaboration
    • open question: how do we report "artifact saved" to the user
      • log it?
        • makes self-service hard/impossible
      • save as created-resource?
        • not in a repo, just "naked" content
        • unexplored territory for Pulp
    • see 2024-06-18 meeting minutes for a LOT of previous discussion
    • break up the problem?
      • Step 1: create/report (logging only) the created artifact via UUID
      • Step 2: provide an API that could provide access to that UUID (assuming not cleaned up)
    • what if we just provide logging option first?
    • AI: bmbouter to post a summary in ticket

June 18

June 11

  • CI duckup
    • Fun with self-signed certs/azure/s3/pulp-replication setups
    • should we drop back to http:?
    • should we not-validate TLS?
      • (note: in CI)
    • see https://github.com/pulp/pulpcore/pull/5466/files for an experiment
    • pulp-python ONLY accepts SSL-remotes
      • syncing between test-pulp-instances is affected
    • is this an EL9 container issue?
      • pulp-container inheriting host-certs isn't trusted (suddenly)
    • "something" changed externally to make the workaround no longer work
      • "requests" update?
    • where do the certs come from?
      • self-signed openssl calls
      • container-startup generates one
      • trustme library? - not the current issue
    • we may not have a current expert in this area (Fabricio?)
    • need to get this addressed one way or another before we can move forward on (any) releases
    • AI: can we organize a few people to really dig into solving this problem please?
      • ggainey, mdellweg
  • Do we need any other blocker on the Milestone for the next Breaking Change release (3.55)???
  • Next breaking change release?
    • last time we went 15 releases
    • that seems to have worked well for us?
    • consensus: 3.70 will be the next breaking-change release
  • ipanova Orphan cleanup is opened only to the admin. This might be inconvenient in the domain enabled deployment where owner of the domain cannot free-up his storage bucket
    • orphan-cleanup already is per-domain
    • BUT, only instance-admin can run it
    • should a "domain admin" be able to run orphan cleanup on "their" domain?
      • consensus: yes please
      • AI: [dalley] open an issue and start design discussion there
    • larger question around "should orphan cleanup just be a Thing pulp does "for you" instead of user-intervention
    • discussion to be had
  • ipanova Task-group-issued tasks should be limited in workers usage. Certain actions like core import, replica, file ACS rpm prune create a task group that can potentially dispatch hundreds of tasks. We should add an option/setting to cap the % usage of the workers.
    • limit concurrent-tasks-per-user would be useful - but not this problem
    • limit concurrent-tasks-per-domain - similarly
    • worker-categorization/tagging possibility (eg, "only workers 1 and 5 know how to sign things") - also not this problem, but is something we should be considering as we expand the multi-user exposure of a Pulp instance
      • "cheap"/"expensive" workers concept?
    • consensus: let's solve the immediate task-group-ad-hockery inside of task-group, and then refactor the ad-hoc users to take advantage of that
    • consensus: the "related issues" above need to be solved
      • but design/architecture/brainstorming needs to happen first
  • Do we want to backport this?
    • https://github.com/pulp/pulpcore/pull/5463
    • "create api.,json without needing a running pulp instance"
    • drastically simplifies the CI
    • should help packaging teams as well
    • if we're willing to ignore the "hey! you're backporting a feature!" warnings, we can backport
      • mark as .misc?
      • maybe just hand-name it as .misc in the backport branches
    • consensus: yes please, figure out details at backport time

June 04

May 28

  • Discussion about db indices

    • Waiting on Grant

  • Do we want to stop publishing bindings?

    • AI: Publish a Discouse topic about this
    • Generate the bindings is very unique to each usage
      • Different plugins, domains enabled or not.
      • All those things make the bindings unusabled between different configurations
    • Service pulp users are generating their own bindings with very specific plugins
    • Which users would be affected by not publishing the bindings anymore?
    • Kept it on the milestone unless it's the only thing blocking the new breaking release

May 21

  • Use AWS STS for S3 credentials instead of saving them in Pulp
    • https://github.com/pulp/pulpcore/issues/5346
    • Should it be handled by django-storages, if it's possible.
    • There's a scenario where the application retrive the token using boto, the retrieve of the token can be very specific
    • Sources, a Red Hat Service, interacts with AWS STS.
    • We need to pass through Sources, to aquire a session token.
    • Have an internal pulp plugin with a new storage class to deal with it. Few code as possible on upstream pulpcore.
    • Make domains extensible

May 14

  • Kafka integration for the tasking system
    • https://github.com/pulp/pulpcore/issues/5337
    • Kevin Howell is going to work on this feature
    • several/many pulp users have expressed interest in such a facility
    • Kafka is popular - is it the best option for us?
      • is it still amqp under the covers? (if so, we'd then work w/ any amqp broker)
      • payload standard is…exciting
    • websocket/webhook support? (for, e.g., pulp-cli?, galaxy_ng?)
      • works well in 1-client/1-server models, not so much in Pulp's multiple-actor-model
    • one kind-of architecture is, service notifies to kafka/amqp Thing, and there exists a listener/shim to implement webhook/websocket layer
    • django-channels discussion
      • whole app would need to be asgi instead of wsgi
    • future thoughts:
      • who is it for?
      • how configurable is/should this be?
        • is it like setting up a remote? - consensus: Hell No :)
        • or is it an admin-setup?
        • if admin - what settings belong to Pulp, vs Kafka admin/cfg?
          • server-url/cnx info
          • topic
      • what happens if you can't talk to msg-queue?
        • several options on how "fatal" not being able to send/queue msgs could be
  • Improve replication
    • https://github.com/pulp/pulpcore/issues/5214#issuecomment-2104590545
      • key changes are:
        • replicate labels downstream
        • only replicate label-affected repos
        • do not clear-out repos not identified by labels
          • opens up window for collisions opn repo-names
      • creating a subset of repos without clearing out non-matching-repos in the downstream?
        • can this be flag-controlled? - needs discussion
        • collision needs to not-500, but to fail Obviously To The User
    • https://github.com/pulp/pulpcore/issues/4271
      • lack of this is swamping upstream/replication process
      • easy win
      • needs someone to pick this up
      • what if we limit number-of-sync-tasks-at-once?
        • we have a pattern for this in import/export already
      • consensus: use this ticket to handle multi-sync-limit
  • Add ability to migrate S3 storage credentials
    • It's an important upstream feature, and we could use some input on testing it
    • https://github.com/pulp/pulpcore/pull/4298
    • needs some not-gerrod, not-services-team testing, if at all possible
    • can this be testable in CI?
      • multi-bucket-testing requires commands to backend
      • current tests do file -> minio and back - that sounds good
    • clarification around default-domain is biggest thing keeping it in draft
      • what do we do when storage-settings-in-file no longer match storage-settings-in-db?
        • where do we check?
        • see SHA-SUM check as an example
      • issue discovered regarding dflt-storage not honoring env-var (see dkliban for details)

May 6, 2024

  • q filter still tech preview? https://github.com/pulp/pulpcore/issues/5319
  • django-storages dependabot discussion
    • 1.14.3 adds some env-vars that domains should know about
    • decision: hold off on these updates till gerrod has a chance to add that support, then merge gerrod's PR and these updates simultaneously
    • AI: decko to revert the django-storages[google] update before releasing today
  • discussion around API_ROOT work
    • let's put an end-date on waiting for feedback?
      • ie, "We'd really like to get this released to the world, but want to make sure we aren't missing anything you need"?

April 30, 2024

  • Using docker compose in our CI [dkliban]
    • podman-compose maybe?
      • not at feature-parity w/ docker-compose
    • problems due to old images/versions used in CI
    • whatever we do needs to be the same in the dev-env as is used in CI
    • currently ansible setup does do that?
      • maybe we can clean that up?
      • current setup is inherited from Travis - we don't have to do it That Way
      • should be able to massively simplify?
      • plugins still need to have control at pre-install-time?
    • AI: [dkliban] will file an epic and work w/ x9c4 to flesh out specific tasks. ggainey to learn, decko also.
  • core/3.53 releasing today - breaking change is coming, check deprecations please
  • ok to stop publishing changelogs on old docs now?
    • start publishing to new-place
    • redirect old-changelogs to new-location?
    • OR, note in old-changelogs pointing to new-location

April 23, 2024

  • Sentry / GlitchTip integration
  • discussion RE releasing
    • currently problem building/releasing (old-style) docs
    • holding off on releases until we get green-CI (sez ggainey who is release-nanny this month :) )
    • pedro/lmjachky investigating "removing possibly-unused dependency" that is causing our problem
    • consensus:
      • let's Just Do It
      • if something currently-unknown breaks the release-process, ggainey promises not to yell at anyone

April 16, 2024

  • an API to use with the livenessProbe in kubernetes - https://github.com/pulp/pulpcore/issues/5243

    • do it The Kubernetes Way
    • provide liveness-prob-api on content-app
    • database heartbeat?
    • cmd to run to check worker-liveness
    • api-liveness will also test db-liveness and connectivity?
      • let's not correlate these two answers
    • heartbeats all end up in the database - check that instead?
      • no, doesn't tell us "you can get to pulp from outside"
    • The Point here, is to let k8s decide "do I need to restart the pods" - and current approach (/status) sometimes takes Too Long
    • Conclusions:
      • YES, do a simple /healthz for api-app
      • YES, something similar for content-app
      • YES, cmd to check worker liveness
      • ADDITIONAL: worker that can't talk to DB "three" times in a row should die

  • updating timestamps seems to be taking minutes during a sync

    • "touch" bites us again
    • let's get an EXPLAIN/ANALYZE on the query
    • might be domain-index-related Thing?
    • can we get debug-tooling in staging env?

April 9, 2024

April 2, 2024

  • [pedro] Discuss OpenPG support draft PR (might interest COPR):
  • Take a look at supported branches list
    • https://github.com/pulp/pulpcore/blob/main/template_config.yml#L84
    • discussion ensues
    • the more branches we have, the more "in danger" the oci-image-creation is due to size limitations
    • discussion: we can't enforce this - but we can help/suggest/guide our downstreams to try and be mindful of the "hidden costs" of choosing a pulpcore
    • can we try dropping release branches?
      • we stop backporting dropped branches, which can cause all sorts of Fun
      • trying to do a lot of, say, CI updates, all at once, is…problematic
  • [decko] PR for ThirdPartyAuthenticationScheme

March 26, 2024

  • Remote Authentication, Pulp behind API Gateways and OpenAPI schema
    • OpenAPI advertises only Basic/Session auth
    • info put into OAPI via drf-spectacular
    • auth-backend being used in services is Different - looks for a third-party header
    • need to augment OAPI in a way to "know about" this
    • Fun With Pulp-CLI will then ensue :)
      • as long as the pulp admin sets this up following the OAPI Standard schema for a security-workflow, pulp-cli should be able to consume it
    • AI: [dkliban/decko] get an issue up for pulpcore-side work, docs
  • rubygems.org MFA
    • proposal:
      • all current users of the 'pulp' login at rubygems.org get a 'personal' rg.org account
        • ggainey, dkliban, bbouterse, dalley
      • all such accounts turn on MFA
      • THEN, all such accounts are marked as 'owners' of pulp's gems
      • THEN, pulp login turns on MFA
    • AI: ggainey to setup login, MFA, ownership and explore API access

March 19, 2024

March 12, 2024

  • [lmjachky] migrating large amounts of data in pulp-container

    • artifacts store JSON data, we want to get rid of the artifacts and have the data stored in the Model instead
      • migration vs django-command (cost–benefit analysis)
      • considering downtime (pulp/Katello/Satellite)
    • transaction timeout?; migration can run once, if it fails, it fails
    • storage far-away can/will increase time
    • what if artifacts missing?
    • pulp_gem process
      • new-model
      • migration added new-model
      • mgt-command to populate new-model
      • old-model would continue being available until all-new-models available
    • what if:
      • migration adds field
      • code knows both old/new way
      • django-admin then migrates the data
    • "agreed solution":
      • 1 migration to add the field, raise warning to run the management command
      • add codepaths to accept both versions
      • management command to migrate the data and report on missing artifacts (can be rerun)
      • Later release remove extra codepaths and management command (force that the new field is not None in the migration or migrate the rest) and add the migration that forces the "migration"
      • Document zero downtime upgrade procedure

  • [dkliban] Replication feature request

    • https://github.com/pulp/pulpcore/issues/4637
    • 4637 will be treated as a bug (because it's eminently backportable), dkliban to open an issue/RFE "handle making direct modifications to a Replicant, and then re-replicating"

March 5, 2024

  • django-lifecycle update breaks us
  • AI: all - if there are breaking-changes desired, we have 4 weeks to get them in
    • 3.55 is Coming!!
  • nice to haves RE release-process:
    • changes to requirements.txt being noted as "releaseable"
      • AI: [ggainey] to investigate updating check_release to look for these
    • what about test-infra?
      • don't need changelog entries (or at best .misc)
      • "I need a release of a fixture that my plugin needs to use" is the reason for a z-release with just-test-update
    • teach towncrier/release-script to auto-create a changelog entry for all req.txt changes since last Z
    • maybe keep this in the bnack of our heads as "something to automate better" when we have some free cycles

February 27, 2024

  • chunked uploads of big file are slow because of default nginx client_body_max_size is 1mb
  • task unblocked_at is ready for reviews

February 20, 2024

February 13, 2024

  • Github PulpOrganization/Core membership
    • help!
    • what about pypi/rubygems.org?
      • more coverage needed
      • we need to figure out how to handle 2FA for these sites
      • more volunteers needed
    • bitwarden locker?
    • need to fix the immediate need (not enough coverage) now
    • long-term, maybe discuss Better Ways
    • dalley, mdellweg, ggainey, dkliban volunteers for pypi access-keys
    • mdellweg, ggainey, dalley, bmbouter volunteers for RubyGems.org
    • ggainey wants to be added to the pulp org in github
    • we should audit permissions "regularly" (once/yr)
  • memory spike fix discussion https://github.com/pulp/pulpcore/pull/5049
    • affects large/mature Pulp installations, sometimes quite badly
    • large change - more eyes please, once it's out of draft
  • "supported branches" and "lower bounds tests"
    • pulp_deb had, as a lower-bounds, a not-in-supported-branches pulpcore
      • examples: pytest<8 and black<24 and pulp-smash-removal issues
    • all LBs "should" match a supported-branch
    • Probably not solvable in a generic way. We'll fix this when it comes up.

February, 6, 2024

  • Python 3.9 and CentOS 9
    • katello wants py3.11 "now"
    • configurable py-vers and updated base-image - how do we make sure they don't conflict?
    • we already know how to tell branches to use a specific-image-tag 9eg, we have older branches that need py3.6)
    • centos9 doesn't "have" py3.8
    • do we want to bump main-compat to 3.9+ ?
      • yes please
      • 3.8 is EOL "soon" (upstream)
      • downstream(s) are on 3.9 already
    • to do this:
      • images to centos9 (ie py3.9)
      • then configurable-py-version
      • older-supported-branches?
        • new image gets anew name
        • current branches continue to use existing name
        • plugin template needs to enable specifying image-by-name (it may already)
        • AFTER this all works, review supported branches and decide which ones need to "move up" py-vers-images
      • need images for diff py-vers (oci-images already discussing making this happen, and making it selectable)
    • https://github.com/pulp/pulp-oci-images/pull/575 is the in-progress work
      • currently stalled (s6 deprecated functionality)
      • AI: gerrod to take over PR, gerrod/dalley to sit down w/ mikedep to resolve current issue
      • dkliban is a resource for finding the right s6 folks to talk to
  • Repository deletion memory spike https://github.com/pulp/pulp_rpm/issues/2280#issuecomment-1928769513
    • problem affects both up- and downstream
    • publishedartifact "seems to be" the current culprit
    • see possible approaches in links mentioned in 2280
    • this prob needs to be addressed at pulpcore level
      • let's handle this at a generic/consistent level (see Hap's blog post)
      • poke Hap and ask if he wants author-credit if we expand on his examples
    • dalley to experiment in pulp_rpm, and move to pulpcore once we have something working

January 30, 2024

  • pytest<8

    • core, deb, rpm, certguard are addressed
      • core needs backports
      • AI: ggainey to sheperd those backports
    • every plugin needs to update their test-requirements
    • do we want to install into base-image?
      • pytest shouldn't be in production image
      • if installed in the image, we might not "notice" problems until much later
    • plugins have their own test-requirements
    • shows us (again) upperbounds-required protects us
    • should we have something in plugin_template to address this as part of ci-update-next?
      • AI: mdellweg to prepare such a thing
    • do we really need to have an upperbound on Everything?
      • lint/test reqs are safer for products to not-require-UB?
      • dependabot would yell at us when updates come up
        • does it? - yes
        • would allow us to plan accepting updates
      • yes please?
      • AI: [all] plugin leads need to open an issue and update their test-requirements to "reasonably pin" upperbounds
      • AI: [ggainey] add to discourse-discussion with an explanation for this
        • note on "why we don't use pip-compile"

  • Dependencies on release branches.

    • is there some way to evaluate dependencies for release-branches?
    • changing deps to allow "newer" can introduce issues
    • new deps can be incompatible with other entangled deps
      • if just relax UB, shouldn't be an issue
    • discussion around pip-compile and "pin exact versions always"
      • prob not a Good Idea for us
    • if we were using Pulp to curate our own content, we would maybe have not been affected by this?
      • better to break our CI than our upstream
      • "our" scale is significantly different than "pulp users" scales

  • Pulpcore certguard docs still not able to publish.

  • core/3.22.X to be released (jinja update)

    • backport releases are often done by not-the-release-sheperd

  • core docs conversion update

January 23, 2024

  • orphan_for filter ready for review: https://github.com/pulp/pulpcore/pull/4668
  • LTS branches
    • what is The Process?
      • can't support more than 5 +/- 3 branches at a time
      • OTOH - product-lifecycle suggests "support for 3 years"
      • can we add a "list of supported versions" to the spreadsheet/matrix? Can we make that happen with Spreadsheet Magic?
    • downstream/Pulp communication is a fertile ground for missed communications
      • stakeholder/pulp teams need to be Very Explicit about this
    • what if we choose "breaking + 2" as "always a supported release"?
      • any new feature request/migration issues would mitigate against this
    • what can we do/control?
      • no drawing to an inside straight, for stakeholders :)
      • we can be more explicit about "these are the branches we currently allow backports to"
        • lead-to-stakeholder communication
        • "look at template.cfg", and/or support matrix
      • each core-team-mtg, review currently-supported branches and decide if something has gone EOL
    • problem: stakeholder moving to a new branch wants no-regressions
    • if you backport to "A" supported branch, you need to backport to all supported branches in between
    • if stakeholder moves to "not latest but newer than current-lastest-supported", then they might be missing backports
    • AI: stakeholders need to stay in close comms w/ stakeholder leads RE supported versions
    • AI: stakeholders cannot move to "random" branches
    • AI: backports have to happen "in supported branch order" (No Regressions)
    • AI: support-matrix is Controlling - be aware of it!
    • AI: let's think about how we actually retire old branches?
      • always a challenge
    • AI: [mdellweg] open issue in plugin-template to separate "latest release" and "stakeholder supported" branch-lists for us
  • mdellweg to do next release
    • certguard merge blocking 3.45 - let's get it removed

January 9, 2024

  • Today's release may have some Issues due to pypi API access changes
    • mdellweg continues as release-nanny for now
  • Do we want to bump minimum Python version to 3.9?
    • oci-images (PR draft?), plugin-template prob need changes
    • PR bumping oci-images to EL9 and Python 3.9 https://github.com/pulp/pulp-oci-images/pull/554
    • what does 3.9 get us?
      • still needs monkeypatch to enable TLS-proxy support
      • need 3.11 to not-do-that
    • bumping in oci-images means we no longer test on 3.8
    • can/should we get signoff from stakeholder leads?
      • RHUI/Satellite on same rpms (3.9 for current-support)
    • what if oci-images had multiple python-version images, and branches used "oldest supported"?
    • what's the proposed "long term" python-version to be supported?
      • maybe whatever is in F40?
      • 3.12 on F39
      • maybe pick 3.11?
    • conclusion: we're not ready to do this yet
  • bumping aiohttp to 3.9.0+ and python-cryptography to 41.0.6+
    • stakeholders want bugfixes
    • need to think about which branches need them
    • need to make sure branch-upper-bound allows fixed version(s) to be installed (main branch already ok)
    • mdellweg to pick this up
  • Orphan cleanup: https://github.com/pulp/pulpcore/pull/4841
    • Can't find a way to be compatible with orphan_for filter: https://github.com/pulp/pulpcore/pull/4668
    • maybe close 4841 in pref for 4668, and then change tests in https://github.com/pulp/pulp_rpm/pull/3313 to take advantage
      • conclusion: yes please
    • Note: if pulp stopped allowing content outside of repositories, the need for this goes away
      • implications on uploads (mostly)
      • impacts "what is the change-diff" (can address in other ways)
    • see also container's "pending blobs" solution
    • last-touched and sync implications
  • PyPI credentials update
Last changed by 
 
Pulp
Design/Discussion/Meeting minutes for https://pulpproject.org/
0
32

Read more

CLI Team

https://github.com/pulp/pulp-cli/pulls

Apr 9, 2025
Pulpcore team meeting

Core SME List

Apr 8, 2025
Pulpcore/katello/pulp_deb integration meeting

See also: https://discourse.pulpproject.org/t/katello-pulp-community-integration-meeting-minutes/39

Apr 1, 2025
RPM meeting

discuss adopting zero-downtime-migration strategy

Mar 6, 2025
Read more from Pulp
Published on HackMD