Try   HackMD

Pulpcore 2023 archive

December 05, 2023

  • Ignoring dependabot PRs for Django in pulpcore?
    • can be done by responding to dependabot w/ "ignore"
      • "@dependabot ignore this dependency"
    • do we respond to Z-releases in django?
      • we look at z-releases and only accept if they affect us?
    • dependabot won't show us 4.2.z anymore, now that 5 is out
    • consensus: ignore dependabot for django in pulpcore
  • Switch on auto-merge together with "dismiss stale reviews".
    • note: "dismiss stale" means "cancel approvals w/ new push"
    • ggainey: yes please
      • don't forget : "approve" means "ok if this merges instantly"
    • bmbouter: approved
    • dkliban to Push The Buttons

November 28, 2023

  • Showcase: Auto-merge and dismiss stale PR review.
    • Prior art: pulp-cli-*, pulp_gem, pulp-ansible
    • This requires a certain mindset:
      • What you approve is what you accept to merge.
      • Merging is not the last guard to get in but approve is.
  • Can we please delete all the nightly published dev versions of the clients from pypi and rubygems?
    • [dkliban] will clean up the ruby gems
  • Generated bindings are weird in strange ways.
  • We have 6 active release branches in pulpcore.
    • What can we do to reduce this number?

November 14, 2023

  • If not done already, please, transfer issues from the pulp_file repository to pulpcore
    • https://github.com/pulp/pulp_file/issues
      • no in-flight work, so we should be able to do the transfer
      • done!
    • archive the repo?
      • can't make it read-only - we do have to be able fix older releases
      • we need to mark 1.15 as "default" as the first-place-to-fix?
      • should fix in pulpcore first, then apply to 1.15, then cherry-pick
      • need to think about pulpcore-issue-links if/when/as we need to backport fixes
        • think about this when it happens?
      • in readme announce the "Hw to ask for pulp_file changes/report bugs"
      • AI: mdellweg to add this process-discussion to pulp_file README.md
    • disable the creation of new issues
      • yes please
      • Just One Thing (mdellweg) : how then would we deal with CHANGES/ and possible pulpcore/pulp_file overlaps?
      • are we going to lose links to issues? (see discussion above)
      • AI: ipanova to turn off issue-creation in pulp_file
    • we should create a separate section for pulp-file. like we have for CHANGES/plugin_api/ we could have CHANGES/pulp_file otherwise changelog will have these all merged in one list and it will be difficult to distinguish what is pulpcore and what change is pulp-file
      • already handled!
    • need to merge pulp_file docs into pulpcore
      • do we move pulp_file docs into the pulpcore docs/ dir?
        • this is a non-trivial task
        • this is a question for docs-working-group?
      • we can still publish from old repo when/if/as needed
        • is that "enough" for now?
      • pulpcore issue: https://github.com/pulp/pulpcore/issues/4625
    • need some discourse announces and note to pulp_file's README.md for this
    • is there work to do on oci-images and/or oci_env due to this change?
      • oci_env
        • workaround: check out 1.16 and list in your deps
        • how do we fix oci_env to "know about" subprojects?
      • pulp-oci-images
        • do we need to make changes there?
        • for now - we install empty-pkg to show that this doesn't break anything
        • "soon", we may remove that need - but not till we're convinced everything works
        • do we need to call out, in docs, that pulp_file is autoinstalled?
          • seems to be "ok for now", docs-merge may require a link change
  • https://github.com/pulp/plugin_template/issues/792
    • github-actions are WAY outdated
    • we need someone with org-admin to allow these workflows
    • AI: (dkliban) allow new version of workflow
  • what's the magic command to check for releases? talk to mdellweg offline pleasae
  • should we invite rchan-directs@ on invitee list?
    • inviting a mailing-list is Suboptimal

October 31, 2023

  • re-evaluate state of S3 timeout issue
    • It's fixed!
  • REMINDER: no meeting next week due to PulpCON
  • release-nanny/mtg-lead rotation
    • can we ask bmbouters if he's still up for being on the release-lead list?
  • pulpcore 3.40
    • https://github.com/pulp/pulpcore/milestone/9
    • update to pytest seems to have broken our test-run
      • mdellweg has a fix in his PR
    • would be wonderful if we could get both PRs merged and get 3.40 out on Halloween!
    • pulp_file docs are not in pulpcore - that will need its own PR
      • pulp_file in pulpcore, shipped in same pypi pkg, it's a separate app
      • tests moved into pulpcore hierarchy to get them run in current CI setup
      • we don't currently know how to publish multiple-app-docs
      • can we merge pulp_file doc into pulpcore docs hierarchy?
        • needs an issue
    • need one more release of pulp_file (1.6) showing compat w/ 3.40+
      • empty package, will keep Everything Else happy
    • next on the merge-into-core list : pulp-certguard
      • what should certguard declare compat with? 3.44?
      • can we get an issue open in pulpcore for the certguard merge? [dkliban]
      • discourse thread as well
        • in pulp_file-merge thread [mdellweg]
    • once 3.40 releases - all plugins need a compat release #SOON
    • make sure to make something of a splash in discourse
    • nightly images aren't building, due to 3.40/plugin compat fun (this is expected)
  • Question: should we open a task to re-evaluate testing-strategy across the entire project?
    • core/plugins seem to be in good shape
      • "what do we test, where" is not as easy to answer as it should be
      • i.e., "what pulp-instantiation-scenarios are we testing, and where"
        • from-source, from-pypi, from-images, etc
    • are there areas (in this context) where we aren't covering actual-user-workflows?
    • oci-image testing may need some help/evaluation
    • what about operator-tests?
      • runs container/ansible doc-scripts
    • let's open such a task-force?
      • 2-3 people, 2-3 weeks, post-pulpcon
      • people-bandwidth, as always, is going to be Exciting
    • how about a one-hour, pulp-project, strategic-planning mtg? (this would be just one such issue)
      • this test-strategy/coverage isssue is one
      • docs (general updates)
        • esp installation
      • move docs to markdown
      • ???
      • bmbouter to create such a mtg, let's pay attention to this
  • Revisit the multiple cert guards

October 24, 2023

October 17, 2023

  • AAP and pulp_file-into-pulpcore
    • only concern is "hide pulp_file from users"
    • RBAC should make that possible
    • "at some point" AAP will want to use pulp_file (#soon)
  • do we want there to be a default-off RBAC policy for pulp_file?
    • don't want the complexity of a feature-flag in core
    • disable-by-dflt will make Pulp's life harder, for not-the-main-path usecase of Pulp
  • Do we want to make domain compatibility a plugin requirement with 3.55?
    • because all-plugins are included in images, can't turn on domains
    • ggainey: this is labor-intensive, and we are short of bandwidth
    • ggainey: can we have domain-enabled/plugin-limited images?
      • same bandwidth issue, plus ongoing maintenance on the OCI team
      • OR - "domain on" is what the images have, and non-domain-enabled plugins no longer get installed into OCI images - will prob happen "at some point"
    • what "major" plugins are missing?
      • container
      • debian
      • ansible
      • python
    • each plugin needs to make it happen on their own timeframe
    • smaller plugins could prob be contained as side-projects?
    • not in a place to make this a 3.55 requirement?
      • can def revisit as Things Change
      • plugins should def consider making this a priority if/when/as we can
  • [decko] Maybe it's time discuss the one-commit per PR rule again?
    • continue discussion from last week
    • what is best for the project? it would be easier to answer if we had written criteria for how we determine if something benefits the project
    • 2 main pain-points:
      • patchback can't handle multiple commits
      • squash-merge affects merge-msg/consistency
    • if we can come up with answers to these - re-raise this issue?
    • currently, you can use multi-comits and have tests run, multi-commit-red doesn't block merges (not required)
      • it's red because github only recognizes "fail", not "warning"
    • current workflow allows for multi-commit features
    • can we update machinery to recognize the diff between feature/bugfix?
  • we need to make our github-related values explicit
    • contribution friction
    • maintaining clean GH history
    • testability/quality impacts
    • practicality/automate-ability
    • user-acquisition/increase Pulp userbase
    • question: "What would change my mind on $SOME-TOPIC?"
    • AI: ggainey: pull together a draft "What do we Value?" doc, get it onto Pulp Team agenda (Retro? Monday Team mtg? 3 Month Planning? PulpCon?)
      • make sure there is a really clear WHY
        • need a more-organized way of making decisions
        • proposal: ask "Why" as a google form w/ closed results, look for themes in responses
          • results closed to isolate, not to anonymize - publish later
  • FYI, RFE from Services group here: https://github.com/pulp/pulpcore/issues/4583
    • high-priority ask - add comments/observations to the issue please!

October 10, 2023

  • grats to decko on one year anniversary at RH! +1

    • [decko] It's been a wonderful jorney with you all. Thanks

  • Breaking changes release (3.40) upcoming in a few weeks

    • Is there anything we want to put in there that isn’t already on the list?
    • last breaking change release was 3.25 (May 10)
    • What would the next breaking release?
      • 3.55 - since landing in the 4-6 mo window seems to work
    • 3.38, 3.39 release notes should include reminders "3.40 is coming, update your requirements!"
      • dkliban takes AI to post/pin this warning
    • should we limit deprecations (not allow in 3.38/39)
      • would let plugins adapt to 3.40 changes
      • might not be practical right now due to current in-progress changes
    • lets be explicit about the gap between "3.40 release" and "images available"
      • if the release-to-image gap covers multiple core releases, it's…Suboptimal

  • Should we merge pulp_file into pulpcore?

    • We can collect arguments as ai for next week.
    • features added in pulpcore, tests end up in pulp_file, not infrequently
      • suboptimal test/CI experience
    • do we continue to produce separate package? or does pulpcore deliver core and file?
    • would we move pulp_file into pulpcore repo?
      • subrepo?
      • sep django app in same repo?
      • move pulp_file into pulpcore app?
    • downside: you can't not-have pulp_file
      • "very few" users don't use pulp_file
      • def need to publicize if/when/as we decide to do this
      • this might impact things like security/attack-surface audits, if we're adding pulp_file to Things that don't install it now
    • what about packaging/dependencies?
      • def check if there's anything new in pulp_file
      • run proposal by Odilon RE packaging impacts
    • 3.40 is the best place to do this?
      • kinda tight
    • AI: dkliban will post to discourse on the proposal
    • can we get a go/nogo from stakeholders by Thurs 12-OCT?
      • AI mdellweg to talk to AAP Downstream about impacts

  • [decko] Move away from pycryptography 38.0.x given CVE-2023-23931

    • Quay is complaining about 38.0 having a CVE
    • 39.0 has some serious breaking changes
    • pulpcore specifies cryptography>=38.0.1,<41.0.5
    • does something else pin it down more?
    • latest image finds both 3.41 and 3.38 and chooses 3.38 "for some reason"
    • concern w/ raising LB to (say) 39.0.1 would only be "does this affect packaging"
      • add to dkliban discussion w/ Odilon :)
    • pyOpenSSL is a req't of pulp-certguard:
pyOpenSSL==22.1.0
└── cryptography [required: >=38.0.0,<39, installed: 38.0.4]

  • [decko] Maybe it's time discuss the one-commit per PR rule again?

    • It make harder to contribute with other users on the same branch
    • Can't accept contributions from other users directly on the discussion page
    • switch to squash-and-merge on accepting PR?
    • patchback will have problems - but it's onlt used for bugfixes
    • current rules mark CI as red, but that isn't a showstopper
    • for features, can make sense to have multiple commits
    • letting github do squash-merge has downsides
      • breaks CI assumptions
      • changes commit-msg to not match our assumptions
    • we can take advantage of multiple-commits in PRs right now, as long as one contributor does the manual squash-step before final merge
    • AI check to see that oci-images has same rules?
    • discussion around "how should we be making decisions like this?"

  • Do we want to make domain compatibility a plugin requrement with 3.55?

  • AIs

    • dkliban takes AI to post/pin this warning (3.40 coming)
    • dkliban will post to discourse on the proposal to move pulp_file into pulpcore
    • mdellweg to talk to AAP Downstream about impacts of pulp_file-into-pulpcore
    • dkliban to talk to Odilon about impacts of pulp_file-into-pulpcore and crypto-requirement-update
    • ggainey to investigate/release a pulp-certguard that can get us off of crypto-38.0

October 3, 2023

  • Breaking changes release (3.40) upcoming in a few weeks
    • Is there anything we want to put in there that isn't already on the list?
    • last breaking change release was 3.25 (May 10)
  • New content guard needed - https://github.com/pulp/pulpcore/issues/4518

September 26, 2023

September 12, 2023

  • Deleting objects can take too much time for the gunicorn worker.

  • Problems with the pulpcore-api entrypoint

    • operator overrides the init script in the pulp-mimial container
    • the init script waits for all other services - this should be removed
    • need to bring operator back inline with the container installs
    • have our docker compose files test like pulp operator in our pulp-oci-images CI

  • Release branch pipeline fix [urgent]

  • Question about disk_space PR

    • https://github.com/pulp/pulpcore/pull/4376
    • lots of good discussion ensues
    • this is a very expensive calculation
      • actual numbers would be helpful
    • could it be a periodic task?
      • how often?
      • is there any setting that wouldn't be "wrong" for at least some users?
      • calculate at-sync-end, and use periodic to pick up on-demand content being actually stored
    • could it be a separate endpoint?
      • puts the calculation-burden on only the users that want this info, rather than on all users of the list-endpoint
    • could/should it be an async endpoint?
      • generates a report and stores it, and makes it retreivable (in task, or as a separately-retrievable file?)
    • doing it as part of the repo-list-endpoint is Counterproductive (in terms of perrformance)
    • is there a difference (or even A Way) to calculate "disk currently used" vs "disk that MIGHT be used by on_demand content"
    • can this be a django-admin command?
      • don't we already have Publication-related pulpcore-manager commands? (rpm-publication-report)
      • can we think about reporting in general?
        • this is a Minefield
      • it's likely that at least one use-case will want this as an API call (controlled by RBAC/domains)
        • let's hit up that affected user and see if we're right about that assumption
    • what about "size of all Artifacts currently?"
      • also, per-domain is Important
    • do we ever want this per-repo? or just only when you ask for a list-of-repos?
    • is there a way to store the resulting report as a "permanent" object and have the task hand back a pointer to that entity to the user when the task is completed?
      • storing "inside the task" will eventually run into scale-problems
      • how does the end-user "download" the resulting report?
    • TL;DR decision: let's start with a pulpcore-manager command to generate this, and deal with API questions IFF a/the user insists

Sept 5, 2023

  • [dkliban][mdellweg] to look into potentially affected areas, prep questions for davidn related to proxy request https://github.com/pulp/pulpcore/issues/4207
    • discussion ensues
    • LOTS of architecture-thinking needs to happen here
    • Proposal: make HREFs independent from current settings (ie API_ROOT/domain-off-on/proxy-gateway)
    • "new reserved resource" json-array field? or just add into existing?
    • mdellweg to add notes into #4207
    • who has people to work on this? or should?
      • prob better for core-pulp to do the backend, may be able to get assistance on the serializer-side
    • "required" date is "early 2024", BUT POC "sooner rather than later" would be Good
    • maybe get an oci-env profile for testing multi-gateway-approach "soon"
  • https://github.com/pulp/pulpcore/pull/4249
    • gerrod/ggainey to review
  • https://github.com/pulp/pulp_file/issues/896
    • discussion ensues
    • for michal and #869
      • extend existing create-content to accept a Remote
      • add new-remote URL and path-in-that-dir and sha256-checksum
      • download, check checksum, put at existing rel-path argument
    • https://github.com/pulp/pulp_file/issues/628
      • new pulp_file Remote
      • at sync-time, pulp_file recognizes Remote and "magic code" understand the downstream dir-structure and knows how to fetch therefrom

Aug 29, 2023

  • Immutable content.
    • There have been some places where we said "Yes, I know content should be immutable, but…".
    • This seems to always end in hells kitchen.
    • I want us to reinforce the no 1 pulp rule, that content must be immutable.
      • Reason there is no sharing if it can change.
    • I want us to find all the places where we were lax and agressively address them.
      • pulp_rpm modulemd snippet issue - code deleted
      • pulp_container had something like this - code deleted
      • pulp_ansible 'is_latest' gets updated
    • data-repair is a different (related) issue
      • when you're fixing content, you want to fix it in all shared locations
    • "normalizing" as much as possible supports this
      • metadata changes (as opposed to artifact-changes)
  • Pulpcore team code review configuration on GH
    • is auto-assigning a good idea, or not?
    • discussion ensues
    • consensus: auto-assignment of reviewers isn't really useful for the way this team operates - let's turn it off

Aug 22, 2023

Aug 15, 2023

Aug 1, 2023

  • AI: [dkliban] open issue/start work on this
  • should we yank releases due to import issues?
    • import/export is broken on the last 2 z-streams of several branches
    • do we need to notify downstream users
    • improve import/export tests?
      • it's not a test-problem, it's Something Else (currently unknown)
  • would it be useful to have a nightly-test-run, in plugins, that would run against pulpcore/main (instead of against released-pulpcore)
    • this is in service of getting ahead of pulpcore-engine-changes that inadvertently break plugin-behvior, before releasing it into the wild
    • proposal: do so, and have a feature-merge-deadline for core of "Monday Afternoon GMT-5"?
    • discussion ensued
      • would be nice to catch core-changes earlier
      • more friction is Bad
      • this is an exception case we're addressing
      • run plugin tests against oci-image-from-source
      • make it a manual-only job?
    • prob not worth more than a day's work or so
      • if somebody is Real Excited About This - go4it!
      • otherwise, let's not

July 25, 2023

  • [lmjachky] Should we leave the stream runner enabled? What are the pros and cons of having it? It is unreliable.
    • relies on sftp, which is not thread-safe
    • BUT - sftp is only storage that doesn't support redirection, which is what we have it for
    • can we have a specific test for this, that doesn't rely on the stream-runner?
    • the test exists for a reason, BUT…
    • discussion: having unreliable tests is bad, let's remove it
      • disable? or remove?
      • plugin-api-docs "DON'T DO THIS" section?
      • consensus: remove stream-section from plugin_template
      • AI: [dkliban] open issue/start work on this
  • [lmjachky] Do we want to review issues flagged as "Severity-High": https://github.com/pulp/pulpcore/labels/Severity%3A High?
    • first step: let's lose the labels left over from the redmine migration that we never use
    • consensus: there is plenty of on-fire stuff to do, let's not pick up things nobody is currently complaining about
  • We have a custom pytest hook in pulp-smash implemented in pulpcore.
    • is anyone using it?
    • Should we make the implementation failsafe for now?
    • Can we move the hook to pulpcore?
      • what does pytest do if a hook is implemented in two places?
    • if we remove this hook from pulpcore, the pulp-smash invocation will just do-nothing
    • let's Do This

July 18, 2023

  • [decko] Use poetry for dependency management (and possibly as build-system???)

    • Can deal better with some conflicting scenarios
    • Lock file
    • Dependabot supports it
    • "One file to rule them all."
    • [dalley] poetry and "arbitrary/muiltiple plugins" didn't work well together previously
    • [mdellweg] need buy-in from Build Gang
    • [mdellweg] example problem: PyYaml/Cython collision just happened, poetry is having a lot of issues as a result
    • [dalley] is pip-new-dep-solve the default now?
      • we "think so"? (yes, 2021)
    • [ipanova] We have already done some PoC poetry evaluation time ago
    • AI: [decko] can we get all this into a discourse thread please

  • tasks that leverage task groups can potentially block whole instance (i.e. import task)

    • Example of a problem:
      • import spawns a task for each repo
      • tasks line up and are handled one-per-worker
      • everything else lines up behind them
      • in a specific customer issue, 129 repos in a single import blocks task-processing for…a very long time
    • mdellweg/ggainey have been discussing already
    • discussion around workarounds possible
    • need to decouple "make improvements to import" from "make tasking-system more sophisticated"
    • AI: [ggainey] open issue to provide POC for import not using all-workers all the time
    • can we make another working-group RE tasking system?
      • does anybody have the cycles right now?
    • AI: we need an RFE "allow cancelling a task-GROUP"

  • changelog entries and releases

    • entries got "messed up" on main branch
    • dkliban has a PR open to fix
    • there's possibly a non-atomic-problem happening w/ release?
      • was this a once-only problem due to release-workflow-problems?
    • 3.29.1 had a problem as well?
    • we prob need to have a discussion "soon"

July 11, 2023

  • Pulp 4 when?
    • why - because there are "suboptimal design decisions" that we currently can't change due to API issues"
    • AI: we def need to start/have a list of things-that-would-change-the-API, that would drive this
    • when that list is "large enough", then it might be time
    • is there a way to change "broken apis" that is not an X-update?
    • AI: let's get a discourse thread started on this
    • AI: let's discuss at PulpCON!
  • Can we teach users that content in Pulp is only safe when attached to a repository?
    • RBAC demands this workflow anyway.
    • How about a feature flag ALLOW_NAKED_UPLOAD?
    • thoughts
      • could get rid of touch()
      • sync() would need to change (since it leaves things around until create-version)
        • we can already break sync (run cleanup/prot-time=0 during sync)
  • review needed:
  • slack topic gets outdated (was showing pulpcore 3.23 the other day)
    • worth adding this step to the release automation? or exclude the pulpcore number from the topic?
    • not just Slack - matrix as well - done, removed the release version number
    • Mastodon(?) and Twitter as well
      • mastodon isn't there yet
    • is it worth having this at all?
    • let's just have the changelog?
    • what about a list of current-release-versions for core/all-plugins?
      • only if we can get this automatically
  • General performance degredation reported.
    • https://github.com/pulp/pulpcore/issues/3970
    • maybe add to the incoming content-performance-analysis?
    • major features added pre(3.22)/post(3.25):
      • domains
      • otel
      • django4
    • can we get the scripts/procedures used on the galaxy-side to evaluate?
    • AI: [gerrod] spin up a performance-analysis group
      • ask for interest in Team Mtg?
      • ggainey, lmjachky volunteer to be in the group? +1
    • need for this analysis is going to go from "important" to "on fire" pretty quickly
  • current-problem: latest pulp-glue is later than what pulpcore-wants
    - installing pulp-cli onto a pulp-instance breaks/doesn't install
    - what about broken-versions of pulp-cli on pypi?
    - need to yank (only pulp-cli)
    - [ ] 0.18.0
    - [ ] 0.19.0
    - [ ] 0.19.1
    - [ ] 0.19.2
    - [ ] 0.20.0
    - need to make pulpprojerct pypi perms be more-sane
    -

June 27, 2023

  • [COPIED] Logic operators in query parameter
    • Anticipated to only need "or", "for now"
    • __in-filters do not fit the bill.
    • create custom filter field (with or-lookups)
      • needs schema addition
      • need a separator (| may be good)
    • django-filters allow or with lists
      • breaking change
      • needs schema adjustment (bindings generation)
  • [COPIED] Need ability to filter content by a list of repository versions.
    • There is a desire to query packages accross domains.
      • Pulp api calls are supposed to only ever touch one domain.
      • A service could access the database directly.
      • Maybe add a query behind a feature flag.
      • This kind of query may be a performance concern (unrelated?).
      • We could enable a domain to list items from all domains.
      • We could add the old endpoints with list only ability.
        • This can be shut with a feature flag for real multi-tennant systems.

June 20, 2023

Pending AIs

  • AI: Dennis invite DavidN/JustinS to pulpcore meeting

June 13th, 2023

  • Logic operators in query parameter
  • Gerrod showed off to the team how cool the release versions script is
    • A Z-release is needed for 3.18, New Version: 3.18.20
    • A Z-release is needed for 3.21, New Version: 3.21.10
    • A Z-release is needed for 3.22, New Version: 3.22.7
    • A Z-release is needed for 3.23, New Version: 3.23.7
    • A new Y-release is needed!, New Version: 3.28.0
  • ZDU (zero downtime upgrades) https://github.com/pulp/pulpcore/issues/3278
    • There are open issues for safe zdu

Pending AIs

June 06, 2023

  • pulpcore

    • 3.28 (NEW)
    • 3.27 (current release)
    • 3.23 (galaxyNG/4.7)
    • 3.22 (katello/4.9)
    • 3.21 (katello/4.7, galaxyNG/4.6)
    • 3.18 (katello/4.5)
    • 3.16 (katello/4.3)

  • pulp_file

    • 1.15 (NEW)
    • 1.14 (current)
    • 1.12 (katello/4.9)
    • 1.11 (katello/4.7)
    • 1.10 (katello/4.3. 4.5)

May 30, 2023

  • pulpcore

    • 3.27 (NEW)
    • 3.26 (current release)
    • 3.23 (galaxyNG/4.7)
    • 3.22 (katello/4.9)
    • 3.21 (katello/4.7, galaxyNG/4.6)
    • 3.18 (katello/4.5)
    • 3.16 (katello/4.3)

  • pulp_file

    • 1.15 (NEW)
    • 1.14 (current)
    • 1.12 (katello/4.9)
    • 1.11 (katello/4.7)
    • 1.10 (katello/4.3. 4.5)

  • pulp 3.25 image

May 23, 2023

  • pulpcore

    • 3.27 (NEW)
    • 3.26 (current release)
    • 3.23 (galaxyNG/4.7)
    • 3.22 (katello/4.9)
    • 3.21 (katello/4.7, galaxyNG/4.6)
    • 3.18 (katello/4.5)
    • 3.16 (katello/4.3)

  • pulp_file

    • 1.15 (NEW)
    • 1.14 (current)
    • 1.12 (katello/4.9)
    • 1.11 (katello/4.7)
    • 1.10 (katello/4.3. 4.5)

  • Are the deprecation checks in the CI still useful?

    • on main?
      • Yes!
    • on release branches?
      • No, please disable when you see them.
      • Needing to actively ignore them is tedious.

May 16, 2023

  • pulpcore

    • 3.26 (NEW)
    • 3.25 (current release)
    • 3.23 (galaxyNG/4.7)
    • 3.22 (katello/4.9)
    • 3.21 (katello/4.7, galaxyNG/4.6)
    • 3.18 (katello/4.5)
    • 3.16 (katello/4.3)

  • pulp_file

    • 1.15 (NEW)
    • 1.14 (current)
    • 1.12 (katello/4.9)
    • 1.11 (katello/4.7)
    • 1.10 (katello/4.3. 4.5)
  • difficulty in getting core-PRs reviewed
    • think about this for next week's mtg
    • "should we continue requiring 2 reviewers"?
      • still ok, except when everyone is on PTO?
      • problem w/ finding 2 SMEs at all times, moreso than github-access-issue
      • even just "extra eyes" (ie, not SME eyes) is useful
  • "keeping branch warm" discussion
    • remove 1.13 from pulp_file update_ci_branches? - YES
    • remove 3.24 from pulpcore update_ci_branches? - Yes(ish)
    • "whatever is used in downstream" and "currently-released branch"
    • what will be the implications for our upstream?
      • will they just stay on "downstream" branches (making them even more like an LTS?)
    • should core/3.24 be an exception?
      • if so - we can add it when we decide we need to
    • We should delete the backport labels for cold branches.
    • AI: [ggainey] make sure release-process is updated to be explicit about this
  • Create a working-group to improve CI automation to support weekly releases
    • Still too many manual steps in release process, quite a burden
    • only dkliban and x9c4 on the "CI Team" currently
    • still many manual-steps at Y-release time
    • lots of improvements discussed, not enough ppl to pick them up
    • let's get a doc together and add ideas to it
    • who would like to champion for this?
      • gerrod to helm this
      • volunteers: ggainey, decko, x9c4
  • How to handle this migration:
  • This PR lacks a changelog and issue
  • OpenTelemetry is beta - discussion

May 9, 2023

  • pulpcore
    • 3.25 (NEW)
    • 3.24
    • 3.23
    • 3.22
    • 3.21
    • 3.18
    • 3.16
  • pulp_file
    • 1.15 (NEW)
    • 1.14
    • 1.13
    • 1.12
    • 1.11
    • 1.10
  • Issue discussion
    • https://github.com/pulp/pulp_rpm/issues/3134
    • add a repo-versions__in
    • how can we make this performant?
    • filters could use perf-analysis/fixes
    • needs a POC to understand implications
      • materialized views? (django? or postgres?)
  • A proposal from telemetry workgroup
    • current state
      • Dependencies declared in the pulp-ci-centos build
      • Code missing in the pulpcore's main branch. We can't merge the code if we don't add the opentelemetry basic stack as a direct dependency on pulpcore's requirements.
      • A new PR for OpenTelemetry aiohttp-server instrumentation project.
        • needed for pulp-content instrumentation
      • pulp-api is in a great spot
      • aiohttp is pretty good, but we really want it released upstream to consume
      • nothing for tasking yet - phase-2 perhaps?
      • Got a oci-env profile for development purposes.
        • Still not merged given some instabilities with CI's docker tests.
          • need some help/eyes
    • The new proposal
      • Given the new pulpcore release process
        • "We can undo it later if needed"
      • Phase 1
        • We want to add a trimmed opentelemetry stack list of dependencies directly into pulpcore.
          • opentelemetry-distro[otlp]>=0.38b0,<=0.38b0
          • opentelemetry-exporter-otlp-proto-http>=1.17.0,<=1.17.0
          • opentelemetry-instrumentation-django>=0.38b0,<=0.38b0
          • opentelemetry-instrumentation-wsgi>=0.38b0,<=0.38b0
        • We have a PR open for instrumenting pulp-api
        • We are about to open a PR for opentelemetry-instrumentation-aiohttp-server
        • Will have a PR for instrumenting pulp-content after getting the otel PR merged
          • We can release a package with the instrumentator if the PR got stalled for some reason(ex: got no reviewers)
          • Or we can add it directly into pulpcore code base(vendor it?)
      • Phase 2
        • instrument the workers?
        • Any specific pulp related code?
    • discussion
    • decision:
      • merge reqs/PR to pulpcore
      • merge otel-work in oci_env
      • release as soon as it's merged (3.25? .26?)
  • difficulty in getting core-PRs reviewed
    • think about this for next week's mtg
  • 3.25 Today?
  • Can we make a bot or something to tell us when there are un-released commits on our branches
    • this would be great!
    • maybe a cmd to be run on a local copy of the repo?
    • a tues-workflow to check branches and auto-release?
    • let's gather some info and anyone who's Very Excited about this can start pulling something together
  • discussion around drop-trailing-slash PR
    • gerrod's refactoring would make this easier to implement
    • do we hold up 3.25 for this, or not
    • if we're going to change this to opt-in, it can happen "whenever"
    • decision: remove from blocker-list, release core/3.25

May 2nd, 2023

  • Weekly release check-in
    • pulpcore
      • 3.24
      • 3.23
      • 3.22
      • 3.21
      • 3.18
      • 3.16
    • pulp_file
      • 1.14
  • core/3.24
  • Let's nail down what cadence we will use as "number of Y-releases without breaking changes"
  • With the new release policy, is it still useful to have tests for new pulpcore features living in pulp_file only?
    • In the long run, it does not matter if the tests are in pulpcore or pulp_file.
    • "Test should come with the feature" suggests adding them to pulpcore if it's a pulpcore only change.
    • What was the original reason to move tests using pulp_file to the plugin?
      • mdellweg's memory: We had compatibility issues with all the branches moving around. But we fixed that and can now depend on stable pulp_file releases.
    • if there is any code-change in pulp_file for a Thing, then its test needs to live in pulp_file
    • if there is no pulp_file code-change (e.g., core is adding a Thing that pulp_file gets "for free" due to inheritance), then the test for it "can" live in pulpcore
    • this own't solve the problem - pulp_file changes that require core-changes still collide oddly/badly
    • what about fixtures? can you use pulp_file fixtures from inside pulpcore?
      • yes if pulp_file is a pytest plugin
    • discussion: should core and file be the same project?

April 25, 2023

April 18 (JD 2 460 052) (1681822800)

  • Weekly release check-in
    • pulpcore
      • 3.24
      • 3.23
      • 3.22
      • 3.21
      • 3.18
      • 3.16
    • pulp_file
      • 1.14
  • Django 4.2 updates
  • move from Twitter to mastodon? (fosstodon.org maybe?)
    • AI: [ggainey] bring up at Team mtg next week
  • https://github.com/pulp/pulpcore/issues/3710 needs discussion
    • Labels? - not available at repo-version level
      • maybe start w/ Labels, and see how far we can get?
    • AH/repo-mgt - already talked about repo-group concept?
      • Pulp2 had something like
    • What about repo-version being a piece of content that could be held by a repository? (RepoCeption!)
      • need a content-object that is-a repo-version?
    • plugin implications?
  • Docs issue in plugin CI's
    • plugin docs-build (pulp_file for sure)
    • we install pulp "outside" the container
    • conflicts w/ cli/glue, pip-installing, plugin-vs-core docs-requirements conflicts
    • have a bandaid in-place(ish) for core
    • need some thought on how to fix the problem "appropriately"
    • root-cause(ish): plugin-doc-build requires pulpcore-doc-requirements-install, and then plugin-doc-requirements-install, and core can therefore break the plugins w/out knowing
    • AI: [mdellweg] to investigate better workaround in template
    • AI: [ggainey] report back on any breakage in today's pulp_file release (attempt)

April 11, 2023

  • Circular CI testing dependencies
  • List endpoint optimizations
  • [ggainey] pulpcore z-releases today
    • 3.23.2, 3.22.4, 3.21.7, 3.18.17, 3.16.17
    • let ggainey know if he should wait, before noon-GMT-5 today
  • [decko] pulp-maven permissions
    • need github perms to at least re-run actions
    • same as merge-pull-request perms
    • who has access? - ttereshc to fix
    • AI: [ggainey] update our permissions-matrix so we have some idea of who can do what, where

April 4, 2023

  1. Django 4.2 released
    • Possibility to upgrade to Psycopg3
    • ASGI vs WSGI
    • decko is working on OTEL :(
    • Must to line up with 3.25, and "before August" to meet a stakeholder ask
    • Make Postgres/12+ required as well
    • do we have a prio-list issue?
    • x9c4 to pick up the work?
  2. When to release z-streams
    • If there's anything unreleased on a Tuesday, Push The Button
  3. Release pulpcore?
  • only z-releases
  1. Release pulp_file?
  • pytest plugin PR needs a release
  • release 1.14 to get PulpReplicator out

March 28, 2023

  1. Do we want to run the test suite in the release pipeline?
    • we run it before merging every commit
    • thoughts: we want faster releases, and we can always put this back if it starts causing us issues
    • AI: [mdellweg] remove it
  2. Do we want to run the lowerbounds CI check with == or ~= pins?
  3. Can we raise the lower bound (Z version) of a requirement in a plugin when releasing a bug fix?
    • pulpcore Z version bump when releasing a plugin
    • regression testing assessment (problems with accepting bug fixes)
  4. More friendly docs needed for certguard?
  5. Are we ready to stop publishing anything nightly?
    • we don't publish clients any more, do publish docs
    • docs updated at release-time
    • changelog updated when that file changes
    • w/out nightly-docs, need to point into github directly (which is ok)
    • what is generating /latest/ ?
      • need to make sure that works first
    • AI: [mdellweg] investigate and stop doing nightly-publish if we can

March 21, 2023

  • Can we delete all the nightly published bindings from pypi and rubygem?
    • I do not see any way they are useful.
    • There are a lot.
    • At best they are compatible with a specific commit per day.
    • AI: [ggainey] bring up at katello integration mtg
    • AI: [somebody] PyPi/rubygem.org needs to be cleaned up
  • AI: [somebody] get quba42 access to pulp_deb & pulp-cli-deb & pulp-glue-deb on pypi
  • We need to bump the pulpcore version in pulp_file (and pulp-certguard) and perform a backport/release
  • please review + merge + release this PR for pulp_file
    • an ask from @bmbouter to allow upstream testing of the 3.23 replication feature
  • file/1.13 and lower-bounds
    • create a PR by hand to update release branch
    • yank 1.13.0 from pypi/rubygem
  • discussion around PyPI permissions
    • Fun!
  • why is decko not marked as a "Developer" on discourse? Let's fix!
  • CI Update is stuck on certguard tests not being domain compatible
    • domains-and-certguard-tests doesn't work currently
    • certguard isn't pytest-style tests yet
    • let's separate concerns - do 2 CI changes
  • discussion: should we be shipping tests in the pypi installs?
    • prob not "expected" behavior
    • currently we do
    • originally done to support django-unit-tests (which expect things to be part of the app)
    • want tests to be runnable in pulp-users' integration CIs
  • discussion around Django/4 investigation
  • discussion around pulling together an Official List of who has access to what/clean house

March 14, 2023

  • release core/3.23 today?

    • yes please!
    • AI: ggainey to turn the release crank today

  • moved some AIs from dkliban/bmbouter to decko (lucky man)

  • opentelemetry scope-of-work discussion

    • upstream user-interest/conversations
      • talk to decko if you want to be involved
    • developers
    • Scope of Work
      • Develop and merge OTEL instrumentation in pulpcore
        • code-changes only in pulpcore-workers
        • autoinstrumenting handling Other Parts (wsgi, aiohttp, postgres)
      • Write user-facing pulpcore documentation on how to configure metrics and tracing
      • Ensure oci-env has a profile that fully sets up prometheus, jaeger, grafana, and collector
      • Record 5-minute demo showing metrics collection with Grafana and Prometheus (pulp-administrator-facing)
      • Record 5-minute demo showing tracing with Jaeger (developer-facing)
      • demos need to be sent to youtube-admin and then publicized

  • open telemetry performance update

  • open telemetry dependency for pulp

    • would include the various autoinstrumentation pkgs (postgres, redis, wsgi, etc) and utilities
    • prob ~10ish new pkgs
    • decision needs to be made on whether pulp requires these "always" or "optionally"
      • needs discussion and decision
    • even if deps are present, maybe env-var to turn on/off?
    • is operator interested?
      • yes - primarily worker-metrics, for scaling decisions
      • kubernetes envs will handle scaling-due-to-hardware-issues (ie, CPU-utilization)

  • Can this be reviewed/merged before 3.23 release?

  • What about this one:

  • observation/evaluation of https://analytics.pulpproject.org/ status

    • discussion around understanding why so many "old" core-versions
    • should/can we think about which "downstream" is reporting analytics?
    • lots of discussion around how to interpret results

March 7, 2023

  • Can we get a Pulp/Katello Integration Mtg representative to cover dkliban's leave?
    • ggainey is It
    • [AI] ggainey to suggest every-other-week for this mtg
  • core/3.23 - release when?
    • domains PR almost merged - today
    • pulp-cli/glue will have a release
    • then pulpcore
    • then pulp_file
  • calVer recap
    • current thinking: let's just release faster and not switch to calver right now
    • not possible anyway before 3.25
  • Next breaking change Release after 3.25?
  • Analytics proposal

February 28, 2023

  • gerrod is out on pto, Matthias is meeting lead
  • Upgrading Django, 4.2 (LTS) is coming in April
    • 3.2 upstream support is till Dec 1, 2023.
    • Any known changes which affect us?
    • AAP is asking for 4.1 even now, pushing back to wait for 4.2
    • With django 4.* and django-channels, we may be able to consolidate the codebase (sync and async parts) on django.
    • Do migrations break with this change?
    • SUMMARY: We want to switch to 4.2 as soon as possible. Pulpcore 3.25 is the best opportunity. If we find FTE, someone should experiment with 4.1 asap.
    • [AI DKliban] Look for an issue; add it to 3.25 blockers.
    • [AI DKliban] advertise the Django change on Discourse.
  • crazy idea time: Can we turn Upload into a content type?
    • Chunks are really "just" artifacts.
    • We'd get orphan cleanup for free.
    • One less type of object to store in the storage bucket.
    • [AI MDellweg] Write a story
  • Open Telemetry update
    • Do we want this to be feature flagged?
      • YES
    • How do you want it turned on/off?
      • Question of deployment
    • Should it be on-by-default?
    • Can we make it an optional dependency?
    • [AI BBouters] Present <= 10' about telemetry
  • What is the process to add metrics to analytics?

February 21, 2023

February 14

  • Faster release schedule (and no more waiting on blockers) is needed if we want our cloud deployment downstreams to not deploy from the main branch.

    • Are migrations guaranteed to be stable between adding to main and releasing?
    • We could defer merging PRs with migrations until later in the cycle?
      • Might not be a later phase if we are releasing faster. Need to review faster
    • Goal is to release often so we don't run into a scenario where we want to release a plugin that is waiting on a pulpcore release
      • Cloud services might end up using pulpcore from main anyway
        • But they may also consume pulp as a library from only ga versions.
    • Release every week?
      • Y release if features present, else will be a z-stream release
      • What about bugfixes? How does this impact backporting?
      • Could be a backport nightmare, need to have unofficial LTS versions that we get products to share
      • Need to choose a new larger future compatible pulpcore version for plugins to declare compatibility with, maybe 10 versions: 3.35?
        • Potentially change pulpcore versioning to date base format?
      • [AI] dkliban to write a discourse post discussing how many new versions we want to declare future compatibility against

  • [lmjachky] Review of https://github.com/pulp/pulpcore/issues/3429

    • Again, my take on this is that we can mark almost all of the features in tech preview as production ready NOW.
    • https://discourse.pulpproject.org/t/features-in-tech-preview/737
      • Based on the comments, we have direct feedback from users who request features. Why do we need analytics first in order to mark the features as production ready?

  • Let's use merge queues for Pulpcore

  • Stream runner (analysis)

  • Replication PR

January 31

January 24

  • [dkliban] to update pulpcore release docs to include OCI images release information
  • Pulp replica discussion
    • fun with repo-attached Distributions and auto-publish

January 17

January 10

January 3, 2023

  • "stream" test is really unreliable
    • we need to fix this problem, or disable the unreliable tests - they teach us to "ignore red"
    • [team] decided to keep the stream job on and leave this work for Michal, as planned. The bug is sporadic so should not have high impact
  • Upstream high-value tickets, should we prio-list them?
  • [ttereshc] added Mike and Humberto as optional attendees when and if they have some questions or need clarifications on pulp architecture, or pulpcore related topics, which are easier to bring up over a call. If you think, it's a wrong venue, blame me :), and I'm open for suggestions.
Last changed by 
 
Pulp
Design/Discussion/Meeting minutes for https://pulpproject.org/
0
78

Read more

Pulpcore team meeting

Core SME List

Apr 29, 2025
Pulpcore/katello/pulp_deb integration meeting

See also: https://discourse.pulpproject.org/t/katello-pulp-community-integration-meeting-minutes/39

Apr 16, 2025
CLI Team

https://github.com/pulp/pulp-cli/pulls

Apr 9, 2025
RPM meeting

discuss adopting zero-downtime-migration strategy

Mar 6, 2025
Read more from Pulp
Published on HackMD