# Pulpcore 2021 Archive
# December 14th, 2021
* [bmbouter] expanding pulpcore committers to SMEs
* https://discourse.pulpproject.org/t/expanding-pulpcore-committers/266
* [bmbouter] feature proposal for 3.18
* https://discourse.pulpproject.org/t/reducing-memory-usage-during-sync/277
* https://pulp.plan.io/issues/9635
* (dalley) related to memory, we should address https://pulp.plan.io/issues/9262
# December 7, 2021
* [ipanova] Are we going to participate in the upcoming conferences? Devconf and Fosdem. Do we plan any booths or just talks/workshops
* dkliban will submit a talk to FOSDEM on using the single container
* [ipanova] https://pulp.plan.io/issues/9603 reclaim disk space for whole pulp, katello would benefit from this
* The proposed solution looks good
* [bmbouter] interested in adding an additional commiter
* [bmbouter] forming telemetry working group in Jan
* https://discourse.pulpproject.org/t/proposal-telemetry/259/2
* [mdellweg] Do we want to make 3.20 a remove deprecations only release?
* let's see what happens
* probably not
# November 30, 2021
* Custom download factory
* https://github.com/pulp/pulp_container/pull/436
* Roles PR broke pulp_container
* https://github.com/pulp/pulp_container/pull/440
* should we have a deprecation for this change?
# November 23, 2021
* branch: master -> main
* pulpcore docs x plugin docs x installer docs
* https://docs.pulpproject.org/pulpcore/installation/storage.html
* https://docs.pulpproject.org/pulp_installer/objectstorage/
* from pulpcore and pulp_installer docs, it seems all plugins support azure and S3
* should plugins have some kind of feature/support table?
* object storage is just one example, it may be happening with other features
* [Idea] create a working group + melanie who will meet X times and work through docs re-structuring. When? Who?
* Yes/No on blog post declaring 3.20 as the next plugin API breaking-change release
* https://github.com/pulp/pulpproject.org/pull/399
* outcome: Decided yes to merge it
* Discussion about switching to aiohttp.server for tests
* https://github.com/pulp/pulpcore/pull/1728/files
# November 16, 2021
* github migration
* pulpcore total issues - https://drive.google.com/drive/folders/12jpPh84YBMzI035gSq9oKwFmuysrhhTw?usp=sharing
* NEW: 245
* ASSIGNED: 11
* POST: 20
* pulp-operator - https://github.com/pulp/pulp-operator/issues
* pulp_npm - https://github.com/pulp/pulp_npm/issues
* https://pulp.plan.io/issues/9524.json
* any objections to avoid using /tmp?
* https://pulp.plan.io/issues/9569
* should we add a check into CI for that?
* open floor proposal: extending release timeline
* https://discourse.pulpproject.org/t/what-to-do-to-reduce-unecessary-plugin-compatibility-releases/236
* Roles PR
* Is this interface what we want to manage roles on objects?
* https://github.com/pulp/pulpcore/pull/1693
# November 2, 2021
* release pulp-file 1.10.1 - https://github.com/pulp/pulp_file/pull/593
# October 26, 2021
* 3.17 relese date - before or after Pulpcon?Nov 4 or Nov 16?
* Let's release on Nov 16th
* Docs day. November 3rd? https://pulp.plan.io/issues?query_id=174
* Yes!
* Feedback on Patchback proposal
* we need to move to github issues before we focus on automating the backports
# October 19, 2021
* today discourse is not loading any pages apart from the main one
* we still need to review the backporting proposal
* https://discourse.pulpproject.org/t/automating-backports/153
* found reviewers for some pulpcore PRs, merged some dependabot minor verison bumps
* roles PR is being verified and adjusted based on pulp_container tests running with it
* https://github.com/pulp/pulpcore/pull/1627
# October 12, 2021
* Removing `mirror` parameter from pulpcore `RepositorySyncURLSerializer`
* Doesn't make sense for all plugins
* Not flexible enough for some plugins
* Proposal: Remove from pulpcore, have plugins provide it for API backwards compatibility
* Deprecate + add to plugins in 3.17, remove in 3.18
* Some discussion here:
* https://pulp.plan.io/issues/9316#note-20
* David - anything before I leave?
* Will give up commit bit
# October 5, 2021
* https://pulp.plan.io/issues/9289#note-5
* Will move to a separate repository and will try to simply tag it without releasing to PyPI
* 3.14 tagged all-in-one container image (was) a month old, should we trigger / automate them more frequently?
* 3.16.0 release
* ACS prs are merged
* plugins will check on compatibility during Open Floor today
# September 28, 2021
* Proposal: Redis should be optional by default
* https://pulp.plan.io/issues/8997#note-15
* Supported auth in url format?
* https://bugzilla.redhat.com/show_bug.cgi?id=2007655
* Looks like pulp_file supports it but not pulp_rpm
* Migration plugin migrates url to username/password
* Reject urls with auth (validation in pulpcore)
* Check-in on Roles progress
* models are finished
* permissions are being checked.
* some helper functions for queryset filtering and checking permissions
* creation_hooks (formerly known as permissions_assignment) with hooks for roles
* roles are part of the access policy
* may change due to concerns
* want to allow admins to modify roles, but not the rest of access policy
* options
* keep it that way
* move default roles as a class attribute in the viewset; post migrate signal; add REST for roles +1
* add REST for roles, ship default as a migration
* rewritten access policy for tasks as an example
* Still need API views for listing Roles, to assign roles, and to remove roles
# September 21, 2021
* Roles targeted for 3.16?
* Need buy in on the approach taken here: https://github.com/pulp/pulpcore/pull/1623
# September 14, 2021
* Relaxing the cryptopgraphy requirement https://pulp.plan.io/issues/9367
* Something like ">=3.1,<4"
* FYI: added the ability to return a task_group from viewsets
* https://github.com/pulp/pulpcore/pull/1618
* disable django_allow_unsafe for content app https://pulp.plan.io/issues/9354
* DB encryption key rotation
* Is there a way to change the key used to encrypt passwords in the database?
* Not currently, no
* Can we provide a pulpcore-manager command? Needs to reencrypt all entries.
* +1 from me
# September 7, 2021
* More reports from Katello users of deadlocks https://pulp.plan.io/issues/8750
* need to test with 3.15 fixes
* meeting lead
* dkliban starts next week
## Action items
* [ipanova] reach out to Ansible folks about retained_versions testing
* [bmbouter] comment on https://github.com/pulp/pulpcore/pull/1385
* [ttereshc, bmbouter] review ACS PRs opened by daviddavis
# August 31, 2021
* JSONField fix https://github.com/pulp/pulpcore/pull/1585
* pulp_ansible needs this released today
* it affects plugins that have JSONField and import/export
* mdellweg to review [done], daviddavis to release
* DB connection issues in content app https://pulp.plan.io/issues/9276
* aligned to 3.16, needs a reproducer
* UI for OpenAPI schema
* /pulp/api/v3/
* You need to login to see the entire schema
* Currently not tested
* Does any stakeholder uses it?
* Should we remove?
* https://github.com/pulp/plugin_template/pull/477/files#r695716549
* agreed to remove
* Download problems:
* pulp might have problems handling Content-Encoding
* https://pulp.plan.io/issues/9213?pn=1#note-7
* https://bugzilla.redhat.com/show_bug.cgi?id=1994275#c14
* I'm guessing we didn't catch this because we only test the file plugin with .iso, no .txt, .gz or other file types
* dkliban to investigate
* users are still hitting timeouts in some situations
* https://pulp.plan.io/issues/9301
* https://bugzilla.redhat.com/show_bug.cgi?id=1993773
* https proxy support needed for Katello by the end of September
* https://pulp.plan.io/issues/9304
* https://bugzilla.redhat.com/show_bug.cgi?id=1993917
* bmbouter is on it
* pulp-manifest tool
* https://pulp.plan.io/issues/9289
* in a separate repo or as a part of pulp_file?
* currently lives in pulp_file
* pulpcore 3.16 release plans
* suggest Sep-28 at open floor
* What to do about https://github.com/pulp/community
* discussions are disabled, only wiki left
* discuss at open floor
* meeting lead
* dkliban will take over early, September 14th
## Action items
* [dkliban] to reach out to newswangerd/osapryki to get the list of proposed projects for token auth support and check with security teams
# August 24, 2021
* 3.14.z release
* waiting on the last item to be investigated (encoding issue observed in rpm but potentially a pulpcore problem)
* dalley to finish cherry-picks
* ttereshc to release
* fao89 to release installer
* Move docs day to next week? +1
* 3.15 release on Thursday (tentative)
* everything needs to be merged by tomorrow EOB
# August 17, 2021
* 3.15 release is postponed till next week
* refactor code to stop running async code inside a transaction, dkliban is on it
* mdellweg to release next week
* Issues with gunicorn
* concurrent chunk upload, with concurrency 25+, e.g. docker push
* gunicorn prematurely closes the connection
* need a lead, might become urgent
* Docs day query https://tinyurl.com/uc59aa5k
* What we already have for the docs day https://tinyurl.com/aajxytrm
# August 10, 2021
* 3.15 pulpcore workers use 10 times as many connections to the DB during sync as they did in 3.14
* in 3.14 - 1 extra db connection during file repo sync
* in future 3.15, without the wrapper - extra 9 connections
* in future 3.15 with the wrapper - extra 12 connections (but overall number of connections can be limited)
* Full Django 3 support will possibly require a few new patches in core areas to address the connection issue
* Technically they might not fit the backport policy upstream
* We can always apply them downstream-only, but also..
* Without the model changes (using the new JSONField) there will be a tremendous number of deprecation warnings logged on startup
* Should we find a way to hide the warnings and proceed with backports or should we make 3.15 the minimum recommended version for supporting Django 3?
* No backports, 3.15 only for official support
* [osapryki] State of Token Authentication support in pulpcore.
* no progress, won't be in 3.15
* need to create a new plugin
* [osapryki] drf auth token has security concerns, encourage not to use it
* needs to be in 3.16
* pulp_file release with pulpcore 3.15 compatibility?
* let's wait for now, there are might be changes required for 3.15 compatibility
* many duplicate key errors in postresql logs
* no functionality is affected
* major pain is huge queries which fill up logs ans require a lot of space
* BZ https://bugzilla.redhat.com/show_bug.cgi?id=1991527
* is it intentional or do we overuse try/except?
* 3.14.4
* cherry picks are done
* dralley to release
## Action items
* [osapryki and dkliban] to get the list of proposed projects for token auth support and check with security teams
# August 3, 2021
* What is the status of ACS implementation?
* david - hoping to start on this after 3.15
* pulpcore's docs style: https://github.com/pulp/pulpcore/pull/1532
* Loosen the constraints around what can be in a new-repository-version
* https://hackmd.io/@ggainey/rel_path_braindump
* https://github.com/pulp/pulpcore/pull/1513/
* need to decide where to merge asap
* Docs day Aug 12?
* yes if the 3.15 release date doesn't slip; if it does, we'll adjust docs day daye as well
## Action items
* [dkliban] to close @dannysauer PRs
* [ipanova] to schedule docs day and prep a query for the next meeting
# July 27, 2021
* repair and reclaim disk features need plugin input in order to correctly function
* https://pulp.plan.io/issues/9128 and https://pulp.plan.io/issues/8305#note-16 [no objections to go forward]
* is there a way or in the first place should we prevent content with no Artifact and no RemoteArtifact from being added to new repository versions via one of the validation hooks? We want to keep the historical records around, but the content is no longer useful or functional, so it doesn't make sense to let it spread into new repository versions. [no immediate action, capture on the ticket --force flag which can be specified when new repo-version is being created to allow such content]
* Z releases, requested backports only vs all bug fixes go into the latest Z stream
* https://github.com/pulp/pulpcore/pull/1519#issuecomment-886555176
* Tasking system with shared resources
* https://pulp.plan.io/issues/9148
* pulp-list, handle-artifact-checksums issue
* https://listman.redhat.com/archives/pulp-list/2021-July/msg00025.html
## Action Items
* [ttereshc] to file a task to automate more cherry-picking process
* [fao + dkliban] follow up with bin about upgrade issue
# July 20, 2021
* Backport to 3.14? https://github.com/pulp/pulpcore/pull/1484
* *outcome*: it's backported
* pulp_ci_centos image has Python 3.8 now
* Release branches use this, will be running against 3.8
* Is that a problem? We can make a 3.6 base image
* *outcome*: Don't pursue this for release branches
* https://github.com/pulp/pulp_file/pull/538
* Backport to pulp_file 1.8?
* *outcome*: yes to backport @daviddavis to do so
* django 3.2 - https://github.com/pulp/pulpcore/pull/1464
* *outcome*: merged
* Investigating how to backport migrations
* RBAC work update
* bmbouter to try to make
* `assign_role(role, user_or_group, obj=None)`
* `remove_role(role, user_or_group, obj=None)`
## Action Items
* [daviddavis] to backport https://github.com/pulp/pulp_file/pull/538
# July 13, 2021
* Using SECRET_KEY for db field encryption?
* https://github.com/pulp/pulpcore/pull/1301#discussion_r625923964
* https://github.com/pulp/pulp_installer/pull/669#issuecomment-871428386
* Don't use SECRET_KEY. It could be rotated, would be an overextension of it, etc
* Backporting fixes discussion: https://github.com/pulp/community/discussions/50
* Always use backports to get issues into z-stream releases
* Z-stream changelogs will only contain references to backports
* The cherrypick script handles this
* Z-stream milestones in redmine will only have backport issues
* Not the most optimal solution, revisit in the future?
* How to sort releases in the changelog (by date or release number?)
* Wait until we have user feedback
* Installer team should release pulp_installer
* Issues with the old tasking system are blocking Katello 3.18 (pulpcore 3.7) users
* https://pulp.plan.io/issues/8779
## Action items
* [david] to check with Katello/Satellite about https://pulp.plan.io/issues/8779
# July 6, 2021
* Rotate the release engineering role
* outcome: done, dkliban is 3.15 release lead
* how to allow users to have more control over the routes included in urls
* https://pulp.plan.io/issues/8898#note-2
* for example, needed here: https://docs.pulpproject.org/pulpcore/authentication/keycloak.html#python-social-auth-and-django
* outcome: everyone think it over
* katello indicated they would not recommend investing additional effort in the old tasking system
* Except insofar as it impacts katello 3.18
* what to do with the old epic? CLOSE - WONTFIX?
* https://pulp.plan.io/issues/8912
* outcome: will close WONTFIX and invite anyone to reopen, also recommend users upgrade
* Sync pipeline race condition when syncing repos w/ identical content
* https://pulp.plan.io/issues/8967
* I distinctly remember being able to reproduce this, and it's coming up in HTB, but I can't reproduce any more? Maybe just not enough attempts? But I don't remember this being explicitly fixed
* outcome: @dalley will try to reproduce some if he has time, not an official AI
## Action Items
* [dkliban] pick tentative 3.15 date and schedule first go/no-go meeting
# June 29 2021
* content app possible issue https://pulp.plan.io/issues/8911, https://github.com/pulp/pulp_rpm/pull/2018
* dkliban is lookin into this
* proposal: changing DefaultAccessPolicy in 3.15
* https://pulp.plan.io/issues/8974
* outcome: tag as 3.15 and we'll work on it
* Should we continue the old tasking system investigation?
## Action Items
* [bmbouter] to ask katello if they want pulp to continue investigating old-style tasking system
# June 22 2021
## Topics
* Making new-style tasking system the default with 3.14?
* [+1 from wibbit](https://matrix.to/#/!aVApiNMtnstWbwDcVU:matrix.org/$7ZrOemgXXCEzRR3xRwCCsqi8sfV1SIdf8_fy8FS9pq4?via=libera.chat&via=matrix.org&via=ctrl-c.liu.se)
* content app is shown as online for quite a while (~20 sec) after being stopped
* Confused by https://pulp.plan.io/versions/242
* I thought we were using milestones for releases?
## Action Items
* [ttereshc] file an issue to support graceful shutdown for content app
* [mdellweg] switch new-style tasking to be enabled by default and improve docs on how to switch between
* [ipanova] delete milestone https://pulp.plan.io/versions/242 and move onto 3.15
* [mdellweg] plan a deep dive for the new tasking system
# June 15 2021
## Topics
* Retaining EL7 support in the installer (and using python 3.8)
* https://github.com/pulp/community/discussions/3#discussioncomment-852713
* Adding retry support
* https://github.com/pulp/pulpcore/pull/1401
* Proposal make the new style tasking the default with 3.14
* let's ask wibbit what he thinks
* ask katello if they will upgrade to 3.15, and if it's later than 3.15 maybe upgrade with 3.15
* Need to update release guide and checklist templates
* Moving revised release guides to github community wiki
## Action Items
* [dkliban] to continue working with @dannysauer and help with their PRs
* [ttereshc] come up with a proposal for plugin removal
* [mdellweg and bmbouter] continue old tasking system investigation
* [dkliban] update and move release guide to GH community repo's wiki. email pulp-dev with link to new guide.
* [bmbouter] to file issue on how to generically load urls into pulp
# June 8 2021
## Topics
* release lead rotation
* bmbouter to handle 3.14, @dalley did 3.13 so he's moved to the end [DONE]
* sprint items review
* https://pulp.plan.io/issues?query_id=124
* any objections to add https://pulp.plan.io/issues/7297 blocks container import/export work
* [OUTCOME] decided to add to sprint, and @ipanova will
* [OUTCOME] Decided to not to the review now and instead wait for a final decision on the process
* Dropping Python 3.6 and upgrading to Django 3.2
* https://pulp.plan.io/issues/8853 and https://pulp.plan.io/issues/8488
* Packaging/Katello asked for target of 3.15
* https://pulp.plan.io/issues/8850
* aligned to 3.15
* PR auto-merge
* https://github.blog/changelog/2021-02-04-pull-request-auto-merge-is-now-generally-available/
* https://docs.github.com/en/github/collaborating-with-pull-requests/incorporating-changes-from-a-pull-request/automatically-merging-a-pull-request
* If we go that way, we should do right after a release to reduce the risk
* [NEXT STEP] think it over, unclear on timeline for adoption
* Thoughts on getting this into pulp_file 1.8
* https://github.com/pulp/pulp_file/pull/508
* Thoughts on deprecating `DELETE /pulp/api/v3/orphans/` in favor of `POST /pulp/api/v3/orphans/`
* https://github.com/pulp/pulpcore/pull/1394#discussion_r646741563
* [NEXT STEP] finalize decision via PR
* Tasking issues gain more attention in the downstream
* https://pulp.plan.io/issues/8779
* https://bugzilla.redhat.com/show_bug.cgi?id=1962462
* [NEXT STEP] mdellweg to schedule meeting with @bmbouter and @dalley
* Feedback needed for the logging issue/concern
* https://github.com/pulp/community/discussions/20
## Action Items
* [bmbouter & dalley] add tasking env var to django 3.2 epic (https://pulp.plan.io/issues/8488)
* [dkliban] to continue working with @dannysauer and help with their PRs
* [ttereshc] come up with a proposal for plugin removal
* [dalley and bmbouter] to review tasking system PR from @x9c4
* [dkliban and daviddavis] use new release automation to release pulp_file 1.8.0
* [mdellweg] setup a meeting for tasking issues
# June 1 2021
## Topics
* (RBAC) changes to the default access policy statements:
* new 'action' has been added https://github.com/pulp/pulp_container/pull/320
* existing condition has been modified https://github.com/pulp/pulp_container/pull/321
* Question: how to handle existing objects prior to such changes?
* pulp community feedback https://hackmd.io/8e4jBklyQ5Wnlc68-G1JdQ
* we whether start a thread/discussion with this input to gather more or use the survey Melanie is working on
* there is a slight preference towards the survey
* plugin removal https://pulp.plan.io/issues/7822
* can we prioritize it?
* users are trying to come up with a workaround https://pulp.plan.io/issues/4944#note-5
* but data is still present in db
* and it can be a problem for some plugins (e.g. pulp-2to3-migration)
* if an unneeded plugin is not updated in time or just no longer maintained, users can't upgrade their pulpcore
* e.g. https://listman.redhat.com/archives/pulp-list/2021-June/msg00000.html
* Tasking system needs review
* https://github.com/pulp/pulpcore/pull/1261
* Wibbit is testing it in "production?"
## Action Items
* [bmbouter & dalley] add tasking env var to django 3.2 epic
* [dkliban] to engage contributor @dannysauer and help with their PRs
* [dkliban] file issue to update CI to use python 3.7 and the single container to use python 3.7
* [ipanova] set up a RBAC meeting
* [ipanova] reach out to Melanie and ask for a meeting to address Danny's feedback + team prefers survey
* [ttereshc] come up with a proposal for plugin removal
* [dalley and bmbouter] to review tasking system PR from @x9c4
# May 25, 2021
## Topics
* Is this a bug or intentional?
* https://pulp.plan.io/issues/8798
* [outcome] probably not intentional
* we cache these results to avoid recalculating them constantly
* Non-blocking orphan cleanup https://pulp.plan.io/issues/7659
* Plugin code needs to update timestamp
* [proposal] add `timestamp_of_interest` via migration in 3.14, then in 3.15 have orphan_cleanup become non-blocking and use `timestamp_of_interest`
* Django upgrade to 3.2
* https://pulp.plan.io/issues/8488
* django-guardian now resolved
* https://github.com/pulp/pulpcore/pull/1352#pullrequestreview-667076933
* PR resolves pulp's issues
* https://github.com/pulp/pulpcore/pull/1116
* [goal] attempt to switch to Django 3.2 soon, maybe with 3.14
* Challenges with Python 3.6
* dependencies are dropping support
* Making top-level Authentication page in docs
* https://pulp.plan.io/issues/8799
* Relate backport requests to the original issue
* https://docs.pulpproject.org/pulpcore/bugs-features.html#for-backport-requests
* will we benefit from automation here?
* Danny's feedback
## Action Items
* [dkliban] to engage contributor @dannysauer and help with their PRs
* [bmbouter & dalley] add tasking env var to django 3.2 epic
* [david] email asking users about python 3.6 and django 3.2
* [dkliban] file issue to update CI to use python 3.7 and the single container to use python 3.7
* [ipanova] reach out to Melanie w/r to Danny's feedback so we prepare it in a format that can be shared/discussed with a wide group/community
# May 18, 2021
## Topics
* Docs Backlog ( let's time box it 20mins)
* https://tinyurl.com/36hc2c9h
* 3.13.0 release moved to Monday May 24
* single container requires plugins to be compatible
* https://github.com/pulp/pulp-oci-images/blob/latest/pulp/Containerfile#L14-L22
* 3.11.2 in-progress
* Logging suggestion/concern
* https://bugzilla.redhat.com/show_bug.cgi?id=1961508
## Action Items
* [ipanova] to invite pulp docs day since we identified issues
* [dkliban] to engage contributor @dannysauer and help with their PRs
* [dalley] to move back the 3.13 release to may 24th
* [bmbouter, dalley, gerrod] to help round up 3.13 bocking Prs
* [dkliban] to review upgrade test docs in plugin_template and merge if good
* [x9c4] to review what plugins need releases prior to 3.13 release and bring up at open floor today
* [ttereshc] to file upstream logging bug related to BZ and invite discussion about how to handle
# May 11, 2021
## Topics
* Hold https://pulp.plan.io/issues/8386 for 3.14
* could potentially break plugins (pulp_rpm) that want 3.13
* Possible scheme to "backport idempotent data migrations"
* We cannot backport migrations, but we could wrap the migrating code into a post_migrate action on the backport only.
* This can only work for pure data migrations( no schema change).
* Being on the dot releases, the post_migrate code may be run arbitrarily often before the real migration (coming with the minor release) will run once. Therefore the code must be idempotent.
* The post_migrate code should not appear on master ever, because it may be incompatible with later database schemas.
* Response to checksum migration
* https://github.com/pulp/pulpcore/pull/1313#issuecomment-838075001
* considered 3 options:
* option (1): make it a no-op
* option (2): make it continue even if there are errors
* option (3): leave it as is
* question: should the migration also download content
* overall folks believe no, mostly because the common case is reducing the number of checksums not adding them
* decision: go with option (2) @daviddavis to update his PR and reply to the concern
* Docs Backlog ( let's time box it 20mins)
* https://tinyurl.com/36hc2c9h
* Idea/quesiton: should we add DRF token authentication to pulpcore, can pulp-container use it?
* Backports for django CVE fix
* 3.9 - 3.12 for pulp_deb
* katello & galaxy_ng need: 3.7, 3.9, 3.11
* bmbouter to release backports
## Action Items
[fao] write a report about upgrade tests and send on pulp-dev mailing list
# May 4, 2021
## Topics
* New pulpcore meeting lead?
* New lead rotation
* BaseDistribution drop migration problem
* https://pulp.plan.io/issues/8386#note-5
* Problem: using `run_before` prevents new plugins from being installed
* David to confirm that `run_before` won't work
* pulpcore meeting x open floor
* Goal: be more mindful of which topics can be discussed at open floor
* 3.13 - https://pulp.plan.io/versions/188
* dalley to release (bmbouter if release gets delayed past may 17)
* https://pulp.plan.io/issues/8656
* Django 2.2.21 introduced a change which breaks any workflows which use stages API
* filename validation for file field fails
* https://docs.djangoproject.com/en/2.2/releases/2.2.21/
* individual orphan removal needs grooming
* https://pulp.plan.io/issues/8658
* prep for Docs day
* Installer release
* https://github.com/pulp/pulp_installer/pull/568
* Lately installer has been released by pulpcore team members
* Feedback from pulpcore people are welcome!
## Action Items
* [david] test out run_before for https://pulp.plan.io/issues/8386#note-5
# April 27, 2021
## Topics
* Migration issue on galaxy
* https://github.com/pulp/pulpcore/pull/1174
* During migration it went out of memory
* Worries about migration breaking in the future when model(s) change
* Need better testing around upgrades
* 3.13 - https://pulp.plan.io/versions/188
* Revisit next week
* Move from `gnupg` to `pretty-bad-protocol`?
* https://github.com/isislovecruft/python-gnupg
* Hold off and look/ask for a list of "approved" crypto packages
* 3.12.2/3.7.6 to release
* Investigating another potential issue
* Katello hits this 75% of the time on el8 runs
* https://pulp.plan.io/issues/8603
* daviddavis will look into it
* Testing the waters on removing RemoteArtifact
* TL;DR what if we just use "Artifact" + a small table for remote sources, validated by constraints rather than a separate table schema.
* https://github.com/pulp/pulpcore/compare/master...dralley:delete-remoteartifact#diff-81f6a78175bb93934b6beff952646d3ca1ef3731f1ff14492d4ec77bfc3fdf82R224-R241
* Fixes https://pulp.plan.io/issues/8305
* Simplifies working w/ artifacts
* can filter Artifact by whether it's on_demand or not
* update artifacts in-place when files are downloaded
* delete "RemoteArtifactSaver" stage and _needed_remote_artifacts()
* I have a prototype that works for sync, should I invest more time in it?
* Review old quarter open issues
* https://tinyurl.com/pulpq12021
## Action Items
* [fao89] look at upgrade testing
* [fao89] find timeline for https://pulp.plan.io/issues/8192
# April 20, 2021
## Topics
* Demo videos?
* Was asked about a demo video for a feature
* I guess we're not doing these anymore?
* Fail to record anything due to capacity and commitments
* Record demos ad hoc?
* Ask feature writer to record demo when stakeholder asks for one
* Releasing https://pulp.plan.io/issues/8578
* Problem: BaseDistribution is deprecated but plugin api is missing RepositoryVersionRelatedField
* 3.12.1 or early/small 3.13.0?
* bumping minor release would cause extra churn
* 3.13 - https://pulp.plan.io/versions/188
* Needs date and release lead (rotation on lines 10-11)
* make distribution update synchronous https://pulp.plan.io/issues/7762
* rhui would like to dev freeze by may
* Pulp 2 to Pulp 3 enablement session for Satellite
* [fao] I need some onboarding
* Streamed endpoints
* https://github.com/pulp/pulp_ansible/pull/562
* faster
* query evaluation
* gunicorn timeout - blocks API worker
* Review old quarter open issues
* https://tinyurl.com/pulpq42020
## Action Items
* [david] release 3.12.1 with https://pulp.plan.io/issues/8578
* [dalley] confirm with rhui if https://pulp.plan.io/issues/7762 is still needed given auto-publish/auto-distribute and on the their Dev Freeze timeline
* https://pulp.plan.io/issues/7762#note-7
* will follow up by email
* [david] look at the pulp-oci-images and redmine integration
# April 13, 2021
## Topics
* Orphan endpoint
* As a user I want to see my orphaned content --> generic endpoint
* As a user I want to purge individual orphaned content
* As a user I want to purge orphaned content by type
* Whilst collection v1.2.3 is considered an orphan another v1.2.3 of the same collection can NOT be published into any repo in Automation Hub
* can it enabled in the galaxy viewsets by comparing digest of the uploaded binary data with the digest of the orphaned content ---> if matches, reject upload?
* Orphan cleanup proposal https://pulp.plan.io/issues/7659
* Have orphan cleanup also consider longest running task?
* Use timestamp_of_interest or longest running task(LRT) start time (whichever is greater)
* Leave out for now but recommend users don't set threshold above LRT
* pulp_file release for pulpcore 3.12/3.13 https://pulp.plan.io/issues/8547
* Need to fix deprecations
* reclaim disk space story https://pulp.plan.io/issues/8459
* ROSI announcement
* pulp-smash removal from PyPI
* https://github.com/pulp/Pulp-2-Tests/blob/013dffb728e4da5611b4b64e5310fe9cf1851479/setup.py#L43
* Dynaconf or configuration issue? https://bugzilla.redhat.com/show_bug.cgi?id=1937024#c11
## Action Items
* [david] update orphan cleanup proposal - https://pulp.plan.io/issues/7659
* [all] Look at ROSI announcement
* [all] Look at reclaim story https://pulp.plan.io/issues/8459
* [david] release pulp_file 1.7.0
* [david] ping ggainey about Pulp-2-Test's usage (of pulp-smash).
* Done
# April 6, 2021
## Topics
* Distribution migration
* problem removing the primary key on the reverse migration
* https://github.com/pulp/pulp_file/pull/495
* Changing a primary key: https://vivekanandxyz.wordpress.com/2019/02/25/changing-primary-key-in-django-postgresql-setup/
* Orphan cleanup
* https://pulp.plan.io/issues/7659
* Fixing this issue would mean not having to add global locks to new tasking system
## Action Items
* [all] review orphan cleanup proposal - https://pulp.plan.io/issues/7659
* Be prepared to discuss
* [david] move old notes to another hackmd
# March 30, 2021
## Previous AIs
* [bmbouter] to close https://github.com/pulp/pulpcore/pull/1122 based on the recent convo at pulp_ansible meeting
* Done
* [ttereshc] to review https://github.com/pulp/pulpcore/pull/1194
* Done, merged
## Topics
* another go/no-go meeting? thursday
* MasterModel distribution work update
* daviddavis to take over the completion of the work
* https://github.com/pulp/pulpcore/pull/1198/
* https://github.com/pulp/pulp_file/pull/484/
* bmbouter and mdellweg updating worker redesign PoC
* https://hackmd.io/Y0l9nBm9SFmPiIBXBz8lwQ
* pulp_file 1.6.1
* Going out today
## Actions Items
* [bmbouter] to close https://github.com/pulp/pulpcore/pull/1122 based on the recent convo at pulp_ansible meeting
* Done
# March 23, 2021
## Previous AIs
* ~~[ipanova] to set up meeting about content management (removal, rejecting, etc)~~
* ~~[bmbouter] to merge https://github.com/pulp/pulp/pull/4020/ and reply to thread~~
## Topics
* Do we need a pulp_file release for pulpcore 3.11?
* we have 1.6.0 which allows 3.11
* we need to release pulp_file before 3.12 is out (April 6 is the current tentative release date).
* Files are not being deleted from storage when calling the method delete()
* https://pulp.plan.io/issues/7316
* 3.12 or not?
* not a blocker, it's not a new problem or regression
* Repository version retention PR https://github.com/pulp/pulpcore/pull/1204
* Any thoughts/objections to only cleaning up repo versions on `__exit__`?
* And NOT when the repo is updated
* https://pulp.plan.io/issues/8368#note-7
* opinions split whether to trim versions on the repository update or not
* continue conversation on the PR and on the upcoming meeting for content cleanup improvements
* https://github.com/pulp/pulpcore/pull/1204#discussion_r599518868
## Action Items
* [bmbouter] to close https://github.com/pulp/pulpcore/pull/1122 based on the recent convo at pulp_ansible meeting
* [ttereshc] to review https://github.com/pulp/pulpcore/pull/1194
# March 16, 2021
## Previous AIs
[mdellweg] summarize "safety in deletion" of remotes feature onto https://pulp.plan.io/issues/8305 [done]
## Topics
* deleting and rejecting, timeline and scheduling?
* https://pulp.plan.io/issues/7659 run orphan clean up in parallel
* ask for deleting artifact but keeping the content
* keeping the record but freeing diskspace;
* delete collection-1.2.3 tarball and never allow collection-1.2.3 to be uploaded again
* version consistency + free disk space
* retain N repository versions, request for comment
* https://pulp.plan.io/issues/8368
* improving how users can identify orphans, request for comment
* https://pulp.plan.io/issues/8372
* should this merge? https://github.com/pulp/pulp/pull/4020/
* may simplify satellite carrying downstream patch if they only need to patch pulp_rpm. low-risk, small patch.
* Yes, merge it
* 3 month planning
## Action Items
* [ipanova] to set up meeting about content management (removal, rejecting, etc)
* [bmbouter] to merge https://github.com/pulp/pulp/pull/4020/ and reply to thread
# March 9, 2021
## Previous AIs
* [mdellweg] look for any docs issues that we might need for 3.11
* don't have time to complete for 3.11, so we skip and will revisit for 3.12
* [dalley] get more info about what happens remote gets deleted to revisit next week
* deleting a remote, then recreating the remote, then resyncing it, everything is fixed
## Topics
* Deleting a remote breaks content artifacts: https://pulp.plan.io/issues/8305
* add a validation for remote removal 1)check whether it is used in any of the RA or 2) linked to any repo 3) add the force flag for force removal
* deleting and rejecting
* https://pulp.plan.io/issues/7659 run orphan clean up in parallel
* retain N repository versions
* https://pulp.plan.io/issues/8368
## Action Items
[mdellweg] summarize "safety in deletion" of remotes feature onto https://pulp.plan.io/issues/8305 [done]
# March 2, 2021
## Previous AIs
* [mdellweg] ping quba42 about pulp_deb CI; Done
## Topics
* docs day for 3.11. Does it makes sense to schedule it this time? I don't think anyone(?) has made any docs PR for 3.10 docs day due to the workload[ipanova]
* Skip but mdellweg will check for any docs issues that we might need for 3.11
* Deleting a remote breaks content artifacts: https://pulp.plan.io/issues/8305
* Get more info (eg does recreating the remote fix everything?)
* Revisit next week
* should Pulp sync handle 429?
* It should currently
* removal of tech preview label .doc or .feature?
* We should be consistent. ttereshc demanded we use ".doc"
* Maybe introduce tech preview section in the future?
* https://github.com/pulp/pulpcore/pull/1086#issuecomment-788176427
* Checking remote artifacts for publications
* Pulpcore team to review PRs. Set up follow up discussion.
* worker timeouts
* HDD environments have WAL lock issues
* Removal (or not) of tech-preview labels in 3.11 and beyond!
* https://pulp.plan.io/versions/174
* Discuss at go/no-go meeting
## Action items
* [mdellweg] look for any docs issues that we might need for 3.11
* [dalley] get more info about what happens remote gets deleted to revisit next week
# February 23, 2021
## Previous action items
* [david] Close out https://pulp.plan.io/issues/5510 with comment directing them to labels
* Done
* [david] file/update issues about checks
* Done: https://pulp.plan.io/issues/8265
* [bmbouter] add galaxy tag to track galaxy requests
* Done
## Topics
* 3.11 proposal: Making worker timeouts configurable
* https://pulp.plan.io/issues/8291
* Going forward with status API change
* thanks to @mdellweg for rolling out a CI backwards compatability option
* only one not merged https://github.com/pulp/pulp_deb/pull/247
* CLI will get this PR next:
* https://github.com/pulp/pulp-cli/pull/162
* last: https://github.com/pulp/pulpcore/pull/1102
* 3.12 plugin API hard breaking change w/ Distributors
* drf_spectacular
* https://github.com/pulp/pulp_ansible/runs/1957957095?check_suite_focus=true#step:9:1314
* Investigate pulp_ansible failure
* may need to revert drf-spectacular version bump on pulpcore
*
## Action items
[mdellweg] ping quba42 about pulp_deb CI; Done
# February 16, 2021
## Previous action items
* [bmbouter] to revise https://pulp.plan.io/issues/8202 and https://pulp.plan.io/issues/8167
* Done
* [bmbouter] update working dir changes issue to include docs section about tempfile https://pulp.plan.io/issues/8231
* Done
* [fao] contribute back to drf_spectacular to simplify our code
## Topics
* Close out this issue? https://pulp.plan.io/issues/5510
* System checks
* https://github.com/pulp/pulpcore/blob/master/pulpcore/app/checks.py
* https://git.io/JtX46
* Migration plugin support
* Pulpcore broke migration plugin?
* How long do we support migration plugin?
* For the foreseeable future
* Review items on the sprint
* https://pulp.plan.io/projects/pulp/issues?query_id=124
* Removed items we don't think we'll get to in near future
## Action items
* [david] Close out https://pulp.plan.io/issues/5510 with comment directing them to labels
* [david] file/update issues about checks
* [bmbouter] add galaxy tag to track galaxy requests
# February 9, 2021
## Previous action items
None
## Topics
* https://pulp.plan.io/issues/8202
* Which fields need to be write_only?
* 3.11 release date? volunteer?
* March 2
* ipanova to release
* Discuss rotating the role
* working directory changes
* https://pulp.plan.io/issues/8231
* drf_spectacular
* https://pulp.plan.io/issues/8209
* how to avoid copy code to customize? Is it ok to contribute back to drf_spectacular?
* Content app performance
* https://pulp.plan.io/issues/8180
* At minimum, we need to tune some parameters, optimize codepaths
* We may want to investigate other means of serving content (or caching)
* RHUI and others want this anyway https://pulp.plan.io/issues/7810
Next steps:
* bring up at the katello integration meeting
* involve perf team to extend existing test plans
* start hackmd + schedule a meeting
## Action items
* [bmbouter] to revise https://pulp.plan.io/issues/8202 and https://pulp.plan.io/issues/8167
* update working dir changes issue to include docs section about tempfile https://pulp.plan.io/issues/8231
* [fao] contribute back to drf_spectacular to simplify our code
# February 2, 2021
## Previous action items
* [fao89] will port fr the rate limit code to pulpcore - david + brian will review
* Done: https://github.com/pulp/pulpcore/pull/1091
* [brian and matthias] will meet to talk about security scanners
* Done https://pulp.plan.io/issues/8088
* will send to pulp-dev to advertise for feedback
* [bmbouter] to file story for adding proxy_username and proxy_password and add to 3.11
* Done https://pulp.plan.io/issues/8167
## Topics
* Status API app name changing (bugfix, but backwards incompatible)
* https://github.com/pulp/pulpcore/pull/1102
* 4 options
* do nothing - things update and changed
* use the name attribute and strip off ".app" - this would get you back to the original
~~* have plugins change their label as part of 3.10 compatibility release~~
* defer??
* Decision: use labels
* Encrypting data at rest issue FYI ( get into pulpcore 3.12)
* https://pulp.plan.io/issues/8192
* Revisiting returning secrets in the API for authorized users
* rbac for artifacts
## Action items
None
# January 27, 2021
## Previous action items
## Topics
* Check in on wording for backport process https://github.com/pulp/pulpcore/pull/1089
* +1 from all
* Rate limiting in pulp_ansible - https://pulp.plan.io/issues/7965
* Move work to pulpcore
* FIPS - https://pulp.plan.io/issues/7960
* Add issues to sprint as we work on them
* ByteSore is interested in implementing https://pulp.plan.io/issues/8088 himself, wants to know how & where
* Groups access policy
* https://pulp.plan.io/issues/8159 ( release in 3.10)
* https://pulp.plan.io/issues/8160 ( release in 3.10)
* adding proxy_username and proxy_password to BaseRemote (release in 3.11)
* https://pulp.plan.io/issues/8058#note-5
* Create 3.11 story
* Content and RBAC
## Action items
* [fao89] will port fr the rate limit code to pulpcore - david + brian will review
* [brian and matthias] will meet to talk about security scanners
* [bmbouter] to file story for adding proxy_username and proxy_password and add to 3.11
# January 19, 2021
## Previous action items
* [bmbouter] to ask Satellite if they require this and when https://pulp.plan.io/issues/7683
* Satellite PM doesn't need. David to check with support.
* [ipanova] open story to add write api for users
* Done.
* [daviddavis] schedule fips check in meeting
* Done. Next meeting is Jan 28.
## Topics
* FIPS estimate for pulpcore work
* https://tinyurl.com/pulpcorefips
* Estimate: 2.5 FTE
* Release automation - https://pulp.plan.io/issues/8093
* Added task to automate the pre-release steps https://pulp.plan.io/issues/8119
* Proposal: start using github discussions
* https://github.com/orgs/pulp/teams/core
* let's not use it for now, let's instead explore using github issues as planned fall 2020
## Action Items
# January 12, 2021
## Previous action items
* [dkliban] to close out pulp_file 1.5.0
* Done
* [ttereshc] start discussion about LTS versions of pulp
* https://www.redhat.com/archives/pulp-dev/2021-January/msg00003.html
* Done
## Topics
* Download and verify - https://pulp.plan.io/issues/7683
* Needs to be implemented in pulpcore
* bmbouter to ask Satellite
* Deadline for https://pulp.plan.io/issues/7659?
* Would like to fix by march/april
* write api for users
* All agreed to proceed - AI open a story
* Restrict default permissions to admin users
* https://github.com/pulp/pulpcore/pull/1064
## Action Items
* [bmbouter] to ask Satellite if they require this and when https://pulp.plan.io/issues/7683
* [ipanova] open story to add write api for users
* [daviddavis] schedule fips check in meeting
# January 5, 2021
## Previous action items
* [dkliban] to release pulp_file 1.5.0
* [ttereshc] work on https://pulp.plan.io/issues/7869
* done https://github.com/pulp/plugin_template/pull/325
* [x9c4] write a task to have a test that searches for log messages with deprecation warnings
* https://pulp.plan.io/issues/7997 [Done]
* [x9c4] start a pulp-dev thread about RBAC - objects that do not have access policy are accesible by default
* Done
* https://github.com/pulp/pulpcore/pull/1064
* [dklban] to look at https://github.com/pulp/pulpcore/pull/799
* Done. Needs some installer changes.
* [ttereshc] update the release guide and plugin templates in plan.io clarify when to bump plugin's Y release.
* done https://pulp.plan.io/projects/pulp/wiki/Pulp3_Release_Guide#Criteria-to-have-the-Y-release-for-plugins
* needs feedback https://www.redhat.com/archives/pulp-dev/2020-December/msg00032.html
## Topics
* Move off Travis https://pulp.plan.io/issues/7859
* need a volunteer for the remaining task to publish docs on every push
* Backports https://pulp.plan.io/issues/7950 and https://pulp.plan.io/issues/7951
* on hold till the fixes are merged
* galaxy requirement: pulpcore>=3.7,<3.9 (backport will be needed in 3.7 and 3.8) https://github.com/ansible/galaxy_ng/blob/4.2.0/setup.py#L60
* Object labels https://pulp.plan.io/issues/7127
* Need a volunteer to deliver in pulpcore 3.10
* daviddavis: started on this
* Can we make the users, groups and permissions resources manageble by the rest-api?
* enables us to add cli commands for them
* Sounds good. Check with bmbouter.
* https://pulp.plan.io/issues/8010
* Should backports be backported to every (minor) version between the fix and the requested version
* We need an LTS, 3.7 is defacto an LTS version.
* Mailing list discussion
* new idna (sub-dependency) releases break pulp installation
* https://pulp.plan.io/issues/6169
* Follow up at installer meeting to ensure users are using the newest version of pip (has newest dependency solver)
* Signing service SMEs?
* matthias and dkliban
## Action Items
* [dkliban] to close out pulp_file 1.5.0
* [ttereshc] start discussion about LTS versions of pulp
* https://www.redhat.com/archives/pulp-dev/2021-January/msg00003.html
###### tags: `pulpcore`