Pulpcore team meeting

# Pulpcore team meeting ## Overview * [Core SME List](https://hackmd.io/@pulp/core_sme) * Release rotation (2 months) * CURRENT: dkliban - July, August * [dkliban] dalley bmbouter mdellweg ipanova ggainey decko gerrod * cheat sheet https://hackmd.io/gbTfH231RK-u1J2qjCcMLw * Meeting lead (2 months) * CURRENT: dalley - July, August * [dalley] bmbouter, dkliban, ggainey, gerrod, mdellweg, decko, ipanova * Open PRs * [core non-draft open](https://github.com/pulp/pulpcore/pulls?q=is%3Apr+is%3Aopen+draft%3Afalse+) * [file non-draft open](https://github.com/pulp/pulp_file/pulls?q=is%3Apr+is%3Aopen+draft%3Afalse) * Github issues in the last 7 days * https://github.com/pulp/pulpcore/issues?q=is%3Aissue+is%3Aopen+created%3A%3E2023-01-17 * Prio-list items * https://github.com/pulp/pulpcore/issues?q=is%3Aopen+is%3Aissue+label%3Aprio-list * Security Alerts: - https://github.com/pulp/pulpcore/security/code-scanning - https://github.com/pulp/pulp_file/security/code-scanning ## Meeting Agenda * Pending-AI review * Upcoming items * Open PR review * Prio-list review * moderator posts notes to [Discourse](https://discourse.pulpproject.org/t/pulpcore-meeting-minutes/36/) # Upcoming Agenda # Aug 15 * It's not easy to get a heartbeat in a gunicorn app. * I need suggestions. * https://github.com/pulp/pulpcore/pull/4249 * https://github.com/pulp/pulp-oci-images/pull/532 * https://docs.gunicorn.org/en/stable/custom.html * general consensus seems to be "no objection to the approach, but it needs to be socialized/documented to users before merging" -> discourse [AI mdellweg] * need to know "complete list" of current gunicorn options about which we care * comment in PR (dkliban) * Complex filtering * https://github.com/pulp/pulpcore/issues/2480 * https://github.com/pulp/pulpcore/issues/3914 * options: * simple or advanced? * feature flag or not? * there are some extant implementations * general consensus: mdellweg has a green light to continue his POC into pulpcore # Aug 1 * AI: [dkliban] open issue/start work on this * https://github.com/pulp/plugin_template/issues/774 - DONE * should we yank releases due to import issues? * import/export is broken on the last 2 z-streams of several branches * do we need to notify downstream users * improve import/export tests? * it's not a test-problem, it's Something Else (currently unknown) * would it be useful to have a nightly-test-run, in plugins, that would run against pulpcore/main (instead of against released-pulpcore) * this is in service of getting ahead of pulpcore-engine-changes that inadvertently break plugin-behvior, **before** releasing it into the wild * proposal: do so, **and** have a feature-merge-deadline for core of "Monday Afternoon GMT-5"? * discussion ensued * would be nice to catch core-changes earlier * more friction is Bad * this is an exception case we're addressing * run plugin tests against oci-image-from-source * make it a manual-only job? * prob not worth more than a day's work or so * if somebody is Real Excited About This - go4it! * otherwise, let's not # July 25 * [lmjachky] Should we leave the stream runner enabled? What are the pros and cons of having it? It is unreliable. * relies on sftp, which is not thread-safe * BUT - sftp is only storage that doesn't support redirection, which is what we have it for * can we have a specific test for this, that doesn't rely on the stream-runner? * the test exists for a reason, BUT... * discussion: having unreliable tests is bad, let's remove it * disable? or remove? * plugin-api-docs "DON'T DO THIS" section? * consensus: remove stream-section from plugin_template * AI: [dkliban] open issue/start work on this * [lmjachky] Do we want to review issues flagged as "Severity-High": https://github.com/pulp/pulpcore/labels/Severity%3A%20High? * first step: let's lose the labels left over from the redmine migration that we never use * consensus: there is plenty of on-fire stuff to do, let's not pick up things nobody is currently complaining about * We have a custom pytest hook in pulp-smash implemented in pulpcore. * is anyone using it? * Should we make the implementation failsafe for now? * Can we move the hook to pulpcore? * what does pytest do if a hook is implemented in two places? * if we remove this hook from pulpcore, the pulp-smash invocation will just do-nothing * let's Do This # July 18 * [decko] Use poetry for dependency management (and possibly as build-system???) * Can deal better with some conflicting scenarios * Lock file * Dependabot supports it * "One file to rule them all." * [dalley] poetry and "arbitrary/muiltiple plugins" didn't work well together previously * [mdellweg] need buy-in from Build Gang * [mdellweg] example problem: PyYaml/Cython collision just happened, poetry is having a lot of issues as a result * [dalley] is pip-new-dep-solve the default now? * we "think so"? (yes, 2021) * [ipanova] We have already done some PoC poetry evaluation time ago * https://github.com/pulp/pulp_file/pull/504 * https://github.com/pulp/pulp_file/pull/391 * at that time poetry did not work in editable mode which was main showstopper https://github.com/python-poetry/poetry/issues/34#issuecomment-586440285 * AI: [decko] can we get all this into a discourse thread please * tasks that leverage task groups can potentially block whole instance (i.e. import task) * Example of a problem: * import spawns a task for each repo * tasks line up and are handled one-per-worker * **everything else** lines up behind them * in a specific customer issue, 129 repos in a single import blocks task-processing for...a very long time * mdellweg/ggainey have been discussing already * discussion around workarounds possible * need to decouple "make improvements to import" from "make tasking-system more sophisticated" * AI: [ggainey] open issue to provide POC for import not using all-workers all the time * https://github.com/pulp/pulpcore/issues/4068 * can we make another working-group RE tasking system? * does anybody have the cycles right now? * AI: we need an RFE "allow cancelling a task-GROUP" * changelog entries and releases * entries got "messed up" on main branch * dkliban has a PR open to fix * there's possibly a non-atomic-problem happening w/ release? * was this a once-only problem due to release-workflow-problems? * 3.29.1 had a problem as well? * we prob need to have a discussion "soon" # July 11 - Pulp 4 when? - why - because there are "suboptimal design decisions" that we currently can't change due to API issues" - AI: we def need to start/have a list of things-that-would-change-the-API, that would drive this - when that list is "large enough", then it might be time - is there a way to change "broken apis" that is not an X-update? - taskgroup - https://github.com/pulp/pulpcore/issues/4009 - access-policy-access - content should always live in repositories - created resource in task is not so helpful as real return values - AI: let's get a discourse thread started on this - AI: let's discuss at PulpCON! - Can we teach users that content in Pulp is only safe when attached to a repository? - RBAC demands this workflow anyway. - How about a feature flag `ALLOW_NAKED_UPLOAD`? - thoughts - could get rid of touch() - sync() would need to change (since it leaves things around until create-version) - we can already break sync (run cleanup/prot-time=0 during sync) - review needed: - https://github.com/pulp/pulpcore/pull/3930 - https://github.com/pulp/pulpcore/pull/3996 - slack topic gets outdated (was showing pulpcore 3.23 the other day) - worth adding this step to the release automation? or exclude the pulpcore number from the topic? - not just Slack - matrix as well - done, removed the release version number - Mastodon(?) and Twitter as well - mastodon isn't there yet - is it worth having this at all? - let's just have the changelog? - what about a list of current-release-versions for core/all-plugins? - only if we can get this automatically - General performance degredation reported. - https://github.com/pulp/pulpcore/issues/3970 - maybe add to the incoming content-performance-analysis? - major features added pre(3.22)/post(3.25): - domains - otel - django4 - can we get the scripts/procedures used on the galaxy-side to evaluate? - AI: [gerrod] spin up a performance-analysis group - ask for interest in Team Mtg? - ggainey, lmjachky volunteer to be in the group? +1 - need for this analysis is going to go from "important" to "on fire" pretty quickly - current-problem: latest pulp-glue is later than what pulpcore-wants - installing pulp-cli onto a pulp-instance breaks/doesn't install - what about broken-versions of pulp-cli on pypi? - need to yank (only pulp-cli) - [ ] 0.18.0 - [ ] 0.19.0 - [ ] 0.19.1 - [ ] 0.19.2 - [ ] 0.20.0 - need to make pulpprojerct pypi perms be more-sane # June 27 - [COPIED] Logic operators in query parameter - Anticipated to only need "or", "for now" - `__in`-filters do not fit the bill. - create custom filter field (with or-lookups) - needs schema addition - need a separator (`|` may be good) - django-filters allow or with lists - breaking change - needs schema adjustment (bindings generation) - [COPIED] Need ability to filter content by a list of repository versions. - There is a desire to query packages accross domains. - Pulp api calls are supposed to only ever touch one domain. - A service could access the database directly. - Maybe add a query behind a feature flag. - This kind of query may be a performance concern (unrelated?). - We could enable a domain to list items from all domains. - We could add the `old` endpoints with list only ability. - This can be shut with a feature flag for real multi-tennant systems. # June 20, 2023 - Need ability to filter content by a list of repository versions. - [AI] Inviting JustingSH and DavidNE to the next meeting to discuss REST API filtering. We expect to see more feature requests talking about complex filtering, like: https://github.com/pulp/pulpcore/issues/2480 (strong concerns because of the future maintenance and workload capacity) - https://github.com/pulp/pulpcore/issues/3914 - https://github.com/pulp/pulpcore/issues/3806 # Pending AIs * AI: Dennis invite DavidN/JustinS to pulpcore meeting # June 13th * Logic operators in query parameter * https://github.com/pulp/pulpcore/issues/3914 * A design is needed. DNewswanger may help with usecases and implementation. * AI: Dennis invite DavidN/JustinS to pulpcore meeting * might be related https://github.com/pulp/pulpcore/issues/3806 * Gerrod showed off to the team how cool the release versions script is * A Z-release is needed for 3.18, New Version: 3.18.20 * A Z-release is needed for 3.21, New Version: 3.21.10 * A Z-release is needed for 3.22, New Version: 3.22.7 * A Z-release is needed for 3.23, New Version: 3.23.7 * A new Y-release is needed!, New Version: 3.28.0 * ZDU (zero downtime upgrades) https://github.com/pulp/pulpcore/issues/3278 * There are open issues for safe zdu # Pending AIs # June 06 * pulpcore - [ ] 3.28 (NEW) - [ ] 3.27 (current release) - [ ] 3.23 (galaxyNG/4.7) - [x] 3.22 (katello/4.9) - [x] 3.21 (katello/4.7, galaxyNG/4.6) - [x] 3.18 (katello/4.5) - [ ] 3.16 (katello/4.3) * pulp_file - [ ] 1.15 (NEW) - [ ] 1.14 (current) - [ ] 1.12 (katello/4.9) - [ ] 1.11 (katello/4.7) - [ ] 1.10 (katello/4.3. 4.5) # May 30 * pulpcore - [x] 3.27 (NEW) - [ ] 3.26 (current release) - [x] 3.23 (galaxyNG/4.7) - [x] 3.22 (katello/4.9) - [x] 3.21 (katello/4.7, galaxyNG/4.6) - [x] 3.18 (katello/4.5) - [ ] 3.16 (katello/4.3) * pulp_file - [ ] 1.15 (NEW) - [ ] 1.14 (current) - [ ] 1.12 (katello/4.9) - [ ] 1.11 (katello/4.7) - [ ] 1.10 (katello/4.3. 4.5) * pulp 3.25 image * https://github.com/pulp/pulp-oci-images/pull/481 * Also 3.26 image?NO # May 23 * pulpcore - [ ] 3.27 (NEW) - [ ] 3.26 (current release) - [ ] 3.23 (galaxyNG/4.7) - [ ] 3.22 (katello/4.9) - [ ] 3.21 (katello/4.7, galaxyNG/4.6) - [ ] 3.18 (katello/4.5) - [ ] 3.16 (katello/4.3) * pulp_file - [ ] 1.15 (NEW) - [ ] 1.14 (current) - [ ] 1.12 (katello/4.9) - [ ] 1.11 (katello/4.7) - [ ] 1.10 (katello/4.3. 4.5) * Are the deprecation checks in the CI still useful? * on main? * Yes! * on release branches? * No, please disable when you see them. * Needing to actively ignore them is tedious. # May 16 * pulpcore - [x] 3.26 (NEW) - [ ] 3.25 (current release) - [ ] 3.23 (galaxyNG/4.7) - [ ] 3.22 (katello/4.9) - [ ] 3.21 (katello/4.7, galaxyNG/4.6) - [ ] 3.18 (katello/4.5) - [ ] 3.16 (katello/4.3) * pulp_file - [ ] 1.15 (NEW) - [ ] 1.14 (current) - [ ] 1.12 (katello/4.9) - [ ] 1.11 (katello/4.7) - [ ] 1.10 (katello/4.3. 4.5)  * difficulty in getting core-PRs reviewed * think about this for next week's mtg * "should we continue requiring 2 reviewers"? * still ok, except when everyone is on PTO? * problem w/ finding 2 SMEs at all times, moreso than github-access-issue * even just "extra eyes" (ie, not SME eyes) is useful * "keeping branch warm" discussion * remove 1.13 from pulp_file update_ci_branches? - YES * remove 3.24 from pulpcore update_ci_branches? - Yes(ish) * "whatever is used in downstream" and "currently-released branch" * what will be the implications for our upstream? * will they just stay on "downstream" branches (making them even more like an LTS?) * should core/3.24 be an exception? * if so - we can add it when we decide we need to * We should delete the backport labels for cold branches. * AI: [ggainey] make sure release-process is updated to be explicit about this * Create a working-group to improve CI automation to support weekly releases * Still too many manual steps in release process, quite a burden * https://docs.pulpproject.org/pulpcore/en/main/nightly/release_process.html * only dkliban and x9c4 on the "CI Team" currently * still many manual-steps at Y-release time * lots of improvements discussed, not enough ppl to pick them up * let's get a doc together and add ideas to it * who would like to champion for this? * gerrod to helm this * volunteers: ggainey, decko, x9c4 * How to handle this migration: * https://github.com/pulp/pulpcore/pull/3768/files * requires specific SQL in migration to handle default 'correctly' in DB * we should stay consistent w/ Django metaphor (no strenuous objection) * This PR lacks a changelog and issue * https://github.com/pulp/pulpcore/pull/3632 * changelog and docs - yes please * AI: [decko] volunteered to do this work * cherrypick to release-branch * OpenTelemetry is beta - discussion * there was a LONG matrix discussion on this * https://matrix.to/#/!aVApiNMtnstWbwDcVU:matrix.org/$kWrHKE_GuChpCy_-mwJLqw0qXe2F7-0lMT4Ty2uQ63E?via=libera.chat&via=matrix.org&via=fedora.im * https://github.com/pulp/pulpcore/pull/3632#discussion_r1190196001 * lots of other projects use this already * in both doc and in code explain why we're relying on a "beta" dependency [decko] * [decko] will add missing changelog in his current PR * let's make sure we continue to be good about having discussions **exactly** like this one * we should be asking the Otel Group directly, "when are you going to be not-beta" # May 9 * pulpcore - [X] 3.25 (NEW) - [ ] 3.24 - [X] 3.23 - [ ] 3.22 - [ ] 3.21 - [ ] 3.18 - [ ] 3.16 * pulp_file - [ ] 1.15 (NEW) - [ ] 1.14 - [ ] 1.13 - [ ] 1.12 - [ ] 1.11 - [ ] 1.10 * Issue discussion * https://github.com/pulp/pulp_rpm/issues/3134 * add a repo-versions__in * how can we make this performant? * filters could use perf-analysis/fixes * needs a POC to understand implications * materialized views? (django? or postgres?) * A proposal from telemetry workgroup * current state * Dependencies declared in the pulp-ci-centos build * Code missing in the pulpcore's `main` branch. We can't merge the code if we don't add the opentelemetry basic stack as a direct dependency on `pulpcore`'s requirements. * A new PR for OpenTelemetry aiohttp-server instrumentation project. * needed for pulp-content instrumentation * pulp-api is in a great spot * aiohttp is pretty good, but we really want it released upstream to consume * nothing for tasking yet - phase-2 perhaps? * Got a oci-env profile for development purposes. * Still not merged given some instabilities with CI's docker tests. * need some help/eyes * The new proposal * Given the new `pulpcore` release process * "We can undo it later if needed" * Phase 1 * We want to add a trimmed opentelemetry stack list of dependencies directly into pulpcore. * opentelemetry-distro[otlp]>=0.38b0,<=0.38b0 * opentelemetry-exporter-otlp-proto-http>=1.17.0,<=1.17.0 * opentelemetry-instrumentation-django>=0.38b0,<=0.38b0 * opentelemetry-instrumentation-wsgi>=0.38b0,<=0.38b0 * We have a PR open for instrumenting pulp-api * https://github.com/pulp/pulpcore/pull/3632 * We are about to open a PR for opentelemetry-instrumentation-aiohttp-server * Will have a PR for instrumenting pulp-content after getting the otel PR merged * We can release a package with the instrumentator if the PR got stalled for some reason(ex: got no reviewers) * Or we can add it directly into pulpcore code base(vendor it?) * Phase 2 * instrument the workers? * Any specific pulp related code? * discussion * we need to get initial work out for larger use #soon * what's the cost to add dependencies? * "more dependencies" * performance-overhead even if not using? * https://github.com/open-telemetry/opentelemetry-python-contrib/issues/1556 * **tiny** impact * decision: * merge reqs/PR to pulpcore * merge otel-work in oci_env * release as soon as it's merged (3.25? .26?) * difficulty in getting core-PRs reviewed * think about this for next week's mtg * 3.25 Today? * Can we make a bot or something to tell us when there are un-released commits on our branches * this would be great! * maybe a cmd to be run on a local copy of the repo? * a tues-workflow to check branches and auto-release? * let's gather some info and anyone who's Very Excited about this can start pulling something together * discussion around drop-trailing-slash PR * gerrod's refactoring would make this easier to implement * do we hold up 3.25 for this, **or not** * if we're going to change this to opt-in, it can happen "whenever" * decision: remove from blocker-list, release core/3.25 # May 2nd * Weekly release check-in * pulpcore - [X] 3.24 - [X] 3.23 - [ ] 3.22 - [ ] 3.21 - [ ] 3.18 - [ ] 3.16 * pulp_file - [ ] 1.14 * core/3.24 * can/should we get https://github.com/pulp/pulpcore/pull/3770 reviewed/merged? * let's not be hasty * Let's nail down what cadence we will use as "number of Y-releases without breaking changes" * see previous notes https://hackmd.io/XfI9DxymSAu6UJnblh7prQ#March-7-2023 * suggestion: core/3.40 (15 versions post-3.25) * "probably" six-months-ish * see how well this works/revisit at that point * With the new release policy, is it still useful to have tests for new pulpcore features living in pulp_file only? * In the long run, it does not matter if the tests are in pulpcore or pulp_file. * "Test should come with the feature" suggests adding them to pulpcore if it's a pulpcore only change. * What was the original reason to move tests using pulp_file to the plugin? * mdellweg's memory: We had compatibility issues with all the branches moving around. But we fixed that and can now depend on stable pulp_file releases. * if there is any code-change in pulp_file for a Thing, then its test needs to live in pulp_file * if there is no pulp_file code-change (e.g., core is adding a Thing that pulp_file gets "for free" due to inheritance), then the test for it "can" live in pulpcore * this own't solve the problem - pulp_file changes that require core-changes still collide oddly/badly * what about fixtures? can you use pulp_file fixtures from inside pulpcore? * yes if pulp_file is a pytest plugin * discussion: should core and file be the same project? # April 25 * 3.24 release when? * Any more features that would need to go in? If not, we should release now and just have extra time for 3.25, the breaking changes release. * DB Key rotation (not a need). * https://github.com/pulp/pulpcore/pull/3765 * Not a blocker, but PR is ready * Hide distributions field (not a need) * https://github.com/pulp/pulpcore/issues/3538 * Not a blocker * https://github.com/pulp/pulpcore/issues/3023 is The Blocker * AI: [ggainey] go/nogo for 3.24 * Fri 28-APR # April 18 (JD 2 460 052) (1681822800) * Weekly release check-in * pulpcore - [ ] 3.24 - [X] 3.23 - [ ] 3.22 - [ ] 3.21 - [ ] 3.18 - [ ] 3.16 * pulp_file - [X] 1.14 * Django 4.2 updates * https://github.com/pulp/pulpcore/pull/3735 * https://github.com/pulp/pulpcore/pull/3751 * not-to-merge until after core/3.24 released * plugins will need to test against 3735 "soon" * move from Twitter to mastodon? (fosstodon.org maybe?) * AI: [ggainey] bring up at Team mtg next week * https://github.com/pulp/pulpcore/issues/3710 needs discussion * Labels? - not available at repo-version level * maybe start w/ Labels, and see how far we can get? * AH/repo-mgt - already talked about repo-group concept? * Pulp2 had something like * What about repo-version being a piece of content that could be held by a repository? (RepoCeption!) * need a content-object that is-a repo-version? * plugin implications? * Docs issue in plugin CI's * plugin docs-build (pulp_file for sure) * we install pulp "outside" the container * conflicts w/ cli/glue, pip-installing, plugin-vs-core docs-requirements conflicts * have a bandaid in-place(ish) for core * need some thought on how to fix the problem "appropriately" * root-cause(ish): plugin-doc-build requires pulpcore-doc-requirements-install, and then plugin-doc-requirements-install, and core can therefore break the plugins w/out knowing * AI: [mdellweg] to investigate better workaround in template * AI: [ggainey] report back on any breakage in today's pulp_file release (attempt) # April 11 * Circular CI testing dependencies * https://github.com/pulp/pulp_file/pull/943 * new feature breaks existing test - so core tests fail until pulp_file merge happens * revisit "all pulp_file tests must live in pulp_file"? * in pulpcore-PR, reference which git-commit to install pulp_file to show tests work * at merge time, revert * make test be version-dependent/aware? * do a pulp-file z-stream w/ JUST updated test? * AI: [any] review/merge/backport pulp_file PR * AI: [ggainey] get a 1.14.z released to unblock gerrod * should pulpcore/pulp_file be in The Same Project? * def needs its own discussion * see https://github.com/pulp/pulpcore/blob/main/docs/contributing/git.rst#requiring-other-pull-requests for current doc * List endpoint optimizations * https://github.com/pulp/pulpcore/issues/3711 * https://github.com/pulp/pulpcore/issues/3714 * memoizes pulp-type lookup to make a MAJOR performance gain * wait for requests to backport - but Do It! * AI: [ggainey] bring up to katello in nxt integration-mtg * [ggainey] pulpcore z-releases today * 3.23.2, 3.22.4, 3.21.7, 3.18.17, 3.16.17 * let ggainey know if he should wait, before noon-GMT-5 today * [decko] pulp-maven permissions * need github perms to at least re-run actions * same as merge-pull-request perms * who has access? - ttereshc to fix * AI: [ggainey] update our permissions-matrix so we have some idea of who can do what, where # April 4, 2023 1. Django 4.2 released - Possibility to upgrade to Psycopg3 - ASGI vs WSGI - decko is working on OTEL :( - Must to line up with 3.25, and "before August" to meet a stakeholder ask - Make Postgres/12+ required as well - do we have a prio-list issue? - https://github.com/pulp/pulpcore/issues/3660 - x9c4 to pick up the work? 1. When to release z-streams - If there's anything unreleased on a Tuesday, Push The Button 3. Release pulpcore? - only z-releases 4. Release pulp_file? - pytest plugin PR needs a release - release 1.14 to get PulpReplicator out # March 28, 2023 1. Do we want to run the test suite in the release pipeline? - we run it before merging every commit - thoughts: we want faster releases, and we can always put this back if it starts causing us issues - AI: [mdellweg] remove it 2. Do we want to run the lowerbounds CI check with == or ~= pins? - https://github.com/pulp/plugin_template/blob/b83f4e7b1a0754c1f0ae357fa5b0022425421553/templates/github/.ci/scripts/calc_deps_lowerbounds.py.j2#L18 - we often declare the pulpcore's requirement as pulpcore>=3.21,<3.25; the CI goes for 3.21.0 - should we install pulpcore v3.21.5 (the latest Z release) instead? - we have found a lowerbounds-compat-issue w/ the current behavior - plugin is compatible but tests started failing - decision: accept less-strict pins only on pulp-controlled components 3. Can we raise the lower bound (Z version) of a requirement in a plugin when releasing a bug fix? - pulpcore Z version bump when releasing a plugin - regression testing assessment (problems with accepting bug fixes) 4. More friendly docs needed for certguard? - https://github.com/pulp/pulp-certguard/issues/224 - AI: [ggainey] Adding pointers to How Certs Work is a good idea, add commentary to issue 5. Are we ready to stop publishing anything nightly? * we don't publish clients any more, do publish docs * docs updated at release-time * changelog updated when that file changes * w/out nightly-docs, need to point into github directly (which is ok) * what is generating /latest/ ? * need to make sure that works *first* * AI: [mdellweg] investigate and stop doing nightly-publish if we can # March 21, 2023 * Can we delete all the nightly published bindings from pypi and rubygem? * I do not see any way they are useful. * There are a _lot_. * At best they are compatible with a specific commit per day. * AI: [ggainey] bring up at katello integration mtg * AI: [somebody] PyPi/rubygem.org needs to be cleaned up * AI: [somebody] get quba42 access to pulp_deb & pulp-cli-deb & pulp-glue-deb on pypi * We need to bump the pulpcore version in pulp_file (and pulp-certguard) and perform a backport/release * as part of CI-update: https://github.com/pulp/pulp_file/pull/897 * certguard tests unhappy w/ domains * also (as its own issue) https://github.com/pulp/pulp_file/issues/917 * also (as part of Replication) : https://github.com/pulp/pulp_file/pull/910/files#diff-0240bdbfc39fb2be23c9353ffb227babc9d715ad5e0b8d27279b6567ed49e988R53 * pick **one** of the above to do this in * let pull/910 Do The Deed * pulp-certguard * do we really need to up the lower-bound? * probably not, doesn't really "have to be" * no need to update 1.6 to requiring core/3.23 * certguard lower-bound *does* need an update (still at 3.10?) * please review + merge + release [this PR](https://github.com/pulp/pulp_file/pull/910) for pulp_file * an ask from @bmbouter to allow upstream testing of the 3.23 replication feature * file/1.13 and lower-bounds * create a PR by hand to update release branch * yank 1.13.0 from pypi/rubygem * discussion around PyPI permissions * Fun! * why is decko not marked as a "Developer" on discourse? Let's fix! * CI Update is stuck on certguard tests not being domain compatible * domains-and-certguard-tests doesn't work currently * certguard isn't pytest-style tests yet * let's separate concerns - do 2 CI changes * discussion: should we be shipping tests in the pypi installs? * prob not "expected" behavior * currently we do * originally done to support django-unit-tests (which expect things to be part of the app) * want tests to be runnable in pulp-users' integration CIs * discussion around Django/4 investigation * https://github.com/pulp/pulpcore/pull/3618 * https://discourse.pulpproject.org/t/django-4-2-coming-with-pulpcore-3-25/781 * discussion around pulling together an Official List of who has access to what/clean house # March 14, 2023 * release core/3.23 today? * yes please! * AI: ggainey to turn the release crank today * moved some AIs from dkliban/bmbouter to decko (lucky man) * opentelemetry scope-of-work discussion * upstream user-interest/conversations * talk to decko if you want to be involved * developers * Scope of Work * Develop and merge OTEL instrumentation in pulpcore * code-changes **only** in pulpcore-workers * autoinstrumenting handling Other Parts (wsgi, aiohttp, postgres) * Write user-facing pulpcore documentation on how to configure metrics and tracing * Ensure oci-env has a profile that fully sets up prometheus, jaeger, grafana, and collector * Record 5-minute demo showing metrics collection with Grafana and Prometheus (pulp-administrator-facing) * Record 5-minute demo showing tracing with Jaeger (developer-facing) * demos need to be sent to youtube-admin and then publicized * open telemetry performance update * "a span took around 1/10000s and 1.8k memory on average" * https://github.com/open-telemetry/opentelemetry-python-contrib/issues/1556#issuecomment-1458550924 * open telemetry dependency for pulp * would include the various autoinstrumentation pkgs (postgres, redis, wsgi, etc) and utilities * prob ~10ish new pkgs * decision needs to be made on whether pulp requires these "always" or "optionally" * needs discussion and decision * even if deps are present, maybe env-var to turn on/off? * is operator interested? * yes - primarily worker-metrics, for scaling decisions * kubernetes envs will handle scaling-due-to-hardware-issues (ie, CPU-utilization) * Can this be reviewed/merged before 3.23 release? * https://github.com/pulp/pulpcore/pull/3650 * done! * What about this one: * https://github.com/pulp/pulpcore/pull/3655 * done! * observation/evaluation of https://analytics.pulpproject.org/ status * discussion around understanding why so many "old" core-versions * should/can we think about which "downstream" is reporting analytics? * lots of discussion around how to interpret results # March 7, 2023 * Can we get a Pulp/Katello Integration Mtg representative to cover dkliban's leave? * ggainey is It * [AI] ggainey to suggest every-other-week for this mtg * core/3.23 - release when? * domains PR *almost* merged - today * pulp-cli/glue will have a release * then pulpcore * then pulp_file * calVer recap * current thinking: let's just release faster and not switch to calver right now * not possible anyway before 3.25 * Next breaking change Release after 3.25? * We will need to declare the next 3.Y plugin API breaking change release * https://pastebin.com/raw/y4jNx1BM * https://pastebin.com/raw/d5mnQi88 * 15 ('minimum of a quarter')?> 20? * need to have a decision made as part of the 3.25 release process * Analytics proposal * https://github.com/pulp/analytics.pulpproject.org/issues/84 # February 28, 2023 * gerrod is out on pto, Matthias is meeting lead * Upgrading Django, 4.2 (LTS) is coming in April * 3.2 upstream support is till Dec 1, 2023. * Seems it goes till Apr, 2024. https://www.djangoproject.com/download * Any known changes which affect us? * AAP is asking for 4.1 even now, pushing back to wait for 4.2 * With django 4.* and django-channels, we may be able to consolidate the codebase (sync and async parts) on django. * Do migrations break with this change? * SUMMARY: We want to switch to 4.2 as soon as possible. Pulpcore 3.25 is the best opportunity. If we find FTE, someone should experiment with 4.1 asap. * [AI DKliban] Look for an issue; add it to 3.25 blockers. * [AI DKliban] advertise the Django change on Discourse. * crazy idea time: Can we turn Upload into a content type? * Chunks are really "just" artifacts. * We'd get orphan cleanup for free. * One less type of object to store in the storage bucket. * [AI MDellweg] Write a story * Open Telemetry update * Do we want this to be feature flagged? * YES * How do you want it turned on/off? * Question of deployment * Should it be on-by-default? * Can we make it an optional dependency? * [AI BBouters] Present <= 10' about telemetry * What is the process to add metrics to analytics? * We should document the process around the proposal process in the docs somwhere. * The process is there to collect deails around sanitizing and getting feedback from a privacy-protection perspective. * We have an RFE from Ansible folks https://github.com/pulp/analytics.pulpproject.org/issues/82 # February 21, 2023 * [dkliban] Posted to discourse about future of pulpcore version compatibility * https://discourse.pulpproject.org/t/switching-pulpcore-to-calendar-versioning-scheme/771/8 * Lots of discussion, no agreement so far * Replication PR * https://github.com/pulp/pulpcore/pull/3568 * not draft anymore * 2 features still needed: * dealing with protected content * auto-adding contentguard on the receiving system * having multiple upstream-pulp-configs (handling multiple domains) * release current as tech-preview * stays tech-prev until the 2 features above are released, then rmv tech-prev * do we really need to do something specific for multi-domain upstream? * can we just have a multi-domain POP? * investigation needed * Pulp Domains Ready for Review * https://github.com/pulp/pulp-openapi-generator/pull/80 * https://github.com/pulp/pulpcore/pull/3190 * https://github.com/pulp/pulp-certguard/pull/215 * https://github.com/pulp/pulp-cli/pull/590 * https://github.com/pulp/pulp_file/pull/810 * https://github.com/pulp/pulp_rpm/pull/2920 * Discussion around GCP support * Adding more test runners that run on Azure cloud * https://opensource.microsoft.com/azure-credits/ # February 14 * Faster release schedule (and no more waiting on blockers) is needed if we want our cloud deployment downstreams to not deploy from the main branch. * Are migrations guaranteed to be stable between adding to main and releasing? * We could defer merging PRs with migrations until later in the cycle? * Might not be a later phase if we are releasing faster. Need to review faster * Goal is to release often so we don't run into a scenario where we want to release a plugin that is waiting on a pulpcore release * Cloud services might end up using pulpcore from main anyway * But they may also consume pulp as a library from only ga versions. * Release every week? * Y release if features present, else will be a z-stream release * What about bugfixes? How does this impact backporting? * Could be a backport nightmare, need to have unofficial LTS versions that we get products to share * Need to choose a new larger future compatible pulpcore version for plugins to declare compatibility with, maybe 10 versions: 3.35? * Potentially change pulpcore versioning to date base format? * [AI] dkliban to write a discourse post discussing how many new versions we want to declare future compatibility against * [lmjachky] Review of https://github.com/pulp/pulpcore/issues/3429 * Again, my take on this is that we can mark almost all of the features in tech preview as production ready NOW. * https://discourse.pulpproject.org/t/features-in-tech-preview/737 * Based on the comments, we have direct feedback from users who request features. Why do we need analytics first in order to mark the features as production ready? * Let's use merge queues for Pulpcore * https://github.blog/changelog/2023-02-08-pull-request-merge-queue-public-beta/ * Maybe test in a plugin or wait a couple more weeks since feature is in beta * Stream runner (analysis) * [mpysik] Review https://github.com/pulp/pulpcore/issues/3336 * Discuss a solution based on the observed data * Replication PR * https://github.com/pulp/pulpcore/pull/3568 * Didn't get to, will revisit next week # January 31 * [gerrod] (Prio-listable?) Exposing user task dispatcher: https://discourse.pulpproject.org/t/logging-which-user-started-a-task/733/3 * Can/should we get a pulp_file 1.12 released? * 1.11.0 happened 2022-07-28 * 4 new features, agreed to release * interesting authZ resources * https://storage.googleapis.com/pub-tools-public-publication-data/pdf/10683a8987dbf0c6d4edcafb9b4f05cc9de5974a.pdf * https://github.com/authzed/spicedb * Related: Can one impose as a group and make that group owner of the tasks and objects created? # January 24 * [dkliban] to update pulpcore release docs to include OCI images release information * Filed a priority-list issue for this: https://github.com/pulp/pulpcore/issues/3514 * Pulp replica discussion * fun with repo-attached Distributions and auto-publish # January 17 * [lmjachky] Features in tech preview: https://discourse.pulpproject.org/t/features-in-tech-preview/737 * [dkliban] - Update on the pulpcore release steps doc: https://hackmd.io/gbTfH231RK-u1J2qjCcMLw?both * Need some location to store this doc, dkliban to update pulpcore docs * FYI the zero downtime docs are merged, FYI for anyone making migrations and/or task signature changes * https://github.com/pulp/pulpcore/pull/3448/files * Want to continue experiment of having plugins really think about the migrations they are writing # January 10 * review https://hackmd.io/gbTfH231RK-u1J2qjCcMLw?both * [AI] Dennis to bring this up in the deployers meeting * Galaxy[Bruno] - Add a *ContentLabel* model - https://docs.google.com/document/d/1RIp3ifumYegX6ANIscmEruK8F9BQ6hbR46F3oj-7PNI/edit?usp=sharing * Most likely implement in pulp_ansible for now, hard to be generic enough for all plugins * Could eventually replace AnsibleCollectionDeprecated * Should probably be similar in implementation to pulp_container Tags * *ContentMark* is favorite name * issues with pulp-in-one-container image https://github.com/pulp/pulpcore/issues/3492 * Dennis to continue following up with poster on the issue # January 3, 2023 * "stream" test is really unreliable * we need to fix this problem, or disable the unreliable tests - they teach us to "ignore red" * [ipanova] to my knowledge Michal was working on its fix, not trivial * this https://github.com/pulp/pulpcore/issues/3336 ? * [team] decided to keep the stream job on and leave this work for Michal, as planned. The bug is sporadic so should not have high impact * Upstream high-value tickets, should we prio-list them? * Why? -- Came from user feedback sessions * https://hackmd.io/Nw56k4mISw-yq1NDXMQ3Lg * https://github.com/pulp/pulpcore/issues/3155 * https://github.com/pulp/pulpcore/issues/1817 * [team] not ready to work on them yet, leave it to Tasking system SMEs to look into transparently provided tasks args * [ttereshc] added Mike and Humberto as optional attendees when and if they have some questions or need clarifications on pulp architecture, or pulpcore related topics, which are easier to bring up over a call. If you think, it's a wrong venue, blame me :), and I'm open for suggestions. # December 20, 2022 * "stream" test is really unreliable * we need to fix this problem, or disable the unreliable tests - they teach us to "ignore red" * [ipanova] to my knowledge Michal was working on its fix, not trivial * this https://github.com/pulp/pulpcore/issues/3336 ? * if it's not going to happen Soon, then ggainey would like the test disabled for now * Upstream high-value tickets, should we prio-list them? * Why? -- Came from user feedback sessions * https://hackmd.io/Nw56k4mISw-yq1NDXMQ3Lg * https://github.com/pulp/pulpcore/issues/3155 * https://github.com/pulp/pulpcore/issues/1817 * not much of a quorum - move to Upcoming again * FWIW, ggainey and decko in favor * discussion RE putting more focus on community asks * treat upstream-community as a formal Stakeholder * def a good thing * think about 3-month-planning improvement * ggainey out 27-DEC - cancel the mtg next week? or appoint a facilitator? * ~~AI: ggainey to cancel next week~~ * gerrod becomes facilitator for Jan-Feb * prob should not be "optional" for those months :) * ~~AI: ggainey to voluntold gerrod :)~~ # December 13, 2022 * [decko] reviewed/approved [3427](https://github.com/pulp/pulpcore/pull/3427) (HStore) * Zero-downtime next steps * https://github.com/pulp/pulpcore/pull/3448 * all pulpcore and pulp_file migrations need to be zero-downtime compatible * all tasks need to be backwards compatible * AI: review , ggainey added as reviewer * [3190](https://github.com/pulp/pulpcore/pull/3190) (domains) * discussion ensues * AI: [gerrod] run all/many plugins' CI tests against the PR * who are the SMEs? * gerrod * RTFM group? * gerrod's presentation makes it possible for everyone to understand * until we have confident LG2M from confident SMEs, we should maybe release core/3.22 sans domains, then 3.23 w/ this * core/3.22 discussion around probs caused by merged content-app pr * affects python and container * do we revert? wait for a fix? hold off 3.22? * need to resolve prior to a core/3.22 release * needs to be fixed/addressed in pulpcore - issue? * "fixing it" in container/python * fails at core/3.22 release if older plugins are installed * what happens if we "fix" this in plugins and backport? * container tinks this is supportable * how many active z-streams? (2 for container) * CI is currently broken because of this problem * about 1/3 tests in container * "some" tests broken in python * reverting the two commits fixes CI - but removes the major drivers for core/3.22 being date-driven * revert-commit-msg needs to explain WHY we're reverting * need to have something in place for core/3.22 and pulp_rpm * we need to expand on pulpcore's interface in this area, in a way that plugins can reliably take advantage of it - **for core/3.25** * AI: [bmbouter] to open an issue, for this issue * https://github.com/pulp/pulpcore/issues/1830 * AI: this needs A LOT of discussion at OpenFloor # December 6, 2022 * Proposal: have pulpcore adopt zero-downtime migration policy * to do this we need to write plugin-writer docs first * https://discourse.pulpproject.org/t/support-zero-downtime-updates/645/18 * https://hackmd.io/dxm3JG0bSEizAJeM6E9HUw * start with pulpcore and pulp_file and see How It All Works, starting with 3.23 * THIS WILL BE AN EXPERIMENT - no guarantees stated or implied! * AI: [bmbouter] open a doc-issue * AI: [bmbouter] write up plugin docs for this * Proposal: give decko write-perms to pulpcore * previously, generally waited a year/major feature addition as SME to add * feels like we need to give it decko now to allow reviews for pytest work to "count" * ~~AI: [dkliban] add decko to pulpcore team~~ * remove ppicka invitation/commit bit since January * AI: [bmbouter] to remove from mtg invite * AI: [team] move ppicka to Contributor * Should settings be semver controlled? * dkliban: leaning yes * bmbouter: users are asking for it, we should give it to them * still can do "emergency" changes (e.g. security issues) if we need to under semver * x9c4: can we get away with not-defining and deal w/ it case-by-case * since semver allows for emergency fixes, prob ok * ggainey: also leaning yes * mdellweg: we can decide for pulpcore, but not for Everyone * maybe we should put up in discourse for a week for discussion * bmbouter: transparency is good * AI: [x9c4] open thread in discourse * AI: [all] revisit and confirm at next mtg * Labels PR is ready and needs review/merge: https://github.com/pulp/pulpcore/pull/3427 * do we want this in 3.22? * it's a bugfix * merge it - "waiting in main doesn't actually help" * 3.22 is 12-DEC - so review ASAP * AI: [team] get 3427 reviewed so we can merge no later than Fri * Q: how easy is it to create a new release? * Proposal: we should do releases more often * plugins that depend on core/main can't release until that main becomes a 'real' release * Proposal: let's start by releasing quickly (two weeks? three?) * also - no go/nogo * strictly time-based * Y-release ONLY if there are features * required-PR has Implications * what about backport policy? * prob OK, for multiple reasons * https://github.com/pulp/pulpcore/issues/3368 * can we make this a prio-list? - yes # November 29, 2022 * Are we going to actually pay attention to the CodeQL output? * decko can't view https://github.com/pulp/pulpcore/security/code-scanning ? * proposal: if we're not going to use, turn it off * proposal: if we *are* going to use, then open issues and fix the reports * [davidd] Pulp's policy around settings changes https://github.com/pulp/pulpcore/issues/3417#issuecomment-1329659503 * We set a number of settings and are concerned about settings changes * If settings are not covered by semver, could there be a documented policy/deprecation cycle? * proposal: settings should follow same semver policy as REST API * i.e.: can deprecate, but not remove * each rename/removal is Going To Be A Pain - "please don't" is the generic dev policy * proposal: doc settings.local more in docs? * already mentioned in https://docs.pulpproject.org/pulpcore/configuration/applying.html#by-configuration-file * pulp_labels * https://github.com/pulp/pulpcore/pull/3427 * currently exposed for distributions, repositories, remotes (so far) * jsonb is overkill * would need to remove generic-relation * exposes some of the functionality in the API * breaking change? * plugin API affected * need to research if anyone is actively "aware of" the current implementation * marked tech-preview - so (technically) "can do anything" * OTOH - we' guarantee that plugin-breaking-changes won't happen until breaking-change releases (e.g. 3.25) * Base model introduction would require Very Complicated Migration * REST API unaffected * poss bug in current REST * pulpcore 3.22 GA is postponed * all thumbs-up * tech-preview discussion * we have a tech-preview problem * just because we say "we can break this", doesn't make it even remotely a good idea * how long can/should something say "tech-preview" * can we move to "how much is it used"? * analytics to the rescue! # November 22, 2022 * Review ggainey's rearranging of this doc to see if it's what the team wants # November 15, 2022 * 3.22 release schedule * ipanova is the release shepherd * We're at 2 months since 3.21 * tentative GA date is Nov 30 * 16 and 23 of Nov go/no-go meetings * what should go in? * domains * needs issue, add to 3.22 blockers [AI: gerrod] * [blockers](https://github.com/pulp/pulpcore/milestones) * django4/core-3.25 discussion * django-4 happens in April * 3.25 needs more time due to likelihood of breaking-changes * don't want to get downstream products "stuck" on older versions that will EOL * "when should 3.23 and 3.24 happen" * IF 3.25 is going to be "django-4" - prob want to have significant burn-in * 3.23 - prob end-of Jan, to get in anything we want to have available for "conference season" in Feb * 3.24 - prob end-of March, for 2-month-cadence/pre-django-4 * 3.25/django-4 linkage is **not required** - let's "see what happens" w/ django and re-evaluate in April * AI: ggainey to post to discourse [DONE] * AI: ggainey to engage in "creative rearranging" of team doc [DONE] * created [2020](https://hackmd.io/@pulp/core_2020), [2021](https://hackmd.io/@pulp/core_2021) archive-files * single archive warned "file too large" * separated [SME list](https://hackmd.io/@pulp/core_sme) to its own doc * added "Meeting Agenda" section * added `tags: pulpcore` to sme/archive/this doc # November 1, 2022 * Failing test related to repo version delete * dkliban to investigate # October 25, 2022 * Filter-process appears to need some more refactoring * x9c4 is willing to do, but needs some buy-in for quick review turnarounds * will fix a couple of openapi-schema bugs * discussion ensues RE opportunity costs * what about new content-types? * how can we best prioritize competing priorities from groups that are not in agreement? * let's get folk from "emerging" opportunities come talk to us directly # October 18, 2022 * CI failures on 3.16 * failure on the local filesystem runner * proposal: skip the single test because it's preventing us from running all tests * observation: this pulp_rpm has a similar failure https://github.com/pulp/pulp_rpm/pull/2838 * actually it's different and it needs an actual code change in pulp_rpm test to fix so it's unrelated * [deadlock PR](https://github.com/pulp/pulpcore/pull/3286) desperately needs another review(er) please * pulp_file needs some TLC * 6 PRs open * [performance tests](https://github.com/pulp/pulp_file/actions/runs/3262257640/jobs/5359466206#step:12:1676) constantly failing - why? looks like it is fixed now :) * design discussion for: https://github.com/pulp/pulpcore/issues/2851 * bmbouter scheduling the zero downtime working group # October 11, 2022 * https://github.com/pulp/pulp-manifest/ needs an owner. * probably the pulpcore team * mention the existence of the script/repository in the pulp_file documentation * https://github.com/pulp/pulp_file/issues/844 * design discussion for: https://github.com/pulp/pulpcore/issues/2851 * zero-downtime working group call for participation: * https://discourse.pulpproject.org/t/support-zero-downtime-updates/645/8 * telemetry updates * see the recent data: https://analytics.pulpproject.org/ # October 4, 2022 * request for zero-downtime upgrades * https://discourse.pulpproject.org/t/support-zero-downtime-updates/645/ * this would be a project-wide change that all plugins would have to adhere to * advertising/discussing here but real discussion happens on discourse * [bmbouter proposal] let's create a ticket and PoC it, we also need to ask for feedback at open floor * [ipanova] this will most likely be a breaking change that will not come in earlier than 3.25. We should then pick when the time is right for the PoC so the code does not get old. * 3.21 release highlights for AH, needed or not? * Hide manifest files https://github.com/pulp/pulp_file/issues/837 * Would appreciate some feedback * S3 storage docs https://issues.redhat.com/browse/AAH-1944 # September 27, 2022 * https://github.com/pulp/pulpcore/issues/3212 * will have a dedicated meeting for this topic * idea 1 * duplicate content between repositories * create a content unit for each remote that syncs that content * idea 2 * add foreign key to RemoteArtifact on ContentArtifact * What are the right branches for the plugin template to be opened up against? * We call them hot branches. We can still release on others, just not that fast. * This plugin_template PR changes how this works: https://github.com/pulp/plugin_template/pull/693 * 3.7(AH), 3.14(Sat), 3.15(AH), 3.16(Sat), 3.18(AH, Sat), 3.21(AH, Sat) * Do we still need 3.7, 3.14? * 3.16, 3.18, 3.21 make a PR * FYI the dependencies are now in-place for pulpcore, we'll watch dependabot do the rest * https://github.com/pulp/pulpcore/blob/main/requirements.txt * FYI analytics.pulpproject.org is flowing * https://analytics.pulpproject.org/ * Need to do some pulpcore-content peformance testing, e.g. 500K req / sec # September 20, 2022 * There's an ask for more opinions * https://github.com/pulp/pulpcore/pull/2963#discussion_r974957182 * Some kanban reflections * Outbound: How do issues move off the Kanban board? Lots of "done" issues currently, clutters the per-repo view * https://github.com/orgs/pulp/projects/8/views/2?filterQuery=status%3A * it is not implemented yet, but Tanya's idea was manually dispatching a workflow to clean the done tasks at the beginning of the planning * Inbound: many plugins have nothing on the board, is that an accurate portrayal of priority or a process issue? Do we need to add items more frequently? * Only 2 "free to take" issues outside of the RPM plugin * Challenge: How do we manage the prio-list label with full inclusivity in an async environment? * Idea: When adding/removing/modifying the prio-list label leave a comment with @pulp/core * for example: https://github.com/pulp/pulpcore/issues/3213#issuecomment-1251065635 * We lack hard limits on the in progress/review columns * Items with reviews must move back to "in progress" if not ready to merge. * What to do with PRs that never had 'prio-list' ( community contributions, some old issue before kanban era, ) # September 13, 2022 * who is going to take care of pulp2 eol annoucement https://discourse.pulpproject.org/t/archiving-pulp-2-plugin-code/582/5 * versioning dependencies * https://github.com/pulp/pulpcore/pull/3176 * components sme list # September 6, 2022 * Update list of SME * content-type change affects a pulp_ansible test * I am currently trying to get an answer whether it is an issue. * https://github.com/pulp/pulp_ansible/pull/1229 * Discussion on https://github.com/pulp/pulpcore/pull/3136 * Suggestion: instead of using uuid directly, provide a pulp_uuid function which we can change the implementation of without needing future migrations * Can be done independently of whether we adopt UUIDv7 right now * Discuss further at 3.21 go-no-go meeting # August 30, 2022 * pulpcore 3.21 release plans? * https://discourse.pulpproject.org/t/pulpcore-3-21-release-schedule/573/ * go/no-go today, mdellweg will run * need someone to regularly go to katello integration call and report for pulpcore * dkliban to take over the reporting role for pulpcore * We need to have rules for approvals. * proposal: Require one Approval from SME and one non-necessarily-SME. * rationale: We do not have enough people to find two SME for each PR. Admin-merge is not a good solution (and an even harder requirement). Each pulpcore committer should be able to find code smells. * We're supposed to have two SMEs for each component, so couldn't we have two SME reviewers? * even when we do - if one (of the two) SMEs submits the PR, we're still short a reviewer * problem: we don't have the SMEs for pulpcore components written down somewhere * solution: mdellweg to write up SME components and list * FYI: pulp-smash proposal to be put into read-only mode on Sept 5th * https://discourse.pulpproject.org/t/retirement-of-pulp-smash/579 * set prio-list label * to review asynchronously and propose 3 items via pulp-dev, then we will review next week * added prio-list query to the top of this hackmd https://github.com/pulp/pulpcore/issues?q=is%3Aopen+is%3Aissue+label%3Aprio-list * review/unblock any assigned tickets, so folks have no more than 2 items assigned (or find a good reason to increase the limit) * https://github.com/issues/assigned * backport discussion * still doing updating automated CI updates for every release branch back to 3.14 * suggestion: have a list of explicit "supported releases" * AI for dkliban to write up and implement * suggestion: after a product release drops active support, we do stop eager backports * e.g. Once Satellite 6.N+1 comes out, "active" support stops and they only patch critical issues, z-streams not planned in advance # August 23th, 2022 * Discussing the Squashing Migrations plan * https://discourse.pulpproject.org/t/squashing-migrations/571 * AI: discuss on discourse please * pulpcore 3.21 release plans? * any stakeholders waiting for it and blockers? * 1 blocker: https://github.com/pulp/pulpcore/milestone/4 * Satellite and AH might both use 3.21 if released on time for their schedule * Proposal: enabling anonymous telemetry submission for end users in 3.21.0 * https://github.com/pulp/pulpcore/pull/3116 * [bmbouter] Telemetry process needs to be in our docs * https://hackmd.io/OuQp2PDdT-CzAm3jMj6i0Q#List-of-Questions-for-Every-Metric-To-Be-Gathered * https://hackmd.io/oGwOJo0WQcejOnLI-nJ-Fw * Are we already using the new label? * prio-list * Releases are currently broken * whl x MANIFEST.in - https://stackoverflow.com/questions/24347450/how-do-you-add-additional-files-to-a-wheel * https://github.com/pulp/plugin_template/pull/680 # August 16th, 2022 * When to upgrade to Django 4? * django timelines for deprecation https://www.djangoproject.com/download/ # August 9th, 2022 * [lmjachky - PTO] Please, take a look at the current implementation of the remotes removal. We decided to implement an option where a user needs to specify the "force=True" header to delete the remote if it is being used by existing content artifacts. Otherwise, an HTTP 406 error is returned by Pulp: https://discourse.pulpproject.org/t/when-to-allow-removal-of-remotes-used-for-on-demand-syncing/489/16 [ipanova] will comment * we can't use force due to it being a DELETE call * let's not create a force now because we'd have to make an entirely new endpoint * let's have a great error message returned * we need tests * [AI] @ipanova to represent ^ changes on the discourse and the PR reviews * PoC pulp-operator in Golang https://docs.google.com/document/d/1j6eanLgVz4HboEslzhjhOGKK4HF-jSSuC2wSC34eVpA/edit#heading=h.mdfz93aqubxv (pros and cons in the doc) # July 19, 2022 * proposal to sync flake8 configs in the project * https://github.com/pulp/plugin_template/pull/558 * no concerns heard at the meeting * Remove david from pypi projects * https://discourse.pulpproject.org/t/removal-from-pypi-projects/541 * Change test assumption to assume all tests will be run on the same system pulp is installed on * https://discourse.pulpproject.org/t/functional-tests-will-now-run-inside-the-container-running-pulp/498/5 * would include retiring pulp-smash * Yank pulp-smash from pypi? # July 12, 2022 * proposal: remove pep8speaks. It's redundant give flake8 is in the linter. currently enabled on pulpcore, pulp_file, plugin_template, and pulp_ansible * https://github.com/organizations/pulp/settings/installations/39250 * Time to squash migrations? * it is almost 100 now (pulpcore) * about 40 in ansible, container, rpm * https://docs.djangoproject.com/en/4.0/topics/migrations/#squashing-migrations * Can this be done without A) special requirements about the upgrade process and B) requiring plugins to squash at the same time? * Call for feedback: an experiment: marking 50% of tests as nightly only * https://discourse.pulpproject.org/t/marking-majority-of-tests-to-only-run-nightly/491/4 * How are the tests chosen? * What would be the revert policy? # July 5, 2022 * Discourse down? * galaxy NG requested changes (if you missed this meeting, check the comments on the issues to see what was agreed) * https://github.com/pulp/pulpcore/issues/2824 * Discussion: Preventing the removal of remotes that are in-use * Need affirmation of design * https://github.com/pulp/pulpcore/pull/2836 * https://discourse.pulpproject.org/t/when-to-allow-removal-of-remotes-used-for-on-demand-syncing/489/ * # June 28, 2022 * need to reduce CI runtime, right now it's 40+ minutes per PR (assuming no failures!) * proposing we mark 70% of tests to only run nightly in pulpcore and pulp_file * https://discourse.pulpproject.org/t/marking-majority-of-tests-to-only-run-nightly/491 * call for interest on generic UI components for Pulp * https://discourse.pulpproject.org/t/react-patternfly-component-interest-group/492 * HA and health checks * https://discourse.pulpproject.org/t/pulp-health-checks/514 * healthy worker? * don't need to look at worker health since it's a totally async component * healthy content app? * use `HEAD /pulp/content/` if it returns 200 it's :thumbsup: * galaxy NG requested changes (if you missed this meeting, check the comments on the issues to see what was agreed) * https://github.com/pulp/pulpcore/issues/2865 * https://github.com/pulp/pulpcore/issues/2825 * https://github.com/pulp/pulpcore/issues/2824 * did not discuss this one, need input from more people ### June 21st * Pulpcore 3.20 release ### June 14th * [lmjachky] What to do next with https://github.com/pulp/pulpcore/pull/2713? * exporting/importing repositories, distributions, and other types * converting to different types of repositories (https://github.com/pulp/pulp_container/pull/826#issuecomment-1153680951) * telemetry update: * dev systems will now report to the "dev analytics site" which gets summarized every 24 hours * once merged https://dev-analytics-pulpproject-org.pulpproject.workers.dev/ * going to make new chart for x.y components instead of x.y.z * going to configure analytics.pulpproject.org to the workers with a "coming soon" page * ordering filter vs sort filter * https://github.com/pulp/pulpcore/issues/2703#issuecomment-1154133290 ## June 7th * telemetry pr in-review * https://github.com/pulp/pulpcore/pull/2118 * CI Update * Functional tests to be run inside the container instead of the host * Installation is a nightmere today * More similar to dev-environment * Reducing CI runtime: * moving most tests to nightly * Collaboration with david N on using galaxy NG generic React + Patternfly components for a future Pulp UI ## May 31st * [mdellweg] Serving artifact urls from the pulp_api for external verification (e.g. Virus scanning) * telemetry PoC request for review * https://github.com/pulp/pulpcore/pull/2118 * changing the tests run by default * https://github.com/pulp/pulp-smash/pull/1293 * LTS continued? * wait on Dennis' results ## May 24th * [ipanova] need to release 3.19.1 https://github.com/pulp/pulpcore/pull/2567 * continued LTS discussion * What is will be the next LTS? * How aggressively should we be backporting to the LTS? * How many LTS branches are supported concurrently? * telemetry PoC request for comment * https://github.com/bmbouter/analytics.pulpproject.org * https://github.com/pulp/pulpcore/pull/2118 ## May 17th * Look through remaining maybe-finished issues * https://github.com/pulp/pulpcore/issues?q=is%3Aopen+is%3Aissue+label%3AFinished%3F * Convert char to text field https://github.com/pulp/pulp_rpm/pull/2515 * guardian removal side effects https://hackmd.io/sPmy4YS6TQ-iGn6xyYZP7g?both * Discussing LTS declaration for pulpcore * https://discourse.pulpproject.org/t/need-to-reduce-the-number-of-release-branches-aka-we-need-an-lts-strategy/449/ * Is 3.18 an acceptable one * we think so, but let's discuss it a little more (bmbouter wants to think it over some more) * What is will be the next LTS? * How aggressively should we be backporting to the LTS? * How many LTS branches are supported concurrently? ## May 10th * Look through remaining maybe-finished issues * https://github.com/pulp/pulpcore/issues?q=is%3Aopen+is%3Aissue+label%3AFinished%3F * z-releases * 3.20 tentative release date? first go/no-go? * status of django 2.2 in pulpcore 3.7 / 3.14 * updating core/3.7 to last 2.2LTS, will require a change in (at least) matching pulp_container version * need to evaluate other plugins (`Required PR:` ?) * file/rpm/container/ansible * https://github.com/pulp/pulpcore/pull/2651 * need a plan for who/when/testing * bmbouter and dkliban are updating pulp_file tests to use pytest, merging them with pulpcore ## May 3rd * Removing the "test keys" from the repo, recently flagged by security scanners and bmbouter was notified * https://github.com/pulp/pulp_file/tree/main/pulp_file/tests/functional/api/from_pulpcore/artifacts/x509 * 3.20 removal of MissingResource exception * https://github.com/pulp/pulpcore/issues/1812#issuecomment-1112403081 * https://github.com/pulp/pulpcore/pull/2633/files * 3.20 tentative date (or not ready yet?) * (dalley) not ready yet IMO * Worth doing new Z-releases today? * https://github.com/pulp/pulpcore/tree/3.18/CHANGES * (no) ## April 26th * Discussion around list APIs for Exporters and Remotes * https://github.com/pulp/pulpcore/issues/2529 * https://github.com/pulp/pulpcore/issues/2530 * Downloader SIG: need to schedule kick-off meeting * Release rotation - time to rotate? * Brian will take care of 3.20 release * what to do with the blogposts on the pulpproject.org? Probably we will drop those and have only discourse release announcements * need to fix deps requirements( upper bound) on older releases and main branch before 3.20 release * decision on compatibility release? * if 3.25 then it wil be 9-10 months roughly until next breaking change. 3.25 it is * https://discourse.pulpproject.org/t/is-the-pulp-logo-available-under-creative-commons-cc0-cc-by-or-cc-by-sa-licenses/432 * our logo is unlicensed * we need to open a ticket with legals - Jeffrey Kaufman is the legal who can help * Streaming support needs a little more work around content ranges and caching; test will probably a new ci scenario with sftp storage * triage security alerts? * yes, add them next to the PR review ( from the next meeting) ## April 19th * test improvements update * unittest switch to pytest is done on master and all branches back to 3.14 * plugins switching in-progress (thanks @fao!) * moving the using_plugin tests to pulp_file * https://github.com/pulp/pulpcore/pull/2535 * https://github.com/pulp/pulp_file/pull/685 * next step: make pulpcore a pytest plugin so its fixtures can be versioned with it * involves moving all pulpcore pytest fixtures from pulp-smash to pulpcore * 3.20 plugin API breaking changes * https://github.com/pulp/pulpcore/milestone/1 * https://github.com/pulp/pulpcore/blob/f57bb9c8fc41831b5e5c392334c863035627b8c1/pulpcore/app/models/base.py#L74 ## April 12th * 3.19 release day * No objections? Will wait for task lock PR and import/export PR to merge * functional test improvements * accomplished * got pulp_file tests running along with the pulpcore tests * https://github.com/pulp/pulpcore/pull/2432 * improved pulp_file tests to be independant * https://github.com/pulp/pulp_file/pull/674 * https://github.com/pulp/pulp_file/pull/678 * the --pulp-no-leftover-objs feature is ready for pulp-smash, still off by default * https://github.com/pulp/pulp-smash/pull/1285 * next steps * merge or revise --pulp-no-leftover-objs * https://github.com/pulp/pulp-smash/pull/1285 * move all pulpcore.tests.functional.api.using_plugin tests to pulp_file * porting tests: * run in parallel * native pytest * use local fixtures (not fixtures.pulpproj.org) * adding list endpoints for remote and alternate content sources * dkliban is switching unittest runner to pytest * https://github.com/pulp/pulpcore/pull/2498/ <- main branch * https://github.com/dkliban/pulpcore/pull/63/ <-- 3.14 branch * https://github.com/pulp/plugin_template/pull/612 * Goal: to not have unittests require pulp-smash to be configured * Solution: need to move conftest.py into the functional folder "down a level" * Will have to do similarly for any plugin that already adopted conftest.py * Should we stop i8ln log messages? * https://github.com/pulp/pulpcore/issues/2477 * Mark issues as "good first issue" * Would make it much easier to hand off issues to interns / people who want to get started with development (e.g. Taft Sanders) ## April 5th * How many releases do we want to keep supporting? * Currently 5, in a few weeks, it will be 6 * https://github.com/pulp/pulpcore/blob/main/template_config.yml#L35 * switch to lower bound which is 3.14 * We can't drop 3.14 * Should we come up with some kind of LTS policy or continue with as-needed unofficial LTS? * we should also audit and establish an upper bound for our requirements.txt deps * FYI here's a proposal to auto-check that tests are always independant * https://discourse.pulpproject.org/t/making-tests-totally-independant/412 * used to make these changes: https://github.com/pulp/pulpcore/pull/2432/files * 3.19 - last call, are there any PRs which need to merge today * let's push back a week * FYI - do not use gettext in log statements https://github.com/pulp/pulpcore/issues/2477 * read and provide feedback ## March 29 * filed issue to remove the queues in 3.20 * https://github.com/pulp/pulpcore/issues/2377 * Closing old/outdated/finished issues: https://github.com/pulp/pulpcore/labels/Finished%3F * Went through and marked issues I thought could be closed * 86 issues! * Options: * Have a meeting to go through some of the issues and discuss if they should be closed * Have everyone look through some of the issues on their own and close them if they feel confident * Unittest update * Added ability to run additional tests in plugin_template * https://github.com/pulp/plugin_template/pull/602 * Using that to run the pulp_file tests * https://github.com/pulp/pulpcore/pull/2432/files * Fail in CI due to pulp_file tests not being a clean environment after pulpcore tests * Goals: * Test independance, run single tests, or in any order * Cleanup all objects except orphans, a test that needs its orphans removed needs to run orphan cleanup at the beginning * move the pulpcore.tests.functional.api.using_plugin tests to pulp_file.tests.functional.api * to run the "pulpcore" tests you have to run pulp_file.tests.functional and pulpcore.tests.functional to be sure * We thought better this than a formal import dependency on pulp_file * switching unittest runner to pytest is also continuing * Do we want to do this eventually https://github.com/pulp/pulpcore/issues/1984? -YES * used in pulp-ansible https://github.com/pulp/pulp_ansible/blob/main/pulp_ansible/app/tasks/collections.py#L398 * and similarly in content app https://github.com/pulp/pulpcore/blob/main/pulpcore/content/handler.py#L774 ## March 22 * memory PoC continued! * 27% reduction in memory by reducing queue count from 1000 to 1 * should we remove the queues entirely? * concern: deadlocking due first-stage waiting on the resolution of a future, or some even the pipeline is supposed to produce * more of a concern now that the pipeline is even less filled * proposal: removing django admin UI views * https://discourse.pulpproject.org/t/roles-in-pulpcore/148/7 * https://github.com/pulp/pulpcore/blob/901499a3566a940516664ae6ca042455efee0115/docs/configuration/settings.rst#L365 * https://github.com/pulp/pulpcore/issues/2374 * no concerns, it is in Tech preview also * bindings docs * should we publish for both: ruby and python? * https://github.com/pulp/plugin_template/pull/599 * let's ask katello * mkdocs switch? * should we? * fabricio will show his PoC * weird "db connection already closed" CI failure https://github.com/pulp/pulpcore/pull/2357#issuecomment-1072333934 * maybe issue with a test, @dkliban will take a look * absolutely weird RemoteArtifactSaver failure in pulp_container * https://hackmd.io/HVtZsbGWSkKAnA4-gXwbiA * https://github.com/pulp/pulpcore/pull/2375/files * Goal: Deduplicating tests between pulpcore and pulp_file ## March 15th * memory PoC concluded * adjusting batch size is effective at reducing memory * see [the results](https://github.com/pulp/pulpcore/issues/2069#issuecomment-1065191935) * dynamically adjusting it doesn't keep it within a "target amount" * it runs extremely slow 10x plus * memory is never returned to the OS, the pages mostly just stay allocated and later * new memory charting debugging feature added: * https://github.com/pulp/pulpcore/pull/2331 * proposal: to expose the batch size to users, the batch size is likely the single best adjustment to reduce memory * we need each plugin to also look at their own first-stage code to see if it can be memory optimized * proposal: https://github.com/pulp/pulpcore/issues/2069#issuecomment-1066853598 * maintainability policy * how far should we keep ensuring old branches are working? * single container currently does latest 3 y-stream * pulp-cli tests with 5 y-streams ## March 8th * memory PoC in-progress * one feature tries to log the relative memory usage of declarative versus everything else * adds a new setting TASK_MEMORY_LIMIT * Attempts to reduce batching and put() calls at first stage * pending openapi schema changes to improve usability * https://github.com/pulp/pulpcore/pull/2292 * update on pytest unittest issue * https://hackmd.io/dNszZ1RCTMK6r8ERxxgYxA * we will need to release 3.15.5 once QE test the patches * [daniel] will take care ## March 1st, 2022 * Mirrorlist / Metalink support in downloaders * Old issue, still a good idea: https://github.com/pulp/pulpcore/issues/1815 * https://github.com/pulp/pulp_rpm/issues/2386 * Consider implementing it in the 3.20 / 3.21 timeframe along with downloader refactor? * (mirrorlists) are a current source of a lot of hacks in the RPM plugin w/ lack of native support * Can we prioritize https://github.com/pulp/pulpcore/issues/2101 ? * yes - ggainey has a branch already * dkliban to keep ggainey on track :) * https://github.com/pulp/pulpcore/pull/2261 needs review, merge and backports * z-stream Releases for 3.17 && 3.18 maybe 3.16 needed. * bmbouter switching unit test runner to use pytest and the django-pytest plugin * will involve rewriting or removing 7 or so unit tests on pulpcore * going into plugin_template as a non-optional change. will advertise at open floor * An effort is underway to have a validating openAPI spec * https://github.com/ansible/galaxy_ng/pull/1136 ## February 22, 2022 * [ggainey] What does the team think about implementing automerge? * 2 approvals, tests green, no merge conflicts == immediate/automatic merge * implies "DO NOT approve a PR unless/until you're willing to see it merge immediately" * [update 2022-03-01] automerge isn't nearly as 'auto' as we want, not nearly as interesting as a result ## February 15, 2022 * rerooting update * pulp-cli and pulpcore is working well * pulp_installer to get some updates to its reverse proxy configs * plan is to test in one of the jobs * review periodic tasks discussion (aka TaskSchedule) * Request to post periodic metrics * https://github.com/pulp/pulpcore/pull/2208 * plugin removal * blocked by CI weirdness around service names in ci container * We fix the ci base container to forget all workarounds for the old tasking system ## February 8, 2022 * periodic tasks request from galaxy_ng for them to run a galaxy_ng task periodically * https://github.com/pulp/pulpcore/issues/2204 * [bmbouter] my proposal is to move [the PeriodicTasks work](https://github.com/pulp/pulpcore/pull/2118/files#diff-b6921076eabe69f8f887286bbac04f5abb8c91b5e2e747b99ea932c5abf17135R332) to its own PR. * proposal: plugin writer only feature, not user-facing * AI: all discuss on issue and review next week * plugin removal test * as a separate job which runs similar to the upgrade test but only nightly? * current PR which struggles to make it work as typical test * https://github.com/pulp/pulpcore/pull/1434/files#r800633658 * Touch() bug fix - when can we get it tested? * https://github.com/pulp/pulpcore/pull/2159/files * 3.18 schedule discourse thread * https://discourse.pulpproject.org/t/pulpcore-3-18-0-release-schedule/330 ## February 1, 2022 * Discuss memory use proposal https://discourse.pulpproject.org/t/reducing-memory-usage-during-sync/277 * Github issues process adjustment suggestions * https://hackmd.io/zFFGO33jQSC1ISRJpXLQ3A#Issues * Changelog cherry-picking problem * No longer using backport issues. Changelog commit removes files from the CHANGES directory. These files need to be present on 'main' branch until the changes are released in the latest release. * We need to figure out Something - because this is the last, larget, manual step blocking releases * Maybe each previous-branch-changelog is a separate file? * How?!? ## January 25th, 2022 * 3.18 release - Feb 8 (tentative) * we have 2 features merged * any blockers/must-haves * redirect content guard * https://github.com/pulp/pulpcore/issues/2151 * plan to schedule go/no-go on Feb 1 * scheduled Feb 2 based on availability * Need to provide the ability to reroot * https://github.com/pulp/pulpcore/issues/2148 * https://github.com/pulp/pulpcore/blob/main/pulpcore/constants.py#L47 * High-priority bugs that need fixing ASAP: * touch() deadlock * https://bugzilla.redhat.com/show_bug.cgi?id=2021406 * import/export unique-constraint failure * https://bugzilla.redhat.com/show_bug.cgi?id=2038432 * https://github.com/pulp/pulpcore/issues/2102 * [PR 2103](https://github.com/pulp/pulpcore/pull/2103) * taskreservedresource migration * https://bugzilla.redhat.com/show_bug.cgi?id=2031154 * Should / Can we provide (general) manage command to help admins to migrate existing permission sets to existing roles? * Idea: manage command to report "unmigrated" permissions and let the admin assign the roles via api * Create the command in pulp_container codebase * Downloader Refactor * started out obvious * grew out of scope * we will slow role and think about separation of concerns again * nothing urgent; please don't hold any other work for it ## January 18th, 2022 * pulpcore is on github issues now! (BZs updated) * we need to rethink triage, planning, Epics, Milestones ... * ggainey has been looking at github-projects to see if we can use it * github projects - not quite rdy for primetime * write scripts to take advantage of GH API? * label-driven queries * only need "current sprint" label * PRs-merged/issues-closed within date-range * epics are an epic pain right now * maybe use projects? * or milestones? (maybe not) * [Example query for issue-triage](https://github.com/pulp/pulpcore/issues?q=is%3Aopen+is%3Aissue+label%3AIssue+-label%3ATriaged) * Discuss memory use proposal https://discourse.pulpproject.org/t/reducing-memory-usage-during-sync/277 * Want to look at basic PoC soon * Downloader logic is convoluted (and broken to some extend) * can we schedule the custom-file-object for removal in 3.20? * [x9c4] file a PR and discuss improvements there * pulpcore deprecation warns for the use of the old orphan interface * It's a valid test that keeps the old interface running and it's going nowhere in the percievable universe * we could try to distiguish rest and plugin api deprecations * we don't warn the user about the use of the old interface * https://github.com/pulp/pulpcore/pull/2100 * ~~[x9c4] remove the deprecation logging and make sure the deprecation is in the api.docs~~ * [bmbouter] keep track of user facing logging * pulpcore is getting parallel test running * https://discourse.pulpproject.org/t/parallel-test-runs-added-to-ci/320 # January 11th, 2022 * Discuss memory use proposal https://discourse.pulpproject.org/t/reducing-memory-usage-during-sync/277 * Hard to decide without seeing some code (PoC is needed) * Even some way to measure where what amount of memory is bound would be helpful * should we protect remotes from being deleted if they are used in other repos? https://github.com/pulp/pulpcore/blob/main/pulpcore/app/models/repository.py#L64 Should we audit all FKs that use on_delete=SET_NULL? * Deleting with SET_NULL can break stuff, open security holes, or just be annoying * https://pulp.plan.io/issues/8305 is a special variant of this * [ipanova] write an issue * upgrades to the new tasking system * It seems to be a good idea to clean up the tasks table (maybe releated tables too) * [ggainey] reach out to katello # January 4th, 2022 * Rotate meeting lead * github migration * There is an API interaction limit; need to run in batches * Bugzilla interaction is confusing ATM * [dkliban] Talk to downstream to (re-)define Bugzilla interaction * patchback - https://discourse.pulpproject.org/t/automating-backports/153/4 * Let's change the Commit validation to accept closed issues on cherry-picks * https://tutorial.octomachinery.dev/en/latest/hall-of-fame.html * open cherry-picks from the newest branch to the oldest needed * to request a backport, leave a comment on a PR, tagging @to-be-decided. * update on improving tests in pulpcore * new pytest fixtures in pulp-smash merged * https://github.com/pulp/pulp-smash/pull/1262/files * new pulp_file fixtures in pulpcore merged * https://github.com/pulp/pulp-smash/pull/1262/files * working on switching unit tests to pytest * https://pulp.plan.io/issues/9643 * any 3.18 plans or deadlines? * no known driver ___ # Archive * [2021 meeting notes](https://hackmd.io/@pulp/core_2021) * [2020 meeting notes](https://hackmd.io/@pulp/core_2020) ###### tags: `pulpcore`

Syntax	Example	Reference
# Header	Header	基本排版
- Unordered List	Unordered List
1. Ordered List	Ordered List
- [ ] Todo List	Todo List
> Blockquote	Blockquote
Bold font	Bold font
Italics font	Italics font
~~Strikethrough~~	~~Strikethrough~~
19^th^	19^th
H~2~O	H₂O
++Inserted text++	Inserted text
==Marked text==	Marked text
[link text](https:// "title")	Link
![image alt](https:// "title")	Image
`Code`	`Code`	在筆記中貼入程式碼
```javascript var i = 0; ```	`var i = 0;`
:smile:		Emoji list
{%youtube youtube_id %}	Externals
$L^aT_eX$	L^aT_eX
:::info This is a alert area. :::	This is a alert area.