# Pulp/Galaxy Check-In Meeting
## 23 March 2021
### Agenda
* CI/CD
* view_group permission bug
* https://issues.redhat.com/browse/AAH-453
* https://pulp.plan.io/issues/8159
* Unpaginated sync timeout
* https://issues.redhat.com/browse/AAH-224
> [name=Calvin Spealman]
Testing results as of March 19, 2021:
- The unpaginated endpoint times out on QA, the number of collections it has to process in one request is too large
- The number of collections on production might be under whatever limit triggers that timeout, but we can't know
- If it does work on production today, we can't predict when the growing collections there will trigger this problem
- I really think we need to explore creative ways to pre-compute some of this data
- The unpaginated endpoint times out on QA, the number of collections it has to process in one request is too large
- The number of collections on production might be under whatever limit triggers that timeout, but we can't know
- If it does work on production today, we can't predict when the growing collections there will trigger this problem
- I really think we need to explore creative ways to pre-compute some of this data
* Last conversation about galaxy metadata/unpaginated endpoints: https://hackmd.io/370wOQiyQpG8gNXicas6nw?both
* Healthcheck feedback
* Limit N repo verisons
* https://pulp.plan.io/issues/8368
* Dependency sync being added to pulp_ansible 0.8
* https://github.com/pulp/pulp_ansible/pull/545/files
## 16 March 2021
### Agenda
* I am the captain now: http://gph.is/XKFswD
* Pulp resource reservation cleanup
* Release version?
* maybe https://docs.pulpproject.org/pulpcore/changes.html#id135
* Sync collection dependencies
* https://github.com/pulp/pulp_ansible/pull/545
* nested dependency example?(use orionutils)
* Operator
* Currently we have CI job "testing" galaxy
* https://github.com/pulp/pulp-operator/runs/2109374622?check_suite_focus=true
* https://pulp.plan.io/issues/8369
* Content provenance
* Namespace sync
* Customer issue
* possibly similar to: https://access.redhat.com/solutions/4976401
```
"description": "403, message='DefaultErrorTemplate', url=URL('http://serverproxy.corp.int:8080')"
```
* Applying md5 fix on AH Stage and AH Prod
* Are `pulpcore-manager handle-artifact-checksums` script and migrations in same or different versions?
* When do we need galaxy-importer FILES.json fix, and data cleanup from pulp persepctive
* https://pulp.plan.io/issues/8292
* https://issues.redhat.com/browse/AAH-403
### Notes
* Namespace sync 'previously solution not viable'. {insert summary of previous solution}
* Namespace is more or less a type of Content (or even a Repository), could we treat it as such <- No. It's more like this: https://github.com/pulp/pulp_container/blob/master/pulp_container/app/models.py#L198
* 403 DefaultErrorTemplate issue... https://access.redhat.com/solutions/4976401 points to a SSL/TLS intercepting/mitm proxy as the cause when seen with Sat
* https://access.redhat.com/support/cases/#/case/02881343 (has full traceback)
* Looks like sso.redhat.com wasn't whitelisted in the proxy used
* sync Collection dependencies
* ansible-galaxy has ad hoc dep solver
* pulp_ansible has different ad hoc dep solver (two?)
* exec env tooling?
* Galaxy_ng to require pulpcore 3.11.0
* pulp_ansible 0.7.1
* pulp_container 2.3.0
### Actions
* [alex] create pulp.plan.io issue for tasking reservation problem
* [fabricio] followup with calvin about using orion utils to generate test collections
* [david] reprioritize data initialization
* [david] create healthcheck endpoint
* [david] get more info on sync issue from lucas
## 10 March 2021
### Agenda
* Want a Pulp t-shirt?
* Sign up here: https://forms.gle/2ApayxEVYdnhPJRA6
* Platform installer blocked on pulp_installer requirements
* `ini_file` module currently, maybe more
* TL;DR: Only want to depend on things in Automation Hub, which excludes `community.general`
* Someone has to vendor or needs to move requirements to `ansible.utils`
* 0.8.0 candidates: https://pulp.plan.io/versions/186
* Pulp ansible test results: https://hackmd.io/eXUOVP5tTPWdTOl-9uZBuQ
### Notes
*
### Actions
* [Calvin?] File the issue related to platform installer on https://pulp.plan.io/issues/new (use installer category) - email pulp-dev (a public list) mailing list. Make sure the new isssue appears on https://pulp.plan.io/issues?query_id=170
## 2 March 2021
### Agenda
* Token auth support for Katello
* https://pulp.plan.io/issues/7118
* 0.7.1
* Current plan: https://pulp.plan.io/versions/190
* Should we include?
* FILES.json issue - https://pulp.plan.io/issues/8292 (not needed)
* DRF token x Bearer token - https://pulp.plan.io/issues/8290#note-4
* auth_url=None => DRF token?
### Notes
* 0.7.1
* galaxy-importer does not currently validate FILES.json
* Need 0.7.1 released ASAP to unblock content sync performance fixes
* Token:
* Community: no auth_url, DRF token
* c.rh.c: auth_url, Bearer token
* satellite: no auth_url, DRF token
* Inclusion of auth_url triggers Bearer token header
* auth_url / keycloak auth response defines what token type to use, but should always be Bearer in practice
* Needs to be backported to pulp_ansible 0.5.x
* pulp_ansible sync from standalone galaxy_ng broken without fix
### Actions
* [awcrosby] add Jira issue for validating FILES.json
## 23 February 2021
### Agenda
* Does the pulp installer automatically handle the new SHA1 and MD5 changes?
* Collection missing FILES.json https://pulp.plan.io/issues/8292
* Delete PRD available for review
* https://docs.google.com/document/d/1-SzizlhQ_43AN_QP6XqpQ78S4r1Ce_MHb-6YzOAwWBw/edit?usp=sharing
* Unpaginated Serializer on 0.7.1? https://github.com/ansible/galaxy_ng/pull/648
* we need to define the fields https://pulp.plan.io/issues/8264
* Unable to sync from galaxy with token https://pulp.plan.io/issues/8290
*
### Notes
* galaxy_ng dropped MD5 & SHA1?
* An existing customer could override in settings.py
* Very old artifacts could have MD5 & SHA1 checksums
* The pulp installer does not handle this
* The sync process won't bring the MD5 & SHA1 checksums down to local hub, if the local hub has these turned off
* Community Galaxy collections missing FILES.json
* Option 1: Remove offending content from community galaxy
* Option 2: Make the sync operation defensive
* Unpaginated serializer
* galaxy_ng needs to release with 0.7.1
* 0.7.1 should include this bug fix
* Bruno's PR is blocked until this lands
* include also this on 0.7.1 https://github.com/pulp/pulp_ansible/pull/523
* Can't sync from AH using a token
* Satellite is trying to sync from standalone AH to Satellite
* https://pulp.plan.io/issues/8290
### Actions
* [galaxy_ng] Need to perform upgrade testing for MD5 & SHA1
* [gerrod] provide the collection name(s) failing on community galaxy due to FILES.json
* [galaxy_ng] will look into which collections are failing and whether or not they can be removed
* [adrian] reach out to Samir (sahja) on the Satellite team and help troubleshoot https://pulp.plan.io/issues/8290. John Hardy can provide access to the VMs.
## 16 February 2021
### Agenda
* Broken Pulp API spec
* OpenAPI spec generator tweaks for more concrete urls
* https://github.com/pulp/pulpcore/pull/1122
* Want to figure out if this can go upstream, and if not, is there a good way to do it downstream.
* Need this for some of the tooling that cloud.redhat.com uses the OpenAPI spec for which currently breaks on the generated spec.
* Are there any lessons learned from this discussion https://github.com/ansible/galaxy_ng/pull/648 ?
* Why was `highest_version` removed? https://github.com/pulp/pulp_ansible/pull/469/files#diff-e11a4087ff75182a1c5c58b3812e77def3dccfe96c12e2f6261760225d5c41d2L217
* it was not removed, it changed https://github.com/pulp/pulp_ansible/blob/master/pulp_ansible/app/galaxy/v3/serializers.py#L67-L82
* available_versions is an ARRAY AGG which brings a list of versions [0.0.1, ..., 10.0.0] and we get the highest at serializer side
* pulp-ansible 0.7.0
* shipped the collections endpoint query fix (would be good to test against a dump from production)
* import/export as tech preview (not handle deprecation yet)
* rate limit feature
* optimize=True by default (no-op requires remote to be associated to repository)
* https://docs.pulpproject.org/pulp_ansible/changes.html
* 0.8.0?
* EE: https://github.com/pulp/pulp_ansible/pull/518
* katello needs pulp_ansible to have token auth support like galaxy_ng has
* https://pulp.plan.io/issues/7118
* A new query to show all glaxy_ng stakeholder work in pulp.plan.io
* https://pulp.plan.io/issues?query_id=170
* what issues need to be tagged in this way?
### Notes
* open API
* This is a hard problem that isn't solvable
* Can cloud.redhat.com enable unsafe chars in their parser to allow for our spec?
### Actions
* [fabricio, awcrosby] reach out to c.rh.c team to figure out if we can fix the open api stuff on their side
* [bmbouter] file an issue to remove the unneeded data from the the repo all collections endpoint
* [bmclaughlin] check in with the builder team to see if https://github.com/pulp/pulp_ansible/pull/518 is still needed for platfor 2.0
## 9 February 2021
### Agenda
* pulpcore 3.10 https://pulpproject.org/2021/02/04/pulpcore-3.10-is-now-generally-available/
* Labels
* https://docs.pulpproject.org/pulpcore/workflows/labels.html
* https://asciinema.org/a/389900
* Tasking system reliability improvements
* rate_limit option on all Remotes
* fosdem talk user feedback/questions
* most upvoted question: Can galaxy_ng connect to LDAP instead of local users?
* second most upvoted question: Can galaxy_ng import from git repos like community galaxy can?
* https://pulp.plan.io/issues/3363
* third most upvoted question: Where is the effort going between galaxy_ng versus community galaxy?
* pulp_ansible 0.7 to be released tomorrow/thursday, to be compatible with pulpcore 3.10 and 3.11
* proposal: adding migration to populate initial data
* https://issues.redhat.com/browse/AAH-281
* update on performance testing progress
* two AWS boxes are setup, initial data population will occur from c.rh.c
### Notes
*
### Actions
*
---
## 2 February 2021
### Agenda
* Make installing initial data easier - https://issues.redhat.com/browse/AAH-281
* pulp dev env ("pulplift") loads data from fixture - https://github.com/pulp/pulp_installer/pull/521
* How to handle CollectionImports when importing/exporting
* Skip exporting/importing of CollectionImports
* Adding negative requirements file (like an exclude list) https://pulp.plan.io/issues/8194
* State of Sync Performance https://pulp.plan.io/issues/7938
* no-op: https://github.com/pulp/pulp_ansible/pull/511
* using the new endpoints: https://github.com/pulp/pulp_ansible/pull/512
* Pulp Ansible 0.7.0 to release Feb 9th
* anything going into this release needs to be merged by EOB on Feb 8th
* https://github.com/pulp/pulp_ansible/pull/505
* Pulp Ansible scale testing work starting next week
* update scripts to test large repository counts https://pulpproject.org/2020/03/05/pulp-3-scale-testing/
* want similar hardware used for galaxy api https://docs.google.com/document/d/1L--GCumzS4dEeOCjIXfCvUv2UbSYOncWxbN0YtqmjHQ/
* Working on resolving RBAC for non-NamedModelViewSets
* https://pulp.plan.io/issues/8048
### Notes
* Installation data
* Create a plan before next meeting, and we can prioritize from there
* Import/Export
* Confirmed that galaxy_ng does not require Collection import logs for any imported (i.e., restored) Collections
* Adding negative requirements file
* Needs to be vetted by product management
* Need a plan for how to implement on c.rh.c
* State of sync performance
* Need to follow and comment on the related PRs
* On track for release of 9 February
* RBAC
* Default policy for all endpoints will by 'deny', unless the user is a 'superuser'
* Not blocking galaxy_ng
### Actions
* Bruno - review and comment on AAH-281, and work with Brian to create a plan.
* Brian - review negative requirements file with John Hardy and develop a plan for c.rh.c
* Bruno - continue following sync performance PRs
* Alex/Gerrod - resolve unanswered questions / missing numbers for scale testing
## 26 January 2021
### Agenda
* Content provenance update.
* One 0.7 feature for core team:
* syncing MANIFSET.json and DIGESTS.json https://github.com/pulp/pulp_ansible/pull/468
* https://pulp.plan.io/issues/7572
* pulpcore 3.10 to release Feb 4th
* 0.7.0 to release on Feb 9th
* two 0.8 features
* syncing dependencies - https://pulp.plan.io/issues/7751
* import/export - https://pulp.plan.io/issues/6738
* May release end of February
* pulp-cli
* supports pulp_ansible
* demo: https://drive.google.com/file/d/1f3A3Zpi4d1nALyeUapTcAW7-2-C-z3cP/view?usp=sharing
* 0.2.0 just released - https://www.redhat.com/archives/pulp-dev/2021-January/msg00045.html
* pulp dev env
* debug: https://drive.google.com/file/d/1EOSne7aDDTfkrrcO5xnUBhf-9Z0mJD9L/view?usp=sharing
* wiki has been updated
* Pulp CentOS 8 single container
* https://github.com/ansible/galaxy_ng/pull/618
* https://github.com/pulp/pulp-oci-images/pull/57
* used on the mentioned demo above
* Tasking system improvements in Pulp 3.10
* https://github.com/pulp/pulpcore/pull/1058
* Do you need this backported?
* pulp_ansible 0.5.x -> pulp 3.9. Would need to upgrade to pulp_ansible 0.7.x
* How to discuss a different way to populate initial data?
## 19 January 2021
### Agenda
* pulp_ansible 0.7 to be released at end of Jan
* sync performance (using new /metadata endpoints) - https://pulp.plan.io/issues/7938
* rate limiting - https://github.com/pulp/pulp_ansible/pull/496
* partner engineering - exposing ansible version
* https://github.com/pulp/pulp_ansible/pull/466#discussion_r552720930
* namespace sync - on hold
* content provenance - ?
* Changes needed for pulpcore 3.10 compatibility
* https://github.com/ansible/galaxy_ng/pull/616
* galaxy_ng dev environment update
* pulp-ansible/galaxy-ng demo FOSDEM/devconf.cz
* single container
* data fixture
* notes here: https://hackmd.io/@pulp/ansible-containers
* (coleslaw? ;-> https://github.com/alikins/coleslaw)
* pulp-cli is getting ansible support
* https://github.com/pulp/pulp-cli
* https://github.com/pulp/pulp-cli/pull/63
## 12 January 2021
### Agenda
* Testing collections inside containers when running on OCP
* A prebuilt image is used to download a collection and test it
* Collections query - post mortem -- take 2
* [bmbouter] pulp_ansible needs to have automated tests as a way to avoid issues like the collection query 500 error
* Make consistent pulp_ansible namespace max_length in Collection and CollectionVersion https://pulp.plan.io/issues/8078
* 0.5.6
* https://pulp.plan.io/versions/172
* Broken pulp core permissions endpoint
* https://pulp.plan.io/issues/8048
* pulpcore 3.10 - releasing Jan 26th
* object labels are a new feature
* openapi fixes
* unset a DRF field
* https://issues.redhat.com/browse/AAH-174
* continuing on the Redis failure/auto-recovery work
* https://github.com/pulp/pulpcore/pull/1058
### Notes
* pulpcore 3.10
* January 26
* Includes OpenAPI spec fix
### Action
[fao] schedule meeting with Calvin, Henderson & Davnd N. to talk about adding more galaxy automated tests for pulp_ansible.
[Bruno] Ask Dennis/IRC how to handle the unset issue on DRF
## 5 January 2021
### Agenda
* Collections query - post mortem
* pulp_ansible failed to communicate an important change on the query
* Should we require reviews from galaxy in the future? QE reviews?
* Tasking System Improvements
* bugfix merged: https://github.com/pulp/pulpcore/pull/1046
* bugfix ready for review: https://github.com/pulp/pulpcore/pull/1058
* Expose requires_ansible
* https://github.com/pulp/pulp_ansible/pull/466
* question: does the importer provide "requires ansible"?
* pulp_container running on OCP. What are the obstacles to overcome?
* we used to require root access for container building but that was fixed.
* Issue https://issues.redhat.com/browse/AAH-179 - Fail to sync a collection from community repo if collection already exists on rh-certified repo:
```python
duplicate key value violates unique constraint "unique_is_highest"
DETAIL: Key (collection_id, is_highest)=(29f416e2-27e6-46ad-9b10
f520dd5d5dfd, t) already exists.
```
vmware collection
### Notes
* Tasking system improvements (pulpcore)
* Backporting fixes to 3.7
* https://pulp.plan.io/issues/7950
* Does not include any migrations
* 'requires_ansible'
* Jira issue created in galaxy_ng for galaxy-importer to expose 'requires_ansible'
* Will be included in Sprint 3
* pulp_container on OCP
* We originally attempted to build and exec container images on OCP (using pulp_container), and ran into issues around 'root' access and a FUSE requirement.
* It 'may' work perfectly fine on OCP for storing and delivering images.
* Has it been tested on OCP 4?
* Has it been tested on OSD 4 (c.rh.c)?
### Action
* [fao] monitor future pulp_ansible PRs and request reviews from the galaxy_ng as appropriate.
* [fao] require galaxy team ack on pulp_ansible release PRs
* [davidd] setup a 'galaxy' team inside the pulp namespace, make it a code owner of the pulp_ansible project, and insure that a code owner is required to sign-off on PRs.
* [awcrosby] plan for testing and deploying tasking system changes to the cloud
* [Bruno] fill pulp plan io issue for 179
## 08 December 2020
### Agenda
* Did pulp_ansible 0.5.4 solve the rate limiting issue?
* Tasking System Improvements
* bugfix merged: https://pulp.plan.io/issues/7907, https://github.com/pulp/pulpcore/pull/1046
* what version of pulpcore to backport to?
* health-check-with-auto-cancellation in progress: https://pulp.plan.io/issues/7912
* Sync metadata improvements:
* https://docs.google.com/document/d/1lK_I7nbGOido4O1IU7qu8uXXVv7bOVryRzTNjZgYCK8/edit#
* pulp_ansible epic: https://pulp.plan.io/issues/7938
* pulp_ansible stories to write (today):
* signing collection metadata [bmbouter]
* rate limiting for CollectionSync [bmbouter]
* ansible base - https://pulp.plan.io/issues/7949
* https://hackmd.io/@pulp/galaxy_and_partner_engineering
* review of 4.3.0 pulp_ansible related items
* https://docs.google.com/document/d/1IEYQCIJEJ4QmSBABL-nn5lv8hQDbSAbitrCjuxtjIYA/edit
* https://github.com/pulp/pulp_ansible/pull/458 (aiohttp tweaks to reuse connections, and limit_per_host) - any promise?
### Notes
*
### Action
* [davidn/house] Communicate the rate limiting limitations, temporary fix, and future plans to the other Ansible App teams
* [davidn] Add "number of downloaders" parameters to Remote Configuration in UI
* [bmbouter] File ticket to backport tasking system improvements to pulpcore 3.7.x. Will investigate backporting the health check as well.
* [awcrosby] Add an issue in Jira to track tasking system updates in 3.7.x. GalaxyNG needs to test and confirm the user experience is reasonable.
* [alikins] Call out the pulp_ansible rate limit feature in the Content Sync Performance architecture doc. Specifically, the UI will need to surface any new remote configuration options.
* [bmbouter] Have an additional think on pulp/pulp_ansible/pull/458.
## 01 December 2020
### Agenda
* Pulp Tasking System
* Pulp doesn't have any task timeouts or stalled tasks detection mechanism. It’s possible, especially with a low number of workers, for the tasking system to deadlock.
* Redis not being reliable, and things getting out of sync should Redis go down
* Solution: remove Redis
* We're treating it as a longterm storage facility, and it's not
* Sudden worker failure, leaving the task in an unknown state
* This is already handled well by Pulp resource manager
* Pulp has not been able to reproduce a situation where a worker dies and resources are not cleaned up
* We have a database dump. Need to inspect it, and maybe start a single worker and resource manager to see what happens to the remaining tasks. They should get canceled.
* If a task is cancelled via the API it is marked as cancelled, but it may still be in the Redis Queue, and it ends up being executed despite the request to cancel. When a worker receives a new task it should check the status and reject a cancelled task.
* Issue: https://pulp.plan.io/issues/7389
* Status: Fixed in pulpcore 3.7.0
* Note: As of 01-12-2020, the automation-hub-prod is deployed from image tag `1326eb5` and uses pulpcore 3.7.1. Reopen and verify the issue.
* We actually don't know if this is busted, because we cancelled the tasks via SQL, not the API.
* 520/403 status (potential blocker)
* reducing sync concurrency helps but does not solve the problem
* rate limiting can affect other services
* possible solutions:
* leaky bucket algorithm: not solve multiple syncs at same time
* proxy
* How to configure number of pulp workers in pulp installer?
```yaml
pulp_workers: 4
```
https://pulp-installer.readthedocs.io/en/latest/roles/pulp_workers/
* pulp_ansible 0.6.0 released with Github Actions
* Can we edit group names?
* What is the minimum number of workers required to run pulp?
* Object label update https://pulp.plan.io/issues/7127
### Notes
*
### Action
* [bmbouter and osaprykin] schedule a meeting for the tasking system issue
## 17 November 2020
### Agenda
* Planning next feature release/feature freeze of pulp_ansible
* Sync with large requirements.yml
* https://www.redhat.com/archives/galaxy-ng/2020-November/msg00003.html
* Total requests (723 collections/ 4922 collections versions):
723 - collections endpoint e.g. https://galaxy.ansible.com/api/v2/collections/pulp/pulp_installer/
723 - collections versions endpoint e.g https://galaxy.ansible.com/api/v2/collections/pulp/pulp_installer/versions/
4922 - collection version endpoint e.g. https://galaxy.ansible.com/api/v2/collections/pulp/pulp_installer/versions/3.8.1/
4922 - docs-blob endpoint e.g. https://galaxy.ansible.com/api/v2/collections/pulp/pulp_installer/versions/3.8.1/docs-blob/
4922 - tar file endpoint e.g. https://galaxy.ansible.com/download/pulp-pulp_installer-3.8.1.tar.gz
* Sync requests x status 520
* only succesfully sync when disabling request to docs-blob and tar file endpoints
* pulp_ansible 0.5.2 x 0.6.0
* 0.5.2 - https://pulp.plan.io/versions/159
* 0.6.0 - https://pulp.plan.io/versions/160
* Main difference, tag name issue e.g. red_hat_ansible_certified_content_for_ibm_z
* Moving pulp CI/CD infrastructure to Github Actions
### Notes
* Investigate alternate location for galaxy_ng upstream RPMs
* maybe we don't need that
* Planning next feature release/feature freeze of pulp_ansible
* Planning may help bring more people to pulp ansible team
* Status 520
* slow down the sync?
* backoff 520?
* proxy configuration?
* rate limiting?
* galaxy_ng 4.2.1 will depend on pulp_ansible 0.5.2
* galaxy_ng 4.3.0 will depend on pulp_ansible 0.6.0+
* CI/CD
* https://travis-ci.com/github/ansible?tab=insights
* https://blog.travis-ci.com/2020-11-02-travis-ci-new-billing
### Action
* [bmbouter/fabricio] pulp_ansible 0.5.2 releasing 19 November
* [fabricio] to schedule a meeting with alex about status 520
## 03 November 2020
### Agenda
* How can we facilitate pulp-ansible contribution?
* https://github.com/ansible/galaxy_ng/pull/522#issuecomment-717923675
* How to test the installer with CI
* Shipping galaxy_ng releases 3-months ahead of downstream releases
* RPM builds
* https://github.com/theforeman/pulpcore-packaging/pull/61
* Fixing the token bearer refresh with every request sync problem
* https://github.com/pulp/pulp_ansible/pull/406
* @jborean requesting a behavioral change to newly changed Galaxy V3 viewset
* https://pulp.plan.io/issues/7775
* Is there a way to configure the `mirror` option for a remote in the db?
* https://github.com/ansible/galaxy_ng/blob/master/galaxy_ng/app/api/v3/views/sync.py#L50
### Notes
* Facilitating pulp_ansible contribution
* Make it possible to run pulp_ansible from a checkout in the galaxy_ng dev environment
* Testing end-user installs
* Handle via CI, rather than attempting to move dev to pulplift
* If we don't get this addressed, there's risk that we can't release because galaxy_ng's release cycle is out of phase with pulpcore and pulp_installer.
* Shipping galaxy_ng releases 3-months ahead of downstream releases
* Most products at RH ship the upstream 3 months ahead of a downstream
* RPM builds - need a place to host our own, separate from Katello
* @jborean requesting a behavioral change to newly changed Galaxy V3 viewset
* There's a bug fix coming in pulp_ansible (0.5.1)
* galaxy_ng will incorporate in a z stream release post GA
* Configuring `mirror` option for a remote
* Not currently stored in the DB
* Needs more discussion to see if it aligns with vision of other stake holders
* For now, galaxy_ng will default `mirror` to True
### Action
[davidn] File an issue in pulp.plan.io regarding `mirror`
[davidn] Modify galaxy_ng to default `mirror` to True
[house] Investigate alternate location for galaxy_ng upstream RPMs
## 27 October 2020
### Agenda
* **[BLOCKER\URGENT]** pulp/pulp-ci latest is built without uuid-ossp extension in PostgreSQL. https://pulp.plan.io/issues/7754
* Deprecation PR status
* Breaks collection search
### Notes
* Deprecation PR status
* UI API in galaxy_ng will work around the collection search issue
### Action
## 20 October 2020
### Agenda
* Hub API structure
* https://hackmd.io/7n_R24frRtGey4EJMIRPLQ
* SQL performance: https://pulp.plan.io/issues/7564
* A request to '/api/automation-hub/v3/collections/' in our CI environment it generates a WHERE clause that includes ALL collections from the entire database.
* PoCs
* https://gist.github.com/cutwater/10228c80ce0da619163769bd989b79b9
* https://github.com/pulp/pulp_ansible/compare/master...fao89:version_index
* CLI - https://github.com/pulp/pulp-cli
* https://pulpproject.org/2020/09/28/pulp-3-cli-poc-call-for-feedback/
* Can galaxy_ng use this to build CLI commands?
* CLI commands for galaxy_ng?
* django-admin
* https://github.com/pulp/pulpcore/pull/838#issuecomment-707147652
* Manifest integration with CLI
* next steps?
* SELinux policy not supporting unix sockets
* I'm not sure how Yanis provisioned those boxes, but the "test suites" require dev installs, and I maintain [a pulp_installer branch with helper scripts & more](https://github.com/mikedep333/ansible-pulp/tree/selinux-dev-el8) to run them. Yanis probably didn't apply that branch.
### Notes
* Hub API structure
* Too big for the scope of this meeting.
* There's 2 or 3 big issues to resolve related to API structure, evolving the CLI to match API structure, evolving the CLI to resolve dependencies across multiple content repositories.
* Needs to involve product management and pulp_ansible teams
* SQL performance
* At run-time we try to determine highest version. Inefficient to compute this over and over.
* Pioritizing things: deprecation, fixing`requirements.yml` (https://pulp.plan.io/issues/7686), and then SQL Performance
* We could potentially do this workaround: https://meet.google.com/linkredirect?authuser=1&dest=https%3A%2F%2Fgithub.com%2Fansible%2Fgalaxy_ng%2Fblob%2Fmaster%2Fgalaxy_ng%2Fapp%2Fapi%2Fui%2Fviewsets%2Fcollection.py%23L36-L53
* CLI
* This is something we should evaluate after GA.
* Probably the thing to use in place of Django Admin
* Django Admin
* Can we use pgadmin instead?
* Maybe building on top of the Pulp CLI?
* SELinux policy not supporting Unix sockets
### Action
## 13 October 2020
### Agenda
* mutable fields - deprecation prototype
* https://github.com/pulp/pulp_ansible/pull/389/files
* Can't have mutable attributes on content
* Deprecation flag was not being copied during sync
* Original issue: https://github.com/ansible/galaxy_ng/issues/438
* pulp_ansible 0.5.0 status and galaxy_ng rc2 release timing
* collection / collection version deletion
* Design
* Can it be delivered for galaxy_ng GA (12 Nov)
* SQL performance: https://pulp.plan.io/issues/7564
* A request to '/api/automation-hub/v3/collections/' in our CI environment it generates a WHERE clause that includes ALL collections from the entire database.
* PoCs
* https://gist.github.com/cutwater/10228c80ce0da619163769bd989b79b9
* https://github.com/pulp/pulp_ansible/compare/master...fao89:version_index
* CLI - https://github.com/pulp/pulp-cli
* https://pulpproject.org/2020/09/28/pulp-3-cli-poc-call-for-feedback/
* Can galaxy_ng use this to build CLI commands?
* CLI commands for galaxy_ng?
* django-admin
* https://github.com/pulp/pulpcore/pull/838#issuecomment-707147652
* requirements.yml validation for v3/collections in URL
* https://issues.redhat.com/browse/AAH-12
* https://ansible.slack.com/files/U5PKWP6GK/F01C4ESNCPQ/image.png
* https://pulp.plan.io/issues/7686
* Hub API structure
* https://hackmd.io/7n_R24frRtGey4EJMIRPLQ
### Notes
* Deprecation PR
* evaluate the impact on synclist
* may need wrapper tasks in pulp_ansible for add_and_remove task that also handles deprecation/metadata etc
* galaxy_ng synclists tasks would need to be updated to use the new "add_and_remove_collections_and_deprecation_stuff_to_repo_versions" tasks from pulp_ansible
* [david] I recommend we add this to pulp_ansible instead (see https://pulp.plan.io/issues/7621)
* Collection / Collection Version Deletion
* Need actual requirements (i.e., define what 'delete' means)
* Can we do something for GA release of on-prem Hub
* Performance tuning
* Need to reach shared goals around database and system sizing, and latency and throughput of specific endpoints
* Django Admin
* Could we deploy PGAdmin?
* Use the Pulp API
* Move the PRs to galaxy_ng for read/write support
### Actions
* [alikins, davidn] Review deprecation PR https://github.com/pulp/pulp_ansible/pull/389
* [house] Create galaxy_ng Jira to track deprecation changes
* [house] Talk to John Hardy regarding priority of Delete function
## 29 September 2020
### Agenda
* Remove 'certification' flag from CollectionVersion
* https://github.com/pulp/pulp_ansible/pull/369
* Performance issues https://pulp.plan.io/issues/7564
* Most of the problematic queries were related to "certification"
* For GalaxyNG `/api/v3/` points to a default repository and contains all of our non-repo scoped functionality (auth, namespaces etc). We need to figure out what the correct functionality of this endpoint should be. Is having it point to a default repo and contain all our non-repo functionality the correct thing to do?
* pulpcore 3.7.1 releasing today
* Pulp lead switch
* fao89 will be new lead, daviddavis will be supporting lead
* bmbouter will start focusing on katello (swapping with daviddavis)
* Deprecation handling in pulp_ansible 0.5.0
* The deprecated attribute of a collection_version will be scoped at the repository level, rather than globally.
* https://github.com/pulp/pulp_ansible/pull/366/files
### Notes
* pulp_ansible 0.4.0 was released - this is the pulp 3.7.0 compatibility relesase.
* pulp_ansible 0.5.0 should release tomorrow, 30 September
* For performance issues the Galaxy team will need to do more testing once pulp_ansible 0.5.0 lands
* preliminary showurls https://gist.github.com/alikins/122eb7450aed624441fb04904e7c58ba
* More focused version with some markup
* https://github.com/ansible/galaxy_ng/blob/master/galaxy_ng/app/tasks/synclist.py#L98-L155 SyncList Tasks that are based on the pulpcore.app.task.repository.add_or_remove task.
* Will these do the right thing for The New Deprecations[tm]?
* Sounds like it likely doesn't
### Actions
* [David] organize a walkthrough of show_urls() and a discussion of how to organize non-repo or global functions.
* [fao89] make sure the 'deprecated' attribute is handled correctly during a copy operation.
* [alikins] work with fao89 to complete the 'deprecated' attribute work, sharing any code from galaxy_ng where we expect 'deprecated' to be handled.
## 22 September 2020
### Agenda
* pulpcore 3.7.0 is going out today
* FIPS supported added in pulpcore 3.7
* Please test out galaxy_ng in a fips env
* Will need to set this in settings.py:
* `ALLOWED_CONTENT_CHECKSUMS = {“sha224”, “sha256”, “sha384”, “sha512”}`
* pulp_ansible 0.4.0 issue roundup
* https://pulp.plan.io/versions/137
* Deprecation as a per-repo-per-collection
* Project Name: "Deprecate Deprecate"
* Current status:
* does not sync deprecation status from remote
* pulp_ansible is the "source of truth" for deprecation
* collection deprecation status is the same for all repos, and can only be updated at pulp_ansible
* Proposal:
* sync deprecation status and save it per repo
* "source of truth" for deprecation would be the remote
* collection deprecation status depends on the repo
* Liveness and Readiness probes for pulp services
* content app
* resource manager
* workers
* Tasking system monitoring
* New way to depend on pulpcore
* https://github.com/pulp/pulpcore/blob/master/docs/plugins/plugin-writer/concepts/index.rst#plugin-api-stability-and-deprecation-policy
### Actions
- pulp_ansible 0.4.0 needed on or before 29 September
### Notes
- galaxy_ng compatibility release for 3.7.0 will be the 4.2.0rc release due October 2ish
- QE - Update CI to test with FIPS enabled
- galaxy_ng 4.2.0rc feature freeze is 30 September
- pulp issues discussed
- https://pulp.plan.io/issues/7504
- https://pulp.plan.io/issues/7428
- openshift readyness probes etc
- https://www.openshift.com/blog/liveness-and-readiness-probes
- https://docs.openshift.com/dedicated/4/applications/application-health.html
## 8 September 2020
### Agenda
* pulp_ansible 0.3.0 blockers:
* sync docs_blob: WIP - https://github.com/pulp/pulp_ansible/pull/355
* get endpoint from CI: https://ci.cloud.redhat.com/api/automation-hub/v3/collections/
* requirements.yml sync: WIP - https://github.com/pulp/pulp_ansible/pull/353
* on_demand policy for syncing:
* https://pulp.plan.io/issues/7470#note-1
* https://github.com/ansible/galaxy_ng/blob/7b535dd9e0ffa4b6b6081f4cdcd301565b907f80/galaxy_ng/app/api/v3/serializers/sync.py#L12
* Can we get pulp ansible 0.3.0 pypi packages in time for thursday?
* What to do about limiting access to pulp_ansible API urls
* pulpcore 3.7.0 includes two fixes for tasking system
* https://github.com/pulp/pulpcore/pull/867
* https://github.com/pulp/pulpcore/pull/891
* pulpcore 3.7.0 FIPS and SELinux update
* in-progress, on-track
* pulpcon is next week, need to cancel this meeting
* https://hackmd.io/@pulp/pulpcon2020_schedule
* How to enable galaxy_ng to override /pulp/api/v3/pulp_ansible/
* Focus on nginx for now. It's all galaxy_ng uses.
* Plan A: [Test/verify that no change is needed for a `location` snippet that overrides it](https://www.digitalocean.com/community/tutorials/understanding-nginx-server-and-location-block-selection-algorithms#how-nginx-chooses-which-location-to-use-to-handle-requests). It should work because it is more specific, even though we include the snippet later.
* Plan B: [Move the include from line 100 to line 79](https://github.com/pulp/pulp_installer/blob/master/roles/pulp_webserver/templates/nginx.conf.j2#L100) (before `location /pulp/api/v3`). Then Test existing plugins w/ snippets w/ this change to ensure no regressions.
* mikedep333 to write this up, and probably work on this. https://pulp.plan.io/issues/7471
### Actions
### Notes
* fixes for requirements.yml and docs_blob should be ready for 0.3.0
* testing for docs blob can be done on the ci server: https://ci.cloud.redhat.com/api/automation-hub/v3/collections/
* 0.3.0 should be ready for thursday
* Disabling pulp_ansible apis
* we can potentially override pulp_ansible webserver config: https://github.com/pulp/pulp_ansible/tree/master/pulp_ansible/app/webserver_snippets
* https://github.com/pulp/pulp_installer/blob/master/roles/pulp_webserver/templates/nginx.conf.j2#L80-L100
## 1 September 2020
### Agenda
- New code freeze/beta release date: September 10th
- Updates on sync issues:
- pagination (https://pulp.plan.io/issues/7396)
- merged: https://github.com/pulp/pulp_ansible/pull/348
- need a galaxy_ng PR to switch to pulp_ansible machinery to de-duple our code
- requirements_file breaking sync (https://pulp.plan.io/issues/7412)
- [fao89]: I plan to take it after fixing the installer (probably today or tomorrow)
- meta data (https://pulp.plan.io/issues/7397)
- merged: https://github.com/pulp/pulp_ansible/pull/351
- requirements_file is max 255 char (https://pulp.plan.io/issues/7434)
- merged: https://github.com/pulp/pulp_ansible/pull/352
- Call for changes to be in pulp_ansible 0.3.0 release?
- https://pulp.plan.io/issues/7428 [not blocker]
- https://pulp.plan.io/issues/7412 [blocker]
- https://pulp.plan.io/issues/7439 [blocker]
- pulpcore 3.7.0 will include a fix to the cancellation bug @osaprykin reported
- https://pulp.plan.io/issues/7389
- it includes a migration, so it can't go out in pulpcore z-stream
- 3.7.0 slated to release Sept 22nd
- FIPS and SELinux work update being done by pulp-team
- add pulp_ansible to the test environment
- Downstream RPMs (and branding)
- Questions about RBAC implementation
- Could something like AutoAddPermissionsMixin be used in galaxy_ng?
- How to debug perms/RBAC issues?
- Need to continue on django-admin models PR
- https://github.com/pulp/pulpcore/pull/838
- https://github.com/pulp/pulp_ansible/pull/313
### Actions
- [newswangerd] test collection sync
-
### Notes
* CI meeting
* As osaprykin pointed, pulpcore 3.5 was not being installed at stable branch: https://github.com/ansible/galaxy_ng/pull/407
* Started a PoC for run the CI locally: https://github.com/ansible/galaxy_ng/pull/405
* kudos to osaprykin for the feedback on the CI
## 25 August 2020
### Agenda
* Task management issues (osapryki)
* https://pulp.plan.io/issues/7386
When task that does not exist in resource-manager \ worker is pushed to queue,
it remains in ``waiting`` state forever.
How to reproduce:
1. Deploy API with newer version of code.
2. Enqueue task.
Use case: New version of worker is deployed earlier than API version.
* https://pulp.plan.io/issues/7387
Tasks not delivered to pulp-resource-manager persist in `waiting` state.
* https://pulp.plan.io/issues/7389
* Travis CI for ``galaxy_ng``, branch ``stable`` is broken.
### Notes
* Haven't met with dkliban yet to talk about UI
* Tasks issues:
* 7387: Duplicate of https://pulp.plan.io/issues/6449
WIP: https://github.com/pulp/pulpcore/pull/712
* 7386: Not assigned yet. Attempting to add to Sprint 80 (ends aug 4th)
* 7389: On Sprint 80 (ends aug 4th)
* pulpcon tentative schedule: https://docs.google.com/spreadsheets/d/1c4HJ47kgCqiL2EknnmIOHvmw8RgaUwLTiohBuGzH47k/edit#gid=0
* Talk on lockless pulp that might be of interest to Ansible Galaxy.
### Actions
* [dkliban] Setup a meeting with David Newswanger and Zita Nemeckova to get a tour of the galaxyng UI
* [fao] schedule a meeting with Oleksandr to talk about the CI
## 18 August 2020
### Agenda
* The drf_spectacular opeanpi v3 and django-admin changes are awesome and make development 5x better! (alikins)
* Surfacing content from multiple repositories (e.g., Search)
* ~~Task management issues (osapryki)~~
* *Not done. Moved on August, 25th.*
* Join galaxy_ng + pulp_ansible marketing meeting
* Wed 11am - 11:30am EDT
* https://hackmd.io/i6bLP9juQleQG8wYD3proQ
* "As this relates to Ansible, we have a great chance of quick publication on opensource.com. Your outline is a good start. The next steps would be to identify the user, what is the main use case(s), why would anyone want to use this, and why is it great."
* @dkliban wants to meet to learn a bit more about the UI
* Move chat to irc
* Improve engagement with community
* Access to full Pulp (#pulp-dev), Satellite, and Build teams
### Notes
### Actions
* [dkliban] Setup a meeting with David Newswanger and Zita Nemeckova to get a tour of the galaxyng UI
* [bmbouters] Invite House and David Newswanger to the marketing meeting
* [osapryki] Share tasking issues with David Davis
* [house] Explore killing the #galaxy channel on Slack to encourage the move to irc
## 11 August 2020
### Agenda
* Groups API - Need 'writable' endpoints
* Holding the 3.6 Pulp release for Groups API changes
* Associating remotes with repositories [#7015](https://pulp.plan.io/issues/7015)
* Done. Will be included in pulpcore 3.6
* Django Admin
* Will be adding support in Pulp 3.7
* pulp_ansible 0.2.0GA release
* bmbouter will review and merge [#342](https://github.com/pulp/pulp_ansible/pull/342)
### Notes
### Actions
* [osapryki] Report details related to task management issues:
* If redis task was not delivered to content (resource?) manager, task remains in `waiting` state forever.
* Content manager processes tasks received from redis even if tasks are in `cancelled` state.
```
2020-08-11 12:26:35,464 WARNING pulpcore.app.models.task: Task __call__() occurred but Task a1b922c3-677c-4fb3-bd1d-3c4f9c727b05 is not at WAITING
```
* [bmbouter] - Port PRs from @alikins for adding models to Django Admin
## 4 August 2020
### Agenda
* Update on RPM testing
* Planning to release pulp_ansible 0.2.0 GA soon after pulpcore 3.6 release on Aug 11th
* What is required to remove "certified" from pulp_ansible?
* https://pulp.plan.io/issues/6715
* pulp_installer update: health checks
* Requesting backports to pulp_ansible
### Notes
* PR for pointing to pulp and pulp_ansible master branches
* Hitting a problem with pulp_container. Pulling from the master branch of pulp_contianer seems to fix it though.
* Latest pulp_container package is not compatible with Pulp 3.6, so guidance is to point to the master branch for pulp_container
* RPMs
* QE reported a bug related to the Pulp install directory not being set. See https://pulp.plan.io/issues/7255
* Please review: https://github.com/pulp/pulp_installer/blob/3.5.0/roles/pulp_common/defaults/main.yml#L18
* Removing "is_certified" flag
* Need to get content promotion moved to the new "multi-repo" world
* pulp_installer health checks
* Checks that all the processes are running. Once all the roles finish we need to know that everything is reliable and ready-to-go.
* Porting health checks to Ansible URI module (from Ruby)
* The check will hit the API through the web server
* Anticipating the work to be done in a matter of days
* RBAC
* Tests are passing
* Just need to update docs
* PR should be merged this week, putting it on schedule for the 3.6 release.
### Actions
## 28 July 2020
### Agenda
* Repo/remote relation (https://pulp.plan.io/issues/7015) has been merged
* Planning needed for the content protection feature
* https://pulp.plan.io/issues/7118
* needs auth_url determined
* needs the way to hand out the initial token handled
* PulpCon (Sept 14-18)
* https://hackmd.io/hIOjFsFiSkGJR7VqtAJ8eQ
* RPMs
* Evgeni was able to build
* Testing?
* Switching galaxy_ng to using pulp_ansible:master and pulpcore:master
* when and how
### Notes
* RBAC - davidn submitted https://pulp.plan.io/issues/7227
* Content protection feature
* Enables protection of a distribution with a content guard
* GalaxyNG uses DRF token auth today
* Will evaluate what GalaxyNG does, and plan next steps and future design
* PulpCon - submit topics @ https://hackmd.io/hIOjFsFiSkGJR7VqtAJ8eQ
* RPMs
* Waiting to get a download link
* Will test with the pulp_installer
* Switching to pulp_ansible:master and pulpcore:master
* GalaxyNG is starting to need things from the master branches (i.e., RBAC)
* Recommended that our source checkouts use the master branches
### Actions
* [miked] will provide an example script for testing pulp_installer with RPMs
* [adrian] create a PR to point to pulp master branches and fix collection import issues. Tracking here: https://github.com/ansible/galaxy_ng/issues/330
## 21 July 2020
### Notes
* Several upcoming Pulp features/fixes that will not be released until August 11th
* Recommend that galaxy use pulpcore/pulp_ansible master temporarily
* RBAC story https://github.com/ansible/galaxy_ng/issues/302
### Actions
* [davidd] get https://pulp.plan.io/issues/7015 merged in the next week
* [davidn] open a pulp issue(s) about APIs needed for rbac to manage users/perms
### Agenda
* Artifact problems in import code - https://pulp.plan.io/issues/6718
* Added a PulpTemporaryFile model to store files separately from Artifacts
* https://github.com/pulp/pulp_ansible/pull/335
* https://github.com/pulp/pulpcore/pull/793
* Will require a similar change in galaxy_ng. Won't be able to release until after August 11th
* Pulpcon topics planning
* https://hackmd.io/hIOjFsFiSkGJR7VqtAJ8eQ
## 14 July 2020
### Agenda
* RBAC
* need clarity on what work needs to be done and when it can be completed
* adding subroute under each protected viewset named 'access_policy'
* Content Sync
* more information on how remotes work and connected with distributions and repositories.
* does pulp support Red Hat SSO Auth?
* how to manage requirements.yml (Automation Hub vs Community Galaxy remotes).
### Notes
* RBAC
* task breakdown and estimates still need to be done
* https://github.com/rsinger86/drf-access-policy/issues/29 (discussions about exposing access policy via API)
* access_policy (introspection, etc)
* LIST - /pulp/api/v3/access_policy/
* GET, PUT - /pulp/api/v3/access_policy/:uuid/
* access_policy for /pulp/api/v3/remotes/file/file/
* GET /pulp/api/v3/access_policy/?url=/pulp/api/v3/remotes/file/file/
* NOTE: not suitable for checking user access
* Content Sync
* Pulp agreed to deliver https://pulp.plan.io/issues/7015, which provides ability to associate a repository with a remote.
* Remote SSO authentication
- implemented, but not released yet.
- https://github.com/pulp/pulp_ansible/pull/326
- New `auth_url` attribute.
* requirements.yml options
- hardcoding restriction in `galaxy_ng`, by remote name?
- additional property, indicating if requirements.yml is required.
- standalone remote type in addition to CollectionRemote.
### Actions
* [house] test https://github.com/pulp/squeezer and possibly update our config role to use
* [bmbouterse] prioritize https://pulp.plan.io/issues/7015
* [davidn] Complete RBAC task breakdown and estimates
## 07 July 2020
### Agenda
* Ansible modules to configure Pulp are almost available
* https://github.com/pulp/squeezer
* Working on pulp-operator docs
* https://pulp-operator.readthedocs.io/en/latest/
* https://github.com/pulp/pulp-operator/pull/45
* RBAC
* Content Sync performance
### Notes
* RBAC
* On track to deliver by Fest
* Usecase meeting happening later today
* Content Sync
* For content change in the golden repo:
* Kick-off n number of tasks where n is the number of org repos
* Synclist configuration happens automatically on login, so n will be large day one.
* We're resource bound on c.rh.c - something we may need to deal with
* Solution:
* on login create a unique distribution for each subscriber account that defaults to the golden repo
* the first time user adds something to their denylist, create a repo, associate it with the distribution, and start a task to copy content from golden repo into new repo
* this reduces the size of n
* Content promotion:
* Possible time delay between new content arrival on c.rh.c and sync to org repo
* One possibility is to point the org distributions to a specific repository version, rather than latest. This could be a way to 'stage' content and promote it to every org at the same time.
* For now, start with the simple solution, but the above might be an alternative in the future.
* Curation task development:
* https://docs.pulpproject.org/plugins/plugin-writer/concepts/index.html#tasks
* Group task report - reports progress on a group of tasks
* Could be used to surface promotion status in UI
### Actions
* [house] test https://github.com/pulp/squeezer and possibly update our config role to use
## 30 June 2020
### Agenda
* Need to test token sync from cloud.redhat.com, how can we get an account that can do this?
* Splitting up the Remotes in pulp_ansible https://github.com/pulp/pulp_ansible/pull/326/files#r444237452
* Could also handle this by checking for auth_url https://github.com/pulp/pulp_ansible/pull/326/files#r447210249
* RPMs for galaxy_ng
* From build team: "Is there a way you could produce a source release that contains everything needed for building? Maybe by modifying the 'sdist' command? Or creating an own (Python) distribution for the UI code?"
* On c.rh.c, each subscriber org will have a synclist and in the UI users can add/remove collections on the list. AH will then curate content from the 'golden repo' into the org repo based on the synclist.
* How to initiate the sync process from golden -> org repo?
* How to update all org repos when new certified collection version lands?
* What performance issues need to be considered?
* Next steps for the installer testing?
* RBAC upate
### Notes
* Galaxy in one container:
* https://github.com/pulp/pulp-oci-images/tree/master/pulp_galaxy_ng
* https://pulpproject.org/pulp-in-one-container/
* Can we add this to galaxy_ng CI?
* Add to galaxy_ng docs for user consumption/testing
* For pulp_ansible token auth, it was decided to not split the Remote
* Content sync list
* Task code that produces an org repository version
* Inside task code run db queries that include all content from golden repo, subtracting denylist items. This can be done via Django ORM querysets.
* Performance suggestions:
* For synclist changes:
* Could we use a periodic task as a way to batch the changes?
* We could also possibly rev org repos when a local sync is initiated?
* For content change in the golden repo:
* kick-off n number of tasks where n is the number of org repos.
### Action Items
* [house] setup meeting to talk RPMs with ahills, shane, alex, evgeni, davdn
* [house] remind Calvin and Alex (when he returns) about AIOC
* [house] track down a production account for sync testing
* [alikins] Send doc link regarding openapi generation and re-start discussion on pulp-dev
ing list
* [alikins] Proposal around serving static metadata
* [bmbouterse] Invite alikins to Pulp RBAC review meetings
* [bmbouterse/alikins] Write up RBAC draft proposal for galaxy_ng
## 23 June 2020
### Agenda
* RBAC update (bmbouter):
* ldap integration and testing is done, users, groups, and membership importing from external sources
* django-admin is working as the administrator's method of read/edit users, groups, permissions also
* PoC to be finished for Repository Content Changes and Sync
* RPMs for galaxy_ng (chouse)
* Need to collaborate with Evgeni Golov <evgeni@redhat.com>
* Adding token support for Sync (fao)
* manually tested galaxy_ng change - it worked!
* currently working on functional test (CI)
* add another test for galaxy.ansible.com
* will be addressing comments - https://github.com/pulp/pulp_ansible/pull/326
* Installer updates
* install-from-rpm support is merged
* Additional update to docs for new role list: ``['pulp_all_services']``
* galaxy_ng playbook update - should it be on hold? https://github.com/ansible/galaxy_ng/pull/243/
* confirmation from @spredzy galaxy_ng installation runing side-by-side AWX intallation well
* Adding galaxy_ng to the Pulp single container so Ansible can also test galaxy_ng CLI integraiton directly
### Notes
* Testing developer install via Pulplift underway. [View docs here](https://github.com/ansible/galaxy_ng/wiki/Installing-from-source)
* Single container work being merged. Will handoff to galaxy_ng team to test.
* S6 used by all-in-one container: https://github.com/just-containers/s6-overlay
* Once the AIO container is available and tested, work with Sivel to add additionaling testing of `ansible-galaxy` against galaxy_ng.
### Action Items
* [house] invite evgeni@redhat.com to slack
* [house/bmbouter] reach out to evgeni via slack and see what support he needs
* [house/bmbouter] continue downstream images convo in slack
* [house/bmbouter] check-in with Shane before reaching out to RCM regarding downstream image build
* [house] stop testing with Centos7!!! Move to Centos8. If any issues file at https://pulp.plan.io/issues/new with category 'installer'
* [house] file an issue for Centos7 at https://pulp.plan.io/issues/new
* [alikins] Send doc link regarding openapi generation and re-start discussion on pulp-dev mailing list
* [alikins] Proposal around serving static metadata
* [bmbouters] Review and merge single container PR that adds galaxy_ng. Once merged, ping the galaxy_ng team with some links and instructions for testing.
## 16 June 2020
### Agenda
* RBAC update (bmbouter):
* PoC still pending. Delayed due to complexity integrating with external users and groups. Expected this week, due to breakthrough made on Thursday
* https://www.redhat.com/archives/pulp-dev/2020-June/msg00041.html
* pulp-operator update and next steps (mikedep333)
* Held meeting with installer team
* Agreed on a bunch of (tentative) details
* pulp-operator development starting as soon as install-from-rpm is
* epic: install from rpms
* installer PR is still under review (mikedep333)
* PR: https://github.com/pulp/pulp_installer/pull/326
* From @ehelms 16 June: "It's in an engineer's hands, and being worked on. There are a quite a lot of dependencies to process, so I am hoping a week or two. We're pretty slammed but it's a top priority."
* We should move forward with reviwing and merging the PR, regardless of when galaxy-ng RPMs land.
* Reference: Katello "pulpcore" repo used: [el7](https://fedorapeople.org/groups/katello/releases/yum/nightly/pulpcore/el7/x86_64/), [el8](https://fedorapeople.org/groups/katello/releases/yum/nightly/pulpcore/el8/x86_64/)
* Adding token support for Sync, PR available (href issue)
* testing against: https://ci.cloud.redhat.com/api/automation-hub/v3/collections/autohubtest2/collection_dep_a_hlrknkqw/
* getting response with /pulp_ansible/galaxy/automation-hub/api/v3/collections/<namespace>/<name>/versions
* instead of /api/automation-hub/v3/collections/<namespace>/<name>/versions
* https://github.com/pulp/pulp_ansible/pull/326#issuecomment-644313611
* Registry push
* pulp_container working towards their GA release
* Installer improvements from @spredzy
* https://github.com/pulp/pulp_installer/pull/331
* https://github.com/pulp/pulp_installer/pull/322
* https://github.com/pulp/pulp_installer/pull/324
* https://github.com/pulp/pulp_installer/pull/325
### Notes
- free operator ebook: https://www.redhat.com/cms/managed-files/cl-oreilly-kubernetes-operators-ebook-f21452-202001-en_2.pdf
### Actions
- [house] test the developer install and update galaxy_ng docs
- [alikins] Send doc link and start discussion on pulp-dev mailing list
- [alikins] Proposal around serving static metadata
- [house] Provide an update on design galaxy_ng multi-repository support
- [alikins] Work with Fabricio regarding token support for content sync and resolving invalid HREF issue (https://github.com/pulp/pulp_ansible/pull/326#issuecomment-644313611)
## 9 June 2020
### Agenda
- Update on Content Sync (@alikins, @davidn)
- pulp_container update
- docker push/pull testing (@dkliban/@ipanova)
* pulp_container 2.0.0 beta 2 released
- ansible-test integration (@bmclaugh)
(https://github.com/pulp/pulp_installer/pull/326)
- pulp_ansible starting to add client token support
- https://pulp.plan.io/issues/6540
- [Draft PR for Installing from RPMs]
- Content Promotion in GalaxyNG
### Notes
- RBAC
- POC landing this week
- Inviting pulp-dev to comment
- Hoping to include as tech-preview in Pulp Core 3.5
- status updates: https://www.redhat.com/archives/pulp-dev/2020-June/msg00006.html
- RPMs
- Need to ask @ehelms for status and synch up on a delivery date
- RPM installer potentially merging this week, so maybe beta RPMs could land next week or two?
- Let @mdepaulo know when RPMs land
- Need a place to host RPMs. Chat on slack with @bmbouter and @ehelms
- Developer Installs
- Helpful for installing and testing latest code
- Test using this playbook [house]: https://github.com/pulp/pulp_installer/pull/316#issuecomment-638933478
- OpenAPI generation
- Static metadata
- Content Sync
- RBAC
- Permissions exist at the repository level
- Permissions get assigned to users or groups
- David and Adrian to attend Pulp RBAC meeting
- OpenAPI observations ported to HackMD [https://hackmd.io/4UYlnTZ5QhKUBqvV4-wUNg (@alikins)](https://hackmd.io/4UYlnTZ5QhKUBqvV4-wUNg)
- Serving static metadata proof of concept (@alikins)
- Works with ansible-galaxy 2.9
- Example of generated static content [https://github.com/alikins/example_galaxycreate](https://github.com/alikins/example_galaxycreate)
- pulp_ansible token support
- Adds support for token authentication to the content sync machinary, enabling sync from source like c.rh.c where authentication is required.
- Needs to follow same logoic as ansible-galaxy client
- @fabricio will be working on this
- More or less need a
- Content promotion
- Need review of PR for 'move' function
### Actions
- [house] test the developer install and update galaxy_ng docs
- [alikins] Send doc link and start discussion on pulp-dev mailing list
- [alikins] Proposal around serving static metadata
- [davidn] Create meeting regarding galaxy_ng content sync questions
## 2 June 2020
### Agenda
- Proposal:
- make galaxy-ng mailing list self-subscribe
- public archives
- Send minutes from this meeting to it after each meeting (copy-paste-send)
- Switch to single-container or continue with the existing docker environment?
- No, galaxy_ng team will continue using their Docker dev environment
- Should the pulp installer support dev installs?
- RPMs?
- https://pulp.plan.io/issues/6793
- Pulp Container release beta going out this week( will contain “push” APIs)
- Discuss some issues of the current generated openapi https://docs.google.com/document/d/1R1fRM_rLWglJwhOX3FKaUcE_ACOcV3vzXmFDBshHqXg/edit?usp=sharing (alikins)
- How to handle API access for standalone if all tokens are for admin? (alikins)
- Socializing idea of adding static metadata content to “published” galaxy content (ala, yum, deb, maven, etc). And potentially using ContentGuards for access control in lieu of having to do API. (alikins)
### Notes
- RBAC update
- https://hackmd.io/kZ1oYp8TTkeuC5KL_ffjGQ
- POC in progress
- Seeking feedback once it’s ready (hopefully EOW)
- Should the Pulp installer support Dev installs?
- Yes, the Pulp installer should support source checkout installs
- Needs to support UI copy and build steps
- https://github.com/fao89/galaxy_ng_prerequisites/blob/master/tasks/main.yml
- Pulp Container
- Pulp_container beta dropping this week
- Need it in AH for the end-user install to consume
- Alex and Brian will update the pulp_container version in AH
- Pulp generated OpenAPI
- [alikins] will review with the Pulp Dev team
- There needs to be a move to v3 schema generation
- Tokens
- Need RBAC to be in a consumable place, so we can start working with it in GalaxyNG
- Serving static metadata content and potentially using ContentGuards for access control
- Makes sense from a conceptual perspective
- Makes it possible to distribute repositories via import/export. Could be a solution, maybe, for the disconnected use case.
- If we’re going to serve static metadata to the client, it conflicts with existing GalaxyNG API, and would require a lot of re-work.
- Needs a proposal
- Use case(s)
- Technical details and scope of work
- Cost / Benefit justification
### Action Items
- [bmbouter] make mailing list self-subscribe and turn on archives
- [house] send minutes to the list
- [bmbouter] update on RBAC
- [house] RPMs - get a conversation started on building and hosting an “upstream packaged version of the the plugin”
- [bmbouter] organize having the pulp installer support developer installs
- [alikins] will review OpenAPI generation with the Pulp Dev team
- [alikins] proposal for serving static metadata
---
# Template
## DATE
### Agenda
*
### Notes
*
### Action
*