# Dist-Git decoupling & ecosystem mapping

[miro board](https://miro.com/app/board/uXjVMkZYfGo=/?moveToWidget=3458764564271478580&cot=14&share_link_id=408451688028)

[dist-git repo](https://pagure.io/pagure-dist-git)

[read-the-docs](https://fedora-arc.readthedocs.io/en/latest/dist-git-move/index.html)

Work to be done:
1. Phase 1 - Gathering information
* Map services that are calling dist-git
* [mkonecny] Pagure
* Pagure is now base for dist-git
* The theme for src.fedoraproject.org could be found in pagure repository https://pagure.io/pagure/blob/master/f/pagure/themes/srcfpo
* The dist-git is using API (https://pagure.io/pagure/blob/master/f/pagure/api), messaging scheme (https://github.com/Pagure/pagure-messages), FAS integration and ACLs (https://pagure.io/pagure/blob/master/f/pagure/lib/git_auth.py) from Pagure
* Pagure is not interacting with dist-git
* Option to give project https://pagure.io/pagure/blob/master/f/pagure/ui/repo.py#_3171
* UI changes that are specific to dist-git
* Issues are just link to bugzilla tracker for the repository in bugzilla dist-git
* Monitoring status setting (backend implemented in dist-git https://pagure.io/pagure-dist-git/blob/master/f/pagure_distgit/plugin.py)
* Orphaning and taking orphaned projects (backend implemented in dist-git https://pagure.io/pagure-dist-git/blob/master/f/pagure_distgit/plugin.py)
* Links to koji, bodhi, bugzilla, packages and koschei
* [mkonecny] COPR
* Repository here https://github.com/fedora-copr/copr
* Deployment here https://copr.fedorainfracloud.org/
* Calling dist-git in https://github.com/fedora-copr/copr/blob/2789f06b36be5c13eae83c8695eb36de03a7728f/frontend/coprs_frontend/coprs/logic/builds_logic.py#L1101-L1109
* `/api/0/<package_name>/pull-request/<pr_id>/flag`
* `/api/0/<package_name>/c/<commit_id>/flag`
* Cloning dist-git repository using `git clone` wrapped in `copr-distgit-client clone --dist-git fedora <pkgname>`
* [mkonecny] fedpkg
* Using lookaside cache https://src.fedoraproject.org/repo/pkgs
* Following API calls
* `api/0/fork` for project forking
* `_dg/anitya/<namespace>/<package_name>` for disabling monitoring
* rpmbuild
* [mkonecny] the-new-hotness (now more than ever - PDC support dropped in favour of interacting with dist-git)
* Validating the monitoring settings in https://github.com/fedora-infra/the-new-hotness/blob/master/hotness/validators/pagure.py
* Validating if the package is retired, checking `dead.package` file in repository
* Cloning the dist git repository using `git clone` in koji builder module https://github.com/fedora-infra/the-new-hotness/blob/master/hotness/builders/koji.py#L119
* [mkonecny] bodhi (`git grep pagure_url` in bodhi source)
* Dist-git URL in ansible configuration https://pagure.io/fedora-infra/ansible/blob/main/f/roles/bodhi2/base/templates/production.ini.j2
* The default config value for dist-git `pagure_url` in https://github.com/fedora-infra/bodhi/blob/develop/bodhi-server/bodhi/server/config.py
* `pagure_url` is also used in https://github.com/fedora-infra/bodhi/blob/develop/bodhi-server/bodhi/server/models.py `Package` class methods. The API call are
* `api/0/<namespace>/<package_name>`
* `api/0/<namespace>/<package_name>/hascommit`
* [akdhar] F.M.N.
* retrieves information about perms for a package
* retrieves what packages a user or a group has access to
* Artifact details
* Ref. https://github.com/fedora-infra/fmn/blob/develop/fmn/api/handlers/misc.py#L55
* A GET endpoint exists in F.M.N. that can be used to retrieve artifact details from Dist Git
* The inputs required by F.M.N. for this operation are
* Name patterns of artifacts which should be returned
* Name of the users whose artifacts are to be returned
* Name of the groups whose artifacts are to be returned
* The endpoint uses a FastAPI dependency called `get_distgit_proxy`
* Ref. https://github.com/fedora-infra/fmn/blob/develop/fmn/backends/pagure.py#L284
* The URL of the existing Dist Git is referenced here to initialize object of type "PagureAsyncProxy" class
* Ref. https://github.com/fedora-infra/fmn/blob/develop/fmn/backends/pagure.py#L284
* The object has methods for the following operations
* Obtain API URL for dist-git
* Ref. https://github.com/fedora-infra/fmn/blob/develop/fmn/backends/pagure.py#L69
* Determine next page parameters
* Ref. https://github.com/fedora-infra/fmn/blob/develop/fmn/backends/pagure.py#L72
* Depending on the current page information
* Iterates through API responses
* Retrieve project details
* Ref. https://github.com/fedora-infra/fmn/blob/develop/fmn/backends/pagure.py#L86
* According to provided `namespace`, `pattern`, `username`, `owner` values
* Depending on whether they are `short`(?) or are a `fork`
* Retrieve projects associated with a user
* Ref. https://github.com/fedora-infra/fmn/blob/develop/fmn/backends/pagure.py#L115
* According to the provided `username` values
* Retrieve users associated with a project
* Ref. https://github.com/fedora-infra/fmn/blob/develop/fmn/backends/pagure.py#L127
* According to the provided `project_path` values
* Retrieve groups associated with a project
* Ref. https://github.com/fedora-infra/fmn/blob/develop/fmn/backends/pagure.py#L142
* According to the provided `project_path` values
* Retrieve projects associated with a group
* Ref. https://github.com/fedora-infra/fmn/blob/develop/fmn/backends/pagure.py#L157
* According to the provided `name` values
* Invalidate on message(?)
* Ok, so what parts need to change if we were to move to https://GitHub.com tomorrow?
* The distgit URL referencing would change
* Ref. https://github.com/fedora-infra/fmn/blob/develop/fmn/backends/pagure.py#L285
* The method `get_api_url` has to be re-defined depending on what the URL pattern is for the place we move dist-git assets to
* Ref. https://github.com/fedora-infra/fmn/blob/develop/fmn/backends/pagure.py#L69
* The `APIClient` class that is inherited by the `PagureAsyncProxy` class is URL agnostic so no need to rework it
* Ref. https://github.com/fedora-infra/fmn/blob/develop/fmn/backends/base.py#L39
* [jrichardson] Packit
* Makes pull requests to dist-git (eg. with version bumps)
* Ref. https://github.com/packit
* Ref. https://github.com/packit/packit/blob/main/packit/distgit.py
* Class that interacts with dist-git and pagure-over-dist-git API
* Clone method clones dist-git repo for selected package and returns this class
* Clone_pakage method clones package from dist-git from Fedora's src or CentOS Stream gitlab
* Some utility methods to return working directory where dist-git specfile is, absolute paths, Access levels of users and groups, push, pull, upstream names, etc.
* Ref. https://github.com/packit/packit/blob/d4f167c80ae38cdbb6cfa21fc2f8d556ef282c4b/packit/pkgtool.py
* packaging wrapper for fedpkg and centpkg, directory arg sets the dist-git repo
* Ref. https://github.com/packit/packit/blob/d4f167c80ae38cdbb6cfa21fc2f8d556ef282c4b/packit/pkgtool.py
* Sets up a source-git repo in an upstream repo taking DS patches, dist-git attribute sets the dist-git repo to be used for initialization
* Ref. https://github.com/packit/packit/blob/d4f167c80ae38cdbb6cfa21fc2f8d556ef282c4b/packit/cli/create_update.py#L40
* cli tool for creating updates
* class BugzillaIDs, various click options for dist-git branches, paths, etc.
* Ref. https://github.com/packit/packit/blob/d4f167c80ae38cdbb6cfa21fc2f8d556ef282c4b/packit/dist_git_instance.py#L28
* utility class returns dist-git instances
* Ref. https://github.com/packit/packit/blob/d4f167c80ae38cdbb6cfa21fc2f8d556ef282c4b/packit/base_git.py#L182
* nothing to commit PackitException
* Ref. https://github.com/packit/packit/blob/main/packit/api.py
* Represents the synchronization status of source-git and dist-git
* Ref. https://github.com/packit/packit/blob/main/packit/base_git.py
* Determines new release string to use in dist-git spec file, based on upstream spec file and given version.
* Ref. https://github.com/packit/packit/blob/main/packit/dist_git_instance.py
* Create an instance from the url and namespace, args for base url of the dist-git and namespace in the dist-git
* [mkonecny] mirror_pagure_ansible
* The mirror_pagure_ansible service on batcave01 receives bus messages about changes in the repository on Pagure, fetches these into /srv/git/mirrors/ansible.git and pushes from there to /srv/git/ansible.git. When this happens, various actions are triggered via git hooks
* Ref. https://docs.fedoraproject.org/en-US/infra/sysadmin_guide/ansible/
* [mkonecny] Toddlers
* Pushes and pulls (sources, spec files etc.)
* User information updates
* clean_retired_packages toddler
* `/_dg/orphan` endpoint called in `orphan_package` method
* `/_dg/bzoverrides` endpoint called in `orphan_package` method
* `/<namespace>/<name>/git/modifyacls` endpoint called in `remove_access` method
* `/<namespace>/<name>/watchers/update` endpoint called in `remove_access` method
* distgit_bugzilla_sync toddler
* `/extras/pagure_poc.json` called in `_get_pagure_projects`
* `/extras/pagure_bz.json` called in `_get_pagure_projects`
* flag_ci_pr toddler
* Listening to following topics
* `org.centos.*.ci.dist-git-pr.test.error`
* `org.centos.*.ci.dist-git-pr.test.complete`
* `org.centos.*.ci.dist-git-pr.test.running`
* `/api/0/<namespace>/<repo>/c/<commit_hash>/flag` called in process method
* flag_commit_build toddler
* `/api/0/<namespace>/<repo>/c/<commit_hash>/flag` called in process method
* packagers_without_bugzilla toddler
* `/extras/pagure_bz.json` called in `get_user_and_groups_dist_git`
* pdc_retired_packages toddler
* `/<namespace>/<repo>/raw/<branch>/f/dead.package` called in `_is_retired_in_dist_git`
* Calling methods in `utils/pagure.py`
* pdc_unretire_packages toddler
* `/<namespace>/<package_name>.git` called in `process_ticket`
* `git clone` called in `process_ticket`
* scm_request_processor toddler
* Calling methods in `utils/pagure.py`
* utils/pagure.py
* The methods could be used for both pagure.io and dist-git, some are specific only for dist-git
* `set_monitoring_status`
* `is_project_orphaned`
* `assign_maintainer_to_project`
* monitor-gating
* (@thrcka)
* [akdhar] tools used in fedora release engineering
* releng repo in pagure
* (@thrcka)
* Map services that are being called by dist-git
* [mkonecny] CI
* gating.yaml file needs to be in dist-git repository - it contains rules for greenwave
* The services in CI are creating corresponding flags to PR or commit
* `/api/0/<package_name>/pull-request/<pr_id>/flag`
* `/api/0/<package_name>/c/<commit_id>/flag`
* The CI on PR is being triggered by Fedora CI listening for `pull-request.new` topic
* Packit
* (?)
* F.M.N.
* Listens to the message bus for events pertaining to dist-git repos
* Toddlers
* Listens to the message bus for events pertaining to dist-git repos
* Might interact with Bugzilla depending on the messsages received from dist-git
* Events like change of email etc.
* list of services that listen in to the messages by dist-git from ansible repo (`git grep \\.git\\.receive`)
* toddlers
* COPR
* List of topics COPR is listening to https://pagure.io/fedora-infra/ansible/blob/f85569f78aeca0fc6c240a8a99f508d65f368c7b/f/roles/copr/frontend/templates/fedora-messaging/copr_messaging.toml#_46-61
* mirror_pagure_ansible
* FMN
* Datanommer
* Datanommer takes all messages from the Fedora Messaging bus and stores them in a postgres database
* It does not connect directly to dist-git but is indirectly affected by the messages that dist-git propagates on the bus
* The real question is "what kind of messages does dist-git propagate on the Fedora Messaging bus?"
* Or to ask in a different manner - "what kind of messages does pagure propagate on the Fedora Messaging bus?"
* Remember - "dist-git" is a plugin of "pagure" after all
* Also, there have been reports of their messages having similar schematics
*
* Does not interact with dist-git
*
* What functionality is needed from dist-git
* Create graph of the dist-git role in packager workflow
* Create graph of interactions with dist git
* Create discourse thread
* Announce the investigation in mailing list (devel, fedora-infra) - Done

2. Phase 2 - Design of new dist-git
* Map API calls that need to remain
* Removing the Pagure from dist-git
* Base git as backend
* What functionality needs to move to dist-git
when git is used as base backend

3. Phase 3 - Share with community for feedback