# Pulp Services Deployment Meetings * Edge Mgmt Road Map View - https://issues.redhat.com/secure/PortfolioPlanView.jspa?id=1411&sid=1416&vid=8158#plan/backlog * Content Templates Road Map View - https://issues.redhat.com/secure/PortfolioReportView.jspa?r=fuapt#plan/backlog * Pulp Services Board - https://issues.redhat.com/secure/RapidBoard.jspa?rapidView=16966&projectKey=HMS * Pulp HMS Issues - https://issues.redhat.com/issues/?filter=12424552 # Priority Item Ideas Areas of work: * onboarding of compliance team to ruby gem usage * https://issues.redhat.com/browse/APPSRE-9884 <----- dkliban first priority * improving pulpcore re-rooting tests - https://github.com/pulp/pulpcore/issues/5010 <----- dkliban second priority (on hold waiting for changes in oci-images) * Add alerting to the "new Pulp service" - mikedep333 first priority * SLO docs * Finished the "service is down", "pod is crashing", 500 code, and latency alerts on staging and on [production](https://issues.redhat.com/browse/HMS-3966) * In the process of [rewriting the formulas/percentages for alerts](https://issues.redhat.com/browse/HMS-3969) * Currently rewriting the CI tests * Under the EPIC: https://issues.redhat.com/browse/HMS-3439 * Let's review the sub-tasks at the next meeting * https://issues.redhat.com/browse/HMS-3441 * https://issues.redhat.com/browse/HMS-3442 * [Ryan's tour](https://docs.google.com/document/d/1_E88ks02YusAC4KYIPHRpsipZ-WKGaNXEw28GGsgC8M/edit) * upstream workers OTEL implementation -decko - first priority * https://github.com/pulp/pulpcore/issues/3821 * https://issues.redhat.com/browse/PULP-19 * Pulp-cli contribution <-------- decko second priority * https://github.com/pulp/pulp-cli/pull/916 * https://issues.redhat.com/browse/PULP-31 * Write pulp SLOs <------ dkliban first priority * Under the EPIC: https://issues.redhat.com/browse/HMS-3439 * Need of the team to check on: * https://issues.redhat.com/browse/HMS-3503 * Are we a solo app on console.dot? * Yes * Our Architecture * Containers * ROUTING for sure * Authz * Impact when api, content or workers are down * Our backup and recovery procedures * Disaster Recovery procedures * ... * An alert will be triggered by a prometheus rule and gonna point to a document regarding the impacts, access, steps to investigate and an escalation policy. * Gonna need to point to dashboards. Maybe reprioritize the dashboard deployment. * https://gitlab.cee.redhat.com/service/app-interface/-/tree/master/docs/console.redhat.com/app-sops/content-sources?ref_type=heads * Schedule some time to discuss the docs topics * Adding SOP(Standard Operating Procedures) and deployment pipeline docs - dkliban second priority * Related to "Write pulp SLOs" task * Add services downstream docs # Done Priority Items * 3scale ConsoleDot Auth method for Pulp - 03/05/24 - decko * deploying pulp to production - 2/26/24 - dkliban * https://issues.redhat.com/browse/RHCLOUD-30862 * https://issues.redhat.com/browse/RHCLOUD-30845 * provide instructions to insights-compliance for setting up a pull through cache of rubygems - 2/26/24 - dkliban * create a user for insights-compliance - 2/26/24 - dkliban * improve migrations - 2/26/24 - mikedep333 - https://github.com/pulp/pulp-service/issues/44 * Add grafana dashboards to staging environment - decko - https://issues.redhat.com/browse/HMS-3229 * misc RBAC related bugs - 2/5/24 - dkliban * Adding initial SLo docs - 1/31/24 - decko - https://issues.redhat.com/browse/HMS-3440 * Create user and permissions for Content Sources application - 1/17/24 - dkliban - https://issues.redhat.com/browse/HMS-3368 * fix the service monitoring - 1/17/24 - decko - Jan 9 to Jan 16 * Add new top-level Pulp service - 1/10/24 - dkliban - https://issues.redhat.com/browse/HMS-3309 * Run pulp_rpm tests, - 1/3/24 - mikedep333 - https://github.com/pulp/pulp-clowder-deployments/issues/103 * Add Pulp Telemetry endpoints to the Prometheus instance - 12/15/23 - https://issues.redhat.com/browse/HMS-2921 - https://gitlab.cee.redhat.com/service/app-interface/-/merge_requests/88705 * S3 repair API issue, 11/29/23 - dkliban - https://github.com/pulp/pulpcore/issues/4776 * Deploy to production - https://issues.redhat.com/browse/HMS-1763 <---- dkliban first priority, Nov 13 * Address the outstanding CVE JIRA backlog - mikedep333 - 11/29/23 * Add OR content guard to Pulp - ggainey - 12/4/23 - https://github.com/pulp/pulpcore/pull/4605 * Operator safety checks improvements -- dkliban - 11/29/23 - https://github.com/pulp/pulp-operator/issues/1100 * https://issues.redhat.com/projects/APPSRE/issues/APPSRE-8438 * Image Builder EC2 worker issue - dkliban - 11/29/23 * addition of head-checking content guard - decko - 10/25/23 - https://github.com/pulp/pulpcore/issues/4518 * Ephemeral CONTENT_ORIGIN race condition being set - dkliban - 10/28/23 - https://github.com/pulp/pulp-clowder-deployments/issues/94 * Helping ansible BU understand how they can port their operator for breaking changes in pulp-oci-image entrypoints -- decko -- 10/18/23 -- https://issues.redhat.com/browse/AAP-17352 * OTEL Test Machinery - decko+lubos - 10/17/23 - https://github.com/pulp/pulpcore/pull/4414 * Operator Improvements - dkliban+hyagi - 10/16/23 - https://github.com/pulp/pulp-operator/issues/1101 * Image entrypoints alignment with compose and operator deployments - decko - 10/11/23 - https://github.com/pulp/pulp-oci-images/issues/544 * Upstream OTEL PR merging -- https://github.com/open-telemetry/opentelemetry-python-contrib/pull/1800 - decko - 10/30/23 * Enabling telemetry in staging environment <---- decko first priority, Oct 23th - Nov 13th * https://issues.redhat.com/browse/HMS-2920 * https://github.com/pulp/pulp-clowder-deployments/pull/104 * Work on the [pulp-service unit_test.sh](https://github.com/pulp/pulp-service/issues/4) -- mikedep333 - finished 1/18 # May 31 AI: decko checkout JIRA access ~~AI: decko workout the ubi pulp image~~ AI: decko polish what's possible on the pulp plugin smaller set https://issues.redhat.com/browse/HMS-1819 AI: mikedep333 to put details of work to be done in https://issues.redhat.com/browse/HMS-1818 AI: mikedep333 to start into https://issues.redhat.com/browse/HMS-1818 AI: decko and mikedep333 gonna trigger the Jenkins pipeline for a first try # May 24 AI: bmbouter additional testing of oci-env metrics environment AI: decko workout the ubi pulp image AI: decko checkout JIRA access AI: mikedep333 to put details of work to be done in https://issues.redhat.com/browse/HMS-1818 AI: mikedep333to start into https://issues.redhat.com/browse/HMS-1818 # May 22 AI: bmbouter additional testing of oci-env metrics environment AI: decko workout the ubi pulp image AI: decko checkout JIRA access * Staging/Prod deployment method ([HMS-1818](https://issues.redhat.com/browse/HMS-1818)) * We probably need a gitlab repo for pulpcore, pulp_rpm, pulp_oci_images, pulp_clowder_deployments * Mirrors of github, but manually pushed for now # May 17 ~~AI: decko respond to feedback on https://github.com/pulp/pulpcore/pull/3828~~ ~~AI: Decko revising pulp/oci-env PR: https://github.com/pulp/oci_env/pull/98~~ ~~Who to review? https://github.com/pulp/pulp-clowder-deployments/pull/2 * It has some hacks in it, but it works.~~ AI: mike to investigate https://issues.redhat.com/browse/AAP-11911 and determine what our options are * Actually it's on hold: https://hackmd.io/ipjKt7-LQM299Zo1KscL7Q?edit ~~AI: bmbouter to move more items from https://docs.google.com/document/d/11-9o3AcOp2XdOZ8AkyBv7J2uHJ6gZur0fP2jPQtyfMo/edit into JIRA~~ * https://issues.redhat.com/browse/HMS-1818 * https://issues.redhat.com/browse/HMS-1819 * https://issues.redhat.com/browse/HMS-1820 AI: bmbouter additional testing of oci-env metrics environment # May 15 ~~AI: decko Opening a PR to vendor the aiohttp-instrumentation into pulpcore~~ ~~AI: decko opened a pr for fixing pulp_ansible given the new pulpcore version. THis could help with pulp-oci-images~~ AI: Mike working on https://issues.redhat.com/browse/HMS-1764 AI: Mike to revise CI to get https://github.com/pulp/pulp-oci-images/pull/477 passing AI: Decko revising pulp/oci-env PR: https://github.com/pulp/oci_env/pull/98 # May 10 AI: Mike working on https://issues.redhat.com/browse/HMS-1764 AI: Mike to revise CI to get https://github.com/pulp/pulp-oci-images/pull/477 passing AI: Decko revising pulp/oci-env PR: https://github.com/pulp/oci_env/pull/98 AI: Decko to open PR to upstream aiohttp auto-instrumentation from this branch: https://github.com/decko/opentelemetry-python-contrib/tree/aiohttp-server-instrumentation # May 8 # May 3 AI: bmbouter to invite @mshriver to give overview of downstream pipeline * https://consoledot.pages.redhat.com/docs/dev/best-practices/cicd.html#_deployment_pipeline AI: Ask jsherrill about NetworkPolicy requirement for staging ## Needs Review * https://github.com/pulp/pulp-oci-images/pull/477 * https://github.com/pulp/pulpcore/pull/3632/files * https://github.com/pulp/oci_env/pull/98