---
# System prepended metadata

title: 'Umbrella issue: k8s.gcr.io => registry.k8s.io solution #1834'

---

# Umbrella issue: k8s.gcr.io => registry.k8s.io solution #1834

This markdown is synced from https://hackmd.io/gN-1GeSpSgyNSvmjKSULbg?edit to https://github.com/kubernetes/k8s.io/issues/1834#issue-841372237 manually by @BobyMCBobs

Scope: https://github.com/kubernetes/k8s.io/wiki/New-Registry-url-for-Kubernetes-(registry.k8s.io)

Design Doc: https://docs.google.com/document/d/1yNQ7DaDE5LbDJf9ku82YtlKZK0tcg5Wpk9L72-x2S2k/edit (shared w/ dev@kubernetes.io and SIG mailing list)

Board: https://github.com/orgs/kubernetes/projects/77

DRAFT AIs that need filled turned into tickets:
https://github.com/orgs/kubernetes/projects/77/views/2?filterQuery=is%3Adraft


# What exactly are you doing? (and how?)
- [x] We are setting up an AWS account with an IAM role and s3 buckets in AWS regions where we see a large percentage of source image pull traffic
  - [x] IAM Role + User for write - [k/k8s.io#3568](https://github.com/kubernetes/k8s.io/issues/3568) 
  - [x] List of Regions with 80% traffic - [oci-proxy#39](https://github.com/kubernetes-sigs/oci-proxy/issues/39)
    - [x] Determine which regions should serve the image layers - [oci-proxy/issues#38](https://github.com/kubernetes-sigs/oci-proxy/issues/38)
      - kubernetes/k8s.io#3568
      - kubernetes/k8s.io#3620
      - kubernetes-sigs/promo-tools#533
- [x] Create Buckets per Region Names
  - [x]  Add Terraform for registry-k8s-io s3 buckets [k/k8s.io#PR3605](https://github.com/kubernetes/k8s.io/pull/3605)
  - [x]  Create s3 Buckets in AWS Regions [k/k8s.io#3595](https://github.com/kubernetes/k8s.io/issues/3595)
- [x] We will iterate on a sandbox url (registry.sandbox.k8s.io) for our experiments and ONLY promote things to (registry.k8s.io) when we have complete confidence
  - [x] standup sandbox deployment [oci-proxy#13](https://github.com/kubernetes-sigs/oci-proxy/issues/13)
    - [x] Bulid sandbox infrastructure for oci-proxy [k8s.io#3317](https://github.com/kubernetes/k8s.io/issues/3317)
      - [x] Auto deploy for archeo in staging to registry-sandbox.k8s.io [k8s.io#3577](https://github.com/kubernetes/k8s.io/issues/3577)
  - [x] Full e2e testing [oci-proxy#24](https://github.com/kubernetes-sigs/oci-proxy/issues/24)
- [x] both registry and registry.sandbox are serving traffic using oci-proxy on google cloud run
    - Is this true now? 
- [x] oci-proxy will be updated to identify incoming traffic from AWS regions based on IP ranges so we can route traffic to s3 buckets in that region. If a specific AWS region do not currently host s3 buckets, we will redirect to the nearest region which does have s3 buckets (tradeoff between storage and network costs)
  - [ ] [oci-proxy should detect per region (80% of total traffic)
#39](https://github.com/kubernetes-sigs/oci-proxy/issues/39#issuecomment-1085230292)
- We will bulk sync existing image layers to these s3 layers as a starting point (from GCS/GCR)
  - [x] [kubernetes/k8s.io#3666](https://github.com/kubernetes/k8s.io/issues/3666)
- We will update image-promoter to push to these s3 buckets as well in addition to the current setup
  - [ ] Update image-promoter - [DRAFT](https://github.com/orgs/kubernetes/projects/77/views/2?filterQuery=Update+image-promoter+to+copy+to+multiple+regions)
- We will set up monitoring/reporting to check on new costs we incur on the AWS infrastructure and update what we do in GCP infrastructure as well to include the new components
  - [ ] AWS Cost Monitoring - [DRAFT](https://github.com/orgs/kubernetes/projects/77/views/2?filterQuery=Research+Reports+for+AWS+Cost+Explorer)
  - [ ] Collection and loading of AWS cost explorer data [kubernetes/k8s.io/infra/aws/aws-costexplorer-export](https://github.com/kubernetes/k8s.io/tree/main/infra/aws/aws-costexplorer-export)
- We will have a plan in place on how we could add additional AWS regions in the future
  - [x]  Document adding additional AWS Regions - [kubernetes/k8s.io/infra/aws/terraform/registry.k8s.io/README.md#extending-regions](https://github.com/kubernetes/k8s.io/tree/main/infra/aws/terraform/registry.k8s.io#extending-regions)
- We will have CI jobs that will run against registry.sandbox as well to monitor stability before we promote code to registry
  - [x] Full e2e testing using one parallel Conformance CI job - [k-sigs/oci-proxy#24](https://github.com/kubernetes-sigs/oci-proxy/issues/24)
    - [x] Add e2e tests - [oci-proxy/pr#32](https://github.com/kubernetes-sigs/oci-proxy/pull/32)
    - [x] Choose an e2e job to use registry-sandbox
    - [x] Configure chosen e2e sandbox with a testgrid dashboard
- We will automate the deployment/monitoring and testing of code landing in the oci-proxy repository
  - [x] [Auto deploy for archeo in staging to registry-sandbox.k8s.io · Issue #3577](https://github.com/kubernetes/k8s.io/issues/3577)
