# registry.k8s.io implementation options Some options for Kubernetes infrastructure container registry migration, away from non-community run container registries. ## Blob managed level DNS A (probably) community-managed [Distribution [1]](https://github.com/distribution/distribution) instance, served at registry.k8s.io, handles low band operatations such as serving of configs, manifests, and hashes. A domain such as blobs.k8s.io is configured with split-horizon, to redirect the request the closest cloud-provider (based on the source IP). Each cloud-provider syncs the same blobs across. At each cloud-provider, an object store will be publically serving blobs, in the exact path per each object store. Each container image [tag] config will have the URLs property filled in with a blobs.k8s.io URL. ## registry level DNS The domain registry.k8s.io will be configured with split-horizon. Requests will resolve to the closest cloud-provider's managed OCI compliant container registry. Providers may require TLS certificates to be synced for redirects to work correctly. This would mean for LetsEncrypt dns01 certs to be generated. Each provider would have scoped read/write access to the DNS records to allow domain verification. Container images will be synced with [k8s-container-image-promoter](https://github.com/kubernetes-sigs/k8s-container-image-promoter). However, currently CIP only supports GCP and as a result it will need to be updated to support all cloud-providers related to this project. For cloud-providers without a container registry, a hosted [Distribution [1]](https://github.com/distribution/distribution) instance will be hosted there. This option is valid if: - cloud-managed registries per-vendor can all support custom domains (unsure currently) Notes: - given some updating, [txtdirect](https://github.com/txtdirect/txtdirect) is another piece of software that could be implemented. It provides TXT record based redirection (mentioned by stp) ## Envoy WASM filters Using an Envoy WASM filter, the body of the request will be dynamically rewritten to reflect the location of blobs in a public object store within the closest cloud-provider container registry, based on the source IP. --- Notes: - [1] [Distribution](https://github.com/distribution/distribution) or [Harbor](https://goharbor.io/) - this doc is available at [https://hackmd.io/@TKToYPauRJ-u_mNRBOh4HQ/HJBH3QF4_](https://hackmd.io/@TKToYPauRJ-u_mNRBOh4HQ/HJBH3QF4_) - kubernetes/k8s.io issue available at https://github.com/kubernetes/k8s.io/issues/1834