DO280 "Red Hat OpenShift Administration II: Operating a Production Kubernetes Cluster" notes in the margin

# DO280 "Red Hat OpenShift Administration II: Operating a Production Kubernetes Cluster" notes in the margin ###### tags: `red hat, openshift, kubernetes, containers, operators` > You will find here notes and links to official docs with additional information on products and technologies that described on RedHat training. > THIS DOCUMENT DOES NOT REPRINT ANY COPYRIGHTED CONTENT FROM REDHAT TRAINING. You will find here only public accessible outline. [Course description: DO280 Red Hat OpenShift Administration II: Operating a Production Kubernetes Cluster](https://www.redhat.com/en/services/training/do280-red-hat-openshift-administration-i) ## :memo: Table of contents [ToC] ### Chapter 1: Describing the Red Hat OpenShift Container Platform #### Describing OpenShift Container Platform Features [Red Hat Docs: OpenShift Container Platform 4.5 Documentation](https://docs.openshift.com/container-platform/4.5/welcome/index.html) [Product Documentation for OpenShift Container Platform 4.5](https://access.redhat.com/documentation/en-us/openshift_container_platform/4.5/) [Red Hat KB: Red Hat OpenShift Container Platform Life Cycle Policy](https://access.redhat.com/support/policy/updates/openshift) [Red Hat Developer: Getting Started with Red Hat OpenShift](https://developers.redhat.com/products/openshift/getting-started) #### Describing OpenShift Container Platform Architecture ![](https://www.openshift.com/hubfs/Imported_Blog_Media/Eventual_consistency.png) [The OpenShift Container Platform control plane](https://docs.openshift.com/container-platform/4.5/architecture/control-plane.html) ![](https://d33wubrfki0l68.cloudfront.net/7016517375d10c702489167e704dcb99e570df85/7bb53/images/docs/components-of-kubernetes.png) ![https://platform.deloitte.com.au/articles/2017/openshift-3-demystified-for-developers/](https://platform.deloitte.com.au/hubfs/Deloitte_November2018%20Theme/Images/2017-10-30-openshift-3-demystified-for-developers-big-picture.png) ![https://platform.deloitte.com.au/articles/2017/openshift-3-demystified-for-developers/](https://platform.deloitte.com.au/hubfs/Deloitte_November2018%20Theme/Images/2017-10-30-openshift-3-demystified-for-developers-misc.png) [OpenShift 3 Demystified. For Developers](https://platform.deloitte.com.au/articles/2017/openshift-3-demystified-for-developers/) [medium.com article: Just-in-Time Kubernetes: A Beginner’s Guide to Understanding Kubernetes Core Concepts](https://medium.com/dzerolabs/just-in-time-kubernetes-a-beginners-guide-to-kubernetes-core-concepts-19ee7acbafa1) [Компоненты Kubernetes](https://kubernetes.io/ru/docs/concepts/overview/components/) [Kubernetes Architecture Deep-dive Walkthrough](https://medium.com/priceline-labs/kubernetes-series-part-1-2babc30fc7b5) [RedHat Blogs: Kubernetes Deep Dive: API Server - part 1](https://www.openshift.com/blog/kubernetes-deep-dive-api-server-part-1) [RedHat Blogs: Kubernetes Deep Dive: API Server - part 2](https://www.openshift.com/blog/kubernetes-deep-dive-api-server-part-2) [RedHat Blogs: Kubernetes Deep Dive: API Server – part 3a](https://www.openshift.com/blog/kubernetes-deep-dive-api-server-part-3a) [Learn Kubernetes - from the beginning, Part I, Basics, Deployment and Minikube](https://softchris.github.io/pages/kubernetes-one.html) [Learn Kubernetes - from the beginning, part II, Pods, Nodes and Services](https://softchris.github.io/pages/kubernetes-two.html) #### Describing Cluster Operators [What is an Operator?](https://www.openshift.com/learn/topics/operators) An Operator is a method of packaging, deploying and managing a Kubernetes-native application. A Kubernetes-native application is an application that is both deployed on Kubernetes and managed using the Kubernetes APIs and kubectl tooling. An Operator is essentially a custom controller. A controller is a core concept in Kubernetes and is implemented as a software loop that runs continuously on the Kubernetes master nodes comparing, and if necessary, reconciling the expressed desired state and the current state of an object. Objects are well known resources like Pods, Services, ConfigMaps, or PersistentVolumes. Operators apply this model at the level of entire applications and are, in effect, application-specific controllers. The Operator is a piece of software running in a Pod on the cluster, interacting with the Kubernetes API server. It introduces new object types through Custom Resource Definitions, an extension mechanism in Kubernetes. These custom objects are the primary interface for a user; consistent with the resource-based interaction model on the Kubernetes cluster. An Operator watches for these custom resource types and is notified about their presence or modification. When the Operator receives this notification it will start running a loop to ensure that all the required connections for the application service represented by these objects are actually available and configured in the way the user expressed in the object’s specification. [WHAT IS AN OPERATOR AFTER ALL?](https://operatorframework.io/what/) An Operator represents human operational knowledge in software, to reliably manage an application. They are methods of packaging, deploying, and managing a Kubernetes application. The goal of an Operator is to put operational knowledge into software. Previously this knowledge only resided in the minds of administrators, various combinations of shell scripts or automation software like Ansible. It was outside of your Kubernetes cluster and hard to integrate. With Operators, CoreOS changed that. Operators implement and automate common Day-1 (installation, configuration, etc) and Day-2 (re-configuration, update, backup, failover, restore, etc.) activities in a piece of software running inside your Kubernetes cluster, by integrating natively with Kubernetes concepts and APIs. We call this a Kubernetes-native application. With Operators you can stop treating an application as a collection of primitives like Pods, Deployments, Services or ConfigMaps, but instead as a single object that only exposes the knobs that make sense for the application. [OperatorHub.io is a new home for the Kubernetes community to share Operators](https://operatorhub.io) [Youtube: Kubernetes Operator simply explained in 10 mins](https://www.youtube.com/watch?v=ha3LjlD6g7g) [Youtube: Custom Operators for OpenShift Administrators](https://www.youtube.com/watch?v=LB0wwmqQxhU) Openshift Cluster Operators provides - Oauth Server and RBAC. Support for Identity providers - Internal CoreDNS - Web Console - Internal Image Registry - Monitoring and Logging stack - [Medium post: What is the Cluster Etcd Operator ?](https://daein.medium.com/introduction-what-is-the-cluster-etcd-operator-7b9028fc4429) [Latest OpenShift releases](https://console.redhat.com/openshift/releases) [Client tools and images for latest OpenShift release](https://mirror.openshift.com/pub/openshift-v4/x86_64/clients/ocp/latest/release.txt) ### Chapter 2: Verifying the Health of a Cluster #### Describing Installation Methods Ways to learn Openshift without need to install it. https://learn.openshift.com/ https://learn.openshift.com/playgrounds https://developers.redhat.com/developer-sandbox [Try your own Red Hat OpenShift 4 cluster ](https://www.openshift.com/try#step-2) | On your computer | In your datacenter | Self-managed | Managed service | |:-------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |:-------------------------------------------------------------------------------------------------------------------------------------------------------- | | * Your laptop or desktop; | * Your IT environment (VMware or bare metal) | * Your account with a supported provider** | * Installed and maintained for you | | * Minimal, pre-configured | * Self-managed | * Self-managed on Red Hat OpenShift Container Platform | * Red Hat-managed | | * Ideal for development and testing | | | | | * Developer-focused resources | | | | | * Self-managed | | | | | [Try it Locally](https://cloud.redhat.com/openshift/install/crc/installer-provisioned) | [Try it in your IT environment](https://cloud.redhat.com/openshift/install#datacenter) | [Try it in your cloud](https://cloud.redhat.com/openshift/install#public-cloud) | [Try Openshifdt Dedicated](https://www.openshift.com/products/dedicated/?hsLang=en-us) [Try OpenShift on IBM Cloud](https://www.ibm.com/cloud/openshift) | [Red Hat CodeReady Containers is a way to install OpenShift on your laptop in a VM](https://developers.redhat.com/products/codeready-containers/overview) [Red Hat Discussion: Pull Secrets and Customer Account User](https://access.redhat.com/discussions/4600331) [Red Hat OpenShift Cluster Manager site](https://cloud.redhat.com/openshift/create) [OpenShift Container Platform installation overview](https://access.redhat.com/documentation/en-us/openshift_container_platform/4.9/html-single/architecture/index#installation-overview_architecture-installation) ![OpenShift Container Platform installation targets and dependencies](https://access.redhat.com/webassets/avalon/d/OpenShift_Container_Platform-4.9-Architecture-en-US/images/6527feee3673861daba1c8118bcfa1b6/targets-and-dependencies.png) [OpenShift Container Platform installation overview](https://docs.openshift.com/container-platform/4.5/architecture/architecture-installation.html) Bootstrapping a cluster involves the following steps: 1. The bootstrap machine boots and starts hosting the remote resources required for the master machines to boot. (Requires manual intervention if you provision the infrastructure) 2. The master machines fetch the remote resources from the bootstrap machine and finish booting. (Requires manual intervention if you provision the infrastructure) 3. The master machines use the bootstrap machine to form an etcd cluster. 4. The bootstrap machine starts a temporary Kubernetes control plane using the new etcd cluster. 5. The temporary control plane schedules the production control plane to the master machines. 6. The temporary control plane shuts down and passes control to the production control plane. 7. The bootstrap machine injects OpenShift Container Platform components into the production control plane. 8. The installation program shuts down the bootstrap machine. (Requires manual intervention if you provision the infrastructure) 9. The control plane sets up the worker nodes. 10. The control plane installs additional services in the form of a set of Operators. ![Creating the bootstrap, master, and worker machines](https://access.redhat.com/webassets/avalon/d/OpenShift_Container_Platform-4.6-Architecture-en-US/images/f64dc89791c50ae0a5138d1dfe9f8444/create-nodes.png) [Youtube Video: OpenShift 4.x Ignition](https://www.youtube.com/watch?v=GcRf0BomQwg) ![](https://www.openshift.com/hubfs/Blog%20-%20OCP%20e2e%20%231%20Plan%20and%20Deploy.png) [RedHat Blogs: OpenShift End-to-End: Plan and Deploy](https://www.openshift.com/blog/openshift-end-to-end-plan-and-deploy) ![](https://www.openshift.com/hubfs/Blog%20-%20OCP%20e2e%20%232%20Core%20Services.png) [RedHat Blogs: OpenShift End-to-End: Core Services](https://www.openshift.com/blog/openshift-end-to-end-core-services) ![](https://www.openshift.com/hubfs/Blog%20-%20OCP%20e2e%20%234%20Users%20and%20onward.png) [RedHat Blogs: OpenShift End-to-End: User Config and Application Deployment](https://www.openshift.com/blog/openshift-end-to-end-user-config-and-application-deployment) [Using Fio to Tell Whether Your Storage is Fast Enough for Etcd](https://www.ibm.com/cloud/blog/using-fio-to-tell-whether-your-storage-is-fast-enough-for-etcd) [How OCP distribution looks like?](https://mirror.openshift.com/pub/openshift-v4/x86_64/clients/ocp/latest/release.txt) [The Community Distribution of Kubernetes that powers Red Hat OpenShift](https://www.okd.io) How to easily install OCP in my lab? In a VM or Bare-metal? In a cloud or on-premise? [ITNext Blogs: Craig Robinson Guide: Installing an OKD 4.5 Cluster UPI](https://itnext.io/guide-installing-an-okd-4-5-cluster-508a2631cbee) [Medium: OpenShift 4 in an Air Gap (disconnected) environment: Part 1 — prerequisites](https://two-oes.medium.com/openshift-4-in-an-air-gap-disconnected-environment-part-1-prerequisites-63f065e8a729) [Medium: OpenShift 4 in an Air Gap (disconnected) environment: Part 2 — installation](https://two-oes.medium.com/openshift-4-in-an-air-gap-disconnected-environment-part-2-installation-1dd8bf085fdd) [Medium: OpenShift 4 in an Air Gap (disconnected) environment: Part 3 — customization](https://two-oes.medium.com/openshift-4-in-an-air-gap-disconnected-environment-part-3-customization-4b315dd17316) [OpenShift Cluster Managed Bare Metal Documentation](https://openshift-kni.github.io/baremetal-deploy/) [ITNext Blogs: Craig Robinson Guide: OKD 4.5 Single Node Cluster on Windows 10 using Hyper-V](https://itnext.io/okd-4-5-single-node-cluster-on-windows-10-using-hyper-v-3ffb7b369245) [The Easiest And Fastest Way To Deploy An OKD 4.5 Cluster In A Libvirt/KVM Host](https://www.anstack.com/blog/2020/07/31/the-fastest-and-simplest-way-to-deploy-okd-openshift-4-5.html) [Medium: Openshift Single Node for Distributed Clouds](https://medium.com/codex/openshift-single-node-for-distributed-clouds-582f84022bd0) [Github RedHatOfficial: OCP4 Helper Node](https://github.com/RedHatOfficial/ocp4-helpernode) [Github Kubernetes SIG: Kubernetes NFS Subdir External Provisioner](https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner) [Ansible Agnostic Deployer](https://github.com/redhat-cop/agnosticd) [Installing OKD4.X with User Provisioned Infrastructure. Libvirt, iPXE, and FCOS](https://cgruver.github.io/okd4-upi-lab-setup) [Building an OKD4 single node cluster with minimal resources](https://cgruver.github.io/okd4-single-node-cluster) [RedHat Blogs: OpenShift 4 Partner Reference Architectures](https://www.openshift.com/blog/openshift-4-partner-reference-architectures) [OCP on VMware validated design](https://docs.vmware.com/en/VMware-Validated-Design/index.html) [VMware Validated Design 6.x: Architecture and Design for a Red Hat OpenShift Workload Domain](https://docs.vmware.com/en/VMware-Validated-Design/6.0.1/sddc-architecture-and-design-for-a-red-hat-openshift-workload-domain/GUID-20F4C4C1-E4BE-4B53-9C45-4E3EB990FC57.html) [VMware Validated Design 6.x: Deployment of a Red hat OpenShift Workload Domain](https://docs.vmware.com/en/VMware-Validated-Design/6.0.1/sddc-deployment-of-a-red-hat-openshift-workload-domain-in-the-first-region/GUID-01F285ED-C06E-4AF0-A684-BE70C6290E33.html) [OpenShift sizing and subscription guide for enterprise Kubernetes](https://www.redhat.com/en/resources/OpenShift-subscription-sizing-guide-detail) [RedHat KB: How to isolate infrastructure workload from other workload and pay less for RedHat subscription?](https://access.redhat.com/solutions/5034771) [RedHat Blogs: OpenShift tip: Resolve a terminating state issue](https://www.redhat.com/sysadmin/openshift-terminating-state) [Support Policies for Red Hat OpenShift Container Platform Clusters - Deployments Spanning Multiple Sites(Data Centers/Regions)](https://access.redhat.com/articles/3220991) [RedHat Blog: Deploying OpenShift Applications to Multiple Datacenters](https://cloud.redhat.com/blog/deploying-openshift-applications-multiple-datacenters) [RedHat Blog: Global Load Balancer for OpenShift clusters: an Operator-Based Approach](https://cloud.redhat.com/blog/global-load-balancer-for-openshift-clusters-an-operator-based-approach) [RedHat KB: Red Hat OpenShift Container Platform High Availability, and Recommended Practices](https://access.redhat.com/articles/3221001) #### Executing Troubleshooting Commands ![](https://learnk8s.io/a/7d3cde9ddb57e4246215de4be2f3dd71.png) [learnk8s blog post: A visual guide on troubleshooting Kubernetes deployments](https://learnk8s.io/troubleshooting-deployments) [download full size in pdf](https://learnk8s.io/a/a-visual-guide-on-troubleshooting-kubernetes-deployments/troubleshooting-kubernetes.v2.pdf) [Exploring the OpenShift etcd with etcdctl](https://www.krenger.ch/blog/exploring-the-openshift-etcd-with-etcdctl) https://blog.freshtracks.io/a-deep-dive-into-kubernetes-metrics-part-5-etcd-metrics-6502693fa58 https://itnext.io/etcd-performance-consideration-43d98a1525a3 https://access.redhat.com/solutions/5060021 [RedHat KB: Red Hat OpenShift Container Platform High Availability, and Recommended Practices](https://access.redhat.com/articles/3221001) How-to fully replace failed master server. [Openshift docs: Replacing an unhealthy etcd member](https://docs.openshift.com/container-platform/4.8/backup_and_restore/replacing-unhealthy-etcd-member.html) [RedHat KB: Clusteroperator is degraded with NodeInstallerDegraded in status (Redeploy static pods) ](https://access.redhat.com/solutions/4849711) [OpenShift tip: Resolve a Project terminating state issue](https://www.redhat.com/sysadmin/openshift-terminating-state) [RedHat KB: Unable to Delete a Project or Namespace in OCP](https://access.redhat.com/solutions/4165791) [](https://access.redhat.com/solutions/2317401) [Howto regenerate system:admin kubeconfig file](https://rcarrata.com/openshift/regenerate-kubeconfig/) #### Introducing OpenShift Dynamic Storage Pods use ephemeral local storage for scratch space, caching, and logs. Issues related to the lack of local storage accounting and isolation include the following: - Pods do not know how much local storage is available to them. - Pods cannot request guaranteed local storage. - Local storage is a best effort resource. - Pods can be evicted due to other pods filling the local storage, after which new pods are not admitted until sufficient storage has been reclaimed. [Kubernetes Docs](https://kubernetes.io/docs/) / [Concepts](https://kubernetes.io/docs/concepts/) / [Storage](https://kubernetes.io/docs/concepts/storage/) / [Persistent Volumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) A _PersistentVolume_ (PV) is a piece of storage in the cluster that has been provisioned by an administrator or dynamically provisioned using [Storage Classes](https://kubernetes.io/docs/concepts/storage/storage-classes/). It is a resource in the cluster just like a node is a cluster resource. PVs are volume plugins like Volumes, but have a lifecycle independent of any individual Pod that uses the PV. This API object captures the details of the implementation of the storage, be that NFS, iSCSI, or a cloud-provider-specific storage system. A _PersistentVolumeClaim_ (PVC) is a request for storage by a user. It is similar to a Pod. Pods consume node resources and PVCs consume PV resources. Pods can request specific levels of resources (CPU and Memory). Claims can request specific size and access modes (e.g., they can be mounted ReadWriteOnce, ReadOnlyMany or ReadWriteMany, see [AccessModes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes)). [Kubernetes Docs](https://kubernetes.io/docs/) / [Concepts](https://kubernetes.io/docs/concepts/) / [Storage](https://kubernetes.io/docs/concepts/storage/) / [Storage Classes](https://kubernetes.io/docs/concepts/storage/storage-classes/) While PersistentVolumeClaims allow a user to consume abstract storage resources, it is common that users need PersistentVolumes with varying properties, such as performance, for different problems. Cluster administrators need to be able to offer a variety of PersistentVolumes that differ in more ways than just size and access modes, without exposing users to the details of how those volumes are implemented. For these needs, there is the _StorageClass_ resource. [RedHat Blogs: Deploying OpenShift Container Storage using Local Devices](https://www.openshift.com/blog/deploying-openshift-container-storage-using-local-devices) [A Guide to Customizing Red Hat Enterprise Linux CoreOS](https://cloud.redhat.com/blog/red-hat-enterprise-linux-coreos-customization) ### Chapter 3. Configuring Authentication and Authorization #### Configuring Identity Providers [OCP 4.5 Docs](https://docs.openshift.com/container-platform/4.5/welcome/index.html) / [Authentication and authorization](https://docs.openshift.com/container-platform/4.5/authentication/understanding-authentication.html) / [Understanding authentication](https://github.com/openshift/openshift-docs/commits/enterprise-4.5/authentication/understanding-authentication.adoc) [OCP 4.5 Docs](https://docs.openshift.com/container-platform/4.5/welcome/index.html) / [Authentication and authorization](https://docs.openshift.com/container-platform/4.5/authentication/understanding-authentication.html) / [Understanding authentication](https://github.com/openshift/openshift-docs/commits/enterprise-4.5/authentication/understanding-authentication.adoc) / [Understanding identity provider configuration](https://docs.openshift.com/container-platform/4.5/authentication/understanding-identity-provider.html) [Keycloak (as an Identity Provider) to secure Openshift](https://medium.com/keycloak/using-keycloak-identity-provider-to-secure-openshift-f929a7a0f7f1) ![](https://miro.medium.com/max/1225/1*x6tuuEXajC92-6zuvt4A6Q.png ) Adding Azure AD as trusted OAuth Identity Provider [Blog post: OpenShift 4 Authentication via Azure AD](https://www.arctiq.ca/our-blog/2020/1/30/ocp4-auth-with-azure-ad/) [Rotating the OpenShift kubeadmin Password](https://blog.andyserver.com/2021/07/rotating-the-openshift-kubeadmin-password/) #### Defining and Applying Permissions Using RBAC [OCP 4.5 Docs](https://docs.openshift.com/container-platform/4.5/welcome/index.html) - [Authentication and authorization](https://docs.openshift.com/container-platform/4.5/authentication/understanding-authentication.html) - [Understanding authentication](https://github.com/openshift/openshift-docs/commits/enterprise-4.5/authentication/understanding-authentication.adoc) - [Understanding identity provider configuration](https://docs.openshift.com/container-platform/4.5/authentication/understanding-identity-provider.html) - [Using RBAC to define and apply permissions](https://docs.openshift.com/container-platform/4.5/authentication/using-rbac.html) Authorization is managed using: | Authorization object |Description | | - | - | |Rules |Sets of permitted verbs on a set of objects. For example, whether a user or service account can `create` pods. |Roles |Collections of rules. You can associate, or bind, users and groups to multiple roles. |Bindings |Associations between users and/or groups with a role. |Default cluster role | Description | | ----- | - | |`admin` |A project manager. If used in a local binding, an `admin` has rights to view any resource in the project and modify any resource in the project except for quota. | |`basic-user` |A user that can get basic information about projects and users. | |`cluster-admin` |A super-user that can perform any action in any project. When bound to a user with a local binding, they have full control over quota and every action on every resource in the project. |`cluster-status` |A user that can get basic cluster status information. |`edit` |A user that can modify most objects in a project but does not have the power to view or modify roles or bindings. |`self-provisioner` |A user that can create their own projects. |`view` |A user who cannot make any modifications, but can see most objects in aproject. They cannot view or modify roles or bindings. [Kubernetes Docs](https://kubernetes.io/docs/) - [Reference](https://kubernetes.io/docs/reference/) - [API Access Control](https://kubernetes.io/docs/reference/access-authn-authz/) - [Using RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) - [User-facing roles](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) <table> <colgroup><col style="width: 25%;" /><col style="width: 25%;" /><col /></colgroup> <thead> <tr> <th>Default ClusterRole</th> <th>Default ClusterRoleBinding</th> <th>Description</th> </tr> </thead> <tbody> <tr> <td><b>cluster-admin</b></td> <td><b>system:masters</b> group</td> <td>Allows super-user access to perform any action on any resource. When used in a <b>ClusterRoleBinding</b>, it gives full control over every resource in the cluster and in all namespaces. When used in a <b>RoleBinding</b>, it gives full control over every resource in the role binding's namespace, including the namespace itself.</td> </tr> <tr> <td><b>admin</b></td> <td>None</td> <td>Allows admin access, intended to be granted within a namespace using a <b>RoleBinding</b>. If used in a <b>RoleBinding</b>, allows read/write access to most resources in a namespace, including the ability to create roles and role bindings within the namespace. This role does not allow write access to resource quota or to the namespace itself.</td> </tr> <tr> <td><b>edit</b></td> <td>None</td> <td>Allows read/write access to most objects in a namespace. This role does not allow viewing or modifying roles or role bindings. However, this role allows accessing Secrets and running Pods as any ServiceAccount in the namespace, so it can be used to gain the API access levels of any ServiceAccount in the namespace.</td> </tr> <tr> <td><b>view</b></td> <td>None</td> <td>Allows read-only access to see most objects in a namespace. It does not allow viewing roles or role bindings. This role does not allow viewing Secrets, since reading the contents of Secrets enables access to ServiceAccount credentials in the namespace, which would allow API access as any ServiceAccount in the namespace (a form of privilege escalation).</td> </tr> </tbody> </table> Applications - [Projects] - [Working with projects](https://docs.openshift.com/container-platform/4.5/applications/projects/working-with-projects.html) - [Providing access permissions to your project using the Developer perspective (Web Console)](https://docs.openshift.com/container-platform/4.5/applications/projects/working-with-projects.html#odc-providing-project-permissions-using-developer-perspective_projects) ![](https://access.redhat.com/webassets/avalon/d/OpenShift_Container_Platform-4.6-Applications-en-US/images/40c7addbf7c13d2c4661b3875c2f2912/odc_project_permissions.png) - [Configuring project creation](https://docs.openshift.com/container-platform/4.5/applications/projects/configuring-project-creation.html) [Disabling project self-provisioning](https://docs.openshift.com/container-platform/4.5/applications/projects/configuring-project-creation.html#disabling-project-self-provisioning_configuring-project-creation) [**Bug 1629524**](https://bugzilla.redhat.com/show_bug.cgi?id=1629524) - Should remove warning message for remove-cluster-role-from-user compared with add-cluster-role-to-user ``` [student@workstation ~]$ oc adm policy remove-cluster-role-from-group self-provisioner system:authenticated:oauth Warning: Your changes may get lost whenever a master is restarted, unless you prevent reconciliation of this rolebinding using the following command: oc annotate clusterrolebinding.rbac self-provisioners 'rbac.authorization.kubernetes.io/autoupdate=false' --overwrite clusterrole.rbac.authorization.k8s.io/self-provisioner removed: "system:authenticated:oauth" ``` [Red Hat E-book: Red Hat OpenShift security guide](https://www.redhat.com/en/resources/openshift-security-guide-ebook) [Krew (kubectl plugin manager)](https://krew.sigs.k8s.io/) [Plugin index for Krew](https://krew.sigs.k8s.io/plugins/) [rakkess (Review Access) - kubectl plugin to show an access matrix for server resources](https://github.com/corneliusweig/rakkess) ![Review Access - kubectl plugin to show an access matrix for server resources](https://github.com/corneliusweig/rakkess/raw/master/doc/demo-user-smaller.png) [RedHat KB: "oc logout is not working when logged in with system:admin" ](https://access.redhat.com/solutions/4876851) ```` Resolution system:admin is created automatically when the infrastructure is defined, mainly for the purpose of enabling the infrastructure to interact with the API securely. By default system:admin uses certificate not the token to login to the cluster. Which mean system:admin doesn't have a password or token like other normal users have. When you try to logout from system:admin it will not get logged out. Root Cause system:admin credentials live in a client certificate data that can be seen in ~/.kube/config. If you get prompted for a password, that means your $KUBECONFIG file does not contain those credentials. To know more about configuring authentication please refer document ```` ### Chapter 4. Configuring Application Security #### Managing Sensitive Information with Secrets [OCP 4.5 Docs](https://docs.openshift.com/) / [4.5](https://docs.openshift.com/container-platform/4.5/welcome/index.html) / Nodes / [Working with pods](https://docs.openshift.com/container-platform/4.5/nodes/pods/nodes-pods-using.html) / [Providing sensitive data to Pods](https://docs.openshift.com/container-platform/4.6/nodes/pods/nodes-pods-secrets.html) [OCP 4.5 Docs](https://docs.openshift.com/) / [4.5](https://docs.openshift.com/container-platform/4.5/welcome/index.html) / [Images](https://docs.openshift.com/container-platform/4.5/openshift_images/configuring-samples-operator.html) / [Managing images](https://docs.openshift.com/container-platform/4.5/openshift_images/managing_images/managing-images-overview.html) / [Using image pull secrets](https://docs.openshift.com/container-platform/4.5/openshift_images/managing_images/using-image-pull-secrets.html) You use this pull secret to authenticate with the services that are provided by the included authorities, including [Quay.io](https://docs.openshift.com/container-platform/4.5/openshift_images/managing_images/quay.io) and [registry.redhat.io](https://docs.openshift.com/container-platform/4.5/openshift_images/managing_images/registry.redhat.io), which serve the container images for OpenShift Container Platform components. ``` $ oc create secret generic <pull_secret_name> \ --from-file=.dockerconfigjson=<path/to/.docker/config.json> \ --type=kubernetes.io/dockerconfigjson $ oc secrets link default <pull_secret_name> --for=pull ``` #### Controlling Application Permissions with Security Context Constraints [RedHat Blogs: Understanding Service Accounts and SCCs](https://www.openshift.com/blog/understanding-service-accounts-sccs) [RedHat Blogs: Managing SCCs in OpenShift](https://www.openshift.com/blog/managing-sccs-in-openshift) ![](https://www.openshift.com/hubfs/Managing%20SCCs%20in%20OpenShift-1.png) [opensource.com blog post (Daniel J Walsh (RedHat)) Just say no to root (in containers) ](https://opensource.com/article/18/3/just-say-no-root-containers) [Medium.com blog post: Processes In Containers Should Not Run As Root](https://medium.com/@mccode/processes-in-containers-should-not-run-as-root-2feae3f0df3b) [How SCC can enforce rules to your application user id and group id](https://docs.openshift.com/container-platform/4.5/storage/persistent_storage/persistent-storage-nfs.html) [RedHat Blogs: A Guide to OpenShift and UIDs](https://www.openshift.com/blog/a-guide-to-openshift-and-uids) [Security Zones in OpenShift worker nodes](https://itnext.io/security-zones-in-openshift-worker-nodes-part-i-introduction-4f85762962d7) [Support node-level user namespace remapping ](https://github.com/kubernetes/enhancements/issues/127) [Improving Kubernetes and container security with user namespaces](https://kinvolk.io/blog/2020/12/improving-kubernetes-and-container-security-with-user-namespaces/#bringing-user-namespaces-to-kubernetes) [YouTube OpenShift: [What's New] OpenShift 4.9 [Oct-2021]](https://youtu.be/Q1j_mt_XLqE?t=469) [Best practices for building images that pass Red Hat Container Certification](https://developers.redhat.com/articles/2021/11/11/best-practices-building-images-pass-red-hat-container-certification) [Red Hat Blog: Enhancing your Builds on OpenShift: Chaining Builds](https://cloud.redhat.com/blog/chaining-builds) ### Chapter 5 Configuring OpenShift Networking for Applications #### Troubleshooting OpenShift Software defined Networking [OCP 4.5 Docs](https://docs.openshift.com/container-platform/4.5/welcome/index.html) / [OpenShift Container Platform ](https://docs.openshift.com/container-platform/4.5/welcome/index.html)/ 4.5 / [Networking](https://docs.openshift.com/container-platform/4.5/networking/understanding-networking.html) / [Understanding the DNS Operator](https://docs.openshift.com/container-platform/4.5/networking/dns-operator.html) [RedHat Blogs: Demystifying Multus](https://www.openshift.com/blog/demystifying-multus) ![](https://www.openshift.com/hubfs/Demystifying%20Multus-1.png) ![](https://www.openshift.com/hubfs/Demystifying%20Multus-4.png) [Youtube Video: Ask an OpenShift Admin (Ep 24): CNI plugins and Multus](https://www.youtube.com/watch?v=aFjyzVUJO70) [Roberto Carratalá blog post: DNS Deep Dive in Openshift 4](https://rcarrata.com/openshift/dns-deep-dive-in-openshift/) [Anatomy of a Linux DNS Lookup – Part I](https://zwischenzugs.com/2018/06/08/anatomy-of-a-linux-dns-lookup-part-i/) ``` [student@workstation ~]$ oc get clusternetwork NAME CLUSTER NETWORK SERVICE NETWORK PLUGIN NAME default 10.8.0.0/14 172.30.0.0/16 redhat/openshift-ovs-networkpolicy [student@workstation ~]$ oc describe clusternetwork Name: default Created: 6 months ago Labels: <none> Annotations: <none> Service Network: 172.30.0.0/16 Plugin Name: redhat/openshift-ovs-networkpolicy ClusterNetworks: CIDR Host Subnet Length ---- ------------------ 10.8.0.0/14 9 [student@workstation ~]$ oc get hostsubnet NAME HOST HOST IP SUBNET EGRESS CIDRS EGRESS IPS master01 master01 192.168.50.10 10.10.0.0/23 master02 master02 192.168.50.11 10.8.0.0/23 master03 master03 192.168.50.12 10.9.0.0/23 # Lets Calculate this subnet networks # http://jodies.de/ipcalc?host=10.8.0.0&mask1=14&mask2=23 Address (Host or Network) Netmask Netmask for sub/supernet (optional) 10.8.0.0 14 move to: 23 Address: 10.8.0.0 00001010.000010 00.00000000.00000000 Netmask: 255.252.0.0 = 14 11111111.111111 00.00000000.00000000 Wildcard: 0.3.255.255 00000000.000000 11.11111111.11111111 => Network: 10.8.0.0/14 00001010.000010 00.00000000.00000000 (Class A) Broadcast: 10.11.255.255 00001010.000010 11.11111111.11111111 HostMin: 10.8.0.1 00001010.000010 00.00000000.00000001 HostMax: 10.11.255.254 00001010.000010 11.11111111.11111110 Hosts/Net: 262142 (Private Internet) Subnets: Netmask: 255.255.254.0 = 23 11111111.111111 11.1111111 0.00000000 Wildcard: 0.0.1.255 00000000.000000 00.0000000 1.11111111 master01: Network: 10.10.0.0/23 00001010.000010 10.0000000 0.00000000 (Class A) Broadcast: 10.10.1.255 00001010.000010 10.0000000 1.11111111 HostMin: 10.10.0.1 00001010.000010 10.0000000 0.00000001 HostMax: 10.10.1.254 00001010.000010 10.0000000 1.11111110 Hosts/Net: 510 (Private Internet) master02: Network: 10.8.0.0/23 00001010.000010 00.0000000 0.00000000 (Class A) Broadcast: 10.8.1.255 00001010.000010 00.0000000 1.11111111 HostMin: 10.8.0.1 00001010.000010 00.0000000 0.00000001 HostMax: 10.8.1.254 00001010.000010 00.0000000 1.11111110 Hosts/Net: 510 (Private Internet) master03: Network: 10.9.0.0/23 00001010.000010 01.0000000 0.00000000 (Class A) Broadcast: 10.9.1.255 00001010.000010 01.0000000 1.11111111 HostMin: 10.9.0.1 00001010.000010 01.0000000 0.00000001 HostMax: 10.9.1.254 00001010.000010 01.0000000 1.11111110 Hosts/Net: 510 (Private Internet) ``` [OCP 4.5 Docs: Troubleshooting OpenShift SDN](https://docs.openshift.com/container-platform/3.11/admin_guide/sdn_troubleshooting.html) The Interfaces on a Node These are the interfaces that the OpenShift SDN creates: - `br0`: The OVS bridge device that containers will be attached to. OpenShift SDN also configures a set of non-subnet-specific flow rules on this bridge. - `tun0`: An OVS internal port (port 2 on `br0`). This gets assigned the cluster subnet gateway address, and is used for external network access. OpenShift SDN configures `netfilter` and routing rules to enable access from the cluster subnet to the external network via NAT. - `vxlan_sys_4789`: The OVS VXLAN device (port 1 on `br0`), which provides access to containers on remote nodes. Referred to as `vxlan0` in the OVS rules. - `vethX` (in the main netns): A Linux virtual ethernet peer of `eth0` in the Docker netns. It will be attached to the OVS bridge on one of the other ports. ![SDN Flows Inside a Node](https://access.redhat.com/webassets/avalon/d/OpenShift_Container_Platform-3.11-Cluster_Administration-en-US/images/5484a5b9d1f394e368f0dfb6066165c1/SDN.png) Depending on what you are trying to access (or be accessed from) the path will vary. There are four different places the SDN connects (inside a node). They are labeled in red on the diagram above. - **Pod:** Traffic is going from one pod to another on the same machine (1 to a different 1) - **Remote Node (or Pod):** Traffic is going from a local pod to a remote node or pod in the same cluster (1 to 2) - **External Machine:** Traffic is going from a local pod outside the cluster (1 to 3) Of course the opposite traffic flows are also possible. #### Exposing Applications for External Access [How about using LetsEncrypt Certificates in Openshift?](https://letsencrypt.org/ru/docs/client-options/#clients-openshift) [openshift-acme is ACME Controller for OpenShift and Kubernetes clusters](https://github.com/tnozicka/openshift-acme) [Medium Blog: OpenShift 4.4 Ingress across multiple projects](https://daein.medium.com/openshift-4-4-ingress-across-multiple-projects-e5ea58e8f35b) [OpenShift ACME Controller Ansible Role](https://github.com/appuio/ansible-role-openshift-acme) [Self-Serviced End-to-end Encryption Approaches for Applications Deployed in OpenShift](https://www.openshift.com/blog/self-serviced-end-to-end-encryption-approaches-for-applications-deployed-in-openshift) ![](https://assets.openshift.com/hubfs/Imported_Blog_Media/OCP-PKI-and-certificates-reverse-proxy.png?width=1200&height=677&name=OCP-PKI-and-certificates-reverse-proxy.png) In the diagram we can see: 1. [Clear text:](https://docs.openshift.com/container-platform/3.11/architecture/networking/routes.html#route-types) the connection is always unencrypted. 2. [Edge](https://docs.openshift.com/container-platform/3.11/architecture/networking/routes.html#secured-routes): the connection is encrypted from the client to the reverse proxy, but unencrypted from the reverse proxy to the pod. 3. [Re-encrypt](https://docs.openshift.com/container-platform/3.11/architecture/networking/routes.html#secured-routes): the encrypted connection is terminated at the reverse proxy, but then re-encrypted. 4. [Passthrough](https://docs.openshift.com/container-platform/3.11/architecture/networking/routes.html#secured-routes): the connection is not encrypted by the reverse proxy. The reverse proxy uses the [Server Name Indication](https://en.wikipedia.org/wiki/Server_Name_Indication) (SNI) field to determine to which backend to forward the connection, but in every other respects it acts as a [Layer 4](https://en.wikipedia.org/wiki/Transport_layer) load balancer. [Openshift github Issue: Move TLS-related secrets out of Route object into a Secret](https://github.com/openshift/origin/issues/2162) [Self-Serviced, End-to-End Encryption for Kubernetes Applications, Part 2: a Practical Example](https://www.openshift.com/blog/self-serviced-end-to-end-encryption-for-kubernetes-applications-part-2-a-practical-example) For automating the certificate provisioning process, we are going to use three [operators](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/): 1. [Cert-manager](https://github.com/jetstack/cert-manager): this operator is responsible for provisioning certificates. It interfaces with the various CAs and brokers certificate provisioning or renewal requests. Provisioned certificates are placed into Kubernetes [secrets](https://kubernetes.io/docs/concepts/configuration/secret/). 2. [Cert-util-operator](https://github.com/redhat-cop/cert-utils-operator): this operator provides utility functions around certificates. In particular, it will be used to inject certificates into OpenShift routes and turn PEM-formatted certificates into java-readable keystores. 3. [Reloader](https://github.com/stakater/Reloader): this operator is used to trigger a deployment when a configmap or secret changes. We are going to use it to restart our applications when certificates get renewed. [ConsoleLabs Blog:OpenShift and Let's Encrypt](https://labs.consol.de/devops/2020/09/16/openshift-letsencrypt.html) [Habrahabr: SSL-сертификаты от Let's Encrypt с cert-manager в Kubernetes](https://habr.com/ru/company/flant/blog/496936/) Adding security layers to your App on OpenShift * [Part 1: Deployment and TLS Ingress](https://itnext.io/adding-security-layers-to-your-app-on-openshift-part-1-deployment-and-tls-ingress-9ef752835599) * [Part 2: Authentication and Authorization with Keycloak](https://itnext.io/adding-security-layers-to-your-app-on-openshift-part-2-8320018bcdd1) * [Part 3: Secret Management with Vault](https://itnext.io/adding-security-layers-to-your-app-on-openshift-part-3-secret-management-with-vault-8efd4ec29ec4) * [Part 4: Dynamic secrets with Vault](https://itnext.io/adding-security-layers-to-your-app-on-openshift-part-4-dynamic-secrets-with-vault-b5fe1fc7709b) * [Part 5: Mutual TLS with Istio](https://itnext.io/adding-security-layers-to-your-app-on-openshift-part-5-mutual-tls-with-istio-a8800c2e4df4) * [Part 6: PKI as a Service with Vault and Cert Manager](https://itnext.io/adding-security-layers-to-your-app-on-openshift-part-6-pki-as-a-service-with-vault-and-cert-e6dbbe7028c7) https://docs.openshift.com/container-platform/4.7/networking/openshift_sdn/assigning-egress-ips.html https://habr.com/ru/company/jetinfosystems/blog/527482/ #### Configuring Network Policies Theory: [RedHat Blogs: What's New in OpenShift 3.5: Network Policy (Tech Preview)](https://www.openshift.com/blog/whats-new-in-openshift-3-5-network-policy-tech-preview) Practice: [RedHat Blogs: Network Policy Objects in Action](https://www.openshift.com/blog/network-policy-objects-action) ![](https://www.openshift.com/hs-fs/hubfs/Imported_Blog_Media/microservices_architecture-1024x597.png?width=1536&height=896&name=microservices_architecture-1024x597.png) [Cilium: Network Policy Editor for Kubernetes](https://editor.cilium.io) [Container Networking Is Simple!](https://iximiuz.com/en/posts/container-networking-is-simple/) ### Chapter 6 Controlling Pod Scheduling #### Controlling Pod Scheduling Behavior [OCP 4.5 Docs](https://docs.openshift.com/) / [4.5](https://docs.openshift.com/container-platform/4.5/welcome/index.html) / Nodes / Controlling pod placement onto nodes (scheduling) / [About pod placement using the scheduler](https://docs.openshift.com/container-platform/4.5/nodes/scheduling/nodes-scheduler-about.html) [Amazon Web Services Regions and Availability Zones](https://aws.amazon.com/ru/about-aws/global-infrastructure/regions_az/) [Azure geographies](https://azure.microsoft.com/en-us/global-infrastructure/geographies/) [Microsoft Azure Regions Interactive Map of Azure regions (unofficial map)](https://build5nines.com/map-azure-regions/) [Azure Availability zones](https://docs.microsoft.com/en-us/azure/availability-zones/az-overview) ![](https://docs.microsoft.com/en-us/azure/availability-zones/media/az-overview/az-graphic-two.png) [OCP 4.5 Docs: Node Placement and Scheduling Explained](https://www.openshift.com/blog/node-placement-scheduling-explained) [Github Project: Cluster Autoscaler FAQ](https://github.com/openshift/kubernetes-autoscaler/blob/master/cluster-autoscaler/FAQ.md) [Openshift Blogs: Descheduler GA in OpenShift 4.7](https://www.openshift.com/blog/descheduler-ga-in-openshift-4.7) #### Limiting Resource Usage by an Application [RedHat Blogs: How Full is My Cluster? Capacity Management and Monitoring on OpenShift](https://www.openshift.com/blog/full-cluster-capacity-management-monitoring-openshift) [Kubernetes Operational View](https://github.com/raffaelespazzoli/kube-ops-view/tree/ocp) ![](https://github.com/raffaelespazzoli/kube-ops-view/raw/ocp/screenshot.png) https://docs.openshift.com/container-platform/3.11/admin_guide/out_of_resource_handling.html https://docs.openshift.com/container-platform/3.11/admin_guide/overcommit.html https://developers.redhat.com/blog/2020/11/10/you-probably-need-liveness-and-readiness-probes/ https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zombie-reaping-problem/ https://www.openshift.com/blog/liveness-and-readiness-probes https://daein.medium.com/startup-liveness-and-readiness-probes-on-openshift-fdcb04f36b53 ![](https://miro.medium.com/max/875/0*ihMQVwVwHbQ_RVeY) Cgroups * [RedHat Blogs: Mark Richters World domination with Cgroups - Part 1 - Cgroup basics](https://www.redhat.com/en/blog/world-domination-cgroups-part-1-cgroup-basics) * [Habr: redhatrussia: Борьба за ресурсы, часть 1: Основы Cgroups](https://habr.com/ru/company/redhatrussia/blog/423051/) * [Habr: redhatrussia: Борьба за ресурсы, часть 2: – Играемся с настройками Cgroups](habr.com/company/redhatrussia/blog/424367) * [Habr: redhatrussia: Борьба за ресурсы, часть 3: Памяти мало не бывает](https://habr.com/ru/company/redhatrussia/blog/425803/) * [Habr: redhatrussia: Борьба за ресурсы, часть 4: Замечательно выходит](https://habr.com/ru/company/redhatrussia/blog/427413/) * [Habr: redhatrussia: Борьба за ресурсы, часть 5: Начиная с нуля](habr.com/company/redhatrussia/blog/429064) * [Habr: redhatrussia: Борьба за ресурсы, часть 6: cpuset или Делиться не всегда правильно](habr.com/company/redhatrussia/blog/430748) #### Scaling an Application https://www.openshift.com/blog/node-placement-scheduling-explained [Как избежать простоя в работе Kubernetes-кластера при помощи PodDisruptionBudgets](https://habr.com/ru/company/nixys/blog/490680/) [Kubernetes Docs: Specifying a PodDisruptionBudget](https://kubernetes.io/docs/tasks/run-application/configure-pdb/#specifying-a-poddisruptionbudget) https://softchris.github.io/pages/kubernetes-two.html https://softchris.github.io/pages/kubernetes-three.html How to delete pods hanging in Terminating state https://access.redhat.com/solutions/2317401 [habr Flant article: ертикальное автомасштабирование pod'ов в Kubernetes: полное руководство](https://habr.com/ru/company/flant/blog/541642/) probes https://dev.to/pavanbelagatti/configure-kubernetes-readiness-and-liveness-probes-tutorial-478p ### Chapter 7. Describing Cluster Updates #### Describing the Cluster Update Process [Youtube Video: Openshift TV: OpenShift Updates and Release Process w/ Rob Szumski and Scott Dodson](https://www.youtube.com/watch?v=DK5RoIsO8_g) [Red Hat Blogs: The Ultimate Guide to OpenShift Release and Upgrade Process for Cluster Administrators](https://www.openshift.com/blog/the-ultimate-guide-to-openshift-release-and-upgrade-process-for-cluster-administrators) [Red Hat Blog: Time Is On Your Side: A Change to the OpenShift 4 Lifecycle](https://cloud.redhat.com/blog/time-is-on-your-side-a-change-to-the-openshift-4-lifecycle) [Red Hat KB: Red Hat OpenShift Container Platform Life Cycle Policy](https://access.redhat.com/support/policy/updates/openshift) [RHLC: Demo video Upgrading OpenShift](https://learn.redhat.com/t5/Expert-Extra-Videos/Demo-video-Upgrading-OpenShift/ba-p/24116) ### Chapter 8. Managing a Cluster with the Web Console #### Performing Cluster Administration [Red Hat User Experience Design (UXD)](https://www.redhat.com/en/about/product-design) [OpenShift Design](http://openshift.github.io/openshift-origin-design/) - This repository includes [designs](https://openshift.github.io/openshift-origin-design/designs/), [conventions](https://openshift.github.io/openshift-origin-design/conventions/), and [research](https://openshift.github.io/openshift-origin-design/research/) related to the OpenShift Container Platform ecosystem. https://access.redhat.com/labs/ocpupgradegraph #### Managing Workloads and Operators Q: What is the Developer Sandbox for Red Hat OpenShift? A: The sandbox provides you with a private OpenShift environment in a shared, multi-tenant OpenShift cluster that is pre-configured with a set of developer tools. [Developer Sandbox for Red Hat OpenShift](https://developers.redhat.com/developer-sandbox) Q: What is the Developer Sandbox for Red Hat OpenShift? A: The sandbox provides you with a private OpenShift environment in a shared, multi-tenant OpenShift cluster that is pre-configured with a set of developer tools. [Youtube Stream: Learning CodeReady Workspaces from Red Hat on OpenShift](https://www.youtube.com/watch?v=RbhUnvlh1dI) [Command-line cluster management with Red Hat OpenShift’s new web terminal (tech preview)](https://developers.redhat.com/blog/2020/10/01/command-line-cluster-management-with-red-hat-openshifts-new-web-terminal-tech-preview/) [Red Hat Blog: A Deeper Look at the Web Terminal Operator](https://www.openshift.com/blog/a-deeper-look-at-the-web-terminal-operator-1) #### Examining Cluster Metrics [RedHat Developers Blog: Understanding Red Hat OpenShift’s Application Monitoring Operator](https://developers.redhat.com/blog/2019/09/10/understanding-red-hat-openshifts-application-monitoring-operator/) [Kubernetes monitoring with Prometheus, the ultimate guide](https://sysdig.com/blog/kubernetes-monitoring-prometheus/) [Red Hat Blog: Monitoring your own workloads in the Developer Console in OpenShift Container Platform 4.6](https://www.openshift.com/blog/monitoring-your-own-workloads-in-the-developer-console-in-openshift-container-platform-4.6) [Red Hat Developer Blog: More for developers in the new Red Hat OpenShift 4.6 web console](https://developers.redhat.com/blog/2020/11/24/more-for-developers-in-the-new-red-hat-openshift-4-6-web-console/) [Red Hat Blog: How Full is My Cluster? Capacity Management and Monitoring on OpenShift](https://www.openshift.com/blog/full-cluster-capacity-management-monitoring-openshift) [Red Hat Blog: Configure OpenShift Metrics with Prometheus backed by OpenShift Container Storage](https://www.openshift.com/blog/configure-openshift-metrics-with-prometheus-backed-by-openshift-container-storage) ### Chapter 9. Comprehensive Review Red Hat Communities of Practice https://github.com/redhat-cop/containers-quickstarts Openshift Evangelists https://github.com/openshift-evangelists/wordpress-quickstart https://github.com/openshift-evangelists/phpbb-quickstart https://github.com/RedHatWorkshops https://github.com/openshift-labs/lab-learning-portal https://www.openshift.com/blog/openshift-4.6-blog-quick-starts [Red Hat Developer e-books](https://developers.redhat.com/ebooks?sc_cid=7013a000002gj5LAAQ) https://t.me/ru_openshift [GitOps Guide to the Galaxy | OpenShift.tv](red.ht/gitops)