# Kubernetes workspaces demo

We are going to "workspace-ify" our Kubernetes cluster by defining 3 different workspaces within our multi-tenant cluster:

* `ops-team`
* `finance-team`
* `backend-team`

## Setup

### KUBECONFIG contexts

For this demo, we have a number of contexts setup:

- `kind-kind` - the 'real' Kubernetes cluster
- `wildcard` - points to `/clusters/*`
- `finance` - points to `/clusters/finance-team`
- `backend` - points to `/clusters/backend-team`

### Install CustomResourceDefinitions:

$ kubectl create --context kind-kind \
    -f ./config/crds/rbac.authorization.clusterscoped.kcp.io_workspacerolebindings.yaml \
    -f ./config/crds/rbac.authorization.clusterscoped.kcp.io_workspaceroles.yaml \
    -f ./config/crds/tenancy.kcp.io_workspaceconfigurations.yaml

### Namespaces

We have a standard 'kind' cluster:

(⎈|kind-kind:N/A)james@Jamess-MacBook-Pro workspaces-apiserver % kubectl get namespaces
NAME                 STATUS   AGE
default              Active   4d6h
kube-node-lease      Active   4d6h
kube-public          Active   4d6h
kube-system          Active   4d6h
local-path-storage   Active   4d6h

We create a few dummy namespaces for our teams:

$ echo \
    "finance-team-operators \
    finance-team-jobs \
    backend-webservices \
    backend-searchservice \
    backend-staging \
    backend-development" | xargs -n 1 kubectl create namespace

We create 'system' namespaces for the three workspaces:

$ echo \
    "workspace-scoped-ops-team \
    workspace-scoped-finance-team \
    workspace-scoped-backend-team" | xargs -n 1 kubectl create namespace

### Demo workloads

Create some demo workloads so we can all see what is going on:

# Create dummy workloads for the backend team
$ kubectl create deployment --image nginx --namespace backend-webservices webserver --replicas=3
$ kubectl create deployment --image nginx --namespace backend-searchservice mycool-search-tool --replicas=3
$ kubectl create deployment --image nginx --namespace backend-staging searchtool-staging --replicas=1
$ kubectl create deployment --image nginx --namespace backend-development searchtool-dev-newfeature --replicas=1
$ kubectl create deployment --image nginx --namespace backend-development searchtool-dev-coolthing --replicas=1

# Create dummy workloads for the finance team
$ kubectl create deployment --image nginx --namespace finance-team-operators moneymaking-operator --replicas=1
$ kubectl create cronjob --image nginx --namespace finance-team-jobs makemoney-job

### WorkspaceConfiguration

This is created in the `kind-kind` cluster and defines the mapping from workspace to namespaces (and where workspace-scoped objects are stored):

apiVersion: tenancy.kcp.io/v1alpha1
kind: WorkspaceConfiguration
metadata:
  name: workspace-apiserver-config
spec:
  workspaces:
  - name: ops-team
    systemNamespace: workspace-scoped-ops-team
    namespaces:
    - kube-system
    - kube-public
    - kube-node-lease
    - local-path-storage
  - name: finance-team
    systemNamespace: workspace-scoped-finance-team
    namespaces:
    - finance-team-operators
    - finance-team-jobs
  - name: backend-team
    systemNamespace: workspace-scoped-backend-team
    namespaces:
    - backend-webservices
    - backend-searchservice
    - backend-staging
    - backend-development

> NOTE: This object is 'low level' and in future would be auto-managed by a controller watching for `Workspace` objects or something similar (tbd)

### Creating 'workspace admins'

# Ops admins
apiVersion: rbac.authorization.clusterscoped.kcp.io/v1
kind: WorkspaceRole
metadata:
  name: workspace-admin
  namespace: workspace-scoped-ops-team
rules:
- apiGroups:
  - "*"
  resources:
  - '*'
  verbs:
  - '*'
---
apiVersion: rbac.authorization.clusterscoped.kcp.io/v1
kind: WorkspaceRoleBinding
metadata:
  name: workspace-admin
  namespace: workspace-scoped-ops-team
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole # todo: this should be WorkspaceRole in future
  name: workspace-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: [e-mail]
---
# Finance admins
apiVersion: rbac.authorization.clusterscoped.kcp.io/v1
kind: WorkspaceRole
metadata:
  name: workspace-admin
  namespace: workspace-scoped-finance-team
rules:
- apiGroups:
  - "*"
  resources:
  - '*'
  verbs:
  - '*'
---
apiVersion: rbac.authorization.clusterscoped.kcp.io/v1
kind: WorkspaceRoleBinding
metadata:
  name: workspace-admin
  namespace: workspace-scoped-finance-team
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole # todo: this should be WorkspaceRole in future
  name: workspace-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: [e-mail]
---
# Backend admins
apiVersion: rbac.authorization.clusterscoped.kcp.io/v1
kind: WorkspaceRole
metadata:
  name: workspace-admin
  namespace: workspace-scoped-backend-team
rules:
- apiGroups:
  - "*"
  resources:
  - '*'
  verbs:
  - '*'
---
apiVersion: rbac.authorization.clusterscoped.kcp.io/v1
kind: WorkspaceRoleBinding
metadata:
  name: workspace-admin
  namespace: workspace-scoped-backend-team
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole # todo: this should be WorkspaceRole in future
  name: workspace-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: [e-mail]

### Running workspaces-apiserver

Get the apiserver running - we configure the OIDC authenticator at the moment, however you would typically configure to delegate authN to the real cluster (via `TokenReview`)

$ ./workspaces-apiserver start \
    --oidc-client-id=<clientid-here> \
    --oidc-issuer-url=https://accounts.google.com \
    --oidc-username-claim=email \
    --workspace-configuration-name=workspace-apiserver-config \
    --workspace-shard-name mycompany:us-west-1 \
    --workspace-cluster-kubeconfig-file $HOME/.kube/config \
    --workspace-cluster-context-name kind-kind

## Usage

With all of this setup, we can see the 'real' view:

(⎈|kind-kind:N/A)james@Jamess-MacBook-Pro kcp % kubectl get namespaces
NAME                            STATUS   AGE
backend-development             Active   37m
backend-searchservice           Active   37m
backend-staging                 Active   37m
backend-webservices             Active   38m
default                         Active   48m
finance-team-jobs               Active   36m
finance-team-operators          Active   36m
kube-node-lease                 Active   48m
kube-public                     Active   48m
kube-system                     Active   48m
local-path-storage              Active   48m
workspace-scoped-backend-team   Active   71s
workspace-scoped-finance-team   Active   71s
workspace-scoped-ops-team       Active   71s

### Via the 'wildcard' context

(⎈|kind-kind:N/A)james@Jamess-MacBook-Pro kcp % kubectl get namespaces --context wildcard
NAME                      STATUS   AGE
backend-development       Active   26m
backend-searchservice     Active   26m
backend-staging           Active   26m
backend-webservices       Active   26m
finance-team-jobs         Active   24m
finance-team-operators    Active   24m
kube-node-lease           Active   37m
kube-public               Active   37m
kube-system               Active   37m
local-path-storage        Active   36m

(⎈|kind-kind:N/A)james@Jamess-MacBook-Pro kcp % kubectl get all --context wildcard --all-namespaces
NAMESPACE                NAME                                             READY   STATUS    RESTARTS      AGE     WORKSPACE
backend-development      pod/searchtool-dev-coolthing-f4d4b8f87-42vc7     1/1     Running   0             13m     backend-team
backend-development      pod/searchtool-dev-newfeature-d94bbb8d-b6hn7     1/1     Running   0             14m     backend-team
backend-searchservice    pod/mycool-search-tool-ccc89d7d4-f9zdz           1/1     Running   0             14m     backend-team
backend-searchservice    pod/mycool-search-tool-ccc89d7d4-p7xl6           1/1     Running   0             14m     backend-team
backend-searchservice    pod/mycool-search-tool-ccc89d7d4-s9czj           1/1     Running   0             14m     backend-team
backend-staging          pod/searchtool-staging-5f8c55bb8c-5cbzf          1/1     Running   0             14m     backend-team
backend-webservices      pod/webserver-667ddc69b6-94bss                   1/1     Running   0             15m     backend-team
backend-webservices      pod/webserver-667ddc69b6-bh4t5                   1/1     Running   0             15m     backend-team
backend-webservices      pod/webserver-667ddc69b6-hmqtm                   1/1     Running   0             15m     backend-team
finance-team-jobs        pod/makemoney-job-28615114-2p7dg                 1/1     Running   0             10m     finance-team
finance-team-jobs        pod/makemoney-job-28615115-mb4hn                 1/1     Running   0             9m39s   finance-team
finance-team-jobs        pod/makemoney-job-28615116-sk7d9                 1/1     Running   0             8m39s   finance-team
finance-team-jobs        pod/makemoney-job-28615117-bl8hk                 1/1     Running   0             7m39s   finance-team
finance-team-jobs        pod/makemoney-job-28615118-bnvlt                 1/1     Running   0             6m39s   finance-team
finance-team-jobs        pod/makemoney-job-28615119-v8nt4                 1/1     Running   0             5m39s   finance-team
finance-team-jobs        pod/makemoney-job-28615120-76vtk                 1/1     Running   0             4m39s   finance-team
finance-team-jobs        pod/makemoney-job-28615121-g5nnc                 1/1     Running   0             3m39s   finance-team
finance-team-jobs        pod/makemoney-job-28615122-xqvlr                 1/1     Running   0             2m39s   finance-team
finance-team-jobs        pod/makemoney-job-28615123-7kb5b                 1/1     Running   0             99s     finance-team
finance-team-jobs        pod/makemoney-job-28615124-9qpng                 1/1     Running   0             39s     finance-team
finance-team-operators   pod/moneymaking-operator-67f995d6f7-52zf4        1/1     Running   0             12m     finance-team
kube-system              pod/coredns-76f75df574-2gnfj                     1/1     Running   1 (36m ago)   41m     ops-team
kube-system              pod/coredns-76f75df574-8rq7f                     1/1     Running   1 (36m ago)   41m     ops-team
kube-system              pod/etcd-kind-control-plane                      1/1     Running   0             36m     ops-team
kube-system              pod/kindnet-59vvq                                1/1     Running   1 (36m ago)   41m     ops-team
kube-system              pod/kindnet-zb5kv                                1/1     Running   1 (36m ago)   41m     ops-team
kube-system              pod/kube-apiserver-kind-control-plane            1/1     Running   0             36m     ops-team
kube-system              pod/kube-controller-manager-kind-control-plane   1/1     Running   1 (36m ago)   41m     ops-team
kube-system              pod/kube-proxy-9q5ps                             1/1     Running   1 (36m ago)   41m     ops-team
kube-system              pod/kube-proxy-ljthv                             1/1     Running   1 (36m ago)   41m     ops-team
kube-system              pod/kube-scheduler-kind-control-plane            1/1     Running   1 (36m ago)   41m     ops-team
local-path-storage       pod/local-path-provisioner-7577fdbbfb-5jzd4      1/1     Running   2 (36m ago)   41m     ops-team

NAMESPACE     NAME               TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                  AGE   WORKSPACE
kube-system   service/kube-dns   ClusterIP   10.96.0.10   <none>        53/UDP,53/TCP,9153/TCP   41m   ops-team

NAMESPACE           NAME                          SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE   WORKSPACE
finance-team-jobs   cronjob.batch/makemoney-job   */1 * * * *   False     11       39s             11m   finance-team

NAMESPACE           NAME                               COMPLETIONS   DURATION   AGE     WORKSPACE
finance-team-jobs   job.batch/makemoney-job-28615114   0/1           10m        10m     finance-team
finance-team-jobs   job.batch/makemoney-job-28615115   0/1           9m39s      9m39s   finance-team
finance-team-jobs   job.batch/makemoney-job-28615116   0/1           8m39s      8m39s   finance-team
finance-team-jobs   job.batch/makemoney-job-28615117   0/1           7m39s      7m39s   finance-team
finance-team-jobs   job.batch/makemoney-job-28615118   0/1           6m39s      6m39s   finance-team
finance-team-jobs   job.batch/makemoney-job-28615119   0/1           5m39s      5m39s   finance-team
finance-team-jobs   job.batch/makemoney-job-28615120   0/1           4m39s      4m39s   finance-team
finance-team-jobs   job.batch/makemoney-job-28615121   0/1           3m39s      3m39s   finance-team
finance-team-jobs   job.batch/makemoney-job-28615122   0/1           2m39s      2m39s   finance-team
finance-team-jobs   job.batch/makemoney-job-28615123   0/1           99s        99s     finance-team
finance-team-jobs   job.batch/makemoney-job-28615124   0/1           39s        39s     finance-team

NAMESPACE     NAME                        DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE   WORKSPACE
kube-system   daemonset.apps/kindnet      2         2         2       2            2           kubernetes.io/os=linux   41m   ops-team
kube-system   daemonset.apps/kube-proxy   2         2         2       2            2           kubernetes.io/os=linux   41m   ops-team

NAMESPACE                NAME                                        READY   UP-TO-DATE   AVAILABLE   AGE   WORKSPACE
backend-development      deployment.apps/searchtool-dev-coolthing    1/1     1            1           13m   backend-team
backend-development      deployment.apps/searchtool-dev-newfeature   1/1     1            1           14m   backend-team
backend-searchservice    deployment.apps/mycool-search-tool          3/3     3            3           14m   backend-team
backend-staging          deployment.apps/searchtool-staging          1/1     1            1           14m   backend-team
backend-webservices      deployment.apps/webserver                   3/3     3            3           15m   backend-team
finance-team-operators   deployment.apps/moneymaking-operator        1/1     1            1           12m   finance-team
kube-system              deployment.apps/coredns                     2/2     2            2           41m   ops-team
local-path-storage       deployment.apps/local-path-provisioner      1/1     1            1           41m   ops-team

NAMESPACE                NAME                                                 DESIRED   CURRENT   READY   AGE   WORKSPACE
backend-development      replicaset.apps/searchtool-dev-coolthing-f4d4b8f87   1         1         1       13m   backend-team
backend-development      replicaset.apps/searchtool-dev-newfeature-d94bbb8d   1         1         1       14m   backend-team
backend-searchservice    replicaset.apps/mycool-search-tool-ccc89d7d4         3         3         3       14m   backend-team
backend-staging          replicaset.apps/searchtool-staging-5f8c55bb8c        1         1         1       14m   backend-team
backend-webservices      replicaset.apps/webserver-667ddc69b6                 3         3         3       15m   backend-team
finance-team-operators   replicaset.apps/moneymaking-operator-67f995d6f7      1         1         1       12m   finance-team
kube-system              replicaset.apps/coredns-76f75df574                   2         2         2       41m   ops-team
local-path-storage       replicaset.apps/local-path-provisioner-7577fdbbfb    1         1         1       41m   ops-team

(⎈|kind-kind:N/A)james@Jamess-MacBook-Pro kcp % kubectl get clusterrole,clusterrolebinding --context wildcard
NAME                                                    CREATED AT
clusterrole.rbac.authorization.k8s.io/workspace-admin   2024-05-28T14:59:54Z
clusterrole.rbac.authorization.k8s.io/workspace-admin   2024-05-28T14:59:54Z
clusterrole.rbac.authorization.k8s.io/workspace-admin   2024-05-28T14:59:54Z

NAME                                                           ROLE                          AGE
clusterrolebinding.rbac.authorization.k8s.io/workspace-admin   ClusterRole/workspace-admin   69m
clusterrolebinding.rbac.authorization.k8s.io/workspace-admin   ClusterRole/workspace-admin   69m
clusterrolebinding.rbac.authorization.k8s.io/workspace-admin   ClusterRole/workspace-admin   69m

### Via the 'finance' context

(⎈|kind-kind:N/A)james@Jamess-MacBook-Pro kcp % kubectl get namespaces --context finance
NAME                     STATUS   AGE
finance-team-jobs        Active   53m
finance-team-operators   Active   53m

(⎈|kind-kind:N/A)james@Jamess-MacBook-Pro kcp % kubectl get all --context finance --all-namespaces
NAMESPACE                NAME                                        READY   STATUS    RESTARTS   AGE
finance-team-jobs        pod/makemoney-job-28615152-vhgjx            1/1     Running   0          8s
finance-team-operators   pod/moneymaking-operator-67f995d6f7-52zf4   1/1     Running   0          39m

NAMESPACE           NAME                          SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE
finance-team-jobs   cronjob.batch/makemoney-job   */1 * * * *   False     8        8s              38m

NAMESPACE           NAME                               COMPLETIONS   DURATION   AGE
finance-team-jobs   job.batch/makemoney-job-28615132   0/1           20m        20m
finance-team-jobs   job.batch/makemoney-job-28615133   0/1           19m        19m
finance-team-jobs   job.batch/makemoney-job-28615134   0/1           18m        18m
finance-team-jobs   job.batch/makemoney-job-28615135   0/1           17m        17m
finance-team-jobs   job.batch/makemoney-job-28615142   0/1           10m        10m
finance-team-jobs   job.batch/makemoney-job-28615147   1/1           5m7s       5m8s
finance-team-jobs   job.batch/makemoney-job-28615148   1/1           4m7s       4m8s
finance-team-jobs   job.batch/makemoney-job-28615149   1/1           3m7s       3m8s
finance-team-jobs   job.batch/makemoney-job-28615150   0/1           2m8s       2m8s
finance-team-jobs   job.batch/makemoney-job-28615151   0/1           68s        68s
finance-team-jobs   job.batch/makemoney-job-28615152   0/1           8s         8s

NAMESPACE                NAME                                   READY   UP-TO-DATE   AVAILABLE   AGE
finance-team-operators   deployment.apps/moneymaking-operator   1/1     1            1           39m

NAMESPACE                NAME                                              DESIRED   CURRENT   READY   AGE
finance-team-operators   replicaset.apps/moneymaking-operator-67f995d6f7   1         1         1       39m

(⎈|kind-kind:N/A)james@Jamess-MacBook-Pro kcp % kubectl get clusterrole,clusterrolebinding --context finance
NAME                                                    CREATED AT
clusterrole.rbac.authorization.k8s.io/workspace-admin   2024-05-28T14:59:54Z

NAME                                                           ROLE                          AGE
clusterrolebinding.rbac.authorization.k8s.io/workspace-admin   ClusterRole/workspace-admin   68m

### Via the 'backend' context

(⎈|kind-kind:N/A)james@Jamess-MacBook-Pro kcp % kubectl get namespaces --context backend
NAME                      STATUS   AGE
backend-development       Active   26m
backend-searchservice     Active   26m
backend-staging           Active   26m
backend-webservices       Active   26m

(⎈|kind-kind:N/A)james@Jamess-MacBook-Pro kcp % kubectl get all --context backend --all-namespaces
NAMESPACE               NAME                                           READY   STATUS    RESTARTS   AGE
backend-development     pod/searchtool-dev-coolthing-f4d4b8f87-42vc7   1/1     Running   0          42m
backend-development     pod/searchtool-dev-newfeature-d94bbb8d-b6hn7   1/1     Running   0          42m
backend-searchservice   pod/mycool-search-tool-ccc89d7d4-f9zdz         1/1     Running   0          43m
backend-searchservice   pod/mycool-search-tool-ccc89d7d4-p7xl6         1/1     Running   0          43m
backend-searchservice   pod/mycool-search-tool-ccc89d7d4-s9czj         1/1     Running   0          43m
backend-staging         pod/searchtool-staging-5f8c55bb8c-5cbzf        1/1     Running   0          42m
backend-webservices     pod/webserver-667ddc69b6-94bss                 1/1     Running   0          44m
backend-webservices     pod/webserver-667ddc69b6-bh4t5                 1/1     Running   0          44m
backend-webservices     pod/webserver-667ddc69b6-hmqtm                 1/1     Running   0          44m

NAMESPACE               NAME                                        READY   UP-TO-DATE   AVAILABLE   AGE
backend-development     deployment.apps/searchtool-dev-coolthing    1/1     1            1           42m
backend-development     deployment.apps/searchtool-dev-newfeature   1/1     1            1           42m
backend-searchservice   deployment.apps/mycool-search-tool          3/3     3            3           43m
backend-staging         deployment.apps/searchtool-staging          1/1     1            1           42m
backend-webservices     deployment.apps/webserver                   3/3     3            3           44m

NAMESPACE               NAME                                                 DESIRED   CURRENT   READY   AGE
backend-development     replicaset.apps/searchtool-dev-coolthing-f4d4b8f87   1         1         1       42m
backend-development     replicaset.apps/searchtool-dev-newfeature-d94bbb8d   1         1         1       42m
backend-searchservice   replicaset.apps/mycool-search-tool-ccc89d7d4         3         3         3       43m
backend-staging         replicaset.apps/searchtool-staging-5f8c55bb8c        1         1         1       42m
backend-webservices     replicaset.apps/webserver-667ddc69b6                 3         3         3       44m

(⎈|kind-kind:N/A)james@Jamess-MacBook-Pro kcp % kubectl get clusterrole,clusterrolebinding --context backend
NAME                                                    CREATED AT
clusterrole.rbac.authorization.k8s.io/workspace-admin   2024-05-28T14:59:54Z

NAME                                                           ROLE                          AGE
clusterrolebinding.rbac.authorization.k8s.io/workspace-admin   ClusterRole/workspace-admin   68m

### Via the 'ops' context

(⎈|kind-kind:N/A)james@Jamess-MacBook-Pro kcp % kubectl get namespaces --context ops
NAME                 STATUS   AGE
kube-node-lease      Active   65m
kube-public          Active   65m
kube-system          Active   65m
local-path-storage   Active   65m

(⎈|kind-kind:N/A)james@Jamess-MacBook-Pro kcp % kubectl get all --context ops --all-namespaces
NAMESPACE            NAME                                             READY   STATUS    RESTARTS      AGE
kube-system          pod/coredns-76f75df574-2gnfj                     1/1     Running   1 (65m ago)   70m
kube-system          pod/coredns-76f75df574-8rq7f                     1/1     Running   1 (65m ago)   70m
kube-system          pod/etcd-kind-control-plane                      1/1     Running   0             65m
kube-system          pod/kindnet-59vvq                                1/1     Running   1 (65m ago)   70m
kube-system          pod/kindnet-zb5kv                                1/1     Running   1 (65m ago)   70m
kube-system          pod/kube-apiserver-kind-control-plane            1/1     Running   0             65m
kube-system          pod/kube-controller-manager-kind-control-plane   1/1     Running   1 (65m ago)   70m
kube-system          pod/kube-proxy-9q5ps                             1/1     Running   1 (65m ago)   70m
kube-system          pod/kube-proxy-ljthv                             1/1     Running   1 (65m ago)   70m
kube-system          pod/kube-scheduler-kind-control-plane            1/1     Running   1 (65m ago)   70m
local-path-storage   pod/local-path-provisioner-7577fdbbfb-5jzd4      1/1     Running   2 (64m ago)   70m

NAMESPACE     NAME               TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                  AGE
kube-system   service/kube-dns   ClusterIP   10.96.0.10   <none>        53/UDP,53/TCP,9153/TCP   70m

NAMESPACE     NAME                        DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
kube-system   daemonset.apps/kindnet      2         2         2       2            2           kubernetes.io/os=linux   70m
kube-system   daemonset.apps/kube-proxy   2         2         2       2            2           kubernetes.io/os=linux   70m

NAMESPACE            NAME                                     READY   UP-TO-DATE   AVAILABLE   AGE
kube-system          deployment.apps/coredns                  2/2     2            2           70m
local-path-storage   deployment.apps/local-path-provisioner   1/1     1            1           70m

NAMESPACE            NAME                                                DESIRED   CURRENT   READY   AGE
kube-system          replicaset.apps/coredns-76f75df574                  2         2         2       70m
local-path-storage   replicaset.apps/local-path-provisioner-7577fdbbfb   1         1         1       70m

(⎈|kind-kind:N/A)james@Jamess-MacBook-Pro kcp % kubectl get clusterrole,clusterrolebinding --context ops
NAME                                                    CREATED AT
clusterrole.rbac.authorization.k8s.io/workspace-admin   2024-05-28T14:59:54Z

NAME                                                           ROLE                          AGE
clusterrolebinding.rbac.authorization.k8s.io/workspace-admin   ClusterRole/workspace-admin   68m