# k8s-resource-client

A library to allow the querying of up-to-date cluster resources and objects from outside a cluster. With two modes of operation, one being a greedy "show me what you got" mode and the other being explictly defined through programatic and config file options.

## Goals

* Provide an up-to-date view of resources and objects.
* Work with RBAC restrictions.
* Work across multiple namespaces.
* Filtering with label selectors.

## Non-Goals

* Work across multiple clusters.
* Access or kubeconfig management.
* Create/update is a seperate concerns.

### Assumptions

A valid Kubernetes API endpoint. This should work on internal or external networks (VPNs, any routable network), but is intended for external or "out of cluster" interactions via the API.

When talking about Kubernetes we will use `Resource(s)` to refer to an API endpoint that is used for accessing a collection of `Object(s)` of a certain kind.

This library will be used to help create out-of-cluster clients, in-cluster clients fall under a different problem space.

The cluster credentials will already be well formed by the time k8s-resource-client recives them; note that this is different from them being valid.

### Problem

Displaying all of the resources in a cluster has a range of complexity depending on the level of access the client has to the cluster API.

A cluster admin is able to easily great a cluster wide SharedInformer that will watch all of the resources in the cluster. Very few clients end up being scoped to cluster admin once a project moves out of development.

Different RBAC configurations allow for a set of verbs - Get, List, Watch, Update, Delete - on resources. These RBAC configurations vary highly and can create challegens when making assumptions about to fetch resourcs from a clusters API.

Examples:
- all verbs for a single namespace
- all verbs for a single resource
- get, list for all resources
- get, list for a single namespace

Currently, `client-go` does not provide an effective solution that transparently displays all of the resources a given client has access to.

Another issue is that if the current authorization is voided; for example in the case of rotating certificate auth. There is no effective way to recover.

### Questions

- What should happen if the current access to one or more resources goes away?
- What should happen if the an error is encountered reaching a resource endpoint?
- What should happen if there is no access to the `Watch` verb?
- Watch is not implemented
- What should happen if there is no access to the `List` verb?
- should return an empty list with a "FailedList" error
- UnknownListError
- PermissionDeniedListError
- ResourceNotFoundListError
- What should happen if these is no access to the `Get` verb?
- should return an empty object with "FailedGet" error
- Is there a minimum RBAC expectation?
- What is a minimally running client?
- What does the API look like? How is it consumed?

### Implementation Strategy

#### API / Interface

## Notes

### REST implementation discussion

`client-go` exposes a rest client ("k8s.io/client-go/rest"). we can explore using this exclusively for the implemntation of the interface

The RESTClient requires hard coding each resource into the request. Addition methods extend this request to be more specific for a given resource.

`restClient.Get().Resource("pod")` -> request for all pods
`restClient.Get().Resource("pod").Name("pod-123")` -> a given pod name

Multiple "filters" are also possible and the order of these methods should not matter (although may have performance implications):
`restClient.Get().Resource("pod").Name("pod-123").Namepsace("default")`

#### Dynamic client, simple.go, wraps REST client
https://github.com/kubernetes/client-go/blob/d3b97581d2e22eef0f02a6667e19dfc193df7a3e/dynamic/simple.go#L69

The Dynamic Client returns a "NamespaceableResourceInterface" that has Get, List, Create, Update, Delete methods.

dynamicClient.Resource(schema.GroupVersionResource{
	Group:    "",
	Version:  "",
	Resource: "",
})

Proof of concept for this would be trying this client in Octant and removing the following items:
- namespace listing
- crd listing
- remove Octant typed errors and embrace k8s-resource-client errors

## Brain Dump

kubevious is written in typescript but created a library called k8s-super-client that is a dedicated resource watching client

**Q**: Would `k8s-resource-client` be a drop-in replacement for client-go? Being setup in the same way? Or would we still expect a user to setup client-go FromConfig .. and pass that to us?
***A***: No. I think that there is an expectation that this would understand the existing client-go Client/KubeConfig implementation.

**Q**:Local cache? resourceRevision? in-memory database? shared informers are essentially in-memory databases ...
***A***: If this route is taken, I think I'd leverage what kcp has done and do it in the form of an in-memory etcd.

**Q**:When we can't do a Watch, Get/List but manually implemented to look like an Informer?
***A***: I don't think it needs to satisfy the informer interface. I think switching the underlying implementation of the exposed interface transparently is perfectly reasonable and that implementation does not *HAVE* to satisfy the Infomer interface.

**Q**: Does it matter if we are exposing a `client`?
***A***: This was more about if the library was to go the route of being a whole hog replacement for client-go and that is not the case.

SharedInformers are the most effecient method for watching a large number of resources but can be hard to coordinate and manage when unable to start a cluster wide shared informer.

Looking in to KCP the idea of seperating the fetching concern from the lookup concern is attractive for something like `k8s-resource-client`. Can any parts of KCP be leveraged to make this client?

Node level watching of resources: https://github.com/kubernetes/community/pull/443/files

How might this leverage something like https://github.com/wwitzel3/subjectaccess ?

Interesting read: https://evancordell.com/posts/kube-apis-crds/

#### Relevant Code / Prior Art
- https://github.com/vmware-tanzu/octant/blob/master/internal/objectstore/dynamic_cache.go
- https://github.com/wwitzel3/clientkube
- https://github.com/kubernetes/client-go/blob/master/dynamic/dynamicinformer/informer.go
- https://github.com/kubernetes/client-go/blob/master/dynamic/dynamiclister/lister.go
- https://github.com/kubernetes/client-go/blob/master/tools/cache/shared_informer.go
- https://github.com/kubernetes/client-go/blob/master/tools/watch/informerwatcher.go
- https://github.com/kcp-dev/kcp/blob/main/pkg/crdpuller/discovery.go
- https://github.com/kcp-dev/kcp/blob/main/pkg/syncer/syncer.go#L136