---
# System prepended metadata

title: Device Plugin register options
tags: [K8s]

---

# Device Plugin register options

###### tags: `K8s`

## DevicePluginServer
* In device plugin server, it defines gRPC interfaces:
    ```protobuf=
        // DevicePlugin is the service advertised by Device Plugins
        service DevicePlugin {
	    // GetDevicePluginOptions returns options to be communicated with Device
	    // Manager
	    rpc GetDevicePluginOptions(Empty) returns (DevicePluginOptions) {}

	    // ListAndWatch returns a stream of List of Devices
	    // Whenever a Device state change or a Device disappears, ListAndWatch
	    // returns the new list
	    rpc ListAndWatch(Empty) returns (stream ListAndWatchResponse) {}

	    // GetPreferredAllocation returns a preferred set of devices to allocate
	    // from a list of available ones. The resulting preferred allocation is not
	    // guaranteed to be the allocation ultimately performed by the
	    // devicemanager. It is only designed to help the devicemanager make a more
	    // informed allocation decision when possible.
	    rpc GetPreferredAllocation(PreferredAllocationRequest) returns (PreferredAllocationResponse) {}

	    // Allocate is called during container creation so that the Device
	    // Plugin can run device specific operations and instruct Kubelet
	    // of the steps to make the Device available in the container
	    rpc Allocate(AllocateRequest) returns (AllocateResponse) {}

	    // PreStartContainer is called, if indicated by Device Plugin during registeration phase,
	    // before each container start. Device plugin can run device specific operations
	    // such as resetting the device before making devices available to the container
	    rpc PreStartContainer(PreStartContainerRequest) returns (PreStartContainerResponse) {}
    }
    ```
* We have to implement <font color=#0000FF>GetDevicePluginOptions()</font>, <font color=#FF0000>GetPreferredAllocation()</font> and <font color=#00BB00>PreStartContainer()</font> to make device plugin options available.
* <font color=#FF0000>GetPreferredAllocation()</font>
    * <font color=#FF0000>GetPreferredAllocation()</font> let kubelet knows, from device plugin point of view, which device(s) is prefered to allocate.
    * This is just a "suggestion", kubelet will determine by itself.
    * <font color=#FF0000>GetPreferredAllocation()</font> needs PreferredAllocationRequest as input arguement.
    ```protobuf=
    // PreferredAllocationRequest is passed via a call to GetPreferredAllocation()
    // at pod admission time. The device plugin should take the list of
    // `available_deviceIDs` and calculate a preferred allocation of size
    // 'allocation_size' from them, making sure to include the set of devices
    // listed in 'must_include_deviceIDs'.
    message PreferredAllocationRequest {
	    repeated ContainerPreferredAllocationRequest container_requests = 1;
    }

    message ContainerPreferredAllocationRequest {
	    // List of available deviceIDs from which to choose a preferred allocation
	    repeated string available_deviceIDs = 1;
	    // List of deviceIDs that must be included in the preferred allocation
	    repeated string must_include_deviceIDs = 2;
	// Number of devices to include in the preferred allocation
	    int32 allocation_size = 3;
    }
    ```
    * available_deviceIDs indicates currently available device(s) from kubelet.
    * must_include_deviceIDs indicates in above devices, which device(s) is mandatory.
    * allocation_size indicates kubelet needs how much of device(s).
    * <font color=#FF0000>GetPreferredAllocation()</font> will also return PreferredAllocationResponse to kubelet.
    ``` protobuf=
    // PreferredAllocationResponse returns a preferred allocation,
    // resulting from a PreferredAllocationRequest.
    message PreferredAllocationResponse {
	    repeated ContainerPreferredAllocationResponse container_responses = 1;
    }

    message ContainerPreferredAllocationResponse {
	    repeated string deviceIDs = 1;
    }
    ```
    * deviceIDs indicates the result which device plugin prefers to allocate.
* <font color=#00BB00>PreStartContainer()</font>
    * kubelet will call <font color=#00BB00>PreStartContainer()</font> to let device plugin knows finally which device(s) is chosen by kubelet, and give a chance to device plugin to do some operation before container creating.
    * <font color=#00BB00>PreStartContainer()</font> will pass an arugement called PreStartContainerRequest.
    ```protbuf=
    // - PreStartContainer is expected to be called before each container start if indicated by plugin during registration phase.
    // - PreStartContainer allows kubelet to pass reinitialized devices to containers.
    // - PreStartContainer allows Device Plugin to run device specific operations on
    //   the Devices requested
    message PreStartContainerRequest {
	    repeated string devices_ids = 1 [(gogoproto.customname) = "DevicesIDs"];
    }
    ```
    * devices_ids indicates which device(s) kubelet will allocate.
* <font color=#0000FF>GetDevicePluginOptions()</font> 
    * <font color=#0000FF>GetDevicePluginOptions()</font> is the function let kubelet knows <font color=#FF0000>GetPreferredAllocation()</font> or/and <font color=#00BB00>PreStartContainer()</font> are available.
    * <font color=#0000FF>GetDevicePluginOptions()</font> will return DevicePluginOptions
    ```protobuf=
    message DevicePluginOptions {
	    // Indicates if PreStartContainer call is required before each container start
	    bool pre_start_required = 1;
	    // Indicates if GetPreferredAllocation is implemented and available for calling
	    bool get_preferred_allocation_available = 2;
    }
    ```
    * Both elements indicate the corresponed option is available or not.
* However, in real environment testing(kubelet v1.23.4), <font color=#0000FF>GetDevicePluginOptions()</font> dosen't be automatically called at all, we have to register these two calls at registeration phase.

## Device Manager Registration Service
* Register request defines the following:
    ```protobuf=
    message RegisterRequest {
	    // Version of the API the Device Plugin was built against
	    string version = 1;
	    // Name of the unix socket the device plugin is listening on
	    // PATH = path.Join(DevicePluginPath, endpoint)
	    string endpoint = 2;
	    // Schedulable resource name. As of now it's expected to be a DNS Label
	    string resource_name = 3;
	    // Options to be communicated with Device Manager
	    DevicePluginOptions options = 4;
    }
    ```
    * DevicePluginOptions is the options indicate <font color=#FF0000>GetPreferredAllocation()</font> or/and <font color=#00BB00>PreStartContainer()</font> is available
    * DevicePluginOptions contains followings:
    ```protobuf=
    message DevicePluginOptions {
	    // Indicates if PreStartContainer call is required before each container start
	    bool pre_start_required = 1;
	    // Indicates if GetPreferredAllocation is implemented and available for calling
	    bool get_preferred_allocation_available = 2;
    }
    ```
## Summary procedure
![](https://i.imgur.com/AiLiI0f.png)
