Try   HackMD

Windows node support

Table of Contents

Summary

There is strong interest in the community for adding support for workloads running on Microsoft Windows. This is non-trivial due to the significant differences in the implementation of Windows from the Linux-based OSes that have so far been supported by Kubernetes.

Motivation

Windows-native workloads still account for a significant portion of the enterprise software space. While containerization technologies emerged first in the UNIX ecosystem, Microsoft has made investments in recent years to enable support for containers in its Windows OS. As users of Windows increasingly turn to containers as the preferred abstraction for running software, the Kubernetes ecosystem stands to benefit by becoming a cross-platform cluster manager.

Goals

  • Enable users to run nodes on Windows servers
  • Document the differences and limitations compared to Linux
  • Test results added to testgrid to prevent regression of functionality

Non-Goals

  • Adding Windows support to all projects in the Kubernetes ecosystem (Cluster Lifecycle, etc)

Proposal

As of 29-11-2018 much of the work for enabling Windows nodes has already been completed. Both kubelet and kube-proxy have been adapted to work on Windows Server, and so the first goal of this KEP is largely already complete.

What works today

1 This list should be available at https://k8s-testgrid.appspot.com/sig-windows but this test setup is not currently working. https://k8s-testgrid.appspot.com/google-windows#windows-prototype is also running against a Windows cluster.

What will work eventually

  • kubectl port-forward hasn't been implemented due to lack of an nsenter equivalent to run a process inside a network namespace.
  • CRIs other than Dockershim: CRI-containerd support is forthcoming

What will never work (without underlying OS changes)

  • Certain Pod functionality
    • Privileged containers
    • Reservations are not enforced by the OS, but overprovisioning could be blocked with --enforce-node-allocatable=pods (pending: tests needed)
  • CSI plugins, which require privileged containers
  • Some parts of the V1 API
  • Overlay networking support in Windows Server 1803 is not fully functional using the win-overlay CNI plugin. Specifically service IPs do not work on Windows nodes. This is currently specific to win-overlay - other CNI plugins (OVS, AzureCNI) work.

Relevant resources/conversations

Risks and Mitigations

Second class support: Kubernetes contributors are likely to be thinking of Linux-based solutions to problems, as Linux remains the primary OS supported. Keeping Windows support working will be an ongoing burden potentially limiting the pace of development.

User experience: Users today will need to use some combination of taints and node selectors in order to keep Linux and Windows workloads separated. In the best case this imposes a burden only on Windows users, but this is still less than ideal.

Graduation Criteria

Implementation History

Other references

Past release proposal for v1.12/13