---
# System prepended metadata

title: Untitled

---

SIG-Windows weekly meeting notes
Meeting Key Info

Meeting Cadence: Weekly Tuesdays at 12:30 PM EST
Meeting Location
https://zoom.us/j/96892680257?pwd=TVNyMzB4VVMwRGZnUkgzT1dnb2szZz09 
Meeting ID: 968 9268 0257
Passcode: 77777
SIG Meeting Calendar: https://calendar.google.com/calendar/embed?src=cgnt364vd8s86hr2phapfjc6uk@group.calendar.google.com&ctz=America/Los_Angeles&pli=1 
SIG-Windows meetings:
https://calendar.google.com/calendar/u/0?cid=dmR2bXVxYjI2Zm83cWI4aGMyOXJsdjY4aThAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ 
SIG-Windows Roadmap/Project Boards
[sig-windows] Issue Tracking (Issues)
https://github.com/orgs/kubernetes/projects/99 (PRs)
Recorded Videos of All Meetings: https://www.youtube.com/playlist?list=PL69nYSiGNLP2OH9InCcNkWNu2bl-gmIU4 
Documentation at https://kubernetes.io/docs/setup/production-environment/windows/
SIG Windows Bi-weekly Backlog Review Meeting
Meeting Cadence: Bi-Weekly(every two weeks) Thursdays at 12:30 PM EST
Meeting Location (passcode 77777):  https://zoom.us/j/94389601840?pwd=MCs2SEJQWG0zUWpBS3Nod0ZNMmVXQT09
Issues · microsoft/Windows-Containers · GitHub
Sig-windows pairing doc: https://hackmd.io/3wPNX3iDT2aU_YkkqXOPpg ← notes ! 
To edit this document please join kubernetes-sig-windows@googlegroups.com

Testing-jobs triage info
https://testgrid.k8s.io/sig-windows-releases 
Notes

Date
Agenda Items
Future
ContainerD work and Hyper-v isolation
Impact from a support perspective of running hyper-v containers when a non-hyper-v hypervisor exists in the stack (public clouds and vmware impact) - cloud providers, can work with Taylor Brown on this are
ContainerD CNI Support - where are we with the major CNI plugins ?
https://github.com/containerd/containerd/pull/4921
Node problem detector discussion
Ideas?
Look for pending reboot actions, possible from updates
Bugcheck reports
06/13/2023
Announcements
1.28 Enhancements Freeze is Jun 15, 2023
New Contributors Welcome
Agenda
Start a new doc?
Aravindh Puthiyaparambil to start a new doc
06/06/2024
Announcements
1.28 Enhancements Freeze is Jun 15, 2023
New Contributors - Welcome
Agenda
[MaRosset] https://kubernetes.slack.com/archives/C0SJ4AFB7/p1685963430263049
https://github.com/kubernetes/kubernetes/issues/118445
[Aravindh, Mateusz] SWDT Windows image creation
https://github.com/prometheus-community/windows_exporter/pull/1218 
https://github.com/kubernetes/kubernetes/pull/116968 
[Aravindh] docs meeting
Start the meeting back up until the big ‘overhaul windows docs’ issue (get link) is closed


05/30/2023
Announcements
New Contributors - Welcome
Agenda
[Failing Test] capz-windows-master · Issue #118300 
[Aravindh, Mateusz] SWDT Windows image creation


05/23/2023
Announcements
New Contributors - Welcome
Agenda
SWDT ~ https://github.com/hashicorp/vagrant/issues/12967 filed
win-svc-proxy updates: clusterIP svcs working in simple scenarios on EKS swapped out win proxy, @daman working on it
Can David / Sravanth answer some Q’s about HNS for everyone


05/16/2023
Announcements
1.28 Enhancements Freeze is Jun 15, 2023
New Contributors - Welcome
Agenda
Windows_exporter Missing metrics from 'container' collector · Issue #1129 · prometheus-community/windows_exporter · GitHub - Tatenda will pick this up
SWDT issue with QEMU and ability to ping the node
https://github.com/ppggff/vagrant-qemu/issues?q=is%3Aissue+is%3Aopen+windows
Daman and Jay working on the Windows server proxy got env on Windows 
EKS. Some active directory issues came up again and again
Need help testing https://github.com/kubernetes-sigs/sig-windows-dev-tools/tree/master-windows-native if anyone wants to help w SWDT , need a windows laptop ! 
How do you get a static IP for a VM in hyper-v?
Use the OS to configure it




05/09/2023
Announcements
1.28 Enhancements Freeze is June 15
New Contributors - Welcome
Agenda
Commit memory update
Sig-windows dev tools discussion
Discussions around using image-builder images and copying over K8s binaries
05/02/2023
Announcements
1.28 Enhancements Freeze is June 15
1.28 Code Freeze is ??
New Contributors - Welcome
Agenda
[Kantesh] Is kubectl for windows being tested using E2E framework?
Except unit tests, are there any other test coverage for kubectl?
#sig-windows thread
1.28 Enhancements
NodeLogViewer
Keep in alpha for 1.28
Kubectl plugin - requirement for beta?
Get e2e tests merged in 1.28
Rootfs file size 
CRI only metrics
CRI stats
PR open for wiring up stats for Windows nodes
Adding Windows support for InPlace Pod Vertical Scaling
Commit memory
Will be discussed at sig-node on the May 9th, 2023 call
Should we update the Windows resource page?
04/25/2023
Announcements
k/k master is open for checkins
New Contributors - Welcome
Mateusz Loskot - Matt says Hi
Tatenda Zifudzi - First meeting joining sig-windows
Agenda
https://github.com/prometheus-community/windows_exporter/issues/1129
Network status not showing up with containerd because we need to query HNS v2 API
Sig-windows-dev-tools on windows
Sometimes works, sometimes it doesn’t. Occasionally running into Windows RM. ⅓ attempts start ⅔ don’t
https://kubernetes.slack.com/archives/C0SJ4AFB7/p1682415223198359 


04/18/2023
CANCELLED FOR KUBECON EU
04/11/2023
Announcements
1.27 release is today 
New Contributors - Welcome
Agenda


[swdt] Thanks to bsankar for getting a vagrant hyperv recipe started…
Static ips on hyper-v? Is this possible?
Secondary nic, internal switch with static ip
Powershell, maybe python
jay asked about automating it. You can automatically set a static IP inside your VM, you could do that through cloud-init (or cloudbase-init for Windows VMs) and a configdrive iso attached to the VM.




Sig-windows channels
How do we make it easier to for new folks to ask question in slack?
[win-svc-proxy] Testing win-svc-proxy on eks 
04/04/2023
Announcements
1.27 expected to release next Tuesday
New Contributors
Agenda


Calico pods not networking in swdt
Daman has win-svc-proxy dev env up 
E2e tests for NodeLogQuery 
https://github.com/kubernetes/kubernetes/pull/117011
emptyDir: https://github.com/microsoft/Windows-Containers/issues/345 
3/28/2023
Announcements
First 1.27 RC cut
New Contributors
Agenda
1.28 work
In place vertical scaling
Rebase fabian’s work
https://github.com/kubernetes/kubernetes/pull/112599 
Cri metrics API
Mansi
Kube-proxy out of tree
[amim] -> Working on getting a few new folks from CNCF
Hyper-v isolation in 1.7
Investigating flakes - https://testgrid.k8s.io/sig-windows-experimental#capz-master-windows-hyperv
Documentation
E2e tests for node log viewer
Issue triaging
Went through backlog and looked at things for 1.28
3/21/2023
Announcements
Test freeze is TODAY 
New Contributors
Agenda
Failing test for release: https://github.com/kubernetes/kubernetes/issues/116782 
Should we revert? Or fix?
Small pr, move forward
Absolute file paths only? Yes
Maybe just add a check to see if host is a file.
No windows specific documentation on this feature
Annual report - please review!
https://github.com/kubernetes/community/pull/7173 
1.27 major themes for SIG
Start thread in slack, need by doc team today
Node log viewer changes - Alpha
Start thread in sig-windows
Also request to create 45s Sig update for kubecon
Up in by end of week.
Call for help - volunteer opportunity
KPNG direction?
Sig-network kep - not finished
Keep moving forward with windows out of tree, others are doing this (antrea, cillium, tiegera)
Get it working and CI, 
Concerns:
Sig-windows maintaining generic kpng components
Last kep would have solved it but it is closed.
1.28 timeline?
Make a decision
Does in-tree get deprecated?
Need at least one go to place
SWDT qemu PR ready to test https://github.com/kubernetes-sigs/sig-windows-dev-tools/pull/238 
3/14/2023
Announcements
Code Freeze - 17:00 PDT Tuesday 14 March
Containerd 1.7.0 released yesterday
New Contributors - Welcome
Agenda
NodeLogQuery feature
All comments addressed
Jordan wants more people to review in depth
Worried about escapes due to how get-winevent is constructed
Will file an exception  for 1.27 if this doesn’t merge by code-freeze
Pramita (qemu/SWDT) 
3/7/2023
Announcements
Code Freeze - 17:00 PDT Tuesday 14 March
Annual Report due soon
Stop using k8s.gcr.io please!
New Contributors - Welcome
Agenda
Metrics collection PR https://github.com/kubernetes/kubernetes/issues/114928 
Merge and label as a breaking change?
Hyperv isolated containers
Available in containerd v1.7
https://testgrid.k8s.io/sig-windows-experimental#capz-master-windows-hyperv
proxy- Whered  https://testgrid.k8s.io/sig-windows-master-release#capz-master-windows-service-proxy go :) .. nvm, its here https://testgrid.k8s.io/sig-windows-experimental#capz-master-windows-service-proxy 
Dev-tools: need QEMU, TPMCRI only status , windows driver help if anyone wants to join us (swtpm….) https://github.com/stefanberger/swtpm/wiki (nvm answer = DISABLE SecureBoot) 
Cri only status kubelet changes. Containerd changes merged should be in 1.7.
Need equivalent changes, to use windows pod stats https://github.com/kubernetes/kubernetes/pull/103095 
2/28/2023
Announcements.
Code Freeze - 17:00 PDT Tuesday 14 March
Welcome Aravindh as new SIG-Windows co-chair
New Contributors - Welcome
Agenda
[June] RKM demo (https://src.redpoint.games/redpointgames/rkm)
E2e Flakes https://github.com/orgs/kubernetes/projects/82/views/2 
2/21/2023
Announcements
Code Freeze - 17:00 PDT Tuesday 14 March
New Contributors - Welcome
Agenda
[MaRosset] - Periodic test pass running KPNG set up https://testgrid.k8s.io/sig-windows-master-release#capz-master-windows-service-proxy
Good opportunity to contribute
Hard to get started with windows containers
Maybe some guides?
Ross has something
Operational readiness for Windows
https://github.com/kubernetes-sigs/windows-operational-readiness 
A way to provide “conformance” for windows
What the windows nodes support 
Another opportunity to contribute
Sig windows dev tools, split experience, proposal ⇒ 
Windows amd64 
→ VMWare workstation (vagrant) ← TODO
Mac os 
m+ → cloud account  (vagrant) or QEMU manual 
amd64 → VMWare Fusion / Workstation (vagrant) 
2/14/2023
Announcements
Code Freeze - 17:00 PDT Tuesday 14th March
New Contributors - Welcome
Agenda
[marosset] - SIG leadership changes
Aravind to become co-chair
Mark to also become TL
[marosset] - buildkit changes and issues for Windows HostProcess containers
https://github.com/containerd/containerd/issues/8070
https://github.com/containerd/containerd/pull/8101 
If using buildx v0.10 or newer you can specify –provenance=none on the docker buildx build call
[Aravindh] - outcome of sig-arch node log discussion
SIG-Architecture Agenda and Meeting Notes
Default the feature to off, don’t use kube-apiserver, just use kubelet
Using this is a first class API is incorrect
API contained within kubelet (node proxy / metrics)
2/7/2023
Announcements
Enhancements Freeze: Thursday, Feb 9th
New Contributors - Welcome
Agenda
1.27 major themes: https://kubernetes.slack.com/archives/C0SJ4AFB7/p1675703546916619
Newcomers [bart…] 
windows @ scale and linux containers, e2es…
Calico host-process 
a/b test against antrea via variables.yaml 
Op readiness anyone want to own next phase , 
maybe ross ?
BootId property for the node? What’s that ?
 nodeInfo:
    architecture: amd64
    bootID: a73baaf1-cf07-420b-b57f-6ea4e8c9d8dc
Updated https://github.com/kubernetes-sigs/sig-windows-samples/blob/master/PAIRING.md with details
Pramita + QEMU → aravindh 
QEMU , need help from someone else, amd64 launch failing…. 
Aravindh , node logs, … sig-arch, come next week for the exciting  (sig archhhhh)
1/31/2023
Announcements
Enhancements Freeze: Thursday, Feb 9th
New Contributors - Welcome
Agenda
[Aravindh] Any update on Windows Container Unconference?
Going to discuss node service log viewer enhancement at SIG node after this meeting
[MaRosset] - windows-service-proxy updates
Container image at https://console.cloud.google.com/gcr/images/k8s-staging-win-svc-proxy/global/kpng
OCI provider merged: https://github.com/kubernetes-sigs/image-builder/pull/1051 
Pause container change in hcsshim: https://github.com/microsoft/hcsshim/pull/1615/files 
Sig-windows dev tools updates from David, upgrading to use HPC
Pramita is getting qemu support working


1/24/2023
Announcements
Enhancements Freeze: Thursday, Feb 9th
New Contributors - Welcome
Agenda
https://github.com/kubernetes/kubernetes/issues/114928
[Aravindh] Jordan’s comment on node log query feature → need to avoid spinning further and get sig-node feedback


1/17/2023
Announcements
Enhancements Freeze: Thursday, Feb 9th
New Contributors - Welcome
Agenda
[MaRosset] - Created an issue for ‘Specify scratch volume size for WIndows containers’ - http://features.k8s.io/3746 
1/10/2023
Announcements
1.27 has officially started
Enhancements Freeze: Thursday, Feb 9th
New Contributors - Welcome
Agenda
https://github.com/kubernetes/kubernetes/pull/114423
KEP for rootfs size for Windows containers??
https://github.com/kubernetes/kubernetes/pull/109702
Thin provisioned, but if all consumed something bad could happen due to space being consumed
Ephemeral storage limits (with eviction) on kubelet might help?
Generally a Yes let’s do a kep
[jay] https://github.com/kubernetes-sigs/windows-service-proxy is live, code mostly structures what were envisioning for windows owned proxy, see https://github.com/kubernetes-sigs/windows-service-proxy/blob/main/cmd/k2s.go for entrypoint  
 https://docs.google.com/document/d/1OdcY8voTjHT4Yk30p_Tb2ilouhmNARRcNafUVKy0Zw8/edit#heading=h.n9gkuwwt8x98 ← “proposal”
01/03/2023
Announcements
1.27 release cycle starts this week
Release Cycle begins: Monday, January 9th
Enhancements Freeze: Thursday, February 9th
Code Freeze: Tuesday, March 14th
Release Day: Tuesday, April 11th
Release team shadow applications close today 
New Contributors - Welcome
Agenda
[James] https://github.com/kubernetes-sigs/sig-windows-tools/pull/271
Docs for joining Windows nodes to a cluster?
[Aravindh] KEP 2258: add node log query update
WIP: Rewire through API server
Revert "Add e2e tests for node log viewer" by fabi200123 · Pull Request #114636 · kubernetes/kubernetes
12/13/2022
Announcements
1.26 has been released!
1.22 reached EoL
HostProcess containers Blog goes live 12/13
https://kubernetes.io/blog/2022/12/13/windows-host-process-containers-ga/ 
Canceling Sig-windows meeting  December 20/27
Enhancements Freeze tentative for 1.27 -  Feb 9 2023
New Contributors - Welcome
Agenda
Docs pairing meeting? Find a new time?
Refocus, start a new meeting next year.
Windows kube-proxy (Jay/Mark/David)	 https://github.com/K8sbykeshed/windows-service-proxy 
New repo in kubernetes sigs being created
12/06/2022
Announcements
K8s 1.26 release delayed until Dec 8th, 2022
New Contributors - Welcome
Agenda
Ping.exe removal
https://github.com/microsoft/hcsshim/issues/1576
Please comment on issue
https://github.com/kubernetes/website/pull/37370
Kubeadm join instructions for Windows
https://github.com/kubernetes/website/pull/38276#discussion_r1039940892 
https://github.com/kubernetes-sigs/sig-windows-tools/pull/239 
Going to split and merge calico. Then follow up with flannel
Does anyone want node-problem-detector to run as a daemonSet w/ HostProcess containers?
Kube proxy out of tree doc https://docs.google.com/document/d/1OdcY8voTjHT4Yk30p_Tb2ilouhmNARRcNafUVKy0Zw8/edit#heading=h.7lll3xzbrz4n 
Review, maybe specific meeting to review.  Follow up in slack
Sig-win-dev-tools and m2/m1 macs ? anyone try it ?
What arch is the Windows VM that is getting created?
TIME TO MOVE TO QEMU
11/29/2022
Announcements
Blog posts for 1.26 should be ready for review today
Maintainer talk for Kubecon EU is open
New contributor for OpReadiness
New Contributors - Welcome
Agenda
HPC blog post should be ready for review by today
https://github.com/kubernetes/website/pull/37370
Include user accoutn stuff from https://github.com/kubernetes/website/pull/37340
Windows KPNG options
Keep gRPC functionality and maintain everything in KPNG
Make sig-windows kubeproxy repo and have everything live there
More
Jay to make a new doc listing options for review at future community meeting
11/22/2022
CANCELLED
11/15/2022
Announcements
11/22/2022 Meeting will be canceled
Doc PRs need to be ready for review by 11/15/2022
KubeCon recordings are up https://www.youtube.com/@cncf :
Windows HostProcess Containers For Configuration And Beyond - James Sturtevant & Mark Rossetti - YouTube
What's New With SIG Windows - Mark Rossetti & James Sturtevant, Jay Vyas, Dimitrie Mititelu - YouTube
New Contributors - Welcome
Agenda




11/8/2022
Announcements
Code Freeze is TONIGHT
New Contributors - Welcome
Agenda
KEP 2258: add node log viewer
KPNG - propose kube-proxy library impl and make that a separate repo. Then people can vendor that to make custom kube-proxy.
Solution proposed for linux but would works well for Windows
Most important part of in-tree proxy is iptables which is deprecated now
(if time , need  sig-windows kube proxy, out of tree)  ? 
11/1/2022
Announcements
Feature Blog Freeze 11/2/2022
Code Freeze 11/9/2022
Docs PR placeholders Freeze 11/10/2022
New Contributors - Welcome
Agenda
OS update for HNS policy syncing taking a long time (https://github.com/kubernetes/kubernetes/pull/109124)
On by default for Windows Server 2022 w/ 10C
Requires some extra steps to enable on Windows Server 2019 w/ Oct 10C
Powershell : reg add HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Policies\Microsoft\FeatureManagement\Overrides /v 3444861581 /t REG_DWORD /d 1 /f
gpupdate /force
schtasks /run /tn "Microsoft\Windows\Flighting\FeatureConfig\ReconcileFeatures"
Reboot the machine (Restart-Computer or shutdown.exe)
KPNG updates
Jay, Amim, Riccardo spending a week hacking on KPNG
New goal is to take KPNG, break it up, and move it in tree (under a staging directory)


10/25/2022
CANCELED FOR KUBECON
10/18/2022


KubeCon
Contributor Summit (schedule)
SIG meet and greet - contributors only (lunch break)
SIG Windows Development Environment (3:20pm)
Contributor Celebration (6PM @ Deluxx Fluxx)
Main Event (schedule)
HostProcessContainers talk (Wed 4:30pm)
OpReadiness (Thurs 3:25pm)
SIG-Windows maintainers talk (Thursday 5:25pm)
Lessons learned for scheduling 20 mil Windows Containers (Friday 11:55am)
SIG meet and greet (Friday 12:30pm)


10/11/2022
Announcements
1.26 code freeze - Tuesdays Nov 8th (PDT)
New Contributors - Welcome
Agenda
Fix for kube-proxy regression discussed last week back ported to v1.24 and v1.25 (https://github.com/kubernetes/kubernetes/issues/112836)
1.26 enhancements
Node service log viewer (alpha)
HostProcess containers (stable)
Windows host network support (alpha)
CRI based stats (alpha) 
Other work for v1.26
Get unit tests running for Windows
Windows file system permissions (https://github.com/kubernetes/kubernetes/pull/104660 and https://github.com/kubernetes/kubernetes/pull/110921)


10/04/2022
Announcements
Enhancements freeze is this Thursday
New contributors -Welcome
Agenda
ebpf demo (James)
Question: testing and UDS with Windows HostProcess pods
Kube-proxy regression in v1.24: https://github.com/kubernetes/kubernetes/issues/112836
9/27/2022
Announcements
KEP freeze is Next thursday


New Contributors - Welcome
Agenda
Call for 1.26 blog posts
Node Service log viewer?
Yes - given PR merges
HPC - Yes
KPNG?<
May next release
Operational readiness - Xinqi?
Retriable Jobs KEP -  https://github.com/kubernetes/enhancements/pull/3463
Currently planning on skipping windows support
Mark took a look, only OOMKill and node-shutdown related issues look like they will be skipped
New HPA tests are failing - it looks like resource-consumer isn’t causing memory usage to increase? Started a thread in slack
Using testlimit.exe 
If can’t allocate all of it, then doesn’t allocate anything
Possible solution: Consume memory in smaller chunks up to limit that causes scale (using argument in url)
CPU still not sure, some consumption is happen, scaling to 2 but not all the way to 3
9/20/2022
Announcements
KubeCon NA contributor summit
V0.1.7 release of https://github.com/jsturtevant/windows-debug
Includes network/hns debugging utils and HCS event provider filter files
New Contributors - Welcome
Agenda
https://github.com/kubernetes/kubernetes/pull/112133 - Removing kube-proxy userspace modes FYI
[fabian] - https://github.com/kubernetes/kubernetes/pull/112599 -
Added the Windows support for In-Place Pod Vertical Scaling
[daschott] - Demo wcnspect tool for network diagnostics
https://github.com/microsoft/wcnspect   
Cri-only stats: https://github.com/kubernetes/enhancements/pull/3439 
9/13/2022
Announcements
V1.26 release schedule announced
Production Readiness Review Freeze — Thursday 29th September 2022
Enhancements Freeze — 18:00 PDT on Thursday 6th October 2022
Code Freeze — 17:00 PDT on Tuesday 8th November 2022


New Contributors - Welcome
Agenda
Should SIG-Windows submit anything for the contributor summit (or KPNG?)
Using GH boards for enhancement tracking
https://github.com/orgs/kubernetes/projects/98 
Enhancements
#3503 Host network support for Windows pods
Support for Windows privileged containers #1981 
Use kubectl to view logs of system services on nodes #2258
[KEP for WinDSR / WinOverlay]
https://github.com/kubernetes/kubernetes/pull/109702 ??
Cri only Pod Stats: https://github.com/kubernetes/enhancements/pull/3439 
9/6/2022
sort of Cancelled
Sig-windows-dev-tools — VT-X not enabled , asked bobby in github-management 
Option: Azure cloud possibly cc @lachie 
8/30/2022
Announcements
KubeCon NA contributor summit
New Contributors - Welcome
Agenda
[andrew] -  9 worker node max pod-per-node updates
https://github.com/kubernetes/community/blob/master/sig-scalability/slos/slos.md 
https://github.com/kubernetes/kubernetes/issues/112111 (wrong memory usage)
8/23/2022
Announcements
1.25 should be released today!
New Contributors - Welcome
Agenda
[mark] - New sig-windows project board https://github.com/orgs/kubernetes/projects/82
[aravindh] Discuss feature: add node log viewer 
8/16/2022
Announcements
1.25 release Aug 23 2022
KubeCon NA 2022 contributor summit registration open
https://www.kubernetes.dev/events/2022/kcsna/registration/ 
This base image for host-process-containers released
https://github.com/microsoft/windows-host-process-containers-base-image 
New Contributors - Welcome
Agenda
Minimum configuration for a Windows node
Windows Container Requirements | Microsoft Docs
8/9/2022
Announcements
V1.25 test freeze is today
KubeCon NA schedule is available
New Contributors - Welcome
Agenda
Go over state-of-the art for CAPI, vs windows-machine-operator, and the original CAPI support in windows CAEP
https://github.com/openshift/windows-machine-config-operator 
Windows operational readiness updates
New repo - https://github.com/kubernetes-sigs/windows-operational-readiness
[Amim] Discuss running tests in a prow job? https://github.com/kubernetes/test-infra/issues/27089 
Fabian windows docs : kubernetes-sigs/sig-windows-tools/236 
8/2/2022
Announcements
New Contributors - Welcome
Agenda
Node Service log viewer - moving to v1.26, will continue to work on implementation instead of taking a break. 
Major themes from SIG-windows for 1.25 release notes
Users are confused on how to add Windows nodes to clusters
Discussions on slack
https://github.com/kubernetes-sigs/sig-windows-tools/issues/217 
Hugo tag to warn users about third party solutions? - use this until external docs have instructions for Windows
KPNG - windows merging ETA this week!
7/26/2022
Announcements
Code freeze is 1 week away
New Contributors - Welcome
Agenda
Enhancement status check
Node service log viewer
At risk 
Pod OS field
On tracks
https://github.com/kubernetes/kubernetes/pull/105919 still open but actively being worked on
Operational readiness
Not tracked in k/k
https://github.com/k8sbykeshed/op-readiness/
Move to k-sigs?
GMSA, networking done
Storage being worked on
CRI based stats (node)
https://github.com/kubernetes/enhancements/pull/3439 
Discussing what fields should be in protobuf objects
PRs needing reviews for 1.25
https://github.com/kubernetes/kubernetes/issues/111438 
https://github.com/kubernetes/kubernetes/pull/110566
Update title and add to v1.25 milestone? 
https://github.com/kubernetes/kubernetes/pull/104660
Discuss more on slack?
Concerned about maintenance 




7/19/2022
Announcements
Removing hns v1 from kubeproxy: https://github.com/kubernetes/kubernetes/pull/110957 
Code freeze is Aug 2nd 2022 (2 weeks away)!
New Contributors - Welcome
Agenda
[demo jstur] Kubectl plugin for windows node access using HostProcess containers: https://github.com/jsturtevant/windows-debug 
[jsturtevant] cri pod sandbox stats for windows: https://github.com/kubernetes/kubernetes/pull/110754 
[marosset] hostprocess containers updates
[demo Xinqi] operational readiness demo (core networking part)
https://github.com/K8sbykeshed/op-readiness
[ClaudiuBelu] Windows Pod DNS Policy
7/12/2022
Announcements
K8s v1.21 is out of support
New Contributors - welcome
Agenda
https://github.com/kubernetes/kubernetes/pull/109702 
What pod spec field should we use for setting the ‘writible container layer’ for window spods?
Ephemeral storage requests? limits?
Kube-proxy caching backports to 1.22 / 1.23 not stable?
Does kube-proxy caching backports make dockershim less stable?
https://testgrid.k8s.io/sig-windows-1.22-release#aks-engine-windows-dockershim-1.22 (w/ proxy caching) vs https://testgrid.k8s.io/sig-windows-1.22-release#aks-engine-windows-dockershim-1.22-46d58cc17 (w/o proxy caching) 
https://testgrid.k8s.io/sig-windows-1.23-release#aks-engine-windows-dockershim-1.23 (w / proxy caching) vs https://testgrid.k8s.io/sig-windows-1.23-release#aks-engine-windows-dockershim-1.23-938a3203c (w/o proxy caching)
[jay]hypothesis: kube-proxy now using slightly more CPU or memory, hence longer pod startup in highly constrained environments
07/05/2022
Announcements
Perf-dash - http://perf-dash.k8s.io/#/?jobname=soak-tests-capz-windows-2019&metriccategoryname=E2E&metricname=CPUUsage&Process=MsMpEng 
New Contributors - welcome
Agenda
Dev tools - cloud init
Discussion on difference between sig-windows-dev-tools and kubeinit


6/28/2022
Announcements
New Contributors - welcome
Agenda
Soak/Perf tests demo
Kpng
Open PR
New KEP
Separate repos for each backend
https://github.com/kubernetes/enhancements/pull/2094 
Kubeadm and sig windows tools
Cluster sizing
Anything like https://kubernetes.io/docs/setup/best-practices/cluster-large/ ?
6/21/2022
Announcements
HPC staying in beta for 1.25: https://github.com/kubernetes/enhancements/pull/3311 
Enhancement freeze this week
New Contributors - Welcome
Agenda
1.22 and 1.23 release delays and next steps https://groups.google.com/u/2/a/kubernetes.io/g/dev/c/os8nyAVTL7A 
Enhancement check in
PodOS https://github.com/kubernetes/enhancements/issues/2802
Kubectl logs https://github.com/kubernetes/enhancements/issues/2258 
Operational readiness
https://github.com/kubernetes/enhancements/issues/2578 
Networking test - new or taint?
Draft: https://github.com/kubernetes/kubernetes/pull/110614 
https://github.com/Azure/AKS/issues/2899 - kube-proxy bug?
KPNG [[dimistriy m update]]
https://github.com/papagalu/kpng/tree/test_snapshot ← new prototype here ! working w sig-win-dev-tools PS C:\Users\vagrant\kpng\backends\windows\kernelspace> New-HNSNetwork  -Type Overlay  -AddressPrefix "100.244.0.0/16"  -Gateway "100.244.1.1"  -Name "External"  -SubnetPolicies @(@{Type = "VSID"; VSID = 9999; })

-AdapterName $vxlanAdapter -Verbose 


6/14/2022
Announcements
Enhancements freeze moved to Jun 23, 2022
Mark on PTO next 2 weeks (returning July 5th)
New contributors - Welcome
Agenda 
https://github.com/kubernetes/kubernetes/pull/110379 (base image for pause container) (carry from last week)
https://github.com/kubernetes/enhancements/issues/995 (carry from last week)
Kube-proxy container image for Window (carry from last week)
How/where to build kube-proxy image for ci/rolling k8s builds.
Making generic kube-proxy image proposal: Windows Generic Kube-proxy 
2022 Azure image issue when getting hcsShim supported features:
https://kubernetes.slack.com/archives/C0SJ4AFB7/p1655217707728239


6/7/2022
Announcements
1.25 enhancement freeze - Thursday June 16th
Docs restructuring PR merged - https://github.com/kubernetes/website/issues/31428 
New Contributors - Welcome
Agenda
ARM toolchain info
https://blogs.windows.com/windowsdeveloper/2022/05/24/create-next-generation-experiences-at-scale-with-windows/ 
https://github.com/kubernetes/kubernetes/pull/110379 (base image for pause container)
https://github.com/kubernetes/enhancements/issues/995 
Kube-proxy container image for Window
How/where to build kube-proxy image for ci/rolling k8s builds.
KPNg-windows syncup (amim, dimitri,...)
5/31/2022
Cancelled - no agenda
5/24/2022
Announcements
1.25 enhancement freeze - Thursday June 16th
New Contributors - Welcome
Agenda
KubeCon reports
https://www.youtube.com/c/cloudnativefdn/videos 
Claudiu met with folks interested in running Windows
New CNI work
Service mesh
Calico folks are interested in bugs
Forza uses Windows containers in K8s
https://customers.microsoft.com/en-us/story/1498781140435260527-forza-horizon-5-crosses-finish-line-fueled-by-azure-kubernetes-service 
SIG-Windows talks - might need to follow up on some questions
CI-signal SIG interested in collaborating
Suggestion to list tests running in parallel w/ other tests
New work on container identities, currently only docker is supported, containerd is on roadmap
Link: https://sched.co/yttL
Brandon
Met with a lot of customers
Interested in performance, image size, image pull times
HostProcessContaienrs
Lots of interest, esp w/ log monitoring
5/17/2022
Canceled for KubeCon EU
5/10/2022
Announcements
Zoom account recovered, will upload past videos soon
1.25 enhancement freeze - Friday June 17th
New Contributors - Welcome
Agenda
Windows docs update PR is up - please review
https://github.com/kubernetes/website/pull/33582 
Windows Server, ver 20H2 support
End of support scheduled for Aug 2022
Current tests run on aks-engine and we are not supporting aks-engine for 1.25 branches
https://github.com/kubernetes/sig-release/tree/master/releases/release-1.25
https://docs.microsoft.com/en-us/windows-server/get-started/windows-server-release-info 
New pull request job for main branch:
https://github.com/kubernetes/test-infra/pull/26238 
/test pull-kubernetes-e2e-capz-windows-containerd
Containerd and hyper-v features
Not required to run but needed for some features, can turn off after
https://github.com/microsoft/Windows-Containers/issues/80 
https://github.com/kubernetes-sigs/image-builder/blob/fdf8a50a3a691b8f94b5b46f349ad0815cbe47fb/images/capi/ansible/windows/roles/systemprep/tasks/main.yml#L139-L166
1.25 planning
Host-process containers to stable (Mark, Danny)
PodOS to stable (Ravi)(https://github.com/kubernetes/enhancements/pull/3303/files)
Pod security updates needed for kube-apiserver +/- version support
Graduation criteria is use-cases in the wild
Operational readiness (Amim, Xinqi)
perf/soak tests (Marius)
HyperV isolated container support in containerd (Danny, dcantah )
https://github.com/containerd/containerd/issues/6862 
Kubeadm for Windows / kube-proxy image (James?, Claudiu?)
Kubeadm for Windows works today but is difficult to configure and lacks documentation.
Msft / kpng handoff hangout (jay, matt, daschott…)




5/3/2022
Announcements
1.24 release ETA 5pm PST today
New Contributors - Welcome
Agenda
Issues with Zoom account - recording uploads may will be delayed
Configuring Windows nodes (pre-requisites - containers features etc, how to install, componentes).  Where should docs on this live?  What should be included
Linux docs on k8s.io used to have specific flannel config details but this was removed due to third party content policies for K8s website
For Windows - Is this information used? Where should it live? What should be included?
Information is useful (Jamie)
Have k8s.io link to a K-sigs repo with Windows docs?
Concerns about versioning
hostProcess folder on sig-windows-tools (planning to add Antrea), ongoing discussions to move to the official repo instead.
4/26/2022
Announcements
1.24 still expected to release May 3rd
New Contributors - Welcome
Agenda
[mark/danny] Demo prototyped host process container volume mount behavior
[daschott] Server 2022 KB released fixes networking issues 
4/19/2022
Announcements
1.24 release delayed until May 3rd
https://github.com/containernetworking/plugins/pull/725 merged (v2 API support for win-overlay CNI)
New Contributors - Welcome
Dimitrie Mititelu - CloudBase - Please connect with Jay


Agenda
[marosset] Cleaning up ‘perma-failing’ test passes https://github.com/kubernetes/kubernetes/issues/109521
https://storage.googleapis.com/k8s-metrics/failures-latest.json
Claudiu can take a look and skip some tests
Some clusters might need to do something similar to https://github.com/kubernetes-sigs/windows-testing/pull/312 
 
[marosset] Eviction on Windows
	TODO Mark: add details
Update operational readiness updates related to grace pod termination / eviction
Windows Server 2022 KB5012637 - releasing today (ETA - 17:00 PST)
4/12/2022
Announcements
1.24.0-rc.0 coming soon
New Contributors - Welcome
Agenda
[arvind] feature: add node log viewer will not make 1.24 and will be targeted for 1.25
Shape of the API has to change
https://github.com/kubernetes/kubernetes/issues/109162 will merge in 1.24.1 (hopefully)
[jamie]Update on GMSA Chart
Most CI is passing - need to work out a go-mod issue
[j] Should we have an option to create the New-NetFirewallRule/New-HNSNetwork in kube-proxy kernelspace (i.e. in kpng) ? 
Calico sets up rule that routes traffic on localhost through its network temporarily until everything gets set up through internal service endpoint to get felix info
Once felix info gets pulled down the rest of the network gets set up
This proposal may make above logic / workarounds not needed 
This would be similar to IPvS behaviors
Do we have enough information to create HNSnetwork at this time? Pod subnet?
What about l2bridge
New configuration interface for kube-proxy (KNI?)
Could simplify some start-up sequences
https://github.com/kubernetes/kubernetes/issues/109438 filed -> https://jayunit100.blogspot.com/2022/03/my-dev-workflow-for-kpng-windows.html 
4/5/2022
Announcements
1.24 release updates
Test freeze tonight
Docs for 1.24 features should be ready to review/merge
New Contributors - Welcome
Agenda
Improvements to winkernel kube-proxy https://github.com/kubernetes/kubernetes/pull/109124 
https://kubernetes.slack.com/archives/C0SJ4AFB7/p1649175162077849?thread_ts=1649175001.560109&cid=C0SJ4AFB7
New test failure after https://github.com/kubernetes/kubernetes/pull/108590/files merged.
Failing on CAPZ (overlay + calico) but passing on overlay + flannel configuration, also passing on l2bridge
Kubernetes e2e suite.[sig-network] Networking Granular Checks: Pods should function for node-pod communication: udp [NodeConformance] 
Plan: exclude tests in CAPZ master jobs and address in calico
https://github.com/flannel-io/flannel/pull/1096 flannel adds a host route policy - do we need this for calico (or other CNIs)?
Is in calico (from 3.20)
Does Antrea pass?
jay/Amim to follow up
https://prow.k8s.io/view/gs/kubernetes-jenkins/logs/ci-kubernetes-e2e-capz-master-containerd-windows/1511351587174354944 
3/29/2022
Announcements
Code freeze - TONIGHT
[daschott] KB5011551 released to support >64 backends per Kubernetes service (up to 1024)
Addresses issue #98680 and #93
Available in 3C now - 4B in April
New Contributors - Welcome
1.24 PRs 
feature: add node log viewer
Exception needed
CLI approved
Node / api review required
Agenda
podOs - merged
Beta apis are off by default in 1.24? Need to follow up
TODO - follow up next meeting
[marosset/dcantah] HostProcess container volume mount updates
Addressing pain point in alpha/beta. Current approach:
Payload on Host dir on under /c/guid
Env that points to it
Incluster config doesn’t work without changes to goclient
New approach:
Union of host OS and container file system
Payload of container is in special folder but only visible to the container
Uses new windows apis not in 2019, trying to backport but may not be able to
Need feedback here


3/22/2022
Announcements
Code freeze next week
Gmsa keyvault ccg plugin open sourced: https://techcommunity.microsoft.com/t5/containers/new-updates-to-group-managed-service-accounts-gmsa-1-3-0-on/ba-p/3260600 
https://github.com/kubernetes-sigs/image-builder/pull/835 
New Contributors - Welcome


Agenda
[jstur] CAPI and gmsa tests: https://github.com/kubernetes/kubernetes/issues/108816 
PodOs field
https://github.com/kubernetes/kubernetes/pull/107859
Node Service Log Viewer
Enabling more conformance tests for windows
https://github.com/kubernetes/kubernetes/pull/108590
https://github.com/kubernetes/kubernetes/pull/108592 
https://github.com/kubernetes/kubernetes/pull/106631
Docs updates
Docs teams main focus is dockershim depreciation
 Plan is to keep prs open to target 1.24 branch, once ready merge after 1.24 goes live. Should be shortly after 1.24 goes live
New networking sub group
Still working on timing
Looking for another active member  
3/15/2022
Announcements
https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/   
New Contributors
Christian Glombek
Agenda
[marosset] Run kubelet in a job object in windows #105077
[cglombek] Request for Comments/Reviews on https://github.com/kubernetes/kubernetes/pull/96120 (feature: add node log viewer)
Demo: https://www.youtube.com/watch?v=yypRwJRj0e4&list=PL69nYSiGNLP2OH9InCcNkWNu2bl-gmIU4&index=9 by Aravindh 
[marosset] https://github.com/containerd/containerd/pull/6618 / https://github.com/kubernetes/kubernetes/issues/97739 / https://github.com/containerd/containerd/issues/4878 
3/08/2022
Announcements
1.24 Code Freeze - March 29th (PST)
2021 SIG annual report available
Agenda
Discussion of https://kubernetes.slack.com/archives/C0SJ4AFB7/p1645789297811109 
Timeline of fix - tentative date of April 19th.  (2022 4C)
Two issues
Loadbalancer service vip not working - pod 2 service
Work around was local traffic policy (for l2bridge)
Pod to pod traffic failing  (l2bridge)
Workaround disable dsr
Follow: https://github.com/microsoft/Windows-Containers/issues/204 
KPNG kube-proxy kernel space “kind of” working w/ Windows
Reach out to Jay on slack if you are interested in contributing
https://github.com/kubernetes-sigs/kpng/pull/217 
[marosset] - Demo of azuredisk csi stuff running in host-process containers
https://github.com/kubernetes-sigs/azuredisk-csi-driver/pull/1201/files 
3/01/2022
Announcements
Windows_exporter image is now available: https://github.com/prometheus-community/windows_exporter/pkgs/container/windows-exporter 
Agenda
-with-containers images for WS2022?
Not available on Azure due to Docker EE transitioning to Miratis.
Currently no equivalent for containerd based images
[jstur] Walk through of windows e2e tests with capz.
https://testgrid.k8s.io/sig-windows-signal#capz-windows-containerd-master 
test-infra job configs  
https://capz.sigs.k8s.io/developers/development.html#conformance-testing 
ci-conformance.sh (entrypoint for PROW jobs)
https://github.com/kubernetes-sigs/sig-windows-tools/tree/master/hostprocess/eventflow-logger


[marosset] - Status of Overhaul Windows documentation #31428
[Xinqi] present gMSA on Operational Readiness (10 min)
2/22/2022
Announcements
[amim/jay/xinqi] prototype of op readiness en route will demo in 2 wks w gmsa/networking results, then storage after that…. 
 GMSA webhook image now being published to k8s-staging registry (instead of sigwindowstools dockerhub) - gcr.io/k8s-staging-gmsa-webhook/k8s-gmsa-webhook:latest
Sig-windows-signal test passes will use CAPZ for 1.24+ 
Agenda
Moved 1.24 jobs to capz: https://github.com/kubernetes/test-infra/pull/25152/files 
Node-log-viewer PR will be open for reviews shortly
PodOS field:
Update to come on slack
2/15/2022
Announcements
1.20 is going out of service after this last set of patches
1.24 code freeze in a few months
Prometheus Windows exporter support merged - https://github.com/prometheus-community/windows_exporter/pull/864 
Next release of GMSA webhook will have a configurable port so you can run as a non-root user
Windows defender performance: https://github.com/kubernetes/kubernetes/pull/98705


Agenda
Meeting to discuss containerd CNI support - discussed in slack https://kubernetes.slack.com/archives/C0SJ4AFB7/p1644449792790369?thread_ts=1644367825.278619&cid=C0SJ4AFB7 
Demo: Prometheus Windows exporter
WS2022 docs updates - need issue in microsoft/windows-contianers to link to
2/8/2022
Announcements
1.24 code freeze is March 30, 2022
Moving from aks-engine to Cluster api for Azure in 1.24: https://groups.google.com/g/kubernetes-sig-release/c/NZdggjRu7-4 
Agenda
https://github.com/containerd/containerd/issues/4851 cni windows question - when will we fix this ? 
Windows CNIs don’t use CNI check today
Nothing should be blocking this from working
Will CNI check call help resolve issue?
Race in Antrea and calico to work with containerd
OVS specific?
Ping Jay in sig-windows for more info/questions
https://github.com/kubernetes/website/pull/31457 - restructuring windows docs
Please review 
Helm charts for GMSA work coming https://github.com/kubernetes-sigs/windows-gmsa/pull/55 


2/1/2022
Announcements
WS2022 support - What is outstanding?
2 NodePort tests failing consistently
Probably requires an OS fix
L2Bridge networking issues (related to node port issues)
TODO (Jing/Muzz/Mark) open issue in Microsoft/Windows-Containers for tracking
TODO (Mark) Update upstream docs to point to above TBD issue
1b patch issues
FeaturesOnDemand (ssh server) install issues on WS2022 (resolved Jan 30, 2022)
Few other misc issues
[Jamie] GMSA 
Helm chart coming soon
Cert manager support
Gmsa credentials can be included in deployment
[Weijuan] Interest check in another Windows containers unconference
Similar to WSL conference
Criag Peters set one up ~2 years
2 thumbs up from Aravindh
https://cloudblogs.microsoft.com/opensource/2019/07/16/first-windows-containers-kubernetes-unconference-2019/ 
Customers showing interest
End-to-end story / app modernization
Maybe 2 tracks/conferences?
One for developers/kubernetes members
One for customers / ops / etc
1/25/2022
Announcements
GMSA helm chart for deployments coming soon
[Amim] Demo: KPNG and Windows Userspace backend ~ 10 min
KPNG Windows Userspace | knabben (thefind.live)
[jstur] WS2022 private image gcr.io/authenticated-image-pulling/windows-nanoserver:v1
https://app.slack.com/client/T09NY5SBT/C0SJ4AFB7
Container Runtime blackbox test when running a container with a new image should be able to pull from private registry with secret


1/18/2022
Announcements
KubeCon EU SIG-Windows talk topics
Container users?
Pod OS field?
[Mark] Kured demo
https://github.com/weaveworks/kured
Windows support PR - https://github.com/weaveworks/kured/pull/460 
[Aravindh] WIP Node service log viewer demo
Use sig-windows-dev-tools and the dev branch shared by Aravind https://github.com/kubernetes/kubernetes/pull/96120 
[jay] on vsphere ~ csi-proxy perf expectations, consistent CPU spikes at scale, normal ? 
1/11/2022
Announcements
1.24 release schedule finalized https://github.com/kubernetes/sig-release/tree/master/releases/release-1.24 
Agenda
https://hub.docker.com/u/sigwindowstools -> https://github.com/orgs/kubernetes-sigs/packages 
https://github.com/kubernetes-sigs/sig-windows-tools/pkgs/container/sig-windows%2Feventflow-logger 
Windows docs updates
Bi-weekly meeting every other Thursday 9:30am PST (alternating weekly with backlog grooming)?
jayunit100/jamie/ State of csi-proxy, vsphere 
Csi-proxy will be integrated with wins in near future
https://github.com/rancher/wins#enabling-csi-proxy-functionality
SMB are most mature plugins (for local storage), vSphere is in alpha
GCE and Azure plugins (for cloud storage)
Add SMB to sig-windows-devs-tools
Operational readiness KEP
Will bring it up in PRR slack
1/4/2022
Announcements
Happy new year
dev@kubernetes.io (https://groups.google.com/a/kubernetes.io/g/dev) 
Do not join group
Other groups will follow in Jan/Feb
1.24 enhancement freeze Jan 27, 2022 (proposed)
MCR 20.10.9 released (https://docs.mirantis.com/mcr/20.10/rn-20-10/20-10-9.html) 
1.24 Enhancements status
OS podspec field
Progress to beta (minimal changes identified for 1.24)
Node service log viewer
Alpha in 1.24
Looking for volunteers to help (esp w/ kubectl changes)
Operational readiness
Reviewed but needs PRR reviewer
Host process containers
Stay in beta (probably)
12/27/2021
CANCELLED
12/21/2021
CANCELLED
12/14/2021
Announcements
K-dev mailing lists migrating Jan 2
DockerShim is removed
Cancel 12/21 and 12/28 meetings?
[Jing] Windows Container Kubernetes document update needed (https://kubernetes.io/docs/setup/production-environment/windows/intro-windows-in-kubernetes)
Volume mounts cannot project files or directories back to the host filesystem.
Expanding the mounted volume (resizefs)
You cannot enable mountPropagation for volume mounts as this is not supported on Windows.
None of the Pod securityContext fields work on Windows
TODO (Mark): set up a time to pair/plan on restructuring docs
[jay] [Windows on Tanzu intro doc] ~ maybe a good intro for newcomers on CAPI windows workflow  https://docs.google.com/document/d/1b-CKvnimNgL_erZhoFK-OoecqQCenVE4q9yHa7Vttlk/edit 
Capz docs: https://capz.sigs.k8s.io/topics/windows.html
[jay] https://github.com/jayunit100/k8sprototypes/tree/master/windows/op-readiness 
12/7/2021
Announcements
- 1.23 release is today
Contributor summit this week. 
[Muzz]Did all the Doc PRs make it?
All the docs PRs we were tracking made it.
[Brandon] DockerMsftProvider 20.10.11 is coming in mid-december
20.10.8 has issue where docker crashes after reboot
File permission issues, delete panic.log file in program/data before restarting the service 
Containerd provider?
On the backlog
https://github.com/microsoft/Windows-Containers/issues/186 
[David Schott] https://github.com/kubernetes/kubernetes/issues/106636
[Brandon] Next steps for Host Process Containers
Support for custom users
File system enhancements
Projects or components?
Win_exporter - pr opened
Kured - pr opened
Kube-proxy
Csi-proxy
Plan is to use directly in drivers instead of proxy
Works in HPC but likely won’t merge
cni’s?
Anything else?
Differences between HPC and server silo containers
HPC are privileged on the host,it is  a process running  on the host
There is Pod security support that went beta in 1.23
Discussion on hostile work loads: https://docs.microsoft.com/en-us/azure/aks/concepts-security#hostile-multi-tenant-workloads 
(jay, might join late!) GMSA question from wenli, https://kubernetes.slack.com/archives/C0SJ4AFB7/p1638548431217300 
If you get ERROR_NO_LOGON_SERVERS.. in a pod, whats the right way to fix it? -> update: looks like this relates back to the nic where dns traffic is flowing through 
[james/jamie] gmsa image to k8s registry: https://github.com/kubernetes-sigs/windows-gmsa/issues/52 
Reboot tests
5 min sleep
https://github.com/kubernetes/kubernetes/pull/104698 
Openshift does this
drain
Wait for new workload to be available
Updated pr with next steps
11/30/2021
Announcements
Docs PRs reviewed today
https://github.com/kubernetes/website/pull/30391 - host process containers
https://github.com/kubernetes/website/pull/30436 - OS field 
Waiting on feedback from Tim
DockerMsftProvider is a couple versions behind Moby versions (https://github.com/microsoft/Windows-Containers/issues/183)​​
moby/moby releases are source-only though, so our choice is to either wait for Mirantis to release new versions that will get piped through DockerMsftProvider, or build moby for Windows ourselves
Is there another option (besides containerd)? Is Microsoft or anyone else already building Moby releases for Windows?
Microsoft is working on building docker engine binaries directly from Moby upstream and making it available for others, but nothing to share quite yet
Can we find out from Mirantis what the timeline is for new 20.10.x versions?
Past experience: 4-6 week delay between a Moby release and the corresponding Mirantis Container Runtime release
Op readiness: https://github.com/kubernetes/enhancements/pull/2975/ ← any other major issues?
11/22/2021
NO MEETING
11/16/2021
Announcements
Code freeze is Nov 16 - TODAY
Docs 
Draft PRs open by Nov 18
PRs reviewable by Nov 23
PRs reviewed by Nov 30
Next week tentative - if no agenda by Monday 5pm pst will cancel
[jasonhall] Tekton workloads on Windows
[jay] https://github.com/kubernetes-sigs/cluster-api-provider-azure/issues/682 curious about CNIs, cloud provider azure, windows
11/09/2021
Announcements
Code freeze is Nov 16
Docs 
Draft PRs open by Nov 18
PRs reviewable by Nov 23
PRs reviewed by Nov 30
PRs / work we want merged for 1.23
Metrics:
https://github.com/kubernetes/kubernetes/pull/105744
https://github.com/kubernetes/kubernetes/pull/104287 
Os field:
Working on another PR to add support in kubelet
Needs docs updates, 
Pod security standards updates
Swagger 
Node log viewer service
At risk
Plumbing is working, working on heuristics
Docs:
storage: document Windows projected volume limitations


Community meeting on Nov 23?
Cancel if no agenda
Windows Operational Readiness KEP merge
https://github.com/kubernetes/enhancements/pull/2975/files
K8s 1.24 priorities
Enhancements
Capacity / scalability / eviction
Docs
Kubeadm and kubeproxy image

[sebsoto] PR draft feedback
WIP: Allow identifying Windows default routes
11/02/2021
Announcements
Code freeze is  Nov 16
Docs PRs for enhancements must have placeholder PR open by Nov 18
Vote in steering committee elections - https://elections.k8s.io/ 
[jasonhall] Tekton workloads on Windows
https://kubernetes.slack.com/archives/C0SJ4AFB7/p1635865180065400 
Finishing off Operational Readiness https://github.com/kubernetes/enhancements/pull/2975/files (finalize kep metadata)
Discussions around conformance vs. readiness. Agree to finish the KEP first and MAYBE talk about conformance later with sig-architecture (if they let us !)
Differentiate between Overlay and Bridge tests on network section https://github.com/kubernetes/kubernetes/issues/101062 
Kubernetes 1.19.x … pause 3.3 from the dependencies(https://raw.githubusercontent.com/kubernetes/kubernetes/release-1.19/build/dependencies.yaml), … mcr.microsoft.com? 
Should be safe to use latest pause image on any cluster.
OS field updates - broken into 3 changes
API changes
Merged - behind a feature gate - PR link
Kubelet changes
Reconcile OS label
Reject pod if OS label does not match
PSP changes
Updating validation logic to depend on OS field may need to be held into this feature is beta / 1.24
10/26/2021
Announcements
Code freeze is ~ 3 weeks
HostProcess containers on containerD support merged in CAPZ
[aravindh] Next steps for documenting projected volume issue
Should we try and upstream the work-arounds for settle projected volume perms when RunAsUser and RunAsUsername are both set?
Not today
Longterm plan is to use OS field to either add RunAsUser or RunAsUserName
Add documentation in projected volume section
On MSDN - Document more clearly capabilities of different container users and highlight potential risks 
Update https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/container-security#when-to-use-containeradmin-and-containeruser-user-accounts
[anfernee] PR review request: https://github.com/kubernetes/kubernetes/pull/99287 
runAsNonRoot validation for ‘Add Pod OS field’
https://github.com/kubernetes/kubernetes/pull/104693#discussion_r734777251
10/19/2021
Announcements
Containerd 1.6 beta
Please remove annotations for host process containers for containerd config
Dockershim will be removed immediately after 1.23 release - https://groups.google.com/g/kubernetes-sig-windows/c/Y7m-A2rEV8Q 
In place upgrades? Are they supported?
Not sure; need to follow up with Sig node.
Install for Containerd
Is it complicated to install?
Possible Winget package?
Big difference from Docker Provider that is there today
Containerd image Pull Timeout?
Not there for containerd
Doesn’t work with containerd
Support image pull progress deadline for containerd #4984   
KubeCon recap
Muzz to share some customer interactions
Dev focus was well received.  
Hostprocess demo was also exciting
WS2022 was also popular topic
Just getting started
Argon (process isolated) backwards compatibility
OSM demo - Service Mesh for Windows!
Issues with existing projects building Windows container images 
Does buildx solve this? Sort of but...
Make it simple to build Windows images.  Github action
[aravindh] Next steps for documenting projected volume issue
[Mark / James] - Is anyone seeing issues with limits on version upgrades maybe related to https://github.com/kubernetes/kubernetes/pull/86101
10/12/2021
NO MEETING - Cancelled for KubeCon NA 2021
10/5/2021
Announcements:
KubeCon NA 2021
NO meeting next week (Oct 12)
SIG-Windows talk - Wed Oct 13 11:00 am PDT
Focus on HostProcess containers and dev environments
Service Mesh on Windows - Fri Oct 15 11:55 am PDT
https://github.com/openshift/windows-machine-config-operator/pull/638
Projected volume doc


9/28/2021
Announcements:
Congrats to Claudiu for TL promotion and thank you to Deep for all the contributions!
Docker 20.10.8 has a fix for issue where containers are never stopped - https://github.com/moby/moby/pull/42613 
KubeCon NA - Is anyone attending in person?
Brandon and Muzz
WIP: Windows file permissions #104660
Library is healthy
Ssid expert to give the library a look. Are these mappings valid?


HostProcess pr in capz: https://github.com/kubernetes-sigs/cluster-api-provider-azure/pull/1672 using images from https://github.com/kubernetes-sigs/sig-windows-tools/pull/161/commits 
[jayunit100] Follow up: scalability tests? Perf-tests is the canonical place for them.  Anyone want to own the Windows initiative?
9/21/2021
https://github.com/kubernetes/kubernetes/issues/88153
Ibrahim to add an e2e test to to test this scenario
Lots of interest across infra providers to look into perf metrics
Time to start a scalability / perf subproject for Windows?
SUSE/Rancher might be able to help lead this effort
Different perf tools for windows: https://hackmd.io/dOXPzNWeSzqiLNqZCtZl2A 
https://kubernetes.slack.com/archives/C09QZTRH7/p1632245074017200 
Possibly create a sub-group to focus on this

J any interest in kube proxy / windows work ? https://github.com/kubernetes-sigs/kpng ! :) 
9/14/2021
Announcements
KEP Updates
Host Process Containers KEP Beta approval
Node Logs
OS field in Pod Spec
https://github.com/kubernetes/kubernetes/issues/104987 host-process container e2es… questions
Use nightly build of containerd for windows https://github.com/kubernetes-sigs/sig-windows-tools/releases/download/windows-containerd-nightly/windows-containerd.tar.gz
Is https://github.com/kubernetes/kubernetes/issues/97807 resolved?
Windows node conformance - Jay?
9/7/2021
Announcements
KEP updates
https://github.com/kubernetes/enhancements/pull/2803
https://github.com/kubernetes/enhancements/pull/2865 
WS 2022
3 container images (NanoServer, ServerCore, Server)
Improved  compatibility commitment for process isolated containers
No SAC releases
Windows Server 2022 and beyond for containers
Windows Server 2022 Now Generally Available
:1809 full & nano images for ltsc2019 - Consider having additional ltsc2019 tags added here & here? 
[hxie] Where to get docs on hostProcess containers
https://kubernetes.io/docs/tasks/configure-pod-container/create-hostprocess-pod/ 
8/31/2021
Announcements
PRR soft deadline for enhancements is Sept 2nd!


[jstur] How to handle Podsandbox errors https://github.com/kubernetes/kubernetes/issues/104635 
[j] Should we add another hostProcess E2E for static pod manifests / restarts ?  https://github.com/kubernetes/kubernetes/issues/104648
TODO: follow up with Brandon Smith around 20H2 images
Addressed in Windows OS Version Support
8/24/2021
Announcements
KubeCon NA registration open
1.23 release schedule is up
Calico now working in windows-dev-env thanks amim!
KEP updates
Identify Windows pods at API admission level authoritatively #2802 - new OS field on Pod will be added
HostProcess containers to beta


Discussion: What is the optimal length of Code Freeze? #1674
8/17/2021
Intros
Announcements
Metrics Perf Work
Context deadline exceed metrics/summary https://github.com/kubernetes/kubernetes/issues/104283
Docker shim makes two calls to hcs https://github.com/kubernetes/kubernetes/issues/104285
Containerd doesn’t return network stats https://github.com/kubernetes/kubernetes/issues/104286 
Concurrent calls to containerd hang https://github.com/kubernetes/kubernetes/issues/104111 
Fixes: https://github.com/kubernetes/kubernetes/pull/104287 
https://github.com/kubernetes/kubernetes/issues/104418 
https://github.com/kubernetes/kubernetes/issues/103671 - Test flakes in ‘Pods should delete a collection of pods’
[aravindh] Projected Volumes, Bound Service Account Token and  Windows Pods
Windows Pod with RunAsUserName and a Projected Volume does not honor file permissions in the volume
8/10/2021
Intros
Announcements
Starting KEP collection for 1.23
Support for Windows privileged containers #1981 beta?
Use kubectl to view logs of system services on nodes #2258 - alpha?
Identify Windows pods at API admission level authoritatively #2802 - alpha?
A Windows-Conformance definition and tooling convergence #2578 - alpha?
SIG-Windows Enhancement liaison volunteers?
[james/mark] Bumping Gmsa api version to beta (or stable) version https://github.com/kubernetes-sigs/windows-gmsa/issues/43 
ContainerD CNI support updates
Vendoring in v1.22 CRI updates https://github.com/containerd/containerd/pull/5836
HostProcess support https://github.com/containerd/containerd/pull/5131


[Sebastian] Windows Kubelet should prefer using the network interface with the default gateway when picking the Node IP
https://github.com/kubernetes/kubernetes/issues/104269 
8/3/2021
Intros
Announcements
Re-visit https://github.com/kubernetes/kubernetes/issues/102849
Windows OS uses same SID for local container users and changing this behavior isn’t currently planned
We might be able to work around issue but manipulating well-known user groups on the host and assigning access to those groups at container runtime - needs an experiment to confirm, maybe kubelet can do this
Current plan - Update docs to raise awareness of this behavior



[WIP] Identify Windows Pods during API Server admission
Runtime classes generally agreed
Details on behaviours still being worked through
Kubelet rejecting node selectors only
Still allow tolerations and node selectors but fail if nodeselector doesn’t mage
Higher level controllers
Scenario that will fail to have runtime
Can we mutate specs?
Pod security admission plugin dynamically query on RunTimeClass
Any blocking issues for users?
[jay] Final home for the burrito server ? https://github.com/kubernetes-sigs/sig-windows-tools/pull/153 
Remove from image-builder repo
Keep in sig-windows-tools https://github.com/kubernetes-sigs/image-builder/issues/672 created as a semaphore





7/27/2021
Intros
Announcements
V1.22 Docs freeze is today
European friendly meetings?
GMT+3
Poor attendance when tried w/ network proxy working group
Suggested targeted meetings?
HostProcess docs merged: https://github.com/kubernetes/website/pull/28413 
E2E tests in Capi: https://github.com/kubernetes-sigs/cluster-api-provider-azure/pull/1388#issuecomment-885082038 	
Is anyone else using CAPI test framework? Should we centralize/consolidate logic for Windows?
Projected volumes
Should we update docs with a warning?
Follow up on issue next week


7/20/2021
Intros
Announcements
July 20 (today) Docs for features must be ready for review
July 27 Docs Freeze
Fix for vxlan/dualstack merged, backport in progress https://github.com/kubernetes/kubernetes/pull/101047 
Doc PRs to review
HostProcess 
K8s.io updates https://github.com/kubernetes/website/pull/28413
Blog post https://github.com/kubernetes/website/compare/main...brasmith-ms:hostprocess-blog
Containerd and hcsshim components: https://github.com/marosset/windows-cri-containerd/releases/tag/hostprocess 
CSI proxy to stable
https://github.com/kubernetes/website/pull/28914 
General updates
https://github.com/kubernetes/website/pull/29017




KEP for Windows Identification
https://github.com/kubernetes/enhancements/issues/2802
https://github.com/kubernetes/enhancements/pull/2803


7/13/2021
Intros
Announcements
1.22 end-game schedule
July 15 - Test Freeze
July 27 - Docs Freeze
Dev-env ~ hyper-v
https://github.com/kubernetes-sigs/sig-windows-dev-tools/ 
Need to discuss which solution: hyperv native or VirtualBox native
Have both working.
New project, Contributions Welcome!
Dev-env hcsshim issue (@danny)?
Couldn’t attach networking to containers?
CSI Proxy to Stable 
Cutting the release tomorrow
Docker log rotation issue resolved with latest release of docker
https://github.com/microsoft/Windows-Containers/issues/106#issuecomment-871804815 
KEP for Windows Identification in 1.23
https://github.com/kubernetes/enhancements/issues/2802 
Please review and comment
7/6/2021
Intros
Announcements
Welcome back Mark Rossetti!
Code Freeze Thursday!
https://github.com/kubernetes-sigs/sig-windows-dev-tools 
Thanks to Friedrich, Slayden, Jay and many others
Is there a clean way to add a defaultGateway to an existing windows route ?
Hcn transparent mode?
Creates a simple hyperv switch
Is hyperv enabled required - just need apis but not hypervisor
Start thread on slack around Virtual box creating two nics and Antrea isn’t handling this correctly.  
https://github.com/kubernetes/kubernetes/pull/103434 
Skip host file mapping if hostprocess
6/29/2021
Intros
Announcements
Code freeze July 8th
Bug fix for sub-attomic file mappings: https://github.com/kubernetes/kubernetes/pull/97642 
Should be directory symlink
[jstur/peri] HostProcess Update
[ravi] Windows node Identification: https://github.com/kubernetes/enhancements/pull/2803/files 
[Erick] Projected Volumes and Container User
https://kubernetes.slack.com/archives/C0SJ4AFB7/p1623439088166000
Chown on volume causes the pod to not start
WIP: windows: Fix Pods with Projected Volumes 
6/22/2021
Intros
Announcements
Code Freeze (July 8th)
[jsturtevant] HostProcess Tests up: 
https://testgrid.k8s.io/sig-windows-master-release#ci-kubernetes-e2e-aks-engine-azure-master-windows-containerd-hostprocess-alpha 
20H2 Tests
Passing in GCE and aks-e
[jsturtevant] Containerd 1.5 requires hcsshim built from master for some fixes
https://testgrid.k8s.io/sig-windows-master-release#aks-engine-windows-containerd-1-5-master
Hcsshim would die and kill all containers
Could not recreate containers in pods
https://github.com/kubernetes-sigs/sig-windows-tools/issues/152 ← build artifacts server perit@vmware.com is working on
6/15/2021
Intros
Announcements
Sonobuoy support: https://sonobuoy.io/sonobuoy-adds-windows-support/ 
Projected volume bug
https://github.com/kubernetes/kubernetes/issues/102849
Plan is to use go-acl and do the equivalent of os.Chown().
WIP: https://github.com/kubernetes/kubernetes/pull/102868/files 
Siloscape: First Known Malware Targeting Windows Containers to Compromise Cloud Environments
[Brandon] Official response
Addressed in January 21, 2021-KB4598296 (OS Build 17763.1728) Preview (microsoft.com)
Log Viewer KEP 
Working through it, might be delayed
6/8/2021
Intros
Announcements
Code freeze for 1.22 is July 8th
GMSA webhook updates
https://github.com/kubernetes-sigs/windows-gmsa/pull/31
New image: docker pull sigwindowstools/k8s-gmsa-webhook:latest 
Also have versioned tags now: v0.1.0-20-gdc75308
[Aravindh] Cloud Controller Manager
cloud-node-manager.yaml
mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.0 
Will this image be updated?
Currently 2019 support
Currently coming in v1.24?
Available in out-of-tree azure provider https://github.com/kubernetes-sigs/cloud-provider-azure/releases/tag/v1.0.0 
Enhancement?
https://github.com/kubernetes/enhancements/tree/master/keps/sig-cloud-provider/azure/2328-ccm-instance-metadata 
[muzz] OpenSSH install for image-builder, binary name?
[Aravindh] Siloscape: First Known Malware Targeting Windows Containers to Compromise Cloud Environments
6/1/2021
Intros
Announcements
TGIK episode
Image builder
Cross platform images and CRI
https://www.youtube.com/watch?v=l3TWbrWkVzY&list=PL7bmigfV0EqQzxcNpmcdTJ9eFRPBe-iZa 
Agenda
OpenSSH install for image-builder (offline) [PT]
Is there download link for the cab file for ssh feature? (for airgap env)
Muzz follow up with team working on  components
Same for containerd?
Containerd can be downloaded out of band 
Ssh needs cab file
Containerd as a feature.
Open issue for windows 
Code freeze for 1.22 is July 8th
5/25/2021
Intros
Announcements
HostProcess merged: https://github.com/kubernetes/kubernetes/pull/99576 
if you plan to try it out you still need to build the containerdshim from https://github.com/microsoft/hcsshim/pull/962 but any version of containerd should work
Peri has a containerd pr to enable host network pods
Kubecon Videos
 Windows Containers in Kubernetes and a... Mark Rossetti, Kalya Subramanian, David Schott & Jay Vyas - YouTube
KubeProxy issue fixed and cherry picked for 1.19,1.20 and 1.21
Ref counting is only applicable to Remote endpoints by sbangari · Pull Request #101358 · kubernetes/kubernetes (github.com) 
Dockershim will be removed in K8s. 1.24 and not earlier.
Windows Server SAC evaluation version 
Muzz is confirming with Windows Licensing team
Node logs KEP
Adding feature flag and PR is in progress
Access denied and log rotation
Are these the same? 
https://github.com/microsoft/Windows-Containers/issues/106
https://github.com/kubernetes/kubernetes/issues/98102 


5/18/2021
Intros
Announcements
CSI proxy is going stable in 1.22
Sig-windows “After hours”
May 23rd for kubecon talks
[jsturtevant] KEP for cAdvisor-less, CRI-full Container and Pod Stats
Looking for feedback for Windows
[muzz] Dockershim deprecation extension to K8s. 1.24 or later?
Soliciting response and feedback
Dockershim is being removed. Windows containerd is newer.  Extending 1 more release?  
Open up issue and post in slack for more feedback.
[jsturtevant] gmsa cert request upgrade


5/11/2021
Intros
Announcements
KEPS
Enhancement freeze Thursday May 13th 
Host Process
Agreement on final item, waiting for review from sig-api
LogViewer
Kep Merged!
Implementation started
PSP vNext KEP
CSI Proxy
Going Stable!
Need to add to tracking spreadsheet
Microsoft/Windows-Containers issues
https://github.com/microsoft/Windows-Containers/issues/106
Still repros with 10c (10.0.17763.1757). 
NPD Demo
https://docs.google.com/document/d/1eiK6KAp_TFR0PgBMu2WCf49fMZcg-HHnBHMc9fALquU/edit#
https://github.com/kubernetes/node-problem-detector/pull/554
Extension model
Config directory, to run this script with success/fail
5/4/2021
Intros
Announcements
Enhancement freeze is next week (May 13th)
HostProcess
Implementation pr in and under review
https://github.com/kubernetes/kubernetes/pull/99576 
LogViewer
Need updates on KEP documentation
https://github.com/kubernetes/enhancements/pull/2271 
PSP vNext KEP (sig-auth)
PSP replacement meeting May 5th 1pm PST
https://github.com/kubernetes/enhancements/pull/2582
Leadership updates
Deep steeping down as TL
Claudiu is nominated as TL
[jay] Windows Testing and acceptance sync
Should we go with dan’s idea for affirmation as opposed to pass/fail? https://github.com/kubernetes/enhancements/pull/2137/files (Versioning+Status results as opposed to pass/fail results)
Similar to conformance profiles?
https://github.com/kubernetes/enhancements/tree/master/keps/sig-architecture/1618-conformance-profiles 
[jeremyje/mcshooter] Node Problem Detector MVP
Working prototype performing critical service health checks.
Windows binaries will be released in future NPD releases.
Feedback and contributions appreciated.
What issues should it be scanning for?
https://docs.google.com/document/d/1eiK6KAp_TFR0PgBMu2WCf49fMZcg-HHnBHMc9fALquU/edit
[james] release informing tests: https://app.slack.com/client/T09NY5SBT/threads/thread/C0SJ4AFB7-1617376478.111400?cdn_fallback=1 
Proposal to replace GCE tests on release-informing dashboards w/ aks-engine tests
Containerd disk:
Symlink issue https://github.com/containerd/containerd/pull/5411 
4/26/2021
Intros
Announcements
1.22 release schedule announced
https://groups.google.com/g/kubernetes-dev/c/jqKdK2H5ayQ/m/vJhkPcbBAAAJ
Enhancement freeze May 13th 
Code Freeze - Jul 8th
v1.22.0 release Aug 4th
Next cherry-pick deadline is May 7th
Mark out of office May 10 -> Early July
KubeCon EU Virtual is next week
Windows licensing updates -
Windows team is working on a community program to provide developer licenses - May take some time
Visual studio subscription / MSDN - you can use these licenses for development purposes
Free Azure credits include Windows server time
Partner program - Available to employees of Microsoft partners
Eval versions should work for 180 days for testing / development
Access to resources in CNCF Azure sub (possible? - contact Mark/James)
PSP vNext / Windows work
PSP vNext is targeting alpha in v1.22
https://github.com/kubernetes/enhancements/pull/2582 
Current plan is to include mechanism to filter out Windows pods for policies
Looking for volunteers to define/implement Windows specific policies
Microsoft/Windows-Containers issues
https://github.com/microsoft/Windows-Containers/issues/106
Might have been fixed in 10c+: 
https://support.microsoft.com/en-us/topic/windows-10-and-windows-server-2019-update-history-725fc2e1-4443-6831-a5ca-51ff5cbcb059 
10c is October 20, 2020—KB4580390 (OS Build 17763.1554) Preview but is fixed in latest images as well.
https://github.com/microsoft/Windows-Containers/issues/109 
Check node resources
Update with some additional info and repo setps


4/19/2021
Intros
Announcements
Local development now works - give it a try https://github.com/FriedrichWilken/KubernetesOnWindows 
Agenda
PSP Windows identifier: https://github.com/kubernetes/enhancements/pull/2582/files#r609093959 
Node Problem Detector
Being worked on to add windows support
Call for contributors
Follow the progress here: https://github.com/kubernetes/node-problem-detector/issues/461
winkernel kube-proxy problems when IPv6DualStack is enabled 
Windows licensing for local development
Muzz will follow up
Windows server developer license? 
Using eval now
Developer Experience 
Configuring WS for use of Web browsers.
Can be set via registry.  
Possible script: https://stackoverflow.com/questions/9368305/disable-ie-security-on-windows-server-via-powershell 
Kubeproxy update and dashboard
Roadmap: https://github.com/orgs/kubernetes/projects/55
Doc: https://docs.google.com/document/d/1yW3AUp5rYDLYCAtZc6e4zeLbP5HPLXdvuEFeVESOTic/edit# 
Kpgn - might be fairly straight forward, doing a prototype soon
Release Cadence KEP:
Releasing 3 times a year instead of 4. 
Approvals rolling in
4/13/2021
Intros
Announcements
1.21 released last week!
Plan for log-viewer KEP
Please review and comment
winkernel kube-proxy problems when IPv6DualStack is enabled 
1.22 priorities and plans
1.22 SIG-Windows planning doc
Container hostPath with symlink to different disk partition issue
https://docs.google.com/document/d/1hMn2LwFjXDbIzn3ZK_xiw4eNYnAajTIIyRXyQKaKa20/edit
Relevant fix https://github.com/adrg/xdg/pull/16  
4/5/2021
Intros (Open invitation for new folks to introduce themselves)
Announcements
[Sladyn] : Sig-Release liaison informal intro/training, get up to speed and I can probably pick up on some coding contributions as well. 
Image Builder images 
Should we use Eval SKUs?
Make this configurable and document how to target different windows SKUs when running image builder?
3/30/2021
Intros (Open invitation for new folks to introduce themselves)
Announcements
Docs deadline is March 31 (reviews must be reviewed by SMEs)
[Jing] Container Teardown Issue
Fail to remove filesystem: CreateFile, access is denied error
https://github.com/microsoft/Windows-Containers/issues/106 
[Jay] Non url based windows build input to img builder  https://github.com/kubernetes-sigs/image-builder/issues/571
[minor issue] https://serverfault.com/questions/1058833/vmware-workstation-on-ubuntu-cant-mount-windows-isos  
3/23/2021
Intros (Open invitation for new folks to introduce themselves)
Announcements
1.21 Test Freeze March 24
1.21 Doc updates deadline March 31
https://www.kubernetes.dev/resources/release/#tldr
Interest check for Asia-friendly community meeting
[peri & jay] 15 char node names, capi, ldap, and https://github.com/kubernetes-sigs/cluster-api-provider-azure/blob/63bd284d14e479a18154afd9150c7b12a2dfa205/azure/scope/machine.go#L280
https://github.com/kubernetes-sigs/cluster-api/issues/2217 
Recommendation ~ short names ~ netbios restr. not going away
Names longer than 15 chars may impact cloud providers in addition to capi-providers
Should 15 char names be enforced in cluster-api?
Next steps - research how host names are set in all various clouds and make proposal for capi-changes
Subproject involvement
kubernetes-sigs/windows-gmsa
Looking for help with
Updating docs
Updating deployment files
Adding e2e test cases (gmsa + shared volumes)
kubernetes-sigs/sig-windows-samples
Looking for help with
Adding content
kubernetes-sigs/windows-testing
Looking for help with
Onboarding testgrid jobs for additional cloud providers / ISVs
Converting aks-engine testgrid jobs to use cluster-api
kubernetes-csi/csi-proxy 
[lubomir] Windows control-plane. Do we have plans for that eventually?
3/16/2021
Intros (Open invitation for new folks to introduce themselves)
Announcements
1.21 Test Freeze 3/24
KubeCon NA CFP Open until April 11 
15 min ~ [jay] delete old subgroups, maybe rehash them to just  “testing”, “ecosystem(networking, priv cont, ..) ” ?
Subprojects: https://github.com/kubernetes/community/tree/master/sig-windows#subprojects
Give folks ownership of projects
3/09/2021
Intros (Open invitation for new folks to introduce themselves)
Announcements
Annual Report
1.21 Code Freeze is TODAY
Add PRs we need merged here!
#99221 Get filesystem stats for files on Windows 
#99670 Include nltest in the windows busybox image
#99576 Windows HostProcess work
#99862 · gmsa output to check values
[WIP] kube-proxy handle terminating endpoints
#96616 Kubelet Check Windows Permissions 
KubeCon NA CFP Open until April 11 
15 min ~ [jay] delete old subgroups, maybe rehash them to just  “testing”, “ecosystem(networking, priv cont, ..) ” ?
[Aravindh] [WIP] kube-proxy handle terminating endpoints needs review from sig-windows
[Jing] csi driver registrar failure
 hcsshim::System::Start: failure in a Windows system call: The virtual machine or container exited unexpectedly. (0xc0370106)
Uses nanoserver as base image
[Jing] os.Stat() does not work for symlink that points to a dir
Access is denied error
https://github.com/microsoft/Windows-Containers/issues/97
[Jing] Memory mode of EmptyDir
3/02/2021
Annual Report
PR  https://github.com/kubernetes/community/pull/5575 
[Jing] Lstat issue on socket file for Windows
https://github.com/kubernetes/kubernetes/pull/99463
Jay: growing the community -> “sig-windows after dark” , anyone up for it ?  (kill tools + sigwin-networking subgroup + kill other sgs?)
[Lauri] Offer to help out with program management: see PR. What are your top three process/alignment/workflow needs right now?
https://github.com/kubernetes/kubernetes/pull/97576 
https://github.com/kubernetes/community/pull/5565 (ways of working agreement)
2/23/2021
KEP Status
Privileged Containers KEP
Node Log KEP
Test images
Nginx source for https://hub.docker.com/repository/docker/claudiubelu/nginx
Claudiu built a x64 ngnix image because there was not one available previously
https://github.com/kubernetes/kubernetes/issues/99325 
TODO: Move ngnix build to sig-windows-tools or windows-testing
Httpd push: https://github.com/kubernetes/test-infra/issues/20884 
CI Signal
https://testgrid.k8s.io/sig-windows-signal 
Peri / J
Windows Netpol conformance - anyone want to help test b4 merging ? Docker images for 99360 ? containerd AD stuff updates ?  
Add new tag we can use in skip regex if/when [LinuxOnly] tag gets removed?
https://support.microsoft.com/en-us/topic/february-16-2021-kb4601383-os-build-17763-1790-preview-9535653c-8012-47f0-0f90-44cdd57cdc36 
Containerd gMSA account fix in 2C


[Mark] ContainerD and disk mount issues found on Windows with in-tree and csi plugins
https://github.com/kubernetes/kubernetes/issues/99342 
Related to 
https://github.com/containerd/containerd/issues/4915
https://github.com/kubernetes/kubernetes/pull/97642
https://github.com/kubernetes/kubernetes/pull/97045
https://github.com/golang/go/issues/40180  
Jay to post CNI and containerd meeting in slack
Conformance tests
https://github.com/kubernetes/kubernetes/issues/96639 
Possible working sub group?


2/16/2021
Announcements
Sig’s need to author annual report for active projects
Will draft and share
KEP status
Exception from release team
Privileged Containers
Open question - Enforce pods to set pids?
Concerns on the future if we support windows mixed pods since windows supports users differently than Linux
Node log kep
API struct need reviewer from API team
Three things:
Kubectl source code review
Explicit Admin rights - Higher RBAC - This is important
Keep it off by default.
Request for features should be expanded for pod logs - pushed back as out of scope
Overall - waiting for sig-cli to take a look and help address comments
Defender Updates
Did `Add-MpPreference` solve perf issues people were experiencing?
Switch test infra to use runtime classes instead of node selector
Widely different experiences between cloud providers
Some use taints, some use tolerations
Ravig to open an issue for discussion
20H2 test i
2/9/2021
Announcements
1.21 KEP updates / discussions
[Aravindh] KEP 2258: Node service log viewer
[Mark]  KEP 1981: Windows privileged container KEP updates for alpha #2288
https://github.com/kubernetes/enhancements/pull/2364
Windows Defender discussion
Guidance is to set defender exclusions using FULL PATH to containerd
Add-MpPreference -ExclusionProcess "c:\program files\containerd\containerd.exe" 
Sig-windows testgrid dashboard updates: 
https://github.com/kubernetes/test-infra/pull/20653
Ready to merge?
Also add ‘CI’ column to SIG-Windows Project
[Peri] Containerd privileged containers
[Ravi] https://github.com/kubernetes/test-infra/pull/20600
2/2/2021
Announcements
Jay hosted TGI Kubernetes Jan 29, 2021 (link)
1.21 KEP updates / discussions
[Aravindh] KEP 2258: Node service log viewer
[Mark]  KEP 1981: Windows privileged container KEP updates for alpha #2288
[James] Sig-windows testgrid dashboard updates: 
https://github.com/kubernetes/test-infra/pull/20653
Windows Defender overhead w/ containerd
MS team is seeing 10% CPU overhead when running containers vs docker
Would like to gather a complete list of scenarios negatively impacted by Defender to bring to the defender team.
Pull image spikes? (Jeremy)
image extraction spikes? (Peri)
Hangs in policy jobs - kubectl issues commands that don’t finish? (Jay / Peri?)
TODO: open an issue against containerd/contianerd to track perf regression vs docker
1/26/2021
[ANNOUNCEMENT ~ PRE-TRIAGE/CI MEETING STARTING 1/26]
TRIAGE : testgrid, issues [15 minutes]
[james, ravi, jay] https://github.com/kubernetes/test-infra/issues/20542  MAKE SIG_WINDOWS DASHBOARD GREEN, delete old tests
https://github.com/kubernetes/kubernetes/pull/98299/files
1.21 KEP updates/discussions
[Aravindh/LorbusChirs] KEP 2258: Use kubectl to view system service logs #2271


[Mark]  KEP 1981: Windows privileged container KEP updates for alpha #2288
Have proof of concept working (will try and demo next week)
https://github.com/microsoft/hcsshim/pull/903
https://github.com/marosset/kubernetes/commit/7abbd2113cea244f0f46326db29da66a078ff634
Add "io.microsoft.container.privileged": "true" to metadata.annotations in the deployment spec.
[Rey] - RFP for k8s security audit. Windows is not in scope for this audit.
https://github.com/kubernetes/community/pull/5416
Sharing # of windows users on various platforms could help strengthen the argument for including windows in next audit - but hopefully not necessary.
1/19/2021
[Mark / Everyone] Intros
[Mark] Announcements
KubeCon EU maintainer talk
Enhancement freeze is Feb 9, 2021
1.21 KEPS
[Aravindh] KEP 2258: Use kubectl to view system service logs #2271
Still designing kubectl interactions (esp to restrict to admin only)
Is there a way to filter?
In Openshift there is a way but not sure how that moves to kubectl
Is there a way to use gmsa?
Openshift it is only Admins
Useful for situations where containers can’t start
Maybe compromise would be to use this for critical logs only and more generic way to get logs would be via privileged containers
Implementations PR’s are open and it works in Openshift
Possible work to be done in kubectl
Need to add sig-cli to review of KEP
[Mark] KEP 1981: Windows privileged container KEP updates for alpha #2288
New updates - take a look
Limiting the difference scenarios on networking side. 
Always be in host network
All containers in a pod will be privileged
Separate kep possible for adding containers to pod network compartment
[Dims] Need containerd based jobs for presubmit CI https://github.com/kubernetes/kubernetes/issues/97304
Job doesn’t need to be pre-submit initially
Do have an optional one for containerd:
pr-aks-engine-azure-windows-containerd


Have periodics for azure file/azure disk?
Pre-submits have some blockers 
https://github.com/kubernetes/kubernetes/issues/93276 
Release informing
Little more lenient on timing
[James] e2e tests broke due to new images: k8s.gcr.io/e2e-test-images/agnhost:2.26
[j] Networkpolicy E2Es running now in windows, seeing many failures in various CNIs… whats the bar going to be ?
Containerd seems to be leaking processes
https://github.com/kubernetes/kubernetes/issues/98123


1/12/2021
[Mark] Announcements
Starting w/ 1.21 SIGS needs to notify release team about which enhancements they plan to delivered https://groups.google.com/g/kubernetes-dev/c/q_UqG6ZnSA0
Proposed enhancement freeze is Feb 9, 2021
https://groups.google.com/g/kubernetes-dev/c/w6oW2hv404s
[James/Mark] CAPI / Windows / containerD / CNI discussions
Install / configure CNI during image build??
Kubeadm docker flow creates a “Host” network and it all works
This doesn’t work in Containerd - there are blocks in containerd code path.
Might be able to use Privileged
Couple options for Containerd:
Install cni’s on machines
Can’t upgrade/install adhoc after cluster is up
Using privileged containers
Openshift
When node comes up operator configures it
GCE also installs cni’s at node start.
OVA - PostKubeAdm setup
Cloudbase-init is resilient to reboots.
HCI
Huge interest in everything as pods - much easier to manage
Lots of work arounds for running as services
[James/Jay] CNI DNS
Calico on azure and EKS
https://github.com/projectcalico/calico/issues/4307
https://github.com/Azure/azure-container-networking/pull/305 
Resolveconf in dockershim: https://github.com/kubernetes/kubernetes/blob/aea228f5dd3ad928dcb4c932fce8a80a74539d7f/pkg/kubelet/dockershim/docker_sandbox.go#L152 
https://github.com/kubernetes/kubernetes/pull/67435 
Containerd: https://github.com/containerd/cri/pull/1244 
1/5/2021
[m2] SIG-Windows leadership updates
Michael Michael stepping down from Chair after 5 years
Ben Moss stepping down from TL
Jay and James recognized as new TLs
[m2] Join https://groups.google.com/g/kubernetes-sig-windows to get the new calendar invite
[m2/Mark] James contributor award
[m2] KubeCon call for maintainers
[m2] 1.21 investments
Privileged containers (KEP PR merged in provisional state)
Brandon Smith to drive
Scaping event logs (KEP needed, owner?)
Aravindh to own (Muzz as reviewer)
LB health checks for external traffic policy
Cluster API
AWS - no provider changes needed. Image builder PR is out!
VSphere - image builder PRs being worked on.
GCP - don’t know if anyone is working on it. Off the radar right now
Azure merged: https://github.com/kubernetes-sigs/cluster-api-provider-azure/pull/1036 
containerD GMSA work (no KEP needed likely)
vSphere CSI proxy (owner?)
Gab to scope and execute on work
csi/storage: Create infra to build/release csi-proxy that supports privileged mode and existing GRPC over named pipes and separate proxy
[completed] GPU/device support
https://github.com/kubernetes/kubernetes/pull/93948
https://github.com/kubernetes/kubernetes/pull/93285 
Node problem detector.
Jeremy to help incubate / kick-start this
PRs to build it for windows are ready. Will make it into a service soon
[m2] Imagebuilder and Azure/AWS/GCP Windows images. Will cloud providers bake kubelet into the images or rely on kubeadm actions to bake Gold Windows images
CAPZ image provisioned by Azure will have everything you need including the kubelet
[Aravindh] Announcing Windows Container Support for Red Hat OpenShift
Demo


12/22 and 12/29
No SIG-Windows meeting - Enjoy the holiday break
12/15/2020
[MaRosset] DockerShim depreciation update
Effort largely focused on linux
Most folks want to move forward and deprecate dockershim
Containerd is not battle tested with cloud providers as dockershim
Dawn proposed we wait for timelines for public cloud providers to validate containerd and leave dockershim for windows only for a few more releases
No set timeline yet. Wait and see adoption
Azure to have public preview with containerD soon
CNIs may have to implement some changes to support
CNI config files need to delegate to sdnbridge/sdnoverlay to work with containerd
Example CNI configs for using flannel with contianerd at windows-container-networking/example at master · microsoft/windows-container-networking (github.com)
sdnbridge/sdnoverlay can be found at Releases · microsoft/windows-container-networking (github.com)
[Aravindh] Update on Windows CAPI discussion
[WIP] Add ability to provision Windows VMs
Windows support via kubeadm
[m2] Additional tech leads for sig-windows
James Sturtevant
Jay Vyas
Merge Windows device plugin API (aka Windows GPU support) (https://github.com/kubernetes/kubernetes/pull/93285) as soon as possible before merge conflicts or other issues break it again. (Unfortunately I cannot join today’s meeting, but maybe you can push this forward anyway - thomacos)
Let’s merge it
[Muzz] 5 mins Issue Triage
K8s meeting calendar (for the sig-windows backlog review)
[Amber] Privileged Containers KEP and Brandon Smith’s intro
KEP needs some updates before it gets pushed for approval
12/08/2020
No sig-windows meeting on Dec 22 and 29
[MaRosset] - Windows Server 20H2 (SAC release) is out
Container image tags 2009 and 20H2 are interchangable
Will update the docs in v1.21
[Amber] Privileged Containers Update
Service mesh scenarios (aligning to pod compartment)
Moving forward with v1.21 alpha
Runtimeclass or pod spec update being decided
Update the KEP to be ready for review/approval
Killer scenarios - CSI proxy
[David Schott] DSR Update
What works with WS 2019 in K8s 1.20 and what doesn't
When trying to delete network resources, ensure that policies are cleaned up before network deletion
David should propose a kubecon talk on this
Client IP preservation fix  for external LB health checks
[MaRosset] DockerShim depreciation and what it means for Windows
Sig-node discussion happening today
[jeremyje] Node Problem Detector will be staffed soon. First PR submitted.
12/01/2020
[m2] k8s contributor celebration
[m2] release notes [additions below]
1. Under Documentation
`Updates on documentation and guidance on containerD support for Windows nodes`
2. Under Feature
`containerD support for Windows nodes is now stable (#24862, @marosset) [SIG Windows and Node]`
`WinOverlay feature graduated to beta (#94807, @ksubrmnn) [SIG Windows]`
`Added limited support to kube-proxy for externalTrafficPolicy=Local setting via Direct Server Return (DSR) load balancers on Windows. Load balancer health checks have not been implemented yet (#93166, @elweb9858) [SIG Network and Windows]`

3. Under Bug or Regression
`Avoid dereferencing same endpoint twice on the deletion or update of a service (#93638, @sbangari)`
`Choosing the right source VIP for local endpoints (#96499, @sbangari)`

4. Under Depreciation
Please change the PR for `Windows hyper-v container featuregate is deprecated in 1.20 and will be removed in 1.21 (#95505, @wawa0210) [SIG Node and Windows]` to kind/deprecation
CAPI updates
AWS provider works and PR for image builder will come out today. No changes in provider
Azure - image builder PR merged. Provider changes will get out for reviews soon
vSphere PR is open for both image builder and provider
Some providers ship monthly
Naadir to check with the maintainers for GCP provider
Ankur Gupta issue in azure data center
Cannot make WMI calls from server to pod
https://github.com/kubernetes/kubernetes/issues/89650
https://github.com/kubernetes/kubernetes/issues/96935 
11/24/2020
No meeting. US Thanksgiving
11/17/2020
[m2] Process handle leak
Mark to take a look after thanksgiving. Needs investigation
Could be a config issue
[Jay] Asking about kube proxy CI to validate code changes with tests
Jocelyn and team can help here
ContainerD docs are going well. Mark to tag Michael on some questions.
[thomacos] Windows device plugin API / GPU access and e2e tests 
E2e test is running in AKS


11/10/2020
[Aravindh] Demo of Community Windows Machine Config Operator (10 minutes)
RH will connect with CAPI team since they started on this work and operator is using modified CAPI
RH to connect on k8s worker node docker logs scraping from event log
[m2] code freeze on Nov 12th
[m2] New invite for bug triage meeting
See top of doc for details
[james s] Node density tests for Windows (http://node-perf-dash.k8s.io/#/builds)
GCE tests node density today as well
Dockerhub rate limit pull update
Switched to Azure Container Registry with anonymous pulls
https://github.com/kubernetes-sigs/windows-testing/pull/214
Will push to both e2eteam dockerhub repo and k8sprow.aazurecr.io for a while
No problems for sig-windows on this front
[thomacos] Windows device plugin API / GPU access and e2e tests 
How do we create the device plugin test image? (See also this comment)
Everything should be ready except how to build the test image
According to docs, the base image needs to have the full Windows OS (not nanoserver). This is image that leverages dxdiag
Need another image that will host the device plugin
Customers will have to download and compile on their own and deploy to K8s as a daemon set
Should place this in sig-windows-tools
Will work with Claudio to push this to pass tests now (push to e2e tests github repo so prow can use it)
[m2] pulse for 1.20
10/27/2020
[Mark/m2] - Cancelling Meeting Nov 3rd
[Deep, Claudiu] Follow up on Dockerhub functional user for sig-windows
Details/notes at https://github.com/kubernetes/test-infra/issues/19477#issuecomment-716341329
Can’t do anything about anonymous pulls
We are the only ones affected b/c gcr does not support windows and most test jobs are going to use mirror.gcr.io (they now contain manifest lists)
Action: Claudiu to work on auth framework and work with Deep on creating functional user that’s exempt
[jay, jiunjen] what should we add for cni guidance to the k8s.io windows docs (calico, antrea, cillium flannel etc)  + what do we ‘own’ in terms of community guidance from wins -> priv containers etc ? 
Team to create a PR and have David Schott review as well
[Mark/James/Muzz] - Windows node issues with high CPU load after v1.18 changes to enforce CPU limits
https://github.com/kubernetes/kubernetes/issues/95735
[ddebroy] csi-proxy Beta.2 released
includes iSCSI support!
10/20/2020
KubeCon NA presentation recording
[Aravindh] Announcing Community Windows Machine Config Operator for OpenShift 4.6+
Windows containers restart in vSphere cluster
10C windows updates [Mark]
DNS fixes for Containers on Windows Server 2019
https://github.com/microsoft/Windows-Containers/issues/59  
https://github.com/microsoft/Windows-Containers/issues/61 
Single file mapping support for Windows Server 2019
https://support.microsoft.com/en-us/help/4464619
Docker hub registry rate limiting
https://github.com/kubernetes/test-infra/issues/19477 
https://www.docker.com/pricing/resource-consumption-updates 
Options 
Opensource dedicated
Deep will follow up on this request
Docker hub user for sig-windows
Mirror pull through. Does not contain windows images
Finish windows image promoter PR’s
Option for 1.19+
Still need other options for 1.16-18
Deep: Nov 1st isn’t hard limit.  But given number of images and jobs will still run into rate limit
10/13/2020
PSA: Kubecon slot will be on the 20th and will be recorded next week, so anything that needs to be bubbled up, please inform m2 
[m2] Envoy support for Windows
Relevant EnvoyCon Presentation
David to co-present
Important for enabling service mesh features on Windows
Caveat:
You need to build the binary, but release process being worked on.
[daschott] not full parity with linux, e.g. signals
#13322
[daschott] Please provide feedback on what features you’re looking for.
[m2] Did anything change from the plan of record for 1.20 as far as KEPs go?
[jing] CSI proxy will stay beta in 1.21
[m2] Best case for privileged containers is GA for 1.23 which gives runway for CSI
[James] Image Builder demo (windows caep: https://github.com/kubernetes-sigs/cluster-api/pull/3616) 
Management cluster can be linux only
[m2] The incremental work for other providers should not be significant other than image building?
[Peri] On premise is a bit different because of drivers and licensing, but the biggest concern is a consistent URI for a Windows image.
[m2] For public cloud, we can use their licensed images
[jing] re: privileged containers, discussed how we can do list-disk etc…
[muzz] still testing privileged containers. There are issues on the backlog looking for contributions.
[m2] How are we going to enable GSMA
[James] Don’t think we need to make changes to Cluster API core as kubeadmbootstrap CRD has a “prekubeadmcommand”
[Jay] Do people generally run Docker Enterprise?
[Mark] Currently we recommend Docker Enterprise, but containerd will be supported from 1.20 and that will be the recommendation going forth. You can also configure moby to consume containerd as its CRI, but is very experimental in Windows. Long term plans are to introduce containerd support for Moby and then Docker EE will use that. Docker EE currently calls the v1 APIs directly.


10/6/2020
[Amber] Privileged Containers Updates
https://github.com/kubernetes/enhancements/pull/2037
https://docs.google.com/document/d/12EUtMdWFxhTCfFrqhlBGWV70MkZZPOgxw0X-LTR0VAo/edit
Network Namespace abstraction broken impacts alpha (between privileged and non-privileged pods)
Could result in changes in CRI API (significant changes), pushing alpha to 1.21. Could release some capabilities in 1.20 to get feedback
[Muzz] What’s lined up for 1.20?
[unchanged, pending privileged container work] CSI to stay beta
containerD to stable
Networking work (No KEP needed)
Local traffic policy
DSR enhancements on loab balancer support
Image builder, cluster-api  to go alpha
[Jing] Can privileged container allows accessing volumes/disks on hosts from inside the container?
Csi-proxy APIs need to run Get-Disk/Get-Volumes/Format-disk etc.
Amber says possibly, but may need to be discussed
If job objects can do this, it should be possible, but Amber to check
[David] DSR reported problem and regression
Anytime HNS service is restarted  a VFP rule is dropped (service traffic from pod fails). Specific to 2019 only. Will be resolved in patch end of October.
See Issue #61
[Mark] Graceful node shutdown
https://github.com/kubernetes/enhancements/issues/2000
Is anyone interested in helping with implementation for Windows?
How will this work with CPI?
9/28/2020
[Kalya] Privileged Containers Updates
Identified another scenario with service meshes
Container Networking team is working on enabling service meshes with Envoy
Open Service Mesh (OSM) has been the focus right now, but open to other service meshes
OSM uses an init container that is privileged to program networking, we’ll need a way to do this on Windows
Problem around mixing privileged and non-privileged containers in the same pod
Deep mentioned that CSI providers also follow the model of mix of privileged and non-privileged containers in a pod
[David] DSR Promotion to BETA - Do we have tests?
there was a reported problem that David will investigate
David to look at endpoint slides tests that are passing according to Claudiu
We don’t have DSR specific tests. David asked to have DSR enabled for all tests. Mark to file an issue on this!
[Jeremy] Node Problem Detector
Jeremy doing the investigation
Looking for feedback, more types of problems that the detector could surface
Contributors welcome
Jing asked about debugging support for windows containers
https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/debugger-download-tools 
https://github.com/microsoft/windows-containers to file tickets
9/22/2020
[m2] Enhancements freeze on 10/6
[Mark] Dockershim deprecation followup from last week
CRI api is still alpha
SIG-cluster-lifecycle expressed concern for upgrade support for existing clusters since change CRIs is not supported
For Windows, dockershim AND kublet contains some workarounds needed to have the kubelet play nicely with docker.
Moby (and transitivvely Docker) for Windows only supports HCSv1 runtime. Experimental support for containers as a runtime was added - but never tested.
For containerd on Windows we have more flexibility to iterate/address issues in the Windows OS itself and/or containerd and have not been adding workarounds for CRI codepaths.
The cri-dockerd example shared still uses a dockershim hosted outside of kubelet and may work but at that point we might as well just use dockershim hosted by kubelet.
As mentioned we've been investing in containerd support for Windows and are hoping to finish stabilizing everything in 1.20. The proposal to add doc user-facing messaging around timelines/expectations for dockershim deprecation satisfies all of my concerns here.
9/15/2020
[m2] Updating the zoom link invite for sig-windows
[Mark] Feature flags/enhancements status tracking doc 
SIG Windows - Feature status
[David] DSR issue explanation - Is it 1.19 or 1.20? 
https://github.com/kubernetes/kubernetes/issues/62046
DSR did land in 1.19. Local traffic policy via DSR will be coming in 1.20
DSR work in 1.19 is required for Calico OSS release
[Muzz or Amber] Privileged Containers Status - Enhancement Issue 
https://github.com/kubernetes/enhancements/issues/1981
To be presented at SIG-Node
Ironing out main scenarios in the KEP and testing
[James] Pre-submit tests for Windows - get agreement from the community 
https://github.com/kubernetes/kubernetes/issues/93276#issuecomment-687344627
James to investigate ...30 min time period for blocking pre-submit jobs
https://testgrid.k8s.io/sig-windows-presubmit 
Suggestion: https://testgrid.k8s.io/sig-windows-containerd#aks-engine-azure-windows-master-containerd
Ideas for jobs
Containerd jobs are fairly stable for last one month
Pick jobs that have high success rate and test basic functionality (kubelet, routing, containers standing up)
Dockershim deprecation
Intention to print a warning at kubelet startup if you are using dockership
Sig-node Want to remove dockershim codebase
Need end user docs and phasing outline
CRI API is in alpha still
CSI proxy read out
Vsphere CSI us