<!-- **Note:** When your KEP is complete, all of these comment blocks should be removed. To get started with this template: - [ ] **Pick a hosting SIG.** Make sure that the problem space is something the SIG is interested in taking up. KEPs should not be checked in without a sponsoring SIG. - [ ] **Create an issue in kubernetes/enhancements** When filing an enhancement tracking issue, please make sure to complete all fields in that template. One of the fields asks for a link to the KEP. You can leave that blank until this KEP is filed, and then go back to the enhancement and add the link. - [ ] **Make a copy of this template directory.** Copy this template into the owning SIG's directory and name it `NNNN-short-descriptive-title`, where `NNNN` is the issue number (with no leading-zero padding) assigned to your enhancement above. - [ ] **Fill out as much of the kep.yaml file as you can.** At minimum, you should fill in the "Title", "Authors", "Owning-sig", "Status", and date-related fields. - [ ] **Fill out this file as best you can.** At minimum, you should fill in the "Summary" and "Motivation" sections. These should be easy if you've preflighted the idea of the KEP with the appropriate SIG(s). - [ ] **Create a PR for this KEP.** Assign it to people in the SIG who are sponsoring this process. - [ ] **Merge early and iterate.** Avoid getting hung up on specific details and instead aim to get the goals of the KEP clarified and merged quickly. The best way to do this is to just start with the high-level sections and fill out details incrementally in subsequent PRs. Just because a KEP is merged does not mean it is complete or approved. Any KEP marked as `provisional` is a working document and subject to change. You can denote sections that are under active debate as follows: ``` <<[UNRESOLVED optional short context or usernames ]>> Stuff that is being argued. <<[/UNRESOLVED]>> ``` When editing KEPS, aim for tightly-scoped, single-topic PRs to keep discussions focused. If you disagree with what is already in a document, open a new PR with suggested changes. One KEP corresponds to one "feature" or "enhancement" for its whole lifecycle. You do not need a new KEP to move from beta to GA, for example. If new details emerge that belong in the KEP, edit the KEP. Once a feature has become "implemented", major changes should get new KEPs. The canonical place for the latest set of instructions (and the likely source of this file) is [here](/keps/NNNN-kep-template/README.md). **Note:** Any PRs to move a KEP to `implementable`, or significant changes once it is marked `implementable`, must be approved by each of the KEP approvers. If none of those approvers are still appropriate, then changes to that list should be approved by the remaining approvers and/or the owning SIG (or SIG Architecture for cross-cutting KEPs). --> ### Enhancement Description - One-line enhancement description (can be used as a release note): Auto-refreshing official kubernetes CVE feed - Kubernetes Enhancement Proposal: - Discussion Link: https://docs.google.com/document/d/1GgmmNYN88IZ2v2NBiO3gdU8Riomm0upge_XNVxEYXp0/edit#heading=h.ash02v8wrjia - Primary contact (assignee): @nehaLohia27 - Responsible SIGs: @kubernetes/sig-security - Enhancement target (which target equals to which milestone): - Alpha release target (x.y): 1.25 - Beta release target (x.y): - Stable release target (x.y): - [ ] Alpha - [ ] KEP (`k/enhancements`) update PR(s): - [ ] Code (`k/k`) update PR(s): - [ ] Docs (`k/website`) update PR(s): <!-- Uncomment these as you prepare the enhancement for the next stage - [ ] Beta - [ ] KEP (`k/enhancements`) update PR(s): - [ ] Code (`k/k`) update PR(s): - [ ] Docs (`k/website`) update(s): - [ ] Stable - [ ] KEP (`k/enhancements`) update PR(s): - [ ] Code (`k/k`) update PR(s): - [ ] Docs (`k/website`) update(s): --> _Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently._ # KEP-NNNN: Auto-Refreshing Official CVE Feed <!-- This is the title of your KEP. Keep it short, simple, and descriptive. A good title can help communicate what the KEP is and should be considered as part of any review. --> <!-- A table of contents is helpful for quickly jumping to sections of a KEP and for highlighting any additional information provided beyond the standard KEP template. Ensure the TOC is wrapped with <code>&lt;!-- toc --&rt;&lt;!-- /toc --&rt;</code> tags, and then generate with `hack/update-toc.sh`. --> <!-- toc --> - [Release Signoff Checklist](#release-signoff-checklist) - [Summary](#summary) - [Motivation](#motivation) - [Goals](#goals) - [Non-Goals](#non-goals) - [Proposal](#proposal) - [User Stories (Optional)](#user-stories-optional) - [Story 1](#story-1) - [Story 2](#story-2) - [Notes/Constraints/Caveats (Optional)](#notesconstraintscaveats-optional) - [Risks and Mitigations](#risks-and-mitigations) - [Design Details](#design-details) - [Test Plan](#test-plan) - [Graduation Criteria](#graduation-criteria) - [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy) - [Version Skew Strategy](#version-skew-strategy) - [Production Readiness Review Questionnaire](#production-readiness-review-questionnaire) - [Feature Enablement and Rollback](#feature-enablement-and-rollback) - [Rollout, Upgrade and Rollback Planning](#rollout-upgrade-and-rollback-planning) - [Monitoring Requirements](#monitoring-requirements) - [Dependencies](#dependencies) - [Scalability](#scalability) - [Troubleshooting](#troubleshooting) - [Implementation History](#implementation-history) - [Drawbacks](#drawbacks) - [Alternatives](#alternatives) - [Infrastructure Needed (Optional)](#infrastructure-needed-optional) <!-- /toc --> ## Release Signoff Checklist <!-- **ACTION REQUIRED:** In order to merge code into a release, there must be an issue in [kubernetes/enhancements] referencing this KEP and targeting a release milestone **before the [Enhancement Freeze](https://git.k8s.io/sig-release/releases) of the targeted release**. For enhancements that make changes to code or processes/procedures in core Kubernetes—i.e., [kubernetes/kubernetes], we require the following Release Signoff checklist to be completed. Check these off as they are completed for the Release Team to track. These checklist items _must_ be updated for the enhancement to be released. --> Items marked with (R) are required *prior to targeting to a milestone / release*. - [ ] (R) Enhancement issue in release milestone, which links to KEP dir in [kubernetes/enhancements] (not the initial KEP PR) - [ ] (R) KEP approvers have approved the KEP status as `implementable` - [ ] (R) Design details are appropriately documented - [ ] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input (including test refactors) - [ ] e2e Tests for all Beta API Operations (endpoints) - [ ] (R) Ensure GA e2e tests for meet requirements for [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) - [ ] (R) Minimum Two Week Window for GA e2e tests to prove flake free - [ ] (R) Graduation criteria is in place - [ ] (R) [all GA Endpoints](https://github.com/kubernetes/community/pull/1806) must be hit by [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) - [ ] (R) Production readiness review completed - [ ] (R) Production readiness review approved - [ ] "Implementation History" section is up-to-date for milestone - [ ] User-facing documentation has been created in [kubernetes/website], for publication to [kubernetes.io] - [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes <!-- **Note:** This checklist is iterative and should be reviewed and updated every time this enhancement is being considered for a milestone. --> [kubernetes.io]: https://kubernetes.io/ [kubernetes/enhancements]: https://git.k8s.io/enhancements [kubernetes/kubernetes]: https://git.k8s.io/kubernetes [kubernetes/website]: https://git.k8s.io/website ## Summary {#summary} Currently it is not possible to filter for issues or PRs that are related to CVEs announced by kubernetes. This KEP addresses this concern by labelling this issues or PRs with the new label **official-cve-feed** using the automation. The in-scope issues are the closed issues for which there is a CVE ID and is officially announced as a Kubernetes CVE by SRC in the past. ## Motivation {#motivation} With the growing number of eyes on Kubernetes, the number of CVEs related to Kubernetes have increased. Although most CVEs are regularly fixed that directly or indirectly or transitively impact Kubernetes, there is no single place to programmatically subscribe or pull the data of fixed CVEs, for the end users of Kubernetes. Current options are either broken or incomplete. An auto-refreshing CVE feed will allow end users to programmatically fetch the list of CVEs and allow them to get up to date information from kubernetes community. ### Goals {#goals} Create a periodically auto-refreshing list of official Kubernetes CVEs ## Proposal {#proposal} ### Pre-requisites - [x] https://github.com/kubernetes/test-infra/pull/23428 - [x] Search and Identify closed issues that have a CVE ID e.g. CVE-1001-12345 in the issue description or summary (This search [filter](https://github.com/kubernetes/kubernetes/issues?q=is%3Aissue+in%3Abody+%22CVSS%3A3.%22+label%3Acommittee%2Fsecurity-response+is%3Aclosed+) is giving the most accurate data so far) - [x] Label those issues with `official-cve-feed` using https://docs.github.com/en/rest/reference/issues REST API - [x] https://github.com/kubernetes/committee-security-response/pull/133 ### Goals - Generate a JSON document using the results from the filtered label on `k/k` repo. - Create a Prow job to periodically generate this JSON document. - Update the JSON doc when needed (e.g. when a new CVE is announced) in `k/website` - Using Hugo, publish the list from this JSON document on official k8s website ### Non-Goals - Triage and vulnerability disclosure: This will continue to be done by SRC - CVEs that are identified in build time dependencies and container images. Only official CVEs announced by SRC will be published in the feed. ### User Stories (Optional) <!-- Detail the things that people will be able to do if this KEP is implemented. Include as much detail as possible so that people can understand the "how" of the system. The goal here is to make this feel real for users without getting bogged down. --> #### Story 1 As a K8s end user, I want a list of CVEs with relevant information that I can fetch programmatically, so I can understand when new CVEs are announced #### Story 2 As a K8s maintainer, I want to create a process that auto-updates CVE feed, when SRC announces new CVEs such that I do not have to do extra work to maintain this feed manually ### Risks and Mitigations #### JSON blob construction will fail If this happens, we expect the job too fail. If blob construction fails, the failure will alert the owners of this feature and we will take action as needed. If the failure can not be fixed in a reasonable amount of time, the CVE feed will be stale until it is fixed. In case of urgency from community to get the refreshed feed, JSON blob will be manually updated via usual PR review and approval process. #### Urgent CVE feed refresh In some extenuating circumstances, we may need to update the CVE feed within minutes of the official CVE announcement, instead of waiting for the merge based or periodical website rebuild. In those situations, manual updates to JSON blob using usual PR reviews and approval process can be implemented. ### Storage of CVE feed blob There are two options to store the CVE feed JSON blob: * __Google Cloud Bucket__: A new google cloud bucket can be created where the CVE feed is written and read using `gsutil` tool. * __Git repository__: Store it as a version controlled artifact in one of the kubernetes org websites. Google cloud option has an advantage of transparent updates to JSON blob where the prow job run will be identical everytime. The disadvantage with Google Cloud bucket is that, it will have an unofficial looking URL which would be hard for an end user to deciper it's authenticity and provenance. Advantage with git repository especially `k/website` hosts the JSON blob the domain name in the URL would be `k8s.io/static/security/official-cve-feed.json` which is much more recognizable, intuitive in terms of trust, TLS enabled and unlikely to be spoofed. The disadvantage though is that this might get delayed by PR review and approval process. However, this can be prevented through use of `skip-review` label. ## Design Details Prow job that automates PR creation to k/website when a new CVE is announced will be used to keep the CVE feed always updated. A wrapper will be need to be implemented based on: https://github.com/kubernetes/test-infra/tree/master/robots/pr-creator The flow will look something like this: - Prow job creates JSON blob based on the information found using github label `/official-cve-feed` applied to `k/k` repo - It compares the new generated JSON blob with existing JSON blob on `k/website` page - JSON blob will be hosted in (https://github.com/kubernetes/website/tree/main/static/security/official-cve-feed.json) - If generated JSON blob is different, than existing JSON blob (including blob does not exist), a new PR is created to update that JSON blob - Example Prow job could be something like this: https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes/test-infra/test-infra-trusted.yaml#L604-L677 - The directory under which the JSON blob exists, will also have an OWNERS file made of aliases. The OWNERS file can include SRC alias `security-response-committee` and a new alias for `sig-security-tooling`. This will allow manual modification when needed. - `skip-review` label will need to be added to k/website. This can be used for PRs created by `pr-creator` bot to reduce / bypass approver and reviewer availability ### Test Plan <!-- **Note:** *Not required until targeted at a release.* Consider the following in developing a test plan for this enhancement: - Will there be e2e and integration tests, in addition to unit tests? - How will it be tested in isolation vs with other components? No need to outline all of the test cases, just the general strategy. Anything that would count as tricky in the implementation, and anything particularly challenging to test, should be called out. All code is expected to have adequate tests (eventually with coverage expectations). Please adhere to the [Kubernetes testing guidelines][testing-guidelines] when drafting this test plan. [testing-guidelines]: https://git.k8s.io/community/contributors/devel/sig-testing/testing.md --> ### Graduation Criteria <!-- **Note:** *Not required until targeted at a release.* Define graduation milestones. These may be defined in terms of API maturity, or as something else. The KEP should keep this high-level with a focus on what signals will be looked at to determine graduation. Consider the following in developing the graduation criteria for this enhancement: - [Maturity levels (`alpha`, `beta`, `stable`)][maturity-levels] - [Deprecation policy][deprecation-policy] Clearly define what graduation means by either linking to the [API doc definition](https://kubernetes.io/docs/concepts/overview/kubernetes-api/#api-versioning) or by redefining what graduation means. In general we try to use the same stages (alpha, beta, GA), regardless of how the functionality is accessed. [maturity-levels]: https://git.k8s.io/community/contributors/devel/sig-architecture/api_changes.md#alpha-beta-and-stable-versions [deprecation-policy]: https://kubernetes.io/docs/reference/using-api/deprecation-policy/ Below are some examples to consider, in addition to the aforementioned [maturity levels][maturity-levels]. #### Alpha - Feature implemented behind a feature flag - Initial e2e tests completed and enabled #### Beta - Gather feedback from developers and surveys - Complete features A, B, C - Additional tests are in Testgrid and linked in KEP #### GA - N examples of real-world usage - N installs - More rigorous forms of testing—e.g., downgrade tests and scalability tests - Allowing time for feedback **Note:** Generally we also wait at least two releases between beta and GA/stable, because there's no opportunity for user feedback, or even bug reports, in back-to-back releases. **For non-optional features moving to GA, the graduation criteria must include [conformance tests].** [conformance tests]: https://git.k8s.io/community/contributors/devel/sig-architecture/conformance-tests.md #### Deprecation - Announce deprecation and support policy of the existing flag - Two versions passed since introducing the functionality that deprecates the flag (to address version skew) - Address feedback on usage/changed behavior, provided on GitHub issues - Deprecate the flag --> ### Upgrade / Downgrade Strategy Not applicable ### Version Skew Strategy Not applicable ## Production Readiness Review Questionnaire Not applicable <!-- Production readiness reviews are intended to ensure that features merging into Kubernetes are observable, scalable and supportable; can be safely operated in production environments, and can be disabled or rolled back in the event they cause increased failures in production. See more in the PRR KEP at https://git.k8s.io/enhancements/keps/sig-architecture/1194-prod-readiness. The production readiness review questionnaire must be completed and approved for the KEP to move to `implementable` status and be included in the release. In some cases, the questions below should also have answers in `kep.yaml`. This is to enable automation to verify the presence of the review, and to reduce review burden and latency. The KEP must have a approver from the [`prod-readiness-approvers`](http://git.k8s.io/enhancements/OWNERS_ALIASES) team. Please reach out on the [#prod-readiness](https://kubernetes.slack.com/archives/CPNHUMN74) channel if you need any help or guidance. --> ### Feature Enablement and Rollback <!-- This section must be completed when targeting alpha to a release. --> ###### How can this feature be enabled / disabled in a live cluster? <!-- Pick one of these and delete the rest. --> - [ ] Feature gate (also fill in values in `kep.yaml`) - Feature gate name: - Components depending on the feature gate: - [ ] Other - Describe the mechanism: - Will enabling / disabling the feature require downtime of the control plane? - Will enabling / disabling the feature require downtime or reprovisioning of a node? (Do not assume `Dynamic Kubelet Config` feature is enabled). ###### Does enabling the feature change any default behavior? <!-- Any change of default behavior may be surprising to users or break existing automations, so be extremely careful here. --> ###### Can the feature be disabled once it has been enabled (i.e. can we roll back the enablement)? <!-- Describe the consequences on existing workloads (e.g., if this is a runtime feature, can it break the existing applications?). NOTE: Also set `disable-supported` to `true` or `false` in `kep.yaml`. --> ###### What happens if we reenable the feature if it was previously rolled back? ###### Are there any tests for feature enablement/disablement? <!-- The e2e framework does not currently support enabling or disabling feature gates. However, unit tests in each component dealing with managing data, created with and without the feature, are necessary. At the very least, think about conversion tests if API types are being modified. --> ### Rollout, Upgrade and Rollback Planning <!-- This section must be completed when targeting beta to a release. --> ###### How can a rollout or rollback fail? Can it impact already running workloads? <!-- Try to be as paranoid as possible - e.g., what if some components will restart mid-rollout? Be sure to consider highly-available clusters, where, for example, feature flags will be enabled on some API servers and not others during the rollout. Similarly, consider large clusters and how enablement/disablement will rollout across nodes. --> ###### What specific metrics should inform a rollback? <!-- What signals should users be paying attention to when the feature is young that might indicate a serious problem? --> ###### Were upgrade and rollback tested? Was the upgrade->downgrade->upgrade path tested? <!-- Describe manual testing that was done and the outcomes. Longer term, we may want to require automated upgrade/rollback tests, but we are missing a bunch of machinery and tooling and can't do that now. --> ###### Is the rollout accompanied by any deprecations and/or removals of features, APIs, fields of API types, flags, etc.? <!-- Even if applying deprecation policies, they may still surprise some users. --> ### Monitoring Requirements <!-- This section must be completed when targeting beta to a release. --> ###### How can an operator determine if the feature is in use by workloads? <!-- Ideally, this should be a metric. Operations against the Kubernetes API (e.g., checking if there are objects with field X set) may be a last resort. Avoid logs or events for this purpose. --> ###### How can someone using this feature know that it is working for their instance? <!-- For instance, if this is a pod-related feature, it should be possible to determine if the feature is functioning properly for each individual pod. Pick one more of these and delete the rest. Please describe all items visible to end users below with sufficient detail so that they can verify correct enablement and operation of this feature. Recall that end users cannot usually observe component logs or access metrics. --> - [ ] Events - Event Reason: - [ ] API .status - Condition name: - Other field: - [ ] Other (treat as last resort) - Details: ###### What are the reasonable SLOs (Service Level Objectives) for the enhancement? <!-- This is your opportunity to define what "normal" quality of service looks like for a feature. It's impossible to provide comprehensive guidance, but at the very high level (needs more precise definitions) those may be things like: - per-day percentage of API calls finishing with 5XX errors <= 1% - 99% percentile over day of absolute value from (job creation time minus expected job creation time) for cron job <= 10% - 99.9% of /health requests per day finish with 200 code These goals will help you determine what you need to measure (SLIs) in the next question. --> ###### What are the SLIs (Service Level Indicators) an operator can use to determine the health of the service? <!-- Pick one more of these and delete the rest. --> - [ ] Metrics - Metric name: - [Optional] Aggregation method: - Components exposing the metric: - [ ] Other (treat as last resort) - Details: ###### Are there any missing metrics that would be useful to have to improve observability of this feature? <!-- Describe the metrics themselves and the reasons why they weren't added (e.g., cost, implementation difficulties, etc.). --> ### Dependencies <!-- This section must be completed when targeting beta to a release. --> ###### Does this feature depend on any specific services running in the cluster? <!-- Think about both cluster-level services (e.g. metrics-server) as well as node-level agents (e.g. specific version of CRI). Focus on external or optional services that are needed. For example, if this feature depends on a cloud provider API, or upon an external software-defined storage or network control plane. For each of these, fill in the following—thinking about running existing user workloads and creating new ones, as well as about cluster-level services (e.g. DNS): - [Dependency name] - Usage description: - Impact of its outage on the feature: - Impact of its degraded performance or high-error rates on the feature: --> ### Scalability <!-- For alpha, this section is encouraged: reviewers should consider these questions and attempt to answer them. For beta, this section is required: reviewers must answer these questions. For GA, this section is required: approvers should be able to confirm the previous answers based on experience in the field. --> ###### Will enabling / using this feature result in any new API calls? <!-- Describe them, providing: - API call type (e.g. PATCH pods) - estimated throughput - originating component(s) (e.g. Kubelet, Feature-X-controller) Focusing mostly on: - components listing and/or watching resources they didn't before - API calls that may be triggered by changes of some Kubernetes resources (e.g. update of object X triggers new updates of object Y) - periodic API calls to reconcile state (e.g. periodic fetching state, heartbeats, leader election, etc.) --> ###### Will enabling / using this feature result in introducing new API types? <!-- Describe them, providing: - API type - Supported number of objects per cluster - Supported number of objects per namespace (for namespace-scoped objects) --> ###### Will enabling / using this feature result in any new calls to the cloud provider? <!-- Describe them, providing: - Which API(s): - Estimated increase: --> ###### Will enabling / using this feature result in increasing size or count of the existing API objects? <!-- Describe them, providing: - API type(s): - Estimated increase in size: (e.g., new annotation of size 32B) - Estimated amount of new objects: (e.g., new Object X for every existing Pod) --> ###### Will enabling / using this feature result in increasing time taken by any operations covered by existing SLIs/SLOs? <!-- Look at the [existing SLIs/SLOs]. Think about adding additional work or introducing new steps in between (e.g. need to do X to start a container), etc. Please describe the details. [existing SLIs/SLOs]: https://git.k8s.io/community/sig-scalability/slos/slos.md#kubernetes-slisslos --> ###### Will enabling / using this feature result in non-negligible increase of resource usage (CPU, RAM, disk, IO, ...) in any components? <!-- Things to keep in mind include: additional in-memory state, additional non-trivial computations, excessive access to disks (including increased log volume), significant amount of data sent and/or received over network, etc. This through this both in small and large cases, again with respect to the [supported limits]. [supported limits]: https://git.k8s.io/community//sig-scalability/configs-and-limits/thresholds.md --> ### Troubleshooting <!-- This section must be completed when targeting beta to a release. The Troubleshooting section currently serves the `Playbook` role. We may consider splitting it into a dedicated `Playbook` document (potentially with some monitoring details). For now, we leave it here. --> ###### How does this feature react if the API server and/or etcd is unavailable? ###### What are other known failure modes? <!-- For each of them, fill in the following information by copying the below template: - [Failure mode brief description] - Detection: How can it be detected via metrics? Stated another way: how can an operator troubleshoot without logging into a master or worker node? - Mitigations: What can be done to stop the bleeding, especially for already running user workloads? - Diagnostics: What are the useful log messages and their required logging levels that could help debug the issue? Not required until feature graduated to beta. - Testing: Are there any tests for failure mode? If not, describe why. --> ###### What steps should be taken if SLOs are not being met to determine the problem? ## Implementation History <!-- Major milestones in the lifecycle of a KEP should be tracked in this section. Major milestones might include: - the `Summary` and `Motivation` sections being merged, signaling SIG acceptance - the `Proposal` section being merged, signaling agreement on a proposed design - the date implementation started - the first Kubernetes release where an initial version of the KEP was available - the version of Kubernetes where the KEP graduated to general availability - when the KEP was retired or superseded --> ## Drawbacks <!-- Why should this KEP _not_ be implemented? --> ## Alternatives <!-- What other approaches did you consider, and why did you rule them out? These do not need to be as detailed as the proposal, but should include enough information to express the idea and why it was not acceptable. --> ## Infrastructure Needed (Optional) <!-- Use this section if you need things from the project/SIG. Examples include a new subproject, repos requested, or GitHub details. Listing these here allows a SIG to get the process for these resources started right away. -->