v24.2.2

fix: config reader handles bool types (#218, @tomplus)

v24.2.1

fixed watch.stream bug of not working with apis with follow kwarg (#216, @mcreng)

v24.2.0

Kubernetes API Version: v1.24.2

API Change

Add 2 new options for kube-proxy running in winkernel mode. --forward-healthcheck-vip, if specified as true, health check traffic whose destination is service VIP will be forwarded to kube-proxy's healthcheck service. --root-hnsendpoint-name specifies the name of the hns endpoint for the root network namespace. This option enables the pass-through load balancers like Google's GCLB to correctly health check the backend services. Without this change, the health check packets is dropped, and Windows node will be considered to be unhealthy by those load balancers. (kubernetes/kubernetes#99287, @anfernee)
Added CEL runtime cost calculation into CustomerResource validation. CustomerResource validation will fail if runtime cost exceeds the budget. (kubernetes/kubernetes#108482, @cici37)
Added a new metric webhook_fail_open_count to monitor webhooks that fail to open. (kubernetes/kubernetes#107171, @ltagliamonte-dd)
Adds a new Status subresource in Network Policy objects (kubernetes/kubernetes#107963, @rikatz)
Adds support for InterfaceNamePrefix and BridgeInterface as arguments to --detect-local-mode option and also introduces a new optional --pod-interface-name-prefix and --pod-bridge-interface flags to kube-proxy. (kubernetes/kubernetes#95400, @tssurya)
CEL CRD validation expressions may now reference existing object state using the identifier oldSelf. (kubernetes/kubernetes#108073, @benluddy)
CRD deep copies should no longer contain shallow copies of JSONSchemaProps.XValidations. (kubernetes/kubernetes#107956, @benluddy)
CRD writes will generate validation errors if a CEL validation rule references the identifier oldSelf on a part of the schema that does not support it. (kubernetes/kubernetes#108013, @benluddy)
CSIStorageCapacity.storage.k8s.io: The v1beta1 version of this API is deprecated in favor of v1, and will be removed in v1.27. If a CSI driver supports storage capacity tracking, then it must get deployed with a release of external-provisioner that supports the v1 API. (kubernetes/kubernetes#108445, @pohly)
Custom resource requests with fieldValidation=Strict consistently require apiVersion and kind, matching non-strict requests (kubernetes/kubernetes#109019, @liggitt)
Feature of DefaultPodTopologySpread is graduated to GA (kubernetes/kubernetes#108278, @kerthcet)
Feature of NonPreemptingPriority is graduated to GA (kubernetes/kubernetes#107432, @denkensk)
Feature of PodOverhead is graduated to GA (kubernetes/kubernetes#108441, @pacoxu)
Fixed OpenAPI serialization of the x-kubernetes-validations field (kubernetes/kubernetes#107970, @liggitt)
Fixed failed flushing logs in defer function when kubelet cmd exit 1. (kubernetes/kubernetes#104774, @kerthcet)
Fixes a regression in v1beta1 PodDisruptionBudget handling of strategic merge patch-type API requests for the selector field. Prior to 1.21, these requests would merge matchLabels content and replace matchExpressions content. In 1.21, patch requests touching the selector field started replacing the entire selector. This is consistent with server-side apply and the v1 PodDisruptionBudget behavior, but should not have been changed for v1beta1. (kubernetes/kubernetes#108138, @liggitt)
Improve kubectl's user help commands readability (kubernetes/kubernetes#104736, @lauchokyip)
Indexed Jobs graduated to stable. (kubernetes/kubernetes#107395, @alculquicondor)
Introduce a v1alpha1 networking API for ClusterCIDRConfig (kubernetes/kubernetes#108290, @sarveshr7)
Introduction of a new "sync_proxy_rules_no_local_endpoints_total" proxy metric. This metric represents the number of services with no internal endpoints. The "traffic_policy" label will contain both "internal" or "external". (kubernetes/kubernetes#108930, @MaxRenaud)
JobReadyPods graduates to Beta and it's enabled by default. (kubernetes/kubernetes#107476, @alculquicondor)
Kube-apiserver: --audit-log-version and --audit-webhook-version now only support the default value of audit.k8s.io/v1. The v1alpha1 and v1beta1 audit log versions, deprecated since 1.13, have been removed. (kubernetes/kubernetes#108092, @carlory)
Kube-apiserver: the metadata.selfLink field can no longer be populated by kube-apiserver; it was deprecated in 1.16 and has not been populated by default since 1.20+. (kubernetes/kubernetes#107527, @wojtek-t)
Kubelet external Credential Provider feature is moved to Beta. Credential Provider Plugin and Credential Provider Config API's updated from v1alpha1 to v1beta1 with no API changes. (kubernetes/kubernetes#108847, @adisky)
Make STS available replicas optional again. (kubernetes/kubernetes#109241, @ravisantoshgudimetla)
MaxUnavailable for StatefulSets, allows faster RollingUpdate by taking down more than 1 pod at a time. The number of pods you want to take down during a RollingUpdate is configurable using maxUnavailable parameter. (kubernetes/kubernetes#82162, @krmayankk)
Non-graceful node shutdown handling is enabled for stateful workload failovers (kubernetes/kubernetes#108486, @sonasingh46)
Omit enum declarations from the static openapi file captured at https://git.k8s.io/kubernetes/api/openapi-spec. This file is used to generate API clients, and use of enums in those generated clients (rather than strings) can break forward compatibility with additional future values in those fields. See https://issue.k8s.io/109177 for details. (kubernetes/kubernetes#109178, @liggitt)
OpenAPI V3 is turned on by default (kubernetes/kubernetes#109031, @Jefftree)
Pod affinity namespace selector and cross-namespace quota graduated to GA. The feature gate PodAffinityNamespaceSelector is locked and will be removed in 1.26. (kubernetes/kubernetes#108136, @ahg-g)
Promote IdentifyPodOS feature to beta. (kubernetes/kubernetes#107859, @ravisantoshgudimetla)
Remove a v1alpha1 networking API for ClusterCIDRConfig (kubernetes/kubernetes#109436, @JamesLaverack)
Renamed metrics evictions_number to evictions_total and mark it as stable. The original evictions_number metrics name is marked as "Deprecated" and has been removed in kubernetes 1.23 . (kubernetes/kubernetes#106366, @cyclinder)
Skip x-kubernetes-validations rules if having fundamental error against the OpenAPIv3 schema. (kubernetes/kubernetes#108859, @cici37)
Support for gRPC probes is now in beta. GRPCContainerProbe feature gate is enabled by default. (kubernetes/kubernetes#108522, @SergeyKanzhelev)
Suspend job to GA. The feature gate SuspendJob is locked and will be removed in 1.26. (kubernetes/kubernetes#108129, @ahg-g)
The AnyVolumeDataSource feature is now beta, and the feature gate is enabled by default. In order to provide user feedback on PVCs with data sources, deployers must install the VolumePopulators CRD and the data-source-validator controller. (kubernetes/kubernetes#108736, @bswartz)
The CertificateSigningRequest spec.expirationSeconds API field has graduated to GA. The CSRDuration feature gate for the field is now unconditionally enabled and will be removed in 1.26. (kubernetes/kubernetes#108782, @cfryanr)
The ServerSideFieldValidation feature has graduated to beta and is now enabled by default. Kubectl 1.24 and newer will use server-side validation instead of client-side validation when writing to API servers with the feature enabled. (kubernetes/kubernetes#108889, @kevindelgado)
The ServiceLBNodePortControl feature has graduated to GA. The feature gate will be removed in 1.26. (kubernetes/kubernetes#107027, @uablrek)
The deprecated kube-controller-manager flag '--deployment-controller-sync-period' has been removed, it is not used by the deployment controller. (kubernetes/kubernetes#107178, @SataQiu)
The feature DynamicKubeletConfig has been removed from the kubelet. (kubernetes/kubernetes#106932, @SergeyKanzhelev)
The infrastructure for contextual logging is complete (feature gate implemented, JSON backend ready). (kubernetes/kubernetes#108995, @pohly)
This adds an optional timeZone field as part of the CronJob spec to support running cron jobs in a specific time zone. (kubernetes/kubernetes#108032, @deejross)
Updated the default API priority-and-fairness config to avoid endpoint/configmaps operations from controller-manager to all match leader-election priority level. (kubernetes/kubernetes#106725, @wojtek-t)
topologySpreadConstraints includes minDomains field to limit the minimum number of topology domains. (kubernetes/kubernetes#107674, @sanposhiho)
Introduce a v1alpha1 networking API for ClusterCIDRConfig (kubernetes/kubernetes#108290, @sarveshr7) [SIG API Machinery, Apps, Auth, CLI, Cloud Provider, Instrumentation, Network and Testing]
Introduction of a new "sync_proxy_rules_no_local_endpoints_total" proxy metric. This metric represents the number of services with no internal endpoints. The "traffic_policy" label will contain both "internal" or "external". (kubernetes/kubernetes#108930, @MaxRenaud) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Instrumentation, Network, Node, Release, Scheduling, Storage, Testing and Windows]
Make STS available replicas optional again, (kubernetes/kubernetes#109241, @ravisantoshgudimetla) [SIG API Machinery and Apps]
Omit enum declarations from the static openapi file captured at https://git.k8s.io/kubernetes/api/openapi-spec. This file is used to generate API clients, and use of enums in those generated clients (rather than strings) can break forward compatibility with additional future values in those fields. See https://issue.k8s.io/109177 for details. (kubernetes/kubernetes#109178, @liggitt) [SIG API Machinery and Auth]
Remove a v1alpha1 networking API for ClusterCIDRConfig (kubernetes/kubernetes#109436, @JamesLaverack) [SIG API Machinery, Apps, Auth, CLI, Network and Testing]
The deprecated kube-controller-manager flag '--deployment-controller-sync-period' has been removed, it is not used by the deployment controller. (kubernetes/kubernetes#107178, @SataQiu) [SIG API Machinery and Apps]
Adds a new Status subresource in Network Policy objects (kubernetes/kubernetes#107963, @rikatz) [SIG API Machinery, Apps, Network and Testing]
Adds support for "InterfaceNamePrefix" and "BridgeInterface" as arguments to --detect-local-mode option and also introduces a new optional --pod-interface-name-prefix and --pod-bridge-interface flags to kube-proxy. (kubernetes/kubernetes#95400, @tssurya) [SIG API Machinery and Network]
CEL CRD validation expressions may now reference existing object state using the identifier oldSelf. (kubernetes/kubernetes#108073, @benluddy) [SIG API Machinery and Testing]
CSIStorageCapacity.storage.k8s.io: The v1beta1 version of this API is deprecated in favor of v1, and will be removed in v1.27. If a CSI driver supports storage capacity tracking, then it must get deployed with a release of external-provisioner that supports the v1 API. (kubernetes/kubernetes#108445, @pohly) [SIG API Machinery, Architecture, Auth, Scheduling, Storage and Testing]
Custom resource requests with fieldValidation=Strict consistently require apiVersion and kind, matching non-strict requests (kubernetes/kubernetes#109019, @liggitt) [SIG API Machinery]
Improve kubectl's user help commands readability (kubernetes/kubernetes#104736, @lauchokyip) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Cluster Lifecycle, Contributor Experience, Instrumentation, Network, Node, Release, Scalability, Scheduling, Security, Storage, Testing and Windows]
Indexed Jobs graduates to stable (kubernetes/kubernetes#107395, @alculquicondor) [SIG Apps, Architecture and Testing]
Introduce a v1alpha1 networking API for ClusterCIDRConfig (kubernetes/kubernetes#108290, @sarveshr7) [SIG API Machinery, Apps, Auth, CLI, Cloud Provider, Instrumentation, Network and Testing]
JobReadyPods graduates to Beta and it's enabled by default. (kubernetes/kubernetes#107476, @alculquicondor) [SIG API Machinery, Apps and Testing]
Kubelet external Credential Provider feature is moved to Beta. Credential Provider Plugin and Credential Provider Config API's updated from v1alpha1 to v1beta1 with no API changes. (kubernetes/kubernetes#108847, @adisky) [SIG API Machinery and Node]
MaxUnavailable for StatefulSets, allows faster RollingUpdate by taking down more than 1 pod at a time. The number of pods you want to take down during a RollingUpdate is configurable using maxUnavailable parameter. (kubernetes/kubernetes#82162, @krmayankk) [SIG API Machinery and Apps]
Non graceful node shutdown handling. (kubernetes/kubernetes#108486, @sonasingh46) [SIG Apps, Node and Storage]
OpenAPI V3 is turned on by default (kubernetes/kubernetes#109031, @Jefftree) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Scheduling, Storage and Testing]
Promote IdentifyPodOS feature to beta. (kubernetes/kubernetes#107859, @ravisantoshgudimetla) [SIG API Machinery, Apps, Node, Testing and Windows]
Skip x-kubernetes-validations rules if having fundamental error against OpenAPIv3 schema. (kubernetes/kubernetes#108859, @cici37) [SIG API Machinery and Testing]
Support for gRPC probes is now in beta. GRPCContainerProbe feature gate is enabled by default. (kubernetes/kubernetes#108522, @SergeyKanzhelev) [SIG API Machinery, Apps, Node and Testing]
The AnyVolumeDataSource feature is now beta, and the feature gate is enabled by default. In order to provide user feedback on PVCs with data sources, deployers must install the VolumePopulators CRD and the data-source-validator controller. (kubernetes/kubernetes#108736, @bswartz) [SIG Apps, Storage and Testing]
The ServerSideFieldValidation feature has graduated to beta and is now enabled by default. Kubectl 1.24 and newer will use server-side validation instead of client-side validation when writing to API servers with the feature enabled. (kubernetes/kubernetes#108889, @kevindelgado) [SIG API Machinery, Architecture, CLI and Testing]
The infrastructure for contextual logging is complete (feature gate implemented, JSON backend ready). (kubernetes/kubernetes#108995, @pohly) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Scheduling and Testing]
This adds an optional timeZone field as part of the CronJob spec to support running cron jobs in a specific time zone. (kubernetes/kubernetes#108032, @deejross) [SIG API Machinery and Apps]
Add 2 new options for kube-proxy running in winkernel mode. --forward-healthcheck-vip, if specified as true, health check traffic whose destination is service VIP will be forwarded to kube-proxy's healthcheck service. --root-hnsendpoint-name specifies the name of the hns endpoint for the root network namespace. This option enables the pass-through load balancers like Google's GCLB to correctly health check the backend services. Without this change, the health check packets is dropped, and Windows node will be considered to be unhealthy by those load balancers. (kubernetes/kubernetes#99287, @anfernee) [SIG API Machinery, Cloud Provider, Network, Testing and Windows]
Added CEL runtime cost calculation into CustomerResource validation. CustomerResource validation will fail if runtime cost exceeds the budget. (kubernetes/kubernetes#108482, @cici37) [SIG API Machinery]
CRD writes will generate validation errors if a CEL validation rule references the identifier "oldSelf" on a part of the schema that does not support it. (kubernetes/kubernetes#108013, @benluddy) [SIG API Machinery]
Feature of DefaultPodTopologySpread is graduated to GA (kubernetes/kubernetes#108278, @kerthcet) [SIG Scheduling]
Feature of PodOverhead is graduated to GA (kubernetes/kubernetes#108441, @pacoxu) [SIG API Machinery, Apps, Node and Scheduling]
Fixes a regression in v1beta1 PodDisruptionBudget handling of "strategic merge patch"-type API requests for the selector field. Prior to 1.21, these requests would merge matchLabels content and replace matchExpressions content. In 1.21, patch requests touching the selector field started replacing the entire selector. This is consistent with server-side apply and the v1 PodDisruptionBudget behavior, but should not have been changed for v1beta1. (kubernetes/kubernetes#108138, @liggitt) [SIG Apps, Auth and Testing]
Kube-apiserver: --audit-log-version and --audit-webhook-version now only support the default value of audit.k8s.io/v1. The v1alpha1 and v1beta1 audit log versions, deprecated since 1.13, have been removed. (kubernetes/kubernetes#108092, @carlory) [SIG API Machinery, Auth and Testing]
Pod-affinity namespace selector and cross-namespace quota graduated to GA. The feature gate PodAffinityNamespaceSelector is locked and will be removed in 1.26. (kubernetes/kubernetes#108136, @ahg-g) [SIG API Machinery, Apps, Scheduling and Testing]
Suspend job to GA. The feature gate SuspendJob is locked and will be removed in 1.26. (kubernetes/kubernetes#108129, @ahg-g) [SIG Apps and Testing]
The CertificateSigningRequest spec.expirationSeconds API field has graduated to GA. The CSRDuration feature gate for the field is now unconditionally enabled and will be removed in 1.26. (kubernetes/kubernetes#108782, @cfryanr) [SIG API Machinery, Apps, Auth, Instrumentation and Testing]
TopologySpreadConstraints includes minDomains field to limit the minimum number of topology domains. (kubernetes/kubernetes#107674, @sanposhiho) [SIG API Machinery, Apps and Scheduling]
CRD deep copies should no longer contain shallow copies of JSONSchemaProps.XValidations. (kubernetes/kubernetes#107956, @benluddy) [SIG API Machinery]
Feature of NonPreemptingPriority is graduated to GA (kubernetes/kubernetes#107432, @denkensk) [SIG Apps, Scheduling and Testing]
Fix OpenAPI serialization of the x-kubernetes-validations field (kubernetes/kubernetes#107970, @liggitt) [SIG API Machinery]
Kube-apiserver: the metadata.selfLink field can no longer be populated by kube-apiserver; it was deprecated in 1.16 and has not been populated by default in 1.20+. (kubernetes/kubernetes#107527, @wojtek-t) [SIG API Machinery, Apps, Auth, Autoscaling, CLI, Cloud Provider, Network, Scheduling, Storage and Testing]
Add a new metric webhook_fail_open_count to monitor webhooks that fail open (kubernetes/kubernetes#107171, @ltagliamonte-dd) [SIG API Machinery and Instrumentation]
Fix failed flushing logs in defer function when kubelet cmd exit 1. (kubernetes/kubernetes#104774, @kerthcet) [SIG Node and Scheduling]
Rename metrics evictions_number to evictions_total and mark it as stable. The original evictions_number metrics name is marked as "Deprecated" and will be removed in kubernetes 1.23 (kubernetes/kubernetes#106366, @cyclinder) [SIG API Machinery, Apps, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scalability, Scheduling, Storage, Testing and Windows]
The ServiceLBNodePortControl feature graduates to GA. The feature gate will be removed in 1.26. (kubernetes/kubernetes#107027, @uablrek) [SIG Network and Testing]
The feature DynamicKubeletConfig is removed from the kubelet. (kubernetes/kubernetes#106932, @SergeyKanzhelev) [SIG Apps, Auth, Instrumentation, Node and Testing]
Update default API priority-and-fairness config to avoid endpoint/configmaps operations from controller-manager to all match leader-election priority level. (kubernetes/kubernetes#106725, @wojtek-t) [SIG API Machinery]

v23.6.0

API Change

Omits alpha-level enums from the static openapi file captured in api/openapi-spec (kubernetes/kubernetes#109179, @liggitt) [SIG Apps and Auth]
Fixes a regression in v1beta1 PodDisruptionBudget handling of "strategic merge patch"-type API requests for the selector field. Prior to 1.21, these requests would merge matchLabels content and replace matchExpressions content. In 1.21, patch requests touching the selector field started replacing the entire selector. This is consistent with server-side apply and the v1 PodDisruptionBudget behavior, but should not have been changed for v1beta1. (kubernetes/kubernetes#108139, @liggitt) [SIG Auth and Testing]
Fix OpenAPI serialization of the x-kubernetes-validations field (kubernetes/kubernetes#108030, @liggitt) [SIG API Machinery]
A new field omitManagedFields has been added to both audit.Policy and audit.PolicyRule so cluster operators can opt in to omit managed fields of the request and response bodies from being written to the API audit log. (kubernetes/kubernetes#94986, @tkashem) [SIG API Machinery, Auth, Cloud Provider and Testing]
A small regression in Service updates was fixed. The circumstances are so unlikely that probably nobody would ever hit it. (kubernetes/kubernetes#104601, @thockin)
Added a feature gate StatefulSetAutoDeletePVC, which allows PVCs automatically created for StatefulSet pods to be automatically deleted. (kubernetes/kubernetes#99728, @mattcary)
Client-go impersonation config can specify a UID to pass impersonated uid information through in requests. (kubernetes/kubernetes#104483, @margocrawf)
Create HPA v2 from v2beta2 with some fields changed. (kubernetes/kubernetes#102534, @wangyysde) [SIG API Machinery, Apps, Auth, Autoscaling and Testing]
Ephemeral containers graduated to beta and are now available by default. (kubernetes/kubernetes#105405, @verb)
Fix kube-proxy regression on UDP services because the logic to detect stale connections was not considering if the endpoint was ready. (kubernetes/kubernetes#106163, @aojea) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Contributor Experience, Instrumentation, Network, Node, Release, Scalability, Scheduling, Storage, Testing and Windows]
If a conflict occurs when creating an object with generateName, the server now returns an "AlreadyExists" error with a retry option. (kubernetes/kubernetes#104699, @vincepri)
Implement support for recovering from volume expansion failures (kubernetes/kubernetes#106154, @gnufied) [SIG API Machinery, Apps and Storage]
In kubelet, log verbosity and flush frequency can also be configured via the configuration file and not just via command line flags. In other commands (kube-apiserver, kube-controller-manager), the flags are listed in the "Logs flags" group and not under "Global" or "Misc". The type for -vmodule was made a bit more descriptive (pattern=N,... instead of moduleSpec). (kubernetes/kubernetes#106090, @pohly) [SIG API Machinery, Architecture, CLI, Cluster Lifecycle, Instrumentation, Node and Scheduling]
Introduce OS field in the PodSpec (kubernetes/kubernetes#104693, @ravisantoshgudimetla)
Introduce v1beta3 API for scheduler. This version
- increases the weight of user specifiable priorities. The weights of following priority plugins are increased
  - TaintTolerations to 3 - as leveraging node tainting to group nodes in the cluster is becoming a widely-adopted practice
  - NodeAffinity to 2
  - InterPodAffinity to 2
- Won't have HealthzBindAddress, MetricsBindAddress fields (kubernetes/kubernetes#104251, @ravisantoshgudimetla)
Introduce v1beta2 for Priority and Fairness with no changes in API spec. (kubernetes/kubernetes#104399, @tkashem)
JSON log output is configurable and now supports writing info messages to stdout and error messages to stderr. Info messages can be buffered in memory. The default is to write both to stdout without buffering, as before. (kubernetes/kubernetes#104873, @pohly)
JobTrackingWithFinalizers graduates to beta. Feature is enabled by default. (kubernetes/kubernetes#105687, @alculquicondor)
Kube-apiserver: Fixes handling of CRD schemas containing literal null values in enums. (kubernetes/kubernetes#104969, @liggitt)
Kube-apiserver: The rbac.authorization.k8s.io/v1alpha1 API version is removed; use the rbac.authorization.k8s.io/v1 API, available since v1.8. The scheduling.k8s.io/v1alpha1 API version is removed; use the scheduling.k8s.io/v1 API, available since v1.14. (kubernetes/kubernetes#104248, @liggitt)
Kube-scheduler: support for configuration file version v1beta1 is removed. Update configuration files to v1beta2(xref: kubernetes/enhancements#2901) or v1beta3 before upgrading to 1.23. (kubernetes/kubernetes#104782, @kerthcet)
KubeSchedulerConfiguration provides a new field MultiPoint which will register a plugin for all valid extension points (kubernetes/kubernetes#105611, @damemi) [SIG Scheduling and Testing]
Kubelet should reject pods whose OS doesn't match the node's OS label. (kubernetes/kubernetes#105292, @ravisantoshgudimetla) [SIG Apps and Node]
Kubelet: turn the KubeletConfiguration v1beta1 ResolverConfig field from a string to *string. (kubernetes/kubernetes#104624, @Haleygo)
Kubernetes is now built using go 1.17. (kubernetes/kubernetes#103692, @justaugustus)
Performs strict server side schema validation requests via the fieldValidation=[Strict,Warn,Ignore]. (kubernetes/kubernetes#105916, @kevindelgado)
Promote IPv6DualStack feature to stable. Controller Manager flags for the node IPAM controller have slightly changed:
1. When configuring a dual-stack cluster, the user must specify both --node-cidr-mask-size-ipv4 and --node-cidr-mask-size-ipv6 to set the per-node IP mask sizes, instead of the previous --node-cidr-mask-size flag.
2. The --node-cidr-mask-size flag is mutually exclusive with --node-cidr-mask-size-ipv4 and --node-cidr-mask-size-ipv6.
3. Single-stack clusters do not need to change, but may choose to use the more specific flags. Users can use either the older --node-cidr-mask-size flag or one of the newer --node-cidr-mask-size-ipv4 or --node-cidr-mask-size-ipv6 flags to configure the per-node IP mask size, provided that the flag's IP family matches the cluster's IP family (--cluster-cidr). (kubernetes/kubernetes#104691, @khenidak)
Remove NodeLease feature gate that was graduated and locked to stable in 1.17 release. (kubernetes/kubernetes#105222, @cyclinder)
Removed deprecated --seccomp-profile-root/seccompProfileRoot config. (kubernetes/kubernetes#103941, @saschagrunert)
Since golang 1.17 both net.ParseIP and net.ParseCIDR rejects leading zeros in the dot-decimal notation of IPv4 addresses, Kubernetes will keep allowing leading zeros on IPv4 address to not break the compatibility. IMPORTANT: Kubernetes interprets leading zeros on IPv4 addresses as decimal, users must not rely on parser alignment to not being impacted by the associated security advisory: CVE-2021-29923 golang standard library "net" - Improper Input Validation of octal literals in golang 1.16.2 and below standard library "net" results in indeterminate SSRF & RFI vulnerabilities. Reference: https://nvd.nist.gov/vuln/detail/CVE-2021-29923 (kubernetes/kubernetes#104368, @aojea)
StatefulSet minReadySeconds is promoted to beta. (kubernetes/kubernetes#104045, @ravisantoshgudimetla)
Support pod priority based node graceful shutdown. (kubernetes/kubernetes#102915, @wzshiming)
The "Generic Ephemeral Volume" feature graduates to GA. It is now enabled unconditionally. (kubernetes/kubernetes#105609, @pohly)
The Kubelet's --register-with-taints option is now available via the Kubelet config file field registerWithTaints (kubernetes/kubernetes#105437, @cmssczy) [SIG Node and Scalability]
The CSIDriver.Spec.StorageCapacity can now be modified. (kubernetes/kubernetes#101789, @pohly)
The CSIVolumeFSGroupPolicy feature has moved from beta to GA. (kubernetes/kubernetes#105940, @dobsonj)
The IngressClass.Spec.Parameters.Namespace field is now GA. (kubernetes/kubernetes#104636, @hbagdi)
The Service.spec.ipFamilyPolicy field is now required in order to create or update a Service as dual-stack. This is a breaking change from the beta behavior. Previously the server would try to infer the value of that field from either ipFamilies or clusterIPs, but that caused ambiguity on updates. Users who want a dual-stack Service MUST specify ipFamilyPolicy as either "PreferDualStack" or "RequireDualStack". (kubernetes/kubernetes#96684, @thockin)
The TTLAfterFinished feature gate is now GA and enabled by default. (kubernetes/kubernetes#105219, @sahilvv)
The kube-controller-manager supports --concurrent-ephemeralvolume-syncs flag to set the number of ephemeral volume controller workers. (kubernetes/kubernetes#102981, @SataQiu)
The legacy scheduler policy config is removed in v1.23, the associated flags policy-config-file, policy-configmap, policy-configmap-namespace and use-legacy-policy-config are also removed. Migrate to Component Config instead, see https://kubernetes.io/docs/reference/scheduling/config/ for details. (kubernetes/kubernetes#105424, @kerthcet)
Track the number of Pods with a Ready condition in Job status. The feature is alpha and needs the feature gate JobReadyPods to be enabled. (kubernetes/kubernetes#104915, @alculquicondor)
Users of LogFormatRegistry in component-base must update their code to use the logr v1.0.0 API. The JSON log output now uses the format from go-logr/zapr (no v field for error messages, additional information for invalid calls) and has some fixes (correct source code location for warnings about invalid log calls). (kubernetes/kubernetes#104103, @pohly)
Validation rules for Custom Resource Definitions can be written in the CEL expression language using the x-kubernetes-validations extension in OpenAPIv3 schemas (alpha). This is gated by the alpha "CustomResourceValidationExpressions" feature gate. (kubernetes/kubernetes#106051, @jpbetz) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Storage and Testing]
Add gRPC probe to Pod.Spec.Container.{Liveness,Readiness,Startup}Probe (#106463, @SergeyKanzhelev) [SIG API Machinery, Apps, CLI, Node and Testing]
Adds a feature gate StatefulSetAutoDeletePVC, which allows PVCs automatically created for StatefulSet pods to be automatically deleted. (#99728, @mattcary) [SIG API Machinery, Apps, Auth and Testing]
Performs strict server side schema validation requests via the fieldValidation=[Strict,Warn,Ignore] query parameter. (#105916, @kevindelgado) [SIG API Machinery, Apps, Auth, Cloud Provider and Testing]
Support pod priority based node graceful shutdown (#102915, @wzshiming) [SIG Node and Testing]
A new field omitManagedFields has been added to both audit.Policy and audit.PolicyRule so cluster operators can opt in to omit managed fields of the request and response bodies from being written to the API audit log. (#94986, @tkashem) [SIG API Machinery, Auth, Cloud Provider and Testing]
Create HPA v2 from v2beta2 with some fields changed. (#102534, @wangyysde) [SIG API Machinery, Apps, Auth, Autoscaling and Testing]
Fix kube-proxy regression on UDP services because the logic to detect stale connections was not considering if the endpoint was ready. (#106163, @aojea) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Contributor Experience, Instrumentation, Network, Node, Release, Scalability, Scheduling, Storage, Testing and Windows]
Implement support for recovering from volume expansion failures (#106154, @gnufied) [SIG API Machinery, Apps and Storage]
In kubelet, log verbosity and flush frequency can also be configured via the configuration file and not just via command line flags. In other commands (kube-apiserver, kube-controller-manager), the flags are listed in the "Logs flags" group and not under "Global" or "Misc". The type for -vmodule was made a bit more descriptive (pattern=N,... instead of moduleSpec). (#106090, @pohly) [SIG API Machinery, Architecture, CLI, Cluster Lifecycle, Instrumentation, Node and Scheduling]
IngressClass.Spec.Parameters.Namespace field is now GA. (#104636, @hbagdi) [SIG Network and Testing]
KubeSchedulerConfiguration provides a new field MultiPoint which will register a plugin for all valid extension points (#105611, @damemi) [SIG Scheduling and Testing]
Kubelet should reject pods whose OS doesn't match the node's OS label. (#105292, @ravisantoshgudimetla) [SIG Apps and Node]
The CSIVolumeFSGroupPolicy feature has moved from beta to GA. (#105940, @dobsonj) [SIG Storage]
The Kubelet's --register-with-taints option is now available via the Kubelet config file field registerWithTaints (#105437, @cmssczy) [SIG Node and Scalability]
Validation rules for Custom Resource Definitions can be written in the CEL expression language using the x-kubernetes-validations extension in OpenAPIv3 schemas (alpha). This is gated by the alpha "CustomResourceValidationExpressions" feature gate. (#106051, @jpbetz) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Storage and Testing]
Ephemeral containers have reached beta maturity and are now available by default. (#105405, @verb) [SIG API Machinery, Apps, Node and Testing]
Introduce OS field in the Pod Spec (#104693, @ravisantoshgudimetla) [SIG API Machinery and Apps]
Introduce v1beta3 api for scheduler. This version
- increases the weight of user specifiable priorities. The weights of following priority plugins are increased
  - TaintTolerations to 3 - as leveraging node tainting to group nodes in the cluster is becoming a widely-adopted practice
  - NodeAffinity to 2
  - InterPodAffinity to 2
- Won't have HealthzBindAddress, MetricsBindAddress fields (#104251, @ravisantoshgudimetla) [SIG Scheduling and Testing]
JSON log output is configurable and now supports writing info messages to stdout and error messages to stderr. Info messages can be buffered in memory. The default is to write both to stdout without buffering, as before. (#104873, @pohly) [SIG API Machinery, Architecture, CLI, Cluster Lifecycle, Instrumentation, Node and Scheduling]
JobTrackingWithFinalizers graduates to beta. Feature is enabled by default. (#105687, @alculquicondor) [SIG Apps and Testing]
Remove NodeLease feature gate that was graduated and locked to stable in 1.17 release. (#105222, @cyclinder) [SIG Apps, Node and Testing]
TTLAfterFinished is now GA and enabled by default (#105219, @sahilvv) [SIG API Machinery, Apps, Auth and Testing]
The "Generic Ephemeral Volume" feature graduates to GA. It is now enabled unconditionally. (#105609, @pohly) [SIG API Machinery, Apps, Auth, Node, Scheduling, Storage and Testing]
The legacy scheduler policy config is removed in v1.23, the associated flags policy-config-file, policy-configmap, policy-configmap-namespace and use-legacy-policy-config are also removed. Migrate to Component Config instead, see https://kubernetes.io/docs/reference/scheduling/config/ for details. (#105424, @kerthcet) [SIG Scheduling and Testing]
Track the number of Pods with a Ready condition in Job status. The feature is alpha and needs the feature gate JobReadyPods to be enabled. (#104915, @alculquicondor) [SIG API Machinery, Apps, CLI and Testing]
Client-go impersonation config can specify a UID to pass impersonated uid information through in requests. (kubernetes/kubernetes#104483, @margocrawf) [SIG API Machinery, Auth and Testing]
IPv6DualStack feature moved to stable. Controller Manager flags for the node IPAM controller have slightly changed:
1. When configuring a dual-stack cluster, the user must specify both --node-cidr-mask-size-ipv4 and --node-cidr-mask-size-ipv6 to set the per-node IP mask sizes, instead of the previous --node-cidr-mask-size flag.
2. The --node-cidr-mask-size flag is mutually exclusive with --node-cidr-mask-size-ipv4 and --node-cidr-mask-size-ipv6.
3. Single-stack clusters do not need to change, but may choose to use the more specific flags. Users can use either the older --node-cidr-mask-size flag or one of the newer --node-cidr-mask-size-ipv4 or --node-cidr-mask-size-ipv6 flags to configure the per-node IP mask size, provided that the flag's IP family matches the cluster's IP family (--cluster-cidr). (kubernetes/kubernetes#104691, @khenidak) [SIG API Machinery, Apps, Auth, Cloud Provider, Cluster Lifecycle, Network, Node and Testing]
Kubelet: turn the KubeletConfiguration v1beta1 ResolverConfig field from a string to *string. (kubernetes/kubernetes#104624, @Haleygo) [SIG Cluster Lifecycle and Node]
A small regression in Service updates was fixed. The circumstances are so unlikely that probably nobody would ever hit it. (kubernetes/kubernetes#104601, @thockin) [SIG Network]
Introduce v1beta2 for Priority and Fairness with no changes in API spec (kubernetes/kubernetes#104399, @tkashem) [SIG API Machinery and Testing]
Kube-apiserver: Fixes handling of CRD schemas containing literal null values in enums. (kubernetes/kubernetes#104969, @liggitt) [SIG API Machinery, Apps and Network]
Kubelet: turn the KubeletConfiguration v1beta1 ResolverConfig field from a string to *string. (kubernetes/kubernetes#104624, @Haleygo) [SIG Cluster Lifecycle and Node]
Kubernetes is now built using go1.17 (kubernetes/kubernetes#103692, @justaugustus) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scheduling, Storage and Testing]
Removed deprecated --seccomp-profile-root/seccompProfileRoot config (kubernetes/kubernetes#103941, @saschagrunert) [SIG Node]
Since golang 1.17 both net.ParseIP and net.ParseCIDR rejects leading zeros in the dot-decimal notation of IPv4 addresses. Kubernetes will keep allowing leading zeros on IPv4 address to not break the compatibility. IMPORTANT: Kubernetes interprets leading zeros on IPv4 addresses as decimal, users must not rely on parser alignment to not being impacted by the associated security advisory: CVE-2021-29923 golang standard library "net" - Improper Input Validation of octal literals in golang 1.16.2 and below standard library "net" results in indeterminate SSRF & RFI vulnerabilities. Reference: https://nvd.nist.gov/vuln/detail/CVE-2021-29923 (kubernetes/kubernetes#104368, @aojea) [SIG API Machinery, Apps, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scalability, Scheduling, Storage and Testing]
StatefulSet minReadySeconds is promoted to beta (kubernetes/kubernetes#104045, @ravisantoshgudimetla) [SIG Apps and Testing]
The Service.spec.ipFamilyPolicy field is now required in order to create or update a Service as dual-stack. This is a breaking change from the beta behavior. Previously the server would try to infer the value of that field from either ipFamilies or clusterIPs, but that caused ambiguity on updates. Users who want a dual-stack Service MUST specify ipFamilyPolicy as either "PreferDualStack" or "RequireDualStack". (kubernetes/kubernetes#96684, @thockin) [SIG API Machinery, Apps, Network and Testing]
Users of LogFormatRegistry in component-base must update their code to use the logr v1.0.0 API. The JSON log output now uses the format from go-logr/zapr (no v field for error messages, additional information for invalid calls) and has some fixes (correct source code location for warnings about invalid log calls). (kubernetes/kubernetes#104103, @pohly) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation and Storage]
When creating an object with generateName, if a conflict occurs the server now returns an AlreadyExists error with a retry option. (kubernetes/kubernetes#104699, @vincepri) [SIG API Machinery]
CSIDriver.Spec.StorageCapacity can now be modified. (kubernetes/kubernetes#101789, @pohly) [SIG Storage]
Kube-apiserver: The rbac.authorization.k8s.io/v1alpha1 API version is removed; use the rbac.authorization.k8s.io/v1 API, available since v1.8. The scheduling.k8s.io/v1alpha1 API version is removed; use the scheduling.k8s.io/v1 API, available since v1.14. (kubernetes/kubernetes#104248, @liggitt) [SIG API Machinery, Auth, Network and Testing]
Kube-controller-manager supports '--concurrent-ephemeralvolume-syncs' flag to set the number of ephemeral volume controller workers. (kubernetes/kubernetes#102981, @SataQiu) [SIG API Machinery and Apps]

v22.6.5

feat: periodically refresh ServiceAccount tokens (#205, @JacobHenner)

v22.6.4

fix regenerate library with correct crd spec (#198, @tomplus)

v22.6.3

feat: configurable heartbeat value for WsApiClient (#195, @GlassOfWhiskey)
feat: added optional explicit config parameter to InClusterConfigLoader (#196, @GlassOfWhiskey)

v22.6.2

fix: add patch to update read buffer size (#192, @tomplus)

v22.6.1

fix: passing Bearer token in authorization header (#190, @tomplus)

v22.6.0

feat: regenerate library with OpenAPI Generator v5.4.0 (#185, @tomplus) Some changes from OpenAPI Generator Changelog (previous version v4.3.0):
- feat(python-asyncio): add support for proxy config using system env vars
- add option to select/detect content-type
- add socket_options to configuration for the rest client
- correct return types if multiple responses are defined
- subclass Python exceptions
- fix Python UTF-8 Encoding Issue
feat: add support for aiohttp proxy config using env vars (#173, @icamposrivera)

API Change

Kube-apiserver: Fixes handling of CRD schemas containing literal null values in enums (#104988, @liggitt) [SIG API Machinery, Apps and Network]
A new score extension for NodeResourcesFit plugin that merges the functionality of NodeResourcesLeastAllocated, NodeResourcesMostAllocated, RequestedToCapacityRatio plugins, which are marked as deprecated as of v1beta2. In v1beta1, the three plugins can still be used in v1beta1 but not at the same time with the score extension of NodeResourcesFit. (kubernetes/kubernetes#101822, @yuzhiquan)
A value of Auto is now a valid for the service.kubernetes.io/topology-aware-hints annotation. (kubernetes/kubernetes#100728, @robscott)
Add DataSourceRef alpha field to PVC spec, which allows contents other than PVCs and VolumeSnapshots to be data sources. (kubernetes/kubernetes#103276, @bswartz)
Add PersistentVolumeClaimDeletePoilcy to StatefulSet API. (kubernetes/kubernetes#99378, @mattcary)
Add a new Priority and Fairness rule that exempts all probes (/readyz, /healthz, /livez) to prevent restarting of healthy kube-apiserver instance by kubelet. (kubernetes/kubernetes#100678, @tkashem)
Add alpha support for HostProcess containers on Windows (kubernetes/kubernetes#99576, @marosset) [SIG API Machinery, Apps, Node, Testing and Windows]
Add distributed tracing to the kube-apiserver. It is can be enabled with the feature gate APIServerTracing (kubernetes/kubernetes#94942, @dashpole)
Add three metrics to the job controller to monitor if a job works in healthy condition. IndexedJob has been promoted to Beta. (kubernetes/kubernetes#101292, @AliceZhang2016)
Added field .status.uncountedTerminatedPods to the Job resource. This field is used by the job controller to keep track of finished pods before adding them to the Job status counters. Pods created by the job controller get the finalizer batch.kubernetes.io/job-tracking Jobs that are tracked using this mechanism get the annotation batch.kubernetes.io/job-tracking. This is a temporary measure. Two releases after this feature graduates to beta, the annotation won't be added to Jobs anymore. (kubernetes/kubernetes#98817, @alculquicondor)
Added new kubelet alpha feature SeccompDefault. This feature enables falling back to the RuntimeDefault (former runtime/default) seccomp profile if nothing else is specified in the pod/container SecurityContext or the pod annotation level. To use the feature, enable the feature gate as well as set the kubelet configuration option SeccompDefault (--seccomp-default) to true. (kubernetes/kubernetes#101943, @saschagrunert) [SIG Node]
Adds the ReadWriteOncePod access mode for PersistentVolumes and PersistentVolumeClaims. Restricts volume access to a single pod on a single node. (kubernetes/kubernetes#102028, @chrishenzie)
Alpha swap support can now be enabled on Kubernetes nodes with the NodeSwapEnabled feature flag. See KEP-2400 for details. (kubernetes/kubernetes#102823, @ehashman)
Because of the implementation logic of time.Format in golang, the displayed time zone is not consistent. (kubernetes/kubernetes#102366, @cndoit18)
Corrected the documentation for escaping dollar signs in a container's env, command and args property. (kubernetes/kubernetes#101916, @MartinKanters) [SIG Apps]
Enable MaxSurge for DaemonSet by default. (kubernetes/kubernetes#101742, @ravisantoshgudimetla)
Enforce the ReadWriteOncePod PVC access mode during scheduling (kubernetes/kubernetes#103082, @chrishenzie)
Ephemeral containers are now allowed to configure a securityContext that differs from that of the Pod. Cluster administrators should ensure that security policy controllers support EphemeralContainers before enabling this feature in clusters. (kubernetes/kubernetes#99023, @verb)
Exec plugin authors can override default handling of standard input via new interactiveMode kubeconfig field. (kubernetes/kubernetes#99310, @ankeesler)
If someone had the ProbeTerminationGracePeriod alpha feature enabled in 1.21, they should update/delete any workloads/pods with probe terminationGracePeriods < 1 before upgrading (kubernetes/kubernetes#103245, @wzshiming)
Improved parsing of label selectors (kubernetes/kubernetes#102188, @alculquicondor) [SIG API Machinery]
Introduce minReadySeconds api to the StatefulSets. (kubernetes/kubernetes#100842, @ravisantoshgudimetla)
Introducing Memory quality of service support with cgroups v2 (Alpha). The MemoryQoS feature is now in Alpha. This allows kubelet running with cgroups v2 to set memory QoS at container, pod and QoS level to protect and guarantee better memory quality. This feature can be enabled through feature gate Memory QoS. (kubernetes/kubernetes#102970, @borgerli)
Kube API server accepts Impersonate-Uid header to impersonate a user with a specific UID, in the same way that you can currently use Impersonate-User, Impersonate-Group and Impersonate-Extra. (kubernetes/kubernetes#99961, @margocrawf)
Kube-apiserver: --service-account-issuer can be specified multiple times now, to enable non-disruptive change of issuer. (kubernetes/kubernetes#101155, @zshihang) [SIG API Machinery, Auth, Node and Testing]
Kube-controller-manager: the --horizontal-pod-autoscaler-use-rest-clients flag and Heapster support in the horizontal pod autoscaler, deprecated since 1.12, is removed. (kubernetes/kubernetes#90368, @serathius)
Kube-scheduler: a plugin enabled in a v1beta2 configuration file takes precedence over the default configuration for that plugin. This simplifies enabling default plugins with custom configuration without needing to explicitly disable those default plugins. (kubernetes/kubernetes#99582, @chendave)
New node-high priority-level has been added to Suggested API Priority and (kubernetes/kubernetes#101151, @mborsz)
NodeSwapEnabled feature flag was renamed to NodeSwap

The flag was only available in the 1.22.0-beta.1 release, and the new flag should be used going forward. (kubernetes/kubernetes#103553, @ehashman) [SIG Node]
Omit comparison with boolean constant (kubernetes/kubernetes#101523, @chuntaochen) [SIG CLI and Cloud Provider]
Removed the feature flag for probe-level termination grace period from Kubelet. If a user wants to disable this feature on already created pods, they will have to delete and recreate the pods. (kubernetes/kubernetes#103168, @raisaat) [SIG Apps and Node]
Revert addition of Add PersistentVolumeClaimDeletePoilcy to StatefulSetAPI. (kubernetes/kubernetes#103747, @mattcary)
Scheduler could be configured to consider new resources beside CPU and memory, GPU for example, for the score plugin of NodeResourcesBalancedAllocation. (kubernetes/kubernetes#101946, @chendave) [SIG Scheduling]
Server Side Apply now treats all Selector fields as atomic (meaning the entire selector is managed by a single writer and updated together), since they contain interrelated and inseparable fields that do not merge in intuitive ways. (kubernetes/kubernetes#97989, @Danil-Grigorev) [SIG API Machinery]
Suspend Job feature graduated to beta. Added the action label to Job controller sync metrics job_sync_total and job_sync_duration_seconds. (kubernetes/kubernetes#102022, @adtac)
The API documentation for the DaemonSet's spec.updateStrategy.rollingUpdate.maxUnavailable field was corrected to state that the value is rounded up. (kubernetes/kubernetes#101296, @Miciah)
The CSIServiceAccountToken graduates to Ga and is unconditionally enabled. (kubernetes/kubernetes#103001, @zshihang)
The CertificateSigningRequest.certificates.k8s.io API supports an optional expirationSeconds field to allow the client to request a particular duration for the issued certificate. The default signer implementations provided by the Kubernetes controller manager will honor this field as long as it does not exceed the --cluster-signing-duration flag. (kubernetes/kubernetes#99494, @enj)
The EndpointSlicen Mirroring controller no longer mirrors the last-applied-configuration annotation created by kubectl to update EndpointSlices. (kubernetes/kubernetes#102731, @sharmarajdaksh)
The NetworkPolicyEndPort is graduated to beta and is enabled by default. (kubernetes/kubernetes#102834, @rikatz)
The PodDeletionCost feature has been promoted to beta, and enabled by default. (kubernetes/kubernetes#101080, @ahg-g)
The Server Side Apply treats certain structs as atomic. Meaning the entire selector field is managed by a single writer and updated together. (kubernetes/kubernetes#100684, @Jefftree)
The ServiceAppProtocol feature gate has been removed. It reached GA in Kubernetes (kubernetes/kubernetes#103190, @robscott)
The TerminationGracePeriodSeconds on pod specs and container probes should not be negative. Negative values of TerminationGracePeriodSeconds will be treated as the value 1s on the delete path. Immutable field validation will be relaxed in order to update negative values. In a future release, negative values will not be permitted. (kubernetes/kubernetes#98866, @wzshiming)
The kube-scheduler component config v1beta2 API available Three scheduler plugins deprecated (NodeLabel, ServiceAffinity, NodePreferAvoidPods). (kubernetes/kubernetes#99597, @adtac)
The pod/eviction subresource now accepts policy/v1 eviction requests in addition to policy/v1beta1 eviction requests (kubernetes/kubernetes#100724, @liggitt)
The podAffinity, NamespaceSelector and the associated CrossNamespaceAffinity quota scope features graduate to Beta and they are now enabled by default. (kubernetes/kubernetes#101496, @ahg-g)
The pods/ephemeralcontainers API now returns and expects a Pod object instead of EphemeralContainers. This is incompatible with the previous alpha-level API. (kubernetes/kubernetes#101034, @verb) [SIG Apps, Auth, CLI and Testing]
The v1.Node and .status.images[].names are now optional. (kubernetes/kubernetes#102159, @roycaihw)
The deprecated flag --algorithm-provider has been removed from kube-scheduler. Use instead ComponentConfig to configure the set of enabled plugins. (kubernetes/kubernetes#102239, @Haleygo)
The options --ssh-user and --ssh-key are removed. They only functioned on GCE, and only in-tree. Use the apiserver network proxy instead. (kubernetes/kubernetes#102297, @deads2k)
Track Job completion through status and Pod finalizers, removing dependency on Pod tombstones. (kubernetes/kubernetes#98238, @alculquicondor) [SIG API Machinery, Apps, Auth and Testing]
Track ownership of scale subresource for all scalable resources i.e. Deployment, ReplicaSet, StatefulSet, ReplicationController, and Custom Resources. (kubernetes/kubernetes#98377, @nodo) [SIG API Machinery and Testing]
Revert addition of Add PersistentVolumeClaimDeletePoilcy to StatefulSet API. (kubernetes/kubernetes#103747, @mattcary) [SIG API Machinery and Apps]
Added field .status.uncountedTerminatedPods to the Job resource. This field is used by the job controller to keep track of finished pods before adding them to the Job status counters.

Pods created by the job controller get the finalizer batch.kubernetes.io/job-tracking

Jobs that are tracked using this mechanism get the annotation batch.kubernetes.io/job-tracking. This is a temporary measure. Two releases after this feature graduates to beta, the annotation won't be added to Jobs anymore. (kubernetes/kubernetes#98817, @alculquicondor) [SIG API Machinery, Apps, Auth and CLI]
Ephemeral containers are now allowed to configure a securityContext that differs from that of the Pod.

Cluster administrators should ensure that security policy controllers support EphemeralContainers before enabling this feature in clusters. (kubernetes/kubernetes#99023, @verb) [SIG API Machinery, Apps, Auth and Node]
If someone had the ProbeTerminationGracePeriod alpha feature enabled in 1.21, they should update/delete any workloads/pods with probe terminationGracePeriods < 1 before upgrading (kubernetes/kubernetes#103245, @wzshiming) [SIG Apps and Node]
Introducing Memory QoS support with cgroups v2 (Alpha) The MemoryQoS feature is now in Alpha. This allows kubelet running with cgroups v2 to set memory QoS at container, pod and QoS level to protect and guarantee better memory quality. This feature can be enabled through feature gate MemoryQoS. (kubernetes/kubernetes#102970, @borgerli) [SIG Node and Storage]
NodeSwapEnabled feature flag was renamed to NodeSwap

The flag was only available in the 1.22.0-beta.1 release, and the new flag should be used going forward. (kubernetes/kubernetes#103553, @ehashman) [SIG Node]
Removed the feature flag for probe-level termination grace period from Kubelet. If a user wants to disable this feature on already created pods, they will have to delete and recreate the pods. (kubernetes/kubernetes#103168, @raisaat) [SIG Apps and Node]
Track Job completion through status and Pod finalizers, removing dependency on Pod tombstones. (kubernetes/kubernetes#98238, @alculquicondor) [SIG API Machinery, Apps, Auth and Testing]
When using kubectl replace (or the equivalent API call) on a Service, the caller no longer needs to do a read-modify-write cycle to fetch the allocated values for .spec.clusterIP and .spec.ports[].nodePort. Instead the API server will automatically carry these forward from the original object when the new object does not specify them. (kubernetes/kubernetes#103532, @thockin) [SIG Apps and Network]
A new score extension for NodeResourcesFit plugin that merges the functionality of NodeResourcesLeastAllocated,NodeResourcesMostAllocated,RequestedToCapacityRatio plugins, which are marked as deprecated as of v1beta2. In v1beta1, the three plugins can still be used in v1beta1 but not at the same time with the score extension of NodeResourcesFit
Add DataSourceRef alpha field to PVC spec, which allows contents other than PVCs and VolumeSnapshots to be data sources. (kubernetes/kubernetes#103276, @bswartz) [SIG API Machinery, Apps and Storage]
Add PersistentVolumeClaimDeletePoilcy to StatefulSet API. (kubernetes/kubernetes#99378, @mattcary) [SIG API Machinery and Apps]
Add distributed tracing to the kube-apiserver. It is can be enabled with the feature gate: APIServerTracing=true (kubernetes/kubernetes#94942, @dashpole) [SIG API Machinery, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Storage and Testing]
Added new kubelet alpha feature SeccompDefault. This feature enables falling back to the RuntimeDefault (former runtime/default) seccomp profile if nothing else is specified in the pod/container SecurityContext or the pod annotation level. To use the feature, enable the feature gate as well as set the kubelet configuration option SeccompDefault (--seccomp-default) to true. (kubernetes/kubernetes#101943, @saschagrunert) [SIG Node]
Adds the ReadWriteOncePod access mode for PersistentVolumes and PersistentVolumeClaims. Restricts volume access to a single pod on a single node. (kubernetes/kubernetes#102028, @chrishenzie) [SIG Apps, CLI, Node, Scheduling and Storage]
Alpha swap support can now be enabled on Kubernetes nodes with the NodeSwapEnabled feature flag. See for details. (kubernetes/kubernetes#102823, @ehashman) [SIG Node]
CSIServiceAccountToken is GA. (kubernetes/kubernetes#103001, @zshihang) [SIG Auth and Storage]
Enforce the ReadWriteOncePod PVC access mode during scheduling (kubernetes/kubernetes#103082, @chrishenzie) [SIG Apps, CLI, Node, Scheduling and Storage]
Improved parsing of label selectors (kubernetes/kubernetes#102188, @alculquicondor) [SIG API Machinery]
Kube API server accepts Impersonate-Uid header to impersonate a user with a specific UID, in the same way that you can currently use Impersonate-User, Impersonate-Group and Impersonate-Extra (kubernetes/kubernetes#99961, @margocrawf) [SIG API Machinery, Auth and Testing]
Kube-scheduler: a plugin enabled in a v1beta2 configuration file takes precedence over the default configuration for that plugin; this simplifies enabling default plugins with custom configuration without needing to explicitly disable those default plugins. (kubernetes/kubernetes#99582, @chendave) [SIG Scheduling]
Scheduler could be configured to consider new resources beside CPU and memory, GPU for example, for the score plugin of NodeResourcesBalancedAllocation. (kubernetes/kubernetes#101946, @chendave) [SIG Scheduling]
Suspend Job feature graduated to beta Added the "action" label to Job controller sync metrics job_sync_total and job_sync_duration_seconds (kubernetes/kubernetes#102022, @adtac) [SIG Apps, Instrumentation and Testing]
TerminationGracePeriodSeconds on pod specs and container probes should not be negative. Negative values of TerminationGracePeriodSeconds will be treated as the value 1s on the delete path. Immutable field validation will be relaxed in order to update negative values. In a future release, negative values will not be permitted. (kubernetes/kubernetes#98866, @wzshiming) [SIG API Machinery, Apps and Node]
The API documentation for the DaemonSet's spec.updateStrategy.rollingUpdate.maxUnavailable field was corrected to state that the value is rounded up. (kubernetes/kubernetes#101296, @Miciah) [SIG Apps and CLI]
The CertificateSigningRequest.certificates.k8s.io API supports an optional expirationSeconds field to allow the client to request a particular duration for the issued certificate. The default signer implementations provided by the Kubernetes controller manager will honor this field as long as it does not exceed the --cluster-signing-duration flag. (kubernetes/kubernetes#99494, @enj) [SIG API Machinery, Apps, Auth, CLI, Instrumentation, Node, Security and Testing]
The ServiceAppProtocol feature gate has been removed. It reached GA in Kubernetes 1.20. (kubernetes/kubernetes#103190, @robscott) [SIG Network]
Because of the implementation logic of time.Format in golang, the displayed time zone is not consistent (kubernetes/kubernetes#102366, @cndoit18) [SIG Apps, Auth, Autoscaling, CLI, Cluster Lifecycle, Instrumentation, Network, Node and Testing]
Endpoint slices mirroring controller no longer mirrors the last-applied-configuration annotation created by kubectl to updated endpoint slices (kubernetes/kubernetes#102731, @sharmarajdaksh) [SIG API Machinery, Apps, Cloud Provider, Network, Release, Scheduling, Storage and Testing]
Exec plugin authors can override default handling of standard input via new interactiveMode kubeconfig field (kubernetes/kubernetes#99310, @ankeesler) [SIG API Machinery, Auth, CLI and Testing]
Kube-scheduler component config v1beta2 API available Three scheduler plugins deprecated (NodeLabel, ServiceAffinity, NodePreferAvoidPods) (kubernetes/kubernetes#99597, @adtac) [SIG Scheduling]
Network Policy EndPort is graduated to beta and is enabled by default (kubernetes/kubernetes#102834, @rikatz) [SIG Network]
--ssh-user and --ssh-key options are removed. They only functioned on GCE, and only in-tree. Use the apiserver network proxy instead. (kubernetes/kubernetes#102297, @deads2k) [SIG API Machinery, Cloud Provider and Testing]
Enable MaxSurge for DS by default (kubernetes/kubernetes#101742, @ravisantoshgudimetla) [SIG Apps and Testing]
Introduce minReadySeconds api to the StatefulSets. (kubernetes/kubernetes#100842, @ravisantoshgudimetla) [SIG API Machinery, Apps and Testing]
Kube-controller-manger: the --horizontal-pod-autoscaler-use-rest-clients flag and Heapster support in the horizontal pod autoscaler, deprecated since 1.12, is removed. (kubernetes/kubernetes#90368, @serathius) [SIG API Machinery, Apps, Autoscaling, Cloud Provider and Instrumentation]
The deprecated flag --algorithm-provider has been removed from kube-scheduler. Use instead ComponentConfig to configure the set of enabled plugins (kubernetes/kubernetes#102239, @Haleygo) [SIG Cloud Provider and Scheduling]
Add alpha support for HostProcess containers on Windows (kubernetes/kubernetes#99576, @marosset) [SIG API Machinery, Apps, Node, Testing and Windows]
Add three metrics to job controller to monitor if Job works in a healthy condition. IndexedJob promoted to Beta (kubernetes/kubernetes#101292, @AliceZhang2016) [SIG Apps, Instrumentation and Testing]
Corrected the documentation for escaping dollar signs in a container's env, command and args property. (kubernetes/kubernetes#101916, @MartinKanters) [SIG Apps]
Omit comparison with boolean constant (kubernetes/kubernetes#101523, @GreenApple10) [SIG CLI and Cloud Provider]
Pod Affinity NamespaceSelector and the associated CrossNamespaceAffinity quota scope graduated to beta (kubernetes/kubernetes#101496, @ahg-g) [SIG API Machinery, Apps and Testing]
V1.Node .status.images[].names is now optional (kubernetes/kubernetes#102159, @roycaihw) [SIG Apps and Node]
"Auto" is now a valid value for the service.kubernetes.io/topology-aware-hints annotation. (kubernetes/kubernetes#100728, @robscott) [SIG Apps, Instrumentation and Network]
Kube-apiserver: --service-account-issuer can be specified multiple times now, to enable non-disruptive change of issuer. (kubernetes/kubernetes#101155, @zshihang) [SIG API Machinery, Auth, Node and Testing]
New "node-high" priority-level has been added to Suggested API Priority and Fairness configuration. (kubernetes/kubernetes#101151, @mborsz) [SIG API Machinery]
PodDeletionCost promoted to Beta (kubernetes/kubernetes#101080, @ahg-g) [SIG Apps]
SSA treats certain structs as atomic (kubernetes/kubernetes#100684, @Jefftree) [SIG API Machinery, Auth, Node and Storage]
Server Side Apply now treats all Selector fields as atomic (meaning the entire selector is managed by a single writer and updated together), since they contain interrelated and inseparable fields that do not merge in intuitive ways. (kubernetes/kubernetes#97989, @Danil-Grigorev) [SIG API Machinery]
The pods/ephemeralcontainers API now returns and expects a Pod object instead of EphemeralContainers. This is incompatible with the previous alpha-level API. (kubernetes/kubernetes#101034, @verb) [SIG Apps, Auth, CLI and Testing]
The pod/eviction subresource now accepts policy/v1 Eviction requests in addition to policy/v1beta1 Eviction requests (kubernetes/kubernetes#100724, @liggitt) [SIG API Machinery, Apps, Architecture, Auth, CLI, Storage and Testing]
Track ownership of scale subresource for all scalable resources i.e. Deployment, ReplicaSet, StatefulSet, ReplicationController, and Custom Resources. (kubernetes/kubernetes#98377, @nodo) [SIG API Machinery and Testing]
We have added a new Priority & Fairness rule that exempts all probes (/readyz, /healthz, /livez) to prevent restarting of "healthy" kube-apiserver instance(s) by kubelet. (kubernetes/kubernetes#100678, @tkashem) [SIG API Machinery]

v21.7.1

fix: add missing change in latest regnerated lib (#184, @tomplus)

v21.7.0

API Change

Kube-apiserver: Fixes handling of CRD schemas containing literal null values in enums (#104989, @liggitt) [SIG API Machinery, Apps and Network]
"Auto" is now a valid value for the service.kubernetes.io/topology-aware-hints annotation. (kubernetes/kubernetes#100728, @robscott) [SIG Apps, Instrumentation and Network]
We have added a new Priority & Fairness rule that exempts all probes (/readyz, /healthz, /livez) to prevent restarting of "healthy" kube-apiserver instance(s) by kubelet. (kubernetes/kubernetes#101111, @tkashem) [SIG API Machinery]
1. PodAffinityTerm includes a namespaceSelector field to allow selecting eligible namespaces based on their labels.
2. A new CrossNamespacePodAffinity quota scope API that allows restricting which namespaces allowed to use PodAffinityTerm with corss-namespace reference via namespaceSelector or namespaces fields. (kubernetes/kubernetes#98582, @ahg-g) [SIG API Machinery, Apps, Auth and Testing]
Add Probe-level terminationGracePeriodSeconds field (kubernetes/kubernetes#99375, @ehashman) [SIG API Machinery, Apps, Node and Testing]
Added .spec.completionMode field to Job, with accepted values NonIndexed (default) and Indexed. This is an alpha field and is only honored by servers with the IndexedJob feature gate enabled. (kubernetes/kubernetes#98441, @alculquicondor) [SIG Apps and CLI]
Adds support for endPort field in NetworkPolicy (kubernetes/kubernetes#97058, @rikatz) [SIG Apps and Network]
CSIServiceAccountToken graduates to Beta and enabled by default. (kubernetes/kubernetes#99298, @zshihang)
Cluster admins can now turn off /debug/pprof and /debug/flags/v endpoint in kubelet by setting enableProfilingHandler and enableDebugFlagsHandler to false in the Kubelet configuration file. Options enableProfilingHandler and enableDebugFlagsHandler can be set to true only when enableDebuggingHandlers is also set to true. (kubernetes/kubernetes#98458, @SaranBalaji90)
DaemonSets accept a MaxSurge integer or percent on their rolling update strategy that will launch the updated pod on nodes and wait for those pods to go ready before marking the old out-of-date pods as deleted. This allows workloads to avoid downtime during upgrades when deployed using DaemonSets. This feature is alpha and is behind the DaemonSetUpdateSurge feature gate. (kubernetes/kubernetes#96441, @smarterclayton) [SIG Apps and Testing]
Enable SPDY pings to keep connections alive, so that kubectl exec and kubectl portforward won't be interrupted. (kubernetes/kubernetes#97083, @knight42) [SIG API Machinery and CLI]
FieldManager no longer owns fields that get reset before the object is persisted (e.g. "status wiping"). (kubernetes/kubernetes#99661, @kevindelgado) [SIG API Machinery, Auth and Testing]
Fixes server-side apply for APIService resources. (kubernetes/kubernetes#98576, @kevindelgado)
Generic ephemeral volumes are beta. (kubernetes/kubernetes#99643, @pohly) [SIG API Machinery, Apps, Auth, CLI, Node, Storage and Testing]
Hugepages request values are limited to integer multiples of the page size. (kubernetes/kubernetes#98515, @lala123912) [SIG Apps]
Implement the GetAvailableResources in the podresources API. (kubernetes/kubernetes#95734, @fromanirh) [SIG Instrumentation, Node and Testing]
IngressClass resource can now reference a resource in a specific namespace for implementation-specific configuration (previously only Cluster-level resources were allowed). This feature can be enabled using the IngressClassNamespacedParams feature gate. (kubernetes/kubernetes#99275, @hbagdi)
Jobs API has a new .spec.suspend field that can be used to suspend and resume Jobs. This is an alpha field which is only honored by servers with the SuspendJob feature gate enabled. (kubernetes/kubernetes#98727, @adtac)
Kubelet Graceful Node Shutdown feature graduates to Beta and enabled by default. (kubernetes/kubernetes#99735, @bobbypage)
Kubernetes is now built using go1.15.7 (kubernetes/kubernetes#98363, @cpanato) [SIG Cloud Provider, Instrumentation, Node, Release and Testing]
Namespace API objects now have a kubernetes.io/metadata.name label matching their metadata.name field to allow selecting any namespace by its name using a label selector. (kubernetes/kubernetes#96968, @jayunit100) [SIG API Machinery, Apps, Cloud Provider, Storage and Testing]
One new field "InternalTrafficPolicy" in Service is added. It specifies if the cluster internal traffic should be routed to all endpoints or node-local endpoints only. "Cluster" routes internal traffic to a Service to all endpoints. "Local" routes traffic to node-local endpoints only, and traffic is dropped if no node-local endpoints are ready. The default value is "Cluster". (kubernetes/kubernetes#96600, @maplain) [SIG API Machinery, Apps and Network]
PodDisruptionBudget API objects can now contain conditions in status. (kubernetes/kubernetes#98127, @mortent) [SIG API Machinery, Apps, Auth, CLI, Cloud Provider, Cluster Lifecycle and Instrumentation]
PodSecurityPolicy only stores "generic" as allowed volume type if the GenericEphemeralVolume feature gate is enabled (kubernetes/kubernetes#98918, @pohly) [SIG Auth and Security]
Promote CronJobs to batch/v1 (kubernetes/kubernetes#99423, @soltysh) [SIG API Machinery, Apps, CLI and Testing]
Promote Immutable Secrets/ConfigMaps feature to Stable. This allows to set immutable field in Secret or ConfigMap object to mark their contents as immutable. (kubernetes/kubernetes#97615, @wojtek-t) [SIG Apps, Architecture, Node and Testing]
Remove support for building Kubernetes with bazel. (kubernetes/kubernetes#99561, @BenTheElder) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scalability, Scheduling, Storage, Testing and Windows]
Scheduler extender filter interface now can report unresolvable failed nodes in the new field FailedAndUnresolvableNodes of ExtenderFilterResult struct. Nodes in this map will be skipped in the preemption phase. (kubernetes/kubernetes#92866, @cofyc) [SIG Scheduling]
Services can specify loadBalancerClass to use a custom load balancer (kubernetes/kubernetes#98277, @XudongLiuHarold)
Storage capacity tracking (= the CSIStorageCapacity feature) graduates to Beta and enabled by default, storage.k8s.io/v1alpha1/VolumeAttachment and storage.k8s.io/v1alpha1/CSIStorageCapacity objects are deprecated (kubernetes/kubernetes#99641, @pohly)
Support for Indexed Job: a Job that is considered completed when Pods associated to indexes from 0 to (.spec.completions-1) have succeeded. (kubernetes/kubernetes#98812, @alculquicondor) [SIG Apps and CLI]
The BoundServiceAccountTokenVolume feature has been promoted to beta, and enabled by default.
- This changes the tokens provided to containers at /var/run/secrets/kubernetes.io/serviceaccount/token to be time-limited, auto-refreshed, and invalidated when the containing pod is deleted.
- Clients should reload the token from disk periodically (once per minute is recommended) to ensure they continue to use a valid token. k8s.io/client-go version v11.0.0+ and v0.15.0+ reload tokens automatically.
- By default, injected tokens are given an extended lifetime so they remain valid even after a new refreshed token is provided. The metric serviceaccount_stale_tokens_total can be used to monitor for workloads that are depending on the extended lifetime and are continuing to use tokens even after a refreshed token is provided to the container. If that metric indicates no existing workloads are depending on extended lifetimes, injected token lifetime can be shortened to 1 hour by starting kube-apiserver with --service-account-extend-token-expiration=false. (kubernetes/kubernetes#95667, @zshihang) [SIG API Machinery, Auth, Cluster Lifecycle and Testing]
The EndpointSlice Controllers are now GA. The EndpointSliceController will not populate the deprecatedTopology field and will only provide topology information through the zone and nodeName fields. (kubernetes/kubernetes#99870, @swetharepakula)
The Endpoints controller will now set the endpoints.kubernetes.io/over-capacity annotation to "warning" when an Endpoints resource contains more than 1000 addresses. In a future release, the controller will truncate Endpoints that exceed this limit. The EndpointSlice API can be used to support significantly larger number of addresses. (kubernetes/kubernetes#99975, @robscott) [SIG Apps and Network]
The PodDisruptionBudget API has been promoted to policy/v1 with no schema changes. The only functional change is that an empty selector ({}) written to a policy/v1 PodDisruptionBudget now selects all pods in the namespace. The behavior of the policy/v1beta1 API remains unchanged. The policy/v1beta1 PodDisruptionBudget API is deprecated and will no longer be served in 1.25+. (kubernetes/kubernetes#99290, @mortent) [SIG API Machinery, Apps, Auth, Autoscaling, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Scheduling and Testing]
The EndpointSlice API is now GA. The EndpointSlice topology field has been removed from the GA API and will be replaced by a new per Endpoint Zone field. If the topology field was previously used, it will be converted into an annotation in the v1 Resource. The discovery.k8s.io/v1alpha1 API is removed. (kubernetes/kubernetes#99662, @swetharepakula)
The controller.kubernetes.io/pod-deletion-cost annotation can be set to offer a hint on the cost of deleting a Pod compared to other pods belonging to the same ReplicaSet. Pods with lower deletion cost are deleted first. This is an alpha feature. (kubernetes/kubernetes#99163, @ahg-g)
The kube-apiserver now resets managedFields that got corrupted by a mutating admission controller. (kubernetes/kubernetes#98074, @kwiesmueller)
Topology Aware Hints are now available in alpha and can be enabled with the TopologyAwareHints feature gate. (kubernetes/kubernetes#99522, @robscott) [SIG API Machinery, Apps, Auth, Instrumentation, Network and Testing]
Users might specify the kubectl.kubernetes.io/default-exec-container annotation in a Pod to preselect container for kubectl commands. (kubernetes/kubernetes#97099, @pacoxu) [SIG CLI]
Add Probe-level terminationGracePeriodSeconds field (kubernetes/kubernetes#99375, @ehashman) [SIG API Machinery, Apps, Node and Testing]
CSIServiceAccountToken is Beta now (kubernetes/kubernetes#99298, @zshihang) [SIG Auth, Storage and Testing]
Discovery.k8s.io/v1beta1 EndpointSlices are deprecated in favor of discovery.k8s.io/v1, and will no longer be served in Kubernetes v1.25. (kubernetes/kubernetes#100472, @liggitt) [SIG Network]
FieldManager no longer owns fields that get reset before the object is persisted (e.g. "status wiping"). (kubernetes/kubernetes#99661, @kevindelgado) [SIG API Machinery, Auth and Testing]
Generic ephemeral volumes are beta. (kubernetes/kubernetes#99643, @pohly) [SIG API Machinery, Apps, Auth, CLI, Node, Storage and Testing]
Implement the GetAvailableResources in the podresources API. (kubernetes/kubernetes#95734, @fromanirh) [SIG Instrumentation, Node and Testing]
The Endpoints controller will now set the endpoints.kubernetes.io/over-capacity annotation to "warning" when an Endpoints resource contains more than 1000 addresses. In a future release, the controller will truncate Endpoints that exceed this limit. The EndpointSlice API can be used to support significantly larger number of addresses. (kubernetes/kubernetes#99975, @robscott) [SIG Apps and Network]
The PodDisruptionBudget API has been promoted to policy/v1 with no schema changes. The only functional change is that an empty selector ({}) written to a policy/v1 PodDisruptionBudget now selects all pods in the namespace. The behavior of the policy/v1beta1 API remains unchanged. The policy/v1beta1 PodDisruptionBudget API is deprecated and will no longer be served in 1.25+. (kubernetes/kubernetes#99290, @mortent) [SIG API Machinery, Apps, Auth, Autoscaling, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Scheduling and Testing]
Topology Aware Hints are now available in alpha and can be enabled with the TopologyAwareHints feature gate. (kubernetes/kubernetes#99522, @robscott) [SIG API Machinery, Apps, Auth, Instrumentation, Network and Testing]
1. PodAffinityTerm includes a namespaceSelector field to allow selecting eligible namespaces based on their labels.
2. A new CrossNamespacePodAffinity quota scope API that allows restricting which namespaces allowed to use PodAffinityTerm with corss-namespace reference via namespaceSelector or namespaces fields. (kubernetes/kubernetes#98582, @ahg-g) [SIG API Machinery, Apps, Auth and Testing]
Add a default metadata name labels for selecting any namespace by its name. (kubernetes/kubernetes#96968, @jayunit100) [SIG API Machinery, Apps, Cloud Provider, Storage and Testing]
Added .spec.completionMode field to Job, with accepted values NonIndexed (default) and Indexed (kubernetes/kubernetes#98441, @alculquicondor) [SIG Apps and CLI]
Clarified NetworkPolicy policyTypes documentation (kubernetes/kubernetes#97216, @joejulian) [SIG Network]
DaemonSets accept a MaxSurge integer or percent on their rolling update strategy that will launch the updated pod on nodes and wait for those pods to go ready before marking the old out-of-date pods as deleted. This allows workloads to avoid downtime during upgrades when deployed using DaemonSets. This feature is alpha and is behind the DaemonSetUpdateSurge feature gate. (kubernetes/kubernetes#96441, @smarterclayton) [SIG Apps and Testing]
EndpointSlice API is now GA. The EndpointSlice topology field has been removed from the GA API and will be replaced by a new per Endpoint Zone field. If the topology field was previously used, it will be converted into an annotation in the v1 Resource. The discovery.k8s.io/v1alpha1 API is removed. (kubernetes/kubernetes#99662, @swetharepakula) [SIG API Machinery, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network and Testing]
EndpointSlice Controllers are now GA. The EndpointSlice Controller will not populate the deprecatedTopology field and will only provide topology information through the zone and nodeName fields. (kubernetes/kubernetes#99870, @swetharepakula) [SIG API Machinery, Apps, Auth, Network and Testing]
IngressClass resource can now reference a resource in a specific namespace for implementation-specific configuration(previously only Cluster-level resources were allowed). This feature can be enabled using the IngressClassNamespacedParams feature gate. (kubernetes/kubernetes#99275, @hbagdi) [SIG API Machinery, CLI and Network]
Introduce conditions for PodDisruptionBudget (kubernetes/kubernetes#98127, @mortent) [SIG API Machinery, Apps, Auth, CLI, Cloud Provider, Cluster Lifecycle and Instrumentation]
Jobs API has a new .spec.suspend field that can be used to suspend and resume Jobs (kubernetes/kubernetes#98727, @adtac) [SIG API Machinery, Apps, Node, Scheduling and Testing]
Kubelet Graceful Node Shutdown feature is now beta. (kubernetes/kubernetes#99735, @bobbypage) [SIG Node]
Limit the quest value of hugepage to integer multiple of page size. (kubernetes/kubernetes#98515, @lala123912) [SIG Apps]
One new field "InternalTrafficPolicy" in Service is added. It specifies if the cluster internal traffic should be routed to all endpoints or node-local endpoints only. "Cluster" routes internal traffic to a Service to all endpoints. "Local" routes traffic to node-local endpoints only, and traffic is dropped if no node-local endpoints are ready. The default value is "Cluster". (kubernetes/kubernetes#96600, @maplain) [SIG API Machinery, Apps and Network]
PodSecurityPolicy only stores "generic" as allowed volume type if the GenericEphemeralVolume feature gate is enabled (kubernetes/kubernetes#98918, @pohly) [SIG Auth and Security]
Promote CronJobs to batch/v1 (kubernetes/kubernetes#99423, @soltysh) [SIG API Machinery, Apps, CLI and Testing]
Remove support for building Kubernetes with bazel. (kubernetes/kubernetes#99561, @BenTheElder) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scalability, Scheduling, Storage, Testing and Windows]
Setting loadBalancerClass in load balancer type of service is available with this PR. Users who want to use a custom load balancer can specify loadBalancerClass to achieve it. (kubernetes/kubernetes#98277, @XudongLiuHarold) [SIG API Machinery, Apps, Cloud Provider and Network]
Storage capacity tracking (= the CSIStorageCapacity feature) is beta, storage.k8s.io/v1alpha1/VolumeAttachment and storage.k8s.io/v1alpha1/CSIStorageCapacity objects are deprecated (kubernetes/kubernetes#99641, @pohly) [SIG API Machinery, Apps, Auth, Scheduling, Storage and Testing]
Support for Indexed Job: a Job that is considered completed when Pods associated to indexes from 0 to (.spec.completions-1) have succeeded. (kubernetes/kubernetes#98812, @alculquicondor) [SIG Apps and CLI]
The apiserver now resets managedFields that got corrupted by a mutating admission controller. (kubernetes/kubernetes#98074, @kwiesmueller) [SIG API Machinery and Testing]
controller.kubernetes.io/pod-deletion-cost annotation can be set to offer a hint on the cost of deleting a pod compared to other pods belonging to the same ReplicaSet. Pods with lower deletion cost are deleted first. This is an alpha feature. (kubernetes/kubernetes#99163, @ahg-g) [SIG Apps]
Cluster admins can now turn off /debug/pprof and /debug/flags/v endpoint in kubelet by setting enableProfilingHandler and enableDebugFlagsHandler to false in their kubelet configuration file. enableProfilingHandler and enableDebugFlagsHandler can be set to true only when enableDebuggingHandlers is also set to true. (kubernetes/kubernetes#98458, @SaranBalaji90) [SIG Node]
The BoundServiceAccountTokenVolume feature has been promoted to beta, and enabled by default.
- This changes the tokens provided to containers at /var/run/secrets/kubernetes.io/serviceaccount/token to be time-limited, auto-refreshed, and invalidated when the containing pod is deleted.
- Clients should reload the token from disk periodically (once per minute is recommended) to ensure they continue to use a valid token. k8s.io/client-go version v11.0.0+ and v0.15.0+ reload tokens automatically.
- By default, injected tokens are given an extended lifetime so they remain valid even after a new refreshed token is provided. The metric serviceaccount_stale_tokens_total can be used to monitor for workloads that are depending on the extended lifetime and are continuing to use tokens even after a refreshed token is provided to the container. If that metric indicates no existing workloads are depending on extended lifetimes, injected token lifetime can be shortened to 1 hour by starting kube-apiserver with --service-account-extend-token-expiration=false. (kubernetes/kubernetes#95667, @zshihang) [SIG API Machinery, Auth, Cluster Lifecycle and Testing]
Adds support for portRange / EndPort in Network Policy (kubernetes/kubernetes#97058, @rikatz) [SIG Apps and Network]
Fixes using server-side apply with APIService resources (kubernetes/kubernetes#98576, @kevindelgado) [SIG API Machinery, Apps and Testing]
Kubernetes is now built using go1.15.7 (kubernetes/kubernetes#98363, @cpanato) [SIG Cloud Provider, Instrumentation, Node, Release and Testing]
Scheduler extender filter interface now can report unresolvable failed nodes in the new field FailedAndUnresolvableNodes of ExtenderFilterResult struct. Nodes in this map will be skipped in the preemption phase. (kubernetes/kubernetes#92866, @cofyc) [SIG Scheduling]
Enable SPDY pings to keep connections alive, so that kubectl exec and kubectl port-forward won't be interrupted. (kubernetes/kubernetes#97083, @knight42) [SIG API Machinery and CLI]
Change the APIVersion proto name of BoundObjectRef from aPIVersion to apiVersion. (kubernetes/kubernetes#97379, @kebe7jun) [SIG Auth]
Promote Immutable Secrets/ConfigMaps feature to Stable. This allows to set Immutable field in Secrets or ConfigMap object to mark their contents as immutable. (kubernetes/kubernetes#97615, @wojtek-t) [SIG Apps, Architecture, Node and Testing]

v20.13.0

API Change

Kube-apiserver: Fixes handling of CRD schemas containing literal null values in enums (#104990, @liggitt) [SIG API Machinery, Apps and Network]
We have added a new Priority & Fairness rule that exempts all probes (/readyz, /healthz, /livez) to prevent restarting of "healthy" kube-apiserver instance(s) by kubelet. (kubernetes/kubernetes#101112, @tkashem) [SIG API Machinery]
Fixes using server-side apply with APIService resources (kubernetes/kubernetes#100714, @kevindelgado) [SIG API Machinery, Apps and Testing]
Regenerate protobuf code to fix CVE-2021-3121 (kubernetes/kubernetes#100501, @joelsmith) [SIG API Machinery, Apps, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node and Storage]
Kubernetes is now built using go1.15.8 (kubernetes/kubernetes#98962, @cpanato) [SIG Cloud Provider, Instrumentation, Release and Testing]
TokenRequest and TokenRequestProjection features have been promoted to GA. This feature allows generating service account tokens that are not visible in Secret objects and are tied to the lifetime of a Pod object. See https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#service-account-token-volume-projection for details on configuring and using this feature. The TokenRequest and TokenRequestProjection feature gates will be removed in v1.21.
- kubeadm's kube-apiserver Pod manifest now includes the following flags by default "--service-account-key-file", "--service-account-signing-key-file", "--service-account-issuer". (kubernetes/kubernetes#93258, @zshihang) [SIG API Machinery, Auth, Cluster Lifecycle, Storage and Testing]
A new nofuzz go build tag now disables gofuzz support. Release binaries enable this. (kubernetes/kubernetes#92491, @BenTheElder) [SIG API Machinery]
Add WindowsContainerResources and Annotations to CRI-API UpdateContainerResourcesRequest (kubernetes/kubernetes#95741, @katiewasnothere) [SIG Node]
Add a serving and terminating condition to the EndpointSlice API. serving tracks the readiness of endpoints regardless of their terminating state. This is distinct from ready since ready is only true when pods are not terminating. terminating is true when an endpoint is terminating. For pods this is any endpoint with a deletion timestamp. (kubernetes/kubernetes#92968, @andrewsykim) [SIG Apps and Network]
Add dual-stack Services (alpha). This is a BREAKING CHANGE to an alpha API. It changes the dual-stack API wrt Service from a single ipFamily field to 3 fields: ipFamilyPolicy (SingleStack, PreferDualStack, RequireDualStack), ipFamilies (a list of families assigned), and clusterIPs (inclusive of clusterIP). Most users do not need to set anything at all, defaulting will handle it for them. Services are single-stack unless the user asks for dual-stack. This is all gated by the "IPv6DualStack" feature gate. (kubernetes/kubernetes#91824, @khenidak) [SIG API Machinery, Apps, CLI, Network, Node, Scheduling and Testing]
Add support for hugepages to downward API (kubernetes/kubernetes#86102, @derekwaynecarr) [SIG API Machinery, Apps, CLI, Network, Node, Scheduling and Testing]
Adds kubelet alpha feature, GracefulNodeShutdown which makes kubelet aware of node system shutdowns and result in graceful termination of pods during a system shutdown. (kubernetes/kubernetes#96129, @bobbypage) [SIG Node]
AppProtocol is now GA for Endpoints and Services. The ServiceAppProtocol feature gate will be deprecated in 1.21. (kubernetes/kubernetes#96327, @robscott) [SIG Apps and Network]
Automatic allocation of NodePorts for services with type LoadBalancer can now be disabled by setting the (new) parameter Service.spec.allocateLoadBalancerNodePorts=false. The default is to allocate NodePorts for services with type LoadBalancer which is the existing behavior. (kubernetes/kubernetes#92744, @uablrek) [SIG Apps and Network]
Certain fields on Service objects will be automatically cleared when changing the service's type to a mode that does not need those fields. For example, changing from type=LoadBalancer to type=ClusterIP will clear the NodePort assignments, rather than forcing the user to clear them. (kubernetes/kubernetes#95196, @thockin) [SIG API Machinery, Apps, Network and Testing]
Document that ServiceTopology feature is required to use service.spec.topologyKeys. (kubernetes/kubernetes#96528, @andrewsykim) [SIG Apps]
EndpointSlice has a new NodeName field guarded by the EndpointSliceNodeName feature gate.
- EndpointSlice topology field will be deprecated in an upcoming release.
- EndpointSlice "IP" address type is formally removed after being deprecated in Kubernetes 1.17.
- The discovery.k8s.io/v1alpha1 API is deprecated and will be removed in Kubernetes 1.21. (kubernetes/kubernetes#96440, @robscott) [SIG API Machinery, Apps and Network]
External facing API podresources is now available under k8s.io/kubelet/pkg/apis/ (kubernetes/kubernetes#92632, @RenaudWasTaken) [SIG Node and Testing]
Fewer candidates are enumerated for preemption to improve performance in large clusters. (kubernetes/kubernetes#94814, @adtac)
Fix conversions for custom metrics. (kubernetes/kubernetes#94481, @wojtek-t) [SIG API Machinery and Instrumentation]
GPU metrics provided by kubelet are now disabled by default. (kubernetes/kubernetes#95184, @RenaudWasTaken)
If BoundServiceAccountTokenVolume is enabled, cluster admins can use metric serviceaccount_stale_tokens_total to monitor workloads that are depending on the extended tokens. If there are no such workloads, turn off extended tokens by starting kube-apiserver with flag --service-account-extend-token-expiration=false (kubernetes/kubernetes#96273, @zshihang) [SIG API Machinery and Auth]
Introduce alpha support for exec-based container registry credential provider plugins in the kubelet. (kubernetes/kubernetes#94196, @andrewsykim) [SIG Node and Release]
Introduces a metric source for HPAs which allows scaling based on container resource usage. (kubernetes/kubernetes#90691, @arjunrn) [SIG API Machinery, Apps, Autoscaling and CLI]
Kube-apiserver now deletes expired kube-apiserver Lease objects:
- The feature is under feature gate APIServerIdentity.
- A flag is added to kube-apiserver: identity-lease-garbage-collection-check-period-seconds (kubernetes/kubernetes#95895, @roycaihw) [SIG API Machinery, Apps, Auth and Testing]
Kube-controller-manager: volume plugins can be restricted from contacting local and loopback addresses by setting --volume-host-allow-local-loopback=false, or from contacting specific CIDR ranges by setting --volume-host-cidr-denylist (for example, --volume-host-cidr-denylist=127.0.0.1/28,feed::/16) (kubernetes/kubernetes#91785, @mattcary) [SIG API Machinery, Apps, Auth, CLI, Network, Node, Storage and Testing]
Migrate scheduler, controller-manager and cloud-controller-manager to use LeaseLock (kubernetes/kubernetes#94603, @wojtek-t) [SIG API Machinery, Apps, Cloud Provider and Scheduling]
Modify DNS-1123 error messages to indicate that RFC 1123 is not followed exactly (kubernetes/kubernetes#94182, @mattfenwick) [SIG API Machinery, Apps, Auth, Network and Node]
Move configurable fsgroup change policy for pods to beta (kubernetes/kubernetes#96376, @gnufied) [SIG Apps and Storage]
New flag is introduced, i.e. --topology-manager-scope=container|pod. The default value is the "container" scope. (kubernetes/kubernetes#92967, @cezaryzukowski) [SIG Instrumentation, Node and Testing]
New parameter defaultingType for PodTopologySpread plugin allows to use k8s defined or user provided default constraints (kubernetes/kubernetes#95048, @alculquicondor) [SIG Scheduling]
NodeAffinity plugin can be configured with AddedAffinity. (kubernetes/kubernetes#96202, @alculquicondor) [SIG Node, Scheduling and Testing]
Promote RuntimeClass feature to GA. Promote node.k8s.io API groups from v1beta1 to v1. (kubernetes/kubernetes#95718, @SergeyKanzhelev) [SIG Apps, Auth, Node, Scheduling and Testing]
Reminder: The labels "failure-domain.beta.kubernetes.io/zone" and "failure-domain.beta.kubernetes.io/region" are deprecated in favor of "topology.kubernetes.io/zone" and "topology.kubernetes.io/region" respectively. All users of the "failure-domain.beta..." labels should switch to the "topology..." equivalents. (kubernetes/kubernetes#96033, @thockin) [SIG API Machinery, Apps, CLI, Cloud Provider, Network, Node, Scheduling, Storage and Testing]
Server Side Apply now treats LabelSelector fields as atomic (meaning the entire selector is managed by a single writer and updated together), since they contain interrelated and inseparable fields that do not merge in intuitive ways. (kubernetes/kubernetes#93901, @jpbetz) [SIG API Machinery, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Storage and Testing]
Services will now have a clusterIPs field to go with clusterIP. clusterIPs[0] is a synonym for clusterIP and will be synchronized on create and update operations. (kubernetes/kubernetes#95894, @thockin) [SIG Network]
The ServiceAccountIssuerDiscovery feature gate is now Beta and enabled by default. (kubernetes/kubernetes#91921, @mtaufen) [SIG Auth]
The status of v1beta1 CRDs without "preserveUnknownFields:false" now shows a violation, "spec.preserveUnknownFields: Invalid value: true: must be false". (kubernetes/kubernetes#93078, @vareti)
The usage of mixed protocol values in the same LoadBalancer Service is possible if the new feature gate MixedProtocolLBService is enabled. The feature gate is disabled by default. The user has to enable it for the API Server. (kubernetes/kubernetes#94028, @janosi) [SIG API Machinery and Apps]
This PR will introduce a feature gate CSIServiceAccountToken with two additional fields in CSIDriverSpec. (kubernetes/kubernetes#93130, @zshihang) [SIG API Machinery, Apps, Auth, CLI, Network, Node, Storage and Testing]
Users can try the CronJob controller v2 using the feature gate. This will be the default controller in future releases. (kubernetes/kubernetes#93370, @alaypatel07) [SIG API Machinery, Apps, Auth and Testing]
VolumeSnapshotDataSource moves to GA in 1.20 release (kubernetes/kubernetes#95282, @xing-yang) [SIG Apps]
WinOverlay feature graduated to beta (kubernetes/kubernetes#94807, @ksubrmnn) [SIG Windows]
API priority and fairness graduated to beta 1.19 servers with APF turned on should not be run in a multi-server cluster with 1.20+ servers. (kubernetes/kubernetes#96527, @adtac) [SIG API Machinery and Testing]
Add LoadBalancerIPMode feature gate (kubernetes/kubernetes#92312, @Sh4d1) [SIG Apps, CLI, Cloud Provider and Network]
Add WindowsContainerResources and Annotations to CRI-API UpdateContainerResourcesRequest (kubernetes/kubernetes#95741, @katiewasnothere) [SIG Node]
Add a 'serving' and terminating condition to the EndpointSlice API.

serving tracks the readiness of endpoints regardless of their terminating state. This is distinct from ready since ready is only true when pods are not terminating. terminating is true when an endpoint is terminating. For pods this is any endpoint with a deletion timestamp. (kubernetes/kubernetes#92968, @andrewsykim) [SIG Apps and Network]
Add support for hugepages to downward API (kubernetes/kubernetes#86102, @derekwaynecarr) [SIG API Machinery, Apps, CLI, Network, Node, Scheduling and Testing]
Adds kubelet alpha feature, GracefulNodeShutdown which makes kubelet aware of node system shutdowns and result in graceful termination of pods during a system shutdown. (kubernetes/kubernetes#96129, @bobbypage) [SIG Node]
AppProtocol is now GA for Endpoints and Services. The ServiceAppProtocol feature gate will be deprecated in 1.21. (kubernetes/kubernetes#96327, @robscott) [SIG Apps and Network]
Automatic allocation of NodePorts for services with type LoadBalancer can now be disabled by setting the (new) parameter Service.spec.allocateLoadBalancerNodePorts=false. The default is to allocate NodePorts for services with type LoadBalancer which is the existing behavior. (kubernetes/kubernetes#92744, @uablrek) [SIG Apps and Network]
Document that ServiceTopology feature is required to use service.spec.topologyKeys. (kubernetes/kubernetes#96528, @andrewsykim) [SIG Apps]
EndpointSlice has a new NodeName field guarded by the EndpointSliceNodeName feature gate.
- EndpointSlice topology field will be deprecated in an upcoming release.
- EndpointSlice "IP" address type is formally removed after being deprecated in Kubernetes 1.17.
- The discovery.k8s.io/v1alpha1 API is deprecated and will be removed in Kubernetes 1.21. (kubernetes/kubernetes#96440, @robscott) [SIG API Machinery, Apps and Network]
Fewer candidates are enumerated for preemption to improve performance in large clusters (kubernetes/kubernetes#94814, @adtac) [SIG Scheduling]
If BoundServiceAccountTokenVolume is enabled, cluster admins can use metric serviceaccount_stale_tokens_total to monitor workloads that are depending on the extended tokens. If there are no such workloads, turn off extended tokens by starting kube-apiserver with flag --service-account-extend-token-expiration=false (kubernetes/kubernetes#96273, @zshihang) [SIG API Machinery and Auth]
Introduce alpha support for exec-based container registry credential provider plugins in the kubelet. (kubernetes/kubernetes#94196, @andrewsykim) [SIG Node and Release]
Kube-apiserver now deletes expired kube-apiserver Lease objects:
- The feature is under feature gate APIServerIdentity.
- A flag is added to kube-apiserver: identity-lease-garbage-collection-check-period-seconds (kubernetes/kubernetes#95895, @roycaihw) [SIG API Machinery, Apps, Auth and Testing]
Move configurable fsgroup change policy for pods to beta (kubernetes/kubernetes#96376, @gnufied) [SIG Apps and Storage]
New flag is introduced, i.e. --topology-manager-scope=container|pod. The default value is the "container" scope. (kubernetes/kubernetes#92967, @cezaryzukowski) [SIG Instrumentation, Node and Testing]
NodeAffinity plugin can be configured with AddedAffinity. (kubernetes/kubernetes#96202, @alculquicondor) [SIG Node, Scheduling and Testing]
Promote RuntimeClass feature to GA. Promote node.k8s.io API groups from v1beta1 to v1. (kubernetes/kubernetes#95718, @SergeyKanzhelev) [SIG Apps, Auth, Node, Scheduling and Testing]
Reminder: The labels "failure-domain.beta.kubernetes.io/zone" and "failure-domain.beta.kubernetes.io/region" are deprecated in favor of "topology.kubernetes.io/zone" and "topology.kubernetes.io/region" respectively. All users of the "failure-domain.beta..." labels should switch to the "topology..." equivalents. (kubernetes/kubernetes#96033, @thockin) [SIG API Machinery, Apps, CLI, Cloud Provider, Network, Node, Scheduling, Storage and Testing]
The usage of mixed protocol values in the same LoadBalancer Service is possible if the new feature gate MixedProtocolLBSVC is enabled. "action required" The feature gate is disabled by default. The user has to enable it for the API Server. (kubernetes/kubernetes#94028, @janosi) [SIG API Machinery and Apps]
This PR will introduce a feature gate CSIServiceAccountToken with two additional fields in CSIDriverSpec. (kubernetes/kubernetes#93130, @zshihang) [SIG API Machinery, Apps, Auth, CLI, Network, Node, Storage and Testing]
Users can try the CronJob controller v2 using the feature gate. This will be the default controller in future releases. (kubernetes/kubernetes#93370, @alaypatel07) [SIG API Machinery, Apps, Auth and Testing]
VolumeSnapshotDataSource moves to GA in 1.20 release (kubernetes/kubernetes#95282, @xing-yang) [SIG Apps]
- TokenRequest and TokenRequestProjection features have been promoted to GA. This feature allows generating service account tokens that are not visible in Secret objects and are tied to the lifetime of a Pod object. See https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#service-account-token-volume-projection for details on configuring and using this feature. The TokenRequest and TokenRequestProjection feature gates will be removed in v1.21.
- kubeadm's kube-apiserver Pod manifest now includes the following flags by default "--service-account-key-file", "--service-account-signing-key-file", "--service-account-issuer". (kubernetes/kubernetes#93258, @zshihang) [SIG API Machinery, Auth, Cluster Lifecycle, Storage and Testing]
Certain fields on Service objects will be automatically cleared when changing the service's type to a mode that does not need those fields. For example, changing from type=LoadBalancer to type=ClusterIP will clear the NodePort assignments, rather than forcing the user to clear them. (kubernetes/kubernetes#95196, @thockin) [SIG API Machinery, Apps, Network and Testing]
Services will now have a clusterIPs field to go with clusterIP. clusterIPs[0] is a synonym for clusterIP and will be synchronized on create and update operations. (kubernetes/kubernetes#95894, @thockin) [SIG Network]
Add dual-stack Services (alpha). This is a BREAKING CHANGE to an alpha API. It changes the dual-stack API wrt Service from a single ipFamily field to 3 fields: ipFamilyPolicy (SingleStack, PreferDualStack, RequireDualStack), ipFamilies (a list of families assigned), and clusterIPs (inclusive of clusterIP). Most users do not need to set anything at all, defaulting will handle it for them. Services are single-stack unless the user asks for dual-stack. This is all gated by the "IPv6DualStack" feature gate. (kubernetes/kubernetes#91824, @khenidak) [SIG API Machinery, Apps, CLI, Network, Node, Scheduling and Testing]
Introduces a metric source for HPAs which allows scaling based on container resource usage. (kubernetes/kubernetes#90691, @arjunrn) [SIG API Machinery, Apps, Autoscaling and CLI]
New parameter defaultingType for PodTopologySpread plugin allows to use k8s defined or user-provided default constraints (kubernetes/kubernetes#95048, @alculquicondor) [SIG Scheduling]
GPU metrics provided by kubelet are now disabled by default (kubernetes/kubernetes#95184, @RenaudWasTaken) [SIG Node]
New parameter defaultingType for PodTopologySpread plugin allows to use k8s defined or user provided default constraints (kubernetes/kubernetes#95048, @alculquicondor) [SIG Scheduling]
Server Side Apply now treats LabelSelector fields as atomic (meaning the entire selector is managed by a single writer and updated together), since they contain interrelated and inseparable fields that do not merge in intuitive ways. (kubernetes/kubernetes#93901, @jpbetz) [SIG API Machinery, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Storage and Testing]
Status of v1beta1 CRDs without "preserveUnknownFields:false" will show violation "spec.preserveUnknownFields: Invalid value: true: must be false" (kubernetes/kubernetes#93078, @vareti) [SIG API Machinery]
A new nofuzz go build tag now disables gofuzz support. Release binaries enable this. (kubernetes/kubernetes#92491, @BenTheElder) [SIG API Machinery]
A new alpha-level field, SupportsFsGroup, has been introduced for CSIDrivers to allow them to specify whether they support volume ownership and permission modifications. The CSIVolumeSupportFSGroup feature gate must be enabled to allow this field to be used. (kubernetes/kubernetes#92001, @huffmanca) [SIG API Machinery, CLI and Storage]
Added pod version skew strategy for seccomp profile to synchronize the deprecated annotations with the new API Server fields. Please see the corresponding section in the KEP for more detailed explanations. (kubernetes/kubernetes#91408, @saschagrunert) [SIG Apps, Auth, CLI and Node]
Adds the ability to disable Accelerator/GPU metrics collected by Kubelet (kubernetes/kubernetes#91930, @RenaudWasTaken) [SIG Node]
Custom Endpoints are now mirrored to EndpointSlices by a new EndpointSliceMirroring controller. (kubernetes/kubernetes#91637, @robscott) [SIG API Machinery, Apps, Auth, Cloud Provider, Instrumentation, Network and Testing]
External facing API podresources is now available under k8s.io/kubelet/pkg/apis/ (kubernetes/kubernetes#92632, @RenaudWasTaken) [SIG Node and Testing]
Fix conversions for custom metrics. (kubernetes/kubernetes#94481, @wojtek-t) [SIG API Machinery and Instrumentation]
Generic ephemeral volumes, a new alpha feature under the GenericEphemeralVolume feature gate, provide a more flexible alternative to EmptyDir volumes: as with EmptyDir, volumes are created and deleted for each pod automatically by Kubernetes. But because the normal provisioning process is used (PersistentVolumeClaim), storage can be provided by third-party storage vendors and all of the usual volume features work. Volumes don't need to be empty; for example, restoring from snapshot is supported. (kubernetes/kubernetes#92784, @pohly) [SIG API Machinery, Apps, Auth, CLI, Instrumentation, Node, Scheduling, Storage and Testing]
Kube-controller-manager: volume plugins can be restricted from contacting local and loopback addresses by setting --volume-host-allow-local-loopback=false, or from contacting specific CIDR ranges by setting --volume-host-cidr-denylist (for example, --volume-host-cidr-denylist=127.0.0.1/28,feed::/16) (kubernetes/kubernetes#91785, @mattcary) [SIG API Machinery, Apps, Auth, CLI, Network, Node, Storage and Testing]
Kubernetes is now built with golang 1.15.0-rc.1.
- The deprecated, legacy behavior of treating the CommonName field on X.509 serving certificates as a host name when no Subject Alternative Names are present is now disabled by default. It can be temporarily re-enabled by adding the value x509ignoreCN=0 to the GODEBUG environment variable. (kubernetes/kubernetes#93264, @justaugustus) [SIG API Machinery, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scalability, Storage and Testing]
Migrate scheduler, controller-manager and cloud-controller-manager to use LeaseLock (kubernetes/kubernetes#94603, @wojtek-t) [SIG API Machinery, Apps, Cloud Provider and Scheduling]
Modify DNS-1123 error messages to indicate that RFC 1123 is not followed exactly (kubernetes/kubernetes#94182, @mattfenwick) [SIG API Machinery, Apps, Auth, Network and Node]
The ServiceAccountIssuerDiscovery feature gate is now Beta and enabled by default. (kubernetes/kubernetes#91921, @mtaufen) [SIG Auth]
The kube-controller-manager managed signers can now have distinct signing certificates and keys. See the help about --cluster-signing-[signer-name]-{cert,key}-file. --cluster-signing-{cert,key}-file is still the default. (kubernetes/kubernetes#90822, @deads2k) [SIG API Machinery, Apps and Auth]
When creating a networking.k8s.io/v1 Ingress API object, spec.tls[*].secretName values are required to pass validation rules for Secret API object names. (kubernetes/kubernetes#93929, @liggitt) [SIG Network]
WinOverlay feature graduated to beta (kubernetes/kubernetes#94807, @ksubrmnn) [SIG Windows]

v19.15.1

fix: watch returns raw_object if detection of returned objects fail (#177, @tomplus)

v19.15.0

feat: Kubernetes API Version: v1.19.15

API Change

We have added a new Priority & Fairness rule that exempts all probes (/readyz, /healthz, /livez) to prevent restarting of "healthy" kube-apiserver instance(s) by kubelet. (kubernetes/kubernetes#101113, @tkashem) [SIG API Machinery]
Fixes using server-side apply with APIService resources (kubernetes/kubernetes#100713, @kevindelgado) [SIG API Machinery, Apps, Scheduling and Testing]
Regenerate protobuf code to fix CVE-2021-3121 (kubernetes/kubernetes#100515, @joelsmith) [SIG API Machinery, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node and Storage]
Kubernetes is now built using go1.15.8 (kubernetes/kubernetes#99093, @cpanato) [SIG Cloud Provider, Instrumentation, Release and Testing]
Fix conversions for custom metrics. (kubernetes/kubernetes#94654, @wojtek-t) [SIG Instrumentation]
A new alpha-level field, SupportsFsGroup, has been introduced for CSIDrivers to allow them to specify whether they support volume ownership and permission modifications. The CSIVolumeSupportFSGroup feature gate must be enabled to allow this field to be used. (kubernetes/kubernetes#92001, @huffmanca) [SIG API Machinery, CLI and Storage]
Added pod version skew strategy for seccomp profile to synchronize the deprecated annotations with the new API Server fields. Please see the corresponding section in the KEP for more detailed explanations. (kubernetes/kubernetes#91408, @saschagrunert) [SIG Apps, Auth, CLI and Node]
Adds the ability to disable Accelerator/GPU metrics collected by Kubelet (kubernetes/kubernetes#91930, @RenaudWasTaken) [SIG Node]
Admission webhooks can now return warning messages that are surfaced to API clients, using the .response.warnings field in the admission review response. (kubernetes/kubernetes#92667, @liggitt) [SIG API Machinery and Testing]
CertificateSigningRequest API conditions were updated:
- a status field was added; this field defaults to True, and may only be set to True for Approved, Denied, and Failed conditions
- a lastTransitionTime field was added
- a Failed condition type was added to allow signers to indicate permanent failure; this condition can be added via the certificatesigningrequests/status subresource.
- Approved and Denied conditions are mutually exclusive
- Approved, Denied, and Failed conditions can no longer be removed from a CSR (kubernetes/kubernetes#90191, @liggitt) [SIG API Machinery, Apps, Auth, CLI and Node]
Cluster admins can now turn off /logs endpoint in kubelet by setting enableSystemLogHandler to false in their kubelet configuration file. enableSystemLogHandler can be set to true only when enableDebuggingHandlers is also set to true. (kubernetes/kubernetes#87273, @SaranBalaji90) [SIG Node]
Custom Endpoints are now mirrored to EndpointSlices by a new EndpointSliceMirroring controller. (kubernetes/kubernetes#91637, @robscott) [SIG API Machinery, Apps, Auth, Cloud Provider, Instrumentation, Network and Testing]
CustomResourceDefinitions added support for marking versions as deprecated by setting spec.versions[*].deprecated to true, and for optionally overriding the default deprecation warning with a spec.versions[*].deprecationWarning field. (kubernetes/kubernetes#92329, @liggitt) [SIG API Machinery]
EnvVarSource api doc bug fixes (kubernetes/kubernetes#91194, @wawa0210) [SIG Apps]
Fix bug in reflector that couldn't recover from "Too large resource version" errors (kubernetes/kubernetes#92537, @wojtek-t) [SIG API Machinery]
Fixed: log timestamps now include trailing zeros to maintain a fixed width (kubernetes/kubernetes#91207, @iamchuckss) [SIG Apps and Node]
Generic ephemeral volumes, a new alpha feature under the GenericEphemeralVolume feature gate, provide a more flexible alternative to EmptyDir volumes: as with EmptyDir, volumes are created and deleted for each pod automatically by Kubernetes. But because the normal provisioning process is used (PersistentVolumeClaim), storage can be provided by third-party storage vendors and all of the usual volume features work. Volumes don't need to be empt; for example, restoring from snapshot is supported. (kubernetes/kubernetes#92784, @pohly) [SIG API Machinery, Apps, Auth, CLI, Instrumentation, Node, Scheduling, Storage and Testing]
Go1.14.4 is now the minimum version required for building Kubernetes (kubernetes/kubernetes#92438, @liggitt) [SIG API Machinery, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Storage and Testing]
Hide managedFields from kubectl edit command (kubernetes/kubernetes#91946, @soltysh) [SIG CLI]
K8s.io/apimachinery - scheme.Convert() now uses only explicitly registered conversions - default reflection based conversion is no longer available. +k8s:conversion-gen tags can be used with the k8s.io/code-generator component to generate conversions. (kubernetes/kubernetes#90018, @wojtek-t) [SIG API Machinery, Apps and Testing]
Kube-proxy: add --bind-address-hard-fail flag to treat failure to bind to a port as fatal (kubernetes/kubernetes#89350, @SataQiu) [SIG Cluster Lifecycle and Network]
Kubebuilder validation tags are set on metav1.Condition for CRD generation (kubernetes/kubernetes#92660, @damemi) [SIG API Machinery]
Kubelet's --runonce option is now also available in Kubelet's config file as runOnce. (kubernetes/kubernetes#89128, @vincent178) [SIG Node]
Kubelet: add '--logging-format' flag to support structured logging (kubernetes/kubernetes#91532, @afrouzMashaykhi) [SIG API Machinery, Cluster Lifecycle, Instrumentation and Node]
Kubernetes is now built with golang 1.15.0-rc.1.
- The deprecated, legacy behavior of treating the CommonName field on X.509 serving certificates as a host name when no Subject Alternative Names are present is now disabled by default. It can be temporarily re-enabled by adding the value x509ignoreCN=0 to the GODEBUG environment variable. (kubernetes/kubernetes#93264, @justaugustus) [SIG API Machinery, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scalability, Storage and Testing]
Promote Immutable Secrets/ConfigMaps feature to Beta and enable the feature by default. This allows to set Immutable field in Secrets or ConfigMap object to mark their contents as immutable. (kubernetes/kubernetes#89594, @wojtek-t) [SIG Apps and Testing]
Remove BindTimeoutSeconds from schedule configuration KubeSchedulerConfiguration (kubernetes/kubernetes#91580, @cofyc) [SIG Scheduling and Testing]
Remove kubescheduler.config.k8s.io/v1alpha1 (kubernetes/kubernetes#89298, @gavinfish) [SIG Scheduling]
Reserve plugins that fail to reserve will trigger the unreserve extension point (kubernetes/kubernetes#92391, @adtac) [SIG Scheduling and Testing]
Resolve regression in metadata.managedFields handling in update/patch requests submitted by older API clients (kubernetes/kubernetes#91748, @apelisse)
Scheduler: optionally check for available storage capacity before scheduling pods which have unbound volumes (alpha feature with the new CSIStorageCapacity feature gate, only works for CSI drivers and depends on support for the feature in a CSI driver deployment) (kubernetes/kubernetes#92387, @pohly) [SIG API Machinery, Apps, Auth, Scheduling, Storage and Testing]
Seccomp support has graduated to GA. A new seccompProfile field is added to pod and container securityContext objects. Support for seccomp.security.alpha.kubernetes.io/pod and container.seccomp.security.alpha.kubernetes.io/... annotations is deprecated, and will be removed in v1.22. (kubernetes/kubernetes#91381, @pjbgf) [SIG Apps, Auth, Node, Release, Scheduling and Testing]
ServiceAppProtocol feature gate is now beta and enabled by default, adding new AppProtocol field to Services and Endpoints. (kubernetes/kubernetes#90023, @robscott) [SIG Apps and Network]
SetHostnameAsFQDN is a new field in PodSpec. When set to true, the fully qualified domain name (FQDN) of a Pod is set as hostname of its containers. In Linux containers, this means setting the FQDN in the hostname field of the kernel (the nodename field of struct utsname). In Windows containers, this means setting the this means setting the registry value of hostname for the registry key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters to FQDN. If a pod does not have FQDN, this has no effect. (kubernetes/kubernetes#91699, @javidiaz) [SIG Apps, Network, Node and Testing]
The CertificateSigningRequest API is promoted to certificates.k8s.io/v1 with the following changes:
- spec.signerName is now required, and requests for kubernetes.io/legacy-unknown are not allowed to be created via the certificates.k8s.io/v1 API
- spec.usages is now required, may not contain duplicate values, and must only contain known usages
- status.conditions may not contain duplicate types
- status.conditions[*].status is now required
- status.certificate must be PEM-encoded, and contain only CERTIFICATE blocks (kubernetes/kubernetes#91685, @liggitt) [SIG API Machinery, Architecture, Auth, CLI and Testing]
The HugePageStorageMediumSize feature gate is now on by default allowing usage of multiple sizes huge page resources on a container level. (kubernetes/kubernetes#90592, @bart0sh) [SIG Node]
The Kubelet's --node-status-max-images option is now available via the Kubelet config file field nodeStatusMaxImage (kubernetes/kubernetes#91275, @knabben) [SIG Node]
The Kubelet's --seccomp-profile-root option is now marked as deprecated. (kubernetes/kubernetes#91182, @knabben) [SIG Node]
The Kubelet's --bootstrap-checkpoint-path option is now removed. (kubernetes/kubernetes#91577, @knabben) [SIG Apps and Node]
The Kubelet's --cloud-provider and --cloud-config options are now marked as deprecated. (kubernetes/kubernetes#90408, @knabben) [SIG Cloud Provider and Node]
The Kubelet's --enable-server and --provider-id option is now available via the Kubelet config file field enableServer and providerID respectively. (kubernetes/kubernetes#90494, @knabben) [SIG Node]
The Kubelet's --kernel-memcg-notification option is now available via the Kubelet config file field kernelMemcgNotification (kubernetes/kubernetes#91863, @knabben) [SIG Cloud Provider, Node and Testing]
The Kubelet's --really-crash-for-testing and --chaos-chance options are now marked as deprecated. (kubernetes/kubernetes#90499, @knabben) [SIG Node]
The Kubelet's --volume-plugin-dir option is now available via the Kubelet config file field VolumePluginDir. (kubernetes/kubernetes#88480, @savitharaghunathan) [SIG Node]
The DefaultIngressClass feature is now GA. The --feature-gate parameter will be removed in 1.20. (kubernetes/kubernetes#91957, @cmluciano) [SIG API Machinery, Apps, Network and Testing]
The alpha DynamicAuditing feature gate and auditregistration.k8s.io/v1alpha1 API have been removed and are no longer supported. (kubernetes/kubernetes#91502, @deads2k) [SIG API Machinery, Auth and Testing]
The kube-controller-manager managed signers can now have distinct signing certificates and keys. See the help about --cluster-signing-[signer-name]-{cert,key}-file. --cluster-signing-{cert,key}-file is still the default. (kubernetes/kubernetes#90822, @deads2k) [SIG API Machinery, Apps and Auth]
The unused series.state field, deprecated since v1.14, is removed from the events.k8s.io/v1beta1 and v1 Event types. (kubernetes/kubernetes#90449, @wojtek-t) [SIG Apps]
Unreserve extension point for scheduler plugins is merged into Reserve extension point (kubernetes/kubernetes#92200, @adtac) [SIG Scheduling and Testing]
Update Golang to v1.14.4 (kubernetes/kubernetes#88638, @justaugustus) [SIG API Machinery, Cloud Provider, Release and Testing]
Updated the API documentation for Service.Spec.IPFamily to warn that its exact semantics will probably change before the dual-stack feature goes GA, and users should look at ClusterIP or Endpoints, not IPFamily, to figure out if an existing Service is IPv4, IPv6, or dual-stack. (kubernetes/kubernetes#91527, @danwinship) [SIG Apps and Network]
Users can configure a resource prefix to ignore a group of resources. (kubernetes/kubernetes#88842, @angao) [SIG Node and Scheduling]
Ingress and IngressClass resources have graduated to networking.k8s.io/v1. Ingress and IngressClass types in the extensions/v1beta1 and networking.k8s.io/v1beta1 API versions are deprecated and will no longer be served in 1.22+. Persisted objects can be accessed via the networking.k8s.io/v1 API. Notable changes in v1 Ingress objects (v1beta1 field names are unchanged):
- spec.backend -> spec.defaultBackend
- serviceName -> service.name
- servicePort -> service.port.name (for string values)
- servicePort -> service.port.number (for numeric values)
- pathType no longer has a default value in v1; "Exact", "Prefix", or "ImplementationSpecific" must be specified Other Ingress API updates:
- backends can now be resource or service backends
- path is no longer required to be a valid regular expression (kubernetes/kubernetes#89778, @cmluciano) [SIG API Machinery, Apps, CLI, Network and Testing]
NodeResourcesLeastAllocated and NodeResourcesMostAllocated plugins now support customized weight on the CPU and memory. (kubernetes/kubernetes#90544, @chendave) [SIG Scheduling]
PostFilter type is added to scheduler component config API on version v1beta1. (kubernetes/kubernetes#91547, @Huang-Wei) [SIG Scheduling]
RequestedToCapacityRatioArgs encoding is now strict (kubernetes/kubernetes#91603, @pancernik) [SIG Scheduling]
v1beta1 Scheduler Extender encoding is case-sensitive (v1alpha1/v1alpha2 was case-insensitive), its httpTimeout field uses duration encoding (for example, one second is specified as "1s"), and the enableHttps field in v1alpha1/v1alpha2 was renamed to enableHTTPS. (kubernetes/kubernetes#91625, @pancernik) [SIG Scheduling]
Adds the ability to disable Accelerator/GPU metrics collected by Kubelet (kubernetes/kubernetes#91930, @RenaudWasTaken) [SIG Node]
Kubernetes is now built with golang 1.15.0-rc.1.
- The deprecated, legacy behavior of treating the CommonName field on X.509 serving certificates as a host name when no Subject Alternative Names are present is now disabled by default. It can be temporarily re-enabled by adding the value x509ignoreCN=0 to the GODEBUG environment variable. (kubernetes/kubernetes#93264, @justaugustus) [SIG API Machinery, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scalability, Storage and Testing]
A new alpha-level field, SupportsFsGroup, has been introduced for CSIDrivers to allow them to specify whether they support volume ownership and permission modifications. The CSIVolumeSupportFSGroup feature gate must be enabled to allow this field to be used. (kubernetes/kubernetes#92001, @huffmanca) [SIG API Machinery, CLI and Storage]
The kube-controller-manager managed signers can now have distinct signing certificates and keys. See the help about --cluster-signing-[signer-name]-{cert,key}-file. --cluster-signing-{cert,key}-file is still the default. (kubernetes/kubernetes#90822, @deads2k) [SIG API Machinery, Apps and Auth]
Added pod version skew strategy for seccomp profile to synchronize the deprecated annotations with the new API Server fields. Please see the corresponding section in the KEP for more detailed explanations. (kubernetes/kubernetes#91408, @saschagrunert) [SIG Apps, Auth, CLI and Node]
Custom Endpoints are now mirrored to EndpointSlices by a new EndpointSliceMirroring controller. (kubernetes/kubernetes#91637, @robscott) [SIG API Machinery, Apps, Auth, Cloud Provider, Instrumentation, Network and Testing]
Generic ephemeral volumes, a new alpha feature under the GenericEphemeralVolume feature gate, provide a more flexible alternative to EmptyDir volumes: as with EmptyDir, volumes are created and deleted for each pod automatically by Kubernetes. But because the normal provisioning process is used (PersistentVolumeClaim), storage can be provided by third-party storage vendors and all of the usual volume features work. Volumes don't need to be empt; for example, restoring from snapshot is supported. (kubernetes/kubernetes#92784, @pohly) [SIG API Machinery, Apps, Auth, CLI, Instrumentation, Node, Scheduling, Storage and Testing]
Remove BindTimeoutSeconds from schedule configuration KubeSchedulerConfiguration (kubernetes/kubernetes#91580, @cofyc) [SIG Scheduling and Testing]
Resolve regression in metadata.managedFields handling in update/patch requests submitted by older API clients (kubernetes/kubernetes#91748, @apelisse) [SIG API Machinery and Testing]
The CertificateSigningRequest API is promoted to certificates.k8s.io/v1 with the following changes:
- spec.signerName is now required, and requests for kubernetes.io/legacy-unknown are not allowed to be created via the certificates.k8s.io/v1 API
- spec.usages is now required, may not contain duplicate values, and must only contain known usages
- status.conditions may not contain duplicate types
- status.conditions[*].status is now required
- status.certificate must be PEM-encoded, and contain only CERTIFICATE blocks (kubernetes/kubernetes#91685, @liggitt) [SIG API Machinery, Architecture, Auth, CLI and Testing]
The Kubelet's --cloud-provider and --cloud-config options are now marked as deprecated. (kubernetes/kubernetes#90408, @knabben) [SIG Cloud Provider and Node]
CertificateSigningRequest API conditions were updated:
- a status field was added; this field defaults to True, and may only be set to True for Approved, Denied, and Failed conditions
- a lastTransitionTime field was added
- a Failed condition type was added to allow signers to indicate permanent failure; this condition can be added via the certificatesigningrequests/status subresource.
- Approved and Denied conditions are mutually exclusive
- Approved, Denied, and Failed conditions can no longer be removed from a CSR (kubernetes/kubernetes#90191, @liggitt) [SIG API Machinery, Apps, Auth, CLI and Node]
EnvVarSource api doc bug fixes (kubernetes/kubernetes#91194, @wawa0210) [SIG Apps]
Fixed: log timestamps now include trailing zeros to maintain a fixed width (kubernetes/kubernetes#91207, @iamchuckss) [SIG Apps and Node]
The Kubelet's --node-status-max-images option is now available via the Kubelet config file field nodeStatusMaxImage (kubernetes/kubernetes#91275, @knabben) [SIG Node]
The Kubelet's --seccomp-profile-root option is now available via the Kubelet config file field seccompProfileRoot. (kubernetes/kubernetes#91182, @knabben) [SIG Node]
The Kubelet's --enable-server and --provider-id option is now available via the Kubelet config file field enableServer and providerID respectively. (kubernetes/kubernetes#90494, @knabben) [SIG Node]
The Kubelet's --really-crash-for-testing and --chaos-chance options are now marked as deprecated. (kubernetes/kubernetes#90499, @knabben) [SIG Node]
The alpha DynamicAuditing feature gate and auditregistration.k8s.io/v1alpha1 API have been removed and are no longer supported. (kubernetes/kubernetes#91502, @deads2k) [SIG API Machinery, Auth and Testing]
NodeResourcesLeastAllocated and NodeResourcesMostAllocated plugins now support customized weight on the CPU and memory. (kubernetes/kubernetes#90544, @chendave) [SIG Scheduling]
PostFilter type is added to scheduler component config API on version v1beta1. (kubernetes/kubernetes#91547, @Huang-Wei) [SIG Scheduling]
kubescheduler.config.k8s.io is now beta (kubernetes/kubernetes#91420, @pancernik) [SIG Scheduling]
EnvVarSource api doc bug fixes (kubernetes/kubernetes#91194, @wawa0210) [SIG Apps]
The Kubelet's --really-crash-for-testing and --chaos-chance options are now marked as deprecated. (kubernetes/kubernetes#90499, @knabben) [SIG Node]
NodeResourcesLeastAllocated and NodeResourcesMostAllocated plugins now support customized weight on the CPU and memory. (kubernetes/kubernetes#90544, @chendave) [SIG Scheduling]
K8s.io/apimachinery - scheme.Convert() now uses only explicitly registered conversions - default reflection based conversion is no longer available. +k8s:conversion-gen tags can be used with the k8s.io/code-generator component to generate conversions. (kubernetes/kubernetes#90018, @wojtek-t) [SIG API Machinery, Apps and Testing]
Kubelet's --runonce option is now also available in Kubelet's config file as runOnce. (kubernetes/kubernetes#89128, @vincent178) [SIG Node]
Promote Immutable Secrets/ConfigMaps feature to Beta and enable the feature by default. This allows to set Immutable field in Secrets or ConfigMap object to mark their contents as immutable. (kubernetes/kubernetes#89594, @wojtek-t) [SIG Apps and Testing]
The unused series.state field, deprecated since v1.14, is removed from the events.k8s.io/v1beta1 and v1 Event types. (kubernetes/kubernetes#90449, @wojtek-t) [SIG Apps]
Kube-proxy: add --bind-address-hard-fail flag to treat failure to bind to a port as fatal (kubernetes/kubernetes#89350, @SataQiu) [SIG Cluster Lifecycle and Network]
Remove kubescheduler.config.k8s.io/v1alpha1 (kubernetes/kubernetes#89298, @gavinfish) [SIG Scheduling]
ServiceAppProtocol feature gate is now beta and enabled by default, adding new AppProtocol field to Services and Endpoints. (kubernetes/kubernetes#90023, @robscott) [SIG Apps and Network]
The Kubelet's --volume-plugin-dir option is now available via the Kubelet config file field VolumePluginDir. (kubernetes/kubernetes#88480, @savitharaghunathan) [SIG Node]
A new IngressClass resource has been added to enable better Ingress configuration. (kubernetes/kubernetes#88509, @robscott) [SIG API Machinery, Apps, CLI, Network, Node and Testing]
API additions to apiserver types (kubernetes/kubernetes#87179, @Jefftree) [SIG API Machinery, Cloud Provider and Cluster Lifecycle]
Add Scheduling Profiles to kubescheduler.config.k8s.io/v1alpha2 (kubernetes/kubernetes#88087, @alculquicondor) [SIG Scheduling and Testing]
Added GenericPVCDataSource feature gate to enable using arbitrary custom resources as the data source for a PVC. (kubernetes/kubernetes#88636, @bswartz) [SIG Apps and Storage]
Added support for multiple sizes huge pages on a container level (kubernetes/kubernetes#84051, @bart0sh) [SIG Apps, Node and Storage]
Allow user to specify fsgroup permission change policy for pods (kubernetes/kubernetes#88488, @gnufied) [SIG Apps and Storage]
AppProtocol is a new field on Service and Endpoints resources, enabled with the ServiceAppProtocol feature gate. (kubernetes/kubernetes#88503, @robscott) [SIG Apps and Network]
BlockVolume and CSIBlockVolume features are now GA. (kubernetes/kubernetes#88673, @jsafrane) [SIG Apps, Node and Storage]
Consumers of the 'certificatesigningrequests/approval' API must now grant permission to 'approve' CSRs for the 'signerName' specified on the CSR. More information on the new signerName field can be found at https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/1513-certificate-signing-request/README.md/#signers (kubernetes/kubernetes#88246, @munnerz) [SIG API Machinery, Apps, Auth, CLI, Node and Testing]
CustomResourceDefinition schemas that use x-kubernetes-list-map-keys to specify properties that uniquely identify list items must make those properties required or have a default value, to ensure those properties are present for all list items. See https://kubernetes.io/docs/reference/using-api/api-concepts/#merge-strategy for details. (kubernetes/kubernetes#88076, @eloyekunle) [SIG API Machinery and Testing]
Fixed missing validation of uniqueness of list items in lists with x-kubernetes-list-type: map or x-kubernetes-list-type: set in CustomResources. (kubernetes/kubernetes#84920, @sttts) [SIG API Machinery]
Fixes a regression with clients prior to 1.15 not being able to update podIP in pod status, or podCIDR in node spec, against >= 1.16 API servers (kubernetes/kubernetes#88505, @liggitt) [SIG Apps and Network]
Ingress: Add Exact and Prefix maching to Ingress PathTypes (kubernetes/kubernetes#88587, @cmluciano) [SIG Apps, Cluster Lifecycle and Network]
Ingress: Add alternate backends via TypedLocalObjectReference (kubernetes/kubernetes#88775, @cmluciano) [SIG Apps and Network]
Ingress: allow wildcard hosts in IngressRule (kubernetes/kubernetes#88858, @cmluciano) [SIG Network]
Introduces optional --detect-local flag to kube-proxy. Currently the only supported value is "cluster-cidr", which is the default if not specified. (kubernetes/kubernetes#87748, @satyasm) [SIG Cluster Lifecycle, Network and Scheduling]
Kube-controller-manager and kube-scheduler expose profiling by default to match the kube-apiserver. Use --profiling=false to disable. (kubernetes/kubernetes#88663, @deads2k) [SIG API Machinery, Cloud Provider and Scheduling]
Kube-scheduler can run more than one scheduling profile. Given a pod, the profile is selected by using its .spec.SchedulerName. (kubernetes/kubernetes#88285, @alculquicondor) [SIG Apps, Scheduling and Testing]
Move TaintBasedEvictions feature gates to GA (kubernetes/kubernetes#87487, @skilxn-go) [SIG API Machinery, Apps, Node, Scheduling and Testing]
Moving Windows RunAsUserName feature to GA (kubernetes/kubernetes#87790, @marosset) [SIG Apps and Windows]
New flag --endpointslice-updates-batch-period in kube-controller-manager can be used to reduce number of endpointslice updates generated by pod changes. (kubernetes/kubernetes#88745, @mborsz) [SIG API Machinery, Apps and Network]
New flag --show-hidden-metrics-for-version in kubelet can be used to show all hidden metrics that deprecated in the previous minor release. (kubernetes/kubernetes#85282, @serathius) [SIG Node]
Removes ConfigMap as suggestion for IngressClass parameters (kubernetes/kubernetes#89093, @robscott) [SIG Network]
Scheduler Extenders can now be configured in the v1alpha2 component config (kubernetes/kubernetes#88768, @damemi) [SIG Release, Scheduling and Testing]
The apiserver/v1alph1 #EgressSelectorConfiguration API is now beta. (kubernetes/kubernetes#88502, @caesarxuchao) [SIG API Machinery]
The storage.k8s.io/CSIDriver has moved to GA, and is now available for use. (kubernetes/kubernetes#84814, @huffmanca) [SIG API Machinery, Apps, Auth, Node, Scheduling, Storage and Testing]
VolumePVCDataSource moves to GA in 1.18 release (kubernetes/kubernetes#88686, @j-griffith) [SIG Apps, CLI and Cluster Lifecycle]

v18.20.1

feat: load kubeconfig from dict (#169, @tomplus)

v18.20.0

Important Information:

The library versioning scheme has been changed. Starting from this release, the library uses a version format vY.Z.P where Y and Z are respectively from the Kubernetes version v1.Y.Z and P would incremented due to changes on the library side itself. Ref: kubernetes-client/python#1244

API Deprecations:

The following deprecated APIs can no longer be served:
All resources under apps/v1beta1 and apps/v1beta2 - use apps/v1 instead
daemonsets, deployments, replicasets resources under extensions/v1beta1 - use apps/v1 instead
networkpolicies resources under extensions/v1beta1 - use networking.k8s.io/v1 instead
podsecuritypolicies resources under extensions/v1beta1 - use policy/v1beta1 instead (#85903, @liggitt) [SIG API Machinery, Apps, Cluster Lifecycle, Instrumentation and Testing]

API Change:

Fix bug in reflector that couldn't recover from "Too large resource version" errors (#92537, @wojtek-t) [SIG API Machinery]
Fixed: log timestamps now include trailing zeros to maintain a fixed width (#91207, @iamchuckss) [SIG Apps and Node]
Fixed: log timestamps now include trailing zeros to maintain a fixed width (#91207, @iamchuckss) [SIG Apps and Node]
Resolve regression in metadata.managedFields handling in update/patch requests submitted by older API clients (#92007, @apelisse) [SIG API Machinery and Testing]
A new IngressClass resource has been added to enable better Ingress configuration. (#88509, @robscott) [SIG API Machinery, Apps, CLI, Network, Node and Testing]
The CSIDriver API has graduated to storage.k8s.io/v1, and is now available for use. (#84814, @huffmanca) [SIG Storage]
autoscaling/v2beta2 HorizontalPodAutoscaler added a spec.behavior field that allows scale behavior to be configured. Behaviors are specified separately for scaling up and down. In each direction a stabilization window can be specified as well as a list of policies and how to select amongst them. Policies can limit the absolute number of pods added or removed, or the percentage of pods added or removed. (#74525, @gliush) [SIG API Machinery, Apps, Autoscaling and CLI]
Ingress:
- spec.ingressClassName replaces the deprecated kubernetes.io/ingress.class annotation, and allows associating an Ingress object with a particular controller.
- path definitions added a pathType field to allow indicating how the specified path should be matched against incoming requests. Valid values are Exact, Prefix, and ImplementationSpecific (#88587, @cmluciano) [SIG Apps, Cluster Lifecycle and Network]
The alpha feature AnyVolumeDataSource enables PersistentVolumeClaim objects to use the spec.dataSource field to reference a custom type as a data source (#88636, @bswartz) [SIG Apps and Storage]
The alpha feature ConfigurableFSGroupPolicy enables v1 Pods to specify a spec.securityContext.fsGroupChangePolicy policy to control how file permissions are applied to volumes mounted into the pod. (#88488, @gnufied) [SIG Storage]
The alpha feature ServiceAppProtocol enables setting an appProtocol field in ServicePort and EndpointPort definitions. (#88503, @robscott) [SIG Apps and Network]
The alpha feature ImmutableEphemeralVolumes enables an immutable field in both Secret and ConfigMap objects to mark their contents as immutable. (#86377, @wojtek-t) [SIG Apps, CLI and Testing]
The beta feature ServerSideApply enables tracking and managing changed fields for all new objects, which means there will be managedFields in metadata with the list of managers and their owned fields.
The alpha feature ServiceAccountIssuerDiscovery enables publishing OIDC discovery information and service account token verification keys at /.well-known/openid-configuration and /openid/v1/jwks endpoints by API servers configured to issue service account tokens. (#80724, @cceckman) [SIG API Machinery, Auth, Cluster Lifecycle and Testing]
CustomResourceDefinition schemas that use x-kubernetes-list-map-keys to specify properties that uniquely identify list items must make those properties required or have a default value, to ensure those properties are present for all list items. See https://kubernetes.io/docs/reference/using-api/api-concepts/#merge-strategy for details. (#88076, @eloyekunle) [SIG API Machinery and Testing]
CustomResourceDefinition schemas that use x-kubernetes-list-type: map or x-kubernetes-list-type: set now enable validation that the list items in the corresponding custom resources are unique. (#84920, @sttts) [SIG API Machinery]

To read the full CHANGELOG visit here.

v12.1.2

fix: handle 401 returned by GKE (#154, @tomplus)

v12.1.1

fix: Watch() raises exceptions for received errors (#151, @tomplus)

v12.1.0

feat: add function to create objects from dict (#143, @tomplus)
feat: Increase aiohttp read buffer to 2MiB (#138, @JacobHenner)
fix: Missing create_from_yaml_single_item import (#133, @VideoSystemsTech)
fix: show warning if config not loaded #127 (#127, @tomplus)

v12.0.1

fix: remove checking headers if not preloaded content is returned from rest api (#123, @tomplus)

v12.0.0

feat: regenerate client for Kubernetes API Version: 1.16.14 using OpenAPI 4.3.1
fix: Removed shlex args mangling (#110, @WoLpH)
fix: remove redundant close() in Watch class (#119, @tomplus)

API Change:

Resolve regression in metadata.managedFields handling in update/patch requests submitted by older API clients (#91748, @apelisse) [SIG API Machinery and Testing]
Fix bug where sending a status update completely wipes managedFields for some types. (#90033, @apelisse) [SIG API Machinery and Testing]
The MutatingWebhookConfiguration and ValidatingWebhookConfiguration APIs have been promoted to admissionregistration.k8s.io/v1:
- failurePolicy default changed from Ignore to Fail for v1
- matchPolicy default changed from Exact to Equivalent for v1
- timeout default changed from 30s to 10s for v1
- sideEffects default value is removed, and the field made required, and only None and NoneOnDryRun are permitted for v1
- admissionReviewVersions default value is removed and the field made required for v1 (supported versions for AdmissionReview are v1 and v1beta1)
- The name field for specified webhooks must be unique for MutatingWebhookConfiguration and ValidatingWebhookConfiguration objects created via admissionregistration.k8s.io/v1
The AdmissionReview API sent to and received from admission webhooks has been promoted to admission.k8s.io/v1. Webhooks can specify a preference for receiving v1 AdmissionReview objects with admissionReviewVersions: ["v1","v1beta1"], and must respond with an API object in the same apiVersion they are sent. When webhooks use admission.k8s.io/v1, the following additional validation is performed on their responses:
- response.patch and response.patchType are not permitted from validating admission webhooks
- apiVersion: "admission.k8s.io/v1" is required
- kind: "AdmissionReview" is required
- response.uid: "<value of request.uid>" is required
- response.patchType: "JSONPatch" is required (if response.patch is set) (#80231, @liggitt)
The CustomResourceDefinition API type is promoted to apiextensions.k8s.io/v1 with the following changes:
- Use of the new default feature in validation schemas is limited to v1
- spec.scope is no longer defaulted to Namespaced and must be explicitly specified
- spec.version is removed in v1; use spec.versions instead
- spec.validation is removed in v1; use spec.versions[*].schema instead
- spec.subresources is removed in v1; use spec.versions[*].subresources instead
- spec.additionalPrinterColumns is removed in v1; use spec.versions[*].additionalPrinterColumns instead
- spec.conversion.webhookClientConfig is moved to spec.conversion.webhook.clientConfig in v1
- spec.conversion.conversionReviewVersions is moved to spec.conversion.webhook.conversionReviewVersions in v1
- spec.versions[*].schema.openAPIV3Schema is now required when creating v1 CustomResourceDefinitions
- spec.preserveUnknownFields: true is disallowed when creating v1 CustomResourceDefinitions; it must be specified within schema definitions as x-kubernetes-preserve-unknown-fields: true
- In additionalPrinterColumns items, the JSONPath field was renamed to jsonPath in v1 (fixes kubernetes/kubernetes#66531) The apiextensions.k8s.io/v1beta1 version of CustomResourceDefinition is deprecated and will no longer be served in v1.19. (#79604, @liggitt)
The ConversionReview API sent to and received from custom resource CustomResourceDefinition conversion webhooks has been promoted to apiextensions.k8s.io/v1. CustomResourceDefinition conversion webhooks can now indicate they support receiving and responding with ConversionReview API objects in the apiextensions.k8s.io/v1 version by including v1 in the conversionReviewVersions list in their CustomResourceDefinition. Conversion webhooks must respond with a ConversionReview object in the same apiVersion they receive. apiextensions.k8s.io/v1 ConversionReview responses must specify a response.uid that matches the request.uid of the object they were sent. (#81476, @liggitt)
Add scheduling support for RuntimeClasses. RuntimeClasses can now specify nodeSelector constraints & tolerations, which are merged into the PodSpec for pods using that RuntimeClass. (#80825, @tallclair)
Kubelet should now more reliably report the same primary node IP even if the set of node IPs reported by the CloudProvider changes. (#79391, @danwinship)
Omit nil or empty field when calculating container hash value to avoid hash changed. For a new field with a non-nil default value in the container spec, the hash would still get changed. (#57741, @dixudx)
Property conditions in apiextensions.v1beta1.CustomResourceDefinitionStatus and apiextensions.v1.CustomResourceDefinitionStatus is now optional instead of required. (#64996, @roycaihw)
When the status of a CustomResourceDefinition condition changes, its corresponding lastTransitionTime is now updated. (#69655, @CaoShuFeng)

v11.3.0

fix: watch closes http session (#104, @tomplus)

v11.2.0

feat: regenerate client against openapi-generator v4.3.0 (context-manager and close function for http client instead of using __del__ method) (#99, @jnschaeffer)

v11.1.0

feat: add compatability for follow methods (#98, @playground-julia)

v11.0.0

feat: regenerate library using the latest version of openapi-generator (4.3.x) (gen/#146, @tomplus)
test: add tests for Python 3.8. (#86, @tomplus)
fix: prevent installing aiohttp 4.0 and up for now (#88, @sepulworld)
fix: watch.stream stores resource_version for the next call (#89, @tomplus)
chore: remove unused path import (#92, @aK0nshin)

API Change:

Introduce ExtensionsV1beta1RuntimeClassStrategyOptions and PolicyV1beta1RuntimeClassStrategyOptions. Add RuntimeClass restrictions & defaulting to PodSecurityPolicy kubernetes/kubernetes#73795
Introduce V1WindowsSecurityContextOptions. Add Windows specific options in Pod Security Context and Container Security Context kubernetes/kubernetes#77147
Split V1beta1Webhook into V1beta1MutatingWebhook and V1beta1ValidatingWebhook kubernetes/kubernetes#78491
Introduce parameter allow_watch_bookmarks in list options for requesting watch bookmarks from apiserver. The implementation in apiserver is hidden behind feature gate WatchBookmark (currently in Alpha stage) kubernetes/kubernetes#74074
Add V1DeleteOptions parameters (dry_run, grace_period_seconds, orphan_dependents, propagation_policy) to delete collection APIs kubernetes/kubernetes#77843
Add ListMeta.RemainingItemCount. When responding a LIST request, if the server has more data available, and if the request does not contain label selectors or field selectors, the server sets the ListOptions.RemainingItemCount to the number of remaining objects kubernetes/kubernetes#75993
Add controller_expand_secret_ref in V1SecretReference to store CSI volume expansion secrets kubernetes/kubernetes#77516
Introduce preemption_policy field to V1PriorityClass kubernetes/kubernetes#74614
Add port configuration to service reference in Admission webhook configuration, AuditSink webhook configuration, CRD Conversion webhook configuration and kube-aggregator kubernetes/kubernetes#74855
Introduce inline_volume_spec to V1PersistentVolumeSpec kubernetes/kubernetes#77703
Add fields x_kubernetes_embedded_resource, x_kubernetes_int_or_string, x_kubernetes_preserve_unknown_fields to V1beta1JSONSchemaProps kubernetes/kubernetes#77207

v10.0.1

fix: when _preload_content=False Websocket Stream returns 401 error (#84, @kexirong)
fix: e2e tests use apps/v1 api instead of removed extensions/v1beta1 (#85, @tomplus)

v10.0.0

feat: add create_form_yaml() functionality (#76, @PidgeyBE)
feat: custom objects can be merged by json-patch (gen/#119, @tomplus)
fix: parse microseconds in data-time fields (#80, @tomplus)
feat: upgrade to API spec from Kubernetes 1.14 (#83, @tomplus)

API Change:

Remove the AdmissionregistrationV1alpha1 API group, containing only the InitializationConfiguration type kubernetes/kubernetes#72972
Promote Lease API to v1 kubernetes/kubernetes#72239
The Ingress API is now available via NetworkingV1beta1Api. ExtensionsV1beta1Api Ingress objects are deprecated and will no longer be served in Kubernetes v1.18 kubernetes/kubernetes#74057
Introduce RuntimeClass to NodeV1alpha1Api and NodeV1beta1Api kubernetes/kubernetes#74433
Graduate PriorityClass API to GA SchedulingV1Api kubernetes/kubernetes#73555
Introduce CSINodeInfo and CSIDriver to StorageV1beta1Api kubernetes/kubernetes#74283
The alpha Initializers feature, admissionregistration.k8s.io/v1alpha1 API version, Initializers admission plugin, and use of the metadata.initializers API field have been removed. Discontinue use of the alpha feature and delete any existing InitializerConfiguration API objects before upgrading. The metadata.initializers field will be removed in a future release. The parameter include_uninitialized has been removed. kubernetes/kubernetes#72972

v9.1.0

feat: check whether an object key is present on watch (#71, @mickours)
feat: merging kubeconfig files (#69, @tomplus)

v9.0.0

feat: switch to openapi-generator (#58, @tomplus)
feat: add fieldSelector parameter to list/watch methods in custom objects spec (gen/#106)
feat: upgrade to API spec from Kubernetes 1.13 (#58, @tomplus)

Breaking Changes:

Model v1beta1WebhookClientConfig is renamed to AdmissionregistrationV1beta1WebhookClientConfig, to avoid naming conflict with ApiextensionsV1beta1WebhookClientConfig introduced in: kubernetes/kubernetes#67006
Delete request's body parameter is optional kubernetes/kubernetes#70032

v8.1.0

feat: watch improvement (context manager, close method) (#61, @hubo1016)

v8.0.3

fix: use yaml.safe_load, yaml.safe_dump for security reasons (#57, @tomplus)

v8.0.2

feat: remove dependency to urllib3 from kube_config, pin urlllib>=1.23 due to CVE-2018-20060 (#56, @tomplus)

v8.0.1

fix: kubeconfig loading failure when server uri contains trailing slash (#53, @tomplus)
feat: regenerate client with thread-pool optimization (#54, @tomplus)

v8.0.0

feat: upgrade to API spec from Kubernetes 1.12 (#50, @tomplus)

v7.0.0

first stable release 7.0.0

v1.0.0-beta9

fix: option verify-ssl impacts on server certs only (#46, @tomplus)

v1.0.0-beta8

feat: add debug logs to KubeConfigLoader (#45, @tomplus)
feat: exec-plugins support in kubeconfig (#44, @tomplus)
fix: read config data with bytes (python3) (#41, @tomplus)

v1.0.0-beta7

feat: add OIDC auth support (#36, @bpicolo)

v1.0.0-beta6

feat: regenerate with latest client gen to get custom object status scale api (#37, @juliantaylor)
fix: handling timeout by watch loop (#39, @tomplus)

v1.0.0-beta5

feat: upgrade to spec from Kuberentes 1.11 (#34, @tomplus)

v1.0.0-beta4

fix: aiohttp with verify_ssl=False (#33, @bpicolo)

v1.0.0-beta3

feat: watch work forever if timeout is not specified (#30, @tomplus)

v1.0.0-beta2

feat: support Python 3.7 (#28, @tomplus)

v1.0.0-beta1

feat: make function load_kube_config asynchronous
feat: function to auto-refresh gke token
feat: remove synchronous libraries from dependencies

v1.0.0-alpha4

feat: watch stops the iterator for empty responses and do not process ERROR responses (#22, @olitheolix)
feat: replace urllib3 by http.client in e2e initializer (#20, @tomplus)
feat: new example - tail.py (#19, @tomplus)
feat: new example - simultaneously watch multiple event streams without threads (#13, @olitheolix)
fix: fix few typos in setup.py (#18, @evemorgen)
fix: requirement for requests and urllib3 version (#16, @tomplus)

v1.0.0-alpha3

fix e2e and unit tests
fix Watch, Stream

v1.0.0-alpha2

fix requirements.txt

v1.0.0-alpha1

first release

Files

CHANGELOG.md

Latest commit

History

CHANGELOG.md

File metadata and controls

v24.2.2

v24.2.1

v24.2.0

API Change

v23.6.0

API Change

v22.6.5

v22.6.4

v22.6.3

v22.6.2

v22.6.1

v22.6.0

API Change

v21.7.1

v21.7.0

API Change

v20.13.0

API Change

v19.15.1

v19.15.0

API Change

v18.20.1

v18.20.0

v12.1.2

v12.1.1

v12.1.0

v12.0.1

v12.0.0

v11.3.0

v11.2.0

v11.1.0

v11.0.0

v10.0.1

v10.0.0

v9.1.0

v9.0.0

v8.1.0

v8.0.3

v8.0.2

v8.0.1

v8.0.0

v7.0.0

v1.0.0-beta9

v1.0.0-beta8

v1.0.0-beta7

v1.0.0-beta6

v1.0.0-beta5

v1.0.0-beta4

v1.0.0-beta3

v1.0.0-beta2

v1.0.0-beta1

v1.0.0-alpha4

v1.0.0-alpha3

v1.0.0-alpha2

v1.0.0-alpha1