Skip to content

Instantly share code, notes, and snippets.

@dims

dims/features.md

Last active April 11, 2025 03:31
Show Gist options
  • Save dims/a0ecf5de51235968e12b9d6b62ac3a83 to your computer and use it in GitHub Desktop.
Save dims/a0ecf5de51235968e12b9d6b62ac3a83 to your computer and use it in GitHub Desktop.
Download ZIP
`Feature Gates in Kubernetes master branch on Jan 11, 2025`
Raw
features.md
Feature Alpha Beta GA Deprecated Links
APIResponseCompression 1.8 1.16 ☑️ code KEPs
APIServerIdentity 1.20 1.26 ☑️ code KEPs
APIServerTracing 1.22 1.27 ☑️ code KEPs
APIServingWithRoutine 1.30 code KEPs
AggregatedDiscoveryRemoveBetaType 1.0 1.33 ☑️ code KEPs
AllowDNSOnlyNodeCSR 1.0 ☑️ 1.31 🔴 code KEPs
AllowInsecureKubeletCertificateSigningRequests 1.0 ☑️ 1.31 🔴 code KEPs
AllowOverwriteTerminationGracePeriodSeconds 1.0 ☑️ 1.32 🔴 code KEPs
AllowParsingUserUIDFromCertAuth 1.33 ☑️ code KEPs
AllowServiceLBStatusOnNonLB 1.0 ☑️ 1.29 🔴, 1.32 🔴 🔐 code KEPs
AllowUnsafeMalformedObjectDeletion 1.32 code KEPs
AnonymousAuthConfigurableEndpoints 1.31 1.32 ☑️ code KEPs
AnyVolumeDataSource 1.18 1.24 ☑️ 1.33 ☑️ 🔐 code KEPs
AuthorizeNodeWithSelectors 1.31 1.32 ☑️ code KEPs
AuthorizeWithSelectors 1.31 1.32 ☑️ code KEPs
BtreeWatchCache 1.32 ☑️ 1.33 ☑️ 🔐 code KEPs
CBORServingAndStorage 1.32 code KEPs
CPUManagerPolicyAlphaOptions 1.23 code KEPs
CPUManagerPolicyBetaOptions 1.23 ☑️ code KEPs
CPUManagerPolicyOptions 1.22 1.23 ☑️ 1.33 ☑️ 🔐 code KEPs
CRDValidationRatcheting 1.28 1.30 ☑️ 1.33 ☑️ 🔐 code KEPs
CSIMigrationPortworx 1.23 1.31 ☑️ 1.33 ☑️ 🔐 code KEPs
CSIVolumeHealth 1.21 code KEPs
CloudControllerManagerWebhook 1.27 code KEPs
ClusterTrustBundle 1.27 1.33 code KEPs
ClusterTrustBundleProjection 1.29 1.33 code KEPs
ComponentFlagz 1.32 code KEPs
ComponentStatusz 1.32 code KEPs
ConcurrentWatchObjectDecode 1.31 code KEPs
ConsistentListFromCache 1.28 1.31 ☑️ code KEPs
ContainerCheckpoint 1.25 1.30 ☑️ code KEPs
ContainerStopSignals 1.33 code KEPs
CoordinatedLeaderElection 1.31 1.33 code KEPs
CronJobsScheduledAnnotation 1.28 ☑️ 1.32 ☑️ 🔐 code KEPs
CrossNamespaceVolumeDataSource 1.26 code KEPs
CustomCPUCFSQuotaPeriod 1.12 code KEPs
CustomResourceFieldSelectors 1.30 1.31 ☑️ 1.32 ☑️ 🔐 code KEPs
DRAAdminAccess 1.32 code KEPs
DRADeviceTaints 1.33 code KEPs
DRAPartitionableDevices 1.33 code KEPs
DRAPrioritizedList 1.33 code KEPs
DRAResourceClaimDeviceStatus 1.32 1.33 ☑️ code KEPs
DeclarativeValidation 1.33 ☑️ code KEPs
DeclarativeValidationTakeover 1.33 code KEPs
DeploymentReplicaSetTerminatingReplicas 1.33 code KEPs
DevicePluginCDIDevices 1.28 1.29 ☑️ 1.31 ☑️ 🔐 code KEPs
DisableAllocatorDualWrite 1.31 1.33 code KEPs
DisableCPUQuotaWithExclusiveCPUs 1.33 ☑️ code KEPs
DisableNodeKubeProxyVersion 1.29 1.31 🔴, 1.33 ☑️ code KEPs
DynamicResourceAllocation 1.26 1.32 code KEPs
ElasticIndexedJob 1.27 ☑️ 1.31 ☑️ 🔐 code KEPs
EventedPLEG 1.26 code KEPs
ExecProbeTimeout 1.20 ☑️ code KEPs
ExternalServiceAccountTokenSigner 1.32 code KEPs
GitRepoVolumeDriver 1.0 ☑️ 1.33 🔴 code KEPs
GracefulNodeShutdown 1.20 1.21 ☑️ code KEPs
GracefulNodeShutdownBasedOnPodPriority 1.23 1.24 ☑️ code KEPs
HPAConfigurableTolerance 1.33 code KEPs
HPAScaleToZero 1.16 code KEPs
HonorPVReclaimPolicy 1.23 1.31 ☑️ 1.33 ☑️ 🔐 code KEPs
ImageMaximumGCAge 1.29 1.30 ☑️ code KEPs
ImageVolume 1.31 1.33 code KEPs
InPlacePodVerticalScaling 1.27 1.33 ☑️ code KEPs
InPlacePodVerticalScalingAllocatedStatus 1.32 1.33 🔴 code KEPs
InPlacePodVerticalScalingExclusiveCPUs 1.32 code KEPs
InTreePluginPortworxUnregister 1.23 code KEPs
JobBackoffLimitPerIndex 1.28 1.29 ☑️ 1.33 ☑️ 🔐 code KEPs
JobManagedBy 1.30 1.32 ☑️ code KEPs
JobPodReplacementPolicy 1.28 1.29 ☑️ code KEPs
JobSuccessPolicy 1.30 1.31 ☑️ 1.33 ☑️ 🔐 code KEPs
KMSv1 1.0 ☑️ 1.28 ☑️, 1.29 🔴 code KEPs
KubeletCgroupDriverFromCRI 1.28 1.31 ☑️ code KEPs
KubeletCrashLoopBackOffMax 1.32 code KEPs
KubeletEnsureSecretPulledImages 1.33 code KEPs
KubeletFineGrainedAuthz 1.32 1.33 ☑️ code KEPs
KubeletInUserNamespace 1.22 code KEPs
KubeletPSI 1.33 code KEPs
KubeletPodResourcesDynamicResources 1.27 code KEPs
KubeletPodResourcesGet 1.27 code KEPs
KubeletRegistrationGetOnExistsOnly 1.0 ☑️ 1.32 🔴 code KEPs
KubeletSeparateDiskGC 1.29 1.31 ☑️ code KEPs
KubeletServiceAccountTokenForCredentialProviders 1.33 code KEPs
KubeletTracing 1.25 1.27 ☑️ code KEPs
LegacySidecarContainers 1.0 ☑️ 1.33 🔴 code KEPs
ListFromCacheSnapshot 1.33 code KEPs
LoadBalancerIPMode 1.29 1.30 ☑️ 1.32 ☑️ 🔐 code KEPs
LocalStorageCapacityIsolationFSQuotaMonitoring 1.15 1.31 code KEPs
LogarithmicScaleDown 1.21 1.22 ☑️ 1.31 ☑️ 🔐 code KEPs
MatchLabelKeysInPodAffinity 1.29 1.31 ☑️ 1.33 ☑️ 🔐 code KEPs
MatchLabelKeysInPodTopologySpread 1.25 1.27 ☑️ code KEPs
MaxUnavailableStatefulSet 1.24 code KEPs
MemoryManager 1.21 1.22 ☑️ 1.32 ☑️ 🔐 code KEPs
MemoryQoS 1.22 code KEPs
MultiCIDRServiceAllocator 1.27 1.31 1.33 ☑️ code KEPs
MutableCSINodeAllocatableCount 1.33 code KEPs
MutatingAdmissionPolicy 1.32 code KEPs
NFTablesProxyMode 1.29 1.31 ☑️ 1.33 ☑️ 🔐 code KEPs
NodeInclusionPolicyInPodTopologySpread 1.25 1.26 ☑️ 1.33 ☑️ 🔐 code KEPs
NodeLogQuery 1.27 1.30 code KEPs
NodeSwap 1.22 1.30 ☑️ code KEPs
OpenAPIEnums 1.23 1.24 ☑️ code KEPs
OrderedNamespaceDeletion 1.33 ☑️ code KEPs
PodAndContainerStatsFromCRI 1.23 code KEPs
PodDeletionCost 1.21 1.22 ☑️ code KEPs
PodDisruptionConditions 1.25 1.26 ☑️ 1.31 ☑️ 🔐 code KEPs
PodIndexLabel 1.28 ☑️ 1.32 ☑️ 🔐 code KEPs
PodLevelResources 1.32 code KEPs
PodLifecycleSleepAction 1.29 1.30 ☑️ code KEPs
PodLifecycleSleepActionAllowZero 1.32 1.33 ☑️ code KEPs
PodLogsQuerySplitStreams 1.32 code KEPs
PodObservedGenerationTracking 1.33 code KEPs
PodReadyToStartContainersCondition 1.28 1.29 ☑️ code KEPs
PodSchedulingReadiness 1.26 1.27 ☑️ 1.30 ☑️ 🔐 code KEPs
PodTopologyLabelsAdmission 1.33 code KEPs
PortForwardWebsockets 1.30 1.31 ☑️ code KEPs
PreferSameTrafficDistribution 1.33 code KEPs
ProcMountType 1.12 1.33 ☑️ code KEPs
QOSReserved 1.11 code KEPs
RecoverVolumeExpansionFailure 1.23 1.32 ☑️ code KEPs
RecursiveReadOnlyMounts 1.30 1.31 ☑️ 1.33 ☑️ 🔐 code KEPs
ReduceDefaultCrashLoopBackOffDecay 1.33 code KEPs
RelaxedDNSSearchValidation 1.32 1.33 ☑️ code KEPs
RelaxedEnvironmentVariableValidation 1.30 1.32 ☑️ code KEPs
ReloadKubeletServerCertificateFile 1.31 ☑️ code KEPs
RemoteRequestHeaderUID 1.32 1.33 ☑️ code KEPs
ResilientWatchCacheInitialization 1.31 ☑️ code KEPs
ResourceHealthStatus 1.31 code KEPs
RetryGenerateName 1.30 1.31 ☑️ 1.32 ☑️ 🔐 code KEPs
RotateKubeletServerCertificate 1.7 1.12 ☑️ code KEPs
RuntimeClassInImageCriApi 1.29 code KEPs
SELinuxChangePolicy 1.32 1.33 ☑️ code KEPs
SELinuxMount 1.30 1.33 code KEPs
SELinuxMountReadWriteOncePod 1.25 1.28 ☑️ code KEPs
SchedulerAsyncPreemption 1.32 1.33 ☑️ code KEPs
SchedulerPopFromBackoffQ 1.33 ☑️ code KEPs
SchedulerQueueingHints 1.32 ☑️ code KEPs
SeparateCacheWatchRPC 1.28 ☑️ 1.33 🔴 code KEPs
SeparateTaintEvictionController 1.29 ☑️ code KEPs
ServiceAccountNodeAudienceRestriction 1.33 ☑️ code KEPs
ServiceAccountTokenJTI 1.29 1.30 ☑️ 1.32 ☑️ 🔐 code KEPs
ServiceAccountTokenNodeBinding 1.29 1.31 ☑️ 1.33 ☑️ 🔐 code KEPs
ServiceAccountTokenNodeBindingValidation 1.29 1.30 ☑️ 1.32 ☑️ 🔐 code KEPs
ServiceAccountTokenPodNodeInfo 1.29 1.30 ☑️ 1.32 ☑️ 🔐 code KEPs
ServiceTrafficDistribution 1.30 1.31 ☑️ 1.33 ☑️ 🔐 code KEPs
SidecarContainers 1.28 1.29 ☑️ 1.33 ☑️ 🔐 code KEPs
SizeMemoryBackedVolumes 1.20 1.22 ☑️ 1.32 ☑️ 🔐 code KEPs
StatefulSetAutoDeletePVC 1.23 1.27 ☑️ 1.32 ☑️ 🔐 code KEPs
StatefulSetStartOrdinal 1.26 1.27 ☑️ 1.31 ☑️ 🔐 code KEPs
StorageCapacityScoring 1.33 code KEPs
StorageNamespaceIndex 1.30 ☑️ 1.33 ☑️ code KEPs
StorageVersionAPI 1.20 code KEPs
StorageVersionHash 1.14 1.15 ☑️ code KEPs
StorageVersionMigrator 1.30 code KEPs
StreamingCollectionEncodingToJSON 1.33 ☑️ code KEPs
StreamingCollectionEncodingToProtobuf 1.33 ☑️ code KEPs
StrictCostEnforcementForVAP 1.30 1.32 ☑️ 🔐 code KEPs
StrictCostEnforcementForWebhooks 1.30 1.32 ☑️ 🔐 code KEPs
StrictIPCIDRValidation 1.33 code KEPs
StructuredAuthenticationConfiguration 1.29 1.30 ☑️ code KEPs
StructuredAuthorizationConfiguration 1.29 1.30 ☑️ 1.32 ☑️ 🔐 code KEPs
SupplementalGroupsPolicy 1.31 1.33 ☑️ code KEPs
SystemdWatchdog 1.32 ☑️ code KEPs
TopologyAwareHints 1.21 1.24 ☑️ 1.33 ☑️ 🔐 code KEPs
TopologyManagerPolicyAlphaOptions 1.26 code KEPs
TopologyManagerPolicyBetaOptions 1.28 ☑️ code KEPs
TopologyManagerPolicyOptions 1.26 1.28 ☑️ 1.32 ☑️ code KEPs
TranslateStreamCloseWebsocketRequests 1.29 1.30 ☑️ code KEPs
UnauthenticatedHTTP2DOSMitigation 1.29 ☑️ code KEPs
UnknownVersionInteroperabilityProxy 1.28 code KEPs
UserNamespacesPodSecurityStandards 1.29 code KEPs
UserNamespacesSupport 1.25 1.33 ☑️ code KEPs
VolumeAttributesClass 1.29 1.31 code KEPs
WatchCacheInitializationPostStartHook 1.31 code KEPs
WatchFromStorageWithoutResourceVersion 1.27 1.33 🔴 🔐 code KEPs
WatchList 1.27 1.32 ☑️ code KEPs
WinDSR 1.14 1.33 ☑️ code KEPs
WinOverlay 1.14 1.20 ☑️ code KEPs
WindowsCPUAndMemoryAffinity 1.32 code KEPs
WindowsGracefulNodeShutdown 1.32 code KEPs
WindowsHostNetwork 1.26 1.33 🔴 code KEPs

Legend: ☑️ - enabled, 🔴 - disabled 🔐 - locked to default

Raw
kube_features_test.go
func TestGenerateMarkDownOfAllKubeFeaturesRegistered(t *testing.T) {
var sb strings.Builder
sb.WriteString("| Feature | Alpha | Beta | GA | Deprecated | Links |\n")
sb.WriteString("|---------|-------|------|----|----|----|\n")
// Extract keys from the map
keys := make([]string, 0, len(defaultVersionedKubernetesFeatureGates))
for key := range defaultVersionedKubernetesFeatureGates {
keys = append(keys, string(key))
}
sort.Strings(keys)
for _, feature := range keys {
specs := defaultVersionedKubernetesFeatureGates[featuregate.Feature(feature)]
alpha, beta, ga, deprecated := "", "", "", ""
for _, spec := range specs {
switch spec.PreRelease {
case featuregate.Alpha:
alpha = spec.Version.String()
if spec.Default {
alpha += " :ballot_box_with_check:"
}
if spec.LockToDefault {
alpha += " :closed_lock_with_key:"
}
case featuregate.Beta:
beta = spec.Version.String()
if spec.Default {
beta += " :ballot_box_with_check:"
}
if spec.LockToDefault {
beta += " :closed_lock_with_key:"
}
case featuregate.GA:
ga = spec.Version.String()
if spec.Default {
ga += " :ballot_box_with_check:"
}
if spec.LockToDefault {
ga += " :closed_lock_with_key:"
}
case featuregate.Deprecated:
depVer := spec.Version.String()
if spec.Default {
depVer += " :ballot_box_with_check:"
} else {
depVer += " :red_circle:"
}
if spec.LockToDefault {
depVer += " :closed_lock_with_key:"
}
if len(deprecated) > 0 {
deprecated += ", "
deprecated += depVer
} else {
deprecated = depVer
}
}
}
link_to_code := fmt.Sprintf("[code](https://cs.k8s.io/?q=%%5Cb%s%%5Cb&i=nope&files=&excludeFiles=CHANGELOG&repos=kubernetes/kubernetes)", feature)
link_to_enhancements := fmt.Sprintf("[KEPs](https://cs.k8s.io/?q=%%5Cb%s%%5Cb&i=nope&files=&excludeFiles=CHANGELOG&repos=kubernetes/enhancements)", feature)
sb.WriteString(fmt.Sprintf("| %s | %s | %s | %s | %s |%s %s\n", feature, alpha, beta, ga, deprecated, link_to_code, link_to_enhancements))
}
sb.WriteString("\n\n Legend: :ballot_box_with_check: - enabled, :red_circle: - disabled\n")
sb.WriteString("\t\tclosed_lock_with_key: - locked to default\n")
fmt.Print(sb.String())
}
@aojea
Copy link

aojea commented Jan 13, 2025

Love it 🥇

@pacoxu
Copy link

pacoxu commented Jan 17, 2025

Can we sort it by last updated version?

@dims
Copy link
Author

dims commented Jan 17, 2025

@pacoxu we can generate a CSV that can then be used in excel or google sheets. Also feel free to fork this gist to experiment!

@pacoxu
Copy link

pacoxu commented Jan 21, 2025 •
edited
Loading

https://docs.google.com/spreadsheets/d/1IxBtCeWOI8qWjF5jauT7pzfKwJssEhHErS4L6VcHgnU/edit?usp=sharing

I created one like this and updated status of some KEPs with @carlory.

  • We mainly update SIG-Node and SIG-Storage KEPs.

@pacoxu
Copy link

pacoxu commented Jan 22, 2025 •
edited
Loading

We have updated all keps that has no update after v1.26 now.

https://docs.google.com/spreadsheets/d/1IxBtCeWOI8qWjF5jauT7pzfKwJssEhHErS4L6VcHgnU/edit?gid=1555406139#gid=1555406139

I divide it into 3 pages

  • not update after v1.22
  • updated between 1.23 and v1.29:
    • v1.23-v1.28
    • v1.29
  • updated after v1.30

@BenTheElder
Copy link

After 1.33 there will be no alpha + enabled (WindowsHostNetwork was removed and we have a presubmit check now)

@dims
Copy link
Author

dims commented Apr 10, 2025

@BenTheElder let me regenerate the data now that we cut rc.0

@dims
Copy link
Author

dims commented Apr 11, 2025

@BenTheElder we still have WindowsHostNetwork, defaulted to false and deprecated.

	WindowsHostNetwork: {
		{Version: version.MustParse("1.26"), Default: false, PreRelease: featuregate.Alpha},
		{Version: version.MustParse("1.33"), Default: false, PreRelease: featuregate.Deprecated},
	},

https://github.com/kubernetes/kubernetes/blob/88dfcb225d41326113990e87b11137641c121a32/pkg/features/kube_features.go#L1863-L1866

@BenTheElder
Copy link

@BenTheElder we still have WindowsHostNetwork, defaulted to false and deprecated.

Er right, but not alpha+default enabled. Should've noticed that earlier scrolling this table :-)

@BenTheElder
Copy link

Very neat use of the versioned FeatureGates info , something like this would be pretty cool on kubernetes.io

@pacoxu
Copy link

pacoxu commented Apr 11, 2025

@BenTheElder we still have WindowsHostNetwork, defaulted to false and deprecated.

	WindowsHostNetwork: {
		{Version: version.MustParse("1.26"), Default: false, PreRelease: featuregate.Alpha},
		{Version: version.MustParse("1.33"), Default: false, PreRelease: featuregate.Deprecated},
	},

https://github.com/kubernetes/kubernetes/blob/88dfcb225d41326113990e87b11137641c121a32/pkg/features/kube_features.go#L1863-L1866

In the PR, the logic of this FG are totally removed: this is not like a traditional deprecation.

@BenTheElder
Copy link

@pacoxu oof, lets take this to slack or the PR and follow up there

I'm also reaching out to the compat version folks about this table, @siyuanfoundation mentioned there's a tracking issue to implement something similar in the kubernetes docs using the feature gate yaml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment