v2.12.4

Image

registry-cn-hangzhou.ack.aliyuncs.com/acs/cloud-controller-manager:v2.12.4

New features:

CLB

• The default instance charge type for newly created CLB instances is changed from PayBySpec to PayByCLCU.

Node

• Lingjun nodes: support automatically cleaning up Lingjun node resources in the cluster after the Lingjun instance is released.
• Skip reconcilation of hybrid-cloud nodes.

Improvements:

• When processing node change events, ignore Services for serviceBackendType=eni.
• Improve CLB/NLB processing performance.
• Perform a limited number of attempts when NLB OpenAPI calls are throttled.
• Optimize metrics related to the time spent synchronizing Services, routes, and nodes.
• Change the readinessGate retry wait time from exponential backoff to a fixed interval.
• When it cannot find the ENI corresponding to a backend Pod IP, the error logs include the Pod name (targetRef) and the node information.

Fixed bugs

• Fixed an issue that it failed to automatically configure the backend targetPort as the health check port when NLB is manually configured with a listener port range and health checks.
• Fixed an issue in ECS + ECI/ACS mixed deployment scenarios where ECI/ACS instances could not be attached or backend weights were set incorrectly.
• Fixed a panic during Service synchronization when querying NLB information or when waiting for asynchronous task calls failed.

v2.11.2

Image

registry-cn-hangzhou.ack.aliyuncs.com/acs/cloud-controller-manager:v2.11.2

New features

CLB & NLB

• The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ignore-weight-update can be used to ignore the update for backend weight.

NLB

• The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-listener-port-range can be used to configure the port range for listeners.
• Support specify custom NLB OpenAPI Endpoint via the environment variable NLB_ENDPOINT.

CLB

• Support multiple ACL IDs separated by commas for the annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-id.

Improvements

• Reimplement the node and route controller for using batch OpenAPI to reduce the total API calls and increase the reconciliation speed.
• Parallelize the listener and servergroup actions for NLB & CLB to reduce single service reconciliation time.
• Reduce the API calls for service reconciliation.
• Use nil pointer instead of empty string for AllocationId and IPv4Addr when calling OpenAPI for creating NLB instance.
• Use NextToken instead of PageSize for DescribeNetworkInterfaces OpenAPI.

Fixed bugs

• Fix the bug that the NLB controller won't retry if there is one or more pods are not ready when using ReadinessGate.

v2.10.0

Image

registry-cn-hangzhou.ack.aliyuncs.com/acs/cloud-controller-manager:v2.10.0

Features:

CLB & NLB

• The ReadinessGate feature is supported.
• The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags now can be used to modify tags for existing instances.
• The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-preserve-lb-on-delete can be used to preserve the loadbalancer instance after the Service is deleted.

NLB

• The annotations service.beta.kubernetes.io/alibaba-cloud-loadbalancer-alpn and service.beta.kubernetes.io/alibaba-cloud-loadbalancer-alpn-policy can be used to specify a TCP/SSL listener for the Application-Layer Protocol Negotiation (ALPN) policy.

Node

• Adding the node.alibabacloud.com/nodepool-id and node.alibabacloud.com/instance-charge-type annotations to nodes.

Improvements:

• The reconcileID parameter is added to the log output.

Fixed bugs:

• Check the LoadBalancer service type again in reconciliation.

What's Changed

• metrics: clb latency&operation related metrics by @xuancheng131 in #408
• update go version to 1.22.0 by @gujingit in #409
• support modify loadbalancer tag after loadbalancer created & reused by @Lyt99 in #421
• add nodepool id & instance charge type labels for node by @Lyt99 in #417
• nlb support alpn by @Lyt99 in #418
• support nlb for service.beta.kubernetes.io/class annotation by @Lyt99 in #422
• preserve lb on delete for clb&nlb by @Lyt99 in #415
• check loadbalancer class again to cleanup by @Lyt99 in #416
• add extra info to reconcile logs by @Lyt99 in #420
• support readiness gates for CLB & NLB by @Lyt99 in #419
• fix e2e test compile error and failure by @Lyt99 in #423

New Contributors

• @xuancheng131 made their first contribution in #408

Full Changelog: v2.9.1...v2.10.0

v2.9.1

Image

registry-cn-hangzhou.ack.aliyuncs.com/acs/cloud-controller-manager-amd64:v2.9.1

Features

• The default resource group ID when creating CLB & NLB instances can be configured via ResourceGroupID in cloud config.

CLB

• The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-xforwardedfor-slbport can be used to enable the X-Forwarded-SLBPort request header.
• The service.beta.kubernetes.io/alibaba-cloud-loadbalancer-xforwardedfor-clientsrcport annotation can be used to enable the X-Forwarded-Client-srcport request header.

NLB

• NLB instances can be reused across VPCs.
• The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-bandwidth-package-id can be used to specify the ID of an elastic IP address (EIP) bandwidth plan.
• The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-delete-protection can be used to enable deletion protection. By default, this feature is enabled for newly created NLB instances.
• The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-modification-protection can be used to enable the configuration modification protection. By default, this mode is enabled for newly created NLB instances.
• The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vgroup-port annotation can be used to reuse a server group. This annotation takes effect only when an existing NLB instance is reused.
• When an NLB instance is reused by multiple Services, the annotationservice.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight  can be used to specify the weight of the traffic received by the current Service. This annotation takes effect only when an existing vserver group is reused.
• The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-backend-ip-version: ipv6 can be used to add IPv6 backend servers for dual-stack NLB instances .
• The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ipv6-address-type can be used to specify the IPv6 network address type for dual-stack NLB instances .
• The annotations service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-ep-id-enabled, service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-eps-id-enabled, and service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-vpc-id-enabled can be used to pass the VpcId, PrivateLinkEpId, and PrivateLinkEpsId information to backend servers over Proxy Protocol.

Node

• Ithe IPv6 addresses of ECS instances can be automatically added to nodes.

Improvements:

• EndpointSlice feature gate is enabled by default.
• A verification is added to check whether the route table ID is an empty string.
• A verification is added to check the return values of API operations in reuse scenarios.
• The resourceVersion=0 parameter setting is used when you initiate a LIST request.

Fixed bugs:

• Fixed the NetworkUnavailable state is not set during node initialization in Flannel mod.
• Fixed the NLB server group's is incorrect when the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id annotation is used to specify a resource group.

What's Changed

• support vpc with secondary cidrs by @gujingit in #387
• add resourceVersion=0 for list requests by @Lyt99 in #405
• set HealthCheckInterval to 5 to fix e2e error by @Lyt99 in #398
• bugfix: fix route table id is empty string by @gujingit in #399
• bugfix: fix node controller do not init routeconfig by @gujingit in #406
• enable EndpointSlice feature by default by @Lyt99 in #396
• filter services when node reconcile by @Lyt99 in #397
• node add ipv6 address by @Lyt99 in #395
• check loadbalancer id from API when reuse by @Lyt99 in #393
• create with specified resource group id for clb&nlb by @Lyt99 in #394
• add node max syncs in cloud config by @gujingit in #401
• feat: support reusing nlb of another vpc by @gujingit in #403
• Support nlb modification & delete protection by @gujingit in #402
• support X-Forwarded-SLBPort & X-Forwarded-Client-srcport for CLB by @gujingit in #400
• upgrade nlb sdk to v3.0.0 & support nlb ppv2 privatelink by @Lyt99 in #390
• nlb ipv6 backend & address type support by @Lyt99 in #391
• support nlb vgroup-port and weight & fix clb listener vgroup reuse by @Lyt99 in #392
• [Bug] Optimize ZoneMappings comparison by @yang-wang11 in #404
• fix backend ip version check for clb by @Lyt99 in #407

New Contributors

• @yang-wang11 made their first contribution in #404

Full Changelog: v2.8.0...v2.9.1

v2.8.0

Image

registry-cn-hangzhou.ack.aliyuncs.com/acs/cloud-controller-manager-amd64:v2.8.0

Features

• Support addon token authorization.
• NLB supports creating IP type server groups through the annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-server-group-type. Please refer to the doc for NLB server group description.
• Support pprof (default port is 6060).

Improvements

• Disable cache for node, service, endpoint and endpointSlice.

NLB

• Optimize the server group creation to avoid repeatedly creating server groups.

CLB

• Add IP address verification when using the ENI mode. The IP address must be in the cluster VPC.
• If service.beta.kubernetes.io/alibaba-cloud-loadbalancer-instance-charge-type is set to PayByCLCU, the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec annotation will be ignored.

What's Changed

• chore: chore: pkg imported more than once by @testwill in #367
• fix: typo in loadbalancer by @mitingjin in #370
• chore: replace github.com/ghodss/yaml with sigs.k8s.io/yaml by @Juneezee in #368
• Bugfix/servergroup by @gujingit in #374
• Bugfix/no cache for node by @gujingit in #375
• feat: filter pods whose ip not in vpc cidr in eni mod by @gujingit in #377
• Feature/dependency update by @gujingit in #376
• feat: update instanceChargeType && instanceSpec by @gujingit in #378
• Feature/addon token by @gujingit in #380
• support configuring server group batch size by @gujingit in #381
• test: update route controller tests by @gujingit in #382
• Feature/ip mode by @gujingit in #379
• add nlb ip tests; update expect to support paybyclcu & serverGroupTDype by @gujingit in #384

New Contributors

• @testwill made their first contribution in #367
• @mitingjin made their first contribution in #370
• @Juneezee made their first contribution in #368

Full Changelog: v2.7.0...v2.8.0

v2.7.0

Image

registry-cn-hangzhou.ack.aliyuncs.com/acs/cloud-controller-manager-amd64:v2.7.0

Features

• The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip can be used to create an intranet loadbalancer with the specifying IP address.

Improvements

• Optimize the reconciling of CLB and NLB server groups to reduce errors caused by insufficient Quota.
• Update Service Hash Function to reduce the change of Hash value caused by operations such as cluster upgrade.

Fixed bugs

• Fix the bug that the Service could not be reconciled after setting the EIP Annotation.
• Fix the bug that HTTP protocol cannot be set for other ports after setting ForwardPort Annotation.

v2.6.0

Image

registry-cn-hangzhou.ack.aliyuncs.com/acs/cloud-controller-manager-amd64:v2.6.0

Features

• The node label alpha.service-controller.kubernetes.io/exclude-balancer which excludes a node from cloud load balancers (using Service Type=LoadBalancer) is deprecated in favor of node.kubernetes.io/exclude-balancer.
• Support load balancers with mixed protocol types, enables the creation of a LoadBalancer Service that has different port definitions with different protocols.

Only for CLB

• The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch can be used to disable health check for TCP and UDP listeners.
• The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol can be used to configure the ProxyProtocol protocol for TCP and UDP listeners.

Please note that this function does not support online smooth migration. Switching to proxy protocol requires service shutdown and upgrade.

• The validity period of the certificate will be verified when synchronizing the HTTPS listener. When the certificate expires, the CLB synchronization will fail.

Only for NLB

• The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-security-group-ids can be used to configure security groups for NLB.

Improvements

• The resource lock is changed to leases.
• Continue to update the virtual server groups when the update of the load balancer attributes (such as name, resource group, etc.) fails.
• Synchronize services only when the ready condtion of the node changes, ignoring other condtions change.

Fixed bugs

• Fix the bug of occasional misjudgment of node NotReady when Kubernetes Version=1.24 & CCM Version=v2.5.1

v2.5.1

Image

registry.${region}.aliyuncs.com/acs/cloud-controller-manager-amd64:v2.5.1

Features

• Support to manage NLB (Network Load Balancer) instances. If you set Type=LoadBalancer & loadbalancerClass=alibabacloud.com/nlb for a service, the CCM automatically creates a NLB instance for the serivce, and configures listeners and backend server groups. See help doc for more usage.
  Only supported for Kubernetes 1.24 and above.
• Support to create different types of load balancers accroding to the service loadBalancerClass.
  If you not set loadBalancerClass for a service, the CCM creates a CLB instance by default; if you set loadbalancerClass=alibabacloud.com/nlb, the CCM creates a NLB instance.
  Only supported for Kubernetes 1.24 and above.

Improvements

• Fix the bug that the service which reuses a IPv6 CLB cannot be deleted.
• Fix the bug that can not delete nodes occasionally.
• Call openAPIs with HTTPS protocol as default.

v2.4.0

Image

registry.${region}.aliyuncs.com/acs/cloud-controller-manager-amd64:v2.4.0

Features

• The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-instance-charge-type can be used to set loadbalancer instance charge type .
• The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-tls-cipher-policy can be used to configure security policy. Each security policy contains TLS protocol versions and cipher suites available for HTTPS. Only HTTPS are supported.
• CCM will automatically patch the node.spec.providerID field of the node if this field is empty when adding a node.
• Adding the service.k8s.alibaba/loadbalancer-id label to the service in order to record the loadbalancer instance id associated with the service.

Improvements

• When a node has the ToBeDeletedByClusterAutoscaler taint, the node will not be added to the loadbalancer instance backends.
• Fixed an issue where conflicting routes could not be deleted when the route CIDR was the same.
• Optimize the processing of concurrent route synchronization to reduce false warning events.

v2.3.0

Image

registry.${region}.aliyuncs.com/acs/cloud-controller-manager-amd64:v2.3.0

Features

• The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-hostname can be used to set hostname for service.
• The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-established-timeout can be used to configure the established timeout for an SLB instance. Only TCP is supported.
• The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-request-timeout can be used to configure the request timeout for an SLB instance. Only HTTP & HTTPS are supported.
• The annoation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-method can be used to configure the health check method for HTTP health check.

Improvements

• Verify the format of virtual server group id when reusing an existing virtual server group.
• Optimize the switch selection to avoid the default switch being empty.
• Optimize virtual server group synchronization in order to reduce OpenAPI calls.