一文搞定 K8s 版本升级:1.30.2 升级 1.30.4,含节点驱逐、故障恢复方案
一文搞定 K8s 版本升级1.30.2 升级 1.30.4含节点驱逐、故障恢复方案Kubernetes 集群升级升级必要性升级注意事项升级流程升级步骤升级前准备升级 master升级 kubeadm升级 master 节点组件升级其他 master 节点驱逐 master 节点升级 kubelet 和 kubectl取消节点不可调度升级 worker升级 worker31升级 worker32从故障状态恢复Kubernetes 集群升级摘要本文详细介绍了 Kubernetes 集群从 1.30.2 升级到 1.30.4 的完整操作流程。首先阐述了升级的必要性和注意事项然后按照先 master 节点后 worker 节点的顺序逐步演示了 kubeadm、控制平面组件、kubelet 和 kubectl 的升级步骤。文章包含大量实际操作命令和输出示例涵盖了升级前环境检查、升级计划、组件升级、节点驱逐与恢复等关键环节最后还提供了从故障状态恢复的方法。整个升级过程遵循官方推荐的最佳实践确保集群在升级期间的稳定性和数据安全。学习参考kubernetes 升级升级必要性功能性增强。漏洞修复。等等升级注意事项如果 Kubernetes 集群只有一个master节点则升级 master 节点过程中将导致集群不可用。如果 Kubernetes 集群只有两个worker节点则升级第二个 worker 节点时负载会在第一个worker节点重建进而导致应用短暂不可用不可用时间取决于pod重建时间。升级流程从节点角度看先逐个升级 master 节点再逐个升级 worker 节点。从软件层面看任何节点都需要先升级kubeadm再利用 kubeadm 升级各个组件最后升级kubelet和kubectl等。升级步骤本次实验环境从kubernetes 1.30.2 升级到 1.30.4。升级前准备# 确认当前安装的版本rootmaster30:~# kubectl versionClient Version: v1.30.2 Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3 Server Version: v1.30.2 rootmaster30:~# kubectl get nodesNAME STATUS ROLES AGE VERSION master30.lz.cloud Ready control-plane 53d v1.30.2 worker31.lz.cloud Readynone53d v1.30.2 worker32.lz.cloud Readynone53d v1.30.2# 确认仓库中提供的版本rootmaster30:~# apt list kubeadm -a|grep amd64WARNING:aptdoes not have a stable CLI interface. Use with cautioninscripts. kubeadm/unknown1.30.4-1.1 amd64[upgradable from:1.30.2-1.1]kubeadm/unknown1.30.3-1.1 amd64 kubeadm/unknown,now1.30.2-1.1 amd64[installed,upgradable to:1.30.4-1.1]kubeadm/unknown1.30.1-1.1 amd64 kubeadm/unknown1.30.0-1.1 amd64升级 master升级 kubeadm# 升级 kubeadmrootmaster30:~# apt install -y kubeadm1.30.4-1.1如果 kubeadm 被hold住了使用以下命令解除。# 升级 kubeadmrootmaster30:~# apt-mark unhold kubeadm升级 master 节点组件# 升级帮助rootmaster30:~# kubeadm upgrade apply --help# --dry-run 模拟升级# --etcd-upgrade 升级 etcd默认值true。设置为 false 则不升级etcd。# -f, --force 强制升级即使部分条件不满足。使用该选项还可以避免二次确认升级。# 升级计划rootmaster30:~# kubeadm upgrade plan[preflight]Running pre-flight checks.[upgrade/config]Reading configuration from the cluster...[upgrade/config]FYI: You canlookat this configfilewithkubectl -n kube-system get cm kubeadm-config -o yaml[upgrade]Running cluster health checks[upgrade/health]FATAL:[preflight]Some fatal errors occurred:[ERROR CreateJob]: Jobupgrade-health-check-tk52jinthe namespacekube-systemdid not completein15s: no condition oftypeComplete[preflight]If you know what you are doing, you canmakea check non-fatal with--ignore-preflight-errors...To see the stack trace of this error execute with--v5or higherrootmaster30:~# kubeadm upgrade plan[preflight]Running pre-flight checks.[upgrade/config]Reading configuration from the cluster...[upgrade/config]FYI: You canlookat this configfilewithkubectl -n kube-system get cm kubeadm-config -o yaml[upgrade]Running cluster health checks[WARNING CreateJob]: Jobupgrade-health-check-c45ncinthe namespacekube-systemdid not completein15s: no condition oftypeComplete[upgrade]Fetching available versions to upgrade to[upgrade/versions]Cluster version:1.30.2[upgrade/versions]kubeadm version: v1.30.4 W0905 09:07:28.91866563943version.go:104]could not fetch a Kubernetes version from the internet: unable to get URLhttps://dl.k8s.io/release/stable.txt:Gethttps://cdn.dl.k8s.io/release/stable.txt:context deadline exceeded(Client.Timeout exceededwhileawaiting headers)W0905 09:07:28.91900463943version.go:105]falling back to thelocalclient version: v1.30.4[upgrade/versions]Target version: v1.30.4[upgrade/versions]Latest versioninthe v1.30 series: v1.30.4 Components that must be upgraded manually after you have upgraded the control plane withkubeadm upgrade apply:COMPONENT NODE CURRENT TARGET kubelet master30.lz.cloud v1.30.2 v1.30.4 kubelet worker31.lz.cloud v1.30.2 v1.30.4 kubelet worker32.lz.cloud v1.30.2 v1.30.4 Upgrade to the latest versioninthe v1.30 series: COMPONENT NODE CURRENT TARGET kube-apiserver master30.lz.cloud v1.30.2 v1.30.4 kube-controller-manager master30.lz.cloud v1.30.2 v1.30.4 kube-scheduler master30.lz.cloud v1.30.2 v1.30.4 kube-proxy1.30.2 v1.30.4 CoreDNS v1.11.1 v1.11.1 etcd master30.lz.cloud3.5.12-03.5.12-0 You can now apply the upgrade by executing the following command: kubeadm upgrade apply v1.30.4 _____________________________________________________________________ The table below shows the current state of component configs as understood by this version of kubeadm. Configs that have ayesmarkintheMANUAL UPGRADE REQUIREDcolumnrequire manual config upgrade or resetting to kubeadm defaults before a successful upgrade can be performed. The version to manually upgrade to is denotedinthePREFERRED VERSIONcolumn. API GROUP CURRENT VERSION PREFERRED VERSION MANUAL UPGRADE REQUIRED kubeproxy.config.k8s.io v1alpha1 v1alpha1 no kubelet.config.k8s.io v1beta1 v1beta1 no _____________________________________________________________________升级 master 组件rootmaster30:~# kubeadm upgrade apply v1.30.4 --force[preflight]Running pre-flight checks.[upgrade/config]Reading configuration from the cluster...[upgrade/config]FYI: You canlookat this configfilewithkubectl -n kube-system get cm kubeadm-config -o yaml[upgrade]Running cluster health checks[WARNING CreateJob]: Jobupgrade-health-check-wpclkinthe namespacekube-systemdid not completein15s: no condition oftypeComplete[upgrade/version]You have chosen to change the cluster version tov1.30.4[upgrade/versions]Cluster version: v1.30.2[upgrade/versions]kubeadm version: v1.30.4[upgrade/prepull]Pulling images requiredforsetting up a Kubernetes cluster[upgrade/prepull]This might take a minute or two, depending on the speed of your internet connection[upgrade/prepull]You can also perform this actioninbeforehand usingkubeadm config images pullW0905 09:12:29.40698766867checks.go:844]detected that the sandbox imageregistry.aliyuncs.com/google_containers/pause:3.9of the container runtime is inconsistent with that used by kubeadm.It is recommended to useregistry.k8s.io/pause:3.9as the CRI sandbox image.[upgrade/apply]Upgrading your Static Pod-hosted control plane to versionv1.30.4(timeout: 5m0s)...[upgrade/etcd]Upgrading to TLSforetcd[upgrade/staticpods]Preparingforetcdupgrade[upgrade/staticpods]Current and new manifests of etcd are equal, skipping upgrade[upgrade/etcd]Waitingforetcd to become available[upgrade/staticpods]Writing new Static Pod manifests to/etc/kubernetes/tmp/kubeadm-upgraded-manifests3246055680[upgrade/staticpods]Preparingforkube-apiserverupgrade[upgrade/staticpods]Renewing apiserver certificate[upgrade/staticpods]Renewing apiserver-kubelet-client certificate[upgrade/staticpods]Renewing front-proxy-client certificate[upgrade/staticpods]Renewing apiserver-etcd-client certificate[upgrade/staticpods]Moved new manifest to/etc/kubernetes/manifests/kube-apiserver.yamland backed up old manifest to/etc/kubernetes/tmp/kubeadm-backup-manifests-2024-09-05-09-12-29/kube-apiserver.yaml[upgrade/staticpods]Waitingforthe kubelet to restart the component[upgrade/staticpods]This can take up to 5m0s[apiclient]Found1Podsforlabel selectorcomponentkube-apiserver[upgrade/staticpods]Componentkube-apiserverupgraded successfully![upgrade/staticpods]Preparingforkube-controller-managerupgrade[upgrade/staticpods]Renewing controller-manager.conf certificate[upgrade/staticpods]Moved new manifest to/etc/kubernetes/manifests/kube-controller-manager.yamland backed up old manifest to/etc/kubernetes/tmp/kubeadm-backup-manifests-2024-09-05-09-12-29/kube-controller-manager.yaml[upgrade/staticpods]Waitingforthe kubelet to restart the component[upgrade/staticpods]This can take up to 5m0s[apiclient]Found1Podsforlabel selectorcomponentkube-controller-manager[upgrade/staticpods]Componentkube-controller-managerupgraded successfully![upgrade/staticpods]Preparingforkube-schedulerupgrade[upgrade/staticpods]Renewing scheduler.conf certificate[upgrade/staticpods]Moved new manifest to/etc/kubernetes/manifests/kube-scheduler.yamland backed up old manifest to/etc/kubernetes/tmp/kubeadm-backup-manifests-2024-09-05-09-12-29/kube-scheduler.yaml[upgrade/staticpods]Waitingforthe kubelet to restart the component[upgrade/staticpods]This can take up to 5m0s[apiclient]Found1Podsforlabel selectorcomponentkube-scheduler[upgrade/staticpods]Componentkube-schedulerupgraded successfully![upload-config]Storing the configuration usedinConfigMapkubeadm-configinthekube-systemNamespace[kubelet]Creating a ConfigMapkubelet-configinnamespace kube-system with the configurationforthe kubeletsinthe cluster[upgrade]Backing up kubelet configfileto /etc/kubernetes/tmp/kubeadm-kubelet-config1570193532/config.yaml[kubelet-start]Writing kubelet configuration tofile/var/lib/kubelet/config.yaml[bootstrap-token]Configured RBAC rules to allow Node Bootstrap tokens to get nodes[bootstrap-token]Configured RBAC rules to allow Node Bootstrap tokens to post CSRsinorderfornodes to get long term certificate credentials[bootstrap-token]Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token[bootstrap-token]Configured RBAC rules to allow certificate rotationforallnodeclient certificatesinthe cluster[addons]Applied essential addon: CoreDNS[addons]Applied essential addon: kube-proxy[upgrade/successful]SUCCESS!Your cluster was upgraded tov1.30.4.Enjoy![upgrade/kubelet]Now that your control plane is upgraded, please proceed with upgrading your kubeletsifyou havent alreadydoneso.kubeadm upgrade apply做了以下工作检查你的集群是否处于可升级状态:API 服务器是可访问的所有节点处于Ready状态控制面是健康的强制执行版本偏差策略。确保控制面的镜像是可用的或可拉取到服务器上。如果组件配置要求版本升级则生成替代配置与/或使用用户提供的覆盖版本配置。升级控制面组件或回滚如果其中任何一个组件无法启动。应用新的CoreDNS和kube-proxy清单并强制创建所有必需的 RBAC 规则。如果旧文件在 180 天后过期将创建 API 服务器的新证书和密钥文件并备份旧文件。升级其他 master 节点rootmaster30:~# kubeadm upgrade nodekubeadm upgrade node在其他控制平节点上执行以下操作从集群中获取 kubeadmClusterConfiguration。可选操作备份 kube-apiserver 证书。升级控制平面组件的静态 Pod 清单。为本节点升级 kubelet 配置驱逐 master 节点# 驱逐 master节点并设置为不可调度rootmaster30:~# kubectl drain master30.lz.cloud --ignore-daemonsetsrootmaster30:~# kubectl get nodesNAME STATUS ROLES AGE VERSION master30.lz.cloud Ready,SchedulingDisabled control-plane 42h v1.30.2 worker31.lz.cloud Readynone42h v1.30.2 worker32.lz.cloud Readynone42h v1.30.2升级 kubelet 和 kubectl# 升级 kubectl和kubeletrootmaster30:~# apt install -y kubelet1.30.4-1.1 kubectl1.30.4-1.1# 重启 kubelet 服务rootmaster30:~# systemctl restart kubelet.service取消节点不可调度# 设置master节点可调度rootmaster30:~# kubectl uncordon master30.lz.cloud# 验证结果rootmaster30:~# kubectl get nodesNAME STATUS ROLES AGE VERSION master30.lz.cloud Ready control-plane 48m v1.30.4 worker31.lz.cloud Readynone37m v1.30.2 worker32.lz.cloud Readynone37m v1.30.2升级 worker升级 worker31# 升级kubeadmrootworker31:~# apt install -y kubeadm1.30.4-1.1# 升级 worker节点组件rootworker31:~# kubeadm upgrade node# 以上命令在工作节点上完成以下工作# 1.从集群取回 kubeadm ClusterConfiguration。# 2.为本节点升级 kubelet 配置。# 驱逐节点上 pod并取消节点不可调度rootmaster30:~# kubectl drain worker31.lz.cloud --ignore-daemonsetsrootmaster30:~# kubectl get nodesNAME STATUS ROLES AGE VERSION master30.lz.cloud Ready control-plane 50m v1.30.4 worker31.lz.cloud Ready,SchedulingDisablednone39m v1.30.2 worker32.lz.cloud Readynone39m v1.30.2# 升级 kubectl 和 kubeletrootworker31:~# apt install -y kubelet1.30.4-1.1 kubectl1.30.4-1.1# 重启kubelet服务rootworker31:~# systemctl restart kubelet.service# 取消节点不可调度rootmaster30:~# kubectl uncordon worker31.lz.cloudrootmaster30:~# kubectl get nodesNAME STATUS ROLES AGE VERSION master30.lz.cloud Ready control-plane 51m v1.30.4 worker31.lz.cloud Readynone40m v1.30.4 worker32.lz.cloud Readynone40m v1.30.2升级 worker32# 升级 kubeadmrootworker32:~# apt install -y kubeadm1.30.4-1.1# 升级 worker 节点组件rootworker32:~# kubeadm upgrade node# 驱逐节点上pod并取消节点不可调度rootmaster30:~# kubectl drain worker32.lz.cloud --ignore-daemonsetsrootmaster30:~# kubectl get nodesNAME STATUS ROLES AGE VERSION master30.lz.cloud Ready control-plane 52m v1.30.4 worker31.lz.cloud Readynone40m v1.30.4 worker32.lz.cloud Ready,SchedulingDisablednone40m v1.30.2# 升级 kubectl 和 kubeletrootworker32:~# apt install -y kubelet1.30.4-1.1 kubectl1.30.4-1.1# 重启 kubelet 服务rootworker32:~# systemctl restart kubelet.service# 取消节点不可调度rootmaster30:~# kubectl uncordon worker32.lz.cloudrootmaster30:~# kubectl get nodesNAME STATUS ROLES AGE VERSION master30.lz.cloud Ready control-plane 53d v1.30.4 worker31.lz.cloud Readynone53d v1.30.4 worker32.lz.cloud Readynone53d v1.30.4# 确认集群版本rootmaster30:~# kubectl versionClient Version: v1.30.4 Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3 Server Version: v1.30.4从故障状态恢复如果kubeadm upgrade失败并且没有回滚例如由于执行期间节点意外关闭 你可以再次运行kubeadm upgrade。 此命令是幂等的并最终确保实际状态是你声明的期望状态。要从故障状态恢复你还可以运行kubeadm upgrade apply --force而无需更改集群正在运行的版本。在升级期间kubeadm 向/etc/kubernetes/tmp目录下的如下备份文件夹写入数据kubeadm-backup-etcd-date-timekubeadm-backup-manifests-date-timekubeadm-backup-etcd包含当前控制面节点本地 etcd 成员数据的备份。 如果 etcd 升级失败并且自动回滚也无法修复则可以将此文件夹中的内容复制到/var/lib/etcd进行手工修复。如果使用的是外部的 etcd则此备份文件夹为空。kubeadm-backup-manifests包含当前控制面节点的静态 Pod 清单文件的备份版本。 如果升级失败并且无法自动回滚则此文件夹中的内容可以复制到/etc/kubernetes/manifests目录实现手工恢复。 如果由于某些原因在升级前后某个组件的清单未发生变化则 kubeadm 也不会为之生成备份版本。

相关新闻

ESP32驱动RGB-LED:从PWM控制到HSV色彩渐变实战

ESP32驱动RGB-LED:从PWM控制到HSV色彩渐变实战

1. 项目缘起:从点灯到玩色,ESP32的RGB初体验 玩Arduino的朋友,估计都是从点亮一颗LED开始的。从最基础的闪烁,到呼吸灯,再到用PWM调光,这个过程充满了成就感。但当你觉得单色LED已经玩得差不多了&#xff0…

2026/11/20 19:33:34 阅读更多 →
北京丰科卓辰:全国产6U VPX高速大容量存储板卡

北京丰科卓辰:全国产6U VPX高速大容量存储板卡

FKC-6UV7-M8全国产存储板卡是基于高性能国产FPGA与NVMe先进存储架构定制开发的一款高端数据采集存储板卡。该板卡为标准6U 5HP VPX架构产品,选取复旦微公司的JMF7V690T系列高性能FPGA,对外提供多路的高速数据接口;内部采用先进的NVME存储架构…

2026/11/4 10:58:33 阅读更多 →
一个双非软件工程本科生的逆风翻盘经历

一个双非软件工程本科生的逆风翻盘经历

写在前面 我的高考分数不高,只上了一个很一般的学校,也对自己的专业不感兴趣。但庆幸的是,大学四年,我慢慢的对编程产生了兴趣,学会了如何学习并走入了编程的大门。 后来还算顺利吧!考上了某211硕&#xff…

2026/10/24 13:59:58 阅读更多 →

最新新闻

Electron桌面应用开发实战:从环境搭建到打包发布全流程

Electron桌面应用开发实战:从环境搭建到打包发布全流程

做前端这些年,我大部分时间都在跟浏览器标签页打交道,直到第一次把桌面应用跑起来,才发现原来自己一直被限制在地址栏和刷新按钮围起来的框框里。Electron这个词,最开始是在某段时间频繁出现在同事的讨论里,一个用HTML…

2026/11/19 21:38:27 阅读更多 →
滑盖智能锁T109深度解析:结构原理、安装调试与故障排查

滑盖智能锁T109深度解析:结构原理、安装调试与故障排查

简介:亚太天能T109滑盖智能锁产品说明书是一份PDF格式的官方技术文档,主要面向智能锁安装人员、终端用户及售后维护者,帮助快速掌握产品安装、配置与常见故障处理。全包仅1个PDF文件,大小4.37MB,内容结构清晰&#xff…

2026/11/19 21:38:49 阅读更多 →
PMSM弱磁控制原理与Simulink建模:高速AGV驱动系统设计指南

PMSM弱磁控制原理与Simulink建模:高速AGV驱动系统设计指南

上个月我在调试一台高速AGV的驱动系统,整车要求空载跑到8m/s,电机转速已经拉到额定点,驱动器的输出电压也顶到了母线电压极限附近。再往上踩速度指令,转速纹丝不动,三相电流反而开始震荡。那会儿我的第一反应就是&…

2026/11/19 21:39:09 阅读更多 →
C++核心知识点:从内存管理到STL的实战指南

C++核心知识点:从内存管理到STL的实战指南

C 这门语言我前前后后折腾了十几年,用它写过嵌入式驱动、做过游戏服务端,也搭过基础组件库。每次看到新人一头扎进"C 核心知识点"这个搜索词,我其实挺感慨的——这门语言的难点从来不是语法多,而是知识点之间的耦合太深…

2026/11/19 21:36:50 阅读更多 →
T109滑盖智能锁怎么装?安装调试与故障排查全攻略

T109滑盖智能锁怎么装?安装调试与故障排查全攻略

简介:亚太天能T109滑盖智能锁产品说明书以PDF格式发布,适合智能锁用户、安装人员与售后维修人员阅读。内容围绕T109系列指纹密码刷卡锁展开,先介绍前面板、后面板、锁体、机械钥匙、电池盒盖等部件名称与功能,再给出完整安装步骤&…

2026/11/19 21:37:45 阅读更多 →
LLM知识降熵:从信息垄断到理解权下放

LLM知识降熵:从信息垄断到理解权下放

你有没有过这种经历:翻开源码注释、论文摘要或者一份专业报告,每个字都认识,连在一起却像天书。想找人聊聊,身边要么没人懂,要么懂的懒得跟你解释。后来我试着把同一段内容丢给一个大语言模型,几秒钟之后&a…

2026/11/19 21:36:20 阅读更多 →

日新闻

周新闻

月新闻