1.概述本文通过 nfs-subdir-external-provisioner 在 Kubernetes 集群中通过PVC请求自动创建PV分配nfs共享存储通过StorageClass 达到统一配置存储的行为。此方式不支持动态扩容pvc、pv容量如果想配置动态存储扩容请参考NFS CSI驱动部署。2.前置条件NFS 服务器已搭建并共享目录如 /nfs_share/k8s。Kubernetes 集群版本 ≥ 1.20网络插件如 Calico正常运行。镜像仓库确保能拉取 nfs-subdir-external-provisioner 镜像3.扩容流程3.1 流程说明用户触发通过 kubectl edit pvc 或 YAML 修改 PVC 的存储容量。系统响应Kubernetes 更新 PVC/PV 状态。Provisioner 确保 StorageClass 允许扩容。存储层处理NFS 服务器检查共享目录所在磁盘的剩余空间。PV 容量标记更新逻辑层。文件系统调整若支持在线扩容如 ext4/xfskubelet 自动扩展文件系统。应用生效挂载点容量更新应用可直接使用新空间部分场景需重启 Pod3.2 流程图[用户操作]│ ▽ 编辑 PVC 请求扩容例如 1Gi → 2Gi │ ▽ Kubernetes 检测到 PVC 变更 │ ▽ Provisioner 检查 StorageClass 允许扩容 │ ▽ NFS 服务器验证磁盘空间是否充足 │ ▽ PV 容量标记更新1Gi → 2Gi │ ▽ 文件系统自动扩容如 ext4/xfs 在线扩展 │ ▽ Pod 挂载点生效无需重启* │ ▽ 应用验证新容量df-h/data4.下载 Provisioner 的部署文件访问 GitHub 仓库 nfs-subdir-external-provisioner下载最新版本的部署模板wgethttps://raw.githubusercontent.com/kubernetes-sigs/nfs-subdir-external-provisioner/master/deploy/class.yamlwgethttps://raw.githubusercontent.com/kubernetes-sigs/nfs-subdir-external-provisioner/master/deploy/rbac.yamlwgethttps://raw.githubusercontent.com/kubernetes-sigs/nfs-subdir-external-provisioner/master/deploy/deployment.yaml5.部署步骤5.1 创建namespacekubectl create ns nfs-provisioner5.2 修改Provisioner部署文件并执行5.2.1 rbac.yaml文件此文件不需要修改apiVersion: v1 kind: ServiceAccount metadata: name: nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: nfs-provisioner --- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: name: nfs-client-provisioner-runner rules: - apiGroups:[]resources:[nodes]verbs:[get,list,watch]- apiGroups:[]resources:[persistentvolumes]verbs:[get,list,watch,create,delete]- apiGroups:[]resources:[persistentvolumeclaims]verbs:[get,list,watch,update]- apiGroups:[storage.k8s.io]resources:[storageclasses]verbs:[get,list,watch]- apiGroups:[]resources:[events]verbs:[create,update,patch]--- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: run-nfs-client-provisioner subjects: - kind: ServiceAccount name: nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: nfs-provisioner roleRef: kind: ClusterRole name: nfs-client-provisioner-runner apiGroup: rbac.authorization.k8s.io --- kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: name: leader-locking-nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: nfs-provisioner rules: - apiGroups:[]resources:[endpoints]verbs:[get,list,watch,create,update,patch]--- kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: leader-locking-nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: nfs-provisioner subjects: - kind: ServiceAccount name: nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: nfs-provisioner roleRef: kind: Role name: leader-locking-nfs-client-provisioner apiGroup: rbac.authorization.k8s.io参数详解**ServiceAccount**作用为 Provisioner Pod 分配身份需与deployment.yaml中的serviceAccountName一致。关键字段name和namespace。2. **ClusterRole**作用定义 Provisioner 需要的集群级权限如管理 PV、PVC、StorageClass。关键权限persistentvolumes创建/删除 PV。persistentvolumeclaims更新 PVC 状态。storageclasses读取存储类配置。3. **ClusterRoleBinding**作用将ServiceAccount绑定到ClusterRole授予权限。修改注意事项命名空间一致性所有资源必须部署到同一命名空间如nfs-provisioner。权限最小化不要随意扩大verbs范围如避免*。5.2.2 class.yaml 需要修改apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: nfs-client provisioner: k8s-sigs.io/nfs-subdir-external-provisioner# or choose another name, must match deployments env PROVISIONER_NAMEparameters: archiveOnDelete:trueallowVolumeExpansion:truereclaimPolicy: Retain参数详解**provisioner**作用指定存储分配器名称必须与deployment.yaml中的PROVISIONER_NAME完全一致。示例k8s-sigs.io/nfs-subdir-external-provisioner。2. **allowVolumeExpansion**作用是否允许动态扩容 PVC。可选值true允许或false禁止。3. **reclaimPolicy**作用定义 PV 回收策略。可选值Retain删除 PVC 后保留 PV 和数据需手动清理。Delete删除 PVC 时自动删除 PV 和 NFS 子目录。推荐值生产环境建议Delete测试环境可用Retain。4. **parameters.archiveOnDelete**作用删除 PVC 时如何处理 NFS 子目录。可选值true将子目录重命名为archived-原目录名保留数据。false直接删除子目录彻底清理。推荐值测试环境用false生产环境谨慎选择。5.2.3 deployment.yaml需要修改registry.k8s.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2 这个镜像我是通过国外服务器下载的如需要请联系我apiVersion: apps/v1 kind: Deployment metadata: name: nfs-client-provisioner labels: app: nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: nfs-provisioner spec: replicas:1strategy: type: Recreate selector: matchLabels: app: nfs-client-provisioner template: metadata: labels: app: nfs-client-provisioner spec: serviceAccountName: nfs-client-provisioner containers: - name: nfs-client-provisioner image:172.16.4.177:8090/ltzx/registry.k8s.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2 volumeMounts: - name: nfs-client-root mountPath: /persistentvolumes env: - name: PROVISIONER_NAME value: k8s-sigs.io/nfs-subdir-external-provisioner - name: NFS_SERVER value:172.16.4.60 - name: NFS_PATH value: /nfs_share/k8s/nfs-provisioner volumes: - name: nfs-client-root nfs: server:172.16.4.60 path: /nfs_share/k8s/nfs-provisioner参数详解**image**作用NFS Provisioner 的容器镜像地址。推荐版本使用最新稳定版如v4.0.2。私有仓库若需认证需配置imagePullSecrets。2. **env环境变量****PROVISIONER_NAME**必须与class.yaml中的provisioner完全一致。**NFS_SERVER**NFS 服务器 IP 或域名。**NFS_PATHNFS 共享目录的绝对路径。3. volumes数据卷****nfs.server和nfs.path**必须与NFS_SERVER和NFS_PATH一致。**readOnly默认为false确保可写。4. replicas和strategy****replicas: 1**单副本运行避免多副本竞争。**strategy.type: Recreate**更新时先删除旧 Pod再创建新 Pod。5.3 执行文件CODEBLOCK_PLACEHOLDER_0007 CODEBLOCK_PLACEHOLDER_00085.4 验证部署查看 nfs-provisioner pod 状态[rootmaster1 nfs]# kubectl get pods -n nfs-provisionerNAME READY STATUS RESTARTS AGE nfs-client-provisioner-5bd66c9447-hj8qz1/1 Running010h查看 storageclass 确认 “allowVolumeExpansion”:true允许动扩容[rootmaster1 nfs]# kubectl get storageclass nfs-client -o yamlallowVolumeExpansion:trueapiVersion: storage.k8s.io/v1 kind: StorageClass metadata: annotations: kubectl.kubernetes.io/last-applied-configuration:|{allowVolumeExpansion:true,apiVersion:storage.k8s.io/v1,kind:StorageClass,metadata:{annotations:{},name:nfs-client},parameters:{archiveOnDelete:true},provisioner:k8s-sigs.io/nfs-subdir-external-provisioner,reclaimPolicy:Retain}creationTimestamp:2025-04-02T09:06:32Zname: nfs-client resourceVersion:10513520uid: e1496b73-1d18-46b7-9fe6-598a9229f125 parameters: archiveOnDelete:trueprovisioner: k8s-sigs.io/nfs-subdir-external-provisioner reclaimPolicy: Retain volumeBindingMode: Immediate6.单副本测试6.1 部署busybox进行测试CODEBLOCK_PLACEHOLDER_0011 CODEBLOCK_PLACEHOLDER_00126.2 测试步骤#查看pod busybox-test是否正常运行[rootmaster1 test]# kubectl get podsNAME READY STATUS RESTARTS AGE busybox-test-01/1 Running06s#进入pod验证存储[rootmaster1 test]# kubectl exec -it busybox-test-0 -- /bin/sh#容器内查看挂载点/# df -h /mnt/storageFilesystem Size Used Available Use% Mounted on172.16.4.60:/nfs_share/k8s/nfs-provisioner/default-data-busybox-test-0-pvc-f5396e40-f0b3-4019-9d38-e738e1a37d18299.4G90.7G208.7G30% /mnt/storage#创建测试文件验证pv权限是否正常/# echo NFS Storage Test /mnt/storage/testfile.txt/# cat /mnt/storage/testfile.txtNFS Storage Test#查看pvc状态[rootmaster1 test]# kubectl get pvcNAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE>#查看pv状态[rootmaster1 test]# kubectl get pvpvc-f5396e40-f0b3-4019-9d38-e738e1a37d18 1Gi RWO Retain Bound default/data-busybox-test-0 nfs-client 92m在nfs服务器上验证“/nfs_share/k8s/nfs-provisioner” 是上边配置的nfs共享目录#进到nfs共享目录中[rootlocalhost ~]# cd /nfs_share/k8s/nfs-provisioner#查看自动生成的pv目录[rootlocalhost nfs-provisioner]# lsdefault-data-busybox-test-0-pvc-f5396e40-f0b3-4019-9d38-e738e1a37d18#查看上边创建的文件内容[rootlocalhost nfs-provisioner]# cd default-data-busybox-test-0-pvc-f5396e40-f0b3-4019-9d38-e738e1a37d18/[rootlocalhost default-data-busybox-test-0-pvc-f5396e40-f0b3-4019-9d38-e738e1a37d18]# cat testfile.txtNFS Storage Test7.多副本测试7.1 将副本数修改为3个或者多个CODEBLOCK_PLACEHOLDER_0015 CODEBLOCK_PLACEHOLDER_00167.2 验证#查看pod状态[rootmaster1 test]# kubectl get podsNAME READY STATUS RESTARTS AGE busybox-test-01/1 Running03m24s busybox-test-11/1 Running09s busybox-test-21/1 Running05s#查看pv、pvc状态[rootmaster1 test]# kubectl get pv | grep ^pvcpvc-32f0aa55-b4f5-4ad5-a11c-04c63a2ac29c 1Gi RWO Retain Bound default/data-busybox-test-1 nfs-client 94m pvc-f02fd5d2-b4d5-431b-8243-971184ed2182 1Gi RWO Retain Bound default/data-busybox-test-2 nfs-client 94m pvc-f5396e40-f0b3-4019-9d38-e738e1a37d18 1Gi RWO Retain Bound default/data-busybox-test-0 nfs-client 97m[rootmaster1 test]# kubectl get pvcNAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE>#查看nfs共享挂载点发现已经有三个目录[rootlocalhost ~]# cd /nfs_share/k8s/nfs-provisioner[rootlocalhost nfs-provisioner]# lsdefault-data-busybox-test-0-pvc-f5396e40-f0b3-4019-9d38-e738e1a37d18 default-data-busybox-test-2-pvc-f02fd5d2-b4d5-431b-8243-971184ed2182 default-data-busybox-test-1-pvc-32f0aa55-b4f5-4ad5-a11c-04c63a2ac29c8.参考文档https://www.cnblogs.com/lori/p/17944619