kubernetes中用NFS做后端存储支不支持PVC扩容?
在虚拟机时代,我们经常会收到给磁盘扩容的需求,一般我们就是添加一块磁盘,然后做LVM扩容操作,那在容器时代,我们可以给PVC扩容吗?
之前发了持久化存储相关的知识--PV和PVC,StorageClass,本文来给你介绍一下用NFS做后端存储能不能扩容的问题,当然在生产上没有哪个愿意用NFS做后端存储......
你首先需要准备好NFS存储
首先我们介绍一下pv和pvc模式,也就是纯手动创建的模式。
(1)、首先我们创建一个PV的YAML文件(pv2.yaml)
apiVersion: v1kind: PersistentVolumemetadata: name: my-pv02 labels: storage: pvspec: accessModes: - ReadWriteOnce capacity: storage: 500Mi persistentVolumeReclaimPolicy: Recycle nfs: path: /data/k8s server: 10.1.10.128
创建pv
# kubectl apply -f pv2.yamlpersistentvolume/my-pv02 created# kubectl get pvNAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE 30smy-pv02 500Mi RWO Recycle Bound default/pvc-test 30s
(2)、创建PVC(pvc.yaml),我们这里创建一个比较小的pvc,如下
apiVersion: v1kind: PersistentVolumeClaimmetadata: name: pvc-testspec: accessModes: - ReadWriteOnce resources: requests: storage: 500Mi
然后创建pvc
# kubectl apply -f pvc.yaml# kubectl get pvcNAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGEpvc-test Bound my-pv02 500Mi RWO 58s
我们其实可以看到,我在pvc里定义的是500Mi,它会自动匹配刚好合适的PV。
这时候如果我们修改my-pv02的YAML文件,如下:
apiVersion: v1kind: PersistentVolumemetadata: name: my-pv02 labels: storage: pvspec: accessModes: - ReadWriteOnce capacity: storage: 2Gi persistentVolumeReclaimPolicy: Recycle nfs: path: /data/k8s server: 10.1.10.128
然后我们尝试创建这个PV,如下:
# kubectl apply -f pv2.yamlpersistentvolume/my-pv02 configured# kubectl get pvNAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGEmy-pv02 2Gi RWO Recycle Bound default/pvc-test 116m# kubectl get pvcNAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGEpvc-test Bound my-pv02 500Mi RWO 116m
我们看到我们my-pv02的CAPACITY变成了2Gi,我们的pvc还是500Mi,那么我们能不能对PVC进行扩容呢?我尝试的修改了pvc的配置文件,如下:
apiVersion: v1kind: PersistentVolumeClaimmetadata: name: pvc-testspec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi
然后我们执行这个配置文件
]# kubectl apply -f pvc.yamlError from server (Forbidden): error when applying patch:{"metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"v1\",\"kind\":\"PersistentVolumeClaim\",\"metadata\":{\"annotations\":{},\"name\":\"pvc-test\",\"namespace\":\"default\"},\"spec\":{\"accessModes\":[\"ReadWriteOnce\"],\"resources\":{\"requests\":{\"storage\":\"1Gi\"}}}}\n"}},"spec":{"resources":{"requests":{"storage":"1Gi"}}}}to:Resource: "/v1, Resource=persistentvolumeclaims", GroupVersionKind: "/v1, Kind=PersistentVolumeClaim"Name: "pvc-test", Namespace: "default"Object: &{map["apiVersion":"v1" "kind":"PersistentVolumeClaim" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"v1\",\"kind\":\"PersistentVolumeClaim\",\"metadata\":{\"annotations\":{},\"name\":\"pvc-test\",\"namespace\":\"default\"},\"spec\":{\"accessModes\":[\"ReadWriteOnce\"],\"resources\":{\"requests\":{\"storage\":\"500Mi\"}}}}\n" "pv.kubernetes.io/bind-completed":"yes" "pv.kubernetes.io/bound-by-controller":"yes"] "creationTimestamp":"2019-12-24T10:46:05Z" "finalizers":["kubernetes.io/pvc-protection"] "name":"pvc-test" "namespace":"default" "resourceVersion":"15059" "selfLink":"/api/v1/namespaces/default/persistentvolumeclaims/pvc-test" "uid":"6a905af6-5e44-4940-ba87-e56f6e99df24"] "spec":map["accessModes":["ReadWriteOnce"] "resources":map["requests":map["storage":"500Mi"]] "volumeMode":"Filesystem" "volumeName":"my-pv02"] "status":map["accessModes":["ReadWriteOnce"] "capacity":map["storage":"500Mi"] "phase":"Bound"]]}for: "pvc.yaml": persistentvolumeclaims "pvc-test" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
我们发现报错了,错误信息如下:
persistentvolumeclaims "pvc-test" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
它的意思就是说只有动态供应的pvc可以调整大小,供应pvc的存储类必须支持调整大小。也就是说我们手动创建PV,PVC模式是不支持调整大小的。
我们现在来验证一下用storageclass创建的PVC。
(1)、因为我们使用nfs作为后端存储,所以我们需要先创建一个nfs-client,YAML文件如下:
---apiVersion: v1kind: ServiceAccountmetadata: name: nfs-client-provisioner---apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRolemetadata: name: nfs-client-provisioner-clusterrolerules: - apiGroups: [""] resources: ["persistentvolumes"] verbs: ["get", "list", "watch", "create", "delete"] - apiGroups: [""] resources: ["persistentvolumeclaims"] verbs: ["get", "list", "watch", "update"] - apiGroups: ["storage.k8s.io"] resources: ["storageclasses"] verbs: ["get", "list", "watch"] - apiGroups: [""] resources: ["events"] verbs: ["list", "watch", "create", "update", "patch"] - apiGroups: [""] resources: ["endpoints"] verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]---apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRoleBindingmetadata: name: nfs-client-provisioner-clusterrolebindingsubjects:- kind: ServiceAccount name: nfs-client-provisioner namespace: defaultroleRef: kind: ClusterRole name: nfs-client-provisioner-clusterrole apiGroup: rbac.authorization.k8s.io---apiVersion: extensions/v1beta1kind: Deploymentmetadata: name: nfs-client-prosionerspec: replicas: 1 strategy: type: Recreate template: metadata: labels: app: nfs-client-prosioner spec: serviceAccountName: nfs-client-provisioner containers: - name: nfs-client-prosioner image: registry.cn-hangzhou.aliyuncs.com/rookieops/nfs-client-provisioner:v0.1 imagePullPolicy: IfNotPresent volumeMounts: - name: nfs-client-root mountPath: /data/pv env: - name: PROVISIONER_NAME value: rookieops/nfs - name: NFS_SERVER value: 10.1.10.128 - name: NFS_PATH value: /data/k8s volumes: - name: nfs-client-root nfs: server: 10.1.10.128 path: /data/k8s
创建YAML文件
# kubectl apply -f !$kubectl apply -f nfs-client.yamlserviceaccount/nfs-client-provisioner createdclusterrole.rbac.authorization.k8s.io/nfs-client-provisioner-clusterrole createdclusterrolebinding.rbac.authorization.k8s.io/nfs-client-provisioner-clusterrolebinding createddeployment.extensions/nfs-client-prosioner created# kubectl get podNAME READY STATUS RESTARTS AGEnfs-client-prosioner-56f44c675b-t2ml9 1/1 Running 0 6s
(1)我们创建一个普通的StorageClass(storageclass.yaml)
apiVersion: storage.k8s.io/v1kind: StorageClassmetadata: name: nfs-client-storageclassprovisioner: rookieops/nfs
创建这个YAML文件
# kubectl apply -f storageclass.yamlstorageclass.storage.k8s.io/nfs-client-storageclass created# kubectl get storageclasses.storage.k8s.ioNAME PROVISIONER AGEnfs-client-storageclass rookieops/nfs 34s
创建一个PVC(storageclass-pvc.yaml)
apiVersion: v1kind: PersistentVolumeClaimmetadata: name: test-nfs-pvc2 annotations: volume.beta.kubernetes.io/storage-class: "nfs-client-storageclass"spec: accessModes: - ReadWriteMany resources: requests: storage: 1Mi
创建这个YAML文件
# kubectl apply -f storageclass-pvc.yamlpersistentvolumeclaim/test-nfs-pvc2 created# kubectl get pvcNAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGEpvc-test Bound my-pv02 500Mi RWO 26dtest-nfs-pvc2 Bound pvc-9dd9c608-6875-4609-829c-c92920aaa783 1Mi RWX nfs-client-storageclass 21s
我们看到已经Bound了,我们现在直接修改pvc(storageclass-pvc.yaml)
apiVersion: v1kind: PersistentVolumeClaimmetadata: name: test-nfs-pvc2 annotations: volume.beta.kubernetes.io/storage-class: "nfs-client-storageclass"spec: accessModes: - ReadWriteMany resources: requests: storage: 100Mi
然后我们执行这个YAML文件
# kubectl apply -f storageclass-pvc.yamlError from server (Forbidden): error when applying patch:{"metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"v1\",\"kind\":\"PersistentVolumeClaim\",\"metadata\":{\"annotations\":{\"volume.beta.kubernetes.io/storage-class\":\"nfs-client-storageclass\"},\"name\":\"test-nfs-pvc2\",\"namespace\":\"default\"},\"spec\":{\"accessModes\":[\"ReadWriteMany\"],\"resources\":{\"requests\":{\"storage\":\"100Mi\"}}}}\n"}},"spec":{"resources":{"requests":{"storage":"100Mi"}}}}to:Resource: "/v1, Resource=persistentvolumeclaims", GroupVersionKind: "/v1, Kind=PersistentVolumeClaim"Name: "test-nfs-pvc2", Namespace: "default"Object: &{map["apiVersion":"v1" "kind":"PersistentVolumeClaim" "metadata":map["annotations":map["kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"v1\",\"kind\":\"PersistentVolumeClaim\",\"metadata\":{\"annotations\":{\"volume.beta.kubernetes.io/storage-class\":\"nfs-client-storageclass\"},\"name\":\"test-nfs-pvc2\",\"namespace\":\"default\"},\"spec\":{\"accessModes\":[\"ReadWriteMany\"],\"resources\":{\"requests\":{\"storage\":\"1Mi\"}}}}\n" "pv.kubernetes.io/bind-completed":"yes" "pv.kubernetes.io/bound-by-controller":"yes" "volume.beta.kubernetes.io/storage-class":"nfs-client-storageclass" "volume.beta.kubernetes.io/storage-provisioner":"rookieops/nfs"] "creationTimestamp":"2020-01-20T06:04:20Z" "finalizers":["kubernetes.io/pvc-protection"] "name":"test-nfs-pvc2" "namespace":"default" "resourceVersion":"29483" "selfLink":"/api/v1/namespaces/default/persistentvolumeclaims/test-nfs-pvc2" "uid":"9dd9c608-6875-4609-829c-c92920aaa783"] "spec":map["accessModes":["ReadWriteMany"] "resources":map["requests":map["storage":"1Mi"]] "volumeMode":"Filesystem" "volumeName":"pvc-9dd9c608-6875-4609-829c-c92920aaa783"] "status":map["accessModes":["ReadWriteMany"] "capacity":map["storage":"1Mi"] "phase":"Bound"]]}for: "storageclass-pvc.yaml": persistentvolumeclaims "test-nfs-pvc2" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
发现报与上面一样的错,这是因为要支持动态扩容需要满足两个条件:
后端底层存储支持卷扩展(后端存储保证足够资源)
- 需要在StorageClass对象中设置allowVolumeExpansion为true
我们由于是测试,申请的资源比较少,我们直接对StorageClass对象进行修改(storageclass.yaml),如下:
apiVersion: storage.k8s.io/v1kind: StorageClassmetadata: name: nfs-client-storageclassprovisioner: rookieops/nfsallowVolumeExpansion: true
然后我们重新申明一下
# kubectl apply -f storageclass.yamlstorageclass.storage.k8s.io/nfs-client-storageclass configured
我们再次创建前面的storageclass-pvc.yaml文件
# kubectl apply -f storageclass-pvc.yamlpersistentvolumeclaim/test-nfs-pvc2 configured
我们发现这次没有报错,我们查看一下这个PVC
# kubectl get pvcNAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGEpvc-test Bound my-pv02 500Mi RWO 26dtest-nfs-pvc2 Bound pvc-9dd9c608-6875-4609-829c-c92920aaa783 1Mi RWX nfs-client-storageclass 14m
我们一看,发现并没有扩展成功,这是为什么呢?我们describe一下这个pvc,发现如下提示:
# kubectl describe pvc test-nfs-pvc2...Warning ExternalExpanding 7m43s volume_expandIgnoring the PVC: didn't find a plugin capable of expanding the volume; waiting for an external controller to process this PVC.
报错信息为:没有找到可扩展的插件。
我上官方网站一看,原来人家已经说的很清楚了:Although the feature is enabled by default, a cluster admin must opt-in to allow users to resize their volumes. Kubernetes v1.11 ships with volume expansion support for the following in-tree volume plugins: AWS-EBS, GCE-PD, Azure Disk, Azure File, Glusterfs, Cinder, Portworx, and Ceph RBD.
我们的NFS并不被支持(用NFS做后端存储的小伙伴注意了哈)。
完
©著作权归作者所有:来自51CTO博客作者mb5ff97f7b72697的原创作品,如需转载,请注明出处,否则将追究法律责任更多相关文章
- 使用Operator部署Prometheus
- Weblogic中间件创建文件权限问题解决
- ListView适配器 Activity案例
- Android 开发后台
- Postgresql管理_创建数据库
- 初次撩MYCAT小姐姐
- 第三周
- 实战:从Mysql数据库frm文件中,提取表结构创建SQL语句
- 如何知道你创建的MySQL索引是高效,合理的