인프런 커뮤니티 질문&답변

안녕하세요

여러 방법으로 재시도 해보았지만 설치 시 [InvalidDiskCapacity kubelet invalid capacity 0 on image filesystem] 관련 오류가 해결되지 않아서 문의드립니다.

확인 부탁드리겠습니다.

kubeadm init 을 하면, control-plane 노드가 잠시 수행은 되나, 아래 로그와 같이 disk 용량 부족으로 중지되는 것 같습니다.

환경은 aws ec2 ubuntu 20.04 버전을 사용했습니다. (22.04 버전도 동일함)

ec2 inbound 규칙에 관련 port 정보는 다 열려 있고,

용량은 부족하지 않게 할당하였습니다.

ubuntu@ip-172-31-11-187:~$ lsblk

NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT

loop0 7:0 0 24.9M 1 loop /snap/amazon-ssm-agent/7628

loop1 7:1 0 55.7M 1 loop /snap/core18/2790

loop2 7:2 0 63.5M 1 loop /snap/core20/2015

loop3 7:3 0 40.9M 1 loop /snap/snapd/20290

loop4 7:4 0 91.9M 1 loop /snap/lxd/24061

loop5 7:5 0 64M 1 loop /snap/core20/2264

xvda 202:0 0 100G 0 disk

├─xvda1 202:1 0 99.9G 0 part /

├─xvda14 202:14 0 4M 0 part

└─xvda15 202:15 0 106M 0 part /boot/efi

<인스턴스 유형>

t2.medium (2core, memory 4g)

<kubernetes version>

ubuntu@ip-172-31-11-187:~$ kubelet --version

Kubernetes v1.30.0

ubuntu@ip-172-31-11-187:~$ kubectl version

Client Version: v1.30.0

Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3

Server Version: v1.30.0

ubuntu@ip-172-31-11-187:~$ kubeadm version

kubeadm version: &version.Info{Major:"1", Minor:"30", GitVersion:"v1.30.0", GitCommit:"7c48c2bd72b9bf5c44d21d7338cc7bea77d0ad2a", GitTreeState:"clean", BuildDate:"2024-04-17T17:34:08Z", GoVersion:"go1.22.2", Compiler:"gc", Platform:"linux/amd64"}

감사합니다.

ubuntu@ip-172-31-11-187:~$ kubectl get nodes

NAME STATUS ROLES AGE VERSION

ip-172-31-11-187 NotReady control-plane 18s v1.30.0

ubuntu@ip-172-31-11-187:~$

ubuntu@ip-172-31-11-187:~$

ubuntu@ip-172-31-11-187:~$

ubuntu@ip-172-31-11-187:~$ kubectl describe node ip-172-31-11-187

Name: ip-172-31-11-187

Roles: control-plane

Labels: beta.kubernetes.io/arch=amd64

beta.kubernetes.io/os=linux

kubernetes.io/arch=amd64

kubernetes.io/hostname=ip-172-31-11-187

kubernetes.io/os=linux

node-role.kubernetes.io/control-plane=

node.kubernetes.io/exclude-from-external-load-balancers=

Annotations: kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/containerd/containerd.sock

node.alpha.kubernetes.io/ttl: 0

volumes.kubernetes.io/controller-managed-attach-detach: true

CreationTimestamp: Thu, 09 May 2024 06:47:35 +0000

Taints: node.kubernetes.io/not-ready:NoExecute

node-role.kubernetes.io/control-plane:NoSchedule

node.kubernetes.io/not-ready:NoSchedule

Unschedulable: false

Lease:

HolderIdentity: ip-172-31-11-187

AcquireTime: <unset>

RenewTime: Thu, 09 May 2024 06:47:58 +0000

Conditions:

Type Status LastHeartbeatTime LastTransitionTime Reason Message

---- ------ ----------------- ------------------ ------ -------

MemoryPressure False Thu, 09 May 2024 06:47:48 +0000 Thu, 09 May 2024 06:47:34 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available

DiskPressure False Thu, 09 May 2024 06:47:48 +0000 Thu, 09 May 2024 06:47:34 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure

PIDPressure False Thu, 09 May 2024 06:47:48 +0000 Thu, 09 May 2024 06:47:34 +0000 KubeletHasSufficientPID kubelet has sufficient PID available

Ready False Thu, 09 May 2024 06:47:48 +0000 Thu, 09 May 2024 06:47:34 +0000 KubeletNotReady container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized

Addresses:

InternalIP: 172.31.11.187

Hostname: ip-172-31-11-187

Capacity:

cpu: 2

ephemeral-storage: 101430960Ki

hugepages-2Mi: 0

memory: 4002720Ki

pods: 110

Allocatable:

cpu: 2

ephemeral-storage: 93478772582

hugepages-2Mi: 0

memory: 3900320Ki

pods: 110

System Info:

Machine ID: ec26d309223de68472068db4b74ceec4

System UUID: ec227c44-1b90-af54-a50d-03c7a515ac13

Boot ID: 637a1613-30b3-4577-bbbc-682dfd7b51fb

Kernel Version: 5.15.0-1058-aws

OS Image: Ubuntu 20.04.6 LTS

Operating System: linux

Architecture: amd64

Container Runtime Version: containerd://1.6.31

Kubelet Version: v1.30.0

Kube-Proxy Version: v1.30.0

PodCIDR: 192.168.0.0/24

PodCIDRs: 192.168.0.0/24

Non-terminated Pods: (5 in total)

Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age

--------- ---- ------------ ---------- --------------- ------------- ---

kube-system etcd-ip-172-31-11-187 100m (5%) 0 (0%) 100Mi (2%) 0 (0%) 25s

kube-system kube-apiserver-ip-172-31-11-187 250m (12%) 0 (0%) 0 (0%) 0 (0%) 25s

kube-system kube-controller-manager-ip-172-31-11-187 200m (10%) 0 (0%) 0 (0%) 0 (0%) 25s

kube-system kube-proxy-vktm2 0 (0%) 0 (0%) 0 (0%) 0 (0%) 9s

kube-system kube-scheduler-ip-172-31-11-187 100m (5%) 0 (0%) 0 (0%) 0 (0%) 25s

Allocated resources:

(Total limits may be over 100 percent, i.e., overcommitted.)

Resource Requests Limits

-------- -------- ------

cpu 650m (32%) 0 (0%)

memory 100Mi (2%) 0 (0%)

ephemeral-storage 0 (0%) 0 (0%)

hugepages-2Mi 0 (0%) 0 (0%)

Events:

Type Reason Age From Message

---- ------ ---- ---- -------

Normal Starting 7s kube-proxy

Normal Starting 25s kubelet Starting kubelet.

Warning InvalidDiskCapacity 25s kubelet invalid capacity 0 on image filesystem

Normal NodeHasSufficientMemory 25s kubelet Node ip-172-31-11-187 status is now: NodeHasSufficientMemory

Normal NodeHasNoDiskPressure 25s kubelet Node ip-172-31-11-187 status is now: NodeHasNoDiskPressure

Normal NodeHasSufficientPID 25s kubelet Node ip-172-31-11-187 status is now: NodeHasSufficientPID

Normal NodeAllocatableEnforced 25s kubelet Updated Node Allocatable limit across pods

Normal RegisteredNode 10s node-controller Node ip-172-31-11-187 event: Registered Node ip-172-31-11-187 in Controller

ubuntu@ip-172-31-11-187:~$

ubuntu@ip-172-31-11-187:~$ cat /var/lib/kubelet/kubeadm-flags.env

KUBELET_KUBEADM_ARGS="--container-runtime-endpoint=unix:///var/run/containerd/containerd.sock --pod-infra-container-image=registry.k8s.io/pause:3.9"

ubuntu@ip-172-31-11-187:~$

ubuntu@ip-172-31-11-187:~$

ubuntu@ip-172-31-11-187:~$ kubectl get nodes -o wide

NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME

ip-172-31-11-187 NotReady control-plane 3m15s v1.30.0 172.31.11.187 <none> Ubuntu 20.04.6 LTS 5.15.0-1058-aws containerd://1.6.31