weave 적용했으나 마스터노드가 notReady 에요

Question

weave 적용 했음에도 마스터노드 상태가 notReady 입니다. 원인을 못찾겠어요. 아마도 cni 문제 같은긴한데... kubeadm reset 한번 하면서 weave 를 삭제하고 /opt/cni/weave 는 없는 상태입니다. 아래 명령어 실행해도 /opt/cni 에 weave plugin이 생기지 않네요. $ kubectl apply - f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '
')" MemoryPressure False Fri, 23 Oct 2020 12:56:41 +0900 Fri, 23 Oct 2020 12:03:12 +0900 KubeletHasSufficientMemory kubelet has sufficient memory available DiskPressure False Fri, 23 Oct 2020 12:56:41 +0900 Fri, 23 Oct 2020 12:03:12 +0900 KubeletHasNoDiskPressure kubelet has no disk pressure PIDPressure False Fri, 23 Oct 2020 12:56:41 +0900 Fri, 23 Oct 2020 12:03:12 +0900 KubeletHasSufficientPID kubelet has sufficient PID available Ready False Fri, 23 Oct 2020 12:56:41 +0900 Fri, 23 Oct 2020 12:03:12 +0900 KubeletNotReady [container runtime status check may not have completed yet, runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized]

CLOUD SECURITY LAB · Answer

안녕하세요 강사 최일선입니다. cni 구성에 문제가 있으시다면 weavnet 말고 flannel이나 calico로 시도해보시겠습니까? 계속 업무가 너무 많은터라 빠른 답변을 못드려서 죄송합니다 ㅠ  최대한 빨리 답변 남기도록 노력하겠습니다.

kbh · Answer

swapoff -a 는 설치시 적용해두었습니다. 처음 설치시는 정상적으로 다 이루어집니다. kubeadm reset 이후에 발생합니다. 로그는 cni 구성 실패로 발생하는 내용들인 것 같습니다. W1025 14:49:09.142187 22111 nvidia.go:61] NVIDIA GPU metrics will not be available: no NVIDIA devices found W1025 14:49:09.174784 22111 docker_service.go:565] Hairpin mode set to "promiscuous-bridge" but kubenet is not enabled, falling back to "hairpin-veth" W1025 14:49:09.174948 22111 cni.go:239] Unable to update cni config: no networks found in /etc/cni/net.d W1025 14:49:47.681811 22661 cni.go:239] Unable to update cni config: no networks found in /etc/cni/net.d E1025 14:49:48.986306 22661 aws_credentials.go:77] while getting AWS credentials NoCredentialProviders: no valid providers in chain. Deprecated. For verbose messaging see aws.Config.CredentialsChainVerboseErrors E1025 14:49:48.995962 22661 kubelet.go:1218] Image garbage collection failed once. Stats initialization may not have completed yet: failed to get imageFs info: unable to find data in memory cache E1025 14:49:49.020108 22661 kubelet.go:2103] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized E1025 14:49:49.022834 22661 kubelet.go:1765] skipping pod synchronization - [container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful] E1025 14:49:49.053945 22661 factory.go:354] devicemapper filesystem stats will not be reported: unable to find thin_ls binary E1025 14:49:49.125220 22661 kubelet.go:1765] skipping pod synchronization - container runtime status check may not have completed yet F1025 14:49:49.183549 22661 kubelet.go:1296] Failed to start ContainerManager failed to initialize top level QOS containers: failed to update top level Burstable QOS cgroup : failed to set supported cgroup subsystems for cgroup [kubepods burstable]: failed to find subsystem mount for required subsystem: pids

CLOUD SECURITY LAB · Answer

안녕하세요 강사 최일선입니다. 혹시 sudo swapoff -a 명령은 잘 수행하셨나요? 대부분 시작되지 않는 현상이 이문제로 발생하곤 합니다. journalctl -u kubelet -f를 사용해 서비스의 로그를 확인해보시는 것도 좋을 것 같아요. 감사합니다.