쿠버네티스 클러스터 구축시 CRI 에러

[preflight] Running pre-flight checks.

[preflight] Some fatal errors occurred:

        [ERROR CRI]: unable to check if the container runtime at "/var/run/dockershim.sock" is running: fork/exec /usr/bin/crictl -r /var/run/dockershim.sock info: no such file or directory

[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`

centos7 환경에서 했구요 vm으로 구축중입니다

kubeadm join  을 워커노드 만들려고 하는 곳에 했는데 이런 에러가 발생 합니다 

로그는 워커노드에서 복사한것입니다  

--ignore 옵션을 줘서 했는데 마스터노드에서 kubectl get nodes 를 했을때 worker노드가 안나와서 

이 문제를 해결해야될거 같아서 질문합니다

일단 kubelet 했을때 에러가

I0625 11:25:10.209563    4571 feature_gate.go:226] feature gates: &{{} map[]}

W0625 11:25:10.214818    4571 cni.go:171] Unable to update cni config: No networks found in /etc/cni/net.d

I0625 11:25:10.235874    4571 server.go:376] Version: v1.10.0

I0625 11:25:10.235929    4571 feature_gate.go:226] feature gates: &{{} map[]}

I0625 11:25:10.236103    4571 plugins.go:89] No cloud provider specified.

W0625 11:25:10.236145    4571 server.go:517] standalone mode, no API client

W0625 11:25:10.266236    4571 server.go:433] No api server defined - no events will be sent to API server.

I0625 11:25:10.266260    4571 server.go:613] --cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /

I0625 11:25:10.266507    4571 container_manager_linux.go:242] container manager verified user specified cgroup-root exists: /

I0625 11:25:10.266519    4571 container_manager_linux.go:247] Creating Container Manager object based on Node Config: {RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime:docker CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:cgroupfs KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[{Signal:memory.available Operator:LessThan Value:{Quantity:100Mi Percentage:0} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.1} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.inodesFree Operator:LessThan Value:{Quantity:<nil> Percentage:0.05} GracePeriod:0s MinReclaim:<nil>} {Signal:imagefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.15} GracePeriod:0s MinReclaim:<nil>}]} ExperimentalQOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalCPUManagerReconcilePeriod:10s ExperimentalPodPidsLimit:-1 EnforceCPULimits:true}

I0625 11:25:10.266628    4571 container_manager_linux.go:266] Creating device plugin manager: true

I0625 11:25:10.266672    4571 state_mem.go:36] [cpumanager] initializing new in-memory state store

I0625 11:25:10.266735    4571 state_mem.go:87] [cpumanager] updated default cpuset: ""

I0625 11:25:10.266749    4571 state_mem.go:95] [cpumanager] updated cpuset assignments: "map[]"

W0625 11:25:10.269731    4571 kubelet_network.go:139] Hairpin mode set to "promiscuous-bridge" but kubenet is not enabled, falling back to "hairpin-veth"

I0625 11:25:10.269757    4571 kubelet.go:556] Hairpin mode set to "hairpin-veth"

I0625 11:25:10.271450    4571 client.go:75] Connecting to docker on unix:///var/run/docker.sock

I0625 11:25:10.271468    4571 client.go:104] Start docker client with request timeout=2m0s

W0625 11:25:10.272624    4571 cni.go:171] Unable to update cni config: No networks found in /etc/cni/net.d

I0625 11:25:10.276863    4571 docker_service.go:244] Docker cri networking managed by kubernetes.io/no-op

I0625 11:25:10.287597    4571 docker_service.go:249] Docker Info: &{ID:2WIO:IMHB:ODVY:WZQO:O2HO:LBN5:WS6G:HGBI:Q2VE:LKYE:PD4P:UIOZ Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:9 Driver:overlay2 DriverStatus:[[Backing Filesystem xfs] [Supports d_type true] [Native Overlay Diff true]] SystemStatus:[] Plugins:{Volume:[local] Network:[bridge host macvlan null overlay] Authorization:[] Log:[]} MemoryLimit:true SwapLimit:true KernelMemory:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6tables:true Debug:false NFd:16 OomKillDisable:true NGoroutines:46 SystemTime:2020-06-25T11:25:10.279440086+09:00 LoggingDriver:journald CgroupDriver:systemd NEventsListener:0 KernelVersion:3.10.0-1062.18.1.el7.x86_64 OperatingSystem:CentOS Linux 7 (Core) OSType:linux Architecture:x86_64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:0xc4206e09a0 NCPU:2 MemTotal:2405302272 GenericResources:[] DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:k8s-node1 Labels:[] ExperimentalBuild:false ServerVersion:1.13.1 ClusterStore: ClusterAdvertise: Runtimes:map[docker-runc:{Path:/usr/libexec/docker/docker-runc-current Args:[]} runc:{Path:docker-runc Args:[]}] DefaultRuntime:docker-runc Swarm:{NodeID:m2c9t50qh7aa0yopj7inyr4zu NodeAddr:192.168.0.54 LocalNodeState:pending ControlAvailable:false Error: RemoteManagers:[{NodeID:kjtuefhbkp88pi4ln1h6aog8s Addr:192.168.0.53:2377}] Nodes:0 Managers:0 Cluster:0xc4206d08c0} LiveRestoreEnabled:false Isolation: InitBinary:/usr/libexec/docker/docker-init-current ContainerdCommit:{ID: Expected:aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1} RuncCommit:{ID:66aedde759f33c190954815fb765eedc1d782dd9 Expected:9df8b306d01f59d3a8029be411de015b7304dd8f} InitCommit:{ID:fec3683b971d9c3ef73f284f176672c44b448662 Expected:949e6facb77383876aeff8a6944dde66b3089574} SecurityOptions:[name=seccomp,profile=/etc/docker/seccomp.json]}

F0625 11:25:10.287744    4571 server.go:233] failed to run Kubelet: failed to create kubelet: misconfiguration: kubelet cgroup driver: "cgroupfs" is different from docker cgroup driver: "systemd"

가 나와서 

1.

/etc/systemd/system/kubelet.service.d/10-kubeadm.conf 에

Environment="KUBELET_CGROUP_ARGS=--cgroup-driver=systemd“   

추가 하고 

systemctl show --property=Environment kubelet | cat 

확인 후에 했는데도

에러가 나와서

 다른 곳은 systemd로 수정했는데도 다르다고 나오기도하고  cri에러와  kubelet에 에러가 나와서 질문 합니다 .

[root@instance-1 ~]# kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.5", GitCommit:"e6503f8d8f769ace2f338794c914a96fc335df0f", GitTreeState:"clean", BuildDate:"2020-06-26T03:45:16Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

[root@instance-1 ~]# uname -a
Linux instance-1 3.10.0-1127.10.1.el7.x86_64 #1 SMP Wed Jun 3 14:28:03 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

[root@instance-1 ~]# docker -v
Docker version 19.03.12, build 48a66213fe

[root@instance-1 ~]# kubectl get nodes
NAME         STATUS   ROLES    AGE     VERSION
instance-1   Ready    master   14m     v1.18.5
instance-2   Ready    <none>   9m37s   v1.18.5