Wondering how to resolve Kubernetes error “context deadline exceeded”? We can help you.

As part of our Server Management Services, we assist our customers with several Kubernetes queries.

Today, let us see how our Support techs proceed to resolve it.

How to resolve Kubernetes error “context deadline exceeded”?

The error ‘context deadline exceeded’ means that we ran into a situation where a given action was not completed in an expected timeframe.

Usually, issue occurs if Pods become stuck in Init status.

Typical error will look as shown below:

Warning FailedCreatePodSandBox 93s (x8 over 29m) kubelet, 97011e0a-f47c-4673-ace7-d6f74cde9934 Failed to create pod sandbox: rpc error: code = DeadlineExceeded desc = context deadline exceeded Normal SandboxChanged 92s (x8 over 29m) kubelet, 97011e0a-f47c-4673-ace7-d6f74cde9934 Pod sandbox changed, it will be killed and re-created.

You would see errors for containers from kubelet.stderr.log.

Example:

E0114 14:57:13.656196 9838 remote_runtime.go:128] StopPodSandbox "ca05be4d6453ae91f63fd3f240cbdf8b34377b3643883075a6f5e05001d3646b" from runtime service failed: rpc error: code = DeadlineExceeded desc = context deadline exceeded
...
E0114 14:57:13.656256 9838 kuberuntime_manager.go:901] Failed to stop sandbox {"docker" "ca05be4d6453ae91f63fd3f240cbdf8b34377b3643883075a6f5e05001d3646b"}
...
W0114 14:57:30.151650 9838 cni.go:331] CNI failed to retrieve network namespace path: cannot find network namespace for the terminated container "ca05be4d6453ae91f63fd3f240cbdf8b34377b3643883075a6f5e05001d3646b"

You can also validate the status of the node-agent-hyperbus by running the following nsxcli command from the node (as root):

sudo -i

/var/vcap/jobs/nsx-node-agent/bin/nsxcli

“at the nsx-cli prompt, enter”: get node-agent-hyperbus status

Expected output:

HyperBus status: Healthy

In this scenario you would see the following error instead:

% An internal error occurred

This causes a loop of DEL (delete) requests to the nsx-node-agent process

Today, let us see the simple steps followed by our Support techs to resolve it.

Restarting the nsx-node-agent process will workaround this issue:

— Use “bosh ssh” command to access the worker node

Then, run the below commands.

sudo -i

monit restart nsx-node-agent

— Wait for nsx-node-agent to restart: watch monit summary

This must resolve the issue.

[Stuck in between? We’d be glad to assist you]

Conclusion

In short, today we saw steps followed by our Support Techs resolve Kubernetes failed to start in the docker desktop error.