The Ultimate Troubleshooting Guide for Exit Code 143

If you were to create exit codes for reasons why people leave a bar, you'd probably reserve code 143 for scenarios where bargoers gracefully "finish their whiskey and beer," as Semisonic put it, and then head home quietly at closing time. When containers exist with code 143, it basically means they've shut down without making a scene.

That's a more favorable scenario than, say, exit code 137, which is like getting kicked out of a bar because you drank too much and became a disruption when your ex didn’t answer the phone. In the world of Docker container exit codes, 137 refers to containers that are forced to shut down because they're consuming too many resources and risk destabilizing other containers.

By most measures, terminating gracefully due to exit code 143 is preferable to being bounced out in a code-137 scenario. Still, exit code 143 isn't always desirable. You typically don't want your containers to shut down unexpectedly at all, even if they shut down peacefully. So, when you encounter a Docker container exit code labeled 143, you'll want to troubleshoot it to figure out what caused the exit and how to prevent it from happening again.

Read on for guidance on this task as we unpack exit code 143 causes and look at ways to prevent them.

How to check for container exit code 143

Let's begin our discussion by explaining how you determine if your container exited with code 143.

You can check the exit code of a container by running the command:

The output will look something like this:

As you can see, the STATUS column records information about the container's exit code.

Alternatively, if you just want the exit code without all of the other information, this one-liner will do the trick:

What causes exit code 143?

If you discover that a container stopped with exit code 143, it means the container shut down because it received and obeyed the SIGTERM signal. So, at a high level, you can say that exit code 143 happens because a container was SIGTERMed. But to understand what that really means, you have to understand the meaning of SIGTERM.

In Linux, SIGTERM is a type of signal that any process can send to another process to ask it to shut down. For example, if you run the kill [process ID] command on Linux, it will cause a SIGTERM request to be sent to the process you target. The process is allowed to ignore the request if it wants – which is why frustrated Linux admins sometimes resort to commands like kill -9 [process ID], which tells the operating system to shut down a process forcibly rather than simply asking it nicely to terminate on its own. Kill -9 is kind of like the process termination equivalent of "sudo make me a sandwich", but we digress.

Anyway, in the context of containers, SIGTERM is a way of asking containers to terminate and giving them the opportunity to do so gracefully. As long as a container responds to SIGTERM by shutting down, it will exit with code 143.

Other Docker container exit codes are reserved for scenarios where containers don't shut down willingly. For example, you'll see code 139 if your container was forcibly shut down because it tried to request memory that wasn't assigned to it. And code 137, as we noted above, appears if the container receives a SIGKILL signal, which forces it to shut down immediately whether it wants to or not.

Docker exit code 143 is probably the “best” exit code you can encounter other than code 0, which means the container shut down entirely on its own because it completed whichever task it was running. When you see code 143, you know your container did what it was supposed to do when it was asked to terminate. Instead of crashing or leaving data half-processed, the container gracefully shut down.

Nonetheless, as we'll explain in more detail below, exit code 143 can be associated with undesirable events. If your container doesn't automatically restart and keep running, you'll need to figure out why it is exiting with code 143.

Possible causes of exit code 143

Operating systems and orchestrators don't send SIGTERM requests to containers and trigger exit code 143 events just to be annoying. They do it for a reason, and understanding what caused exit code 143 is essential for ensuring that your containers can restart and keep operating properly.

There are many potential causes of exit code 143. The most common include:

 • The Pod that the container belongs to is being moved to a different server.

 • The cluster is running out of resources and Kubernetes decides to kill one Pod in order to free up resources for another.

 • The container is using too many memory resources and Kubernetes asks it to shut down in order to free up resources for other containers.

 • Kubernetes wants to remove the container's host node from the cluster, so it sends SIGTERM to the container in order to drain the node – which tells the container, in effect, "you don't have to go home, but you can't stay here."

 • An admin manually sends a SIGTERM request to the container using a command like kill or docker stop. This is rare, but it could happen in situations where someone wants a container to shut down but doesn't have access to the Kubernetes cluster that the container belongs to.

In some of these scenarios, exit code 143 isn't a problem because the container should restart on its own. For example, if a node is being drained and the container shuts down for that reason, Kubernetes should restart the container (and its Pod) on a different node.

But in other cases, exit code 143 indicates a problem. If your container is SIGTERMed because it's eating up too much memory, for instance, it could be a sign that you have memory leak issues or that you’re not managing memory requests and limits effectively.

How to troubleshoot exit code 143

If a container stops unexpectedly with code 143 and doesn't restart on its own, there are three main places to look for information about what's wrong:

 • The operating system logs of the host node (which you can find at /var/log/syslog on most Linux distributions). Grep the logs for terminated or SIGTERM to check for events that might have caused the container to receive the SIGTERM signal.

 • The Kubelet log for the node, which is typically located at /var/log/kubelet.log. If the SIGTERM request came from Kubernetes, you'll likely find information about it in the Kubelet log, since Kubelet is responsible for interfacing between individual nodes (and containers running on them) and the cluster.

 • Logs from the container, if you have them. Ideally, a well-designed container will register SIGTERM events inside a log file so that you'll know exactly when the container received the SIGTERM request and what it did in response.

If you're lucky, one of these logs will tell you what caused the container to receive a SIGTERM request, or at least where the request came from. That information should reveal whether the container was misbehaving, or whether it was SIGTERMED for a routine reason, like the draining of a node.

Advanced troubleshooting techniques

If you're not fortunate enough to find the cause of exit code 143 in syslog or kubelet.log, you may need to do some experimentation with the container and dig deeper.

Begin by restarting the container normally and monitoring its resource consumption patterns, which you can track using docker stats or an external container monitoring tool. If you see the container's memory or CPU consumption increasing over time without an apparent cause, it's likely that you have a memory management problem inside the container that is leading to it being SIGTERMed.

You can also check the Pod priority levels that you've configured for the Pod to which the container belongs. It could be the case that Kubernetes is shutting the container down because you'd configured it to be part of a low-priority workload, in which case changing the priority level (or allocating more resources to your cluster) should prevent exit code 143 from happening again.

Consider, too, inspecting other containers that belong to the same Pod. If they're also exiting with code 143, there's a good chance that the issue is linked to the way Kubernetes is managing the Pod, rather than being a problem specific to the individual container.

Best practices to prevent exit code 143

Strategies for preventing exit code 143 in Docker containers align pretty closely with standard best practices for creating and deploying containers:

 • Test the code that you include in the container to ensure it’s stable and won't leak memory.

 • Ensure that your container can record events about signals it has received to a log file, so that you can troubleshoot error codes based on log data specific to your container.

 • Avoiding running multiple applications inside the same container. Although doing so won't directly trigger a 143 error code, it could make the codes harder to troubleshoot because you won't know which application is leading to the SIGTERM request.

 • Ensure that you properly configure Pod priority levels, resource requests and resource limits in Kubernetes to avoid having containers shut down because they are consuming more resources than the cluster can spare.

 • Use autoscaling where available so that Kubernetes can acquire more resources automatically. This helps to avoid shutting containers down due to lack of available resources.