https://methakur.info/tags/troubleshooting/Recent content in Troubleshooting on Munish Thakur | DevOps EngineerHugoen-usSun, 10 Nov 2024 00:00:00 +0000https://methakur.info/blog/debugging-oomkilled-pods/Sun, 10 Nov 2024 00:00:00 +0000https://methakur.info/blog/debugging-oomkilled-pods/<p>It&rsquo;s Monday morning, and Slack is blowing up. The backend service is down. Again.</p> <p>I SSH into the node and run <code>kubectl get pods</code>:</p> <div class="highlight"><div style="color:#f8f8f2;background-color:#282a36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"> <table style="border-spacing:0;padding:0;margin:0;border:0;"><tr><td style="vertical-align:top;padding:0;margin:0;border:0;"> <pre tabindex="0" style="color:#f8f8f2;background-color:#282a36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">1 </span><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">2 </span></code></pre></td> <td style="vertical-align:top;padding:0;margin:0;border:0;;width:100%"> <pre tabindex="0" style="color:#f8f8f2;background-color:#282a36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>NAME READY STATUS RESTARTS AGE </span></span><span style="display:flex;"><span>backend-django-7d8f9c-abcd 0/1 OOMKilled <span style="color:#bd93f9">5</span> 10m </span></span></code></pre></td></tr></table> </div> </div><p>If you&rsquo;ve worked with Kubernetes, you&rsquo;ve seen this. <strong>OOMKilled</strong> means your pod tried to use more memory than allowed and got terminated by the kernel.</p> <p>Here&rsquo;s how I debug it.</p>