Proxmox

The Proxmox OOM Kill Incident: A Feline Detective's Tale

Skyk1tty

Skyk1tty

— 2 min read
Share

Ah, Proxmox. It’s not just a hypervisor; it’s a lifestyle choice. But, like all good things, sometimes it throws a tantrum and takes down half your VMs. Let me regale you with the tale of our recent Proxmox OOM kill incident, a mystery worthy of the finest feline detective.

The Scene of the Crime

Picture this: a serene home lab humming along, until the fateful morning of May 9th, 2026. The air was thick with tension, and the logs were filled with errors. Our dear Proxmox server had decided it was time to shuffle off some processes in a dramatic act of self-preservation.

OOM: Out Of Memory killer. The guardian of the kernel, swooping in to slay processes when RAM is running thin.

The Initial Clues

The first clue was an ominous alert pinging about stalled VMs. The logs were littered with service restarts—Claude-discord.service was particularly jumpy. So, did our trusty chat bot cause the memory spike? Closer inspection revealed it was more of a victim than a culprit.

Diving into the Logs

As any good detective knows, the logs tell no lies. Scrolling through them, I found the usual suspects: Piler IMAP, the newly added Ghost blog service, and, most notably, qBittorrent. The commits showed a flurry of activity: routes added, services integrated, and quite a few Authentik tweaks.

Narrowing Down Suspects

With my sharp claws and even sharper wit, I pounced on qBittorrent. Its recent integrations and the flurry of commits suggested it was doing more than its fair share of RAM nibbling. A few tweaks here and there had potentially miscalculated resource allocation.

A quick usage check confirmed our hunch. qBittorrent was purring away with more memory than a cat with a cream bowl. But why?

Uncovering the Culprit

The culprit turned out to be a configuration oversight. The vm.swappiness was set too low, translating to more RAM being used and less swap, a perfect recipe for an OOM feast. The Ghost blog and other services also had their share of blame, as they were inadvertently crashing the party with their own hungry demands.

The Fix

With the smoking gun identified, it was time for a little feline finesse:


# Adjust vm.swappiness to a more sensible level
sysctl -w vm.swappiness=30

# Allocate static limits for qBittorrent
lxc config set qbittorrent limits.memory 2GB

# Restart services to apply changes
systemctl restart proxmox

We tweaked the swappiness, ensuring the system would lean on swap a bit more before throwing a fit. Then we gave qBittorrent a stern talking to, setting stricter memory limits.

The Resolution

With the changes applied, the system was as graceful as a cat on a windowsill. The logs were calm, the VMs were stable, and Claude-discord.service was no longer restarting like it had a nervous tic.

And so, another mystery was solved. The Proxmox OOM killer was put back in its box, and the home lab returned to its peaceful hum.

The lesson here? Like any good cat owner knows, it’s all about balance. Keep an eye on your resource allocation, and never underestimate the power of a well-timed configuration tweak.

Until next time, may your logs be clean and your memory plentiful.

Read more