PostgreSQL memory management and OOM in containers

The classic container OOM report looks like this: PostgreSQL restarts every few hours, dmesg shows Out of memory: Killed process ... (postgres), yet the team insists the pod has "plenty of memory" because free -m inside the container shows gigabytes free. Both observations are true, and they do not contradict each other: free reads /proc/meminfo, which is not namespaced — inside a container it reports the host's memory, not the cgroup limit the kernel actually enforces. The limit that kills you lives in the cgroup, not in /proc/meminfo.

Where the real limit is read from

On cgroup v2 hosts, the effective limit is in memory.max (and the current charge in memory.current); on cgroup v1 it is memory.limit_in_bytes and memory.usage_in_bytes:

# cgroup v2 (Kubernetes 1.31+, most modern distros)
cat /sys/fs/cgroup/memory.max
cat /sys/fs/cgroup/memory.current

# cgroup v1
cat /sys/fs/cgroup/memory/memory.limit_in_bytes

See the kernel documentation for cgroup v2 and the cgroup v1 memory controller for the full file list. Any sizing tool or runbook that derives "available memory" from /proc/meminfo inside a container is measuring the wrong box.

The budget that must fit inside the limit

PostgreSQL does not read cgroup limits. It sizes itself from postgresql.conf, so the configuration must be derived from the container limit, not from host RAM. The pieces that must fit (see Resource Consumption in the PostgreSQL 18 docs):

• shared_buffers — fixed at startup;
• work_mem × concurrent sorts/hashes — this is per operation, not per session: one query with several hash joins can consume several times work_mem, so max_connections × work_mem is only a rough ceiling;
• maintenance_work_mem (or autovacuum_work_mem) × concurrent vacuum workers and maintenance commands;
• temp_buffers per backend, WAL buffers, and per-connection overhead of a few MB per backend process;
• plus everything else in the container (sidecars, monitoring agents) and the page cache PostgreSQL depends on.

As a starting point, not a rule: on a dedicated PostgreSQL container, shared_buffers around 25% of the cgroup limit leaves room for connections and page cache; raise it only after measuring. The full configuration checklist is in Server configuration.

What PostgreSQL counts vs what the cgroup counts

Two accounting differences explain most "we were nowhere near the limit" surprises:

• Page cache is charged to the cgroup. Reads and writes of heap and WAL files are buffered by the kernel and charged to the cgroup that first touches each page. A container whose processes hold little anonymous memory can still sit at its limit because the rest is page cache — which is normal and mostly reclaimable, until a burst of anonymous allocations (a big work_mem spike, a new connection wave) arrives while the cache is dirty or actively referenced.
• RSS grows with touched pages. Backend processes share shared_buffers, and tools that sum RSS across processes double-count those shared pages. Do not size the limit by summing ps output; size it from memory.current under peak load.

The kernel only kills when there is no reclaimable memory left at the moment of allocation, which is why OOM kills cluster under load spikes rather than at steady state.

OOM killer scoring and oom_score_adj

When the cgroup (or the host) hits its limit, the kernel picks a victim by oom_score, adjustable per process through /proc/<pid>/oom_score_adj (range -1000 to 1000; -1000 exempts the process entirely — see the proc filesystem documentation). Two operational facts matter:

• Protecting the postmaster with -1000 is the standard recipe, but oom_score_adj is inherited by child processes: every backend forked afterwards also becomes unkillable, which is the opposite of what you want. If the postmaster is protected, backends must reset their own score — for example by re-setting oom_score_adj from a wrapper, or by accepting that on Kubernetes you control this at pod level instead.
• On Kubernetes you cannot set oom_score_adj per container; the kubelet assigns it from the pod's QoS class: Guaranteed pods get -997, BestEffort pods get 1000, Burstable pods land in between. This is documented under node-pressure eviction. One database per pod is what makes this protection meaningful.

Also set vm.overcommit_memory = 2 considerations aside carefully: the PostgreSQL documentation's Linux Memory Overcommit section explains the trade-off — raising vm.overcommit_memory to 2 lowers the chance of the OOM killer being invoked at all, at the cost of failing allocations earlier.

Kubernetes requests, limits, and QoS

For a database pod, the defensible pattern from the Kubernetes memory resource documentation and Pod QoS classes:

• Set requests.memory equal to limits.memory so the pod is Guaranteed: it will not be evicted under node pressure before Burstable/BestEffort pods, and it gets the favorable oom_score_adj.
• Derive postgresql.conf from the limit (previous section), not from the node size — pods get rescheduled to larger nodes and silently change their environment.
• Leave headroom in the limit for page cache and spikes: sizing shared_buffers + connections to consume nearly 100% of the limit guarantees the next work_mem burst is fatal.

Diagnosing an OOM kill

Work from the kernel evidence inward to the query:

# Kernel record of the kill (host or via kubectl logs of the node)
dmesg -T | grep -i -E 'out of memory|oom_kill'
journalctl -k | grep -i oom

# cgroup v2: oom_kill counter increments per kill in this cgroup
cat /sys/fs/cgroup/memory.events
# low 0 / high 0 / max <n> / oom <n> / oom_kill <n>

# cgroup v1 equivalent
cat /sys/fs/cgroup/memory/memory.oom_control

memory.events also shows max events (limit hits that triggered reclaim without a kill) — a rising max count with zero oom_kill means you are permanently at the ceiling and a kill is a matter of time.

On the PostgreSQL side, look for the memory consumers active just before the kill. Temp-file usage is the classic work_mem overflow signature; it is tracked per database, not per session:

SELECT datname, temp_files, pg_size_pretty(temp_bytes) AS temp_written
FROM pg_stat_database
ORDER BY temp_bytes DESC;

temp_bytes is cumulative, so compare deltas around the incident window. Attribution to specific statements comes from the log: set log_temp_files = 0 (or a threshold) so every spill records the query that caused it, then correlate with pg_stat_activity sessions that were active at the kill time. The monitoring pipeline for this is covered in Monitoring and logging.

Prevention checklist

• One postmaster per pod; Guaranteed QoS (requests == limits).
• shared_buffers, work_mem, maintenance_work_mem, and max_connections derived from the cgroup limit, not host RAM — re-derive after any limit change.
• Connection pooler in front of the database so max_connections × work_mem stays bounded.
• log_temp_files = 0 (or a threshold) and alerts on temp_bytes growth.
• Alert on memory.events oom_kill increments and on max counter growth.
• Never trust free, top, or htop inside a container for capacity decisions.

My PostgreSQL 18 runs in a Kubernetes pod with memory limit <LIMIT_GB> GiB.

Current settings:
- shared_buffers = <value>
- work_mem = <value>
- maintenance_work_mem = <value>
- max_connections = <value>
- autovacuum_max_workers = <value>

1. Estimate worst-case memory consumption and check it against the cgroup limit.
2. Propose corrected values derived from the limit, with the reasoning for each.
3. Give the kubectl / cgroup commands to verify the real current usage (memory.current, memory.events).
4. List the pg_stat_activity / pg_stat_database queries that would reveal which sessions drive temp-file and memory pressure.

Treat the output as a hypothesis: apply it in a test pod and watch memory.current and memory.events under peak load before changing production.

Related pages

• Deriving postgresql.conf values → Server configuration
• Temp-file and statement-level evidence → Monitoring and logging
• When metadata itself eats the memory → Too many tables are a problem