The mechanics behind the headlines

Back
How Operating Systems Work · Part 12

Signals and Shutdown: The System Comes Full Circle

Signals are how the kernel and processes communicate asynchronously — a mechanism for process control, error notification, and graceful shutdown. Here is how they work, how they are delivered, and how the system uses them to shut itself down.

June 18, 2026 · 15 min read

A signal is a small integer. It carries almost no information — just a number, and sometimes a small supplementary structure. It arrives asynchronously, interrupting whatever the recipient was doing. It cannot be queued in the way a message queue holds messages, and multiple instances of the same signal may collapse into one before they are delivered.

Despite all of this, signals are the mechanism that shuts down every process on the system, translates critical keyboard sequences like Ctrl-C into process control events, notifies every parent when a child exits, and terminates every program that accesses memory it should not. They are not elegant, but they are fundamental.

What Signals Are

A signal is a notification sent to a process or thread. The kernel sends signals to notify processes of events — a memory fault, a floating-point exception, a terminal hangup. Processes send signals to other processes for control — pause, resume, terminate. Users send signals from the keyboard or from kill.

Each signal is identified by a number and a name. The name is a macro defined in <signal.h>; the number is what the kernel actually uses. Standard signal numbers are defined by POSIX and are consistent across Linux systems.

The most commonly encountered signals fall into a few categories:

Process control

  • SIGTERM (15) — request termination. The default action is to exit. Can be caught and handled.
  • SIGKILL (9) — force termination. Cannot be caught, blocked, or ignored. The kernel kills the process directly.
  • SIGSTOP (19) — suspend execution. Cannot be caught, blocked, or ignored. The process enters state T.
  • SIGCONT (18) — resume a stopped process. Sent by the shell when you run fg or bg.

Terminal events

  • SIGINT (2) — interrupt from keyboard (Ctrl-C). Default action: terminate.
  • SIGQUIT (3) — quit from keyboard (Ctrl-). Default action: terminate and produce a core dump.
  • SIGHUP (1) — hang up. Sent when the controlling terminal closes. Daemons use it as a convention to reload configuration.

Error conditions

  • SIGSEGV (11) — segmentation fault. The process accessed memory it is not allowed to access.
  • SIGBUS (7) — bus error. Misaligned memory access or access to a non-existent physical address.
  • SIGFPE (8) — floating-point exception. Includes integer divide-by-zero.
  • SIGILL (4) — illegal instruction. The process executed an invalid CPU instruction.

Child and I/O events

  • SIGCHLD (17) — a child process has stopped or terminated. Sent to the parent.
  • SIGPIPE (13) — write to a pipe or socket with no readers. Default action: terminate.
  • SIGALRM (14) — timer expiry, set with alarm().

To see all signals defined on the current system:

kill -l

Real-time signals (SIGRTMIN through SIGRTMAX) are also available — unlike standard signals, they are queued and delivered in order. They are used for application-level event notification but not covered in depth here.

The Signal Lifecycle: Sent, Pending, Delivered

A signal passes through three states before it affects a process.

Sent — a signal is generated. This can happen in several ways: the kernel generates it in response to a hardware event (a page fault generates SIGSEGV), a process calls kill() targeting another process, a user presses Ctrl-C (the terminal driver sends SIGINT to the foreground process group), or a timer expires.

Pending — the signal has been sent but not yet delivered. The kernel records it in the target process's signal set. If the signal is currently blocked by the process's signal mask, it stays pending until unblocked. Standard signals do not queue — if SIGTERM is pending and another SIGTERM arrives before the first is delivered, only one is delivered. Real-time signals do queue.

Delivered — the kernel acts on the signal. Delivery happens at a specific point: when the kernel is about to return execution to user space, either after handling a system call or after handling an interrupt. The kernel checks the pending signal set at this point. If a signal is pending and not blocked, it is delivered.

This delivery window is important. A signal sent to a process that is currently executing in kernel space (inside a system call) is not delivered until the kernel is ready to return to user space. A process cannot receive a signal mid-syscall and have its handler run inside the kernel. Execution always shifts back to user space before the signal handler runs.

Signal Delivery: The Stack Frame

When the kernel delivers a signal to a process that has registered a handler, it does not simply call the handler function. It cannot — the kernel cannot call user space functions directly, and the process's current execution state needs to be preserved so it can resume after the handler returns.

Instead, the kernel manipulates the process's user-space stack.

Before returning to user space, the kernel saves the process's complete current execution state — registers, instruction pointer, signal mask — onto the process's user-space stack. This saved state is called a signal frame. The kernel then modifies the instruction pointer to point at the signal handler and sets up the stack as if the handler had been called normally, with a special return address pointing to a small piece of code called the signal trampoline.

When the signal handler returns, execution reaches the trampoline, which calls sigreturn() — a system call whose sole purpose is to restore the saved signal frame. The kernel pops the saved state off the stack, restores the registers and instruction pointer to where they were before the signal was delivered, and returns execution to exactly the point that was interrupted.

This stack manipulation is transparent to the handler. The handler is a normal C function that receives a signal number. It has no direct knowledge that the kernel rearranged the stack to invoke it. It returns normally. sigreturn() restores the original state. The interrupted code continues as if nothing happened — unless the handler modified shared state or the signal interrupted a blocking system call, causing it to return EINTR.

The signal trampoline on modern Linux systems is provided by the vDSO — a small kernel-generated library mapped into every process's address space automatically. It does not need to be in the program's own code.

Signal Handlers

A process registers a signal handler with sigaction():

struct sigaction sa;
sa.sa_handler = my_handler;
sigemptyset(&sa.sa_mask);
sa.sa_flags = SA_RESTART;
sigaction(SIGTERM, &sa, NULL);

When SIGTERM is delivered, my_handler runs instead of the default action (terminate). The sa_mask field specifies additional signals to block while the handler is executing — preventing the handler itself from being interrupted by another signal. SA_RESTART causes system calls interrupted by the signal to restart automatically rather than return EINTR.

Async-Signal Safety

Signal handlers run asynchronously — they can interrupt the process at any point, including points where the process is in the middle of a non-reentrant operation. This creates a dangerous class of bugs.

Consider a process that is halfway through a malloc() call. malloc() maintains internal data structures — free lists, heap metadata — that are in a partially updated state mid-operation. If a signal is delivered at that moment and the signal handler calls malloc(), the handler enters malloc() again while the internal structures are inconsistent. The result is a deadlock (if malloc() uses a mutex that the interrupted call already holds) or heap corruption (if it does not).

A function that is safe to call from a signal handler is called async-signal-safe. The POSIX standard defines a specific list of async-signal-safe functions. The list is small and conservative: write(), _exit(), signal(), kill(), and a handful of others. Notably absent: malloc(), printf(), syslog(), and any function that acquires a lock or uses global state.

The practical rule for signal handlers: do as little as possible. The common pattern is to set a volatile flag that the main loop checks, then return immediately. The main loop handles the actual work when it detects the flag. This keeps the handler to a few instructions that cannot deadlock or corrupt state.

Signal Masking

Each thread has a signal mask — a set of signals that are currently blocked. A blocked signal is not ignored; it is held as pending until it is unblocked. When unblocked, it is delivered.

The signal mask is per-thread. A multi-threaded process can have different threads block different signals. A common pattern for signal handling in multi-threaded programs is to block all signals in every thread except one dedicated signal-handling thread, which calls sigwait() in a loop. This avoids the async-signal-safety problem entirely — the handler runs in a normal thread context, not in an interrupt, and can call any function safely.

# View the signal mask of your current shell in hex
grep SigBlk /proc/$$/status

The SigBlk field is a bitmask. Each bit corresponds to a signal number. A set bit means that signal is currently blocked. Convert the hex value to binary to see which signals are blocked.

sigpending() returns the set of signals that are pending — sent but not yet delivered, either because they are blocked or because delivery has not yet occurred.

SIGKILL and SIGSTOP

Two signals stand apart from all others: SIGKILL and SIGSTOP cannot be caught, blocked, or ignored.

SIGKILL causes the kernel to terminate the process unconditionally. There is no handler, no cleanup, no opportunity for the process to flush buffers or close files. The kernel removes the process from the scheduler's runqueue, releases its resources, and marks it as a zombie until the parent calls wait(). A process in an uninterruptible sleep state (state D) will not receive SIGKILL until it leaves that state — the kernel cannot safely interrupt certain I/O operations mid-flight.

SIGSTOP suspends the process. The process enters state T. It receives no CPU time and makes no progress until SIGCONT is sent. Unlike SIGTERM, which the process can catch and handle by saving state before exiting, SIGSTOP gives no opportunity for the process to react.

These two signals are the kernel's override mechanism. No matter how a process is written — whether it catches every other signal, whether it has a buggy handler, whether it is deliberately trying to avoid termination — SIGKILL ends it and SIGSTOP freezes it. This is why container runtimes and init systems use SIGTERM first (giving the process a chance to clean up) and SIGKILL as the final resort.

Core Dumps

Several signals have a default action of terminating the process and producing a core dump: SIGSEGV, SIGILL, SIGFPE, SIGQUIT, and SIGABRT, among others.

A core dump is a snapshot of the process's state at the moment of termination. The kernel freezes all threads in the process, then writes the process's virtual address space — every mapped memory region, their contents, and the saved register state of each thread — to a file. The result is a file (traditionally named core or core.<pid>) that a debugger can load to inspect what the process was doing at the moment it died.

On modern systems with systemd, core dumps are typically handled by systemd-coredump. Instead of writing to the current directory, the kernel pipes the core data to a helper process defined in /proc/sys/kernel/core_pattern. systemd-coredump compresses and stores the dump in /var/lib/systemd/coredump/ and makes it accessible via coredumpctl.

coredumpctl list

This lists recent core dumps with the signal that caused them, the executable, and the timestamp. coredumpctl debug opens the most recent dump in gdb.

Core dumps can be disabled per-process by setting the core size limit to zero:

ulimit -c 0

Or enabled with no size limit:

ulimit -c unlimited

Zombie Processes

When a process exits — for any reason, including signal delivery — it does not disappear immediately. Its task_struct remains in the kernel's process table in zombie state (Z) until its parent calls wait() or waitpid() to collect its exit status.

The zombie holds almost nothing: its PID, its exit status, and its accounting information. It consumes no CPU and negligible memory. But it holds its PID, and PIDs are a finite resource — a system accumulating thousands of zombies eventually runs out.

SIGCHLD is sent to a parent whenever a child changes state: exits, stops, or continues. A well-written parent either calls wait() in a SIGCHLD handler to collect each child as it exits, or calls waitpid(-1, NULL, WNOHANG) in a loop to collect all available exits without blocking.

If a parent exits before calling wait(), its zombie children are reparented to the nearest subreaper — typically PID 1. systemd calls wait() for all its adopted children, so zombies from reparented processes are promptly reaped. A zombie that persists is almost always a sign that the original parent is still running but not calling wait().

The Shutdown Sequence

When the system shuts down, the kernel does not simply switch off. Every process must exit, every filesystem must flush pending writes, every device driver must release hardware, and every CPU must be brought to a safe halt. PID 1 orchestrates this.

On a systemd system, systemctl poweroff or shutdown -h now sends a message to systemd over its D-Bus socket. systemd transitions to the shutdown target. Here is the sequence:

1. Stop services. systemd sends SIGTERM to every service and unit it manages, in reverse dependency order. Services are given TimeoutStopSec (default 90 seconds) to exit cleanly. A service that has not exited by the deadline receives SIGKILL. This is why well-written services catch SIGTERM and implement graceful shutdown — closing network connections, flushing buffers, completing in-flight requests before exiting.

2. Kill remaining processes. After managed services are stopped, systemd sends SIGTERM to every remaining process in the system, then SIGKILL to any that survive the timeout. At this point no user-space processes should be running.

3. Unmount filesystems. With no processes holding file descriptors open, filesystems are unmounted. The VFS layer flushes the page cache — all dirty pages are written to disk. Journaling filesystems write a clean journal commit, marking the filesystem as cleanly unmounted. This is the step that prevents filesystem corruption on shutdown.

4. Disable interrupts. The kernel disables hardware interrupts on all CPU cores. No new work can arrive from devices.

5. Halt or power off. The kernel executes a final sequence: flushes CPU caches, writes any remaining data to disk, and executes the platform-specific halt or power-off instruction. On x86, this is typically the hlt instruction or an ACPI power-off call.

A Graceful Shutdown Handler

The practical consequence of all this: any long-running process — a server, a container entrypoint, a background daemon — should catch SIGTERM and shut down cleanly rather than letting the kernel force-kill it after the timeout.

A minimal example in bash:

#!/usr/bin/env bash

cleanup() {
    echo "SIGTERM received — draining connections..."
    sleep 2
    echo "Clean exit."
    exit 0
}

trap cleanup SIGTERM

echo "Running. PID: $$"
while true; do
    sleep 1
done

Run this script, then send it SIGTERM from another terminal:

kill -SIGTERM <pid>

The script catches the signal, runs cleanup(), and exits cleanly within two seconds. Without the trap, SIGTERM would terminate the script immediately with no cleanup.

In a container, Kubernetes sends SIGTERM to PID 1 of the container when a pod is terminated, waits terminationGracePeriodSeconds (default 30 seconds), then sends SIGKILL. A container that catches SIGTERM and drains active connections before exiting deploys and terminates cleanly. One that does not gets force-killed mid-request.

The System, Complete

This machine is now fully described.

At power-on, the CPU executed firmware from a fixed address. UEFI initialised hardware, verified the bootloader's signature, and handed control to GRUB. GRUB loaded the kernel and initramfs into memory. The kernel set up page tables, initialised the scheduler, mounted the root filesystem, and spawned PID 1.

From that point, everything came into existence: virtual address spaces constructed from page tables, processes created through clone(), system calls crossing the ring 3 to ring 0 boundary through the IDT entry for entry_SYSCALL_64, hardware interrupts routing through the APIC to kernel handlers, the scheduler's timer ticking every 4 milliseconds, files resolved through the VFS to inodes to disk blocks, processes communicating through pipes and sockets and shared memory, threads sharing address spaces while each maintaining their own stack and registers.

Every mechanism is now named.

References