EasyStack

OS Internals

Understand how the operating system uses stacks everywhere: process management, system calls, interrupts, threads, and memory protection.

User Stack vs Kernel Stack

Every running process has not one but two stack regions associated with it: a user-mode stack and a kernel-mode stack. The user stack lives in the process address space and is used whenever your program executes user-mode code. The kernel stack lives in kernel memory and is used whenever the CPU executes kernel-mode code on behalf of that process.

The kernel stack is typically small, often just 8 KB to 16 KB, because kernel-mode code is expected to use stack space conservatively. The user stack, by contrast, can grow much larger and is limited by the virtual memory configuration of the system.

These two stacks are separate regions precisely so that a buggy user program cannot overwrite kernel state. The CPU's privilege level, determined by the code segment selector, dictates which stack is active at any moment.

How stack memory works in detail

How System Calls Use the Stack

When your program performs a system call such as read(), write(), or open(), the CPU transitions from user mode to kernel mode. This is the classic mode switch. The transition swaps to the kernel stack automatically.

On many architectures, the hardware saves the current user-mode state, including the stack pointer, instruction pointer, and flags, onto the kernel stack. The kernel then runs its own handler, which pushes its local variables and return addresses onto the kernel stack. When the system call completes, the saved user state is restored from the kernel stack and control returns to user mode, with the user stack pointer restored to exactly where it was before the call.

This is why syscall overhead includes a stack switch: the CPU must save and restore state across two different stack regions on every system call.

Interrupt Handling and the Stack

Interrupts are asynchronous events from hardware, such as a timer tick, a keyboard press, or a network packet arrival. Like system calls, they force a mode transition, but they can occur at almost any time, even while user code is running.

When an interrupt fires, the CPU saves the interrupted program's state onto the appropriate kernel stack. The interrupt handler then executes its own code on that stack. Because interrupts can be nested, many kernels use a dedicated interrupt stack separate from the normal kernel stack, preventing one deeply nested interrupt from overflowing the handler's space.

Critically, the interrupt handler must restore the interrupted program's state exactly when it finishes. Any mistake in the push/pop order on the stack will corrupt the program and crash the system. This is why interrupt handlers are written with extreme care about stack discipline.

How call stacks and interrupts relate

Thread-Local Storage

Each thread in a multithreaded process gets its own independent user stack. Local variables declared inside functions are thread-local by virtue of living on that thread's stack. Two threads running the same code will push their local frames onto two different stacks and never interfere.

The OS allocates a stack region at thread creation and records its bounds in the thread control block. When a thread is scheduled, the CPU loads that thread's stack pointer. When a thread exits, its stack is reclaimed.

Beyond the main stack, threads may use thread-local storage (TLS) for globals that should be unique per thread. On many platforms, the thread's stack region also stores a pointer to a thread-local storage area, so each stack provides the anchor for that thread's TLS block.

Stack Size Limits

Operating systems impose a limit on stack size. On Linux, the default stack size for the main thread is typically 8 MB, configurable with ulimit -s. Secondary threads are often created with a smaller explicit size passed to pthread_attr_setstacksize().

On Windows, the default reserved stack size for each thread is 1 MB, set through the linker's /STACK option or the STACKSIZE directive. On macOS, the main thread stack is limited to 8 MB by default, and the limit can be inspected with ulimit -s.

These limits matter in practice. Deep or unbounded recursion will eventually hit the limit and raise a stack overflow, crashing the program. You can inspect and raise the limit from the shell:

# Linux / macOS
ulimit -s            # view stack size in KB
ulimit -s unlimited  # remove the limit (not recommended)

# Windows (PowerShell)
wmic process get name, commandline   # inspect running processes

Why stack depth matters for recursion

How the OS Allocates Stack Memory

The user stack grows downward: the stack pointer starts high in the region and decreases as frames are pushed. To avoid pre-committing all of the stack's virtual memory at once, the OS maps the stack lazily.

A guard page sits just below the currently committed bottom of the stack. When the stack pointer crosses into the guard page, the CPU raises a page fault. The kernel's page fault handler decides what to do:

  • If the stack has more room to grow, the OS commits another page and moves the guard page down.
  • If the stack has reached its configured limit, the OS raises a stack overflow signal (Linux SIGSEGV, Windows STATUS_STACK_OVERFLOW) and typically terminates the thread.

This guard-page mechanism is what makes stack growth automatic while still catching runaway growth. It is also why stack overflow and heap corruption are detected separately: the guard page stops a stack from silently writing into the heap region that sits below it.

Detailed stack memory allocation guide

Context Switching and Stack Saving

When the OS switches from one thread to another, it performs a context switch. Part of that work is saving the outgoing thread's CPU state, including its stack pointer, and loading the incoming thread's saved state.

Each kernel thread has its own kernel stack, which holds the saved registers during the switch. The context switch routine pushes the current registers onto the old thread's kernel stack, then loads the new thread's stack pointer and pops those registers off the new thread's kernel stack.

This is a beautiful use of the stack: the scheduling data structure stores the base state, while the kernel stack itself temporarily holds the live register state during each switch. Both the user stack and the kernel stack are essential to making multitasking work.

Security: Stack Canaries and ASLR

Stacks store local variables next to return addresses, which makes them a target for buffer overflow attacks. An attacker who overflows a local buffer can overwrite the saved return address and redirect execution to malicious code.

To defend against this, compilers insert a stack canary, a random sentinel value placed between the local variables and the saved return address. Before a function returns, the canary is verified. If it changed, the program aborts before the corrupted return address can be used.

ASLR (Address Space Layout Randomization) is the second key defense. The OS randomizes the base address of the stack, heap, and libraries on each program launch. Because the runtime stack address is unpredictable, an attacker cannot hard-code the address to jump to, making reliable exploitation much harder.

On Linux the canary is enabled by default with gcc -fstack-protector-all or by the distro's hardening flags. On Windows, the /GS flag adds canaries, and modern 64-bit builds enable ASLR by default.

Writing memory-safe stacks in C

Key Takeaway

The stack is not just a data structure you use in algorithms. It is the foundational runtime mechanism of every operating system: the user stack runs your program, the kernel stack runs system calls and interrupts, guard pages manage growth safely, and canaries plus ASLR keep the whole thing secure.

Explore all EasyStack guides

Frequently Asked Questions

What is the difference between the user stack and the kernel stack?

The user stack lives in the process user space and holds local variables and return addresses for user-mode code. The kernel stack lives in kernel space, is assigned per process, and holds frames for kernel-mode code such as system call handlers and interrupt routines. They are separate regions and the CPU switches between them during mode transitions.

How is stack memory allocated and protected in an OS?

The OS maps a virtual memory region for the stack and places a guard page just below it. The stack grows downward, and when a thread touches the guard page, the OS catches the page fault, expands the stack mapping, or raises a stack overflow signal. Guard pages prevent a growing stack from silently corrupting adjacent heap data.

What are stack canaries and how does ASLR protect the stack?

A stack canary is a random sentinel value placed between local data and the return address. Before a function returns, the canary is checked; if it changed, a buffer overflow was attempted and the program aborts. ASLR randomizes the base address of the stack on every run so attackers cannot reliably predict addresses, making stack-smashing exploits much harder.