Value Score: 90/100 Role: Linux memory owner Proof Level: Conceptual, lab-routed
Do not use Understanding the Linux Kernel, Chapter 9, or the local linux_kernel_v2.6_virtual memory manager folder as the primary source for current Linux MM. They are useful for the timeless idea that VMAs describe policy and page tables describe translations, but they predate modern VMA storage, locking, folios, GUP/pinning work, THP, userfaultfd, memcg/reclaim evolution, and several hardening changes.
Use this current path instead:
- Modern Linux memory manager reading map
docs.kernel.org/mm/process_addrs.htmlformm_struct, VMAs, Maple Tree, and VMA/page-table locking.docs.kernel.org/mm/page_tables.htmlfor current generic page-table terminology.docs.kernel.org/mm/page_cache.htmlfor page cache and folio vocabulary.docs.kernel.org/core-api/maple_tree.htmlfor the VMA index data structure.- Target kernel source:
include/linux/mm_types.h,mm/mmap.c,arch/<arch>/mm/fault.c,mm/memory.c,mm/filemap.c, andmm/rmap.c.
Read old Chapter 9 only after this, and annotate what is obsolete: VMA rbtree emphasis, old locking names, old page-cache terminology, and old fault-path assumptions.
From a Linux MM perspective, a task is not "a process with memory." It is:
- a
task_struct - optionally an
mm_struct - a kernel stack
- saved register state for entry/exit
Threads usually have different task_structs but share the same mm_struct.
Important nuance:
- userspace threads typically share one
mm_struct - kernel threads usually have
task->mm == NULL - kernel threads can temporarily borrow an
active_mm
That is already a senior-level detail because it shows you distinguish scheduling context from address-space ownership.
At a high level, mm_struct tracks:
- the top-level userspace page table root
- the VMA map
- virtual layout metadata like
brk, mmap base, and stack-related bounds - accounting state such as RSS and total virtual memory
- synchronization for map updates and page-table operations
Core transitions:
fork()duplicates the address-space metadata and page tables, with many pages shared initially by COWexecve()throws away the old userspace image and creates a new oneclone(CLONE_VM)shares the samemm_struct
Windows interview material often separates reserved, committed, and resident memory explicitly. Linux uses different API vocabulary, but the same conceptual separation is essential.
| Concept | Linux meaning | What to avoid saying |
|---|---|---|
| Address range exists | A VMA covers a virtual range, created by mmap, brk, the loader, stack growth, shared memory, or another mapping path. |
Do not say every byte has a physical page. |
| Backing is promised or accounted | Anonymous private memory may be charged or allowed according to overcommit policy, MAP_NORESERVE, memcg limits, and mapping type. File-backed clean pages can often be reloaded from the file instead of needing private swap-style backing. |
Do not treat Linux "commit" as identical to Windows MEM_COMMIT. |
| Page is resident | A physical page is currently in RAM and counted in RSS/smaps-style views. |
Do not treat RSS as total allocated or total virtual memory. |
malloc() is a user-space allocator interface, not a kernel allocation primitive. The allocator may satisfy a small allocation from an existing arena that was obtained earlier through brk or mmap. It may satisfy a large allocation with a new anonymous mmap. In both cases, returning a pointer does not prove that all pages are resident. The first write to each untouched page usually triggers a page fault that allocates or maps a physical page.
mmap() creates a VMA, but the backing behavior depends on the flags and file descriptor:
- anonymous private mappings usually allocate physical pages lazily on write faults
- file-backed private mappings can initially share page-cache pages and become anonymous after COW writes
- file-backed shared mappings reflect shared backing-object semantics
MAP_NORESERVEcan weaken swap/commit reservation expectationsmlock/MAP_LOCKEDaffects residency goals, not the existence of the VMA itself
Strong interview answer:
A Linux process can have virtual memory that is mapped but not resident, memory that is accounted or allowed by overcommit policy but not touched, and allocator-returned heap ranges whose physical pages appear only on fault.
malloc,mmap, VMA size, RSS, and commit-style accounting are different views.
A vm_area_struct represents a contiguous virtual range with uniform semantics:
- permissions
- backing type
- fault behavior
- sharing/COW semantics
- special flags like grows-down, locked, or PFN-mapped
Good interviewer sentence:
VMAs describe what should exist and how it should behave; page tables describe what is actually mapped right now.
That distinction is one of the most important in Linux memory management.
You do not need the exact struct layout memorized, but you should know what kinds of state matter:
vm_start,vm_end: address boundsvm_flags: readable, writable, executable, shared, stack-like, special, and related semanticsvm_file: file backing if anyvm_pgoff: backing offset or PFN-oriented meaning for special mappingsvm_ops: VMA-specific operations, especially fault-time behavioranon_vma: ancestry and reverse-mapping support for anonymous/COW memory
Senior nuance:
- VMA permissions and current PTE permissions are not always identical
- a VMA may say "writable" while the current PTEs are temporarily read-only for COW or tracking purposes
Modern kernels store VMAs in a maple tree under the mm_struct.
Why that matters:
- fault lookup is an address-range search problem
- VMA splitting and merging must stay fast
- lockless or RCU-friendly reads matter on hot paths
VMAs are not only used during faults. They are central to:
mmapmunmapmprotectmremapbrkfork- core dumps
- procfs memory inspection
Linux presents a generic multi-level hierarchy:
PGDP4DPUDPMDPTE
Architectures may fold unused levels, but the model remains hierarchical because sparse address spaces are huge and mostly empty.
The CPU does not allocate this hierarchy. On x86/x64, CR3 is just the privileged register that names the active top-level table by physical address. On arm64, TTBR registers play the same conceptual role. The kernel must reserve or allocate physical pages for the top-level table and lower levels, fill valid entries, and then load the root register.
Early boot is special because the normal page allocator does not exist yet. Firmware and the boot loader place the kernel in memory and provide a physical memory map. Linux early boot code uses simple boot-time allocation, reserves pages for initial page tables, builds enough mappings for the kernel image, entry code, direct map or identity mappings as needed by the architecture, then enables paging or switches to the final paging mode. After the MM subsystem initializes memblock, the buddy allocator, and page metadata, later page-table pages come from normal kernel allocation paths.
There is no ordinary user process before this. The initial kernel address space is represented by kernel boot page tables and init_mm-style state. The idle/swapper task runs in that kernel address space. Later, kernel_init/init creation and execve produce PID 1's user address space through normal MM setup. For ordinary processes, Linux allocates an mm_struct, allocates a pgd, initializes architecture-required kernel/shared entries, then faults or maps user pages on demand. Context switching loads the appropriate physical page-table root into CR3 or the architecture equivalent.
The chicken-and-egg answer is: architecture boot code creates the first usable top-level tables before normal process management exists. On x86-64, Linux's generic top-level name is pgd; in 4-level hardware terms that top level corresponds to the PML4, and in 5-level mode the hierarchy grows above it. Early boot code uses statically reserved or boot-allocated physical pages, fills enough entries to reach the kernel image, entry code, early allocator data, and direct/identity mappings required by that phase, then privileged code loads the physical base into CR3. Only after that can the normal virtual-memory, physical-page, and scheduler machinery become ordinary C code.
After MM initialization, new user address spaces are created by kernel code, not hardware. The high-level path is: allocate or initialize an mm_struct; call architecture MM helpers such as pgd_alloc() to allocate the top-level page-table page; initialize the kernel/shared portion from the kernel's master mappings where the architecture requires it; store the resulting kernel virtual pointer in mm->pgd; then populate user entries by fork(), execve(), mmap(), and page-fault handling. The value loaded into CR3 is the physical address derived from mm->pgd, usually combined with architecture bits such as PCID or no-flush flags. So the kernel saves the address-space root in kernel memory as mm_struct state, but the saved field is normally a kernel pointer to the page-table page, not a raw user-visible physical address.
Different creation paths reuse that machinery differently. fork() without CLONE_VM creates a new mm_struct and new top-level root, then shares most data pages initially through COW rather than copying all memory. clone(CLONE_VM) shares the existing mm. execve() builds a replacement address space for the continuing task identity and installs it, then drops the old mm when references are gone. Kernel threads usually do not allocate a user mm; they run with a borrowed active_mm or kernel context because the CPU still needs valid kernel mappings.
Do not reduce this to "exactly one PGD/PML4 per PID." The Linux owner of a user address space is the mm_struct, not the PID number and not an individual task_struct. A multithreaded process has many tasks that usually share one mm_struct because they were created with CLONE_VM; those tasks therefore share the same top-level page-table root for that address space. A fork() without CLONE_VM creates a child with a separate mm_struct and separate root, even though most physical pages may initially be shared through COW. An execve() keeps the task/PID identity but replaces the user address space, so the old mm/page-table hierarchy is dropped and a new one is installed. A zombie or exiting task can remain visible after its mm has been released.
"Live address space" in Linux terms means an mm_struct still has references and its page-table hierarchy has not been torn down. It can belong to tasks that are running, runnable, sleeping, stopped, or frozen; it does not need to be loaded in any CPU's CR3 at that instant. Kernel threads are the opposite case: they normally have task->mm == NULL because they do not own a user address space, but they still need a hardware root while running. Linux handles this with active_mm, borrowing a previous or kernel memory context so the CPU has valid kernel mappings.
With x86 PTI/KPTI, the cardinality changes again. Linux manages a full kernel page-table view and a restricted userspace view used while returning to/running user mode. The userspace view keeps only the minimal kernel mappings needed for safe entry/exit, while kernel entry switches to the fuller kernel copy. Therefore "one mm_struct has one pgd pointer" is a useful source-level starting point, but the hardware may see related user/kernel root variants depending on architecture and mitigation configuration.
Page-table pages that remain part of a live walkable hierarchy must stay resident. That rule applies to every architecture level, not just the final PTE page. In Linux names this means live pgd, p4d, pud, pmd, and pte pages where those levels exist. In x86 PAE mode, CR3 names a page-directory-pointer table (PDPT). In x86-64 4-level paging, CR3 names a page map level 4 table (PML4); with 5-level paging, it names a PML5. x86-64 still has a page-directory-pointer-table/PDPTE level below PML4, followed by page-directory pages and final page-table pages. Linux can encode swapped-out user pages in non-present PTEs and fault those pages back later, but it cannot swap out the paging-structure page the CPU needs in order to read that PTE. If a VMA is unmapped, a process exits, or an empty page-table subtree is torn down, the kernel can free the now-unused page-table pages. That is different from paging a live paging hierarchy to disk.
Five-level x86-64 paging matters because it changes canonical-address and layout assumptions. Four-level x86-64 uses 48-bit canonical linear addresses, while five-level LA57 uses 57-bit canonical linear addresses. For Linux this can expand the sparse user and kernel virtual layout available for ASLR, guard gaps, huge mappings, vmalloc/module/direct-map layout, and debugging or sanitizer reservations. It is not magic security: entropy improves only where the kernel and runtime spend it, and an info leak still collapses layout uncertainty. The bug-hunting value is often in bad assumptions: constants based on the old 47-bit user limit, pointer compression schemes, top-bit masking, address validators, JIT sandboxes, or code that assumes page-table levels are always folded.
Sleeping or unscheduled is not the same as exited. A task can be off-CPU, blocked, frozen, or memory-pressure-trimmed while its mm_struct still exists and may be scheduled again. When no thread using that mm_struct is currently running, its page-table root is not necessarily loaded in any CPU's CR3, but the live root page and walkable paging-structure pages remain allocated in physical RAM. Its data pages can be nonresident, but its live page-table hierarchy must be available for the next hardware walk. On exit, Linux tears down the user VMAs and page tables during address-space cleanup; zombie/task metadata can remain until the parent reaps status, but the user address space is not preserved as a swappable page directory.
A terminal PTE usually contains:
- a PFN
- access-control bits
- status bits
- architecture-specific control bits
Common concepts across architectures:
- present/valid
- writable
- user vs privileged
- executable vs NX
- accessed/young
- dirty
- huge-page or block mapping at higher levels
That list is the hardware-facing layer, not the whole memory-management state. Linux also tracks:
| Layer | Examples | Why it exists |
|---|---|---|
| VMA policy bits | VM_READ, VM_WRITE, VM_EXEC, VM_SHARED, may-permission bits, growth, lock, dump, I/O/PFNMAP, huge-page, soft-dirty style tracking. |
Describes what a virtual range is allowed to do before any PTE exists and tells the fault path how to resolve missing pages. |
| PTE/PMD/etc. bits | Present/valid, PFN, writable, user, NX/XN, accessed/young, dirty, global, cache type, huge/block mappings, swap/nonpresent encodings. | Describes the current translation or software fault state consumed by the MMU or kernel fault code. |
struct page/folio state |
Refcount, mapcount, LRU, dirty/writeback, locked, slab, swap-backed, page-cache identity, pins. | Tracks the physical page independently of any one virtual address so reclaim, COW, migration, DMA, and page cache can work. |
| Backing and reverse-map state | File address_space, inode, page cache, anon_vma/rmap, swap entries, VMA ops. |
Connects page contents to files, anonymous memory, COW ancestry, and other mappings. |
| Cached/lower enforcement | TLB, page-walk caches, I-cache/D-cache coherency, IOMMU tables, KVM EPT/NPT/stage-2 tables. | Hardware and hypervisor/device layers may retain or add translation state beyond the Linux PTE. |
The layers are separate because Linux can create a VMA long before any page is present, can map one physical page into many VMAs, can encode swap or migration in a non-present PTE, and can reclaim or pin a page for reasons unrelated to one virtual address.
Huge pages change the shape of the translation, not just an allocation size. In the ordinary x86-64 4 KB path, the CPU uses indexes from the virtual address to walk PGD/P4D/PUD/PMD/PTE-style levels, then uses the low 12 bits as the page offset. With a 2 MB huge page, the PMD-level entry is the leaf mapping: the CPU does not descend to a final PTE page, and the low 21 address bits are the offset. With a 1 GB huge page, the PUD/PDPTE-level entry is the leaf, and the low 30 bits are the offset. On arm64 the names differ, but the same idea appears as block mappings at higher translation levels.
The TLB caches translations at their page size. A PMD-sized THP or HugeTLB mapping can occupy one large-page TLB translation that covers the whole 2 MB range instead of requiring up to 512 separate 4 KB translations. A 1 GB huge mapping is even larger. On a TLB hit, the CPU does not touch page tables at all; on a miss, the walk stops at the huge PMD/PUD leaf and does not consult lower tables that do not exist for that range. Page-walk caches may still remember upper-level entries, but the main performance win is TLB reach plus skipping lower-level page-table memory reads.
Linux has two major user-visible families:
| Mechanism | How it behaves | Security and debugging consequences |
|---|---|---|
HugeTLB / hugetlbfs / MAP_HUGETLB |
Explicit huge-page pool and huge-page-backed mappings. Pages are usually reserved from a configured pool and cannot be swapped like ordinary anonymous pages. | Strong performance predictability, but coarse granularity, reservation/admin policy, strict alignment, and limited flexibility. Shortage or wrong sizing can fail at mmap or fault time depending on reservation behavior. |
| Transparent Huge Pages (THP) | Kernel opportunistically uses huge mappings for suitable ranges and can fall back to base pages. PMD-sized THP can be split into PTE mappings when an operation needs finer granularity. Modern multi-size THP can use larger folios that may still be PTE-mapped. | Good default performance for large hot regions, but it adds collapse/split races, latency spikes, and forensic ambiguity: a range may change between base pages and huge mappings over time. |
The pros are mostly performance and memory-management overhead: fewer page-table pages, fewer TLB misses, fewer page faults for large regions, and less page-walk pressure. The cons are coarse permissions and bigger failure domains: internal fragmentation, allocation latency, physical-contiguity pressure, more expensive splitting, and less precise 4 KB-level control over mprotect, COW, soft-dirty, page migration, or forensic attribution. When Linux splits a huge mapping into base-page PTEs, it must also invalidate the stale huge-page TLB translation before the new 4 KB permissions and PFNs are authoritative.
From a security perspective, a huge-page leaf is high impact. If a bug or attacker changes RW, NX, user/supervisor, PFN, cache, dirty/accessed, or global-like state on a huge PMD/PUD entry, the effect covers an entire 2 MB or 1 GB range. Conversely, a defender inspecting final PTEs must remember that final PTEs may not exist for a huge-mapped range; the authoritative permission lives at the higher-level leaf. Huge pages can also expose layout and physical-contiguity assumptions that matter for cache side channels, Rowhammer-style physical effects, and JIT/sandbox bounds that assumed 4 KB granularity.
Page tables do not merely store addresses. They define the meaning of memory:
- is it present?
- can userspace touch it?
- can it execute?
- is it large-page mapped?
- is it shared or COW-mediated?
That is why arbitrary writes into page tables can be stronger than many "normal" arbitrary writes into ordinary data objects.
▸ Case Study — Meltdown (CVE-2017-5754)
What broke: Before kernel page-table isolation (KPTI), every process's user-mode page tables contained full read-only mappings of kernel physical memory. The CPU's out-of-order execution engine would speculatively dereference these entries and load kernel data into registers before the permission fault actually fired and discarded the architecturally-visible result.
Primitive gained: Arbitrary kernel memory read from unprivileged userspace. No kernel bug in the traditional sense — the hardware speculated past an access-control enforcement point.
Why it worked: Page-table entries for kernel pages were marked supervisor-only, so a normal architectural access would fault. But out-of-order execution fetched the data transiently before the fault resolved. Although the faulting load's result was suppressed architecturally, the speculative read left a trace in the L1 data cache. A Flush+Reload side-channel loop then recovered the data one byte at a time.
In the wild: Discovered independently by Jann Horn (Project Zero), Cyberus Technology, and the Graz University of Technology teams, disclosed in January 2018. Immediately recognized as a mechanism for breaking kernel ASLR and dumping secrets from kernel memory without any software vulnerability.
What changed: Linux merged KPTI (Kernel Page-Table Isolation), separating user-mode and kernel-mode page tables. Under KPTI, the user-side tables contain only minimal entry/exit stubs — full kernel mappings are absent. Intel, AMD, and ARM also shipped microcode and hardware mitigations. KPTI introduced measurable syscall-path overhead because CR3 must now be switched on every kernel entry and exit.
The TLB is the CPU's translation cache. So a correct MM explanation always includes this sentence:
Updating a page table entry is not enough; the kernel must also deal with stale TLB state.
This matters for:
mprotectmunmap- COW
- permission changes
- page migration
- reclaim
- PTI transitions
Senior-level point:
- a page-table bug is often really a page-table-plus-TLB bug
- stale cached translations can survive after the "real" tables change unless invalidation and ordering are correct
A page fault is not automatically an error. It is often how normal demand paging works.
The CPU faults because of one of a few broad categories:
- missing translation
- protection violation
- invalid/reserved entry
- access type mismatch
The kernel then decides whether this is:
- a legal demand-population event
- a COW event
- a swap-in event
- a backing-object problem
- a genuine invalid access
High-level flow:
- CPU raises a fault exception.
- Architecture entry code saves state and enters kernel mode.
- Kernel identifies the faulting address and access type.
- Architecture-specific fault code, such as an
arch/<arch>/mm/fault.cpath, classifies user/kernel, access type, and special cases. - Kernel stabilizes the relevant VMA metadata through
mmap_lock, VMA lock/RCU lookup, or related locking depending on the path. - Kernel looks up the covering VMA in the
mm_structMaple Tree. - Kernel checks policy and chooses a resolution path.
- Kernel installs or changes PTEs, handles TLB consequences, or signals the task.
Key conceptual dispatcher: handle_mm_fault() and helpers around it.
Current source landmarks:
arch/<arch>/mm/fault.c: architecture page-fault entry and classification.mm/memory.c:handle_mm_fault()and architecture-independent fault resolution.mm/filemap.c: file-backed page-cache faults.mm/rmap.c: reverse mapping, COW/reclaim/migration relationships.mm/mmap.c: VMA creation, splitting, merging, unmapping, and protection changes.
Anonymous memory often begins with no private physical page yet.
Useful subtlety:
- some read faults can map a shared zero page
- write faults allocate and attach a private page
File-backed faults usually resolve from the page cache:
- maybe immediately if cached
- maybe with storage I/O if not
COW is implemented by temporary write-protection plus fault-time copying.
This is the strong answer:
Copy-on-write is sharing implemented through deliberate read-only mappings that become private on the first write fault.
Good rule of thumb:
SIGSEGV: invalid address or invalid permissions for the taskSIGBUS: the mapping exists but the backing object cannot satisfy the access properly
Common errors people make:
- assuming two aliases remain aliases after a write
- forgetting that a private file mapping becomes anonymous after COW
- forgetting that page pins and GUP references complicate replacement
- assuming the page a fault handler produced is definitely the final page userland keeps seeing after later write-protect/COW behavior
Dirty COW-class bugs are interesting because they are failures of invariants under concurrency, not just failures of "copy this buffer correctly."
▸ Case Study — Dirty COW (CVE-2016-5195)
What broke: A race condition in the kernel's
get_user_pages()fault-retry loop allowed a write to reach the original file-backed page rather than the private COW copy the kernel was supposed to have allocated.Primitive gained: Write to any file-backed read-only mapping accessible to the process, including
/etc/passwd,/etc/shadow, and on-disk SUID binaries. No traditional write permission required.Why it worked: When a write fault COWed a page, the kernel could throw away the freshly allocated private page if
madvise(MADV_DONTNEED)raced in from a second thread. On the inevitable retry, the kernel'sget_user_pages()path inadvertently dropped theFOLL_WRITEflag — because the allocation step was already "done" — and returned the original shared page-cache page instead. The subsequent write landed directly on the file-backed page, mutating the on-disk data. Two threads were all that was needed: one callingwrite()through/proc/self/memtargeting the mapping, one in a tightmadvise(MADV_DONTNEED)loop.In the wild: Public PoC appeared within hours of responsible disclosure (October 2016). Actively exploited on Android devices for local root and malicious APK injection, where OS patch delivery was far slower than desktop Linux. Documented exploitation of
/etc/passwdfor root privilege escalation on many unpatched systems.What changed: The
FOLL_WRITEhandling through the COW fault-retry path was reworked to guarantee write intent is never silently dropped on retry. The broader fix also hardened the interaction betweenget_user_pages()flags and concurrent VMA mutations, and was followed by several years of related auditing around GUP semantics and COW correctness.
userfaultfd allows userspace to handle or coordinate faults for registered ranges.
Legitimate uses include:
- post-copy migration
- application-level paging
- controlled memory population
Security relevance:
- it gives attacker-controlled fault timing
- it can become a very precise race-orchestration primitive
- Linux has tightened access because fault handling around kernel-visible state is security-sensitive
▸ Case Study —
userfaultfdas a Kernel Race Primitive (class-level pattern)What broke: Nothing breaks in
userfaultfditself — the mechanism works as designed. The problem is that any kernel code path touching user memory can be indefinitely stalled at a fault boundary while an attacker's registered fault handler runs in userspace.Primitive gained: Controlled interleaving of kernel execution. The attacker can pause the kernel mid-operation — after one object has been modified but before a second one is updated — then inspect or manipulate kernel state before allowing the fault to complete. This converts time-of-check/time-of-use races that would ordinarily require millions of attempts into near-deterministic one-shot windows.
Why it worked: Kernel copy-from/to-user paths, page-population paths, and certain read/write syscall handlers all block at a page fault and yield execution to the userspace fault handler. From the fault handler's perspective, the kernel thread is frozen at a precise instruction boundary, making multi-step race conditions trivially reproducible.
In the wild: Used as a stabilization primitive across a wide range of Linux local privilege escalation chains targeting pipe, io_uring, and slab-allocation races from roughly 2016 onward. The ability to freeze kernel execution at a fault dramatically raised the reliability of exploits that would otherwise require spray-and-pray timing.
What changed: Linux progressively restricted unprivileged
userfaultfduse. Since kernel 5.11,userfaultfdrequires eitherCAP_SYS_PTRACEor thevm.unprivileged_userfaultfdsysctl set to 1 (off by default on most distributions from 2021 onward). The restriction specifically targets the "pause arbitrary kernel code mid-race" capability.
If you need a crisp summary in an interview:
Linux memory management is a layered system.
mm_structowns the address space, VMAs define allowed ranges and backing semantics, page tables define current translations, the TLB caches them, and faults are how the kernel lazily turns policy into actual mappings. Most deep MM bugs happen when one of those layers changes state but another layer still believes the old state is true.