Complete memory layout of threads and processes: tasks, stacks, page tables and VMA
I have talked about it separately in previous articles. VMA, page table, physical page, zone, buddy and slab. Now bring the perspective back to a real process:
When there are multiple threads in a process, which memory structures are shared and which are independent for each thread?
This matter is easy to confuse, because the real scheduling unit of the Linux kernel is not the "abstract process in traditional textbooks", but the task. A process in user mode usually has multiple tasks sharing the same address space; a thread in user mode also has its own thread in the kernel. task_struct.
First give the core picture of this article. It is divided into two layers: the upper part is "what each thread owns", and the lower part is "the user address space that these threads jointly point to".
process P: three user threads
per thread/task I have it myself:
task_struct T1 task_struct T2 task_struct T3
├─ kernel stack 1 ├─ kernel stack 2 ├─ kernel stack 3
├─ user SP -> stack T1 ├─ user SP -> stack T2 ├─ user SP -> stack T3
├─ thread / thread_info ├─ thread / thread_info ├─ thread / thread_info
└─ mm ───────────────┐ └─ mm ───────────────┐ └─ mm ───────────────┐
│ │ │
└──────────► same mm_struct ◄─────────────────────────┘
│
├─ pgd: Same user page table root
├─ VMA tree: The same set of legal address ranges
├─ mmap / heap / stack boundary
└─ rss / total_vm / locked_vm ...
same mm_struct The complete set of user virtual address spaces described:
high address
┌─────────────────────────────────────────────────────────────┐
│ user stack T1 ← T1 The user stack pointer usually falls here │
├─────────────────────────────────────────────────────────────┤
│ user stack T2 ← T2 The user stack pointer usually falls here │
├─────────────────────────────────────────────────────────────┤
│ user stack T3 ← T3 The user stack pointer usually falls here │
├─────────────────────────────────────────────────────────────┤
│ mmap Area: anonymous mapping, file mapping, dynamic library │
├─────────────────────────────────────────────────────────────┤
│ heap │
├─────────────────────────────────────────────────────────────┤
│ code / data / bss │
└─────────────────────────────────────────────────────────────┘
low address
Concluding in one sentence:
Each thread has its own task_struct, user stack and kernel stack; ordinary threads in the same process share the same mm_struct, thus sharing the VMA, page tables, and user address space.
The "same user address space" here does not refer to an additional shared memory, nor does it mean that all threads share the same stack. It refers to: these threads task_struct->mm point to the same mm_struct, so they face the same set of user-mode virtual address ranges, the same VMA tree, and the same user page table root. The user stack of a thread is just a different VMA interval in this shared address space. The user stack pointer of each thread usually falls in its own stack interval.
1. Linux schedules tasks
In the Linux kernel,task_struct Is the core descriptor of the scheduling entity. A single-threaded process has a task_struct;A multi-threaded process has multiple task_struct.
each task_struct Each has its own scheduling state, register context, kernel stack, signal-related state and thread private information. When thread switching, the scheduler switches the task running on the current CPU.
User mode can be accessed from /proc/<pid>/task/ See this:
/proc/<pid>/task/
├─ <tid1>/
├─ <tid2>/
└─ <tid3>/
here every <tid> All correspond to a kernel task. The tid of the main thread of a process is usually equal to the pid, and other threads have their own tids.
So don't write the relationship as:
a process -> one task_struct -> multiple threads
A closer Linux statement is:
a thread group -> multiple task_struct
│
└─ These task usually share the same mm_struct
two,task_struct->mm Points to user address space
mm_struct It is the general entrance to the user address space. I have seen it repeatedly in the previous articles:
mm_struct
├─ pgd Top-level page table pointer
├─ mmap / mm_mt VMA Management structure, commonly used in modern kernels maple tree
├─ start_code / end_code
├─ start_data / end_data
├─ start_brk / brk
├─ start_stack
├─ arg_start / arg_end
├─ env_start / env_end
└─ total_vm / locked_vm / rss Related statistics
passed within the same process pthread_create() Ordinary threads created, usually through clone() bring CLONE_VM, thus sharing the same mm_struct.
shared mm_struct The direct meaning is:
- They see the same set of user virtual address spaces.
- They see the same VMA tree.
- They use the same page table root, which is the same set of user-mode page table mappings.
- a thread
mmap()The address that comes out can also be accessed by other threads. - a thread
munmap()ormprotect()If the address space is changed, other threads will be affected.
can put mm_struct Understood as "the user address space object commonly used by this thread group".
3. Sharing address space does not mean sharing all stacks
Threads share the user address space, but each thread still has its own user stack.
This sentence seems contradictory, but it is not contradictory:The user stack mentioned here is also a certain VMA in the same address space, but each thread agrees to use a different stack area.
same one mm_struct Managed user address space
high address
┌──────────────────────────────┐
│ user stack T1 │ T1 Function calls, local variables, return addresses
├──────────────────────────────┤
│ guard page / unmapped gap │
├──────────────────────────────┤
│ user stack T2 │ T2 Function calls, local variables, return addresses
├──────────────────────────────┤
│ guard page / unmapped gap │
├──────────────────────────────┤
│ user stack T3 │
├──────────────────────────────┤
│ mmap district / heap / code ... │
└──────────────────────────────┘
low address
These stack intervals are all in the same mm_struct , so from the perspective of "address space visibility", one thread can theoretically read and write the user stack address of another thread. Just from a program semantics and thread safety perspective, this is usually a disaster.
So a more accurate expression is:
User address space: thread sharing
user stack VMA: Belong to different intervals in the shared address space
Current stack pointer: independent for each thread
4. Each thread also has an independent kernel stack.
The user stack is in the user address space. Meaning: The user stack is a user-mode virtual address that can be /proc/<pid>/maps You can see the corresponding VMA here, and user mode code can read and write it using ordinary instructions.
The kernel stack is not in user address space. Meaning: It is not a certain address in the user-mode VMA tree of this process. User-mode code cannot use a pointer to directly read and write its own kernel stack, nor will it /proc/<pid>/maps see it here. The kernel stack is located in the kernel address space and will only be used when the CPU has entered the kernel state and executed with kernel permissions.
How does a thread "access" its own kernel stack?
To be more precise, it is not that the user mode thread actively accesses the kernel stack, but that after the task enters the kernel mode, the CPU and kernel switch the current stack to the kernel stack of the task. After the switch, the kernel code is executed; the function calls, local variables, and saved register context of the kernel code fall on this kernel stack.
When a thread executes in user mode, the CPU uses the thread's user stack. After a system call, interrupt or exception occurs, the CPU enters the kernel state, and the kernel will switch to the kernel stack corresponding to this task for execution.
thread T1 Execute in user mode
│
│ use T1 user stack
▼
syscall / interrupt / exception
│
▼
Enter kernel mode
│
│ Cut to T1 kernel stack
▼
Kernel code passes current turn up T1 of task_struct
│
▼
When you need to access the user address space, pass T1->mm turn up mm_struct
4.1 task->stack Points to the kernel stack
The kernel stack is not found in a vacuum.task_struct It is a kernel object in itself, and the kernel can of course save the kernel address in it. For each task, the kernel will task_struct Record its kernel stack location. In modern mainline kernels it can be simplified to:
task_struct
├─ stack -> this task kernel stack
├─ stack_vm_area -> CONFIG_VMAP_STACK record vmalloc stack area
├─ thread -> Architecture related registers/context state
└─ mm -> user address space
│
└─ VMA tree -> The user state thread stack is a certain user segment here VMA
The most common misunderstanding here is that thread stack the name. There are places in the Linux source code that contain task->stack It's called thread stack because it is "a stack for each kernel task/thread". This term is from the perspective of the kernel scheduling entity, not the user stack of pthread.
In order to avoid ambiguity, this article will be called as follows:
task->stack / thread stack -> kernel stack
pthread user stack -> A stack in the user address space VMA
When a new task is created, the kernel allocates two types of things:
1. task_struct Ontology
└─ usually from task_struct dedicated slab cache
2. this task kernel stack
├─ CONFIG_VMAP_STACK=y: usually via vmalloc/vmap distributed and possibly reused per-CPU cached stack
└─ CONFIG_VMAP_STACK=n: Usually contiguous pages are allocated directly as the kernel stack
After the allocation is completed, the kernel records the kernel stack address to task_struct->stack. After that, the kernel needs to find the kernel stack of a certain task, and the process is similar to task_stack_page(task) The essence of such a helper is to start from task->stack Remove the stack address.
The actual action of cutting to this stack is architecture-related. Taking x86-64 as an example, when entering the kernel state from user mode, the CPU will use the kernel stack pointer prepared by the kernel for the current task; when the schedule switches to another task, the kernel will also switch to the kernel stack and register context saved by the other task. After entering the kernel state, ordinary C function calls naturally use the kernel stack pointed to by the current CPU stack pointer.
4.2 The user stack is jointly characterized by VMA and user SP
Do not use the user stack from task->stack try to find. The user stack is first a user virtual address range, consisting of mm_struct VMA description below. When the thread is running in user mode, the top of the current user stack is in the user stack pointer register of the CPU.
Therefore, "the user stack is in task_struct "Is there only one SP pointer?" is not accurate enough. A more accurate statement is:
User stack address range: mm_struct -> VMA tree a certain stack in VMA
Current user stack top: CPU User stack pointer register, e.g. x86-64 user mode RSP
Save location after entering the kernel: architecture-related trap scene, such as on the kernel stack pt_regs
There is no universal cross-architecture task_struct->user_sp The field long represents the user stack. When running in user mode, the user SP is the CPU register value. After a system call, interrupt or exception occurs, the user mode register state will be saved according to architectural rules; when the kernel returns to user mode, the user SP will be restored from the saved register state.
Whether the user SP is legal or not does not depend on task_struct A certain field in it is guaranteed in advance. CPU executes user mode push, call, ret, or when the program directly reads and writes the stack address, it essentially accesses a user virtual address. Whether this address is legal or not, VMA and page table are still used:
User mode SP/RSP visit an address
│
▼
Page table translation
│
├─ present and permissions allow -> Access successful
└─ not present / Permission not allowed
│
▼
page fault
│
├─ VMA Legal, may be a growable stack or an unallocated page -> Handling missing pages
└─ not found VMA or permissions mismatch -> SIGSEGV
In other words, user mode can completely change the stack pointer to an incorrect address; the kernel will not assume that it is legal just because "this is the stack pointer". When the program actually uses this SP to access the memory, or when the kernel needs to put a signal frame on the user stack, it will use the VMA, page table and copy_to_user / access_ok This type of check determines whether it can be accessed; if it cannot be accessed, it will fail. The common result is SIGSEGV.
The most common mistake here is "a process has a kernel stack". For Linux threads, more accurately:
Each task has its own kernel stack. There are multiple tasks in a multi-threaded process, so there are multiple kernel stacks.
The kernel stack is very small and can usually only hold kernel state call chains, a small number of local variables and saved context. It is not intended for user programs to recurse, allocate large arrays, or carry malloc() For data.
five,thread_info The location depends on the architecture and version
Many old materials will be drawn like this:
low address
┌─────────────┐
│ thread_info │
├─────────────┤
│ kernel stack│
│ ... │
└─────────────┘
high address
This picture is very common in the context of some old kernels and old architectures:thread_info Place it at the bottom of the kernel stack, and you can quickly find the low-level information of the current task by masking the stack pointer.
But the same cannot be said for modern kernels. Many architectures and configurations have been thread_info Embed task_struct, or use a different current How to get it. Motivations for this include reducing kernel stack overflow damage thread_info risks, as well as adapting to more complex stack layouts and security features.
So this article adopts a more stable abstraction:
task_struct
├─ thread_info / thread: Architecture-dependent low-level thread state
├─ stack: point to this task kernel stack
└─ mm: Points to user address space
When understanding the main line, there is no need to thread_info Fixed drawing stuck at the bottom of the kernel stack. What should be remembered is that it belongs to the low-level execution state of each task, and the specific location is related to the architecture, kernel version, and configuration.
6. The page table root is mm_struct inside
As I said before when talking about page tables,mm_struct->pgd Points to the top-level page table. Thread sharing mm_struct, which means they share the same user address space page table root.
task_struct T1 task_struct T2
│ │
└──────► same mm_struct ◄──────┘
│
▼
mm->pgd
│
▼
top page table -> ... -> PTE -> physical page
When the CPU switches from one process to another, if the two tasks mm_struct If different, you need to switch the hardware page table root. Taking x86-64 as an example, user address space switching will be reflected in changes in the page table root pointed to by CR3.
But when switching between two threads within the same process, they usually share mm_struct, the user page table root does not need to be switched due to different address spaces. The scheduler still has to switch register contexts, kernel stacks, thread states, etc., but the user address space itself is the same.
The above process corresponds to this ASCII diagram:
Scheduler selection next task
│
▼
next->mm == prev->mm ?
│
├─ Yes: In the same user address space, the page table root usually does not need to be cut.
│
└─ No: Different user address space, switch page table root
│
▼
Switch register context / kernel stack / Thread status
│
▼
run next task
7. Typical kernel threads do not have their own ordinary users mm
Not all tasks represent user threads in a user process. A typical kernel thread only executes kernel code and does not return to user mode to run user programs. Therefore, there is no common user address space such as user code, user stack, and user VMA.
For this type of kernel thread,task_struct->mm Usually NULL. What this sentence means is: it does not have "its own set of user address space objects" and cannot pass through it like an ordinary user thread. task->mm Find your own VMA tree, user page table root, and user stack range.
but task->mm == NULL It does not mean that the CPU can run "without page tables". The CPU still needs a valid page table root when executing instructions, because the kernel virtual addresses of kernel code, kernel data, and kernel stack also need to be translated by the page table. Taking common x86-64 Linux as an example, the page table of each user process not only has its own user mapping, but also a set of shared kernel address space mappings. When the CPU is running in kernel mode, it relies on these kernel maps to access kernel code and data.
Therefore, although the kernel thread does not have its own user address space, the runtime still needs a "currently used address space context". this is active_mm role.
can be simplified to:
Ordinary user thread:
task->mm -> own process mm_struct
task->active_mm -> This is usually the same mm_struct
Typical kernel thread:
task->mm -> NULL
task->active_mm -> Borrowed address space context
"Borrowing" here is easy to misunderstand. It does not mean that the kernel thread becomes part of that user process, nor does it mean that the kernel thread owns the user memory of that process. It borrows from Page table context currently available to the CPU, mainly to continue using the kernel mapping in it.
It can be understood like this:
user process A Running
│
│ CPU The current page table root points to A->mm
▼
Schedule to kernel thread K
│
│ K->mm == NULL, Does not have its own user address space
│ but CPU Page tables are still needed to translate kernel addresses
▼
K->active_mm You can continue to quote A->mm
│
│ K Using the kernel map shared in
│ K don't own it A User address space semantics of
▼
Schedule to another user process B
│
▼
Then switch to the B->mm page table root
This also has a practical benefit: the kernel thread does not have to construct a complete set of user address space separately for "I only run kernel code", and it can also reduce unnecessary page table root switching. The kernel handles this behavior together with mechanisms such as lazy TLB.
So "can I borrow it?" Yes, because this is a state explicitly maintained by kernel scheduling and memory management, and is not privately shared by user-mode programs. The kernel will manage active_mm Reference counting and switching rules. The key boundaries are:
task->mm == NULL
-> this task Does not have its own normal user address space
task->active_mm != NULL
-> CPU The page table context to use when running it
As long as you remember this difference, you will not active_mm Misread as "kernel threads share user memory of a process". If the kernel thread really needs to access user memory on behalf of a certain process, it will follow special interface and context binding rules, not because active_mm After borrowing a certain page table, you can use the user's address as your own address.
So in this section "Kernel threads have no ordinary users. mm" is talking about the common status of a typical kernel thread:task->mm == NULL,but task->active_mm Still records the page table context that the CPU used to run it. Don't interpret this to mean "kernel threads don't have address translation", nor do you mean active_mm Understood as "it owns the address space of a user process".
This explains why when talking about process/thread memory layout, it is necessary to separate "user threads" and "kernel threads". This article mainly discusses ordinary user threads in user processes.
8. How to string together these structures in a system call
Put the dynamic paths together and look at them. Assume that thread T1 calls read():
T1 User mode
│
│ User code runs in T1 user stack superior
▼
syscall
│
▼
CPU Enter the kernel state and switch to T1 kernel stack
│
▼
Kernel passed current turn up T1 task_struct
│
├─ Look at scheduling, signals, permissions, etc. task level status
│
└─ pass current->mm turn up mm_struct
│
├─ check VMA: user buffer Is it a legal address?
├─ Lookup page table: whether the user page has been mapped
└─ Trigger page faults, allocation pages, and creation when necessary PTE
│
▼
System call returns
│
▼
CPU Return to user mode and continue using T1 user stack
In this path:
task_structAnswer "Which thread is currently running?"- The kernel stack carries this kernel mode execution.
mm_structAnswer "What is the user address space this thread belongs to".- VMA answers "Is this user address legal and permissions allowed?"
- The page table answers "Is this virtual page currently mapped to a physical page?"
8.1 current How to find the current task
It says "Kernel passed current Find T1's task_struct", here's current It is not a variable that can be seen in user mode, nor is it passed in from the system call parameters. It is an architecture-related macro or inline function in the kernel, which means:
current -> current CPU running task_struct
How to implement it depends on the architecture. Taking modern x86-64 Linux as an example,current It's roughly a chain like this:
current
│
▼
get_current()
│
▼
Read current CPU of per-CPU variable current_task
│
▼
Get the currently running task_struct *
That is to say,current depends on per-CPU current task pointer. Each CPU has its own current_task, indicating which task this CPU is executing at the moment.
"Where" this per-CPU pointer is on x86-64 can be seen in conjunction with the fifth article in the previous page series:GS and swapgs in the kernel, and modern TSS. As mentioned in that article, x86-64 Linux will use kernel mode GS to point to the per-CPU data area of the current CPU after entering the kernel:
Kernel state GS.base -> current CPU of per-CPU data area
│
├─ current_task
├─ Scheduler run queue
├─ interrupt/Preemption count
└─ Every CPU Statistics
So on x86-64,current It can be understood as: first locate the per-CPU area of the current CPU through GS base, and then read out the per-CPU area of this CPU from the fixed offset. current_task. It is not a field hanging under a process object, but each CPU's own copy of kernel data.
That current_task How did the value come from? It is updated when the schedule switches. After the scheduler selects the next task to run, the architecture-related context switching code will change the current CPU's current_task Change to this next task.
can be simplified to:
Scheduler selection next task
│
▼
Switch address space / register / Kernel stack and other contexts
│
▼
current CPU of current_task = next
│
▼
After this CPU kernel code execution on current
│
▼
What you get is next of task_struct
so current Not "the current thread found out from some process structure", but a quick entry into the kernel of the fact "who is currently running on the CPU". After the system call enters the kernel, the CPU is already executing in the kernel context of this task; at this time, reading current, naturally get the task that initiated this system call.
There is another common saying in old information: through the bottom of the current kernel stack thread_info turn up current. This is true on some old kernels or architectures, but modern kernels have many architectures changed to per-CPU current_task or other implementation. Therefore, this article only retains abstract conclusions:
current It is a quick entry related to architecture.
The function is to get the current CPU running task_struct
Its value is used when the schedule switches to a certain task updated when
9. Experiment: Observe threads, user stacks and shared address space
The following program does four things:
- Create 3 pthread threads.
- from
/proc/self/task/Print the task tid in the current process. - Each thread prints its tid, user stack range, and the address of a local variable.
- Each thread reads the same anonymous block
mmapmap, and calculate what you see/proc/self/task/<tid>/mapssummary.
If multiple threads share the same mm_struct, then what they see maps The contents should be consistent; if each thread has its own user stack, their stack range and local variable addresses should be different.
The complete code is as follows:
#define _GNU_SOURCE
#include <dirent.h>
#include <errno.h>
#include <inttypes.h>
#include <pthread.h>
#include <stdatomic.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#include <sys/syscall.h>
#include <sys/utsname.h>
#include <unistd.h>
#define THREADS 3
struct maps_summary {
unsigned long lines;
uint64_t hash;
};
struct thread_report {
pid_t tid;
unsigned long pthread_value;
void *stack_base;
size_t stack_size;
void *local_addr;
struct maps_summary maps;
int shared_value;
};
static pthread_barrier_t barrier;
static struct thread_report reports[THREADS];
static _Atomic int *shared_page;
static pid_t gettid_linux(void)
{
return (pid_t)syscall(SYS_gettid);
}
static struct maps_summary summarize_maps(pid_t tid)
{
char path[128];
char buf[4096];
struct maps_summary result = {0, 1469598103934665603ULL};
snprintf(path, sizeof(path), "/proc/self/task/%ld/maps", (long)tid);
FILE *fp = fopen(path, "r");
if (!fp) {
fprintf(stderr, "fopen %s: %s\n", path, strerror(errno));
exit(1);
}
while (fgets(buf, sizeof(buf), fp)) {
result.lines++;
for (unsigned char *p = (unsigned char *)buf; *p; p++) {
result.hash ^= (uint64_t)*p;
result.hash *= 1099511628211ULL;
}
}
fclose(fp);
return result;
}
static void print_task_list(void)
{
DIR *dir = opendir("/proc/self/task");
struct dirent *de;
if (!dir) {
perror("opendir /proc/self/task");
exit(1);
}
printf("task tids:");
while ((de = readdir(dir)) != NULL) {
if (de->d_name[0] >= '0' && de->d_name[0] <= '9')
printf(" %s", de->d_name);
}
printf("\n");
closedir(dir);
}
static void fill_report(int index)
{
pthread_attr_t attr;
int local = index;
if (pthread_getattr_np(pthread_self(), &attr) != 0) {
perror("pthread_getattr_np");
exit(1);
}
if (pthread_attr_getstack(&attr, &reports[index].stack_base,
&reports[index].stack_size) != 0) {
perror("pthread_attr_getstack");
exit(1);
}
pthread_attr_destroy(&attr);
reports[index].tid = gettid_linux();
reports[index].pthread_value = (unsigned long)pthread_self();
reports[index].local_addr = &local;
pthread_barrier_wait(&barrier);
if (index == 0)
atomic_store(shared_page, 12345);
pthread_barrier_wait(&barrier);
reports[index].shared_value = atomic_load(shared_page);
reports[index].maps = summarize_maps(reports[index].tid);
pthread_barrier_wait(&barrier);
}
static void *thread_main(void *arg)
{
fill_report((int)(intptr_t)arg);
sleep(1);
return NULL;
}
int main(void)
{
pthread_t threads[THREADS];
struct utsname uts;
pid_t pid = getpid();
size_t page_size = (size_t)sysconf(_SC_PAGESIZE);
if (uname(&uts) != 0) {
perror("uname");
return 1;
}
shared_page = mmap(NULL, page_size, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (shared_page == MAP_FAILED) {
perror("mmap");
return 1;
}
atomic_store(shared_page, 0);
if (pthread_barrier_init(&barrier, NULL, THREADS) != 0) {
perror("pthread_barrier_init");
return 1;
}
printf("machine=%s sysname=%s release=%s\n",
uts.machine, uts.sysname, uts.release);
printf("pid=%ld page_size=%zu shared_mapping=%p\n",
(long)pid, page_size, (void *)shared_page);
for (int i = 0; i < THREADS; i++) {
int err = pthread_create(&threads[i], NULL,
thread_main, (void *)(intptr_t)i);
if (err) {
fprintf(stderr, "pthread_create: %s\n", strerror(err));
return 1;
}
}
usleep(100000);
print_task_list();
for (int i = 0; i < THREADS; i++) {
int err = pthread_join(threads[i], NULL);
if (err) {
fprintf(stderr, "pthread_join: %s\n", strerror(err));
return 1;
}
}
printf("\nthread reports:\n");
for (int i = 0; i < THREADS; i++) {
printf("thread[%d] tid=%ld pthread=0x%lx "
"stack=[%p, %p) stack_size=%zu local=%p "
"shared_value=%d maps_lines=%lu maps_hash=0x%016" PRIx64 "\n",
i,
(long)reports[i].tid,
reports[i].pthread_value,
reports[i].stack_base,
(char *)reports[i].stack_base + reports[i].stack_size,
reports[i].stack_size,
reports[i].local_addr,
reports[i].shared_value,
reports[i].maps.lines,
reports[i].maps.hash);
}
pthread_barrier_destroy(&barrier);
munmap((void *)shared_page, page_size);
return 0;
}
Compile and run in x86_64 container:
docker run --rm --platform linux/amd64 \
-v /tmp:/tmp -w /tmp \
gcc:13 \
bash -lc 'gcc -O2 -Wall -Wextra -pthread /tmp/thread-layout-demo.c -o /tmp/thread-layout-demo && /tmp/thread-layout-demo'
A real output I ran in an x86_64 container:
machine=x86_64 sysname=Linux release=6.12.65-linuxkit
pid=1 page_size=4096 shared_mapping=0x7fffff7c9000
task tids: 1 14 15 16
thread reports:
thread[0] tid=14 pthread=0x7fffff5dc6c0 stack=[0x7ffffeddd000, 0x7fffff5dd000) stack_size=8388608 local=0x7fffff5dbe6c shared_value=12345 maps_lines=58 maps_hash=0x772fbe087b6341ca
thread[1] tid=15 pthread=0x7ffffeddb6c0 stack=[0x7ffffe5dc000, 0x7ffffeddc000) stack_size=8388608 local=0x7ffffeddae6c shared_value=12345 maps_lines=58 maps_hash=0x772fbe087b6341ca
thread[2] tid=16 pthread=0x7ffffe5da6c0 stack=[0x7ffffdddb000, 0x7ffffe5db000) stack_size=8388608 local=0x7ffffe5d9e6c shared_value=12345 maps_lines=58 maps_hash=0x772fbe087b6341ca
This set of results can directly correspond to the structure diagram of this article.
First, the program does run in an x86_64 container:
machine=x86_64
second,/proc/self/task/ You can see multiple tasks:
task tids: 1 14 15 16
here 1 is the main thread pid/tid in the container,14, 15, 16 are the tids of the three pthread worker threads. They belong to the same process, but are independent tasks in the kernel.
Third, the user stack ranges of the three threads are different:
thread[0] stack=[0x7ffffeddd000, 0x7fffff5dd000)
thread[1] stack=[0x7ffffe5dc000, 0x7ffffeddc000)
thread[2] stack=[0x7ffffdddb000, 0x7ffffe5db000)
The local variable address of each thread also falls in its own stack interval:
thread[0] local=0x7fffff5dbe6c
thread[1] local=0x7ffffeddae6c
thread[2] local=0x7ffffe5d9e6c
Fourth, all three threads read the value in the same anonymous map:
shared_mapping=0x7fffff7c9000
shared_value=12345
This shows that this mmap The user virtual address that comes out is located in the shared user address space. After one thread writes, other threads can see it.
Fifth, what three threads saw /proc/self/task/<tid>/maps The summary is exactly the same:
maps_lines=58 maps_hash=0x772fbe087b6341ca
This is not what proves them task_struct are the same, but indicate that they are shared by mm_struct See the same VMA layout.
10. Nothing can be seen from this experiment
User mode experiments can observe user stack, tid,maps, shared mapping, but don't see many kernel internal addresses.
Especially:
- Can't see
task_structThe real kernel virtual address. - The real kernel stack address of each task cannot be seen.
- Can't see
mm_structpointer value. - Can't see
mm->pgdreal address.
This is normal. Modern kernels do not casually expose these addresses to user mode for reasons including security and kernel address randomization.
Therefore, the experiments in this article are only used to verify observable phenomena in user mode:
multiple tid -> multiple task
Different user stacks -> Independent user execution stack per thread
same maps summary -> Share the same user address space layout
shared mmap value -> same one mm_struct The mapping under is visible to all threads
The internal relationship of the kernel must be understood in conjunction with the source code structure:
task_struct -> mm_struct -> VMA tree / pgd
task_struct -> kernel stack
task_struct -> thread / thread_info
11. Several points that are easily confused
First,Threads share address space, which does not mean there is only one stack.
Each thread has its own user stack area. It’s just that these stack areas all belong to the same process address space and are controlled by the same mm_struct Tube.
second,User stack and kernel stack are not the same thing.
The user stack carries user mode function calls. The kernel stack carries the call chain after this task enters the kernel. Each thread has its own kernel stack, and we cannot say "one kernel stack for one process".
third,task_struct and mm_struct Not a one-to-one relationship.
In a single-threaded process it usually looks like one-to-one. In a multi-threaded process, there are multiple task_struct point to the same mm_struct. A typical kernel thread does not have its own ordinary user mm:its task->mm Usually NULL, when running, rely on active_mm Saves the currently used page table context.
fourth,VMA is a structure of the process address space, not a thread-private structure..
One thread in the same process mmap(), munmap(), mprotect() What is changed is the shared VMA layout, and other threads will see the same changes.
fifth,thread_info The pictures cannot be copied from old information..
The old picture puts it at the bottom of the kernel stack, but this may not be the case with modern kernels and different architectures. When talking about the concept, it should be emphasized that it is the low-level thread state of the task, and the specific location depends on the architecture and configuration.
12. Continuing with the structure of the previous articles
Now put the entire picture back into the main line of the series:
CPU A thread is running
│
▼
current task_struct
│
├─ kernel stack: The stack after the current thread enters the kernel
├─ thread / thread_info: Architecture-related thread status
└─ mm
│
▼
mm_struct
│
├─ VMA tree: Is the address legal and what are the permissions?
├─ pgd: Page table root
└─ Address space statistics and boundaries
│
▼
page fault / mmap / brk Waiting path
│
▼
buddy Allocate user pages, page table pages
slab distribute task_struct, vm_area_struct Wait for kernel small objects
This article answers "How are these structures organized in a process/thread?"Next articleWill wrap up the entire series along the dynamic path: once malloc, a page fault, and a kernel object allocation, which structures will be encountered respectively.
References
- Linux kernel documentation: The /proc Filesystem
- Linux kernel documentation: The Linux kernel user-space API guide
- Linux source:
kernel/fork.c - Linux source:
include/linux/sched/task_stack.h - Linux source:
arch/x86/include/asm/current.h - Linux source:
arch/x86/kernel/process_64.c - Linux man-pages: clone(2)
- Linux man-pages: proc(5)