Processes (Transcript of the podcast)

Introduction

In this episode we’ll tackle a topic that joins many parts of the systems and so is hard to fully cover. It has a relationship with everything in the system, it glues it together.
We’re going to be discussing processes on Unix.

What is a process

At the same time?

What uses does a computer have if it doesn’t run any programs? What uses does it have if it only runs a single program?

We can run many different sorts of programs at “the same time” on a Unix-like operating system.
Isn’t that convenient?
But what does it mean to run programs at “the same time”?

Let’s take the opposite, what does it mean to run a single program instance on a machine. That would mean this program would be holistic, it would do everything by itself and could have control of the machine’s entire resources, memory and processors during its lifetime. The switch from one program to another would mean the end of existence of the first program and it would be the role of that program to load the next one in the queue.

If you’ve read the book Flatland this is metaphorically equivalent to the one dimensional being. But we live in the sixth dimension now, things are more complex and subtle.

To run multiple instances of programs we need to be able to split the resources amongst them. We don’t have one physical CPU and RAM for every new process we spawn, though this could be interesting to think about, and so we share spaces in memory, and processor running time.
And that is the problem to tackle: How to run more processes than there are cpus, how to provide the illusion of endless processors and memories.
There are many more or less satisfactory solutions to this problem.

The terms we use to describe those are:

Multiprogramming
Multiprocessing
Multitasking
Timesharing
Multithreading

Let’s talk grossly about what they mean.
Multiprogramming is the case of a single program holding the processor resource at a time, while other programs are loaded in the memory and waiting for the first one to finish executing its instructions or to let go of the processor, be it because it’s doing some I/O or because it’s completed. Then the processor context-switch to that other task.

This is a bit different than having a single program on a machine, as you can imagine, because multiple programs can be loaded in memory.

Multiprocessing is a generic term that refers to the concept of having many hardware processors and having tasks run in parallel on different CPUs aka “at the same time”. Having a new processor for every new program that wants to run would be a case of multiprocessing.

Multitasking is similar to multiprogramming, as in the processor executes only one task at a time, but it adds to it that the task can be interrupted and the CPU can be reassigned/context-switched to another one, which gives the illusion of parallelism, but is more precisely referred to as concurrent tasks. It also goes further, the tasks don’t have to be the whole program they could be sub-parts of it, defined as threads of execution.

Now about timesharing, both multiprogramming and multitasking are timesharing, as in programs share CPU time or resource time. In multiprogramming one program as a whole keeps running until it blocks and in multitasking each running program takes only a fair quantum or slice of the CPU time.

Lastly, multithreading is about having threads, sub-parts of the same program, being ran concurrently. The context-switching in this case is lighter as it doesn’t have to switch virtual memory address space because it’s within the same program. It only has to switch processor state from one thread to another which is efficient.

Ok, that’s a lot of concepts to grasp but this gives us an overview of the concepts. And for those to work we have to define our main actor here: The task, the running program, the unit that will use that CPU time, the process.

We kind of understand why we need them but what are they?

What are they?

To get anything done on a Unix system you need processes. The running programs or computing tasks are running inside them.
Programs themselves are a set of passive machine code, instructions and data stored in an executable on the disk, it’s a non-changing and static entity.

The operating system takes that executable and forms something useful from it. When the program gets active, dynamic, alive, it becomes a process. It’s the state of the program while being executed by the OS, it’s the program instructions in-action. That also means that the same instructions can be loaded in different processes and that program instructions within the same processes can be changed.

The instance of the program is loaded in memory and the instructions finally get to be executed by the processor. But the process is more than that. First and foremost it’s an operating system abstraction.

A process is an independent container or bundle for a program running. Inside of it you can find the program execution, its state, the metadata that describes it, environment. The container also has input and output and can send messages to other containers.

This specific way of seing processes makes more sense when you think about them as owner of resources to which their running program currently needs.

That means that the current executing process and activities are separated from other processes. One process crashing shouldn’t make another one crash (in theory). Processes shouldn’t be able to communicate with the rest of the running ones except through certain specific kernel mechanisms.

The process entity has a lifetime, a start at the fork() system call and end at the exit(). It may execute different program instructions over its lifetime. It has its own address space and control points, its own state, it has its own execution environment, and is the unit used for scheduling, managing what holds which resources at one time.

This is the abstraction that a process provides: A processing unit.

Viewing it in another way: processes are tasks, which means they take up time, which is in opposition to taking up space.

We’ll discuss their history, their memory representation, their structural representation, their state, their scheduling, their lifecycle, their communication, and much more in the following sections.

Types Of Processes

There’s always a process active on Unix, at any time, as soon as the machine boots up there’s a need for at least one process.

That first process is the init process, which is the parent of all other processes.

This is a conundrum as you need at least a process to create another process. So you need that first stable process that never disappears. This is one of the role of that first process.

We’ll see what this all means later on.

Whenever you issue a command, on the shell or wherever, it might start a new process and/or suspend another process, but not always as it might look like there’s another process running but it might be a built-in command being executed. This is a confusion because of the conflation between processes and programs. A different program doesn’t always mean a new process.

The process management will also be discussed later.

Generally speaking there are 3 process types: User processes, daemon processes, and kernel processes.

User processes are the processes that are initiated by the regular user, they run in “user space”.

Daemon processes are processes that are specifically designed to run properly in the background and offer some service. (See podcast about daemons)

Kernel processes are processes that live in the kernel space. They are similar to daemon but have full access to the kernel data structure and are less flexible configuration wise.

Those grossly are the types of processes you’ll encounter.

Basic attributes & Permissions

Let’s now start the discussion about some elements of the structure representing a process and certain specificities they have which we can further elaborate later.

There need to be a way to pinpoint a specific process, to know which one you are talking about in the list. This is done through a unique identifier/identification called the process identifier or PID for short.

There’s usually a limit to the number of IDs and thus a limit to the number of processes that can live in a system, we call that the PID pool. Those IDs are in some systems assigned sequentially and in some others they’re assigned randomly.

Similarly, there are other identifiers attached to processes:

The Parent Process id, PPID which points to the PID of the parent of this process, the process in question being called the child.

The process-group id, PGID, and the SID, the group and the session the process belongs to respectively.

A process is also like a sovereign land, with its owner, delimitation, and rules. That’s why it contains some security attributes such as the process owner, UID, and permissions/privileges.

The user id, UID, is the user who own this process. When the process tries to accesses a file it checks if the user has the right to do so in its permission field.

However keep in mind that ownership can change in some specific cases such as with the setuid and setgid bits.

There are a lot of other things in the process structure as we’ll see but for now let’s move to how the multiple programs are handled in memory.

Scheduling

This isn’t a podcast dedicated to neither scheduling nor virtual memory but we’re still going to give a quick overview of them.

Those are the mechanism that make it possible to run multiple programs at the same time while using a single resource.

Both of those are parts of the process mechanism that happens in system mode and not in user mode, usually there’s a secure mechanism to switch between both modes, the system calls.

When a process, task, or thread requests something and has to wait to get a reply, for example when it does I/O, or when its cpu timeslice runs out, or when it receives an interrupt be it hardware or software, then the OS can take the CPU away from that process and make it eligible to be swapped to disk, it’s state is changed to blocked or sleeping.

This specific activity is called context switching.

The scheduler, a part of the kernel, will choose which is the most appropriate process to run/wakeup next and switch to. This one chosen is certainly a process that isn’t currently waiting for I/O or some other resources, otherwise it’ll create an infinite recursion, it should be a “runnable processes”. Remember that Unix is a timesharing system and processes take turns referred to as quantum time or cpu timeslices.

There are multiple kinds of scheduling strategies, also called policies, to ensure fairness and effectiveness amongst processes competing for cpu resources.

How those schedulers work internally is out of scope.

Let’s just mention three or four interesting things.

First, some Unix-like operating systems allow to dynamically switch the scheduler and some have it built-in at compile time in the kernel, this is a trade-off between efficiency and convenience. The scheduler is one of the most executed part of the os, having the extra dynamic step may make it slower. Linux has the ability to dynamically switch and most BSDs have it built-in at compile time.

Second, most of today’s scheduler have some mathematics behind them, instead of plain roundrobin between processes. They calculate the next process to execute on a time quantum according to heuristic and feedback mechanism regarding their priority, which is also changed according to this heuristic. The process scheduling priority is another one of those attributes on the process structure.

Yet another of those attribute is the current CPU the process is running on. We call that processor affinity, there’s no reason why a process should not run on a different processor everytime it is chosen but sometimes it’s less costly, less complex, and give performance overhead to do so instead of reassigning it, because of the cache.

There’s also a lot of thought going on to choose an appropriate size for the cpu slice of time so that processes are responsive and at the same time have a high throughput. And there are thoughts going on about knowing after how many number of cpu slices the OS should recalculate the priorities.

For that there needs to be a sort of clock or counter that ticks when the process is in run mode and fires the scheduler when the process is out of time. Remember that other processes can’t stop the current running process, it has to stop itself or get in a “wait” mode.

Technically, according to my research, on FreeBSD and Linux the cpu time slice is by default 100 milliseconds. But it’s not a fixed quantum, it can vary between between 10ms-200ms depending on the priority and policy mechanism in place.

There exists commands and system calls to change the priority of running processes. Namely nice and renice. And so you can affect the math behind how the scheduler assigns cpu time slices if you so desire.

The niceness runs from -20 to 19, where -20 is the highest priority and 19 the lowest but only root can decrease niceness and thus increase priority. The default niceness is inherited from the parent process.

Apart from those timers there are a bunch of other timers stored in the process structure to keep track of some time or tick related functions.

For instance, the kernel keeps track of the process creation time, as well as the CPU time that the process consumed during its lifetime, the time it consumes in user mode, plus some specific counters.

Those counters are usually used to send different signals when they run out.

One of those counter ticks in real time (as in seconds) and when it runs out it’ll send SIGALRM to the process.

Another one ticks only when the process is actually running, it sends SIGVTALRM to the process.

Another ticks when the process is running plus when its running in system mode too, it sends SIGPROF.

Ok, so we mainly know that when the process runs out of time it’s taken away and another process can have the CPU for itself.

What does that “take away” move imply, this “swapping to memory or disk”, storing its state, and restoring its state, and how do processes manage using the same memory at the same time.

Again, this isn’t a podcast about memory management, so let’s skim over some of the parts that are interesting to us.

Memory & Paging - Multiple programs running

A process is a running program, its instructions are executed on the processor and its state keeps changing.

This running process has a list of states, pieces that it can update, that it acesses and that are vital during the course of its executation.

One of those is the data in memory, we call that the address space of a process. Memory usually contains instructions and variables.

Another part is the values currently in the registers of the processor. The Processor explicitly uses them to do operations and to keep track of where the process currently is in it’s instruction sets in memory, where the variables are, be them dynamic or local, etc..

Another one of those is the input/output information, the file descriptors pointing to the files currently being accessed.

So when the process runs out of cpu time what happens to this dynamic state?

The answer is that it is swapped, aka suspended to memory.

This consists of saving, creating a snapshot of the whole current machine state in an appropriate structure and to store it in the memory.

Then the new process that wants to run after this one is restored the same way.

The swapping of processes takes place at the end of the scheduler, after the next process has been chosen.

So, is memory split up amongst processes? Do processes have to manage that memory everytime they access their state and instructions? What about memory fragmentation?

Yes, there was a time when programs had to contain the logic to manage shared memory and fragmentation but that time is gone. These days programs use something called virtual memory, a role now delegated to the kernel, which makes processes believe that they have their own continuous clean memory space, aka their own “address space”. It takes off a huge burden from processes.

Virtual memory works by using a combination of hardware and software techniques. There’s a data structure called a page table that has entries in it used to map virtual memory addresses to real physical memory addresses.

There may exist some hardware translation in the CPU called a memory management unity (MMU) that will automatically translate those addresses. (using its translation lookaside buffer (TLB), which is a cache storing those mappings, and the kernel may extend on this hardware capability.)

Usually the memory is assigned by chunks called pages, this makes it easier to translate and assign memory as you can point to the page number. Moreover those pages have a specific size, the page size, which makes it convenient too. For example it might be 8192 bytes, 8KB or 4096 bytes, 4KB, which is frequent , but the size maybe differ per processor implementations.

Moreover, there’s also a technique called paging out, which when the physical memory runs out will move, or page out, the data to disk instead. This allows to use more memory than is actually available at the cost of reduced response time.

So if you join context-switching with the memory management we’ve just discussed you get an idea of how dispersed the state of a process can be.

There’s a lot more to know about virtual memory and context switching, like for example demand paging, when the memory isn’t allocated directly but only when it’s used, and all the specific scheduling techniques, so we might do a future episode about those.

Some History

Where does the ideas about processes and process comes from.

Let’s have a look at a bit of history from the early Unix days at Bell Labs.

In the very early days there weren’t many processes running on a machine, there was only one process running at a time per terminal.

This was still time-sharing because it was shared amongst terminals. It was a sort of multi-programming environment where there could only be one process at a time in memory. It was also very restrictive as disk I/O was done synchronously, you had to wait till it finished until another process could be placed in memory unlike what we’ve previously discussed. It’s only later on that memory management and support for multiple processes was added, partly because new hardware with this capability was obtained by the lab, it got better once UNIX was re-written in C.

In those days, the current process facilities, aka process control, were not present.

Today there are simple mechanism to manage process creation and lifecycle. The system call fork, exec, wait, and exit. Don’t worry too much about them for now we’ll come back to those later, just keep in mind that those are convenient routines.

Here’s how new commands are executed on a shell today:

The shell reads the name of the command from the terminal input and creates a child process using fork. Then the forked child process calls exec with the name of that command to call. Meanwhile the parent, the shell, waits for the command to finish executing. When it’s done it repeats the same procedure, reading from the shell, forking, and executing.

This will sound familiar if you have some knowledge of fork and exec. But at the early Bell labs days there were none of these facilities, no fork, nor wait, nor exec, and even exit meant something different.

This is how a new shell-command/process was started:

The shell closed all opened files, opened a terminal to get the file descriptor 0 and 1 for input and output, then read a command from the terminal. So far so good, the only difference is that it closes all file descriptors first. Now it gets weird: it links the file specifying the command, opens it, removes the link, then copies a small bootstrap program to the top of the memory which instruction is to read the file over the shell code, then it executes it. Which all gives the effect of an exec. And… When the command finishes it calls exit, which causes the system to re-read a fresh version of the shell, because the current code in memory has been tainted by the new command. Finally it repeats itself, cleaning the opened files, etc..

That’s how the primitive execution of new processes started, as a bootstrap over the previous command and a refresh for an exit.

This simple way had some major issues: no support for background processes, no support for retaining memory across different command execution, and there could only be one process per terminal at a time.

It also had to close and reopen all it’s file descriptors everytime, including the terminal file descriptor. Some mini-hacks were made to counter that like linked the terminal file descriptor in the directory where the command is called.

Ok. So how did it evolve to give birth to the sys-calls we now have.

Fork and exec were easily added after that, partly inspired by the division some earlier time-sharing OS at the labs had, the GENIE time-sharing system.

Mainly the separation between creating a new process as a copy of the first one and executing a new process over it. Fork does that, it continues to run the same program as its parent until it performs an explicit exec.

You can see that this is similar to what was already there it just needed to extend the process table, which was already used to handle the one-process-per-terminal time-sharing mechanism.

The code of the parent was swapped to the main disk while it’s forked version ran.

After that the exit system call was also changed. Instead of re-reading a new copy of the shell it could simply clean up its process table.

As for wait, the primitive version was a sort of message passing mechanism, sending one WORD size to a receiver or wait for it to be sent from the sender. But it was a generic message passing mechanism, there didn’t even need to be a relationship between those processes.

It was used like that: the parent shell, after executing a new command, sent a message to the new process and when the command finishes the shell would be waiting for the message back from the child, which then would just exit without sending that message, thus wouldn’t exist anymore, and thus would return an error in the parent saying that it couldn’t get back the message, which finally would let it continue its execution.

This was the primitive for the wait but there was not much use for the message passing and it was replaced by the more simple and less generic wait.

This way of waiting also had the issue that the shell depended on a message that was never be received to continue execution but the forked process could possible send that message back and disrupt the shell.

This all lead to detached processes (with &) and recursive shell commands.

Later on it was found that processes absolutely needed a way to share environment, be it path and file descriptors, with their parents, otherwise commands such as chdir wouldn’t work. Here, chdir was re-implemented as a special command in the shell.

In conclusion, in the early days a process meant “running the program”, its sequence of instructions, on a processor. But gradually it incorporated more mechanism and facilities within itself and the changing system around it. It now means “executing a program and its context”.

Moreover, multiprogramming evolved into multitasking, multiprocessing, and multithreading.

Generic Process State

Let’s go back to our discussion about today’s processes.

Processes are state machines, they have an attribute in their structure, like the other attributes we’ve mentioned, stored in the process, that indicates what its status currently is.

The process scheduler is the one that changes this state.

When a process is first created via the fork system call it is marked as NEW, it’s still undergoing process creation.

Its state changes to NORMAL/READY when it can allocate the resources for execution, putting the executable in memory and filling the stack with the arguments for example. A NORMAL process is just waiting for the CPU to be available to start executing, it’s ready for scheduling.

From that point on it’ll switch between RUNNABLE and STOPPED/SLEEPING according to the signals it receives. SIGCONT and SIGSTOP respectively.

There are multiple categories for the STOPPED/SLEEPING state. A process can be waiting for an event, it could be I/O, or it can be stopped because of signals. The difference between those two is that one is interruptible sleep and another is uninterruptible sleep, as in one can be triggered by software interrupts and the other can only be triggered by hardware interrupts. But that’s also tricky because some interruptible states can be considered uninterruptible because they happen in kernel space but that’s out of scope.

All of that until the process terminates.

When the process terminates it is marked as ZOMBIE, that means it is waiting to be reaped by its parent, it’s undergoing process termination.

If the parent doesn’t reap it then it stays in this state until the parent dies and it’s reparented to PID 1 which will then reap it. I’ve discussed this amply in the podcast related to zombies.

The last state is TERMINATED but that’s not a real state as no process exists as a TERMINATED process.

Life Cycle - Process Creation & Inheritance

We’ve seen the process states now let’s discuss the process life cycle.

Yes, it sounds peculiar but those are two distinct subjects, in the life cycle we have to discuss the actual creation and deletion, inheritance, and control of processes (wait & continue). Well, a life cycle is synonym with life and death.

As you remember from the history we mentioned a bit how processes are created these days using the UNIX operating system API. It uses the two step process: They are “forked” from a parent and inherit their context, and then exec another command over it to become a new program, then they can call exit to stop existing.

This is how processes are born.

But this is a conundrum as there need to be a first process so that other process can emerge from it, creating this sort of tree-like structure of parent-children relationships. There needs to be a root to this tree.

It’s convenient because we have a solution to this and it’s the PID 1, the parent of all processes, the mighty init process that is automatically started at boot time and that promises to stay alive as long as the system is running. Every process is a copy of the init process in a certain way.

fork()

Let’s focus on fork().

As we said multiple times and as you understood from its history, the fork system call creates a copy of the calling process. It inherits everything the parent has in memory (thread structure and virtual memory) and also its file descriptors, it’s almost an exact copy, other than to have a different PID and PPID.

Thus the child will resume execution where its parent left off if it doesn’t call exec directly, meanwhile the parent being suspended.

Fork is the only system call that returns two values, zero for the child and the PID of the child to the parent. It’s made that way so that the parent can identify and wait on the child, as for the child it’s easy to find out its parent pid by calling getppid().

The usage of fork is the base for what we call the process tree and process creation, the tree of parent and children processes. The only process that isn’t forked is the init process.

There are other versions of fork available on some system. For instance the rfork sys-call on some BSDs, and a similar clone() in the Linux kernel, that creates a new process entry that shares only a selected set of resources.

Another one of those version of fork is the vfork call, or in some version the posix_spawn, ensures that the parent will not run until the child does either exec or exit.

With vfork, the child borrows the MMU setup from the parent and memory pages are shared among the parent and child process with no copying done, and in particular with no copy-on-write semantics. Which means that it won’t necessarily allocate more memory, no more page table, but just shares it, which also means it could possibly overwrite it.

The vfork() function has the same effect as fork(2), except that the behavior is undefined if the process created by vfork() either modifies any data other than a variable of type pid_t used to store the return value from vfork(), or returns from the function in which vfork() was called, or calls any other function before successfully calling _exit(2) or one of the exec(3) family of functions.

Exec family

What about exec now.

We’ve also discussed exec before, it replaces the current process with a new one, loading the executable into memory, inheriting some of the context and environment but not the state, nor memory, nor file descriptors.

There is a family of exec system calls. The difference between the variants lays in the way they are called and how the environment of the new process is set up. You have execl, execlp, execle, execv, execvp, execvpe.

Here’s how to make meaning out of those:

All of them have exec in them, what differs is in the last characters.

There are two categories, the execl and execv, which varies only by the way the executable path is specified: The v stands for an array that ends with NULL and the l stands for variadic.

Now with those in mind you can add two things, one is about adding new environment variables and the other is about specifying if the executable should be searched in the path. Respectively those are the ‘e’ and ‘p’.

Wait & Exit

So that’s it the process is running now the parent can wait until it exits.

The parent waits on its children PIDs and when they exit it receives back the exit code/status of that child process. The status ranges from 0 to 255, 0 meaning success.

More precisely the exit routine does the following:

Canceling any pending timers and ticks
Releasing virtual-memory resources
Closing open descriptors

Grouping/Session/Foreground/Background, IPC, And Others

There are still some topics to mention about processes: Grouping and sessions, job control, environment variables, and inter process communication aka IPC.

Both of those are out of scope to this podcast.

Let’s just mention that grouping and session are ways to classify processes hierarchically, even more than just the process tree.

Sessions are processes belonging to the same TTY and groups are just sub-groups part of the same session.

Those are important because it allows processes to know on which terminal they should output or take input from. It’s also useful to create that sort of separation, for example in a session there’s only one process that is active in the foreground at a time but that doesn’t always mean that there’s only one process running at a time, the other processes/jobs could be in the background.

I’ve discussed this a bit in the podcast about terminals. I’ve also dedicated a whole podcast about environment variables.

Regarding IPC there are so many techniques that can be used.

The most interesting for us in this podcast is the one that has a place in the structure of the process, the signals. Though, pipes could be said to be there too as they are file descriptors.

Signal handlers are part of this structure as they are software interrupts, forces context switch, and thus need to be handled at the kernel level.

There’s, again, a whole podcast dedicated to them.

Process Representation

We’ve skimmed the topic of parent and children relationship between processes, we’ve also tackled a bit with the process attributes, and a lot of other things, now let’s talk in more depth about the process structure: How are processes represented in the kernel.

The kernel is a program and like any program it needs structures to organise data. Processes are not an exception. Most of Unix-like OS are written in C and thus use C-structures to represent processes.

The technical name for this is the “process control block”, PCB, aka the task controlling block or task struct, or switch frame.

In my opinion the BSD way of representing this structure is cleaner and less complex than in Linux where it has a bunch of IFDEF mess.

However, most Unix-based OS have the same common components in it.

You can follow along with the show notes if you have them in front of you.

This regroups many of the concepts we’ve discussed thus far with a lot of more fine grained details. As we’ve said earlier, it’s whatever context the process needs to be aware of.

Let’s also remind that threads are equivalent more or less to processes, they are tasks, and that they may use the same structure but share the same virtual memory.

So that PCB contains:

It has its identification information, the PID. The PID is chosen differently according to the operating system. Some assign it randomly and some sequentially. However, as with any resources, this pool is limited and may run out of PID to give.

It has the memory address space for this program, aka its virtual memory. In it you can find the image of the executable, in whatever format or form it takes.

It also contains metadata, links, for accessing shared memory amongst processes, like with libraries.

The structure contains the state the process is currently in: UNUSED, EMBRYO, SLEEPING, RUNNABLE, RUNNING, ZOMBIE

It may contain the path or inode/vnode pointing to the currently executed code.

It contains a list of the file descriptors currently associated with the process.

It has a creation mask, the umask.

It has an inode/vnode pointing to the current directory of the process.

It has some security attributes ID, such as owner, groups, effective uid.

It contains the processor state and context.

It has a link to its parent ID.

It has signal handling attributes, callbacks to what should be executed when they are triggered. That includes their state, their mask, and actions, etc..

It has other security and limit attributes.

The process also has all the timers and counters we’ve mentioned before. Keeping track of CPU utilization and ticks.

It has scheduling stuffs, priority and scheduling class.

It may even have some tracing information for debugging.

All of that and even more subtleties we aren’t going to mention but that you’ll feel more confident reading now.

Data structurally processes are usually stored in multiple lists, queues, and trees.

Queues for scheduling, tree for the process hierarchy, and more.

Specifically for scheduling there might be sub-queues used to store processes of certain types to optimize lookup.

There might be a list with zombie processes that doesn’t need to be schedule but cleaned up and another one with all the other ones.

There might also be a queue for the processes currently stopped and the ones sleeping.

All of this is accessible from kernel space but not everything is from the user-space.

You only have access to the limited set of those attributes that you can manipulate via system calls.

For instance you can’t force changing the PID of a process or change it’s virtual memory directly or force context switching.

Cgroups, jails, & limitation/security

Let’s bring up some broader subjects, some extra topics.

There are many fancier techniques to manage processes, used for limitation, containerization, and security.

Those techniques could be using limits or control groups aka cgroups, or jails, or whatever container.

cgroups for example is a Linux kernel feature that acts on a group of process to limits, acounts for, and isolates resource usage, be it CPU, memory, disk I/O, networking, etc..

It provides an interface to control processes group all together.

For example it helps managing the priority (nice) of a group according to some rules.

The Linux kernel also offers some isolation features, such that processes run in a separate namespace from other processes, used along with cgroups it can keep processes from seeing others resources.

There are other environment isolation mechanism such as jails and chroot.

Debugging

Let’s list some great tools for debugging the processes tree.

The pstree command will display the whole process hierarchy.

The ps command is the must know command to check what’s happening with processes, it has everything in it, the manpage says it all.

There are top and htop for live monitoring of processes.

You can trace the process execution of another process via the ptrace system call, it’s used a lot in the gnu debugger, gdb.

There’s also a tool called vmtouch that you can use to check how much virtual memory things are taking up. (https://hoytech.com/vmtouch/)

Those should be enough to get you going.

Conclusion

Processes are interesting but hard to tackle as a subject.

This podcast felt a bit limited in its scope but that is because processes touch everything around the system.

So let’s wrap this up.

Those were the processes on Unix, with their flaws and qualities.

And as usual, there’s some good content in the show notes so be sure to check those out.

Cheers.

References

Attribution

R.o.t / CC BY-SA (https://creativecommons.org/licenses/by-sa/4.0)

If you want to have a more in depth discussion I'm always available by email or irc. We can discuss and argue about what you like and dislike, about new ideas to consider, opinions, etc..
If you don't feel like "having a discussion" or are intimidated by emails then you can simply say something small in the comment sections below and/or share it with your friends.