Processes, Threads, and Jobs

Process Internals
• Data Structures
• Kernel Variables
• Performance Counters
• Relevant Functions
• Flow of CreateProcess
 Data Structures
•Contains an executive process
block. (EPROCESS)

•Contains executive thread blocks.

(ETHREAD)

•EPROCESS block and related data

structures exists in system space,
except the process environment
block (PEB), which exists in the
process address space.
 Data Structures
• Contents of the EPROCES Block
• Kernel process block / Process control block
• Process identification
• Quota block
• Virtual address descriptors
• Working set information
• Virtual memory information
• Exception local procedure call (LPC) port
Data Structures
 Data Structures
• Virtual Address Descriptors
• Set of data structures to keep track of which virtual addresses have
been reserved in the process’s address space and which have not.
• It describes the status of the process’s address space.
• Binary search tree containing information about the current virtual
space allocation of each process.
• Each node in the tree is a VAD.
• For each bloc of memory allocated to the process a new VAD is
added into the tree.
• Working Set
• Working set is a subset of virtual pages resident in physical
memory.
• Exception Dispatches
• Finds an exception handler that can dispose of the exception.
 Data Structures
• Access Tokens
• Object to identify the security context of a process or thread.
• Consists of information that describes the privileges , accounts, and
groups associated with the process or thread.
 Data Structures
• Kernel Process Block
• Contains basic information that the Windows kernel needs to
schedule threads.
Flow of CreateProcess
 Flow of CreateProcess
1. Open exe and create section object.
2. Create windows process object.
1. Setting up the EPROCESS block
2. Creating the initial process address space
3. Creating the kernel process block
4. Concluding the setup of the process address space.
5. Setting up the PEB
6. Completing the setup of the executive process object.
3. Create windows thread object
4. Notifying windows subsystem about the New process
5. Starting execution of the initial thread.
 Flow of CreateProcess
6. Performing process initialization in the context of the new
process.
 Flow of CreateProcess
1. Opening the image to be executed
1. Finds appropriate windows image to run the specified exe.
2. Creates a section object to later map it into the address space of
the new process.
3. For XP and Server 2003 checks whether software restriction
policies.
 Run Ntvdm.exe

Run Cmd.exe Use .exe directly

win16

MS-DOS .bat or .cmd Windows

What kind of
application is it?

MS-DOS .exe, .com, or .pif

OS/21.x

POSIX

MS-Dos .exe, .com, or

Run Os2.exe .pif

Run Posix.exe
 Flow of CreateProcess
2. Creating the windows executive process object
Has a valid windows exe file and a section object.
1. Setting up the EPROCESS block
1. Allocate and initialize the windows EPROCESS block
2. Inherit the process affinity mask from the parent process.
3. Set working set size to PsMinimumWorkingSet and
PsMaximumWorkingSet.
4. Set the new process’s quota block to the address of its parent
process’s quota block, and increment the reference count for the
parent’s quota block.
5. Inherit the windows device name space.
6. Store the parent process’s ID in the InheritedFromUniqueProcessId
field in the new process object.
7. Create the process’s primary access token.
8. The process table handle is initialized.
9. Set the new process’s exit status to STATUS_PENDING.
 Flow of CreateProcess
2. Creating the initial process address space.
Has a valid windows exe file and a section object.
1. Setting up the EPROCESS block
1. Allocate and initialize the windows EPROCESS block
2. Inherit the process affinity mask from the parent process.
3. Set working set size to PsMinimumWorkingSet and
PsMaximumWorkingSet.
4. Set the new process’s quota block to the address of its parent
process’s quota block, and increment the reference count for the
parent’s quota block.
5. Inherit the windows device name space.
6. Store the parent process’s ID in the InheritedFromUniqueProcessId
field in the new process object.
7. Create the process’s primary access token.
8. The process table handle is initialized.
9. Set the new process’s exit status to STATUS_PENDING.
 Flow of CreateProcess
2. Creating the initial process address space.
1. Three pages are created:
1. Page directory
2. Hyperspace page
3. Working set list
2. Mapping of initial pages. Them no. of pages is deducted from the
kernel variable MmTotalCommittedPages and added to
MmProcessCommit.
3. PsMinimumWorkingSet size is deducted from the
MmResidentAvailablePages.
4. Page table pages for the nonpaged portion of system space and
system cache are mapped into the process.

3. Creating the Kernel Process Block

1. Initializes KPROCESS block
 Flow of CreateProcess
4. Concluding the setup of the process address space.
1. Memory manager initializes the process’s working set list.
2. Section is now mapped into the new process’s address space.
3. Ntdll.dll is mapped into the process.
4. The system wide national language support (NLS) tables are
mapped into the process’s address space.
5. Setting up the PEB
 Flow of CreateProcess
6. Completing the setup of the executive process object.
1. The process’s creation is written to the Security event log, if system
wide auditing of the processes is enabled.
2. The new process if added to the job if its parent was contained in
the job.
3. Selecting a single processor from the available processors to run all
threads on it, if IMAGE_FILE_UP_SYSTEM_ONLY flag is set.
4. If image specifies and explicit processor affinity mask, this value is
copied to the PEB and later set as the default process affinity mask.
5. Inserts the new process block at the end of windows list of active
processes.
6. Process’s creation time is set and handle to it is returned to the
caller.
 Flow of CreateProcess
3. Creating the initial thread and its stack and context.
Has executive process object completely set up.
1. Initializes the stack taking the size from the image
2. Thread is created using NtCreateThread.
3. The address of PEB is used by the initialization code which runs
in the context of this new thread.
4. NtCreateThread calls PspCreateThread which does the
following.
1. Increments the thread count in the process object.
2. Creates ETHREAD and initializes it too.
3. Generates Thread ID.
4. TEB is set up in the user mode address space of the process.
5. User mode thread start address is stored in the ETHREAD.
System supplied thread startup functions:
BaseProcessStart for first thread,
BaseThreadStart for additional threads.
 Flow of CreateProcess
6. KeInitThread is called to setup KTHREAD block.
1. Sets current and base priorities
2. Affinity
3. Quantum
4. Allocates kernel stack for the thread and initializes the machine-
dependent h/w context.
5. Sets thread state to initialized and returns to PspCreateThread.
7. Thread creation notification routines are called.
1. Does an access check on the access token.
2. Thread ready for execution
 Flow of CreateProcess
4. Notifying windows subsystem about the new process.
Has executive process and thread objects.
1. Kernel32.dll sends a message to the windows subsystem to set up for
new process and the thread.
2. In response windows subsystem does,
1. Duplicates the handle for the process and thread.
2. Sets the process priority class if not specified.
3. Csrss block is allocated
4. Exception port is set to be the general function port for the windows
subsystem.
5. Process debugger port is set if debugger process is attached.
6. Csrss thread block is allocated and initialized.
7. Inserts thread in the list of threads for the process.
8. the count of processes in this session is incremented.
9. The process shutdown level is set to 0x280 (the default process shutdown
level—see SetProcessShutdownParameters)
 Flow of CreateProcess
10. The new process block is inserted into the list of Windows subsystem wide
processes.
11. The per-process data structure used by the kernel-mode part of the
Windows subsystem (W32PROCESS structure) is allocated and initialized.
12. The application start cursor is displayed. This cursor is the familiar arrow
with an hourglass attached—the way that Windows says to the user, "I'm
starting something, but you can use the cursor in the meantime." If the
process doesn't make a GUI call after 2 seconds, the cursor reverts to the
standard pointer. If the process does make a GUI call in the allotted time,
CreateProcess waits 5 seconds for the application to show a window. After
that time, CreateProcess will reset the cursor again.
 Flow of CreateProcess
5. Starting execution of the initial thread.
Has process environment determined, resources for its threads to use
have been allocated, the process has a thread and windows subsystem
knows about the new process.
initial thread is resumed, if CREATE_SUSPENDED flag is not set.

6. Performing process initialization in the context of the

new process.
1. Kernel mode startup routine KiThreadStartup starts the thread.
2. Calls the PspUserThreadStartup routine and passes user specified
thread start address as the parameter to it.
3. Then the LdrInitializeThunk is called, which initializes the loader,
heap manager, NLS tables, thread local storage array and critical
section structures. Loads required dlls and calls DLL entry points
with the DLL_PROCESS_ATTACH function code.
4. Finally image begins execution in user mode.
 Thread Internals
Structure of executive thread
block
 Thread Internals
Structure of Kernel Thread Block
 Thread Scheduling
1. Priority driven
2. Preemptive scheduling
1. Ready to run threads are run, but considering the processor affinity.
2. Affinity can be altered using the windows scheduling functions.
3. Threads runs for a specific amount of time called the quantum.
4. Quantum values varies from system to system and process to
process.
It depends on the system configuration settings, foreground/background
status of the process, or use of job object to alter the quantum.
5. Because of preemptive scheduler, the quantum mite not
complete for the thread, if another thread with a higher priority
becomes ready to run.
6. Scheduling is handled by Kernel. The routines that perform
these duties are collectively called the kernels dispatcher.
 Thread Scheduling
7. Need for dispatching may arise due to following reasons:
1. A thread becomes ready to execute—for example, a thread has been newly
created or has just been released from the wait state.
2. A thread leaves the running state because its time quantum ends, it
terminates, it yields execution, or it enters a wait state.
3. A thread's priority changes, either because of a system service call or
because Windows itself changes the priority value.
4. A thread's processor affinity changes so that it will no longer run on the
processor on which it was running.
8. At this junction windows decides which thread to run and does a
context switch.
9. Scheduling decisions are made on thread basis and not on process to
which belongs.
 Priority Levels
Two different perspectives
1. Those of windows API.
2. Of the windows kernel.
Mapping of Windows kernel priorities to the Windows API
 Priority Scheduling
1. In Windows API each thread has a base priority that is a function of
its process priority class and its relative thread priority.
2. Process has only one base priority value.
3. Scheduling decisions are made based on current priority.
4. Priority of threads can be boosted in dynamic range (1 – 15) for
brief periods but not in the real time range.
5. To enter the real time priorities you must have the increase
scheduling priority privilege.
6. Initial base priority of thread is inherited from the process base
priority. This can be overridden on CreatePRocess function or by
using command-line START command. After creation it can be
changed by using SetPriorityClass function or various tools such as
Task manger and Process Explorer.
7. To change the default base priority of process internal system call
NtSetinformationProcess is used.
 Thread States

Figure 6-14. Thread states on Windows 2000 and Windows XP

                                                                                                                                                                                                                                        
 Thread States
• The thread states are as follows:

• Ready :A thread in the ready state is waiting to execute. When looking for a
thread to execute, the dispatcher considers only the pool of threads in the ready
state.
• Standby A thread in the standby state has been selected to run next on a
particular processor. When the correct conditions exist, the dispatcher performs a
context switch to this thread. Only one thread can be in the standby state for
each processor on the system. Note that a thread can be preempted out of the
standby state before it ever executes (if, for example, a higher priority thread
becomes runnable before the standby thread begins execution).
• Running Once the dispatcher performs a context switch to a thread, the thread
enters the running state and executes. The thread's execution continues until its
quantum ends (and another thread at the same priority is ready to run), it is
preempted by a higher priority thread, it terminates, it yields execution, or it
voluntarily enters the wait state.
• Waiting A thread can enter the wait state in several ways: a thread can
voluntarily wait for an object to synchronize its execution, the operating system
can wait on the thread's behalf (such as to resolve a paging I/O), or an
environment subsystem can direct the thread to suspend itself. When the thread's
wait ends, depending on the priority, the thread either begins running
immediately or is moved back to the ready state.
 Thread States
• Transition A thread enters the transition state if it is ready for execution but its
kernel stack is paged out of memory. Once its kernel stack is brought back into
memory, the thread enters the ready state.
• Terminated When a thread finishes executing, it enters the terminated state.
Once the thread is terminated, the executive thread block (the data structure in
nonpaged pool that describes the thread) might or might not be deallocated. (The
object manager sets policy regarding when to delete the object.)
• Initialized This state is used internally while a thread is being created.
Dispatcher Database
 Dispatcher Database
• Variable – KiReadySummary
• Bitmask (32 bits)
• Bitmask of priority levels that have one or more ready threads
• Variable – KiDispatcherReadyListHead
• Array of 32 list entries
• List heads for the 32 ready queues
 Quantum
• A quantum is the amount of time a thread gets to run before
Windows checks to see whether another thread at the same priority
is waiting to run. If a thread completes its quantum and there are no
other threads at its priority, Windows permits the thread to run for
another quantum.
• On Windows 2000 Professional and Windows XP, threads run by
default for 2 clock intervals;
• on Windows Server systems, by default, a thread runs for 12 clock
intervals.
• Quantum Accounting
• Quantum values are stored as multiple of three times
the number of clock ticks. i.e. 6 or 12.
• This is to allow for partial quantum decay on wait
completion. This is done for all threads with priority
less than 14 for above 14 before doing the reduction
the quantum is reset to full turn.
 Quantum
• Controlling the Quantum
• Two settings of quantum
• Short – 2 clock ticks – windows client
• Long – 12 clock ticks – windows server

• Programs setting – default of windows 200 prof and windows XP

• Designates use of short variables.
• Background services – default of Windows Server Systems.
• Designates use of long variables.
• Quantum Boosting
• quantum boosting for foreground and background processes.
• Adjustments of quantums applies only to processes with a
priority higher that idle on systems configured to Programs in
the Performance Options and not changed ofr foreground
process on the system configured to background Services.
 Quantum
•Quantum Settings Registry Value
•HKLM\SYSTEM\CurrentControlSet\Control\Priority\Win32Prior
itySeparation registry is modified

• Short vs. Long A setting of 1 specifies long, and 2 specifies short.

A setting of 0 or 3 indicates that the default will be used (short for
Windows 2000 Professional and Windows XP, long for Windows
Server systems).
• Variable vs. Fixed A setting of 1 means to vary the quantum for
the foreground process, and 2 means that quantum values don't
change for foreground processes.
 Quantum
• A setting of 0 or 3 means that the default (which is variable for
Windows 2000 Professional and Windows XP and fixed for
Windows Server systems) will be used.
• Foreground Quantum Boost This field (stored in the kernel
variable PsPrioritySeparation) must have a value of 0, 1, or 2.
(A setting of 3 is invalid and treated as 2.) It is used as an index
into a three-element byte array named PspForegroundQuantum
to obtain the quantum for the threads in the foreground process.
The quantum for threads in background processes is taken from
the first entry in this quantum table.
 Scheduling Scenarios
Voluntary Switch
 Scheduling Scenarios
Preemption
 Scheduling Scenarios
Quantum End
 Context Switching
• Instruction pointer
• User and kernel stack pointers
• A pointer to the address space in which the thread
runs (the process's page table directory)
 Priority Boosts
1. Priority Boosting after I/O comletion
1. The boost is always applied to a threads base priority.
2. Never boosted above priority 15.
3. The thread gets to run at the elevated priority for its remaining
quantum (as described earlier, quantums are reduced by 1 when
threads exit a wait) before decaying one priority level at a time until
reaches its original base priority.
 Priority Boosts
2. Boosts after waiting for events and semaphores
3. Priority Boosts for foreground Threads after waits
1. Kernel funtion KiUnwaitThread boosts the thread priority by the
current value of PsPrioritySeparation, which reflects the quantum
table index used to select quantums for the threads of foreground
applications.
2. This improves the responsiveness of interactive applications.
3. This boosting applies to all windows systems, and cant be
disabled.
4. Priority Boosts for GUI threads wake up
1. Additional boost of 2 when threads that own windows wake up
because of windowing activity.
2. Applied when KeSetEvent to set an event used to wake up a gui
thread is called.
3. Favors interactive applications.
5. Priority Boosts for CPU starvation
 Priority Boosts
• balance set manager (a system thread that exists primarily to perform memory
management functions ) scans the ready queues for any threads that have
been in the ready state (that is, haven't run) for approximately 4 seconds. If it
finds such a thread, the balance set manager boosts the thread's priority to 15.
On Windows 2000 and Windows XP, the thread quantum is set to twice the
process quantum. On Windows Server 2003, the quantum is set to 4 quantum
units. Once the quantum is expired, the thread's priority decays immediately to its
original base priority. If the thread wasn't finished and a higher priority thread is
ready to run, the decayed thread will return to the ready queue, where it again
becomes eligible for another boost if it remains there for another 4 seconds.

• The balance set manager doesn't actually scan all ready threads every time it runs.
To minimize the CPU time it uses, it scans only 16 ready threads; if there are
more threads at that priority level, it remembers where it left off and picks up
again on the next pass. Also, it will boost only 10 threads per pass—if it finds 10
threads meriting this particular boost (which would indicate an unusually busy
system), it stops the scan at that point and picks up again on the next pass.
 Job Objects
• A job object is a nameable, securable, shareable kernel
object that allows control of one or more process as a
group.
• It allows groups of processes to be managed and
manipulated as a unit.
• a process can be a member of only one job object.
• By default, its association with the job object cant be
broken and all processes created by the process and its
descendents are associated with the same job object as
well.
• Records basic accounting information for all processes
associated with the job and for all processes that were
associated with the job but have since terminated.