Hubbry Logo
Protected modeProtected modeMain
Open search
Protected mode
Community hub
Protected mode
logo
7 pages, 0 posts
0 subscribers
Be the first to start a discussion here.
Be the first to start a discussion here.
Protected mode
Protected mode
Protected mode
View on Wikipedia
from Wikipedia
Part of a series on
Microprocessor modes for the x86 architecture
First supported platform shown in parentheses

In computing, protected mode, also called protected virtual address mode,[1] is an operational mode of x86-compatible central processing units (CPUs). It allows system software to use features such as segmentation, virtual memory, paging and safe multi-tasking designed to increase an operating system's control over application software.[2][3]

When a processor that supports x86 protected mode is powered on, it begins executing instructions in real mode, in order to maintain backward compatibility with earlier x86 processors.[4] Protected mode may only be entered after the system software sets up one descriptor table and enables the Protection Enable (PE) bit in the control register 0 (CR0).[5]

Protected mode was first added to the x86 architecture in 1982,[6] with the release of Intel's 80286 (286) processor, and later extended with the release of the 80386 (386) in 1985.[7] Due to the enhancements added by protected mode, it has become widely adopted and has become the foundation for all subsequent enhancements to the x86 (IA-32) architecture,[8] although many of those enhancements, such as added instructions and new registers, also brought benefits to the real mode.

History

[edit]

The first x86 processor, the Intel 8086, had a 20-bit address bus for its memory, as did its Intel 8088 variant.[9] This allowed them to access 220 bytes of memory, equivalent to 1 megabyte.[9] At the time, 1 megabyte was considered a relatively large amount of memory,[10] so the designers of the IBM Personal Computer reserved the first 640 kilobytes for use by applications and the operating system and the remaining 384 kilobytes for the BIOS (Basic Input/Output System) and memory for add-on devices.[11]

As the cost of memory decreased and memory use increased, the 1 MB limitation became a significant problem. Intel intended to solve this limitation along with others with the release of the 286.[11]

The 286

[edit]
Further information: Intel 80286

The initial protected mode, released with the 286, was not widely used;[11] for example, it was used by Coherent (from 1982),[12] Microsoft Xenix (around 1984)[13] and Minix.[14] Several shortcomings such as the inability to make BIOS and DOS calls due to inability to switch back to real mode without resetting the processor prevented widespread usage.[15] Acceptance was additionally hampered by the fact that the 286 allowed memory access in 64 kilobyte segments, addressed by its four segment registers, meaning that only 4 × 64 KB, equivalent to 256 KB, could be accessed at a time.[11] Because changing a segment register in protected mode caused a 6-byte segment descriptor to be loaded into the CPU from memory, the segment register load instruction took many tens of processor cycles, making it much slower than on the 8086 and 8088; therefore, the strategy of computing segment addresses on-the-fly in order to access data structures larger than 128 kilobytes (the combined size of the two data segments) became impractical, even for those few programmers who had mastered it on the 8086 and 8088.

The 286 maintained backward compatibility with the 8086 and 8088 by initially entering real mode on power up.[4] Real mode functioned virtually identically to the 8086 and 8088, allowing the vast majority of existing software for those processors to run unmodified on the newer 286. Real mode also served as a more basic mode to set up and bootstrap into protected mode. To access the extended functionality of the 286, the operating system would set up some tables in memory that controlled memory access in protected mode, set the addresses of those tables into some special registers of the processor, and then set the processor into protected mode. This enabled 24-bit addressing, which allowed the processor to access 224 bytes of memory, equivalent to 16 megabytes.[9]

The 386

[edit]
Further information: Intel 80386
An Intel 80386 microprocessor

With the release of the 386 in 1985,[7] many of the issues preventing widespread adoption of the previous protected mode were addressed.[11] The 386 was released with an address bus size of 32 bits, which allows for 232 bytes of memory accessing, equivalent to 4 gigabytes.[16] The segment sizes were also increased to 32 bits, meaning that the full address space of 4 gigabytes could be accessed without the need to switch between multiple segments.[16] In addition to the increased size of the address bus and segment registers, many other new features were added with the intention of increasing operational security and stability.[17] Protected mode is now used in virtually all modern operating systems which run on the x86 architecture, such as Microsoft Windows, Linux, and many others.[18]

Furthermore, learning from the failures of the 286 protected mode to satisfy the needs for multiuser DOS, Intel added a separate virtual 8086 mode,[19] which allowed multiple virtualized 8086 processors to be emulated on the 386. Hardware x86 virtualization required for virtualizing the protected mode itself, however, had to wait for another 20 years.[20]

386 additions to protected mode

[edit]

With the release of the 386, the following additional features were added to protected mode:[2]

Entering and exiting protected mode

[edit]

Until the release of the 386, protected mode did not offer a direct method to switch back into real mode once protected mode was entered. IBM devised a workaround (implemented in the IBM AT) which involved resetting the CPU via the keyboard controller and saving the system registers, stack pointer and often the interrupt mask in the real-time clock chip's RAM. This allowed the BIOS to restore the CPU to a similar state and begin executing code before the reset.[clarification needed] Later, a triple fault was used to reset the 286 CPU, which was a lot faster and cleaner than the keyboard controller method.

To enter protected mode, the Global Descriptor Table (GDT) must first be created with a minimum of three entries: a null descriptor, a code segment descriptor and data segment descriptor. Then, the PE bit must be set in the CR0 register and a far jump must be made to clear the prefetch input queue.[22][23] Also, on an IBM-compatible machine, in order to enable the CPU to access all 16 MB of the address space (instead of only the 8 even megabytes), the A20 line (21st address line) must be enabled. (A20 is disabled at power-up, causing each odd megabyte of the address space to be aliased to the previous even megabyte, in order to guarantee compatibility with older software written for the Intel 8088-based IBM PC and PC/XT models).[24] Enabling A20 is not strictly required to run in protected mode; the CPU will operate normally in protected mode with A20 disabled, only without the ability to access half of the memory addresses. 

; MASM program
; enter protected mode (set PE bit)
mov EBX, CR0   ; save control register 0 (CR0) to EBX
or EBX, PE_BIT ; set PE bit by ORing, save to EBX
mov CR0, EBX   ; save EBX back to CR0

; clear prefetch queue; (using far jump instruction jmp)
jmp CLEAR_LABEL
CLEAR_LABEL:

With the release of the 386, protected mode could be exited by loading the segment registers with real mode values, disabling the A20 line and clearing the PE bit in the CR0 register, without the need to perform the initial setup steps required with the 286. [25]

Features

[edit]

Protected mode has a number of features designed to enhance an operating system's control over application software, in order to increase security and system stability.[3] These additions allow the operating system to function in a way that would be significantly more difficult or even impossible without proper hardware support.[26]

Privilege levels

[edit]
Further information: Ring (computer security)
Example of privilege ring usage in an operating system using all rings

In protected mode, there are four privilege levels or rings, numbered from 0 to 3, with ring 0 being the most privileged and 3 being the least. The use of rings allows for system software to restrict tasks from accessing data, call gates or executing privileged instructions.[27] In most environments, the operating system and some device drivers run in ring 0 and applications run in ring 3.[27]

Real mode application compatibility

[edit]

According to the Intel 80286 Programmer's Reference Manual,[28]

the 80286 remains upwardly compatible with most 8086 and 80186 application programs. Most 8086 application programs can be re-compiled or re-assembled and executed on the 80286 in Protected Mode.

For the most part, the binary compatibility with real-mode code, the ability to access up to 16 MB of physical memory, and 1 GB of virtual memory, were the most apparent changes to application programmers.[29] This was not without its limitations. If an application utilized or relied on any of the techniques below, it would not run:[30]

  • Segment arithmetic
  • Privileged instructions
  • Direct hardware access
  • Writing to a code segment
  • Executing data
  • Overlapping segments
  • Use of BIOS functions, due to the BIOS interrupts being reserved by Intel[31]

In reality, almost all DOS application programs violated these rules.[32] Due to these limitations, virtual 8086 mode was introduced with the 386. Despite such potential setbacks, Windows 3.0 and its successors can take advantage of the binary compatibility with real mode to run many Windows 2.x (Windows 2.0 and Windows 2.1x) applications in protected mode, which ran in real mode in Windows 2.x.[33]

Virtual 8086 mode

[edit]
Main article: Virtual 8086 mode

With the release of the 386, protected mode offers what the Intel manuals call virtual 8086 mode. Virtual 8086 mode is designed to allow code previously written for the 8086 to run unmodified and concurrently with other tasks, without compromising security or system stability.[34]

Virtual 8086 mode, however, is not completely backward compatible with all programs. Programs that require segment manipulation, privileged instructions, direct hardware access, or use self-modifying code will generate an exception that must be served by the operating system.[35] In addition, applications running in virtual 8086 mode generate a trap with the use of instructions that involve input/output (I/O), which can negatively impact performance.[36]

Due to these limitations, some programs originally designed to run on the 8086 cannot be run in virtual 8086 mode. As a result, system software is forced to either compromise system security or backward compatibility when dealing with legacy software. An example of such a compromise can be seen with the release of Windows NT, which dropped backward compatibility for "ill-behaved" DOS applications.[37]

Segment addressing

[edit]
Further information: X86 memory segmentation
Virtual segments of 80286

Real mode

[edit]

In real mode each logical address points directly into a physical memory location, every logical address consists of two 16-bit parts: The segment part of the logical address contains the base address of a segment with a granularity of 16 bytes, i.e. a segment may start at physical address 0, 16, 32, ..., 220 − 16. The offset part of the logical address contains an offset inside the segment, i.e. the physical address can be calculated as physical_address = segment_part × 16 + offset, if the address line A20 is enabled, or (segment_part × 16 + offset) mod 220, if A20 is off.[clarification needed] Every segment has a size of 216 bytes.

Protected mode

[edit]

In protected mode, the segment_part is replaced by a 16-bit selector, in which the 13 upper bits (bit 3 to bit 15) contain the index of an entry inside a descriptor table. The next bit (bit 2) specifies whether the operation is used with the GDT or the LDT. The lowest two bits (bit 1 and bit 0) of the selector are combined to define the privilege of the request, where the values of 0 and 3 represent the highest and the lowest privilege, respectively. This means that the byte offset of descriptors in the descriptor table is the same as the 16-bit selector, provided the lower three bits are zeroed.

The descriptor table entry defines the real linear address of the segment, a limit value for the segment size, and some attribute bits (flags).

286

[edit]

The segment address inside the descriptor table entry has a length of 24 bits so every byte of the physical memory can be defined as bound of the segment. The limit value inside the descriptor table entry has a length of 16 bits so segment length can be between 1 byte and 216 byte. The calculated linear address equals the physical memory address.

386

[edit]

The segment address inside the descriptor table entry is expanded to 32 bits so every byte of the physical memory can be defined as bound of the segment. The limit value inside the descriptor table entry is expanded to 20 bits and completed with a granularity flag (G-bit, for short):

  • If G-bit is zero limit has a granularity of 1 byte, i.e. segment size may be 1, 2, ..., 220 bytes.
  • If G-bit is one limit has a granularity of 212 bytes, i.e. segment size may be 1 × 212, 2 × 212, ..., 220 × 212 bytes. If paging is off, the calculated linear address equals the physical memory address. If paging is on, the calculated linear address is used as input of paging.

The 386 processor also uses 32 bit values for the address offset.

For maintaining compatibility with 286 protected mode a new default flag (D-bit, for short) was added. If the D-bit of a code segment is off (0) all commands inside this segment will be interpreted as 16-bit commands by default; if it is on (1), they will be interpreted as 32-bit commands.

Structure of segment descriptor entry

[edit]
80286 Segment descriptor
31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 09 08 07 06 05 04 03 02 01 00
Base[0..15] Limit[0..15]
63 62 61 60 59 58 57 56 55 54 53 52 51 50 49 48 47 46 45 44 43 42 41 40 39 38 37 36 35 34 33 32
Unused P DPL S X C R A Base[16..23]
80386 Segment descriptor
31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 09 08 07 06 05 04 03 02 01 00
Base[0..15] Limit[0..15]
63 62 61 60 59 58 57 56 55 54 53 52 51 50 49 48 47 46 45 44 43 42 41 40 39 38 37 36 35 34 33 32
Base[24..31] G D 0 U Limit[16..19] P DPL S X C R A Base[16..23]

Where:

  • A is the Accessed bit;
  • R is the Readable bit;
  • C (Bit 42) depends on X:[38]
    • if X = 1 then C is the Conforming bit, and determines which privilege levels can far-jump to this segment (without changing privilege level):
      • if C = 0 then only code with the same privilege level as DPL may jump here;
      • if C = 1 then code with the same or a lower privilege level relative to DPL may jump here.
    • if X = 0 then C is the direction bit:
      • if C = 0 then the segment grows up;
      • if C = 1 then the segment grows down.
  • X is the Executable bit:[38]
    • if X = 1 then the segment is a code segment;
    • if X = 0 then the segment is a data segment.
  • S is the Segment type bit, which should generally be cleared for system segments;[38]
  • DPL is the Descriptor Privilege Level;
  • P is the Present bit;
  • D is the Default operand size;
  • G is the Granularity bit;
  • Bit 52 of the 80386 descriptor is not used by the hardware.

Paging

[edit]
Further information: Paging
Common method of using paging to create a virtual address space
Paging (on Intel 80386) with page size of 4K

In addition to adding virtual 8086 mode, the 386 also added paging to protected mode.[39] Through paging, system software can restrict and control a task's access to pages, which are sections of memory. In many operating systems, paging is used to create an independent virtual address space for each task, preventing one task from manipulating the memory of another. Paging also allows for pages to be moved out of primary storage and onto a slower and larger secondary storage, such as a hard disk drive.[40] This allows for more memory to be used than physically available in primary storage.[40]

The x86 architecture allows control of pages through two arrays: page directories and page tables. Originally, a page directory was the size of one page, four kilobytes, and contained 1,024 page directory entries (PDE), although subsequent enhancements to the x86 architecture have added the ability to use larger page sizes. Each PDE contained a pointer to a page table. A page table was also originally four kilobytes in size and contained 1,024 page table entries (PTE). Each PTE contained a pointer to the actual page's physical address and are only used when the four-kilobyte pages are used. At any given time, only one page directory may be in active use.[41]

Multitasking

[edit]
Further information: Computer multitasking

Through the use of the rings, privileged call gates, and the Task State Segment (TSS), introduced with the 286, preemptive multitasking was made possible on the x86 architecture. The TSS allows general-purpose registers, segment selector fields, and stacks to all be modified without affecting those of another task. The TSS also allows a task's privilege level, and I/O port permissions to be independent of another task's.

In many operating systems, the full features of the TSS are not used.[42] This is commonly due to portability concerns or due to the performance issues created with hardware task switches.[42] As a result, many operating systems use both hardware and software to create a multitasking system.[43]

Operating systems

[edit]

Operating systems like OS/2 1.x try to switch the processor between protected and real modes. This is both slow and unsafe, because a real mode program can easily crash a computer. OS/2 1.x defines restrictive programming rules allowing a Family API or bound program to run in either real or protected mode. Some early Unix operating systems, OS/2 1.x, and Windows used this mode.

Windows 3.0 was able to run real mode programs in 16-bit protected mode; when switching to protected mode, it decided to preserve the single privilege level model that was used in real mode, which is why Windows applications and DLLs can hook interrupts and do direct hardware access. That lasted through the Windows 9x series. If a Windows 1.x or 2.x program is written properly and avoids segment arithmetic, it will run the same way in both real and protected modes. Windows programs generally avoid segment arithmetic because Windows implements a software virtual memory scheme, moving program code and data in memory when programs are not running, so manipulating absolute addresses is dangerous; programs should only keep handles to memory blocks when not running. Starting an old program while Windows 3.0 is running in protected mode triggers a warning dialog, suggesting to either run Windows in real mode or to obtain an updated version of the application. Updating well-behaved programs using the MARK utility with the MEMORY parameter avoids this dialog. It is not possible to have some GUI programs running in 16-bit protected mode and other GUI programs running in real mode. In Windows 3.1, real mode was no longer supported and could not be accessed.

In modern 32-bit operating systems, virtual 8086 mode is still used for running applications, e.g. DPMI compatible DOS extender programs (through virtual DOS machines) or Windows 3.x applications (through the Windows on Windows subsystem) and certain classes of device drivers (e.g. for changing the screen-resolution using BIOS functionality) in OS/2 2.0 (and later OS/2) and 32-bit Windows NT, all under control of a 32-bit kernel. However, 64-bit operating systems (which run in long mode) no longer use this, since virtual 8086 mode has been removed from long mode.

See also

[edit]

References

[edit]
[edit]
Revisions and contributorsEdit on WikipediaRead on Wikipedia

Protected mode

View on Grokipedia
from Grokipedia
Protected mode is an operational mode of x86-compatible central processing units (CPUs) introduced by with the 80286 in 1982, serving as the native state of the processor and enabling advanced , , and multitasking through segmentation and paging mechanisms. In protected mode, the processor supports a segmented memory model where is divided into segments defined by descriptors in the (GDT) or Local Descriptor Table (LDT), each specifying base addresses, limits, and access rights to enforce isolation between tasks. Paging provides an additional layer of virtual-to-physical address translation, allowing for efficient implementation and further against unauthorized access. This mode expanded significantly with the Intel 80386 in 1985, introducing 32-bit addressing for up to 4 gigabytes of linear , compared to the 1-megabyte limit of . A core feature of protected mode is its hierarchical protection system, utilizing four privilege levels—known as rings 0 through 3—to control access to resources, with ring 0 reserved for the most privileged operations (typically the operating system kernel) and ring 3 for user applications. Multitasking is facilitated through Task-State Segments (TSS) and task gates, which manage context switches and maintain separate address spaces per task, ensuring hardware-enforced isolation. Entry into protected mode occurs by setting the Protection Enable (PE) bit in the machine status word (MSW) on the 80286 or in the CR0 control register on the 80386 and later processors, a process that requires careful initialization of segment descriptors and often a far jump to flush the processor pipeline. Unlike , which operates in a 16-bit flat without protection mechanisms and supports only single-tasking, protected mode provides robust against faults and malicious code, forming the foundation for like Windows, , and macOS. Later enhancements, such as (PAE) on processors from the era onward, further increased physical memory addressing to 64 gigabytes while maintaining compatibility.

Background

Real Mode Basics

Real mode, also known as real-address mode, is the default operational state of x86 processors upon power-up or reset, providing backward compatibility with the original and 8088 architectures. In this mode, the processor employs a 20-bit space, restricting accessible to a maximum of 1 MB (2^20 bytes). Addressing operates through a segment:offset scheme, where locations are specified by combining a 16-bit segment value from a segment register with a 16-bit offset. The x86 architecture includes four primary 16-bit segment registers in real mode: CS (code segment), which points to the current code segment; DS (data segment), used for data access; SS (stack segment), for stack operations; and ES (extra segment), for additional data. To form a physical address, the segment value is shifted left by 4 bits (effectively multiplied by 16) to establish the segment base, and the offset is then added to this base. For example, the physical address is calculated as Physical address=(segment×16)+offset\text{Physical address} = (\text{segment} \times 16) + \text{offset}. This segmented approach limits each individual segment to a maximum size of 64 KB (2^16 bytes), as the offset is only 16 bits wide. Furthermore, provides no hardware-enforced , allowing programs unrestricted access to the entire addressable memory space without checks for bounds or privileges. These characteristics establish the foundational model from which protected mode evolved to address growing demands for larger memory and security.

Limitations and Motivations

Real mode, the default operating mode of early x86 processors like the , imposed significant constraints that hindered the evolution of personal computing systems. Primarily, it provided no inherent mechanisms, allowing errant or malicious code to access and corrupt any part of the system's 1 MB , which frequently led to crashes or vulnerabilities in multi-program environments. This 1 MB ceiling stemmed from the 20-bit physical addressing scheme, where segmented addressing—combining a 16-bit segment register shifted left by 4 bits with a 16-bit offset—effectively capped usable at 2^20 bytes, restricting scalability as RAM capacities grew beyond this limit in the early 1980s. Furthermore, real mode lacked support for native multitasking, forcing operating systems to rely on cumbersome software techniques like time-slicing or to simulate concurrent program execution, which were inefficient and prone to errors. These limitations motivated the development of protected mode as a foundational shift toward more robust, secure, and scalable architectures. The primary drivers included the need to support multitasking operating systems such as Unix variants, which required to prevent one application from interfering with others, thereby enabling safer multiuser environments on emerging PCs. Protection against malicious software was another key impetus, as growing software complexity in the amplified risks from faulty or intentionally harmful code, necessitating hardware-enforced boundaries to enhance system reliability. Additionally, the demand for larger memory addressing arose from hardware advancements allowing RAM beyond 1 MB, compelling designs that could leverage up to 16 MB of physical memory while maintaining compatibility with existing 8086 software. Historically, protected mode drew inspiration from operating systems that emphasized isolation and resource sharing, influencing 's response to pressures from developers seeking to port advanced OSes to x86 platforms. 's development of the 80286 was informed by six months of into customer requirements for enhanced memory addressing and protection mechanisms. Developers, including those at who ported (a derivative) to the 80286, sought features for advanced OS implementations, as real mode's constraints made reliable systems impractical on PCs. The 80286, designed starting in 1978 and released in 1982, specifically aimed to bridge 16-bit real-mode compatibility with advanced computing paradigms, incorporating protected mode to address these demands while supporting high-level languages and for scientific, engineering, and business applications.

Historical Development

Intel 80286 Implementation

The , released in 1982, introduced protected mode as the first implementation of this feature in the x86 architecture, extending the 16-bit iAPX 86/88 family with advanced capabilities. This processor supported 24-bit physical addressing, enabling access to up to 16 MB of physical memory, a significant expansion from the 1 MB limit of prior real-mode systems that relied on 20-bit addressing. In protected mode, the 80286 provided a of up to 1 GB per task through segmentation, allowing for more efficient multitasking and isolation without hardware paging support. This mode built on real mode's segment:offset addressing model for compatibility but enforced stricter boundaries to prevent unauthorized access. Key innovations in the 80286's protected mode included the Global Descriptor Table (GDT) and Local Descriptor Table (LDT), which defined segment boundaries and attributes for all processes. The GDT, a system-wide table accessed via the GDTR register, held shared segment descriptors, while the LDT, loaded via the LDTR register, provided task-specific segments for private address spaces. Protection was further enhanced by a four-level privilege hierarchy, known as rings 0 through 3, where ring 0 represented the highest privilege for kernel operations and ring 3 the lowest for user applications. These levels were enforced through the Descriptor Privilege Level (DPL) in segment descriptors and the Current Privilege Level (CPL) in the code segment register, preventing lower-privilege code from accessing sensitive resources. Segment descriptors in the GDT and LDT followed an 8-byte , specifying the (starting location of the segment), limit (up to 64 KB in size), and access rights. Access rights included the present bit (P) to indicate segment validity, type fields for code, , or segments, and the DPL for privilege checking. This structure allowed the processor to validate references dynamically, generating exceptions for violations such as invalid descriptors or privilege breaches. Despite these advances, the 80286's protected mode had notable limitations, including restriction to 16-bit operations without support for 32-bit extensions, absence of for running real-mode applications natively, and no built-in paging hardware for demand-paged . Despite these innovations, the 80286's protected mode saw limited adoption in practice, as it lacked support for running real-mode applications without rebooting and was incompatible with the dominant environment, leading most software to run in . Transitioning to protected mode required software initialization: loading the GDT into the GDTR register using the LGDT instruction, followed by setting the Protection Enable (PE) bit in the Machine Status Word (MSW) via the LMSW instruction at privilege level 0, and finally executing an intra-segment jump to flush the prefetch queue. This process ensured a clean switch but demanded careful setup to avoid faults during the mode change.

Intel 80386 Enhancements

The Intel 80386 , released in October 1985, introduced 32-bit protected mode, featuring 32-bit general-purpose registers such as EAX, EBX, ECX, and EDX, along with a 32-bit address bus that supported a 4 GB physical address space. This marked a significant evolution from the 16-bit protected mode of the , extending the latter's four privilege levels while enabling full 32-bit operations for enhanced performance and scalability in multitasking environments. Key additions included Virtual 8086 (V86) mode, which allowed real-mode 8086 applications to execute within protected mode under multitasking supervision, and a built-in paging unit for management. The segment descriptor format was enhanced to support 32-bit addressing, incorporating a granularity bit that permitted segment limits up to 4 GB when set. For compatibility, the 80386 provided Big Real Mode, an unprotected extension enabling 32-bit addressing in to access the full 4 GB space, and enhanced task state segments (TSS) that facilitated efficient context switching for multitasking by storing complete 32-bit processor states. Exclusive to the 80386 were structures like the Page Directory and Page Tables, which formed the basis for demand-paged by mapping 4 KB pages in a hierarchical manner. Additionally, the I/O Privilege Level (IOPL) bits in the EFLAGS register (bits 12 and 13) introduced granular control over I/O instructions, restricting sensitive operations to higher privilege levels and enhancing in protected environments. These features collectively enabled the development of full 32-bit operating systems, such as Microsoft's and 2.0, which leveraged the 80386's capabilities for robust, protected multitasking on personal computers.

Mode Switching

Entering Protected Mode

To enter protected mode on an x86 processor, specific prerequisites must be met to ensure a stable transition from . The (GDT) must be loaded into memory with at least a null descriptor, a descriptor, and a descriptor; these descriptors define the initial memory segments for code execution and data access in protected mode. A stack must also be established within a to handle any immediate subroutine calls or interrupts post-switch. Interrupts, including non-maskable interrupts (NMIs), should be disabled prior to the switch to prevent interference during the transition. The step-by-step procedure to switch to protected mode involves initializing segment registers to a flat memory model, loading the GDT, enabling the protection enable (PE) bit, and reloading the code segment register (CS). First, initialize the segment registers (DS, ES, SS, FS, GS) to point to a flat data segment, typically using a selector value like 0x10 for a 32-bit flat descriptor in the GDT; this ensures data access remains valid during the switch. Next, load the GDTR with the base address and size of the GDT using the LGDT instruction, which accepts a 6-byte pseudo-descriptor containing the 32-bit base address (24-bit for 80286) and 16-bit limit. Then, set the PE bit (bit 0) in the CR0 register to 1 using a MOV to CR0 instruction; this action serializes the processor and activates protected mode semantics. Finally, execute a far jump (JMP FAR) to a protected-mode code segment selector (e.g., 0x08 for a 32-bit code descriptor) and offset, which flushes the prefetch queue, reloads CS with the new selector, and begins execution in protected mode. After the jump, reinitialize the remaining segment registers if needed and re-enable interrupts. The implementation differs from the 80386 in several key aspects during entry. On the 80286, segments are limited to 16-bit operations, and the GDT base is 24 bits wide, requiring a hardware reset to exit protected mode once entered. In contrast, the 80386 supports 32-bit code and data segments (via the D/B flag in descriptors), a full 32-bit GDT base address, and software-based mode switching without reset, allowing larger address spaces and enabling paging if desired post-entry. These enhancements make the 80386 procedure more flexible for . Invalid configurations during entry can trigger exceptions, primarily general protection faults (#GP). For instance, attempting to set the PE bit without a valid GDT or using an invalid segment selector in the far jump causes a #GP(0) exception, halting the processor and requiring error handling via an interrupt service routine if one is already set up. Similarly, loading a null selector into CS or SS post-jump results in a #GP fault. Proper validation of descriptors and selectors is essential to avoid these faults. A minimal assembly example for entering protected mode on an 80386, assuming a pre-built GDT at 0x1000 with flat 32-bit (selector 0x08) and data (0x10) segments, illustrates the sequence:

cli ; Disable interrupts lgdt [gdt_descriptor] ; Load GDT (gdt_descriptor at 0x1000-0x1005) mov eax, cr0 or eax, 1 ; Set PE bit mov cr0, eax jmp 0x08:protected_mode ; Far jump to reload CS protected_mode: mov ax, 0x10 ; Data segment selector mov ds, ax mov es, ax mov fs, ax mov gs, ax mov ss, ax sti ; Re-enable interrupts

cli ; Disable interrupts lgdt [gdt_descriptor] ; Load GDT (gdt_descriptor at 0x1000-0x1005) mov eax, cr0 or eax, 1 ; Set PE bit mov cr0, eax jmp 0x08:protected_mode ; Far jump to reload CS protected_mode: mov ax, 0x10 ; Data segment selector mov ds, ax mov es, ax mov fs, ax mov gs, ax mov ss, ax sti ; Re-enable interrupts

This code performs the essential switch, transitioning to a flat 4 GB model.

Exiting Protected Mode

Exiting protected mode on x86 processors, particularly the 80386 and later, involves a careful sequence of operations to return the CPU to while ensuring compatibility with legacy services and avoiding system instability. This transition disables the protection enable (PE) bit in the CR0 register, flushes the instruction pipeline, and reinitializes segment registers and interrupt handling to mimic the 20-bit addressing and flat memory model of . The procedure requires prior setup of the () with real-mode-compatible descriptors (64 KB limits, byte granularity) to prevent invalid memory accesses during the switch. The step-by-step procedure for exiting protected mode is as follows: First, disable all interrupts to prevent asynchronous disruptions, including maskable interrupts via the CLI instruction and non-maskable interrupts (NMIs) through external masking or by ensuring a valid real-mode vector for NMI (IVT offset 0x08). Second, if paging is enabled, disable it by clearing the PG bit (bit 31) in CR0, zeroing CR3 to invalidate the page directory, and ensuring the CPU is executing from an identity-mapped physical address to avoid translation errors. Third, clear the PE bit (bit 0) in CR0 using a MOV instruction to disable protected mode addressing and protection checks. Fourth, perform a far jump to a real-mode code segment, typically to offset 0xFFF0 in segment 0xF000 (the conventional BIOS entry point), to flush the prefetch queue and reload the code segment register (CS) with real-mode values. Finally, reload the data segment (DS), extra segment (ES), stack segment (SS), FS, and GS registers with real-mode values (usually 0 for flat addressing), load the interrupt descriptor table register (IDTR) via LIDT to point to the real-mode interrupt vector table (IVT) at physical address 0, and re-enable interrupts with STI. The order of segment register reloads is critical: SS should be updated early to ensure a valid stack for any subsequent operations, followed by DS and ES to avoid data access faults. On the Intel 80386, additional considerations apply for safe exit. Paging must be disabled prior to clearing the PE bit, as active paging with protected mode disabled can lead to unpredictable address translations. In multitasking environments using task state segments (TSS), any active task switch must be completed or aborted by popping the task register (TR) and ensuring no pending task gates in the IDT before the transition, to prevent corruption of the task state. Risks during the exit process include potential s if the GDT is not properly configured with valid descriptors before clearing PE, as invalid segment references post-transition can trigger unhandled exceptions leading to a double fault and subsequent , causing a CPU reset. Post-exit, reliance on calls requires the IVT to be correctly initialized, as improper handling can hang the or corrupt . This mode-switching capability has been historically utilized in bootloaders like GRUB for hybrid operations, where is entered for efficient kernel loading but exited to for accessing disk and video services. A representative assembly example for a safe exit on the 80386, assuming paging is disabled and GDT is preset, emphasizes the segment reload sequence:

cli ; Disable maskable interrupts ; Assume NMI masked externally mov eax, cr0 and eax, 0x7FFFFFFE ; Clear PE bit (and PG if needed) mov cr0, eax jmp far 0xF000:0xFFF0 ; Far jump to flush pipeline, CS=0xF000, IP=0xFFF0 (BIOS reset vector) real_mode_cs: mov ax, 0 ; Real-mode segment value mov ds, ax ; Reload DS mov es, ax ; Reload ES mov ss, ax ; Reload SS (critical for stack) mov fs, ax ; Reload FS mov gs, ax ; Reload GS lidt [real_idt] ; Load IVT pointer (base 0, limit 0x3FF) sti ; Re-enable interrupts

cli ; Disable maskable interrupts ; Assume NMI masked externally mov eax, cr0 and eax, 0x7FFFFFFE ; Clear PE bit (and PG if needed) mov cr0, eax jmp far 0xF000:0xFFF0 ; Far jump to flush pipeline, CS=0xF000, IP=0xFFF0 (BIOS reset vector) real_mode_cs: mov ax, 0 ; Real-mode segment value mov ds, ax ; Reload DS mov es, ax ; Reload ES mov ss, ax ; Reload SS (critical for stack) mov fs, ax ; Reload FS mov gs, ax ; Reload GS lidt [real_idt] ; Load IVT pointer (base 0, limit 0x3FF) sti ; Re-enable interrupts

This sequence ensures a clean transition, with the far jump establishing the real-mode CS before other registers are adjusted to prevent faults.

Protection Mechanisms

Privilege Levels

Protected mode in the x86 employs a hierarchical privilege consisting of four rings, numbered 0 through 3, to enforce boundaries between different software components. Ring 0 represents the highest privilege level, typically reserved for operating kernel code with unrestricted access to hardware and resources. Rings 1 and 2 serve as intermediate levels for less trusted services, such as device drivers or executive modules, while Ring 3 is the lowest privilege level, designated for user applications with restricted access to prevent interference with critical operations. The privilege of executing code or accessing data is determined by two key fields: the Current Privilege Level (CPL), which indicates the privilege level of the currently running task and is stored in bits 0-1 of the (CS) register, and the Descriptor Privilege Level (DPL), a 2-bit field (bits 5-6 of the access rights byte) in segment descriptors that specifies the minimum privilege required to access the associated segment or gate. The DPL in segment descriptors acts as the primary enforcement mechanism for privilege checks. For nonconforming code segments, code executing at a given CPL can only load segments whose DPL equals the CPL, ensuring same-privilege direct execution. To invoke more privileged code (lower ring number), transitions must occur through controlled mechanisms like call gates, which validate the caller's CPL against the gate's DPL before allowing the switch. Data access follows similar rules, where a task at CPL can read or write data segments only if the CPL is less than or equal to the segment's DPL, and the requested privilege level (RPL) of the segment selector is also less than or equal to the DPL; violations result in a (#GP). For inter-ring calls to more privileged code using nonconforming segments, the processor performs a stack switch to a new stack segment at the target privilege level, loaded from the task state segment (TSS), and pushes the caller's stack pointer, flags, instruction pointer, and parameters onto it to maintain isolation and enable proper returns. Conforming code segments, marked in their descriptor, allow execution from any CPL greater than or equal to the DPL without a stack switch, facilitating code across privilege levels. Input/output (I/O) operations and handling further enforce privilege separation. I/O instructions from a non-zero ring require the current privilege level to be less than or equal to the I/O privilege level (IOPL) bits in the ; otherwise, a #GP fault occurs, with escalation typically to Ring 0 handlers. and exceptions always vector to Ring 0 via entries in the (IDT), using or trap gates that may disable interrupts to prevent nesting issues. In the implementation, I/O access is strictly limited to Ring 0 unless explicitly permitted via the task's I/O permission , with no dedicated IOPL field for finer Ring 3 control. The Intel 80386 enhances this by introducing IOPL bits that allow Ring 3 code to perform I/O operations directly if IOPL equals 3, providing more flexible control without always relying on bitmaps. For instance, a Ring 3 application attempting to access a Ring 0 without proper triggers a #GP fault, protecting kernel memory from user-mode corruption.

Memory Protection Fundamentals

In protected mode, the x86 employs hardware-enforced checks to prevent unauthorized access, ensuring isolation between , data, and tasks. These checks occur automatically during operations and include verification of bounds, access types, and segment or page presence, as defined in descriptors. Bounds checking confirms that the effective falls within the defined limits of the region, triggering a fault if exceeded. Type checking validates whether the operation (read, write, or execute) aligns with the region's permissions, such as restricting writes to executable segments. The present/not-present bit in descriptors further ensures that only loaded or mapped regions are accessible, with absent regions causing an immediate fault. Violations of these protection rules generate specific exceptions to allow the operating system to handle errors gracefully. The general protection fault (#GP) arises from bounds violations, invalid types, or privilege mismatches, providing an error code identifying the offending selector or operation. The not-present fault (#NP) specifically occurs when a descriptor's present bit is clear, indicating the memory region is not loaded into physical memory. On processors supporting paging, such as the 80386 and later, the page fault (#PF) handles similar issues at the page level, including not-present pages or permission violations, with an error code detailing the cause like writability or user/supervisor access. These faults are fully restartable, enabling precise recovery without . These mechanisms deliver key isolation benefits by enforcing per-task address spaces, where each operates within its allocated regions without interfering with others. This prevents common exploits like buffer overflows from propagating to adjacent areas, enhancing system stability and . Protection integrates with privilege levels (rings 0-3) to add layered enforcement, where access requires matching current privilege level (CPL) against descriptor privilege level (DPL), further restricting sensitive operations to higher-privilege code. In the , the original implementation of protected mode, these checks relied solely on segmentation for coarse-grained protection without paging support, limiting finer-grained isolation until the 80386's enhancements.

Memory Management

Segmentation

In protected mode, segmentation provides a mechanism for dividing the linear into variable-sized segments, each defined by a segment descriptor stored in either the (GDT) or a Local Descriptor Table (LDT). A segment selector, a 16-bit value loaded into one of the segment registers (CS, DS, ES, FS, GS, or SS), serves as an index into the GDT or LDT to retrieve the descriptor, which specifies the segment's base , size limit, and access rights such as readability, writability, and executability. The , consisting of the selector and an offset, is translated to a linear address by adding the offset to the segment base; the processor hardware automatically performs bounds checking to ensure the offset does not exceed the segment limit, generating a general-protection exception if it does. This approach contrasts sharply with segmentation, where segments are fixed at 64 KB and addressed via a simple left-shift of the segment value by 4 bits to form a base address, lacking hardware-enforced bounds or access controls. In protected mode, segments can vary in size without the 64 KB restriction, enabling more efficient utilization, and stack segments support expand-up (growing from low to high addresses) or expand-down (growing from high to low addresses) configurations to accommodate stack operations while respecting the limit. These features introduce robust protection against buffer overflows and unauthorized access, fundamental to the security model of protected mode. The implemented protected mode segmentation with 16-bit offsets, restricting each segment to a maximum size of 64 KB, which aligned offsets and addresses within this limit for compatibility with earlier designs but limited for larger programs. The Intel 80386 enhanced this by introducing 32-bit offsets, allowing segments up to 4 GB in size, and added a bit in the descriptor that, when set, scales the limit to units of 4 KB, permitting segment sizes from 4 KB to 4 GB in 4 KB increments for finer control over memory allocation. These improvements made segmentation more practical for multitasking environments and larger address spaces. Segmentation in protected mode supports various segment types essential for program execution and system operation: code segments hold executable instructions with attributes controlling conformity and readability; data segments manage read-write data areas; and stack segments handle push and pop operations with directionality flags. System segments, such as the Task State Segment (TSS), facilitate by storing processor state for context switching in multitasking scenarios. This segmented model, while flexible, is often combined with other mechanisms in to address memory needs comprehensively.

Paging

The Intel 80386 introduced paging as a key enhancement to its protected mode , providing a mechanism for virtual-to-physical address translation that was absent in the 80286, thereby enabling true support for multitasking operating systems. This paging unit operates on 32-bit linear addresses produced by segmentation, dividing the into fixed-size pages of 4 KB each, with support for larger 4 MB pages through specific page directory entry flags. Paging employs a two-level hierarchical structure: a Page Directory, which holds up to entries and serves as the top-level table, points to Page Tables; each Page Table also contains up to entries that map to individual 4 KB physical pages. The base physical address of the Page Directory is stored in the CR3 , allowing dynamic switching of address spaces during task changes. To enable paging, the PG bit (bit 31) in the CR0 register must be set after protected mode is activated (PE bit in CR0), which initiates for all subsequent accesses. A 32-bit linear address is split into three fields for translation: the upper 10 bits (31–22) index the Page Directory entry, the middle 10 bits (21–12) index the corresponding Page Table entry, and the lower 12 bits (11–0) provide the offset within the 4 KB page. The translation process begins by using CR3 to locate the Page Directory, then fetches the Page Table base address from the indexed directory entry (shifted left by 12 bits to align to a 4 KB boundary), followed by fetching the physical page frame base from the indexed table entry (also shifted left by 12 bits), and finally adding the offset to yield the physical address. For example, given a linear address of 0x402003, the directory index is 0x001 (bits 31–22), the table index is 0x002 (bits 21–12), and the offset is 0x003 (bits 11–0); the physical address is then computed as ((page frame base from table entry) << 12) + offset. Key features include demand paging, where pages are loaded into physical memory only upon access; this is controlled by the Present bit (P, bit 0) in page directory and table entries—if unset, a page fault exception (#PF) is generated to allow the operating system to handle paging in or out. Page-level protections are enforced via bits in the entries: the Read/Write bit (R/W, bit 1) distinguishes read-only from read-write access, and the User/Supervisor bit (U/S, bit 2) restricts access based on the current privilege level (CPL), with supervisor-mode (U/S=0) pages inaccessible from user mode (CPL>0). Effective permissions combine attributes from both the directory and table entries, providing granular control that complements segmentation. Performance is optimized by a Translation Lookaside Buffer (TLB), an on-chip four-way set-associative cache holding 32 recent translations (8 entries per set), which is flushed on CR3 reloads or task switches to ensure coherence. While the 80386 paging supports up to 4 GB of physical memory with 32-bit addresses, later processors introduced (PAE) starting with the to handle more than 4 GB by expanding to 36-bit physical addresses and adding a third level (page directory pointer table) to the hierarchy.

Segment Descriptors

Segment descriptors in the Intel 80386 protected mode are 8-byte entries stored in the (GDT) or local descriptor tables (LDT), defining the base address, size, and access attributes of memory segments. Each descriptor consists of a 32-bit base address, a 20-bit limit field, and various control bits for protection and usage. The structure is as follows:
ByteBitsField NameDescription
00-7Limit (bits 0-7)Lower 8 bits of the 20-bit segment limit; when the granularity bit (G) is 0, units are bytes (maximum 1 MB - 1); when G=1, units are 4 KB pages (maximum 4 GB - 1).
10-7Limit (bits 8-15)Upper 8 bits of the lower 16 bits of the segment limit.
20-7Base (bits 0-7)Lower 8 bits of the 32-bit base address, specifying the starting linear address of the segment.
30-7Base (bits 8-15)Next 8 bits of the 32-bit base address.
40-7Base (bits 16-23)Middle 8 bits of the 32-bit base address.
50-3Type4-bit type field defining the segment category and access rights (bits 40-43); combined with S bit for effective 5-bit type interpretation (detailed below).
4S (System)System flag: 0 for system segments, 1 for code/data segments (bit 44).
5-6DPL (Descriptor Privilege Level)Privilege level (0-3) for the descriptor, used in protection checks; 0 is the most privileged (bits 45-46).
7P (Present)Present bit: 1 indicates the segment is present in memory; 0 indicates it is not (bit 47).
60-3Limit (bits 16-19)Upper 4 bits of the 20-bit segment limit.
4AVL (Available)Available bit for use by system software (bit 52).
5ReservedMust be 0 (bit 53).
6D/B (Default operation size)Default size: 0 for 16-bit mode, 1 for 32-bit mode (bit 54).
7G (Granularity)Granularity bit: 0 for byte granularity, 1 for 4 KB granularity (bit 55).
70-7Base (bits 24-31)Upper 8 bits of the 32-bit base address.
The type field (combined with S for interpretation) distinguishes between code segments, data segments, and system segments. Code segments (type values 8h-Bh when S=1) support execution, with bit 1 indicating readability (e.g., type 0Ah for execute-only non-conforming, 0Bh for execute/read non-conforming). Data segments (type values 0h-7h when S=1) allow data access, with bit 1 for writability and bit 0 for expansion direction (e.g., type 0h for read-only expand-up, 2h for read/write expand-up, or 6h for read/write expand-down for stacks). System segments include types for task state segments (TSS, 9h available or 0Bh busy), local descriptor tables (LDT, 2h), and gates. Gate descriptors enable controlled inter-privilege-level transfers, such as call gates (type Ch for 80386 32-bit). Their format replaces the base and limit with a 32-bit offset to the target , a 16-bit target segment selector, and bits 0-3 of byte 6 specifying the parameter count (0-15 doublewords) to copy from the caller's stack to the target's stack during privilege transitions. The P, DPL, and type fields in byte 5 ensure the gate is present and enforces the required privilege for invocation. Segment descriptor tables are loaded using instructions like LGDT (load GDT register) or LLDT (load LDT register), which set the base and limit of the table in special registers (GDTR or LDTR). Individual descriptors are referenced via 16-bit selectors loaded into segment registers (CS, DS, ES, FS, GS, SS) using instructions such as MOV, far jumps, or calls, which cache the descriptor details for address translation. Compared to the Intel 80286, the 80386 segment descriptors expand the base address to 32 bits (from 24 bits, enabling 4 GB addressing versus 16 MB) and the limit to 20 bits with granularity support (up to 4 GB versus 1 MB fixed). The 80386 also introduces 32-bit offsets in gates and additional flags like D/B and G, absent in the 80286's 16-bit-oriented format.

Compatibility Features

Real Mode Application Support

Protected mode environments, designed primarily for advanced and , initially posed challenges for running legacy real-mode software, which operates under the simpler 8086 addressing model limited to 1 MB of memory. To bridge this gap, developers introduced DOS extenders, software mechanisms that enable 16-bit real-mode applications to leverage protected-mode features without full rewriting. These extenders temporarily switch the processor into protected mode for the application while maintaining compatibility with the underlying environment. A key standardization effort was the DOS Protected Mode Interface (DPMI), a specification released in 1989 that defines an API for 16-bit DOS programs to request protected-mode services, such as allocating extended memory beyond 1 MB and managing descriptors for larger address spaces. DPMI allows applications to execute in protected mode via function calls (interrupt 31h), providing access to up to 4 GB of memory on 80386 processors while intercepting and emulating real-mode DOS and BIOS services as needed. This interface became widely adopted for memory-intensive DOS applications, like games and utilities, by encapsulating protected-mode operations behind a real-mode-compatible facade. Another technique, known as big real mode or unreal mode, emerged as an unofficial but practical workaround on 80386 and later processors. By briefly entering protected mode to load segment descriptors with 32-bit base addresses and limits set to the maximum (0xFFFFFFFF), the system can then return to real mode while retaining these extended descriptors, effectively allowing 32-bit addressing and access to the full 4 GB address space without formal protection mechanisms. This mode lacks memory protection, making it vulnerable to crashes from invalid accesses, but it enabled early 32-bit DOS programs to run with minimal modifications under real-mode DOS. Intel's 1986 programmer's reference manual implicitly supported this behavior through descriptor manipulation, though it was not an intended operating mode. Historically, these techniques saw significant use in Windows 3.x, particularly in 386 mode, where DPMI served as the interface for running DOS applications alongside Windows sessions. Windows 3.0 formalized DPMI support, allowing real-mode DOS programs to request protected-mode execution for memory expansion, while the system managed mode switches to integrate them into the multitasking environment. However, limitations persisted, such as the absence of true multitasking for DOS apps, which ran in isolated sessions without preemptive scheduling, and reliance on cooperative interactions with the host OS. Key challenges in supporting real-mode applications within protected environments include handling interrupts and BIOS calls, which are inherently real-mode constructs incompatible with protected-mode segmentation and privilege checks. DOS extenders address this by implementing mode-switching routines, such as the int86 function family, to temporarily revert to for BIOS interrupts (e.g., for video services) before returning, incurring performance overhead from context switches and descriptor reloads. Failure to manage these transitions properly could lead to system instability or inaccessible hardware resources. With the shift to 64-bit operating systems like and later, native support for real-mode applications has declined sharply, as () omits hardware features like essential for direct 16-bit emulation. 64-bit Windows versions explicitly do not support 16-bit processes or components, rendering traditional DOS extenders obsolete and requiring software emulation tools for legacy real-mode software. serves as an advanced hardware alternative in 32-bit protected mode for more seamless integration.

Virtual 8086 Mode

Virtual 8086 mode, introduced with the 80386 processor, enables the execution of real-mode 8086/8088/80186/80188 software within a protected-mode environment, providing a mechanism for while maintaining and multitasking capabilities. This mode is activated by setting the VM bit (bit 17) in the EFLAGS register while the processor operates in protected mode (PE bit set in CR0), typically through methods such as an IRET instruction from ring 0 with VM=1 or a task switch to a Task State Segment (TSS) marked with VM=1. Once activated, each V86 task functions as a emulating the 8086 processor state, including 16-bit registers, segment-based addressing, and real-mode handling, including additional segment registers (FS and GS) and support for 80186 instructions such as BOUND, while restricted to the 8086 instruction set. Key features of Virtual 8086 mode include the virtualization of sensitive operations, where instructions for I/O (e.g., IN, OUT, INS, OUTS) and privileged actions (e.g., CLI, STI, HLT, INT n) generate exceptions that trap to a ring 0 Virtual 8086 monitor for emulation or direct hardware access, ensuring protected-mode supervision. Interrupts and exceptions in V86 mode vector through the protected-mode (IDT) to privilege level 0 handlers, which can reflect them back to the 8086 code or handle them natively, supporting seamless integration with the host operating system. Multiple V86 instances are supported on 80386 and later processors by enabling paging (PG bit in CR0), which allows distinct linear address spaces for each task, preventing interference while emulating the 8086's 20-bit addressing model (up to 1 MB). This paging integration maps V86 linear addresses—formed by shifting 16-bit selectors left by 4 bits and adding offsets—into the protected-mode virtual address space, enabling address wrapping emulation if needed to mimic 8086 behavior beyond 1 MB. In practice, Virtual 8086 mode facilitated the integration of legacy real-mode applications into protected-mode operating systems, such as in , where it powered "DOS boxes" by running programs in isolated virtual machines under the 80386's enhanced mode. However, the mode imposes limitations, including a per-task capped at 1 MB (with potential for 64 KB overflow in some configurations), reliance on paging for isolation which adds complexity, and performance overhead from frequent supervisor traps for I/O and interrupts, as all V86 tasks execute at current privilege level (CPL) 3 without direct access to hardware. Compatibility differences from true 8086 execution, such as altered instruction timings and exception behaviors (e.g., added page faults), may require OS-level adjustments. For instance, setting the I/O privilege level (IOPL) to 3 in the EFLAGS register permits V86 tasks running at ring 3 to perform I/O operations without trapping to the monitor, enabling user-level emulation in scenarios like application-hosted DOS environments, though lower IOPL values enforce stricter supervision.

Multitasking Support

Task Structures

In protected mode, multitasking is supported through hardware data structures that manage task states and facilitate switches between them. The primary structure is the Task State Segment (TSS), a system segment that holds the complete of a task, including processor registers and stack information, allowing the CPU to save and restore states efficiently during switches. Introduced with the , the TSS is a fixed-format segment with a minimum size of 44 bytes, containing fields such as the back link to the previous TSS, segment registers (CS, DS, ES, SS), general-purpose registers (AX, BX, CX, DX, SI, DI, , SP), the instruction pointer (IP), , and initial stack pointers and selectors for each privilege level (0 through 2). In the 80286, hardware task switching is initiated by a CALL or JMP instruction targeting a task selector, which references a TSS descriptor in the (GDT); this loads the new task's state from its TSS, saves the current task's state, and updates the processor's registers accordingly, enabling basic multitasking without software intervention for context management. The Intel 80386 enhanced the TSS to a minimum size of 104 bytes to accommodate 32-bit operations and additional features, incorporating expanded fields like 32-bit versions of the general-purpose registers (EAX, EBX, etc.), the extended instruction pointer (EIP), extended flags (EFLAGS), additional segment registers (FS, GS), the page directory base register (CR3) for per-task paging, the Local Descriptor Table (LDT) selector, and stack pointers and selectors per privilege ring (ESP0:ESP2, SS0:SS2) to support ring transitions during switches. An optional I/O permission bitmap at the end of the TSS allows fine-grained control over I/O port access for the task, with the bitmap's offset specified in the TSS itself. The Task Register (TR), a special 16-bit register, holds the selector for the current task's TSS descriptor in the GDT or LDT; it is loaded using the LTR instruction (restricted to privilege level 0) and automatically updated during task switches to point to the active TSS. The 80386 introduced the Nested Task (NT) flag in EFLAGS (bit 14), set during CALL-initiated switches to indicate nesting and enable return to the parent task via IRET, while JMP switches clear it for non-nested transitions; additionally, a busy bit in the TSS descriptor (changing type from 9 to B for available to busy) prevents reentrancy by triggering a if a busy TSS is referenced. For , the Descriptor Privilege Level (DPL) in the TSS descriptor (bits 5:6) specifies the minimum privilege required to access the TSS, ensuring that task switches are only permitted if the current privilege level (CPL) is at or below the TSS DPL, thus enforcing inter-task via privilege levels.

Context Switching

In protected mode, context switching enables multitasking by saving the processor state of the current task and restoring that of another, leveraging task state segments (TSS) to hold the complete execution context including registers, segment selectors, and stack pointers. This mechanism supports both inter-task and nested execution flows on x86 processors. Hardware context switching is performed using JMP, CALL, or IRET instructions that target a TSS selector in the (GDT). Upon execution, the processor automatically saves the current task's state—such as general-purpose registers, EFLAGS, EIP, and segment registers—into the old TSS, then loads the new task's state from its TSS into the processor registers. The task register (TR) is updated with the new TSS selector, and the busy flag in the TSS descriptor is atomically set using LOCK semantics to prevent concurrent access. This process ensures isolation between tasks without software intervention. Software context switching, in contrast, is managed entirely by the operating system kernel without hardware TSS involvement, offering greater control and efficiency. The kernel explicitly saves the current task's registers and state to a per-task using instructions like PUSH, POP, or MOV, then restores the next task's state similarly; floating-point and vector unit states are handled via FXSAVE/FXRSTOR or the XSAVE family of instructions. This method is typically invoked during scheduler decisions, such as interrupts for preemptive multitasking. Interrupt-driven context switching is triggered when an interrupt or exception vectors through a task gate in the (IDT), initiating a hardware task switch to a dedicated handler task. The processor saves the interrupted task's state to its TSS, loads the handler's state, and provides a fresh stack for execution, which enhances isolation for event handling; interrupts must often be disabled during the switch to avoid reentrancy. This approach integrates seamlessly with the IDT for asynchronous operations. With the introduction of the Intel 386 processor, task switching gained support for nesting via a back pointer field in the TSS, which stores the selector of the previous task's TSS during switches initiated by CALL or interrupts. When the nested task flag (NT) is set in EFLAGS, an IRET instruction uses this back pointer to return control to the prior task, automatically reloading its saved state and clearing the NT flag. This linkage facilitates chained task execution without manual stack management. Hardware task switching incurs higher latency compared to software methods due to TSS accesses, which often cause cache misses and TLB invalidations, making it less suitable for frequent switches in performance-critical systems. Consequently, favor software approaches for their speed and adaptability, reserving hardware mechanisms for specific isolation needs.

Operating Systems

Early Adopters

Earlier commercial operating systems to adopt protected mode included Microsoft Xenix around 1984 and Coherent in the mid-1980s, which utilized the 80286's protected mode for environments on PC-compatible hardware. One of the earliest widespread commercial operating systems to adopt protected mode was 1.x, jointly developed by and and released in December 1987. It represented a significant implementation of the 80286's protected mode for multitasking, enabling pre-emptive scheduling, multi-threading, and through a segmented while requiring at least a 286 processor. This allowed to support up to 16 MB of RAM and , marking a significant shift from the limitations of real-mode . In 1990, released , which incorporated protected mode in its standard mode for 80286-based systems, utilizing segment descriptors to enforce attributes such as read-only access and prevent errant pointers from corrupting other programs. This segmented approach addressed the 1 MB real-mode barrier, supporting up to 16 MB of physical memory, though it retained 16-bit limitations compared to the enhanced mode on 80386 processors. 's protected mode features facilitated better multitasking and application isolation, contributing to its commercial success with millions of installations. Among Unix variants, emerged as an early adopter, with its project established in the summer of 1989 to port (BSD) to the 80386 platform. It leveraged the 80386's protected mode for 32-bit addressing in a flat memory model, incorporating two-level paging with 4 KB pages to manage efficiently and support a 4 GB per process. While initial releases minimized segmentation for BSD compatibility, future plans included support to enable real-mode application execution within protected environments. Early protected mode adoption encountered significant challenges, including the limited availability and high cost of 80286 and 80386 hardware in the late 1980s, exacerbated by a 1988 RAM price crisis that made even 3 MB systems expensive for typical users. To bridge the gap for DOS-based applications, tools like Phar Lap's 286/ emerged around 1989, allowing protected mode execution on 286 systems by proxying DOS calls and providing access to , though mode-switching overhead limited its use in performance-sensitive software. A pivotal development came in 1993 with Microsoft's release of , which mandated an 80386 processor to deliver full 32-bit protected mode functionality, including advanced paging and portability across architectures, as part of a strategic shift from 16-bit systems to enterprise-grade . This requirement reflected Microsoft's focus on leveraging the 80386's capabilities for robust multitasking and , distinguishing NT from consumer-oriented Windows versions.

Modern Implementations

In contemporary x86 architectures, all CPUs initialize in upon power-on for with legacy firmware, requiring the to transition the processor to protected mode before enabling for 64-bit operation. This sequence ensures that protected mode serves as the foundational execution environment, even in 64-bit systems, where extends its capabilities with a larger while retaining core protection mechanisms like privilege rings and memory isolation. Modern operating systems predominantly adopt a flat memory model in protected mode, configuring segment registers to span the entire with minimal segmentation, thereby simplifying and relying primarily on paging for isolation and . This approach persists in 64-bit , where protected mode features remain essential for legacy support, such as running 32-bit applications in compatibility submode, underscoring the enduring role of protected mode as the baseline for x86 execution beyond . The kernel, powering modern Windows versions, operates exclusively in protected mode with a flat segmentation model augmented by paging to enforce strict separation between user-mode (ring 3) and kernel-mode (ring 0) address spaces, preventing user applications from directly accessing kernel memory or hardware. For legacy 16-bit application compatibility, Windows employs the NT Virtual DOS Machine (NTVDM) subsystem, which leverages within protected mode to emulate a DOS environment without compromising overall system isolation. In , the kernel utilizes protected mode for managing 32-bit processes on 64-bit systems via long mode's compatibility submode, where the code segment selector (CS=0x08) designates 32-bit execution while maintaining protected mode's privilege levels and paging for . This setup allows seamless execution of legacy binaries under the 64-bit kernel, building directly on protected mode foundations extended by long mode's 64-bit registers and addressing. macOS, based on the kernel, minimizes reliance on protected mode segmentation in favor of paging for management, implementing a flat model where page tables handle user-kernel separation and protection across its hybrid Unix foundation. To enhance kernel security, macOS incorporates Intel's Supervisor Mode Execution Prevention (SMEP, introduced in 2012) and (SMAP, introduced in 2013), which prevent the kernel from executing or accessing user-mode memory pages, thereby mitigating common exploit techniques like and data leaks.

References

  1. https://cdrdv2-public.intel.com/671447/325384-sdm-vol-3abcd.pdf
  2. https://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-vol-1-manual.pdf
  3. http://bitsavers.trailing-edge.com/components/intel/80286/210498-005_80286_and_80287_Programmers_Reference_Manual_1987.pdf
  4. https://timeline.intel.com/1978/kicking-off-the-80286
  5. http://www.os2museum.com/wp/theres-more-to-the-286-xenix-story/
  6. http://bitsavers.informatik.uni-stuttgart.de/components/intel/80286/210498-001_iAPX_286_Programmers_Reference_1983.pdf
  7. http://flint.cs.yale.edu/feng/cos/resources/BIOS/procModes.htm
  8. https://timeline.intel.com/1985/raising-the-bar-with-the-386
  9. https://bitsavers.computerhistory.org/components/intel/80386/230985-003_386DX_Microprocessor_Programmers_Reference_Manual_1990.pdf
  10. https://www.osnews.com/story/129268/the-intel-80386-and-windows-nt-part-1-introduction/
  11. https://landley.net/history/mirror/os2/OS2History.html
  12. https://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-vol-3a-part-1-manual.pdf
  13. https://pdos.csail.mit.edu/6.828/2018/readings/i386.pdf
  14. https://docs.pcjs.org/manuals/intel/80386/80386_Returning_To_Real_Mode-1986-01-08.pdf
  15. https://www.gnu.org/software/grub/manual/grub/grub.html
  16. https://cdrdv2-public.intel.com/843836/325384-sdm-vol-3abcd-dec-24.pdf
  17. https://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-vol-3c-part-3-manual.pdf
  18. http://sudleyplace.com/dpmione/dpmispec1.0.pdf
  19. https://www.delorie.com/djgpp/doc/dpmi/
  20. https://www.os2museum.com/wp/a-brief-history-of-unreal-mode/
  21. https://devblogs.microsoft.com/oldnewthing/20230829-00/?p=108661
  22. https://www.delorie.com/djgpp/doc/ug/interrupts/inthandlers2.html
  23. https://learn.microsoft.com/en-us/troubleshoot/windows-client/application-management/x64-windows-not-support-16-bit-programs
  24. https://learn.microsoft.com/en-us/archive/msdn-magazine/2000/july/under-the-hood-happy-10th-anniversary-windows
  25. https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/tr-94-06.pdf
  26. https://bitsavers.trailing-edge.com/components/intel/80386/231746-001_Introduction_to_the_80386_Apr86.pdf
  27. https://cdrdv2-public.intel.com/812386/253668-sdm-vol-3a.pdf
  28. https://www.os2museum.com/wp/os2-history/os2-1-0/
  29. https://386bsd.org/releases/porting-unix-to-the-386-a-practical-approach-designing-the-software-specification-article
  30. https://arstechnica.com/information-technology/2019/11/half-an-operating-system-the-triumph-and-tragedy-of-os2/
  31. https://virtuallyfun.com/2024/01/29/phar-laps-286-dos-extender-why-nobody-used-it-for-games/
  32. https://www.abortretry.fail/p/the-history-of-windows-nt-31
  33. https://wiki.osdev.org/Setting_Up_Long_Mode
  34. https://www.intel.com/content/www/us/en/developer/articles/technical/envisioning-future-simplified-architecture.html
  35. https://wiki.osdev.org/X86-64
  36. https://learn.microsoft.com/en-us/windows/compatibility/ntvdm-and-16-bit-app-support
  37. https://tldp.org/HOWTO/html_single/Linux-i386-Boot-Code-HOWTO/
  38. https://developer.apple.com/library/archive/documentation/Darwin/Conceptual/KernelProgramming/vm/vm.html
  39. https://edc.intel.com/content/www/us/en/design/products-and-solutions/processors-and-chipsets/core-ultra-200h-and-200u-series-processors-datasheet-volume-1-of-2/intel-supervisor-mode-access-prevention-intel-smap/
Add your contribution
Related Hubs
User Avatar
No comments yet.