R10000

Described in press reports during 1992 as MIPS Computer Systems' "next generation RISC part", the R10000 was originally scheduled for delivery in 1994 and intended to form the basis of a family of products, each of which requiring substantially less development effort to bring to market than the core technology itself while incrementally improving performance.^[1] During 1993, MIPS Technologies sought to bring its licencees' resources to bear on the design of the R10000 and related products, with MIPS, Integrated Device Technology, LSI Logic, NEC, Performance Semiconductor, Siemens and Toshiba collectively planning to invest a reported $150 million in this next generation.^[2] The R10000 was eventually unveiled in October 1994 by MIPS and its silicon partners NEC and Toshiba, emphasising the product's potential applications in multiprocessing and massively parallel configurations, and claiming a projected performance of 300 SPECint92, 600 SPECfp92 at 200 MHz.^[3]

The R10000 was introduced in July 1995 at clock frequencies of 175 MHz and 195 MHz. A 150 MHz version was introduced in the O2 product line in 1997, but discontinued shortly after due to customer preference for the 175 MHz version. The R10000 was not available in large volumes until later in the year due to fabrication problems at MIPS's foundries. The 195 MHz version was in short supply throughout 1996, and was priced at US$3,000 as a result.^[4]

On 25 September 1996, SGI announced that R10000s fabricated by NEC between March and the end of July that year were faulty, drawing too much current and causing systems to shut down during operation. SGI recalled 10,000 R10000s that had shipped in systems as a result, which impacted the company's earnings.

In 1997, a version of R10000 fabricated in a 0.25 μm process enabled the microprocessor to reach 250 MHz.

Users of the R10000 include:

• SGI:
  • Workstations: Indigo2 (IMPACT Generation), Octane, O2
  • Servers: Challenge, Origin 2000
  • Supercomputers: Onyx, Onyx2
• NEC, in its Cenju-4 supercomputer
• Siemens Nixdorf, in its servers run under SINIX
• Tandem Computers, in its Himalaya fault-tolerant servers

The R10000 is a four-way superscalar design that implements register renaming and executes instructions out-of-order. Its design is a departure from previous MTI microprocessors such as the R4000, which is a much simpler scalar in-order design that relies largely on high clock rates for performance.

The R10000 fetches four instructions every cycle from its instruction cache. These instructions are decoded and then placed into the integer, floating-point or load/store instruction queues depending on the type of the instruction. The decode unit is assisted by the pre-decoded instructions from the instruction cache, which append five bits to every instruction to enable the unit to quickly identify which execution unit the instruction is executed in, and rearrange the format of the instruction to optimize the decode process.

Each of the instruction queues can accept up to four instructions from the decoder, avoiding any bottlenecks. The instruction queues issue their instructions to their execution units dynamically depending on the availability of operands and resources. Each of the queues except for the load/store queue can issue up to two instructions every cycle to its execution units. The load/store queue can only issue one instruction. The R10000 can thus issue up to five instructions every cycle.

The integer unit consists of the integer register file and three pipelines, two integer, one load store. The integer register file is 64 bits wide and contains 64 entries, of which 32 are architectural registers and 32 are rename registers which implement register renaming. The register file has seven read ports and three write ports. Both integer pipelines have an adder and a logic unit. However, only the first pipeline has a barrel shifter and hardware for confirming the prediction of conditional branches. The second pipeline is used to access the multiplier and divider. Multiplies are pipelined, and have a six-cycle latency for 32-bit integers and ten for 64-bit integers. Division is not pipelined. The divider uses a non-restoring algorithm that produces one bit per cycle. Latencies for 32-bit and 64-bit divides are 35 and 67 cycles, respectively.

The floating-point unit (FPU) consists of four functional units, an adder, a multiplier, divide unit and square root unit. The adder and multiplier are pipelined, but the divide and square root units are not. Adds and multiplies have a latency of three cycles and the adder and multiplier can accept a new instruction every cycle. The divide unit has a 12- or 19-cycle latency, depending on whether the divide is single precision or double precision, respectively.

The square root unit executes square root and reciprocal square root instructions. Square root instructions have an 18- or 33-cycle latency for single precision or double precision, respectively. A new square root instruction can be issued to the divide unit every 20 or 35 cycles for single precision and double precision respectively. Reciprocal square roots have longer latencies, 30 to 52 cycles for single precision (32-bit) and double precision (64-bit) respectively.

The floating-point register file contains sixty-four 64-bit registers, of which thirty-two are architectural and the remaining are rename registers. The adder has its own dedicated read and write ports, whereas the multiplier shares its with the divider and square root unit.

The divide and square root units use the SRT algorithm. The MIPS IV ISA has a multiply–add instruction. This instruction is implemented by the R10000 with a bypass — the result of the multiply can bypass the register file and be delivered to the add pipeline as an operand, thus it is not a fused multiply–add, and has a four-cycle latency.

The R10000 has two comparatively large on-chip caches, a 32 KB instruction cache and a 32 KB data cache. The instruction cache is two-way set-associative and has a 128-byte line size. Instructions are partially decoded by appending four bits to each instruction (which have a length of 32 bits) before they are placed in the cache.

The 32 KB data cache is dual-ported through two-way interleaving. It consists of two 16 KB banks, and each bank are two-way set-associative. The cache has 64-byte lines, uses the write-back protocol, and is virtually indexed and physically tagged to enable the cache to be indexed in the same clock cycle and to maintain coherency with the secondary cache.

The external secondary unified cache supported capacities between 512 KB and 16 MB. It is implemented with commodity synchronous static random access memory (SSRAM). The cache is accessed via its own 128-bit bus that is protected by 9-bits of error correcting code (ECC). The cache and bus operate at the same clock rate as the R10000, whose maximum frequency was 200 MHz. At 200 MHz, the bus yielded a peak bandwidth of 3.2 GB/s. The cache is two-way set associative, but to avoid a high pin count, the R10000 predicts which way is accessed.

MIPS IV is a 64-bit architecture, but to reduce cost the R10000 does not implement the entire physical or virtual address. Instead, it has a 40-bit physical address and a 44-bit virtual address, thus it is capable of addressing 1 TB of physical memory and 16 TB of virtual memory.

The R10000 uses the Avalanche bus, a 64-bit bus that operates at frequencies up to 100 MHz. Avalanche is a multiplexed address and data bus, so at 100 MHz it yields a maximum theoretical bandwidth of 800 MB/s, but its peak bandwidth is 640 MB/s as it requires some cycles to transmit addresses.

The system interface controller supports glue-less symmetrical multiprocessing (SMP) of up to four microprocessors. Systems using the R10000 with external logic can scale to hundreds of processors. An example of such a system is the Origin 2000.

The R10000 consists of approximately 6.8 million transistors, of which approximately 4.4 million are contained in the primary caches.^[5] The die measures 16.640 by 17.934 mm, for a die area of 298.422 mm². It is fabricated in a 0.35 μm process and packaged in 599-pad ceramic land grid array (LGA). Before the R10000 was introduced, the Microprocessor Report, covering the 1994 Microprocessor Forum, reported that it was packaged in a 527-pin ceramic pin grid array (CPGA); and that vendors also investigated the possibility of using a 339-pin multi-chip module (MCM) containing the microprocessor die and 1 MB of cache.^[6]

The R10000 was extended by multiple successive derivatives. All derivatives after the R12000 have their clock frequency kept as low as possible to maintain power dissipation in the 15 to 20 W range so they can be densely packaged in SGI's high performance computing (HPC) systems.

The R12000 is a derivative of the R10000 started by MIPS and completed by SGI. It was fabricated by NEC and Toshiba. The version fabricated by NEC is called the VR12000. The microprocessor was introduced in November 1998. It is available at 270, 300 and 360 MHz. The R12000 was developed as a stop-gap solution following the cancellation of the "Beast" project, which intended to deliver a successor to the R10000. R12000 users include NEC, Siemens-Nixdorf, SGI and Tandem Computers (and later Compaq, after their acquisition of Tandem).

The R12000 improves upon the R10000 microarchitecture by: inserting an extra pipeline stage to improve clock frequency by resolving a critical path; increasing the number of entries in the branch history table, improving prediction; modifying the instruction queues so they take into account the age of a queued instruction, enabling older instructions to be executed before newer ones if possible.

The R12000 was fabricated by NEC and Toshiba in a 0.25 μm CMOS process with four levels of aluminum interconnect. The use of a new process does not mean that the R12000 was a simple die shrink with a tweaked microarchitecture; the layout of the die is optimized to take advantage of the 0.25 μm process.^[7][8] The NEC fabricated VR12000 contained 7.15 million transistors and measured 15.7 by 14.6 mm (229.22 mm²).

The R12000A is a derivative of the R12000 developed by SGI. Introduced in July 2000, it operates at 400 MHz and was fabricated by NEC a 0.18 μm process with aluminum interconnects.

The R14000 is a further development of the R12000 announced in July 2001. The R14000 operates at 500 MHz, enabled by the 0.13 μm CMOS process with five levels of copper interconnect it is fabricated with. It features improvements to the microarchitecture of the R12000 by supporting double data rate (DDR) SSRAMs for the secondary cache and a 200 MHz system bus.^[9]

The R14000A is a further development of the R14000 announced in February 2002. It operates at 600 MHz, dissipates approximately 17 W, and was fabricated by NEC Corporation in a 0.13 μm CMOS process with seven levels of copper interconnect.^[9]

The R16000, code-named "N0", is the last derivative of the R10000. It is developed by SGI and fabricated by NEC in their 0.11 μm process with eight levels of copper interconnect. The microprocessor was introduced on 9 January 2003, debuting at 700 MHz for the Fuel and also used in their Onyx4 Ultimate Vision.^[10] In April 2003, a 600 MHz version was introduced for the Origin 350. Improvements are 64 KB instruction and data caches.

The R16000A refers to R16000 microprocessors with clock rates higher than 700 MHz. The first R16000A is an 800 MHz version, introduced on 4 February 2004. Later, a 900 MHz version was introduced, and this version was, for some time, the fastest publicly known R16000A—SGI later revealed there were 1.0 GHz R16000s shipped to selected customers. R16000 users included HP and SGI. SGI used the microprocessor in their Fuel and Tezro workstations; and the Origin 3000 servers and supercomputers. HP used the R16000A in their NonStop Himalaya S-Series fault-tolerant servers inherited from Tandem via Compaq.

The R18000 is a canceled further development of the R10000 microarchitecture that featured major improvements by Silicon Graphics, Inc. described at the Hot Chips symposium in 2001. The R18000 was designed specifically for SGI's ccNUMA servers and supercomputers. Each node would have two R18000s connected via a multiplexed bus to a system controller, which would interface the microprocessors to their local memory and the rest of the system via a hypercube network.

The R18000 improved the floating-point instruction queues and revised the floating-point unit to feature two multiply–add units, quadrupling the peak FLOPS count. Division and square-root would be performed in separate non-pipelined units in parallel to the multiply–add units. The system interface and memory hierarchy was also significantly reworked. It would have a 52-bit virtual address and a 48-bit physical address. The bidirectional multiplexed address and data system bus of the earlier models would be replaced by two unidirectional DDR links, a 64-bit multiplexed address and write path and a 128-bit read path. The paths could be shared with another R18000 through multiplexing. The bus could also be configured in the SysAD or Avalanche configuration for backwards compatibility with R10000 systems.

The R18000 would have a 1 MB four-way set-associative secondary cache to be included on-die; supplemented by an optional tertiary cache built from single data rate (SDR) or double data rate (DDR) SSRAM or DDR SDRAM with capacities of 2 to 64 MB. The L3 cache would have its cache tags, equivalent to 400 KB, located on-die to reduce latency. The L3 cache would be accessed via a 144-bit bus, of which 128 bits are for data and 16 bits for ECC. The L3 cache's clock rate would be programmable.

The R18000 was to be fabricated in NEC's UX5 process, a 0.13 μm CMOS process with nine levels of copper interconnect. It would have used 1.2 V power supply and dissipated less heat than contemporary server microprocessors in order to be densely packed into systems.