Recent from talks
Nothing was collected or created yet.
Head-mounted display
View on Wikipedia

A head-mounted display (HMD) is a display device, worn on the head or as part of a helmet, that has a small display optic in front of one (monocular HMD) or each eye (binocular HMD). HMDs have many uses including gaming, aviation, engineering, and medicine.[1]
Virtual reality headsets are a type of HMD that track 3D position and rotation to provide a virtual environment to the user. 3DOF VR headsets typically use an IMU for tracking. 6DOF VR headsets typically use sensor fusion from multiple data sources including at least one IMU.
An optical head-mounted display (OHMD) is a wearable display that can reflect projected images and allows a user to see through it.[2]
Overview
[edit]A typical HMD has one or two small displays, with lenses and semi-transparent mirrors embedded in eyeglasses (also termed data glasses), a visor, or a helmet. The display units are miniaturized and may include cathode-ray tubes (CRT), liquid-crystal displays (LCDs), liquid crystal on silicon (LCoS), or organic light-emitting diodes (OLED). Some vendors employ multiple micro-displays to increase total resolution and field of view.
HMDs differ in whether they can display only computer-generated imagery (CGI), or only live imagery from the physical world, or combination. Most HMDs can display only a computer-generated image, sometimes referred to as virtual image. Some HMDs can allow a CGI to be superimposed on real-world view. This is sometimes referred to as augmented reality (AR) or mixed reality (MR). Combining real-world view with CGI can be done by projecting the CGI through a partially reflective mirror and viewing the real world directly. This method is often called optical see-through. Combining real-world view with CGI can also be done electronically by accepting video from a camera and mixing it electronically with CGI.
By using AR technology, the HMDs are allowed to achieve a see-through display. By using virtual reality (VR) technology, the HMDs can realize viewing the images in 360 degrees.[3]
Optical HMD
[edit]An optical head-mounted display uses an optical mixer which is made of partly silvered mirrors. It can reflect artificial images, and let real images cross the lens, and let a user look through it. Various methods have existed for see-through HMD's, most of which can be summarized into two main families based on curved mirrors or waveguides. Curved mirrors have been used by Laster Technologies, and by Vuzix in their Star 1200 product. Various waveguide methods have existed for years. These include diffraction optics, holographic optics, polarized optics, and reflective optics.
Applications
[edit]Major HMD applications include military, government (fire, police, etc.), and civilian-commercial (medicine, video gaming, sports, etc.).
Aviation and tactical, ground
[edit]
In 1962, Hughes Aircraft Company revealed the Electrocular, a compact CRT (7" long), head-mounted monocular display that reflected a TV signal in to transparent eyepiece.[4][5][6][7] Ruggedized HMDs are increasingly being integrated into the cockpits of modern helicopters and fighter aircraft. These are usually fully integrated with the pilot's flying helmet and may include protective visors, night vision devices, and displays of other symbology.
Military, police, and firefighters use HMDs to display tactical information such as maps or thermal imaging data while viewing a real scene. Recent applications have included the use of HMD for paratroopers.[8] In 2005, the Liteye HMD was introduced for ground combat troops as a rugged, waterproof lightweight display that clips into a standard U.S. PVS-14 military helmet mount. The self-contained color monocular organic light-emitting diode (OLED) display replaces the NVG tube and connects to a mobile computing device. The LE has see-through ability and can be used as a standard HMD or for augmented reality applications. The design is optimized to provide high definition data under all lighting conditions, in covered or see-through modes of operation. The LE has a low power consumption, operating on four AA batteries for 35 hours or receiving power via standard Universal Serial Bus (USB) connection.[9]
The Defense Advanced Research Projects Agency (DARPA) continues to fund research in augmented reality HMDs as part of the Persistent Close Air Support (PCAS) Program. Vuzix is currently working on a system for PCAS that will use holographic waveguides to produce see-through augmented reality glasses that are only a few millimeters thick.[10]
Engineering
[edit]Engineers and scientists use HMDs to provide stereoscopic views of computer-aided design (CAD) schematics.[11] Virtual reality, when applied to engineering and design, is a key factor in integration of the human in the design. By enabling engineers to interact with their designs in full life-size scale, products can be validated for issues that may not have been visible until physical prototyping. The use of HMDs for VR is seen as supplemental to the conventional use of CAVE for VR simulation. HMDs are predominantly used for single-person interaction with the design, while CAVEs allow for more collaborative virtual reality sessions.
Head Mounted Display systems are also used in the maintenance of complex systems, as they can give a technician a simulated x-ray vision by combining computer graphics such as system diagrams and imagery with the technician's natural vision (augmented or modified reality).
Medicine and research
[edit]There are also applications in surgery, wherein a combination of radiographic data (X-ray computed tomography (CAT) scans, and magnetic resonance imaging (MRI) imaging) is combined with the surgeon's natural view of the operation, and anesthesia, where the patient vital signs are within the anesthesiologist's field of view at all times.[12]
Research universities often use HMDs to conduct studies related to vision, balance, cognition and neuroscience. As of 2010, the use of predictive visual tracking measurement to identify mild traumatic brain injury was being studied. In visual tracking tests, a HMD unit with eye tracking ability shows an object moving in a regular pattern. People without brain injury are able to track the moving object with smooth pursuit eye movements and correct trajectory.[13]
Gaming and video
[edit]
Low-cost HMD devices are available for use with 3D games and entertainment applications. One of the first commercially available HMDs was the Forte VFX1 which was announced at Consumer Electronics Show (CES) in 1994.[14] The VFX-1 had stereoscopic displays, 3-axis head-tracking, and stereo headphones. Another pioneer in this field was Sony, which released the Glasstron in 1997. It had as an optional accessory a positional sensor which permitted the user to view the surroundings, with the perspective moving as the head moved, providing a deep sense of immersion. One novel application of this technology was in the game MechWarrior 2, which permitted users of the Sony Glasstron or Virtual I/O's iGlasses to adopt a new visual perspective from inside the cockpit of the craft, using their own eyes as visual and seeing the battlefield through their craft's own cockpit.
Many brands of video glasses can be connected to modern video and DSLR cameras, making them applicable as a new age monitor. As a result of the glasses ability to block out ambient light, filmmakers and photographers are able to see clearer presentations of their live images.[15]
The Oculus Rift is a virtual reality (VR) head-mounted display created by Palmer Luckey that the company Oculus VR developed for virtual reality simulations and video games.[16] The HTC Vive is a virtual reality head-mounted display. The headset is produced by a collaboration between Valve and HTC, with its defining feature being precision room-scale tracking, and high-precision motion controllers. The PlayStation VR is a virtual reality headset for gaming consoles, dedicated for the PlayStation 4.[17] Windows Mixed Reality is a platform developed by Microsoft which includes a wide range of headsets produced by HP, Samsung, and others and is capable of playing most HTC Vive games. It uses only inside-out tracking for its controllers.
Virtual cinema
[edit]Some head-mounted displays are designed to present traditional video and film content in a virtual cinema. These devices typically feature a relatively narrow field of view (FOV) of 50–60°, making them less immersive than virtual-reality headsets, but they offer correspondingly higher resolution in terms of pixels per degree. Released in 2011, the Sony HMZ-T1 featured 1280x720 resolution per eye. In approximately 2015, standalone Android 5 (Lollipop) based "private cinema" products were released using various brands such as VRWorld, Magicsee, based on software from Nibiru.
Products released as of 2020 featuring 1920×1080 resolution per eye included the Goovis G2[18] and Royole Moon.[19] Also available was the Avegant Glyph,[20] which incorporated 720P retinal projection per eye, and the Cinera Prime,[21] which featured 2560×1440 resolution per eye as well as a 66° FOV. The rather large Cinera Prime used either a standard support arm or an optional head mount. Expected to be available in late-2021 was the Cinera Edge,[22] featuring the same FOV and 2560×1440 resolution per eye as the earlier Cinera Prime model, but with a much more compact form factor. Other products available in 2021 were the Cinemizer OLED,[23] with 870×500 resolution per eye, the VISIONHMD Bigeyes H1,[24] with 1280x720 resolution per eye, and the Dream Glass 4K,[25] with 1920x1080 resolution per eye. All of the products mentioned here incorporated audio headphones or earphones except for the Goovis G2, the Cinera Prime, the VISIONHMD Bigeyes H1, and the Dream Glass 4K, which instead offered an audio headphones jack.
Remote control
[edit]
First-person view (FPV) drone flying uses head-mounted displays which are commonly called "FPV goggles".[26][27] Analog FPV goggles (such as the ones produced by Fat Shark) are commonly used for drone racing as they offer the lowest video latency. But digital FPV goggles (such as produced by DJI) are becoming increasingly popular due to their higher resolution video.
Since 2010s, FPV drone flying is widely used in aerial cinematography and aerial photography.[28]
Sports
[edit]A HMD system has been developed for Formula One drivers by Kopin Corp. and the BMW Group. The HMD displays critical race data while allowing the driver to continue focusing on the track as pit crews control the data and messages sent to their drivers through two-way radio.[29] Recon Instruments released on 3 November 2011 two head-mounted displays for ski goggles, MOD and MOD Live, the latter based on an Android operating system.[30]
Training and simulation
[edit]A key application for HMDs is training and simulation, allowing to virtually place a trainee in a situation that is either too expensive or too dangerous to replicate in real-life. Training with HMDs covers a wide range of applications from driving, welding and spray painting, flight and vehicle simulators, dismounted soldier training, medical procedure training, and more. However, a number of unwanted symptoms have been caused by prolonged use of certain types of head-mounted displays, and these issues must be resolved before optimal training and simulation is feasible.[31]
Performance parameters
[edit]- Ability to show stereoscopic imagery. A binocular HMD has the potential to display a different image to each eye. This can be used to show stereoscopic images. It should be borne in mind that so-called 'Optical Infinity' is generally taken by flight surgeons and display experts as about 9 meters. This is the distance at which, given the average human eye rangefinder "baseline" (distance between the eyes or Interpupillary distance (IPD)) of between 2.5 and 3 inches (6 and 8 cm), the angle of an object at that distance becomes essentially the same from each eye. At smaller ranges the perspective from each eye is significantly different and the expense of generating two different visual channels through the computer-generated imagery (CGI) system becomes worthwhile.
- Interpupillary distance (IPD). This is the distance between the two eyes, measured at the pupils, and is important in designing head-mounted displays.
- Field of view (FOV) – Humans have an FOV of around 180°, but most HMDs offer far less than this. Typically, a greater field of view results in a greater sense of immersion and better situational awareness. Most people do not have a good feel for what a particular quoted FOV would look like (e.g., 25°) so often manufacturers will quote an apparent screen size. Most people sit about 60 cm away from their monitors and have quite a good feel about screen sizes at that distance. To convert the manufacturer's apparent screen size to a desktop monitor position, divide the screen size by the distance in feet, then multiply by 2. Consumer-level HMDs typically offer a FOV of about 110°.
- Resolution – HMDs usually mention either the total number of pixels or the number of pixels per degree. Listing the total number of pixels (e.g., 1600×1200 pixels per eye) is borrowed from how the specifications of computer monitors are presented. However, the pixel density, usually specified in pixels per degree or in arcminutes per pixel, is also used to determine visual acuity. 60 pixels/° (1 arcmin/pixel) is usually referred to as eye limiting resolution, above which increased resolution is not noticed by people with normal vision. HMDs typically offer 10 to 20 pixels/°, though advances in micro-displays help increase this number.
- Binocular overlap – measures the area that is common to both eyes. Binocular overlap is the basis for the sense of depth and stereo, allowing humans to sense which objects are near and which objects are far. Humans have a binocular overlap of about 100° (50° to the left of the nose and 50° to the right). The larger the binocular overlap offered by an HMD, the greater the sense of stereo. Overlap is sometimes specified in degrees (e.g., 74°) or as a percentage indicating how much of the visual field of each eye is common to the other eye.
- Accommodation support – HMDs that support matching the accommodation and vergence distances of the eyes are more comfortable than those that do not.[32]
- Distant focus (collimation). Optical methods may be used to present the images at a distant focus, which seems to improve the realism of images that in the real world would be at a distance.
- On-board processing and operating system. Some HMD vendors offer on-board operating systems such as Android, allowing applications to run locally on the HMD, and eliminating the need to be tethered to an external device to generate video. These are sometimes referred to as smart goggles. To make the HMD construction lighter producers may move the processing system to connected smart necklace form-factor that would also offer the additional benefit of larger battery pack. Such solution would allow to design lite HMD with sufficient energy supply for dual video inputs or higher frequency time-based multiplexing (see below).
Support of 3D video formats
[edit]

Depth perception inside an HMD requires different images for the left and right eyes. There are multiple ways to provide these separate images:
- Use dual video inputs, thereby providing a completely separate video signal to each eye
- Time-based multiplexing. Methods such as frame sequential combine two separate video signals into one signal by alternating the left and right images in successive frames.
- Side by side or top-bottom multiplexing. This method allocated half of the image to the left eye and the other half of the image to the right eye.
The advantage of dual video inputs is that it provides the maximum resolution for each image and the maximum frame rate for each eye. The disadvantage of dual video inputs is that it requires separate video outputs and cables from the device generating the content.
Time-based multiplexing preserves the full resolution per each image, but reduces the frame rate by half. For example, if the signal is presented at 60 Hz, each eye is receiving just 30 Hz updates. This may become an issue with accurately presenting fast-moving images.
Side-by-side and top-bottom multiplexing provide full-rate updates to each eye, but reduce the resolution presented to each eye. Many 3D broadcasts, such as ESPN, chose to provide side-by-side 3D which saves the need to allocate extra transmission bandwidth and is more suitable to fast-paced sports action relative to time-based multiplexing methods.
Not all HMDs provide depth perception. Some lower-end modules are essentially bi-ocular devices where both eyes are presented with the same image. 3D video players sometimes allow maximum compatibility with HMDs by providing the user with a choice of the 3D format to be used.
Peripherals
[edit]- The most rudimentary HMDs simply project an image or symbology on a wearer's visor or reticle. The image is not bound to the real world, i.e., the image does not change based on the wearer's head position.
- More sophisticated HMDs incorporate a positioning system that tracks the wearer's head position and angle, so that the picture or symbol displayed is congruent with the outside world using see-through imagery.
- Head tracking – Binding the imagery. Head-mounted displays may also be used with tracking sensors that detect changes of angle and orientation. When such data is available in the system computer, it can be used to generate the appropriate computer-generated imagery (CGI) for the angle-of-look at the particular time. This allows the user to look around a virtual reality environment simply by moving the head without the need for a separate controller to change the angle of the imagery. In radio-based systems (compared to wires), the wearer may move about within the tracking limits of the system.
- Eye tracking – Eye trackers measure the point of gaze, allowing a computer to sense where the user is looking. This information is useful in a variety of contexts such as user interface navigation: By sensing the user's gaze, a computer can change the information displayed on a screen, bring added details to attention, etc.
- Hand tracking – tracking hand movement from the perspective of the HMD allows natural interaction with content and a convenient game-play mechanism
See also
[edit]References
[edit]- ^ Shibata, Takashi (1 April 2002). "Head mounted display". Displays. 23 (1–2): 57–64. doi:10.1016/S0141-9382(02)00010-0. ISSN 0141-9382.
- ^ Sutherland, Ivan E. (9 December 1968). "A head-mounted three dimensional display". Proceedings of the December 9-11, 1968, fall joint computer conference, part I on – AFIPS '68 (Fall, part I). ACM. pp. 757–764. CiteSeerX 10.1.1.388.2440. doi:10.1145/1476589.1476686. ISBN 9781450378994. S2CID 4561103. Retrieved 10 June 2018.
- ^ Komura, Shinichi (19 July 2024). "Optics of AR/VR using liquid crystals". Molecular Crystals and Liquid Crystals. 768 (17): 1014–1039. Bibcode:2024MCLC..768.1014K. doi:10.1080/15421406.2024.2379694. ISSN 1542-1406.
- ^ "Science: Second Sight". Time. 13 April 1962.
- ^ Dr. James Miller, Fullerton, CA, research psychologist for the Ground Systems Group at Hughes, "I've Got a Secret", April 9, 1962, on CBS
- ^ "Third Eye for Space Explorers". Popular Electronics. July 1962.
- ^ "'Seeing Things' with Electrocular". Science & Mechanics. August 1962.
- ^ Thompson, Jason I. "A Three Dimensional Helmet Mounted Primary Flight Reference for Paratroopers". Air Force Institute of Technology. Archived from the original on 6 August 2014. Retrieved 6 August 2014.
- ^ "Liteye OLED Helmet Mounted Displays", Defense Update, no. 3, 2005, archived from the original on 19 February 2006
- ^ Shachtman, Noah (11 April 2011). "Darpa's Hologram Goggles Will Unleash Drone Hell". Wired. Retrieved 29 June 2011.
- ^ Wheeler, Andrew (July 2016). "Understanding Virtual Reality (VR) Headsets". Engineering.com.
- ^ Liu, David; Jenkins, Simon A.; Sanderson, Penelope M.; Fabian, Perry; Russell, W. John (2010). "Monitoring with Head-Mounted Displays in General Anesthesia: A Clinical Evaluation in the Operating Room". Anesthesia & Analgesia. 110 (4): 1032–1038. doi:10.1213/ANE.0b013e3181d3e647. PMID 20357147. S2CID 22683908.
- ^ Maruta, J; Lee, SW; Jacobs, EF; Ghajar, J (October 2010). "A unified science of concussion". Annals of the New York Academy of Sciences. 1208 (1): 58–66. Bibcode:2010NYASA1208...58M. doi:10.1111/j.1749-6632.2010.05695.x. PMC 3021720. PMID 20955326.
- ^ Cochrane, Nathan. "VFX-1 Virtual Reality Helmet by Forte". GameBytes. Retrieved 29 June 2011.
- ^ "Video Glasses Can be Connected to DSLR Cameras". Hitari. 30 May 2013. Archived from the original on 22 March 2017. Retrieved 19 June 2013.
- ^ "Oculus Rift – Virtual Reality Headset for 3D Gaming". Retrieved 14 January 2014.
- ^ Makuch, Eddie (13 November 2013). "Xbox One, PS4 "too limited" for Oculus Rift, says creator". GameSpot.
- ^ Takada, Masumi; Yamamoto, Syota; Miyao, Masaru; Takada, Hiroki (2019). Effects of Low/High-Definition Stereoscopic Video Clips on the Equilibrium Function. Lecture Notes in Computer Science. Vol. 11572. Springer, Cham. pp. 669–682. doi:10.1007/978-3-030-23560-4. ISBN 978-3-030-23559-8.
- ^ Kronsberg, Matthew (1 November 2017). "The Royole Moon headset is like an in-flight IMAX cinema strapped to your face". Australian Financial Review.
- ^ "Your Personal Theater | Avegant Video Headset". avegant.com. Retrieved 28 January 2021.
- ^ "PRIME". Cinera. Retrieved 28 January 2021.
- ^ "Meet Cinera Edge, the Next-gen Personal Cinema". Cinera. Retrieved 28 January 2021.
- ^ "Cinemizeroled – One Place to Find Best Products". cinemizeroled.com. Retrieved 29 January 2021.
- ^ "VISIONHMD-BIGEYES H1". visionhmd.com. Retrieved 29 January 2021.
- ^ "Dream Glass 4K/4K Plus". Dream Glass. Archived from the original on 16 November 2021. Retrieved 16 November 2021.
- ^ Cuervo, Eduardo (June 2017). "BEYOND REALITY: Head-Mounted Displays for Mobile Systems Researchers". Getmobile: Mobile Computing and Communications. 21 (2). ACM: 9–15. doi:10.1145/3131214.3131218. S2CID 27061046.
- ^ Bachhuber, Christoph; Eckehard, Steinbach (2017). "Are Today's Video Communication Solutions Ready for the Tactile Internet?". 2017 IEEE Wireless Communications and Networking Conference Workshops (WCNCW). IEEE. pp. 1–6. doi:10.1109/WCNCW.2017.7919060. ISBN 978-1-5090-5908-9. S2CID 45663756.
- ^ David McGriffy. Make: Drones: Teach an Arduino to Fly. Maker Media, 2016. ISBN 9781680451719
- ^ "CDT Acquires Opsys' Dendrimer OLED Business". Archived from the original on 5 July 2008.
- ^ "Recon Instruments' Next Generation Technology Available This Fall". Recon Instruments. 3 November 2011. Archived from the original on 9 March 2012.
- ^ Lawson, B. D. (2014). Motion sickness symptomatology and origins. Handbook of Virtual Environments: Design, Implementation, and Applications, 531–599.
- ^ Koulieris, George-Alex. "Cutting-Edge VR/AR Display Technologies (Gaze-, Accommodation-, Motion-aware and HDR-enabled)". INRIA.HAL.SCIENCE Web Portal. HAL (open archive). Retrieved 20 March 2024.
Bibliography
[edit]- Head Mounted Displays: Designing for the user; Melzer and Moffitt; McGraw Hill, 1997.
- O. Cakmakci and J.P. Rolland. Head-Worn Displays: A Review. IEEE Journal of Display Technology, Vol. 2, No. 3, September 2006..
- O. Cakmakci. Optical Design of Eyewear Displays. SPIE Press, PM388, 2025. ISBN 9781510688339. Publisher page
Head-mounted display
View on GrokipediaIntroduction
Definition and Principles
A head-mounted display (HMD) is a wearable device positioned on the user's head that places screens and optical elements in front of the eyes to project digital images directly into the field of view.[12] These devices can superimpose computer-generated information onto the real-world view in augmented reality configurations or fully replace the physical environment with a synthetic one in virtual reality setups.[13] HMDs are categorized into monocular variants, which present imagery to a single eye while allowing the other to view the unaided real world; binocular variants, which deliver separate images to both eyes; and stereoscopic variants, which provide distinct images to each eye to simulate depth perception.[14] The fundamental optical principles of HMDs rely on miniature displays and lenses to form a virtual image that the eye perceives as if located at a comfortable distance, typically several meters away, rather than at the physical proximity of the device itself. Lenses focus collimated or near-collimated light from the display onto the retina, creating this virtual image often at optical infinity to maximize eye relief—the clearance between the lens and the user's eye for unrestricted blinking and accommodation.[15] Interpupillary distance (IPD) adjustment is essential, as it aligns the optical axes with the user's eye separation (typically 55–75 mm) to prevent visual strain, double vision, or reduced field of view; mismatches can induce phorias or asthenopia.[16] Perceptually, HMDs leverage binocular disparity—the horizontal offset between images presented to each eye—to cue depth, mimicking natural stereopsis where the brain fuses slightly different views for three-dimensional interpretation. This disparity simulates how objects at varying distances project differing retinal positions, enabling perceived depth via the relation , where IPD is the interpupillary distance, is the effective focal length of the optics, and is the lateral disparity. These principles were illustratively demonstrated in the first HMD prototype by Ivan Sutherland, known as the Sword of Damocles, which used optical see-through elements to overlay wireframe graphics and head tracking for basic spatial alignment.[17]Historical Development
The origins of head-mounted displays (HMDs) trace back to early 20th-century patents for stereoscopic viewing devices aimed at enhancing human perception.[1] Post-World War II, precursors emerged in military aviation through flight simulators and optical sighting systems that laid groundwork for integrating aids into pilot helmets to improve targeting accuracy.[18] In the 1960s, pioneering work in computer graphics led to the first true HMD prototype, developed by Ivan Sutherland at Harvard University in 1968; this tethered device, known as the Sword of Damocles due to its ceiling-suspended frame, displayed simple wireframe 3D graphics and tracked head movements to create an immersive viewing experience.[19] The 1970s and 1980s saw further advancements in military innovations, exemplified by the U.S. Army's Integrated Helmet and Display Sighting System (IHADSS) for the AH-64 Apache helicopter, prototyped around 1977 to enable pilots to aim weapons by looking at targets, marking a shift toward integrated sensor fusion in combat environments.[20] The 1980s and 1990s brought a surge in virtual reality (VR) enthusiasm, driven by Jaron Lanier's founding of VPL Research in 1985, which commercialized early HMDs like the EyePhone alongside data gloves, coining the term "virtual reality" and emphasizing immersive simulation for research and entertainment.[21] This era culminated in consumer-facing products, such as the 1991 Virtuality arcade systems, which featured stereoscopic HMDs with head tracking for multiplayer VR gaming experiences in public venues.[21] Sony entered the market in the mid-1990s with the Glasstron, a lightweight video see-through HMD that projected imagery equivalent to a large-screen TV, targeting personal media consumption and early augmented applications.[22] The 2000s and 2010s marked the rise of augmented reality (AR) alongside renewed VR interest, highlighted by the 2012 Oculus Rift Kickstarter campaign, which raised over $2.4 million to develop an affordable consumer HMD with low-latency tracking, revitalizing VR hardware for gaming and beyond after acquiring key patents from VPL.[23] AR gained traction with Google's 2013 Glass, a wearable display offering hands-free information overlays via a prism projector, though it faced privacy concerns and limited adoption.[23] Microsoft's 2015 HoloLens introduced holographic mixed reality through transparent optics and spatial mapping, enabling untethered AR interactions for enterprise uses like design and training.[23] Entering the 2020s, HMDs evolved toward seamless mixed reality integration, with Apple's Vision Pro, released in 2024 and upgraded in October 2025 with an M5 chip, serving as a high-resolution spatial computing headset featuring eye and hand tracking for immersive apps across productivity and entertainment.[24][8] Meta advanced lightweight AR prototypes with Orion in 2024, a holographic waveguide-based glasses form factor weighing under 100 grams, prioritizing all-day wearability for social and assistive experiences, while releasing the Ray-Ban Display AI glasses in September 2025 with integrated displays and EMG wristband controls.[25][26] By 2025, AI integration became a defining trend, enabling real-time rendering, gesture prediction, and contextual overlays in HMDs, such as AI-assisted visualization in smart glasses for industrial and medical applications.[27] Throughout this evolution, key trends reflect a transition from cumbersome, military-centric tools to compact consumer wearables, propelled by Moore's Law, which has exponentially increased transistor density and computational power, enabling smaller displays, efficient sensors, and reduced power consumption in HMD components.[28]Types and Technologies
See-Through Displays
Optical see-through head-mounted displays (HMDs) overlay digital content onto the user's direct view of the real world by employing transparent optical elements that transmit ambient light while reflecting or directing light from an internal display source. Early designs commonly utilized beam splitters or half-silvered mirrors to achieve this combination, where the mirror partially reflects the display image toward the eye and allows a portion of incoming real-world light to pass through unchanged.[29] More advanced implementations incorporate holographic optical elements (HOEs) or diffractive combiners, which selectively diffract specific wavelengths of display light into the user's field of view while maintaining high transparency for the broader spectrum of ambient illumination.[30] These components enable augmented reality (AR) applications by preserving the natural optics of the environment without electronic mediation of the real scene. A primary advantage of optical see-through HMDs is the preservation of natural hand-eye coordination, as users perceive the real world through their own eyes, retaining authentic depth cues and binocular disparity without distortion from cameras or screens.[31] This direct passthrough also results in inherently low latency for AR overlays, avoiding the processing delays associated with video capture and rendering of the physical environment. Additionally, the design enhances safety during real-world navigation, as users maintain unobstructed visibility of surroundings, reducing risks in dynamic settings like aviation or surgery. These benefits trace back to foundational systems, such as Boeing's 1980s head-up displays (HUDs) for fighter pilots, which evolved into full HMDs by integrating see-through optics for enhanced situational awareness in combat aircraft.[32] Despite these strengths, optical see-through HMDs face significant challenges in precise alignment between virtual and physical elements, particularly due to parallax errors arising from the spatial offset between the user's eye and the display optics. Parallax occurs when the virtual image appears shifted relative to real objects at different depths, leading to registration inaccuracies that degrade AR utility. This misalignment can be quantified by the error angle , where represents the axial offset between the eye and display plane, and is the effective focal length of the optical system; even small values can produce noticeable errors in close-range interactions.[33] Historical implementations highlight these issues alongside innovations, such as Boeing's 1990s HMD for the F-16 fighter jet, which employed optical see-through technology via a helmet-mounted combiner to cue weapons and navigation symbology directly in the pilot's line of sight.[32] Modern examples, like the Microsoft HoloLens 2 introduced in 2019, address field-of-view limitations through waveguide-based optics that guide collimated light across transparent substrates, enabling wider angular coverage and reduced parallax in consumer AR applications, though production was discontinued in 2024.[34][35] To mitigate alignment challenges, optical see-through HMDs require robust calibration procedures, including real-time head tracking to dynamically adjust virtual content based on the user's pose and gaze. This involves integrating inertial sensors and optical trackers to estimate head orientation and position, ensuring virtual elements remain registered with the physical world across movements.[36] Such calibration is critical for maintaining accuracy, especially in scenarios with varying eye-display distances or user-specific anatomical differences.Opaque Displays
Opaque head-mounted displays (HMDs) in virtual reality (VR) systems block external light to immerse users in synthetic environments, either through direct light occlusion via opaque screens or by capturing and processing real-world video feeds for compositing virtual elements. This design enables full control over the visual input, creating isolated, high-fidelity simulations without interference from the physical surroundings. In video see-through configurations, forward-facing cameras capture the real environment, which is then digitally altered and overlaid with virtual content to simulate transparency or mixed reality scenes, though the HMD remains optically opaque to prevent direct light passthrough.[37][38] The primary advantages of opaque HMDs include precise manipulation of visual elements for realistic simulations, such as accurate lighting and occlusion in virtual scenes, and the elimination of real-world distractions to enhance user focus and presence. However, these systems introduce drawbacks like increased latency from video capture and processing pipelines, which can cause motion sickness if delays exceed 20-30 milliseconds. Video compositing in mixed setups further amplifies this issue by requiring real-time depth estimation and blending, potentially degrading immersion if not optimized.[39][40] Key technologies in opaque HMDs often incorporate stereo camera pairs for depth mapping, enabling accurate spatial alignment of virtual objects with captured real-world footage in video see-through modes. Latency in these systems is modeled as the total delay , where accounts for sensor readout, includes rendering and compositing computations, and covers scan-out to the screen; minimizing each stage is critical for seamless VR experiences. Advanced processing uses GPU acceleration to handle stereo disparity for 3D reconstruction, supporting immersive interactions.[37][39] More recent consumer examples include the Meta Quest 3, released in 2023, which features dual LCD displays blocking external light for full immersion, inside-out tracking, and low-persistence rendering to reduce blur in standalone VR. The HTC Vive, launched in 2016, extended earlier designs with room-scale tracking using external base stations, allowing users to move within a 2 m × 1.5 m area while maintaining opaque visual isolation. This evolved into standalone devices like the Oculus Quest in 2019, featuring inside-out tracking cameras for untethered, opaque VR without external sensors, powered by a Snapdragon processor for on-device rendering.[41][42][43][44] Immersion in opaque HMDs is enhanced by full 360-degree rendering, which surrounds the user with continuous spherical visuals via head-oriented perspective projection, fostering a sense of presence in the virtual space. Integration of haptic feedback, such as vibrotactile responses from controllers synchronized with visual cues, further deepens this by providing tactile confirmation of interactions, as demonstrated in multisensory VR setups. Audio systems can briefly complement this by delivering spatial soundscapes aligned with the opaque visuals for holistic sensory engagement.Advanced Projection Methods
Waveguide optics represent a pivotal advancement in head-mounted displays (HMDs), particularly for augmented reality (AR) applications, by propagating light through thin substrates using total internal reflection (TIR). This technique confines light rays within a waveguide material, such as glass or polymer, allowing multiple internal bounces before out-coupling to the user's eye, thereby enabling compact form factors without bulky lenses.[45] The efficiency of these systems is often quantified by coupling loss, expressed as , where and denote output and input optical power, respectively; this metric highlights challenges like light leakage during in- and out-coupling, which can reduce overall brightness uniformity.[46] Such designs have facilitated the development of lightweight AR glasses, prioritizing thin profiles over traditional bulk optics.[47] Retinal projection systems offer another sophisticated approach, directly scanning modulated laser light onto the retina to form images, bypassing intermediate lenses and achieving high angular resolution in a compact package. This method leverages the eye's natural optics, projecting pixels at the fovea for sharp focus without the need for wide exit pupils, thus minimizing optical aberrations and bulk.[48] Hybrid implementations, such as those in the Magic Leap One released in 2018, primarily use waveguide optics with digital light field projection to overlay digital content, providing a wide field of view while maintaining see-through transparency.[49] Beyond these, liquid crystal on silicon (LCoS) microdisplays serve as reflective image sources in many HMD projections, utilizing a silicon backplane to modulate polarized light for high contrast and resolution in compact modules.[50] Birdbath optics, prevalent in early portable HMDs, employ a beam splitter and curved partial mirror to fold the optical path, allowing off-the-shelf displays to project images while permitting see-through views, though at the expense of light efficiency due to multiple reflections.[51] By 2025, diffractive waveguides have matured in commercial devices like the Xreal Air 2 (2023), which integrates grating-based in- and out-couplers to achieve full-color AR projection in a form factor weighing 72 grams. Emerging prototypes, such as the October 2025 Magic Leap-Google Android XR smart glasses, further advance waveguide technology for all-day AR wear.[52][53] Complementary techniques, such as foveated rendering, further enhance efficiency by dynamically allocating computational resources to the user's gaze center, reducing peripheral resolution to cut power consumption by up to 50% without perceptible quality loss in HMDs.[54] These innovations underscore key trade-offs: while advanced projection methods improve form factors and immersion, they introduce higher manufacturing costs and design complexity due to precise nanofabrication requirements for elements like gratings and scanners.[55]Key Components
Display and Optics
Head-mounted displays (HMDs) rely on compact, high-performance display panels to deliver immersive virtual or augmented experiences, evolving significantly from early cathode ray tube (CRT) technologies in the 1990s, which were bulky and offered resolutions below 640x480 pixels per eye, to modern microdisplays achieving high resolutions approaching or exceeding 4K per eye, as seen in the Varjo XR-3, released in 2021, with 2880x2720 pixels per eye and up to 70 pixels per degree (PPD) in the foveal area.[56] This progression has been driven by the need for lightweight form factors under 500 grams and power efficiency for prolonged wear. Common display types in HMDs include liquid crystal displays (LCDs), organic light-emitting diode (OLED) panels, and emerging micro-light-emitting diode (micro-LED) arrays, each balancing trade-offs in pixel density, refresh rates, and power consumption. LCDs, often backlit by LEDs, provide high brightness up to 1000 nits but suffer from lower contrast due to light leakage, typically achieving ratios around 1000:1, with pixel densities of 1000-2000 PPI and refresh rates of 60-120 Hz, as in the Meta Quest 3 at 1218 PPI and 120 Hz.[57] OLEDs excel in black levels by self-emission, enabling contrast ratios exceeding 10,000:1, which enhances depth perception in dark scenes, with pixel densities reaching 3000 pixels per inch (PPI) or higher, e.g., ~3386 PPI in the Apple Vision Pro (2024), and refresh rates of 90-120 Hz.[58] Micro-LEDs offer superior efficiency and longevity over OLEDs, with potential PPI above 5000 and refresh rates up to 240 Hz, though current implementations remain costly and limited to prototypes due to fabrication challenges.| Display Type | Typical PPI | Refresh Rate (Hz) | Contrast Ratio | Key Advantage in HMDs |
|---|---|---|---|---|
| LCD | 1000-2000 | 60-120 | ~1000:1 | Cost-effective brightness |
| OLED | 2000-3000 | 90-120 | >10,000:1 | True blacks for immersion |
| Micro-LED | >3000 | 120-240 | >1,000,000:1 | High efficiency, durability |
Sensors and Tracking
Head tracking in head-mounted displays (HMDs) primarily relies on inertial measurement units (IMUs), which integrate gyroscopes to measure angular velocity and accelerometers to detect linear acceleration, enabling real-time estimation of the user's head orientation and position. These sensors provide high-frequency updates, typically at 1000 Hz or more, but suffer from integration drift over time due to cumulative errors in velocity and position calculations. To mitigate this, IMUs are fused with visual-inertial odometry techniques, such as Simultaneous Localization and Mapping (SLAM), which uses onboard cameras to track environmental features and correct drift through map-based pose refinement. This fusion enhances accuracy in dynamic environments, achieving sub-centimeter positional precision in egocentric tracking for mixed reality headsets. Eye tracking in HMDs employs infrared (IR) cameras to illuminate and capture reflections from the user's eyes, determining gaze direction by analyzing pupil position and corneal reflections. The pupil center corneal reflection (PCCR) method, a widely adopted technique, computes gaze vectors from the relative positions of the pupil center and multiple IR glints, supporting applications like foveated rendering where rendering resolution is dynamically allocated to the user's focal point for computational efficiency. Modern implementations achieve gaze estimation accuracy of approximately 0.5 degrees, sufficient for precise interaction in virtual environments while minimizing intrusiveness in compact HMD designs. Environmental sensing in HMDs incorporates depth cameras using time-of-flight (ToF) or structured light principles to generate real-time 3D maps of surroundings, facilitating passthrough augmented reality (AR) by overlaying virtual elements on a digital reconstruction of the physical world. ToF sensors emit modulated IR light and measure phase shifts for depth, offering robust performance in varied lighting, while structured light projects patterns to infer disparity via triangulation. For instance, Intel RealSense cameras, which combine RGB and depth sensing, have been integrated into HMD prototypes for enhanced spatial awareness and obstacle avoidance in AR applications. Pose estimation algorithms in HMDs often utilize Kalman filtering to fuse multi-sensor data, providing optimal state predictions by balancing model predictions with noisy measurements. The standard Kalman filter update equation for pose estimation is given by where is the updated state estimate, is the prior estimate, is the Kalman gain, is the measurement, and is the observation model. This recursive approach corrects IMU drift using visual inputs, ensuring low-latency head pose tracking essential for immersive experiences. By 2025, consumer HMD standards have evolved to support 6 degrees of freedom (6DoF) tracking at 120 Hz refresh rates, as exemplified by the PlayStation VR2 (released in 2023), which uses inside-out camera-based sensing combined with IMUs for seamless room-scale interactions.[59] This performance level aligns tracking updates with display refresh rates to minimize motion artifacts.Audio and Input Systems
Head-mounted displays (HMDs) integrate audio systems to deliver immersive 3D soundscapes, often using bone conduction transducers or in-ear headphones to provide private audio experiences without obstructing environmental awareness. Bone conduction transmits vibrations through the skull directly to the inner ear, enabling users to hear virtual audio while remaining attentive to real-world sounds, as demonstrated in augmented reality applications where spatialized audio enhances situational awareness.[60] In-ear headphones, conversely, offer sealed isolation for deeper bass and clarity in fully immersive environments. These systems frequently employ head-related transfer functions (HRTF) to simulate binaural audio, filtering sounds based on individual head and ear geometry to create realistic directional cues over headphones. HRTF personalization reduces localization errors in virtual spaces, improving the perceived accuracy of sound sources.[61] Spatial audio in HMDs leverages techniques like Ambisonics for rendering full-sphere sound fields, encoding amplitude and phase information from multiple directions for playback via binaural or multichannel setups. Ambisonics supports scalable orders of resolution, with first-order implementations providing basic directionality suitable for real-time HMD processing. A key aspect of its directionality is captured in the intensity pattern for a cardioid component, given bywhere is the maximum intensity and is the angle from the primary axis; this equation models how sound energy concentrates forward while attenuating rearward, foundational to Ambisonics beamforming.[62][63] User input in HMDs extends beyond traditional controllers to natural interfaces, enhancing immersion through gesture and voice modalities. Handheld controllers with six degrees of freedom (6DoF) tracking, such as the Oculus Touch, enable precise positional and rotational input via optical sensors and inertial measurement units, allowing users to manipulate virtual objects intuitively.[64] Eye and hand gesture recognition, powered by AI models like convolutional neural networks, interprets skeletal tracking from cameras to detect commands without physical devices, supporting fluid interactions in untethered environments. Voice commands integrate natural language processing (NLP) for hands-free control, parsing spoken intents to execute actions like navigation or object selection in spatial computing scenarios. Notable implementations include the Apple Vision Pro (released 2024), which features a six-microphone array for environmental passthrough audio, blending real and virtual sounds via beamforming to maintain spatial awareness during mixed-reality use.[58] As extensions, haptic feedback vests like the bHaptics TactSuit provide tactile synchronization with audio cues, vibrating across the torso to simulate impacts or environmental effects in VR gaming.[65] Despite advancements, audio systems in HMDs face challenges such as acoustic leakage in bone conduction designs, where vibrations radiate externally, potentially compromising privacy in shared spaces. Open-ear configurations exacerbate this, as higher volumes increase audible spillover to bystanders. Additionally, always-on microphones for voice activation and passthrough contribute to battery drain, with continuous processing reducing HMD runtime by up to 20-30% in high-interaction scenarios, necessitating optimized power management.[66][67]