Virtual reality headset

The Oculus Rift development kit headset, showing the stereoscopic lens along with the control box.

A virtual reality headset provides immersive virtual reality for the wearer. VR headsets are widely used with computer games but they are also used in other applications, including simulators and trainers. They comprise a stereoscopic head-mounted display (providing separate images for each eye), stereo sound, and head motion tracking sensors^[1] (which may include gyroscopes, accelerometers, structured light systems,^[2] etc.). Some VR headsets also have eye tracking sensors^[3] and gaming controllers.

History

The Forte VFX1 headset from 1994.

An early VR headset, the Forte VFX1, was announced at CES in 1994. The VFX-1 has stereoscopic displays, 3-axis head-tracking, and stereo headphones.^[4] Sony, another pioneer, released the Glasstron in 1997, which has an optional positional sensor, allowing the wearer to view the surroundings, with the perspective moving as his head moves, giving a deep sense of immersion. These VR headsets gave MechWarrior 2 players a new visual perspective of seeing the battlefield from inside the cockpit of their craft. However, these early headsets failed commercially due to their limited technology^[5][6] and were described by John Carmack as like "looking through toilet paper tubes".^[7]

In 2012, a crowdfunding campaign began for a VR headset known as Oculus Rift; the project was led by several prominent video game developers, including John Carmack^[5] who later became the company's CTO.^[8] In March 2014, the project's parent company Oculus VR was acquired by Facebook for US$2 billion.^[9] The final consumer-oriented release of Oculus Rift began shipping on 28 March 2016.^[10]

In March 2014, Sony demonstrated a prototype headset for PlayStation 4,^[11] which was later named PlayStation VR.^[12] In 2014, Valve Corporation demonstrated some headset prototypes,^[13] which lead to a partnership with HTC to produce the Vive, which focuses on "room scale" VR environments that users can naturally navigate within and interact with.^[14] The Vive was released in April 2016^[15] and PlayStation VR in October 2016.^[16]

Samsung Gear VR, a VR headset designed exclusively for use with Samsung Galaxy smartphones.

Virtual reality headsets and viewers have also been designed for smartphones. Unlike headsets with integrated displays, these units are essentially enclosures which a smartphone can be inserted into. VR content is viewed from the screen of the device itself through lenses acting as a stereoscope, rather than using dedicated internal displays. Google released a series of specifications and associated DIY kits for virtual reality viewers known as Google Cardboard; these viewers are capable of being constructed using low-cost materials, such as cardboard (hence the naming). Samsung Electronics parterned with Oculus VR to co-develop the Samsung Gear VR (which is only compatible with recent Samsung Galaxy devices), while LG Electronics developed a headset with dedicated displays for its LG G5 smartphone known as LG 360 VR.^{[17][18][19][20]}

Constraints

Latency requirements

Virtual reality headsets have significantly higher requirements for latency—the time it takes from a change in input to have a visual effect—than ordinary video games.^[21] If the system is too sluggish to react to head movement, then it can cause the user to experience virtual reality sickness, a kind of motion sickness.^[22] According to a Valve engineer, the ideal latency would be 7-15 milliseconds.^[23] A major component of this latency is the refresh rate of the display,^[22] which has driven the adoption of displays with a refresh rate from 90 Hz (Oculus Rift and HTC Vive) to 120 Hz (PlayStation VR).^[24]

The graphics processing unit (GPU) also needs to be more powerful to render frames more frequently. Oculus cited the limited processing power of Xbox One and PlayStation 4 as the reason why they are targeting the PC gaming market with their first devices.^[25]

Asynchronous reprojection/time warp

A common way to reduce the perceived latency^[26] or compensate for a lower frame rate,^[27] is to take an (older) rendered frame and morph it according to the most recent head tracking data just before presenting the image on the screens. This is called asynchronous reprojection^[28] or "asynchronous time warp" in Oculus jargon.^[29]

PlayStation VR synthesizes "in-between frames" in such manner, so games that render at 60 fps natively result in 120 updates per second.^[24][27] SteamVR (HTC Vive) will also use "interleaved reprojection" for games that cannot keep up with its 90 Hz refresh rate, dropping down to 45 fps.^[30]

The simplest technique is applying only projective transformation to the images for each eye (simulating rotation of the eye). The downsides are that this approach cannot take into account the translation (changes in position) of the head. And the rotation can only happen around the axis of the eyeball, instead of the neck, which is the true axis for head rotation. When applied multiple times to a single frame, this causes "positional judder", because position is not updated with every frame.^[26][31][32]

A more complex technique is positional time warp, which uses pixel depth information from the Z-buffer to morph the scene into a different perspective. This produces other artifacts because it has no information about faces that are hidden due to occlusion^[31] and cannot compensate for position-dependent effects like reflections and specular lighting. While it gets rid of the positional judder, judder still presents itself in animations, as timewarped frames are effectively frozen.^[32] Support for positional time warp was added to the Oculus SDK in May 2015.^[33]

Resolution and display quality

Because virtual reality headsets stretch a single display across a wide field of view (up to 110° for some devices according to manufacturers), the magnification factor makes flaws in display technology much more apparent. One issue is the so-called screen-door effect, where the gaps between rows and columns of pixels become visible, kind of like looking through a screen door.^[34] This was especially noticeable in earlier prototypes and development kits,^[6] which had lower resolutions than the retail versions.

Lenses

Image captured from an Oculus Rift DK2, showing compensation for lens distortion and chromatic aberration.

The lenses of the headset are responsible for mapping the up-close display to a wide field of view,^[35][36] while also providing a more comfortable distant point of focus. One challenge with this is providing consistency of focus: because eyes are free to turn within the headset, it's important to avoid having to refocus to prevent eye strain.^[37]

The lens introduce distortion and chromatic aberration, which are corrected in software.^[35]

Uses in various fields

Medicine

Medical training

Virtual reality headsets are being currently used as means to train medical students for surgery. It allows them to perform essential procedures in a virtual, controlled environment. Students perform surgeries on virtual patients, which allows them to acquire the skills needed to perform surgeries on real patients.^[38] It also allows the students to revisit the surgeries from the perspective of the lead surgeon.^[39]

Traditionally, students had to participate in surgeries and often they would miss essential parts. But, now surgeons have been recording surgical procedures and students are now able to watch whole surgeries again from the perspective of lead surgeons with the use of VR headsets, without missing essential parts. Students can also pause, rewind, and fast forward surgeries.^[39]

See also

• Comparison of virtual reality headsets

References