Microsoft AR/VR patent proposes any given direction

CheckCitation/SourcePlease click:XR Navigation Network

(XR Navigation Network January 02, 2024) for the XR headseteye tracking技术正在迅速发展。追踪眼动和确定用户注视方向的一种技术包括分析指向用户眼睛和从用户眼睛反射的红外光信号。例如，头戴式显示器可以配备一个或多个红外光源，从不同的方向向用户的眼睛发射红外光。从用户眼睛反射出来的红外光（闪烁）可以由传感器检测到，并用于确定用户眼睛的XYZ位置和注视方向。

In order to improve the accuracy and processing power of an infrared eye tracking system, it is often necessary to place several different infrared light sources around the user's eye to produce different flashes at the cornea.

Unfortunately, conventional XR systems have certain limitations in terms of where the IR light source can be localized. In particular, legacy systems have historically positioned IR light sources at the peripheral edges of the head unit or other structures where the actual display lens and screen are mounted. This peripheral location of the IR light source is not always optimal and requires the use of additional light sources and/or larger, more powerful light sources. If the light source can be placed closer or better relative to the iris of the user's eye, no other light sources need be used.

There are XR systems that attempt to place the IR light source away from the edge of the display and within the user's field of view to be close to the eyes. However, existing systems of this type are problematic because they create visual obstacles to the user's perspective in mixed reality environments. In particular, existing IR light sources are typically sized in the range of 1.0 mm to 4.0 mm diameter/width. Such dimensions are very noticeable, especially when positioned in the user's field of view. For at least this reason, most conventional systems only position the IR light source in a sub-optimal position at the peripheral edge of the lens/display.

In view of this issue.Microsoft名为“Microled based invisible illumination for eye tracking”的专利申请中提出了一种解决方案。特别是，微软表示所述发明允许红外光源以相对于用户眼睛/虹膜和相应摄像头传感器的最佳方向直接定位在头显透镜的查看区域之内，同时不会妨碍用户对混合现实环境的感知。

The company noted that the approach is to utilize an illumination lens structure with infrared μLEDs that are less than 100 μm in any given direction, and because such uLEDs are not perceived by the user even when embedded in the headset's lens, they do not impede or interfere with the headset's rendering of the mixed reality environment during use.

Turning attention now to FIG. 1, the headset is configured to perform eye tracking based on light reflections (blinks) captured by the camera 130 or other sensors. For example, during use, light is emitted around a light source in the user's eye (.). After the light is emitted, flashes of light are reflected from the user's eyes (in particular the user's iris) and detected by the camera.

Based on the perceived intensity/strength of the light, relative to the source/time of light emitted, the head-up light processing module can detect the localization of the user's eyes/iris (relative position and orientation).

Additional processing of images captured by the system's cameras/sensors can be used to distinguish between the user's pupil and iris. Such images can help the system map the position of the user's eyes and the orientation/gaze of the user's eyes relative to the projected hologram or other objects. The system can also utilize the position of the user's eyes to support mixed reality environments by positioning and re-projecting the hologram at the desired location.

FIG. 2 shows a representation of a user's eye 200 where an infrared light source 210 emits infrared light 220 into the user's eye 200. the infrared light is reflected back as specular and scattered reflections. FIG. 2 also shows how a camera 250 or other sensor may be placed to detect one or more reflections.

The system can determine the relative positioning of the user's eye/iris by using the position of the light source, the time at which the light is emitted from the light source, the position of the camera, and the measured intensity and time of the detected light reflections reflected from the user's eye. This is due to the difference in the reflection of light in different parts of the user's eye, e.g. it reflects differently in the pupil and iris regions of the cornea than in the sclera. This difference is detected and measured based on whether the reflection is specular or scattered.

In order for the light source to be optimally positioned, e.g. close to the user's cornea, it needs to be positioned in the right place. Unfortunately, however, conventional IR light sources are too large (e.g., 1-4 mm) to be positioned within the user's field of view without obstructing the user's view of the environment through the lens.

To help address these issues, Microsoft proposes that an illumination lens configured with an infrared μLED could be used. With this configuration, the light source can be optimally placed close to the user's eyes without having to consider the existing limitations imposed by the physical form factor of the headset mounting structure.

FIG. 3 illustrates an infrared μLED lens structure 300 having 16 infrared μLEDs arranged in a grid fashion. the infrared μLEDs are placed on top of a transparent substrate 310 to form a circuit between the anode and cathode terminals along a conductive trace 320. For example, when the circuit is powered by the power supply of the head unit, the power supply may be electrically connected to the anode and cathode terminals and controlled by a lighting control unit, and the IR μLEDs will activate and emit IR light.

The infrared μLED emits light at a wavelength between 790 μm ~ 1 mm. In preferred embodiments, the infrared μLED emits light at a wavelength of about 800-900 μm, and even more preferably emits light at a wavelength of about 850 μm.

FIGS. 4A-4D show that the number of not-oh-she-that IR uLEDs varies for different illumination lens configurations. Specifically, in FIG. 4A, the distribution pattern of IR uLEDs is somewhat circular/elliptical, but somewhat rectangular in FIG. 4C and somewhat diamond-shaped in FIG. 4D. On the other hand, the embodiment shown in FIG. 4B includes two different patterns, a circular/elliptical outer pattern and a triangular inner pattern.

Further, the illumination lens structure 400A comprises 6 infrared uLEDs, while the illumination lens structures 400C and 400D each comprise 8 infrared uLEDs. The illumination lens structure 400B comprises 19 infrared uLEDs.

The different numbers and patterns of IR uLEDs can be changed to suit different needs and preferences.

In FIG. 5, the head-up display 500 is configured with a transparent illumination lens structure 510, the lens structure 510 having a plurality of infrared μLEDs 540. as shown, the infrared μLEDs 540 are distributed in a circular pattern, eight each for each eye and lens region. The different infrared μLEDs 540 can optionally be connected to a single circuit or two or more different circuits. The electrical traces forming the circuits are not shown.

During use, the light emitted from the infrared uLEDs will be directed at least partially toward the user's eyes, and the light will be reflected back and detected by the device camera 530.

FIG. 6 illustrates an embodiment of a pair of eyeglasses 600, the eyeglasses 600 having a first transparent illumination lens structure 610 including four infrared uLEDs and a second transparent illumination lens structure 620 including eight infrared uLEDs.

This example is used to illustrate that there does not have to be a matching/symmetrical distribution of IR uLEDs on either side of the head unit. Regardless of the number and distribution of IR uLEDs, it should be understood that during use, light emitted from the IR uLEDs will be directed at least partially toward the user's eyes and the light may be reflected back and detected by the device camera 630.

In a related embodiment, the head-up display system 700 shown in FIG. 7 includes a transparent illumination lens structure 710 having a sun visor. in this example, one side of the sun visor has several infrared uLEDs forming a circular pattern. the other side of the sun visor has four infrared uLEDs distributed in a square shape.

Regarding all the previous examples, the IR uLEDs are not shown to scale. In fact, the IR uLEDs are so small (<100 μm), like the thickness of the traces, that they are not visible in the current illustration if represented to scale.

Again, because the IR LEDs are so small, they can be used to illuminate the user's eyes with IR light and at the same time be positioned in the lens through which the user is passing without obstructing the user's field of view through the lens.

The conductive traces are very thin, with widths of less than 50 μm or even less than 25 μm, so that they are visually insignificant and largely invisible when used close to the user's eye. This configuration is particularly advantageous for enabling the traces to be positioned inside the illumination lens structure. Even when positioned directly in front of the user's eye, it is used without obstructing the user's view of the environment as perceived through the lens.

FIG. 8 visualizes a manufacturing process for fabricating the lighting lens structures described in the invention.

As shown, the manufacturing process includes obtaining a wafer 810 comprising one or more infrared μLEDs. for example, the wafer 810 may be an epitaxial wafer formed by an epitaxial growth or deposition process.

The process 800 shown in FIG. 8 also includes obtaining a substrate 820 for transferring the infrared uLEDs thereto. as shown, a transfer process 830 is performed to transfer one or more infrared uLEDs to conductive traces 825 that are already disposed on the substrate 820 and form one or more different circuits 827 on the substrate.

The size of the IR μLEDs removed from the substrate and placed on the substrate is limited to <100 μm in any direction, so the maximum size of any measurable length of the IR μLEDs is less than 100 μm.

The maximum size dimension of the infrared uLED may be less than 75 μm, less than 50 μm, or even less than 20 μm. in one embodiment, the maximum size dimension of the infrared uLED is about 10 μm.

The width of the traces is similarly limited in that the thickness cannot be greater than <50 μm, <40 μm, <30 μm, or even less than 20 μm. in one embodiment, the width of the traces is about 20 μm.

FIG. 9 illustrates a flowchart 900 with a configuration for performing eye tracking, wherein the headset includes an illumination lens comprising a plurality of IR μLEDs, and wherein each IR μLED of the plurality of IR μLEDs has a maximum size of <100 μm.

The system components control the illumination of the infrared uLEDs by emitting infrared light from one or more of the infrared uLEDs in the illumination lens to the user's eye.

Next, the headset is further configured to detect and process flashes of infrared light reflected back from the user's eyes during use of the headset, and to determine a localization of the user's eyes based on the detected and processed flashes.

Attention is now turned to FIG. 10, which illustrates a flowchart 1000 with the fabrication of an illumination lens structure using a plurality of infrared uLEDs.

This includes for obtaining a transparent substrate, applying a plurality of traces to a transparent backing. wherein the plurality of traces conduct electricity and form at least one circuit between an anode terminal and a cathode terminal.

Then, an infrared uLED wafer is obtained comprising a plurality of infrared uLEDs or materials that can be individually extracted as discrete infrared uLEDs with a maximum size of <100 μm. Next, a set of one or more infrared uLEDs is transferred to a substrate in a predetermined pattern and such that the infrared uLEDs are electrically coupled to at least one circuit on the substrate/substrate.

名为“Microled based invisible illumination for eye tracking”的Microsoft patent申请最初在2023年1月提交，并在日前由美国专利商标局公布。

Generally speaking, after a U.S. patent application is examined, it will be automatically published 18 months from the filing date or priority date, or it will be published within 18 months from the filing date at the request of the applicant. Note that publication of a patent application does not mean that the patent is approved. After a patent application is filed, the USPTO requires actual review, which can take anywhere from 1 to 3 years.