Microsoft AR/VR Patent Shares Improved PCIe Data Link Transmission Inside HoloLens

CheckCitation/SourcePlease click:XR Navigation Network

(XR Navigation Network December 27, 2023A wired connection, often referred to as a "hard link", physically connects one node to another. One type of wired connection is PCIe, an interface for connecting high-speed nodes, and while high-speed connectivity endpoints and protocols such as PCIe offer substantial advantages, these types of endpoints require high power to operate. For battery-powered computing architectures such as head-up displays, this can place a significant burden on the system.

在名为“Error recovery and power management between nodes of an interconnection network”的专利申请中，MicrosoftMethods to improve transmission between transmitter nodes and receiver nodes by recovering from error conditions without retransmitting data are presented to help reduce the system burden on the head unit.

In one embodiment, a data streaming is initiated. Said data stream includes different types of data packets. Error correction codes (ECC) are selectively applied to control the data type packet. a transmitter node and a receiver node are connected via a hard link having a plurality of virtual channels. Each virtual channel is associated with a corresponding power consumption node.

When a receiver node receives a control data type packet, it performs error correction without retransmitting, if required. When the final data type packet is sent for each virtual channel, the transmitter node sends the end condition type packet. the corresponding power consumption node corresponding to the corresponding virtual channel transitions from an active state to a low power state.

Particular embodiments are configured to improve power usage of a battery powered device by recognizing when a packet transmission across a virtual channel is complete, and by causing the node to enter a low power state when the packet transmission is complete.

FIG. 2 illustrates an example of an interconnected network 200. The interconnected network 200 may include any number of interconnected nodes, such as node 205 and node 210. examples of nodes 205 and 210 may be any type of computing component, such as an HPU holographic processing unit.

As shown, the node 205 is connected to the node 210 via a high-speed link 215.The high-speed link 215 may be any type of physical channel connection that supports high-speed data transmission.

Embodiments described by the invention may virtualize the high-speed link 215 to include any number of virtual channels. For example, hard link 240 represents high speed link 215. hard link 240 is shown as including any number of virtual channels, such as virtual channels 245, 250, 255, and 260. packets are shown as being transmitted through these different virtual channels, such as packet 265.

Each virtual channel is associated with its respective buffer. For example virtual channel 245 may be associated with buffer 270 and virtual channel 250 may be associated with buffer 275. Other virtual channels are associated with their respective buffers. Each buffer may have a corresponding buffer size 280. in accordance with the principles of the invention, the hardware is able to maintain channel bandwidth balancing among the various requesters (i.e., receiving nodes).

Figures 3 and 4 are examples of head-mounted displays.

The high-speed link 410A can be used to transfer high-speed display data and/or sensor data between a computing unit located in a front-end enclosure and a computing unit located in a rear-end enclosure. High-speed link 410A is designed to have a low latency so that a large number of computing cycles or communications can be passed. In contrast, the low-speed link 405A may be used to provide power, grounding, or switching mechanisms.

The high-speed link 215 in FIG. 2 may represent the high-speed link 410A in FIG. 4, and the nodes 205 and 210 in FIG. 2 may represent any of the computing units shown in FIG. 4, such as the display circuits 420A, the CPUs 420B and 440A, the GPUs 420C and 4406, the SOC 420D, and the HPU 440C.

FIG. 8 lists the operational requirements 800. initially, the operational requirements 800 include real-time data requirements 805. as discussed earlier with respect to head units, a head unit for displaying content to a user has real-time requirements 805 so it is desirable to avoid retransmitting data.

Additionally, the head-up display content changes dynamically based on many different factors, including the pose/orientation of the head-up display, scene changes in the MR scene, interaction with the hologram, etc. So when implemented in a head-up display, data can be required to be transferred from one node in the head-up display to another node in order to meet real-time data requirements.805 This means that the head-up display implementation does not now have the ability to re-transmit the data, as is the case with legacy systems that encounter data errors.

Due to real-time data requirements 805, retransmission to resolve or correct errors is not available. Thus, the Microsoft invention introduces a new technique for responding to errors.

Additionally, the operational requirements 800 include low overhead requirements 810. as previously mentioned, hard link bandwidth between nodes is limited. The principles described in the invention provide error correction techniques that not only meet real-time data requirements 805, but also allow for a small amount of additional overhead to be placed on the bus protocol.

Operational requirement 800 further includes a low power requirement 815. based on specific characteristics of said data streams, embodiments are able to selectively deactivate power consuming nodes to reduce power consumption. For example, in a head-up display environment, embodiments are able to cause the laser to enter a low power state based on detection of specific packets transmitted between nodes.

In order to fulfill the above requirements, embodiments cause a particular type of packet to be transmitted from one node in a data stream to another node. FIG. 9 illustrates an example packet 900 that represents the packet discussed in FIG. 2. Notably, the packet 900 may take different forms, including a control data type packet 905, a non-control data type packet 910, and an end condition packet 915.

After all the virtual channels have finished sending their respective data streams, the last virtual channel that has finished sending its data stream sends an additional final end condition packet to give the status of all the virtual channels. The sending node does not send any new traffic for the virtual channel until all virtual channels finish transmitting their respective data streams for the scan. When a new scan is initiated or a new frame is initiated, the virtual channel will start sending packets again.

Control data type packet 905 This packet type includes control data for controlling how a particular operation is performed. For example, with reference to the header display example, the control data type packet 905 may include timing information regarding when the laser will emit laser light, payload type information, virtual channel information, and may further include placement information regarding where the laser will pulse its laser light.

The control data type packet 905 may also include header information indicating which channel or virtual channel to use. For example, each laser of the head unit may be associated with its respective virtual channel. The control data type packet 905 may include control information for the corresponding laser used for the virtual channel and virtual channel information.

Due to the importance of the control data type packet 905, the packet is protected using the error correction code ECC 925. It should be clarified that because the time and location information contained in the control data type packet 905 is differentially compressed and important, ECC 925 is used to protect the data from errors that may occur during transmission.

If there is an error in the data allowed to be included in the control data type packet 905, the entire scan may be corrupted, resulting in the introduction of artifacts into the resultant image frame.

FIG. 10 provides an additional description of the ECC 1000, which represents the ECC 925 of FIG. 9. Specifically, the ECC 1000 includes at least two types of protection, including single-bit error correction 1005 and two-bit error correction 1010.

In one embodiment, the error protection is x bits of error correction and x+1 bits of error detection. The ECC 1000 is selected to ensure that the ECC 1000 supports the error rate.The ECC 1000 may be 8 bits per control data type packet, or another number of bits depending on the length of the control data type packet.

Returning to FIG. 9, the non-control data type packet 910 is a packet that includes payload information, such as for how to illuminate a particular pixel in a scan, such as color intensity, duration, laser pulse size, and the like. In this way, the non-control data type packet 910 includes pixel data 930 describing the mechanism of how to illuminate the pixel.

Although a single control data type packet may be sent over the virtual channel in each new data stream, any number of non-control data type packets may be sent in the data stream. Also, corruption of any single non-control data type packet may result in only a single pixel being corrupted.

Headsets typically have refresh rates between 90 Hz and 120 Hz and have very high resolution, so users may not notice individual pixel corruption. In view of the described conditions, embodiments selectively avoid applying ECC to non-control data type packets 910. errors occurring in non-control data type packets 910 can simply be accepted as the impact caused by any such errors is negligible.

By applying the ECC 925 to the control data type packet 905, there is no need to retransmit the control data type packet 905 because the ECC 925 allows for error correction and detection at the receiving node, and allows for real-time processing, thereby satisfying the real-time data requirement 805 of FIG. 8.

Similarly, by applying ECC 925 only to control data type packets 905 and not to non-control data type packets 910, embodiments satisfy the low overhead requirement 810 because the additional data is applied only to a single packet and not to multiple packets. So the additional overhead is again insignificant.

In addition, by applying ECC, embodiments are able to automatically recover when an error is detected, thereby satisfying the automatic hardware recovery requirement 825.Thus, embodiments impose different data protection requirements based on data types or rather different packet types.

End Condition Packet 915 is a packet used to be triggered at the completion of the data stream. In other words, the end condition packet 915 is the last packet sent through the data stream after all non-control data type packets have been sent.

If no pixels are illuminated by a particular laser in a particular scan, there may be no control data type packets or non-control data type packets transmitted on the corresponding virtual channel of the laser. Instead, only end condition packets 915 may be transmitted. to help mitigate possible errors, multiple end condition packets may be transmitted.

FIG. 11 shows an example scenario involving a plurality of virtual channels 1100 and three different packet types introduced in FIG. 10 being transmitted. FIG. 11 shows a scenario where there are 12 virtual channels and the data stream flow is flowing from left to right. In addition, the different packet types are illustrated using different coloring techniques. In this example scene, each virtual channel corresponds to a single laser in the headset and each virtual channel has its own corresponding data stream being transmitted.

Figure 11 shows how the data stream for channel 1 begins with one control data type packet, then includes five non-control data type packets, and then includes an end condition packet. Channel 2 is similar in that it includes one control data type packet, two non-control data type packets, and one end condition packet. Channel 1 and Channel 2 are transmitting data because the laser corresponding to the channel will be used to illuminate the pixels in the scan.

On the other hand, channel 3 corresponds to a laser that will not be used to illuminate the scanning pixels. Thus, the data stream transmitted through channel 3 comprises only an end condition packet. This end condition packet beneficially informs the corresponding laser that it will not be used for this particular scan.

Accordingly, if the laser is not intended to be used during a particular scan, the data stream transmitted for the virtual channel for the laser will include only one or more end condition packets.

FIG. 12 focuses on virtual channel 1200, which represents virtual channel 1100 of FIG. 11. FIG. 12 shows how an end condition packet can be transmitted through channel 1. Said end condition packet operates as a trigger 1210 to notify said system that the corresponding power node can be switched from a power consumption mode to a reduced power consumption mode. This trigger operation is beneficial because the system now knows that the power consumption node is idle and so can transition to the reduced power mode.

For example, in a head-up display environment, the head-up display includes a laser transmitter 1215, which itself includes lasers 1215A, 1215B, and 1215C. channel 1 is a virtual channel that corresponds exclusively to laser 1215A. Transmission of the end condition package via channel 1 allows the system to recognize that the use of laser 1215A is now complete, and thus can be transitioned from active mode 1220, which generates laser light, to low power mode 1225 or perhaps even off mode 1230.

As shown in FIG. 12, said end condition packet triggers the operation that laser 1215A is now no longer emitting laser light, while lasers 1215B and 1215C may still be emitting laser light. Referring to FIG. 11, the transmission of the end condition packet at each of the different virtual channels causes the node corresponding to each virtual channel to transition to a low power state, thereby enabling the system to maintain power.

Turning briefly to FIG. 11, channel 1 is shown as the last channel to transmit the end condition packet. Regardless of which virtual channel last transmitted the end condition packet, the data stream of said virtual channel is tasked with performing a so-called encapsulation operation, wherein the end condition packet constitutes a "final" end condition type packet 1315, as shown in FIG. 13.

The final end condition type packet 1315 provides additional information to the receiving node indicating that all data streams for all other virtual channels have been completed. This functionality is beneficial in a head-up display environment because the final end condition type packet 1315 symbolizes the end of the scan.

The Microsoft patent application titled "Error recovery and power management between nodes of an interconnection network" was originally filed in August 2023 and was recently published by the USPTO. published by the U.S. Patent and Trademark Office.

Generally speaking, after a U.S. patent application is examined, it will be automatically published 18 months from the filing date or priority date, or it will be published within 18 months from the filing date at the request of the applicant. Note that publication of a patent application does not mean that the patent is approved. After a patent application is filed, the USPTO requires actual review, which can take anywhere from 1 to 3 years.

In addition, this is only a patent application, which does not necessarily mean that it will be adopted, and it is also uncertain whether it will be actually commercialized and the actual results of its application.