Expert Karl Guttag Shares Vision Pro Native Office Excel App Hands-On Results

CheckCitation/SourcePlease click:kguttag

(XR Navigation Network February 07, 2024)along withappleVisionPro的发售，近眼显示技术专家卡尔·古塔格（Karl Guttag）开始着手研究这款设备。在日前一篇博文中，古塔格分享了关于Apple Vision Pro（AVP）的早期发现，并放出了数张镜后图（用相机拍摄通过显示透镜看到的影像）。

on the basis ofeye trackingThe rendering, perhaps, is "smart and clever."

AVP is sometimes "too smart for its own good" and results in poor visual artifacts. In addition to being used for selection, AVP's eye tracking can also be used to change the resolution and correct color problems (aberrations) in the optical system by pre-processing the image. This makes shooting tricky because the camera lens looks different from the human eye.

今天，我拼凑了些电子表格来测试我拍摄通过AVP光学元件看到的影像的能力。我首先使用AVP的原生Excel app显示两个大型Excel电子表格。一个电子表格是白色背景黑色文本，而这似乎是AVP令文本和线条看起来比原本应该的样子“更粗”。AVP似乎“增强”了电子表格的可读性。

But then I tried white text on a black background, and the boxes following the eye tracking started to appear to flicker. Fortunately, AVP's video captured the effect in the video below:

I want to emphasize that it's not just the camera or the AVP's video that captures the gaze point rendering issue, it's simultaneously what I've seen with my own eyes. I have provided the spreadsheet below so that anyone with an AVP can verify my findings. I tested this in Excel with AVP running. The easiest way to see the effect is if you go to "View" in Excel and use the magnifying glass to zoom in on the view 3 or 4 times to make the text and boxes smaller.

• KGOnTech-XLS-on-White-Large-Array (exel document: black text on white background)
• AVP-XLS-on-BLACK-Large-Array (exel document: white text on black background)

My first post-mirror chart experiment

Due to its eye-tracking based rendering, AVP has difficulty capturing images seen through AVP optics. The tracking effect behaves differently with different cameras and lenses. When setting up the camera, I could see that AVP was changing colors, sometimes resulting in photos with colors different from what my eyes saw.

Obviously, even for static objects like spreadsheets, AVP uses "point-of-view" variable resolution rendering. This re-rendering is based on the eye, and due to variations in 3D spatial locking (i.e. SLAM) results in artifacts seen in the lines of a spreadsheet with white text and a black background.

Also, the resolution of the monitor must be lower than that of the human eye, because if you look at a spreadsheet with white text and a black background, you can easily see the "twisted rope" ripple effect of anti-aliasing.

I should point out that without gaze point rendering, the entire image flickers. However, viewpoint rendering makes it worse because it creates a visible square at the border between the viewpoint area and the low-resolution area. By changing the resolution and thickness of the text and lines, "viewpoint rendering" makes the situation worse. In my opinion, a more elegant degradation would be to render the entire image the same way, make the entire image blink instead of the box, and thicken the borders. The key point is that the AVP's display, while much better than almost any other VR/MR headset, is not, as Apple says, "retina resolution".

Here are some behind-the-scenes images. The first was taken with the R5 camera, 28mm lens, and "pixel shifted" to get a 400 megapixel image.

The second photo below was taken with an Olympus D mark III with a 17mm lens. It doesn't have the resolution of the R5, but AVP's eye tracking performs better with this lens. This camera has a 24 megapixel sensor and then I used its pixel shift feature to capture about 80 megapixel images.

If you slip around the full-resolution image, you can make out the pixel grid in most of the image, but the text gets blurry faster. Initially, this seems to imply gaze point rendering. I haven't had a chance to check, but I suspect the resolution drop is with the squares in the white text black background spreadsheet.

summarize

Designers must be very careful when applying technical solutions. Sometimes, oranges are oranges when they are born in Huainan, but they are hedgehogs when they are born in Huainan.

The biggest problem is probably a bug in the AVP software or the Excel port, and I'm not saying it's the end of the world if it's not improved. There may be a way to "fade" the viewpoint rendering to alleviate the problem, but given the display resolution, I don't think there's any way to eliminate it. In the meantime, the second test I tried caused the viewpoint rendering to "crash/disappear". Since this happens so easily, it's likely to occur elsewhere. Fundamentally, this is largely down to the fact that the resolution of the monitor is not as good as the human eye.