Meta-intelligence research: letting headsets and glasses understand your room layout and furniture

Meta的最新人工智能研究系统可以让头戴设备和眼镜了解您的房间布局和家具。

be known asSceneScript的系统，Meta's system, which Meta says utilizes the same basic technology as large-scale language models (LLMs), with the difference being that instead of predicting the next language segment, it predicts architectural and furniture elements. These elements are predicted in a 3D point cloud capture, which is the same independent localization tracking already performed by the headset.

The output results in a series of primitive 3D shapes with basic boundaries representing the basic boundaries of a given piece of furniture or element.

Quest Quest 3 is able to generate a raw 3D mesh of your room and from this 3D mesh infer the location of walls, floors and ceilings. But the headset does not yet know which shapes in the mesh represent more specific elements, such as doors, windows, tables, chairs, and sofas.The Quest headset does allow users to manually label these objects, such as simple rectangles, but since this is optional and time-consuming, developers can't rely on the user to have already done so.

When technologies like SceneScript are integrated into Quest 3's mixed reality scene setup, developers can automatically place virtual content on or around specific furniture elements. They could replace windows with virtual doors, chairs with virtual seats, or turn sofas into filled-in positions. These features are currently accomplished by requiring users to manually mark furniture or sometimes use manual algorithms to interpret the scene mesh, but SceneScript makes it seamless and automated.

Technologies like SceneScript also have huge potential for future AI assistants on headset devices, and eventually augmented reality glasses.Meta gives examples like "Will this table fit in my bedroom?" or "How many cans of paint do I need to paint this room?" and commands such as "Put [AR/MR app] on the big table."

However, at this point Meta is only describing SceneScript as research, not a recent product feature update. While the technology may eventually be applied to Quest 3, there is no indication that this will happen anytime soon.