8th Wall introduces generative AI tools to its WebAR platform

TL;DR: 8th WallAnnouncing the launch of newGenAImodule that allows developers toInworldandOpenAIintegrated into theirWebARIn experience. The new module simplifies the process of integrating GenAI into WebAR projects, allowing developers to create more dynamic, interactive and engaging experiences. According to 8th Wall, users will now be able to actively shape the experience rather than just passively consuming content.

August 30, 2023 – 8th Wallyes Niantic A subsidiary of Google, a provider of web-based augmented reality (WebAR) development platforms, announced the introduction of Generative AI (GenAI) modules and sample projects on its platform for use in Inworld and OpenAI.

The introduction of GenAI, combined with 8th Wall's engaging WebAR, now offers developers, brands and agencies a new way to tap into a seemingly limitless resource of creativity and give audiences greater control over immersive experiences.

8th Wall noted that the platform already allows developers to integrate with GenAI tools to accelerate development and build WebAR experiences. This technology includes DALL·E and Stable Diffusion for adding text to image inputs, Blockade Labs for adding 360-degree images to the sky, Inworld AI for incorporating human-like interactive characters,Hugging FaceandControlNet, for adding texture to faces, orMove.aiandWonder Dynamics, for adding realistic motion.

With this latest announcement, it is now easier for developers of all skill levels to build WebAR experiences using GenAI, thanks to newly integrated GenAI modules and sample projects from Inworld and OpenAI. According to 8th Wall, these new modules help eliminate the development effort that often hinders adding GenAI technology to WebAR experiences, giving developers a way to introduce the technology into WebAR experiences without tedious work. Developers only need to add the module to the project to enable GenAI-based features in the experience.

AI-based filters on social apps like TikTok have become increasingly popular in recent months, and one of the big reasons may be the sometimes unpredictability of AI-generated content. By combining WebAR with GenAI, developers no longer create one-off content, but combine the two technologies to create dynamic, evolving experiences that respond to user interactions, which helps keep users more engaged and Looking to revisit filters and experiences to create new results.

In addition, with the 8th Wall platform, developers only need to build once to achieve a consistent experience across smartphones, computers, tablets and virtual and mixed reality devices.

Inworld Character Engine

Inworld is a powerful GenAI technology that enables developers to combine their WebAR experiences with interactive and realistic AI-driven characters. These characters, avatars, and brand ambassadors can learn, adapt, and interact with users in a more personal way.

8th Wall said it partnered with Inworld to create Inworld AI Integration Moduleand a sample project. This module allows developers to combine Inworld's characters with 8th Wall's tools to provide new interactive experiences. For example, developers can use Inworld with image targets or world effects to see familiar character avatars on product packaging and engage in natural conversations with them. 8th Wall says that with Inworld's "character brain," characters can mimic the entire range of human expressions, meaning every conversation will be different and contextually relevant.

Developers can now introduce such roles into the 8th Wall experience without the complexity of server configuration or token authentication. Once added to a project, the module handles the backend work, meaning developers only need to enter an API key to start developing. use sample project, developers can build their first Inworld and 8th Wall experiences, using image targets to initiate brand encounters with the fictional brand mascot, Rainbow Ray.

8th Wall mentioned in a blog post: "As the future form of mixed reality gradually emerges, AI characters driven by Inworld will become a key component of building virtual worlds, providing users with unlimited interaction possibilities in the digital space."

New 8th Wall integration module for OpenAI

As part of the latest release, 8th Wall also launched DALL·E andChatGPTNew developer tools for image generation and AI-driven text chat.

Used independently, OpenAI's DALL·E is a powerful GenAI model that creates highly detailed and creative images based on user text prompts. When DALL·E is combined with WebAR, developers can use these text-to-image prompts to create experiences that put users’ creativity in control. According to 8th Wall, users can input unique prompts to generate images that enhance skies, conform to human faces, reimagine real-world objects, or even serve as new textures in WebAR experiences.

Similarly, ChatGPT enables developers to create personalized and interactive WebAR experiences by understanding and responding to user input in real time. 8th Wall says developers can now use this type of interaction to develop text-based conversations with WebAR characters, work with ChatGPT to place objects in the scene, and even create experiences that combine DALL·E and ChatGPT for real-time changes.

Like the Inworld AI Integration Module, the new one is available for OpenAIIntegration Module for OpenAIHelps save time for developers and sets up a project code to call DALL·E and ChatGPT. After developers add their modules to the 8th Wall project, they simply add the OpenAI API key and start building.

Likewise, 8th Wall has created a sample project that developers can use to test the incorporation of AI into WebAR experiences and discover how to use the DALL E to generate 3D textures from text and apply them to virtual objects in real time. In addition, developers can customize the ChatGPT prompt box to provide users with another way to control experience elements.

Ian Curtis, an XR developer at 8th Wall, said of the tools: “What interests me most about incorporating generative AI into augmented reality is that it enables each user to be the architect of their digital world. They don’t No longer passively consuming content, but actively creating it, using their imagination to shape their experience.”

Developers can now exploit the potential of GenAI to transform WebAR experiences with 8th Wall’s newly announced tools, and use GenAI with the platform’s many features, including but not limited to Image Targets, Face Effects, World Effects, Sky Effects, Lightship VPS and Shared AR tools.

For more information about 8th Wall and its tools for developing and creating WebAR experiences, visit the company’s website.

Image/Video Source: 8th Wall/YouTube