AI empowers MR, Microsoft releases Copilot in Dynamics 365 Guides

(XR Navigation Network November 16, 2023)IgniteMicrosoft The 2023 conference announced that it will bring AI assistant Copilot toDynamics 365 Guides, combines the power of generative artificial intelligence and mixed reality to help frontline workers complete complex tasks faster, solve problems, and reduce disruption to workflow.

New technicians need to improve quickly while learning how to operate and maintain equipment. When equipment stops working, staff must act quickly to diagnose the problem, implement repairs, and then document the work.

Copilot in Dynamics 365 Guides can help workers handle complex equipment and processes in industrial settings. Using this tool, instead of searching through reams of digital documents or paper manuals, employees can use natural language and human gestures to ask for relevant information. Copilot uses generative artificial intelligence to search for information from technical documentation, service records, training content and any other data sources managed by customers, then aggregates the information and overlays it on

Microsoft said that the combination of mixed reality and artificial intelligence can present human-centered interfaces and is expected to change the way the real world operates. Now, Copilot can help frontline workers in real time via voice and holograms.

Here’s how Microsoft Plan 365 for Dynamicss unlocks Copilot’s features in the Guides:

Click and ask

Humans have always manually learned and interacted with their environment and asked questions of others at work. Copilot in Dynamics 365 Guides are based on natural interactions and focus on helping employees better understand the world around them. Workers can point to specific parts of equipment and ask Copilot questions, such as "What is the torque limit for this bolt?"

Additionally, new employees can just ask general questions, such as "What is the function of this component?". Copilot recognizes what the user is referring to and provides an answer. 3D models and spatial maps provide the basis for the technology, helping to identify the objects workers are looking at and then projecting holograms to display operating instructions. Employees can ask more complex questions, such as "Tell me where the sensor module is on this machine," and Copilot will display a hologram and location.

Generate spatial content

Copilot in Dynamics 365 Guides can significantly reduce the early investment required to create spatial content, accelerating the out-of-the-box value of mixed reality. Copilot simplifies the creation, management and delivery of mixed reality content using customer-defined data sources, enabling companies to leverage mixed reality faster.

Step-by-step help

OpenAI Copilott supports Service in Dynamics 365 Guides to provide more than one-time answers. Workers can have progressive conversations like, “Tell me the steps to remove the filter,” or “I disconnected the motor, what should I do next?” At this point, Copilot can track the process and conversations using a curated knowledge base, using Express answers with voice, text or 3D holograms, eliminating ambiguity and reducing the need for rework.

information on demand

Microsoft Hololens can display relevant data around workers that they need to reference. Receiving IoT sensors, operational data and service records, Copilot in Dynamics 365 Guides can answer a range of questions, such as "What is the current humidity level?" or "When was the last inspection and what issues were found?"

HoloLens HoloLens 2 allows Copilot to pinpoint specific assets, identify them, and access relevant information sources in real time. Frontline workers can access a wealth of information and troubleshoot or fix glitches with the help of artificial intelligence.

Integrate expert knowledge

Often only experienced workers have the knowledge needed to complete complex tasks or repair critical equipment. But when those workers are no longer available or retire, the knowledge is lost. Copilot in Dynamics 365 Guides can use notes or phone calls from previous expert activities to guide new hires through the same process. This makes it easier and faster for employees to learn from each other, automating knowledge transfer, thereby increasing workforce productivity and business agility.

Streamlined documentation

Workers will be able to use Copilot in Dynamics 365 Field Service to speed up the process of completing and documenting work, mobile services. Workers do not need to fill out written documents, they only need to describe Copilot's progress with their voices, and Copilot will then provide relevant suggestions, effectively check service tasks, add notes, update product quantity and status, and speed up data entry.

Copilot in Dynamics 365 Guides transform the real world into tips and anchors for artificial intelligence. The tool puts people at the heart of the product, fundamentally changing how computing technology helps frontline workers take action in the real world. Copilot in Dynamics 365 Guides offer huge potential for businesses seeking to be more efficient and effective.

Private preview now available, public preview coming soon

Copilot in Dynamics 365 Guides has launched in a private preview with a limited set of features. In the upcoming public preview, it will be available first for HoloLens and later for mobile devices.

Microsoft ultimately emphasized privacy protection, saying that Copilotot in Dynamics 365 Guides was developed based on responsible artificial intelligence standards and is in line with the company's long-term commitment to data security and privacy.