AR远程协作

一、方案介绍AR远程协作子系统由音视频服务、AR眼镜端APP、企业级PC端应用、手机端APP（安卓）等模块组成，通过音视频对话、屏幕共享、截图标注、协作白板等形式，为前端工作人员和后端指挥中心、专家之间的远程协作搭建技术通道。二、系统功能介绍    AR远程协作子系统由音视频服务、AR眼镜端APP、企业级PC端应用、手机端APP（安卓）等模块组成，通过音视频对话、屏幕共享、截图标注、协作白板等形式，为前端工作人员和后端指挥中心、专家之间的远程协作搭建技术通道。三、应用场景    覆盖各应用场景的有效触达，设备远程运维，产线遭遇紧急故障，无需等待专家到场即可快速远程定位解决；供应商远程支持，供应商可同时支持全球范围的多个工厂，协助解决设备紧急故障；远程培训，远程第一视角实操培训和学习分享，帮助企业低成本，快速培育人才。 四、功能优势   AR远程协作系统的核心优势在于可以将第一视角画面进行多方共享，支持一对多、多对一、多对多等各种场景，支持在通讯过程中传输多种类型的文件，包括文档、图片、消息等。远程用户可一键暂停通讯画面或直接在画面进行实时标注，指导结果将同步展现在现场用户视野中。五、功能使用说明1、实时远程通讯    实施远程通讯功能包括音频和视频的双向通道。2、音频双向通道提供高质量音频实时传送服务，实现新型的前后端用户交流方式。前端人员在现场即可享受远程专家的协助，并在专家指导下完成工作。 3、视频双向通道提供高清视频实时传送服务，实现新型的前后端用户交流方式。前端人员在现场即可享受远程专家的协助，并在专家指导下完成工作。 4、账号管理在云VR管理平台新增AR远程协作账号管理模块，支持账号新增、编辑、删除操作，可根据公司名称、账密鉴权并生成远程协作账号二维码，支持复制到粘贴板进行分享扫描登录，支持刷新更新二维码。 5、 AR眼镜端APP5.1智能配网通过固定web地址进行wifi鉴权（如需通过wifi上网），生成二维码供AR眼镜扫一扫智能配网。 5.2远程协作鉴权登录AR眼镜扫描账号二维码，可直接登录远程协作系统，避免手动输入选择账号密码的不便。 5.3人员列表在AR眼镜中可查看人员列表信息、离线在线信息等，同时可以指定人员发起、接听通话。5.4屏幕共享协作请求接通之后，AR眼镜将摄像头画面实时分享给手机端或PC端用户，并可以进行语音通话。5.5截图标注 用户在手机或者PC端发起截屏标注指令后，可对截屏画面进行标注，标注完成后发送至AR眼镜端，此时，AR眼镜将收到截屏标注的整体画面，且支持观看。5.6协作白板 手机或PC端用户在发起远程白板指令后，AR眼镜可实时观看远程白板中的标注、涂鸦等画面。5.7屏幕远程 手机或PC端用户在发起远程屏幕分享指令后，AR眼镜可实时观看远程屏幕内容（文本、图片、PDF、视频等）分享画面。5.8协作视频录制 所有拍摄回传的音视频、照片数据、文字消息都会自动存储，并按照日期进行归档，可以随时调阅回放，同时，允许设置滚动存储机制。6、 企业级PC端应用6.1人员账号 企业账号分管理员和普通人员两类，管理员可对普通人员账号执行新增、编辑、删除、分组操作。6.2权限分配 支持功能模块权限灵活配置，可根据账号角色分配相应权限.6.3远程协作鉴权登录，PC端应用通过账号密码登录远程协作系统。6.4人员管理 从服务器调取人员列表信息、离线在线信息等，通过人员列表可发起、接听其通话。 6.5屏幕共享 协作请求接通之后，建立实时音视频通道，在PC端显示眼镜端摄像头实时画面，并可以进行语音通话。6.6邀请会商 支持邀请其他专家加入视频会商。6.7截图标注 通过点击截屏按钮发起截屏标注指令，可在截屏画面上进行标注并将画面发送至AR眼镜端。6.8协作白板 点击白板按钮，可打开白板，在白板上进行标注、涂鸦等操作并发送至眼镜端。6.9屏幕远程 点击屏幕分享按钮，可将屏幕内容（文本、图片、PDF、视频等）分享至眼镜端。7、手机端APP（安卓）7.1远程协作鉴权登录，手机端APP通过账号密码登录远程协作系统。 7.2人员管理手机端从服务器调取人员列表信息、离线在线信息等，可通过人员列表发起、接听其通话。 7.3屏幕共享协作请求接通之后，协作端将显示现场眼镜端摄像头实时画面，并可以进行语音通话。 7.4邀请会商在手机端邀请其他专家加入视频会商。7.5截图标注 手机端发起截屏标注指令，现场人员可在截屏画面上进行标注并发送至眼镜端，眼镜端人员可查看截屏标注的画面。7.6协作白板在手机端发起远程白板指令，可在白板上进行标注、涂鸦等操作并发送至眼镜端。 7.7屏幕远程 在手机端发起远程屏幕分享指令，可将屏幕内容（文本、图片、PDF、视频等）分享至眼镜端。

1. Solution Introduction
The AR remote collaboration subsystem consists of modules such as audio and video service, AR glasses-side APP, enterprise-level PC application, and mobile APP (Android). It builds a technical channel for remote collaboration between on-site staff and back-end command centers and experts through forms such as audio-video calls, screen sharing, screenshot annotation, and collaborative whiteboard.

2. System Function Introduction
The AR remote collaboration subsystem consists of modules such as audio and video service, AR glasses-side APP, enterprise-level PC application, and mobile APP (Android). It builds a technical channel for remote collaboration between on-site staff and back-end command centers and experts through forms such as audio-video calls, screen sharing, screenshot annotation, and collaborative whiteboard.

3. Application Scenarios
It covers effective coverage of various application scenarios: equipment remote operation and maintenance. When the production line encounters an emergency failure, it can be quickly located and resolved remotely without waiting for experts to arrive on site; supplier remote support, suppliers can support multiple factories around the world at the same time to assist in resolving emergency equipment failures; remote training, remote first-person hands-on training and learning sharing, helping enterprises to cultivate talents quickly at low cost.

4. Functional Advantages
The core advantage of the AR remote collaboration system is that it can share first-person perspective screens with multiple parties, supporting various scenarios such as one-to-many, many-to-one, and many-to-many. It supports transmitting multiple types of files during communication, including documents, images, messages, etc. Remote users can pause the communication screen with one click or perform real-time annotation directly on the screen, and the guidance results will be synchronously displayed in the field of view of on-site users.

5. Function Usage Instructions
1. Real-time Remote Communication
The real-time remote communication function includes two-way audio and video channels.
2. Two-way Audio Channel
Provides high-quality real-time audio transmission services, realizing a new type of communication method between front-end and back-end users. On-site personnel can receive assistance from remote experts and complete work under the guidance of experts.
3. Two-way Video Channel
Provides high-definition real-time video transmission services, realizing a new type of communication method between front-end and back-end users. On-site personnel can receive assistance from remote experts and complete work under the guidance of experts.
4. Account Management
Add an AR remote collaboration account management module on the cloud VR management platform, supporting operations such as account addition, editing, and deletion. It can conduct authentication based on company name and account password, generate remote collaboration account QR codes, support copying to the clipboard for sharing and scanning to log in, and support refreshing and updating QR codes.
5. AR Glasses-side APP
5.1 Smart Network Configuration
Perform Wi-Fi authentication via a fixed web address (if Wi-Fi access is required), and generate a QR code for AR glasses to scan for smart network configuration.
5.2 Remote Collaboration Authentication and Login
Scan the account QR code with AR glasses to directly log in to the remote collaboration system, avoiding the inconvenience of manually entering and selecting account passwords.
5.3 Personnel List
View personnel list information, offline/online status and other information in AR glasses, and can designate personnel to initiate and answer calls.
5.4 Screen Sharing Collaboration
After the collaboration request is connected, the AR glasses will share the camera screen in real-time to mobile or PC users, and voice calls can be conducted.
5.5 Screenshot Annotation
After the user initiates a screenshot annotation instruction on the mobile phone or PC, the screenshot can be annotated. After the annotation is completed, it will be sent to the AR glasses end. At this time, the AR glasses will receive the complete screenshot annotation screen and support viewing.
5.6 Collaborative Whiteboard
After the mobile or PC user initiates a remote whiteboard instruction, the AR glasses can view the annotation, doodling and other content in the remote whiteboard in real-time.
5.7 Remote Screen Sharing
After the mobile or PC user initiates a remote screen sharing instruction, the AR glasses can view the remote screen content (text, images, PDF, video, etc.) sharing screen in real-time.
5.8 Collaborative Video Recording
All captured and returned audio and video, photo data and text messages will be automatically stored and archived by date, which can be retrieved and played back at any time, and a rolling storage mechanism is allowed to be set.
6. Enterprise-level PC Application
6.1 Personnel Account
Enterprise accounts are divided into two categories: administrators and regular personnel. Administrators can perform operations such as adding, editing, deleting, and grouping regular personnel accounts.
6.2 Permission Allocation
Supports flexible configuration of function module permissions, and can allocate corresponding permissions according to account roles.
6.3 Remote Collaboration Authentication and Login
The PC application logs in to the remote collaboration system via account and password.
6.4 Personnel Management
Retrieve personnel list information, offline/online status and other information from the server, and initiate and answer calls through the personnel list.
6.5 Screen Sharing
After the collaboration request is connected, a real-time audio and video channel is established, and the real-time camera screen of the glasses end is displayed on the PC, and voice calls can be conducted.
6.6 Invite Consultation
Supports inviting other experts to join the video consultation.
6.7 Screenshot Annotation
Initiate a screenshot annotation instruction by clicking the screenshot button, annotate on the screenshot screen and send it to the AR glasses end.
6.8 Collaborative Whiteboard
Click the whiteboard button to open the whiteboard, perform annotation, doodling and other operations on the whiteboard and send it to the glasses end.
6.9 Remote Screen Sharing
Click the screen share button to share the screen content (text, images, PDF, video, etc.) to the glasses end.
7. Mobile APP (Android)
7.1 Remote Collaboration Authentication and Login
The mobile APP logs in to the remote collaboration system via account and password.
7.2 Personnel Management
The mobile terminal retrieves personnel list information, offline/online status and other information from the server, and can initiate and answer calls through the personnel list.
7.3 Screen Sharing
After the collaboration request is connected, the collaboration end will display the real-time camera screen of the on-site glasses end, and voice calls can be conducted.
7.4 Invite Consultation
Invite other experts to join the video consultation on the mobile terminal.
7.5 Screenshot Annotation
The mobile terminal initiates a screenshot annotation instruction, and on-site personnel can annotate on the screenshot screen and send it to the glasses end. The glasses end personnel can view the screenshot annotation screen.
7.6 Collaborative Whiteboard
Initiate a remote whiteboard instruction on the mobile terminal, perform annotation, doodling and other operations on the whiteboard and send it to the glasses end.
7.7 Remote Screen Sharing
Initiate a remote screen sharing instruction on the mobile terminal, and share the screen content (text, images, PDF, video, etc.) to the glasses end.

搜集汇总

背景与挑战

背景概述

该数据集描述了一个AR远程协作系统，通过音视频对话、屏幕共享、标注和白板等功能，实现前后端人员的远程协作，适用于设备运维、远程支持和培训等场景。系统支持多端应用，包括AR眼镜、PC和手机，具备实时通讯和文件传输能力。