Skip to content

R1 Pro VR Teleop Usage Tutorial - ROS2

This tutorial is for ATC SDK V2.1.2 and the later version.

1. Preparations Before Startup

1.1 Hardware Preparation

Item Quantity
VR Headset 1
Handheld Controllers 2
Type-C cable 1
Wrist Camera (Optional) 2
Wrist Camera Holder (Optional) 2
USB-A-to-USB-C Cable 1.5m 2
R1 Pro Robot - ROS2 1
PC -Ubuntu 22.04（Non-VM） 1

1.2 Software Preparation

Visit the Galaxea Software Changelog to download the latest SDK.

Download VR Device Configuration SDK in G-Drive:

  • Meta-related-install.zip：For new device activation.
  • platform-tools-latest-windows.zip：ADB files for installing the data collection APP inside the VR headset.
  • vr_teleop-0.1.4.apk：Data collection APP for the VR headset.

1.3 EDP - Emboddied Development Platform

Welcome to use the Emboddied Development Platform by Galaxea! It provides free 100GB of storage and 100 free data download capacity to help you visualize data acquisition, annotate and process data, and export model data.

Here is the Demo account:

Contact us for your FREE account!

2. VR Device Setup

If the VR device you are using is provided by GALAXEA, you may skip Sections 2.1 to 2.5 (which have been pre-configured at the factory) and proceed directly to Section 2.6.

2.1 Activate VR Device Developer Mode

Please refer to the Meta Quest 3 developer mode user guide to complete the activation.

2.2 Install VR Device SDK

  1. Download and extract the platform-tools-latest-windows.zip.
  2. If an old version APK is installed, uninstall it first. VR_2.2_uninstall_old_apk
  3. Connect the VR device to the computer using the Type-C cable. Within the VR device, confirm and allow the USB connection.
    VR_2.2_allow_usb
  4. Extract the .zip and navigate to the folder path. Copy the vr_teleop-0.1.4.apk file to this path.
  5. Open Command Prompt (CMD) in this path and execute the following command to install the app: 
    .\adb.exe install vr_teleop-0.1.4.apk
    If the command displays Success, the installation was successful.

2.3 Connect WiFi

After putting on the VR headset, click the WiFi icon on the left side of the bottom navigation bar on the initial interface to bring up a new dialog box.

Then, click on the WiFi option on the left and connect to the same WiFi network as the robot.

Note：A "Network Restricted" prompt is normal as this network cannot access the external internet.

VR_2.3_connect_wifi

2.4 Disable Quest Boundary System

Open “Setting” → “Advanced” → “Developer” → Turn off“Physical space features”. VR_2.3_connect_wifi

After disabling,

  • The system no longer prompts to set boundaries. Movements during collection won't interrupt the process.
  • Passthrough view cannot be enabled on the main menu page.

Ensure a safe operating environment after disabling boundaries to avoid collision risks.

2.5 Prevent System Sleep

  1. Open “Menu” → “Settings”.
  2. Click “General” → “Power” → “Display off”，choose 4 hours. VR_2.5_sleep

Disabling sleep enables the "Detachable Mounting" solution. Manually power off the VR device after use to prevent battery drain.

Maintain sufficient battery levels for the VR headset and controllers. Low battery affects the pose transmission frame rate of the VR controllers, causing teleoperation lag.

2.6 Obtain VR Device IP Address

Within the VR device, click the connected WiFi, open the network page, scroll down, find and record the IP address. VR_2.6_vr_ip

2.7 Detachable Wearing Solution

To avoid discomfort from prolonged headset wear, we provide a Detachable Wearing Solution. Before using the detachable wearing method for the VR headset, please complete Section 2.4 and Section 2.5.

If your VR device is provided by Galaxea, the relevant configurations have been pre-set during factory shipment, and the above operations are not required. Please use the device in a safe and open environment to ensure no collisions occur during operation.

The usage instructions are as follows: 1. Detach the top strap from the headset top and remove the facial interface for flexible neck mounting. 2. Hang the headset around the neck, adjust it to face forward with the display perpendicular to the ground. VR_2.7_detachable

3. Robot Configuration

Visit the official VR tutorial website to view and copy the code directly.

  1. Log in to the robot. 

    ssh nvidia@${R1_Pro_IP}

  2. Configure the communication environment. 

    echo "export ROS_IP=your_ros_ip" >> ~/.bashrc 
echo "export VR_IP=your_vr_ip" >> ~/.bashrc 
source ~/.bashrc

  3. Install the additional dependencies. Bash sudo apt install ros-humble-rosbag2-storage-mcap pip3 install websockets pyquaternion tos

  4. Configure EDP device serial number 

    # Obtain the robot Serial Number (located on the sticker at the bottom torso joint. 
# If you cannot find it, please contact Galaxea customer support.)
# e.g., S2RL100P25110

sudo mkdir -p /opt/galaxea 
sudo chown -R nvidia:nvidia /opt/galaxea 
mkdir -p /opt/galaxea/body 
echo "S2RL100P25110" | tee /opt/galaxea/body/RSN > /dev/null 
echo "S2RL100P25110" | tee /opt/galaxea/body/RSN_REAL > /dev/null 
echo "R1PRO" | tee /opt/galaxea/body/ROBOT_NAME > /dev/null

# Verify successful write   
cat /opt/galaxea/body/RSN 
cat /opt/galaxea/body/RSN_REAL 
cat /opt/galaxea/body/ROBOT_NAME

  5. Configure Data Collection Module After the upgrade is complete, power off the R1 Pro and restart it. Once restarted, the software package configuration will be finalized, and the VR teleoperation functionality will be ready for use.

4. Teleoperation Startup

Note: All operations in this section must be completed and confirmed every time upon startup.

4.1 Launch the Robot

  1. Log in to the robot. 

    ssh nvidia@${R1_Pro_IP}

  2. Navigate to startup directory. 

    cd /home/nvidia/galaxea/install/startup_config/share/startup_config/script/

  3. Launch the programme. 

    ./robot_startup.sh boot ../sessions.d/ATCStandard/R1PROVRTeleop.d/

4.2 Launch the VR Device

Note: Please ensure that the VR headset is successfully connected to the same WiFi network as the robot. Wear the VR headset properly and hold both controllers, then proceed with the following operations.

4.2.1 Launch GalaxeaVR APP

VR_4.2.1_launch_app

  1. Open GalaxeaVR App, click the black cube icon in the bottom right corner.
  2. Aim the ray emitted by the Bluetooth controller at the green IP input box. Wait for the green input box to a subtal color change from dark to light, then press the T button on the right controller to click the input box.
  3. Once the keyboard pops up, enter the robot’s IP address and click the "Start" button to begin.
  4. Immediately lower both hands naturally to your sides. After waiting 3 seconds, press the G button on both the left and right controllers to resume operation and begin controlling.

The robot will mirror your movements. Ensure safety by moving slightly first, confirming no obstacles nearby.

You can complete the simple operation using the following steps:

  • Stop operation: Long press the B button on the right controller for over 2 seconds to stop VR teleoperation.
  • Control Arms: Hand movements control robot arm movements.
  • Control Grippers: Hand movements control robot arm movements.
  • Pause right arm: Short press the G button on the right controller once to stop the right arm. Press again to unpause.
  • Pause left arm: Short press the G button on the left controller once to stop the left arm. Press again to unpause.

Refer to Section 5.2 for detailed controls.

4.2.2 VR App Display Instruction

VR_5.2_vrapp_button

5. Controller Instructions

5.1 Robot Joystick Controller

VR_5.1_robot_controller

Before powering on the remote: Set ALL switches (SWA/SWB/SWC/SWD) to the UP position. When not in use: Keep all switches in UP position to maintain robot standby state and prevent accidental activation.

Position the switches to specified settings to immediately control the robot's corresponding functions. VR_5.1_robot_controller_table

  • Host Control Mode (DDUU): Host PC must launch chassis control program before operation.
  • Remote Control Mode (DDMU: Directly control chassis via remote after robot powers on. No host PC intervention required.
  • Emergency Stop: Set SWD to DOWN position (any mode); Robot requires reboot after E-stop engagement.Reset SWD to UP before restart.

5.2 VR Handheld Bluetooth Controllers

Please practice operating the robot in an open area while ensuring the safety of personnel and objects. It is recommended to begin formal data collection only after you have become familiar with its operation.

VR_5.2_vrapp_button

6. Mode Switching Instructions

After remote operation is started, the default mode is BIMANUAL Mode. Users can switch different operation modes through the VR device remote controller.

Mode Instruction Note
BIMANUAL Press both T buttons down for 1s To control the chassis, enable the robot joystick controller and ensure SWA, SWB, SWC, SWD are all in the top position.
Torso Press right stick down for 1s -
Torso Press left stick down for 1s -

6.1 Bimannual Control Mode

After teleoperation is initiated, the default mode is the dual-arm Bimannual control mode. Different operating modes can be switched using the VR controllers.

Function Instruction Note
Arm Following Arm end positions follow the movement of the VR handheld controllers. -
Gripper Control Both controllers’ T button controls grippers open/close respectively. -
Arm Pause Both controllers’ G button pause arms respectively.
Press once to pause, press again to unpause.		 When unpausing, ensure the controller position matches the paused position to prevent sudden large arm movements.
Chassis Forward/Back Move the left controller’s stick forward/backward. -
Chassis Translation Move the left controller’s stick left/right. -
Chassis Spin Move the right controller’s stick left/right. -

6.2 Torso Control Mode

After switching to Torso Mode, control the torso pose as follows:

Function Instruction Note
Waist Rotation Move the left controller's stick left/right. Torso Joint 1 movement
Waist Pitch Movement Move the left controller's stick forward/backward. Torso Joint 2 movement
Torso Lift Move the right controller's stick forward/backward. Torso Joint 3 movement
Torso Movement Move the right controller’s T button forward.
Move the right controller’s G button backward.		 Torso Joint 4 movement

6.3 Reset Mode

6.4.1 Preparations Before Connection

Please prepare the following items before connecting the wrist cameras:

Function Instruction Note
Reset Arms to Initial Position Press BOTH two controllers' T + G buttons simultaneously. Press all four buttons simultaneously to reset the robot's arms to their initial startup position. 
# To modify the initial startup position, edit the joint positions in the launch file:
vim ~/galaxea/install/teleoperation_ros2/share/teleoperation_ros2/launch/vr_teleoperation.launch.py
# Modify the initial joint positions here:
{'vr_initializer_left_target_joint_states_r1pro': [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]}, 
{'vr_initializer_right_target_joint_states_r1pro': [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]}
# Modify gripper range here:
'gripper_openning_threshold', default value='80.0' #Max.: 100.0 
'gripper_closing_threshold', default value='0.0'  #Min.: 0.0

6.4.2 Connecting Camera Cables

  1. Install the wrist camera bracket to the robot's wrist and secure the camera.
  2. Use the USB-A-to-USB-C adapter cable to connect any USB-A peripheral interface on the back of the robot and the wrist camera's USB-C interface.

7. Wrist Cameras Connection

7.1 Connect the Camera Cables

  1. Install the wrist camera mount onto the robot's end. Align the three holes on the bottom of the mount with the corresponding empty slots on the surface of the gripper motor, and secure it with screws.
  2. Mount the wrist camera onto the top of the holder, ensuring the Type-C port is facing the inner side.
  3. Use a USB-A to USB-C adapter cable to connect any USB-A peripheral interface on the back of the robot to the Type-C interface of the wrist camera.

7.2 Configure the Cameras

To avoid confusion between the serial numbers of the two cameras during configuration, it is recommended to connect and configure one camera at a time.

  1. Navigate to the wrist camera config directory. 
    cd /home/nvidia/galaxea/install/realsense2_camera/share/realsense2_camera/launch
  2. (e.g.: Connect Left Wrist Camera first) After connecting the left wrist camera cable, run the following command to view and record its Serial Number. 
    rs-enumerate-devices  | grep Serial
    VR_7.2_grep_serial

  3. Connect the right wrist camera cable, then run the command again to view and record its Serial Number. 

    rs-enumerate-devices  | grep Serial
    VR_7.2_grep_serial2

    The order of the serial numbers is not related to the sequence in which the cameras are connected. Tis recommended to connect and record one camera first before proceeding with the second. 4. Use vim to edit the camera serial numbers in the launch file. 

    vim rs_multi_camera_launch.py
# Regardless of connection order, the left wrist camera name is fixed as ‘camera1’, the right as ‘camera2’.
    Enter the previously recorded serial numbers in the corresponding name fields. VR_7.2_vim 5. Restart the program. 
    cd /home/nvidia/galaxea/install/startup_config/share/startup_config/script/ 
./robot_startup.sh kill 
./robot_startup.sh boot ../sessions.d/ATCStandard/R1PROVRTeleop.d/
    6. Check the camera frame rate. 
    cd /home/nvidia/galaxea/install/
ros2 topic hz /hdas/camera_wrist_right/color/image_raw/compressed 
ros2 topic hz /hdas/camera_wrist_left/color/image_raw/compressed
    If values appear for both cameras and are around 15Hz, the connection is successful. VR_7.2_rate

    Wrist cameras only need configuration once per robot. They will start automatically with the robot using the method in section 4.1 thereafter.。

8. Data Collection Instruction

If you are using the Galaxea EDP platform, you may skip sections 8.1 to 8.3 and proceed directly to section 8.4 to begin recording. If not, please follow the steps below.

8.1 Data Format and Data Retrieval

The collected data format is ros2bag, with the file suffix .mcap.

Default storage path: /home/nvidia/GalaxeaDataset/{date}/

(date is the current date in the format: YYYYMMDD (e.g., 20250307).)

8.2 Data Recording Configuration

The default configuration file for data recording is located at: 

/opt/galaxea/data_collection/data_task_config.json

Configuration file describes the information of the collection task. Users can modify it as needed.

{
    "project_info": {
        "project_name": "sop_test"
    },
    "task_info": {
        "task_name": "sop_test_data_collection",
        "task_owner": "san.zhang"  
    },
    "operation_info": {
        "teleoperation_type": "VR",
        "location": "suzhou",
        "operator_name": "si.li"
    }
}

8.3 Data Offloading File Structure

Data is saved in rosbag + json format, with files corresponding one-to-one.

# e.g.: The following folder and JSON file represent one data packet. The folder contains the .mcap data file and its meta information.

S2R12000P18245_20240213173320125_RAW
S2R12000P18245_20240213173320125_RAW.json

# Format: robot_serial_number+timestamp+RAW
# robot_serial_number：Robot serial number, located at /opt/galaxea/body/RSN
# timestamp：Data collection timestamp, precise to milliseconds.
# RAW：Represents raw offloaded collection data.

8.4 Start Recording

Data recording operations are performed using the left VR controller.

Function Instruction Note
Start recording Short press the X button on the left controller. Begins data recording.
Stop recording Short press the Y button on the left controller. Stops data recording
Delete Recording Short press the X button on the left controller. If recording is already in progress, pressing X again stops and deletes the current recording. Data is not saved.

For any issues during installation or startup, please contact us at support@galaxea.ai for technical support!