Boomy Comprehensive Usage Document
I. About the Boomy
(I) Component Information
The Boomy machine has 21 degrees of freedom. One arm has 4 degrees of freedom, including (left-handed coordinate system):
Shoulder joint pitchShoulder joint rollUpper arm joint yawElbow joint pitch
One leg has 6 degrees of freedom, including:
Hip joint pitchHip joint rollThigh joint yawKnee joint pitchAnkle joint pitchAnkle joint roll
The lumbar joint has one degree of freedom, including:
Lumbar joint yaw
Having multiple joint motor degrees of freedom enables the Boomy robot to achieve precise motion and attitude control.
The detailed information is as follows:
| Parameter | Boomy | Boomy-Education Edition |
|---|---|---|
| Height, width, and thickness (standing) | 97cm*35cm*20cm | |
| Weight including battery | About 20.5kg | |
| Total degrees of freedom | 21 | |
| Single leg degree of freedom | 6 | |
| Single arm degree of freedom | 4 | |
| Waist degree of freedom | 1 | |
| Computing power module | / | NVIDIA Jetson Orin NX |
| Joint communication method | EtherCAT & CAN FD | |
| Maximum joint torque | 70 N·m | |
| Lower-Leg and Thigh Length | About 513mm | |
| Arm Span | About 379mm | |
| Power supply method | Lithium battery | |
| Emergency stop function | Software emergency stop & hardware emergency stop | |
| Basic computing power | 6 TOPS | 6 + 117 TOPS |
| Camera model | DC_V93 | Realsense D435I |
| Perception sensors | IMU, camera | |
| Microphone array | Standard configuration | |
| Speaker | Standard configuration | |
| Voice interaction model | ChatGPT | |
| Voice control | Support | |
| Intelligent battery | Quick-release battery | |
| Image transmission | Within the app | During development, hardware can be directly accessed on the computing board |
| Charger | Yes | |
| Mobile APP | Yes | |
| OTA update | Yes | |
| Battery life | 2~3H | |
| Secondary development (HighController) | Yes | |
| Secondary development (LowController) | / | Yes |
| Development language | python C++ Graphical Programming | |
| Component Introduction Diagram | ![]() ![]() | |
(II) Appearance Dimensions



(III) Material Composition

(IV) Electrical Interface
On the back of Boomy, the following electrical interfaces are provided:

| Serial number | Type | Explanation | Quantity |
|---|---|---|---|
| 1 | RJ45 | Switch 1000 BASE-T | 1 |
| 2 | USB | Motion Control Board/Computing Board USB 3.0 | 2 |
| 3 | HDMI | High definition display cable for Motion Control Board/Compute Board | 1 |
The back interface of Boomy with Computing Board is the interface of the Computing Board, while Boomy without Computing Board has the back interface of the Motion Control Board
(V) Airborne Computer
| Type | Model | User | Password | IP | Computing power | Frequency | Memory |
|---|---|---|---|---|---|---|---|
| Motion Control Board | Rockchip RK3576 | The Motion Control Board is not open to users | 192.168.55.102 | 6 TOPS | 2.4GHz | 8+64GB | |
| Secondary development Computing Board | NVIDIA Jetson Orin NX | noetix | Boomy | 192.168.55.101 | 117 TOPS | 2.0GHz | 16+128GB |
The Motion Control Board (Rockchip RK3576) is not open to users. If users access the Motion Control Board without authorization, causing the motion control program to lose control and resulting in losses, the user shall bear the losses themselves
(VI) Camera Information
Boomy-EDU: Realsense D435i
Intel® RealSense™ Depth Camera D435i - Product Specifications | Intel
Boomy Standard: DC_V93

(VII) Battery Information
| Parameter | Information |
|---|---|
| Brand | Grepow |
| Model | GRP6752097-13S1P |
| Capacity | 5100mAh |
| Voltage | 48V |
| Weight | 1360g |
(VIII) Charger Specifications and Parameters
| Parameter | Information |
|---|---|
| Product Model | TA300 Robot |
| Input | 100-240V~50/60Hz 4.5A |
| Battery Type | LiPo/LiFe/LiHV 13S-16S |
| Size | 172*97*46mm |
| Product Name | Smart charger |
| Output | 35-72V ⎓ 7.5A 350W Max |
| Work Mode | Charge |
| Weight | 510g±10g |
II. Instructions for Use
(I) Startup Equipment and Precautions
-
Equipment removal and placement
-
Remove Boomy from the storage position.
-
Hold it steady and make it sit on the ground.
-
Battery installation
-
Insert the battery into the Boomy back battery compartment.
-
Confirm that the battery is installed in place (snap lock).
-
Equipment startup
-
Press the battery switch briefly once to confirm that the battery indicator light is on.
-
Then press and hold the battery switch until the flowing light effect appears.
-
Wait for the robot to complete startup.
-
Initialize attitude adjustment
-
Place Boomy face-up flat on the ground, make sure the ground is flat and non-slip, there are no foreign objects under the Boomy soles or around it, so as to avoid the Boomy automatic stand-up being affected by surrounding objects and failing
-
System initialization instructions
-
After turning on, Boomy will automatically enter full-body enable mode:
- All joints will automatically move to their zero positions
- Do not interfere with or obstruct joint movement during the process
⚠️ Precautions
- Avoid touching joints during startup to prevent pinching or interference with initialization
- Ensure the ground is level to avoid abnormal posture
- Do not power on the robot unless the battery is installed correctly
-
When powering on, ensure that the emergency stop button is in the released (popped-up) position. During use, do not press the emergency stop button except in an emergency.
(II) Boomy Connection Method

After connecting to Boomy, do not select 'Forget Device' in the system settings, otherwise the Bluetooth record will be cleared, causing the device to be unable to connect again. If there is any misoperation, please contact customer support.
(III) Connecting to and Controlling Boomy with the Mobile App
NOETIX Run App Download:
Android:

iOS: NOETIX Run App - App Store
NOETIX Run App usage instructions: NOETIX Run App usage instructions
First use app connect Boomy: NOETIX Run App User Guide (Boomy)
(IV) Emergency Stop Function
This product supports two emergency-stop methods: the emergency-stop button on the Boomy device body and the emergency-stop control in the App:
-
Device body emergency stop button After pressing the button, the device will be immediately powered off and the whole machine will collapse to the ground. Do not press it accidentally during normal operation to prevent the device from falling and being damaged; in case of an accident, please press it immediately to avoid further losses.
-
App emergency stop (global floating button) The button is hidden by default in the sidebar; tap the sidebar to display it. After pressing, the robot will not be powered off, it will only go limp all over. Do not touch it by mistake during normal use to avoid the device falling and being damaged; in case of an accident, please press it immediately to avoid further losses.
App emergency stop process
Emergency stop button

(V) Device Shutdown and Battery Charging
- Equipment storage preparation
a. Perform the one-click storage operation in the App. b. After the storage is completed, hold Boomy steady and make it sit on the ground.
- Device shutdown
c. Press and hold the battery switch until Boomy is completely turned off.
- Battery disassembly
d. Press and hold the battery clip. e. Remove the battery** from the Boomy back battery compartment.
- Battery charging
f. Place the battery on the battery base and ensure it is securely connected. g. Connect the charging adapter to the battery base. h. Insert the adapter into a 220V power supply.
- Charging status confirmation
i. Observe the battery indicator light:
- Indicator light flashing: Charging in progress
- No indication: The battery is fully charged
(VI) Tutorial Videos
1. Boomy Unboxing Video
III. Boomy Development Guide
(I) Boomy Supports the Following Development Interfaces
- DDS: An SDK encapsulated from the DDS interface, supporting
pythonandC++- More lightweight, integrated and encapsulated into simple and easy-to-understand function interfaces
- ROS2: Topics encapsulated and integrated from ROS2, exposed for external access
- More suitable for research laboratory scenarios, offering hardcore, professional-grade ROS2 topic access
noetix_ros2_boomy: https://github.com/Noetix-Robotics/noetix_ros2_boomy
(II)Boomy URDF File
(III) Boomy-DDS-SDK
github link:
GitHub - Noetix-Robotics/noetix_sdk_boomy: This is SDK for Noetix robot Boomy with DDS interface.
SDK Introduction
This document introduces Boomy's communication development based on the DDS message middleware. This SDK adopts a C/S architecture. Under an isolation mechanism, it enables the DDS client program deployed on the secondary-development Compute Board of the Boomy robot and the motion control program on the Motion Control Board, which acts as the server, to issue control commands at a frequency of 500Hz and continuously obtain the current robot status. The DDS used is CYCLONEDDS 0.11.0.
1. System Architecture Diagram

Under the isolation mechanism between the Motion Control Board and the Compute Board, the DDS interface is exposed for the Compute Board to access and develop against.
2. Application Development: HighController
Introduction to Application Development
HighController performs upper-layer scheduling development based on the OEM's motion control strategy. Without changing the underlying control logic, it enables the rapid construction of robot automation applications and supports integration with Foundation Models to create customized intelligent behavior systems.
Typical applications include:
- Combining vision algorithms with the OEM motion control system to achieve autonomous perception and autonomous movement
- Integrating a voice interaction module to enable natural language control
- Integrating Foundation Models to dynamically generate and invoke OEM control actions based on input (this entails certain risks and requires strict safety policies)
- Working in conjunction with an externally mounted perception radar to implement a positioning, navigation, and inspection system
In short, through high-level scheduling and encapsulation of the OEM's control capabilities, HighController enables automated and programmatic invocation of behaviors controllable via a remote controller or App, and on this basis extends certain autonomous and intelligent capabilities.
(1) HighController Architecture Diagram

a. State Reporting Mechanism
Once the client SDK has established a connection with the robot's motion control program, the motion control program will continuously push real-time robot status information to the client, including:
- Full-body motor parameter data (pos: position, vel: velocity, tau: torque, kp, kd, temperature, error: error information)
- IMU status data
- Current working mode (
workmode, indicating the control strategy currently in operation) - AOLION controller data (used to synchronize manual input or auxiliary control)
- Battery level and related system status information
b. Client Control Command Issuance
Based on its own application scenarios, the client sends control commands to the motion control program, mainly including:
-
(x, y, z) movement control data
Valid only in specific modes, such as walking mode, handshake, greeting, cheering, teaching mode, etc.
-
action command
Corresponds to
workmodeand is used to switch or trigger the algorithm strategy currently running on the robot. -
data: additional DDS data
Usually used in
PLAYTEACH(playback teaching mode) to specify additional parameters such as the teaching file index.
c. Internal Processing Flow of the Motion Control Program
After the motion control program receives a command from the client:
- Pass the control parameters to the internal control algorithm
- The algorithm performs policy computation based on a reinforcement learning model
- Generate the motor target setpoints (target position, velocity, or torque) and send them to the motor control interface
- Send the computation results down to the hardware control layer
d. Hardware Execution Layer
After the hardware layer receives the target control data:
- Execute the motor control commands
- Complete the corresponding desired action
Throughout the entire process, users only need to issue some high-level control parameters.
The specific motion planning, dynamic balancing, and motor-level control computations are all handled by the black-box algorithms in the OEM motion control program.
Therefore, under HighController mode, the range of capabilities that users can invoke and automate does not, in principle, exceed the functional boundary originally achievable via the App or controller, but underlying motor-level control permissions are not exposed.
(2) HighController Operation Flowchart

a. Initialization Phase (init)
Upon system startup, the following initialization procedure is performed:
- Establish a communication connection between the client and the robot's motion control program
- Register a callback function for receiving robot status data
b. The DDS callback thread continuously receives the returned status
The motion control program continuously pushes robot status data to the client.
After receiving the data, the callback function encapsulates it and stores it separately in the buffer and in the global state variables:
motor_state_buffer_: motor state bufferimu_buffer_: IMU state bufferjoy_buffer_: controller data buffercurmode: current motion control strategy modeBmsState: current battery information
(3) HighController Common Interfaces and Parameters Table
| HighController Interface | ||
|---|---|---|
| Interface Name | Interface Description | Return Type |
| instance | Get the HighController singleton pointer | Object Pointer |
| init | HighController Object Initialization | None |
| publish_cmd | Publish DDS control command data | None |
| subscribe_robot_hardware_status | Subscribe to robot hardware status information | None |
Get the HighController singleton object
| Function Name | instance |
|---|---|
| Function Prototype | static HighController *Instance() |
| Function Overview | Get the HighController singleton |
| Parameters | None |
| Return Value | HighController singleton object pointer |
| Remarks |
C++ Example:
HighController *ctrl = HighController::Instance();
Python Example:
ctrl = HighController.instance()
HighController Object Initialization
| Function Name | init |
|---|---|
| Function Prototype | bool init() |
| Function Overview | HighController Object Initialization |
| Parameters | None |
| Return Value | bool |
| Remarks |
C++ Example:
ctrl->init();
Python Example:
ctrl.init()
Publish robot control commands
| Function Name | publish_cmd |
|---|---|
| Function Prototype | void publish_cmd(double x, double y, double z, ControlCmd action, uint16_t index) |
| Function Overview | Use HighController to send DDS control commands |
| Parameters |
|
| Return Value | None |
| Remarks | This interface must not be called continuously; otherwise, the robot may receive commands too quickly to respond. A delay of at least 2 ms must be added after each command is sent |
C++ Example:
ctrl->publish_cmd(0, 0, 0, ControlCmd.CHEER, 0);
Python Example:
ctrl.publish_cmd(0, 0, 0, ControlCmd.CHEER, 0)
The action parameter indicates switching the robot to the corresponding mode
| Parameter Name | Define Prototype | Serial No. | Example Value | Description |
|---|---|---|---|---|
ControlCmd | | 0 | WALK | Walking Mode (Half-Body Walking) |
| 1 | SWING | Factory-Preset Action: Wave | ||
| 2 | SHAKE | Factory-Preset Action: Handshake | ||
| 3 | CHEER | Factory-Preset Action: Cheer | ||
| 4 | FULLWALK | Full-Body Walking | ||
| 5 | START | Enable/Disable | ||
| 6 | SWITCH | Preparation Mode | ||
| 7 | STARTTEACH | Start Teaching | ||
| 8 | SAVETEACH | Save Teaching | ||
| 9 | ENDTEACH | End Teaching | ||
| 10 | PLAYTEACH | Play Demonstration | ||
| 11 | DANCE0 | Dance 1 Mode (what is love) | ||
| 12 | FALLTOSTAND | Stand Up | ||
| 13 | STANDTOFALL | Stow | ||
| 14 | DANCE1 | Dance 2 Mode (jueshi) | ||
| 15 | DANCE2 | Dance 3 Mode (gee) | ||
| 16 | ZHOUJI | Elbow Strike | ||
| 17 | DEFAULT | Null Mode |
Purpose of Using DEFAULT (Empty Mode)
The entire robot thread executes the process control function in a periodic loop. The control function is executed periodically in a loop. If teaching-related commands such as
action = ControlCmd::PLAYTEACH; are continuously issued within this loop,
the robot may repeatedly interrupt the teaching playback action before it has finished, causing the robot to appear "stuck and unresponsive."
Correct Approach
PLAYTEACH is a one-shot trigger command (event) and should be sent only once; in subsequent loop iterations, send DEFAULT (an empty command) to avoid repeated triggering.
Example Code (Corrected)
if (is_first) {
action = ControlCmd::PLAYTEACH; // Trigger only once
is_first = false;
} else {
action = ControlCmd::DEFAULT;
}
Effect
- Upon receiving
PLAYTEACH→ the teaching action is triggered and executed to completion - Subsequent
DEFAULT→ no longer interrupts the action - The robot behaves normally and predictably
Action-type commands should be "edge-triggered" and must not be continuously issued within the control loop.
START Enable/Disable
workmode == 30, send START to put the robot into enabled mode
workmode != 30, send START to immediately put the robot into disabled mode
For switching between modes, please follow the state machine below
Boomy State Machine

index is used to specify the teaching file index in teaching-related modes
ctrl.publish_cmd(0, 0, 0, ControlCmd.STARTTEACH, 0) # Start teaching
sleep(10) # Record the teaching action
ctrl.publish_cmd(0, 0, 0, ControlCmd.SAVETEACH, 1) # End and save the teaching action, specifying index 1
sleep(2) # Allow the motion control program to finish the save process to prevent it from being interrupted by the next command
ctrl.publish_cmd(0, 0, 0, ControlCmd.PLAYTEACH, 1) # Play teaching file 1
The parameters x and yaw take effect only when action is one of the following modes:
PLAYTEACH/WALK/DEFAULT
and are used to control robot movement; in other modes, x and yaw have no effect.
Subscribe to robot hardware status information
| Function Name | subscribe_robot_hardware_status |
|---|---|
| Function Prototype | void subscribe_robot_hardware_status(RobotHardwareStatusCallback callback); |
| Function Overview | Subscribe to robot hardware status information |
| Parameters | - callback: Callback function for processing robot status data |
| Return Value | None |
| Remarks | |
C++ Example:
ctrl->subscribe_robot_hardware_status(
[](const RobotHardwareStatus &status) {
latest_hw_status = status;
});
Python Example:
def _on_hw_status(status):
global latest_hw_status
latest_hw_status = status
ctrl.subscribe_robot_hardware_status(_on_hw_status)
| Parameter Name | Define Prototype | Parameters | Description |
|---|---|---|---|
| RobotHardwareStatus | | imu_data | IMU data |
| remote_data | Controller data | ||
| motor_data | Motor data | ||
| bms_data | Battery data | ||
| workmode | Current working mode |
workmode parameter mapping table
The workmode value is valid only in HighController mode
| Value | Mode Description |
|---|---|
| 0 | Enable Mode |
| 1 | Preparation Mode |
| 2 | Half-Body Walking Mode |
| 3 | Full-Body Walking Mode |
| 4 | Run Mode |
| 5 | Dance Mode 0 |
| 8 | Factory Preset Action: Greeting |
| 9 | Factory Default Action: Handshake |
| 10 | Factory Default Action: Cheer |
| 11 | Start Teaching |
| 12 | End Teaching |
| 13 | Play Demonstration |
| 14 | Save Teaching |
| 23 | Play Demonstration |
| 26 | Protection Mode |
| 27 | Get up from the ground |
| 28 | Get up and fall to the ground |
| 29 | Save Demonstrated Motion |
| 30 | Disabled Mode |
| 31 | Dance Mode 1 |
| 32 | Dance Mode 2 |
| 33 | Elbow Strike Motion |
Save Teaching
After the SDK sends SAVETEACH to the robot, the robot will successively return three statuses: Exit Teaching, Save Teaching 1, and Save Teaching 2
Protection Mode
When the robot is under standing control strategies (walking, teaching, dancing, etc.), the motion control program continuously monitors IMU attitude data.
If the attitude is detected to deviate from the stable standing range and exceed the preset threshold, the system will immediately trigger the protection mechanism and forcibly switch to safe mode.
In protection mode:
- All joints enter the disabled state (the motors stop active output)
- The current motion control strategy is terminated
- Further action commands are blocked from execution
The purpose of this mechanism is to:
- Prevent the robot from continuously outputting control torque during a fall or loss of stability
- Avoid secondary damage caused by uncontrolled swinging of the limbs
- Reduce potential damage to the device itself and the surrounding environment
This protection logic is an underlying safety protection mechanism with the highest priority and is not affected by upper-layer control commands.
Controller Data Structure
| Parameter Name | Define Prototype | Example Value | Description |
|---|---|---|---|
| joydata | | axes | Remote controller joystick linear value |
| button | Remote controller button press mapping values: 1 = pressed, 0 = normal |
| Controller Joystick Mapping | Mapping Description | Range |
|---|---|---|
| axes[0] | Right Joystick Horizontal Value | [-1, 1] |
| axes[1] | Left Joystick Vertical Value | [-1, 1] |
For a Boomy equipped with a Compute Board, the rear interface is the Compute Board interface. The Motion Control Board cannot accept a controller receiver, so DDS cannot obtain controller data; therefore, controller data must be obtained directly on the Compute Board. For details, please see the compute board directly obtains controller data
IMU State Data Structure
| Parameter Name | Define Prototype | Example Value | Description |
|---|---|---|---|
| NingImuData | | ori | Attitude Quaternion |
| ori_cov | Attitude Covariance | ||
| angular_vel | Angular Velocity | ||
| angular_vel_cov | Angular Velocity Covariance | ||
| linear_acc | Linear Acceleration | ||
| linear_acc_cov | Linear Acceleration Covariance |
Motor State Data Structure
| Parameter Name | Define Prototype | Example Value | Description |
|---|---|---|---|
| MotorState | | pos | Motor Position |
| vel | Motor Speed | ||
| tau | Motor Torque | ||
| motor_id | Motor ID | ||
| error | Motor Error Codes: 02H: Motor overcurrent 03H: Motor undervoltage 04H: Motor encoder error 06H: Motor brake overvoltage 07H: DRV driver error 08H: Overvoltage 09H: Undervoltage 0AH: Overcurrent 0BH: MOS overtemperature 0CH: Motor coil overtemperature 0DH: Communication loss 0EH: Overload | ||
| temperature | Temperature |
Battery State Data Structure
| Parameter Name | Define Prototype | Example Value | Description |
|---|---|---|---|
| RobotBmsData | | battery_temp_ | Temperature |
| battery_alarm_ | Alarm Information | ||
| battery_soc_ | Battery Level | ||
| battery_soh_ | Battery Health |
(4) Motion Control State Transition Flowchart

Note: During development with HighController, please follow the above flowchart for mode switching to ensure proper and safe operation
(5) Demo Program Introduction
The examples/cpp/example_high.cpp and examples/py/test_high.py files in the SDK folder provide a sample demo for parsing AOLION controller input data and generating corresponding control commands based on controller operations to send to the robot.
3. Motion Control Development: LowController
Motion Control Development Introduction
LowController exposes the motor control interfaces in the motion control program to the SDK, allowing users to directly adjust low-level motor parameters at a control frequency of 500 Hz. Combined with the deployment of reinforcement learning algorithms, this enables deep customization of Boomy's motion capabilities, allowing it to perform diverse new actions and complex behaviors such as dances.
Note: Only the EDU version of Boomy supports LowController development
(1) Boomy Training Framework
Coming soon!
(2) LowController Framework Diagram

a. Status Data Reporting
Once the client SDK has established a connection with the robot's motion control program, the motion control program will continuously push real-time robot status information to the client, including:
- Full-body motor parameter data (pos: position, vel: velocity, tau: torque, kp, kd, temperature, error: error information)
- IMU status data
- AOLION controller data
b. Client-side control computation
The client uses its own algorithms (typically a reinforcement learning model) to perform real-time policy computation, generate target setpoints for each motor (such as target position), and write the motor control command array into the buffer before sending it to the motion control program.
c. Motion Control Program Forwarding Mechanism
After receiving the motor command array sent by the client, the motion control program does not perform high-level policy computation; instead, it directly forwards the control parameters to the hardware control layer.
d. Hardware Execution Layer
After receiving the motor target parameters, the hardware layer executes the control commands and drives the motors to complete the desired actions.
In LowController mode:
- The reinforcement learning model is deployed on the client side
- Users can directly participate in motor-level control
- The motion control program mainly handles communication and safe forwarding
- The control frequency can reach
500Hz
This mode provides higher control freedom and flexibility, but it also means:
- Higher requirements for algorithm stability
- Stricter requirements for real-time performance and system safety
- If the control strategy is unstable, it may directly affect robot motion safety
(3) LowController Operation Flowchart

a. Initialization Phase (init)
Upon system startup, the following initialization steps are performed:
- Establish a communication connection between the client and the robot's motion control program
- Register a callback function for receiving robot status data
- Create the
send_threaddata sending thread
b. Status Data Callback Mechanism
The motion control program continuously pushes robot status data to the client.
After receiving the data, the callback function encapsulates it and stores it in the corresponding buffer or global variable:
motor_state_buffer_: motor state bufferimu_buffer_: IMU state bufferjoy_buffer_: controller data buffer
This phase is only responsible for data reception and caching, and does not participate in control policy computation.
c. send_thread (Data Sending Thread)
The data sending thread also runs at a 2 ms period:
- If
motor_cmd_buffer_is not empty - Read the latest motor control command
- Send the motor target array to the motion control program
The motion control program is only responsible for forwarding data to the hardware layer and does not participate in high-level control decisions.
d. Architecture Features
The core features of LowController mode are:
- The client directly participates in motor-level control
- Reinforcement learning or custom control algorithms run on the client
- The motion control program no longer handles high-level policy computation and only handles communication and safe forwarding
- The control frequency can reach 500Hz
This mode provides higher control freedom and algorithm plasticity, but at the same time places higher demands on:
- Real-time performance
- Algorithm stability
- Thread safety and scheduling accuracy
(4) Joint Motor Information
| ID | Name | Joint | Angle Range | Radian Range | ||
|---|---|---|---|---|---|---|
| 0 | arm_l1_joint | Left arm shoulder joint pitch | -135 | 135 | -2.36 | 2.36 |
| 1 | arm_l2_joint | Left arm shoulder joint roll | -8 | 111 | -0.14 | 1.94 |
| 2 | arm_l3_joint | Left arm joint yaw | -90 | 90 | -1.57 | 1.57 |
| 3 | arm_l4_joint | Left arm elbow joint pitch | -129 | 0 | -2.26 | 0.00 |
| 4 | leg_l1_joint | Left leg hip joint pitch | -120 | 120 | -2.09 | 2.09 |
| 5 | leg_l2_joint | Left leg hip joint roll | -38 | 90 | -0.66 | 1.57 |
| 6 | leg_l3_joint | Left thigh joint yaw | -145 | 145 | -2.53 | 2.53 |
| 7 | leg_l4_joint | Left leg knee joint pitch | 0 | 128 | 0.00 | 2.24 |
| 8 | leg_l5_joint | Left leg ankle joint pitch | -55 | 25 | -0.96 | 0.44 |
| 9 | leg_l6_joint | Left leg ankle joint roll | -10 | 10 | -0.17 | 0.17 |
| 10 | arm_r1_joint | Right arm shoulder joint pitch | -135 | 135 | -2.36 | 2.36 |
| 11 | arm_r2_joint | Right arm shoulder joint roll | -111 | 8 | -1.94 | 0.14 |
| 12 | arm_r3_joint | Right arm joint yaw | -90 | 90 | -1.57 | 1.57 |
| 13 | arm_r4_joint | Right arm elbow joint pitch | -129 | 0 | -2.26 | 0.00 |
| 14 | leg_r1_joint | Right leg hip joint pitch | -120 | 120 | -2.09 | 2.09 |
| 15 | leg_r2_joint | Right leg hip joint roll | -90 | 38 | -1.57 | 0.66 |
| 16 | leg_r3_joint | Right thigh joint yaw | -145 | 145 | -2.53 | 2.53 |
| 17 | leg_r4_joint | Right leg knee joint pitch | 0 | 128 | 0.00 | 2.24 |
| 18 | leg_r5_joint | Right leg ankle joint pitch | -55 | 25 | -0.96 | 0.44 |
| 19 | leg_r6_joint | Right leg ankle joint roll | -10 | 10 | -0.17 | 0.17 |
| 20 | waist_1_joint | Waist joint yaw | -90 | 90 | -1.57 | 1.57 |
MotorCmd.pos corresponds to radian information
(5) AOLION Controller Data Mapping
AOLION controller. The button[] and button mapping table is as follows:
| Button Description | Button index |
|---|---|
| Right shoulder button | 1 |
| Left shoulder button | 2 |
| X | 5 |
| Y | 6 |
| B | 7 |
| A | 8 |
| + | 9 |
| - | 10 |
| None | 11 |
| Left joystick press | 12 |
| Controller Joystick Mapping | Mapping Description |
|---|---|
| axes[0] | Right joystick Y-axis value |
| axes[1] | Left joystick X-axis value |
Controller Joystick Data Range
| Minimum | Maximum |
|---|---|
| -1 | 1 |
(6) Commonly Used LowController Interfaces and Usage Tutorial
| LowController Interface | ||
|---|---|---|
| Interface Name | Interface Description | Return Type |
| instance | Get the LowController singleton pointer | Object pointer |
| init | Initialize the LowController object | None |
| set_joint | Write motor control commands into the buffer | None |
| getJointsIndex | Get the corresponding ID by joint name | int |
| subscribe_robot_hardware_status | Subscribe to robot hardware status information | None |
Get the LowController singleton object
| Function Name | instance |
|---|---|
| Function Prototype | static LowController *Instance() |
| Function Overview | Get the LowController singleton |
| Parameters | None |
| Return Value | LowController singleton object pointer |
| Remarks |
C++ Example:
LowController *ctrl = LowController::Instance();
Python Example:
ctrl = LowController.instance()
LowController Object Initialization
| Function Name | init |
|---|---|
| Function Prototype | bool init() |
| Function Overview | Initialize the LowController object |
| Parameters | None |
| Return Value | bool |
| Remarks |
C++ Example:
ctrl->init();
Python Example:
ctrl.init()
Store motor control command array into buffer
After being stored in the buffer, it will be automatically sent by the send_thread thread.
| Function Name | set_joint |
|---|---|
| Function Prototype | void set_joint(std::array<MotorCmd, 21> motorcmd) |
| Function Overview | Set joint data |
| Parameters | motorcmd: motor command structure |
| Return Value | None |
| Remarks |
| Parameter Name | Definition Prototype | Example Value | Description |
|---|---|---|---|
| MotorCmd | | pos | Motor position |
| vel | Motor speed | ||
| tau | Motor torque | ||
| kp | PID parameter | ||
| kd | PID parameter | ||
| motor_id | Motor ID |
When issuing motor control commands, strictly refer to the specified parameter ranges in.
C++ Example:
std::array<MotorCmd, 21> motorcmd;
// Algorithm logic omitted ...
ctrl->set_joint(motorcmd);
Python Example:
cmds = [MotorCmd() for _ in range(21)]
# Algorithm logic omitted ...
ctrl.set_joint(cmds)
Get the corresponding joint motor ID by joint motor name
| Function Name | getJointsIndex |
|---|---|
| Function Prototype | int getJointsIndex(std::string jointname) |
| Function Overview | Get the corresponding joint motor ID by joint motor name |
| Parameters | jointname: joint motor name |
| Return Value | int: corresponding joint motor ID |
| Remarks |
C++ Example:
int id = ctrl->getJointsIndex("arm_l1_joint");
Python Example:
id = ctrl.getJointsIndex("arm_l1_joint")
Subscribe to robot hardware status information
| Function Name | subscribe_robot_hardware_status |
|---|---|
| Function Prototype | void subscribe_robot_hardware_status(RobotHardwareStatusCallback callback); |
| Function Overview | Subscribe to robot hardware status information |
| Parameters | - callback: callback function for processing robot status data |
| Return Value | None |
| Remarks | Usage is the same as HighController subscribe_robot_hardware_status |
Obtain AOLION controller data (obtained directly on the compute board for Boomy with a compute board)
AoLionDriver class
| AoLionDriver Class Interface | |
|---|---|
| init | Initialize |
| getremotedata | Obtain controller data |
| Function Name | init |
|---|---|
| Function Prototype | bool init(std::string port, int baudrate) |
| Function Overview | Initialize the controller device port |
| Parameters |
|
| Return Value | true: initialization succeeded false: initialization failed |
| Remarks |
| Function Name | getremotedata |
|---|---|
| Function Prototype | const joydata getremotedata() |
| Function Overview | Obtain controller data |
| Parameters | None |
| Return Value | joydata: controller data structure |
| Remarks |
Usage Example
C++
AoLionDriver aoliondriver;
joydata remote_data;
aoliondriver.init("/dev/input/js0", 115200);
const joydata jdata=aoliondriver.getremotedata();
for (int i = 0; i < 14; i++)
std::cout << jdata.button[i] << std::endl;
std::cout << jdata.axes[0] << std::endl;
std::cout << jdata.axes[1] << std::endl;
Python
al = AoLionDriver()
al.init("/dev/input/js0", 115200)
jdata = al.getremotedata()
for i in range(14):
print(jdata.button[i])
print(jdata.axes[0])
print(jdata.axes[1])
4. Audio and Video Development
Introduction to Audio Development
MediaController establishes a communication link with the voice module on the Motion Control Board via DDS topics, enabling control of Boomy audio peripherals (such as speakers and microphones), and providing capabilities for voice interaction agent configuration and data stream management.
Note
Only the EDU version of Boomy supports MediaController development
The open-source interfaces are as follows:
Status control and acquisition interface: supports wakeup control, sleep control, module restart, operating status acquisition, and error information acquisition;
General configuration interface: an interface for configuring and retrieving general parameters, including: non-dialogue sleep timeout duration, wake-up word, wake-up reply word, sleep reply word, as well as audio-visual data stream control;
Audio interface: audio stream push and retrieval, audio control commands (pause/resume, volume adjustment, etc.)
Video interface: video stream push and retrieval, video control commands (pause/resume, etc.)
(1) MediaController Architecture Diagram


a. Media service status and data reporting
After the client SDK establishes a connection with the robot media service program, the media service program will continuously push real-time robot media status information to the client, including:
- Continuous acquisition of internal audio and video data streams (only Boomy models without a Compute Board can acquire internal video streams)
- Partial agent configuration reporting (actively obtained for the first time, and reported after subsequent changes)
- Acquisition of desensitized processed video stream (the version equipped with a Compute Board needs to continuously publish external video streams to the audio-visual interaction system)
b. Delivery of media data streams and configuration data
- Publish external audio stream data to the agent as input, or send it to the internal speaker for output
- Publish external video stream data for agent recognition
- Modify some agent configuration settings, including: volume, timeout, wake word, wake-up response, sleep response, and the data stream routing switch, etc.
The input and output data for audio stream capture currently only support: channels=8, sample_rate=16000, format=2 (16-bit little-endian)

The input and output data for audio stream playback currently only support: channels=2, sample_rate=16000, format=2 (16-bit little-endian)

The captured data and playback data are interleaved
(2) MediaController Startup Flowchart

(3) Common Parameters of MediaController
DDS Packet Header Structure
| Structure Name | Define Prototype | Member | Description |
|---|---|---|---|
| Header | | message_id | DDS Message ID |
| timestamp_us | us timestamp | ||
| sn | The current SN number of the Boomy device |
System Status Control
| SystemControlType | |
|---|---|
| Enumeration | Description |
| TO_WAKEUP | Wake up the audio-visual interaction system agent |
| TO_SLEEP | Put the audio-visual interaction system agent to sleep |
| TO_RESET | Restart the audio-visual interaction system agent |
System Status
| Parameter Name | Define Prototype | Member | Description |
|---|---|---|---|
| SystemStatus | | header | DDS Header |
| value | agent status | ||
| reason | Status Reason |
| WorkStatus | |
|---|---|
| Enumeration | Description |
| READY | Ready State |
| SLEEPED | Sleep State |
| WAKEUPED | Awake State |
| EXIT | Exit Status |
| StatusChangeReason | |
|---|---|
| Enumeration | Description |
| SYSTEM_LAUNCH | System Startup |
| CMD_RESET | Command Reboot |
| AUDIO_WAKEUPED | Voice Wake-up |
| CMD_WAKEUPED | Command Wake-up |
| AUDIO_SLEEPED | Voice Sleep |
| CMD_SLEEPED | Command Sleep |
| TIMEOUT_SLEEPED | Timeout Hibernation |
| ERROR_SLEEPED | Error Sleep |
Voice wake-up refers to triggering through actual human-computer voice interaction; command wake-up refers to triggering through DDS or other forms of commands;
System Error
| Parameter Name | Define Prototype | Member | Description |
|---|---|---|---|
| SystemError | | header | DDS Header |
| code | Error Code | ||
| message | Error Message |
Audio stream data
| Parameter Name | Define Prototype | Example Value | Description |
|---|---|---|---|
| AudioStream | | header | DDS Header |
| timestamp_us | Timestamp (us) | ||
| channels | Number of channels | ||
| sample_rate | Sampling rate | ||
| format | Data format | ||
| duration_ms | Time (ms) | ||
| audio_data | Audio data |
Video stream data
| Parameter Name | Define Prototype | Example Value | Description |
|---|---|---|---|
| VideoStream | | header | DDS Header |
| timestamp_us | Timestamp (us) | ||
| format | Format: 1: YUYV Currently, only the YUYV format is supported | ||
| width | Horizontal pixels per frame | ||
| height | Vertical pixels per frame | ||
| fps | Frame rate | ||
| video_data | Frame image data |
(4) Common Interfaces of MediaController
| MediaController Interface | ||
|---|---|---|
| Instance | Obtain the singleton pointer of MediaController | Object pointer |
| init | MediaController Object Initialization | bool |
| set_system_control | Set the control status of the voice interaction system | None |
| subscribe_system_status | Subscribe to status information of the audio-visual interaction system | None |
| subscribe_system_error | Subscribe to error status information of the audio-visual interaction system | None |
| get_volume | Get the current volume of the audio-visual interaction system Range: 0~200 | int |
| set_volume | Set the volume of the current audio-visual interaction system Range: 0~200 | None |
| get_timeout | Get the inactivity timeout before entering sleep mode | int |
| set_timeout | Set the inactivity timeout before entering sleep mode | None |
| get_audio_cue_enable | Get the enabled status of the prompt tone of the audio-visual interaction system | bool |
| set_audio_cue_enable | Set whether the prompt sound of the audio-visual interaction system is enabled | None |
| get_internal_capture_audio_data_to_agent_enable | Get the enabled status of internal microphone audio stream input to the agent | bool |
| set_internal_capture_audio_data_to_agent_enable | Set the enabled status of internal microphone audio stream input to the agent | None |
| get_external_custom_audio_data_to_agent_enable | Get the enabled status of external audio stream input for the agent | bool |
| set_external_custom_audio_data_to_agent_enable | Set the enabled status of external audio stream input for the agent | None |
| get_internal_agent_audio_data_to_playback_enable | Get the enabled status of agent audio stream output to the internal speaker | bool |
| set_internal_agent_audio_data_to_playback_enable | Set the enabled status of agent audio stream output to the internal speaker | None |
| get_external_custom_audio_data_to_playback_enable | Get the enabled status of external audio stream output to the internal speaker | bool |
| set_external_custom_audio_data_to_playback_enable | Set the enabled status of external audio stream output to the internal speaker | None |
| get_internal_capture_video_data_to_agent_enable | Get the enabled status of internal video stream input to the agent | bool |
| set_internal_capture_video_data_to_agent_enable | Set the enabled status of internal video stream input to the agent | None |
| get_external_custom_video_data_to_agent_enable | Get the enabled status of external video stream input to the agent | bool |
| set_external_custom_video_data_to_agent_enable | Set the enabled status of external video stream input to the agent | None |
| get_external_custom_audio_data_to_agent_use_internal_3a | Get the enabled status of using the internal 3A algorithm for external audio stream input to the agent | bool |
| set_external_custom_audio_data_to_agent_use_internal_3a | Set the enabled status of using the internal 3A algorithm for external audio stream input to the agent | None |
| get_wakeup_response_words | Get wake-up response words | string |
| set_wakeup_response_words | Set wake-up response words | None |
| get_sleep_response_words | Get dormant response words | string |
| set_sleep_response_words | Set sleep response words | None |
| get_wakeup_words | Get all wake words | string |
| add_wakeup_words | Add wake-up words | bool |
| reset_wakeup_words | Reset factory default wake-up words | None |
| subscribe_internal_audio_capture | Subscribe to the audio stream captured by the internal microphone | None |
| subscribe_internal_audio_playback | Subscribe to the audio stream played by the robot's internal speaker | None |
| subscribe_internal_video_capture | Subscribe to the video stream collected by the camera inside the robot (only available for Boomy without Compute Board) | VideoStream struct |
| subscribe_internal_video_desensed | Subscribe to the desensitized video stream processed by the audio-visual interaction system (the Boomy equipped with a Compute Board needs to continuously publish the external video stream) | VideoStream struct |
| pause_video_capture | Pause the internal camera data acquisition of the robot (only available for Boomy without Compute Board) | None |
| resume_video_capture | Resume the internal camera data acquisition of the robot (only available for Boomy without Compute Board) | None |
| publish_external_video_stream | Publish external video streams to the audio-visual interaction system | None |
| publish_external_audio_stream | Publish external audio stream input to the agent | None |
| publish_external_audio_playback_stream | Publish external audio stream output to the internal speaker | None |
| pause_audio_capture | Pause internal microphone audio stream capture | None |
| resume_audio_capture | Resume internal microphone audio stream capture | None |
| pause_audio_playback | Pause the audio stream output of the internal speaker | None |
| resume_audio_playback | Resume the audio stream output of the internal speaker | None |
The call interval of all set agent configuration interfaces (such as changing the volume, changing data switches, etc.) must not be less than 500 ms; otherwise, it may cause the voice interaction system to crash. If this problem occurs, please contact Songyan after-sales support
Get the singleton of the MediaController object
| Function Name | instance |
|---|---|
| Function Prototype | static MediaController *Instance() |
| Function Overview | Get the MediaController singleton |
| Parameters | None |
| Return Value | Singleton pointer to the MediaController object |
| Remarks |
C++ Example:
MediaController *ctrl = MediaController::Instance();
Python Example:
ctrl = MediaController.instance()
MediaController Object Initialization
| Function Name | init |
|---|---|
| Function Prototype | bool init() |
| Function Overview | MediaController Object Initialization |
| Parameters | None |
| Return Value | bool |
| Remarks |
C++ Example:
ctrl->init();
Python Example:
ctrl.init()
Set the status of the audio-visual interaction system
| Function Name | set_system_control |
|---|---|
| Function Prototype | void set_system_control(media::SystemControlType type, bool is_need_reply); |
| Function Overview | Set the status of the audio-visual interaction system (wake-up, sleep, restart) |
| Parameters | - type: system status - is_need_reply: whether to trigger the wake-up/sleep reply words |
| Return Value | None |
| Remarks |
C++ Example:
ctrl->set_system_control(media::SystemControlType::TO_WAKEUP, true);
Python Example:
ctrl.set_system_control(media::SystemControlType::TO_WAKEUP, true)
Subscribe to the status information of the audio-visual interaction system
| Function Name | subscribe_system_status |
|---|---|
| Function Prototype | void subscribe_system_status(SystemstatusCallback callback); |
| Function Overview | Subscribe to status information of the audio-visual interaction system |
| Parameters | - callback: the callback function that needs to be triggered when the state changes |
| Return Value | None |
| Remarks |
C++ Example:
ctrl->subscribe_system_status([&](const media::SystemStatus &status){
printf("status is %d \r\n" , static_cast<int>(status.value));
});
Python Example:
def on_status_switch(status):
print(status.value)
ctrl.subscribe_system_status(on_status_switch)
Subscribe to system error messages of the audio-visual interaction system
| Function Name | subscribe_system_error |
|---|---|
| Function Prototype | void subscribe_system_error(SystemerrorCallback callback); |
| Function Overview | Subscribe to system error messages of the audio-visual interaction system |
| Parameters | - callback: the callback triggered by a DDS message containing error information |
| Return Value | None |
| Remarks |
C++ Example:
ctrl->subscribe_system_error([&](const media::SystemError &error){
printf("status is %s \r\n", error.message.c_str());
});
Python Example:
def on_error(error):
print(error.message)
ctrl.subscribe_system_error(on_error)
Get the current volume
| Function Name | get_volume |
|---|---|
| Function Prototype | int get_volume() |
| Function Overview | Get the current volume of the audio-visual interaction system |
| Parameters | None |
| Return Value | int, volume value, range: 0~200 |
| Remarks |
C++ Example:
int volume = ctrl->get_volume();
std::cout << volume << std::endl;
Python Example:
volume = ctrl.get_volume()
print(volume)
Set the current volume
| Function Name | set_volume |
|---|---|
| Function Prototype | void set_volume(int volume) |
| Function Overview | Set the volume of the current audio-visual interaction system |
| Parameters | volume: volume value, range: 0~200 |
| Return Value | None |
| Remarks |
C++ Example:
ctrl->set_volume(100);
Python Example:
ctrl.set_volume(100)
Get the inactivity timeout before entering sleep
| Function Name | get_timeout |
|---|---|
| Function Prototype | int get_timeout() |
| Function Overview | Get the inactivity timeout before entering sleep |
| Parameters | None |
| Return Value | int, timeout (ms) |
| Remarks |
C++ Example:
int timeout = ctrl->get_timeout();
std::cout << timeout << std::endl;
Python Example:
timeout = ctrl.get_timeout()
print(timeout)
Set the inactivity timeout before entering sleep
| Function Name | set_timeout |
|---|---|
| Function Prototype | void set_timeout(int timeout) |
| Function Overview | Set the inactivity timeout before entering sleep |
| Parameters | timeout: timeout duration (ms) |
| Return Value | None |
| Remarks |
C++ Example:
ctrl->set_timeout(3000);
Python Example:
ctrl.set_timeout(3000)
Check whether the prompt tone is enabled
| Function Name | get_audio_cue_enable |
|---|---|
| Function Prototype | bool get_audio_cue_enable() |
| Function Overview | Get the enabled status of the prompt tone of the audio-visual interaction system |
| Parameters | None |
| Return Value | bool |
| Remarks |
C++ Example:
bool enabled = ctrl->get_audio_cue_enable();
std::cout << (enabled ? "true" : "false") << std::endl;
Python Example:
enabled = ctrl.get_audio_cue_enable()
print(enabled)
Set whether the prompt tone is enabled
| Function Name | set_audio_cue_enable |
|---|---|
| Function Prototype | void set_audio_cue_enable(bool enable) |
| Function Overview | Set whether the prompt sound of the audio-visual interaction system is enabled |
| Parameters | enable: true on, false off |
| Return Value | None |
| Remarks |
C++ Example:
ctrl->set_audio_cue_enable(true);
Python Example:
ctrl.set_audio_cue_enable(True)
Get the enabled status of internal microphone audio stream input to the agent
| Function Name | get_internal_capture_audio_data_to_agent_enable |
|---|---|
| Function Prototype | bool get_internal_capture_audio_data_to_agent_enable() |
| Function Overview | Get the enabled status of internal microphone audio stream input to the agent |
| Parameters | None |
| Return Value | bool |
| Remarks |
C++ Example:
bool enabled = ctrl->get_internal_capture_audio_data_to_agent_enable();
std::cout << (enabled ? "true" : "false") << std::endl;
Python Example:
enabled = ctrl.get_internal_capture_audio_data_to_agent_enable()
print(enabled)
Set the enabled status of internal microphone audio stream input to the agent
| Function Name | set_internal_capture_audio_data_to_agent_enable |
|---|---|
| Function Prototype | void set_internal_capture_audio_data_to_agent_enable(bool enable) |
| Function Overview | Set whether the internal microphone is enabled to collect audio stream and input it to the agent |
| Parameters | enable: true on, false off |
| Return Value | None |
| Remarks |
C++ Example:
ctrl->set_internal_capture_audio_data_to_agent_enable(true);
Python Example:
ctrl.set_internal_capture_audio_data_to_agent_enable(True)
Get the enabled status of external audio stream input to the agent
| Function Name | get_external_custom_audio_data_to_agent_enable |
|---|---|
| Function Prototype | bool get_external_custom_audio_data_to_agent_enable() |
| Function Overview | Get the enabled status of external audio stream input for the agent |
| Parameters | None |
| Return Value | bool: true indicates enabled, false indicates disabled |
| Remarks |
C++ Example:
bool enabled = ctrl->get_external_custom_audio_data_to_agent_enable();
std::cout << (enabled ? "true" : "false") << std::endl;
Python Example:
enabled = ctrl.get_external_custom_audio_data_to_agent_enable()
print(enabled)
Set the enabled status of external audio stream input to the agent
| Function Name | set_external_custom_audio_data_to_agent_enable |
|---|---|
| Function Prototype | void set_external_custom_audio_data_to_agent_enable(bool enable) |
| Function Overview | Set whether the external audio stream input to the agent is enabled |
| Parameters | enable: true on, false off |
| Return Value | None |
| Remarks |
C++ Example:
ctrl->set_external_custom_audio_data_to_agent_enable(true);
Python Example:
ctrl.set_external_custom_audio_data_to_agent_enable(True)
Get the enabled status of agent audio stream output to the internal speaker
| Function Name | get_internal_agent_audio_data_to_playback_enable |
|---|---|
| Function Prototype | bool get_internal_agent_audio_data_to_playback_enable() |
| Function Overview | Get whether the agent audio stream output to the internal speaker is enabled |
| Parameters | None |
| Return Value | bool: true indicates enabled, false indicates disabled |
| Remarks |
C++ Example:
bool enabled = ctrl->get_internal_agent_audio_data_to_playback_enable();
std::cout << (enabled ? "true" : "false") << std::endl;
Python Example:
enabled = ctrl.get_internal_agent_audio_data_to_playback_enable()
print(enabled)
Set the enabled status of agent audio stream output to the internal speaker
| Function Name | set_internal_agent_audio_data_to_playback_enable |
|---|---|
| Function Prototype | void set_internal_agent_audio_data_to_playback_enable(bool enable) |
| Function Overview | Set whether the agent audio stream output to the internal speaker is enabled |
| Parameters | enable: true on, false off |
| Return Value | None |
| Remarks |
C++ Example:
ctrl->set_internal_agent_audio_data_to_playback_enable(true);
Python Example:
ctrl.set_internal_agent_audio_data_to_playback_enable(True)
Get the enabled status of external audio stream output to the internal speaker
| Function Name | get_external_custom_audio_data_to_playback_enable |
|---|---|
| Function Prototype | bool get_external_custom_audio_data_to_playback_enable() |
| Function Overview | Get the enabled status of outputting external audio stream to the internal speaker |
| Parameters | None |
| Return Value | bool: true to enable, false to disable |
| Remarks |
C++ Example:
bool enabled = ctrl->get_external_custom_audio_data_to_playback_enable();
std::cout << (enabled ? "true" : "false") << std::endl;
Python Example:
enabled = ctrl.get_external_custom_audio_data_to_playback_enable()
print(enabled)
Set the enabled status of external audio stream output to the internal speaker
| Function Name | set_external_custom_audio_data_to_playback_enable |
|---|---|
| Function Prototype | void set_external_custom_audio_data_to_playback_enable(bool enable) |
| Function Overview | Set whether the external audio stream is output to the internal speaker |
| Parameters | enable: true on, false off |
| Return Value | None |
| Remarks |
C++ Example:
ctrl->set_external_custom_audio_data_to_playback_enable(true);
Python Example:
ctrl.set_external_custom_audio_data_to_playback_enable(True)
Get the enabled status of internal video stream input to the agent
| Function Name | get_internal_capture_video_data_to_agent_enable |
|---|---|
| Function Prototype | bool get_internal_capture_video_data_to_agent_enable() |
| Function Overview | Get the enabled status of internal video stream input to the agent |
| Parameters | None |
| Return Value | bool: true indicates enabled, false indicates disabled |
| Remarks |
C++ Example:
bool enabled = ctrl->get_internal_capture_video_data_to_agent_enable();
std::cout << (enabled ? "true" : "false") << std::endl;
Python Example:
enabled = ctrl.get_internal_capture_video_data_to_agent_enable()
print(enabled)
Set the enabled status of internal video stream input to the agent
| Function Name | set_internal_capture_video_data_to_agent_enable |
|---|---|
| Function Prototype | void set_internal_capture_video_data_to_agent_enable(bool enable) |
| Function Overview | Set whether to enable the internal video stream input for the agent |
| Parameters | enable: true on, false off |
| Return Value | None |
| Remarks |
C++ Example:
ctrl->set_internal_capture_video_data_to_agent_enable(true);
Python Example:
ctrl.set_internal_capture_video_data_to_agent_enable(True)
Get the enabled status of external video stream input to the agent
| Function Name | get_external_custom_video_data_to_agent_enable |
|---|---|
| Function Prototype | bool get_external_custom_video_data_to_agent_enable() |
| Function Overview | Get the enabled status of external video stream input to the agent |
| Parameters | None |
| Return Value | bool: true indicates enabled, false indicates disabled |
| Remarks |
C++ Example:
bool enabled = ctrl->get_external_custom_video_data_to_agent_enable();
std::cout << (enabled ? "true" : "false") << std::endl;
Python Example:
enabled = ctrl.get_external_custom_video_data_to_agent_enable()
print(enabled)
Set the enabled status of external video stream input to the agent
| Function Name | set_external_custom_video_data_to_agent_enable |
|---|---|
| Function Prototype | void set_external_custom_video_data_to_agent_enable(bool enable) |
| Function Overview | Set whether the external video stream input to the agent is enabled |
| Parameters | enable: true on, false off |
| Return Value | None |
| Remarks |
C++ Example:
ctrl->set_external_custom_video_data_to_agent_enable(true);
Python Example:
ctrl.set_external_custom_video_data_to_agent_enable(True)
Get the enabled status of using the internal 3A algorithm for external audio stream input to the agent
| Function Name | get_external_custom_audio_data_to_agent_use_internal_3a |
|---|---|
| Function Prototype | bool get_external_custom_audio_data_to_agent_use_internal_3a() |
| Function Overview | Get the status of whether the agent enables the internal 3A algorithm when receiving external audio stream input |
| Parameters | None |
| Return Value | bool: true indicates enabled, false indicates disabled |
| Remarks |
C++ Example:
bool enabled = ctrl->get_external_custom_audio_data_to_agent_use_internal_3a();
std::cout << (enabled ? "true" : "false") << std::endl;
Python Example:
enabled = ctrl.get_external_custom_audio_data_to_agent_use_internal_3a()
print(enabled)
Set the enabled status of using the internal 3A algorithm for external audio stream input to the agent
| Function Name | set_external_custom_audio_data_to_agent_use_internal_3a |
|---|---|
| Function Prototype | void set_external_custom_audio_data_to_agent_use_internal_3a(bool enable) |
| Function Overview | Set whether the external audio stream input to the agent uses the internal 3A algorithm |
| Parameters | enable: true on, false off |
| Return Value | None |
| Remarks |
C++ Example:
ctrl->set_external_custom_audio_data_to_agent_use_internal_3a(true);
Python Example:
ctrl.set_external_custom_audio_data_to_agent_use_internal_3a(True)
Get wake-up response words
| Function Name | get_wakeup_response_words |
|---|---|
| Function Prototype | string get_wakeup_response_words() |
| Function Overview | Get wake-up response words |
| Parameters | None |
| Return Value | string, the current wake-up reply word |
| Remarks |
C++ Example:
std::string words = ctrl->get_wakeup_response_words();
std::cout << words << std::endl;
Python Example:
words = ctrl.get_wakeup_response_words()
print(words)
Set wake-up response words
| Function Name | set_wakeup_response_words |
|---|---|
| Function Prototype | void set_wakeup_response_words(string words) |
| Function Overview | Set wake-up response words |
| Parameters | words: content of the wake-up reply word |
| Return Value | None |
| Remarks |
C++ Example:
ctrl->set_wakeup_response_words("I'am");
Python Example:
ctrl.set_wakeup_response_words("I'am")
Get dormant response words
| Function Name | get_sleep_response_words |
|---|---|
| Function Prototype | string get_sleep_response_words() |
| Function Overview | Get dormant response words |
| Parameters | None |
| Return Value | string, the current dormant response word |
| Remarks |
C++ Example:
std::string words = ctrl->get_sleep_response_words();
std::cout << words << std::endl;
Python Example:
words = ctrl.get_sleep_response_words()
print(words)
Set sleep response words
| Function Name | set_sleep_response_words |
|---|---|
| Function Prototype | void set_sleep_response_words(string words) |
| Function Overview | Set up auto-reply messages for idle status |
| Parameters | words: content of the auto-reply for inactivity |
| Return Value | None |
| Remarks |
C++ Example:
ctrl->set_sleep_response_words("ok");
Python Example:
ctrl.set_sleep_response_words("ok")
Get all wake-up words
| Function Name | get_wakeup_words |
|---|---|
| Function Prototype | string get_wakeup_words() |
| Function Overview | Get all wake-up words |
| Parameters | None |
| Return Value | String, all current wake words |
| Remarks |
C++ Example:
std::string words = ctrl->get_wakeup_words();
std::cout << words << std::endl;
Python Example:
words = ctrl.get_wakeup_words()
print(words)
Add wake-up words
| Function Name | add_wakeup_words |
|---|---|
| Function Prototype | bool add_wakeup_words(const std::string &words); |
| Function Overview | Add a new wake word |
| Parameters | - words: new wake word string |
| Return Value | bool: whether the addition via internal Regular Expression succeeded |
| Remarks | Regular Expression: (^(((([a-z]+) )?([^\s]+) ))+@(.+)$) Example: B UW1 M IY1 @boomy |
C++ Example:
if (ctrl->add_wakeup_words("B UW1 M IY1 @boomy"))
std::cout << "add succeed" << std::endl;
else
std::cout << "add fail" << std::endl;
Python Example:
if ctrl.add_wakeup_words("B UW1 M IY1 @boomy"):
print("add succeed")
else:
print("add fail")
Reset wake-up words
| Function Name | reset_wakeup_words |
|---|---|
| Function Prototype | void reset_wakeup_words(); |
| Function Overview | Reset default factory wake-up words |
| Parameters | None |
| Return Value | None |
| Remarks |
C++ Example:
ctrl->reset_wakeup_words();
Python Example:
ctrl.reset_wakeup_words()
Subscribe to the audio stream collected by the robot's internal microphone
| Function Name | subscribe_internal_audio_capture |
|---|---|
| Function Prototype | void subscribe_internal_audio_capture(AudiostreamCallback callback); |
| Function Overview | Subscribe to the audio stream collected by the robot's internal microphone |
| Parameters | - callback: callback function for processing audio stream data |
| Return Value | None |
| Remarks | The collected data is 8-channel data: the first four channels correspond to the four microphones mounted on the top of Boomy, the fifth and sixth channels are reserved channels, and the seventh and eighth channels are for loudspeaker echo acquisition |
C++ Example:
ctrl->subscribe_internal_audio_capture(
[&](const media::AudioStream &stream) {
std::lock_guard<std::mutex> lk(mtx);
if (channels == 0) {
channels = stream.channels;
sample_rate = stream.sample_rate;
}
pcm.insert(pcm.end(), stream.audio_data.begin(),
stream.audio_data.end());
});
Python Example:
def on_audio(stream):
with lock:
if channels_ref[0] == 0:
channels_ref[0] = stream.channels
sample_rate_ref[0] = stream.sample_rate
pcm_all.extend(stream.audio_data)
ctrl.subscribe_internal_audio_capture(on_audio)
Subscribe to the audio stream played by the robot's internal speaker
| Function Name | subscribe_internal_audio_playback |
|---|---|
| Function Prototype | void subscribe_internal_audio_playback(AudiostreamCallback callback); |
| Function Overview | Subscribe to the audio stream played by the robot's internal speaker |
| Parameters | - callback: callback function for processing audio stream data |
| Return Value | None |
| Remarks | The collected data is 2-channel data from the Boomy ontology speaker |
C++ Example:
ctrl->subscribe_internal_audio_playback(
[&](const media::AudioStream &stream) {
std::lock_guard<std::mutex> lk(mtx);
if (channels == 0) {
channels = stream.channels;
sample_rate = stream.sample_rate;
}
pcm.insert(pcm.end(), stream.audio_data.begin(),
stream.audio_data.end());
});
Python Example:
def on_audio(stream):
with lock:
if channels_ref[0] == 0:
channels_ref[0] = stream.channels
sample_rate_ref[0] = stream.sample_rate
pcm_all.extend(stream.audio_data)
ctrl.subscribe_internal_audio_playback(on_audio)
Subscribe to the video stream collected by the camera inside the robot (this interface is only available for Boomy without a Compute Board)
| Function Name | subscribe_internal_video_capture |
|---|---|
| Function Prototype | void subscribe_internal_video_capture(VideostreamCallback callback); |
| Function Overview | Subscribe to the video stream collected by the camera in the robot |
| Parameters | - callback: callback function for processing video stream data |
| Return Value | None |
| Remarks | Only available for Boomy models without a Compute Board |
C++ Example:
ctrl->subscribe_internal_video_capture(
[&](const media::VideoStream &stream) {
printf(
"[DEBUG] video frame: %ux%u format=%u data=%zu bytes\n",
stream.width, stream.height, stream.format,
stream.video_data.size());
if (got.exchange(true))
return;
save_video_frame("internal_video_capture", stream);
});
Python Example:
def on_video(stream):
if got.is_set():
return
result[0] = stream
got.set()
ctrl.subscribe_internal_video_capture(on_video)
Subscribe to the desensitized video stream processed by the audio-visual interaction system
| Function Name | subscribe_internal_video_desensed |
|---|---|
| Function Prototype | void subscribe_internal_video_desensed(VideostreamCallback callback); |
| Function Overview | Subscribe to the desensitized video stream processed by the audio-visual interaction system (the Boomy equipped with a Compute Board needs to continuously publish the external video stream) |
| Parameters | - callback: callback function for processing video stream data |
| Return Value | None |
| Remarks | Boomy equipped with a Compute Board needs to continuously stream external video first before it can be used |
C++ Example:
ctrl->subscribe_internal_video_desensed(
[&](const media::VideoStream &stream) {
printf(
"[DEBUG] video frame: %ux%u format=%u data=%zu bytes\n",
stream.width, stream.height, stream.format,
stream.video_data.size());
if (got.exchange(true))
return;
save_video_frame("internal_video_desensed", stream);
});
Python Example:
def on_video(stream):
if got.is_set():
return
result[0] = stream
got.set()
ctrl.subscribe_internal_video_desensed(on_video)
Suspend the internal camera data acquisition of the robot
| Function Name | pause_video_capture |
|---|---|
| Function Prototype | void pause_video_capture() |
| Function Overview | Pause the internal camera data acquisition of the robot (only applicable to Boomy models without a Compute Board) |
| Parameters | None |
| Return Value | None |
| Remarks | Only available for Boomy models without a Compute Board |
C++ Example:
ctrl->pause_video_capture();
Python Example:
ctrl.pause_video_capture()
Resume the internal camera data acquisition of the robot
| Function Name | resume_video_capture |
|---|---|
| Function Prototype | void resume_video_capture() |
| Function Overview | Resume the internal camera data acquisition of the robot (only applicable to Boomy without a Compute Board) |
| Parameters | None |
| Return Value | None |
| Remarks | Only available for Boomy models without a Compute Board |
C++ Example:
ctrl->resume_video_capture();
Python Example:
ctrl.resume_video_capture()
Publish external video streams to the audio-visual interaction system
| Function Name | publish_external_video_stream |
|---|---|
| Function Prototype | void publish_external_video_stream(VideoStream stream) |
| Function Overview | Publish external video streams to the audio-visual interaction system |
| Parameters | stream: VideoStream struct, containing video frame data |
| Return Value | None |
| Remarks |
C++ Example:
VideoStream stream;
// Populate stream data ...
ctrl->publish_external_video_stream(stream);
Python Example:
stream = VideoStream()
# Populate stream data ...
ctrl.publish_external_video_stream(stream)
Publish external audio stream input to the agent
| Function Name | publish_external_audio_stream |
|---|---|
| Function Prototype | void publish_external_audio_stream(AudioStream stream) |
| Function Overview | Publish external audio stream input to the agent |
| Parameters | stream: AudioStream structure, which contains audio data |
| Return Value | None |
| Remarks |
C++ Example:
AudioStream stream;
// Populate stream data ...
ctrl->publish_external_audio_stream(stream);
Python Example:
stream = AudioStream()
# Populate stream data ...
ctrl.publish_external_audio_stream(stream)
Publish external audio stream output to the internal speaker
| Function Name | publish_external_audio_playback_stream |
|---|---|
| Function Prototype | void publish_external_audio_playback_stream(AudioStream stream) |
| Function Overview | Publish external audio stream output to the internal speaker |
| Parameters | stream: AudioStream structure, which contains audio data |
| Return Value | None |
| Remarks |
C++ Example:
AudioStream stream;
// Populate stream data ...
ctrl->publish_external_audio_playback_stream(stream);
Python Example:
stream = AudioStream()
# Populate stream data ...
ctrl.publish_external_audio_playback_stream(stream)
Pause internal microphone audio stream capture
| Function Name | pause_audio_capture |
|---|---|
| Function Prototype | void pause_audio_capture() |
| Function Overview | Pause internal microphone audio stream capture |
| Parameters | None |
| Return Value | None |
| Remarks |
C++ Example:
ctrl->pause_audio_capture();
Python Example:
ctrl.pause_audio_capture()
Resume internal microphone audio stream capture
| Function Name | resume_audio_capture |
|---|---|
| Function Prototype | void resume_audio_capture() |
| Function Overview | Resume internal microphone audio stream capture |
| Parameters | None |
| Return Value | None |
| Remarks |
C++ Example:
ctrl->resume_audio_capture();
Python Example:
ctrl.resume_audio_capture()
Pause the audio stream output of the internal speaker
| Function Name | pause_audio_playback |
|---|---|
| Function Prototype | void pause_audio_playback() |
| Function Overview | Pause the audio stream output of the internal speaker |
| Parameters | None |
| Return Value | None |
| Remarks |
C++ Example:
ctrl->pause_audio_playback();
Python Example:
ctrl.pause_audio_playback()
Resume the audio stream output of the internal speaker
| Function Name | resume_audio_playback |
|---|---|
| Function Prototype | void resume_audio_playback() |
| Function Overview | Resume the audio stream output of the internal speaker |
| Parameters | None |
| Return Value | None |
| Remarks |
C++ Example:
ctrl->resume_audio_playback();
Python Example:
ctrl.resume_audio_playback()
(5) Demo Test Cases
The SDK provides two complete sample programs with identical functions, implemented in C++ and Python respectively:
- C++:
examples/cpp/example_media.cppcompile and run
./build/example_media <command> [args]
- Python:
examples/py/test_media.pyrun
python3 test_media.py <command> [args]
All output files are saved by default in the out/ folder of the SDK root directory.
Test Case List
- Obtain all common_config configuration information
python3 examples/py/test_media.py
./build/example_media
- Subscribe to the internal microphone to capture a 10-second audio stream
python3 examples/py/test_media.py internal_audio_capture
./build/example_media internal_audio_capture
Subscribe to the internal microphone, capture audio for 10 seconds, and save it to out/internal_audio_capture.wav
- Record the audio stream played by the internal speaker
python3 examples/py/test_media.py internal_audio_playback
./build/example_media internal_audio_playback
Subscribe to the internal speaker, collect data for 10 seconds, and save it to out/internal_audio_playback.wav
- Input the wav audio file to the audio-visual interaction system agent
python3 examples/py/test_media.py external_audio_agent <wav>
./build/example_media external_audio_agent <wav>
Read the wav file, and based on the status of the switches set_internal_capture_audio_data_to_agent_enable and set_external_custom_audio_data_to_agent_enable, then use TO_WAKEUP to wake up the agent, then send the audio in 20ms chunks via publish_external_audio_stream, and resume the switch status after completion
- Publish external audio stream output to the internal speaker
python3 examples/py/test_media.py external_audio_playback <wav>
./build/example_media external_audio_playback <wav>
Read the wav file, convert it to 2-channel interleaved format, split it into 20ms chunks, and transmit them via publish_external_audio_playback_stream for playback
- Subscribe to the internal video stream to capture a frame of image (for models without a Compute Board)
python3 examples/py/test_media.py internal_video_capture
./build/example_media internal_video_capture
Subscribe to the internal video stream and capture a frame to save it to out/internal_video_capture.png
- Subscribe to the internal desensitized video stream to capture a frame of image
python3 examples/py/test_media.py internal_video_desensed
./build/example_media internal_video_desensed
Subscribe to the internal desensitized video stream and capture one frame to save it to out/internal_video_desensed.png
Models equipped with Compute Board need to first publish the external video stream to the agent, as shown in the example below
- Publish external video stream input to the audio-visual interaction program agent
python3 examples/py/test_media.py external_video_publish
./build/example_media external_video_publish
ffmpeg collects /dev/video4 (640x480 yuyv422 @30fps) and publishes it via publish_external_video_stream
5. Camera Example Program
The camera hardware of the Boomy EDU version is directly connected to the Compute Board, which can directly access the hardware interfaces on the Compute Board to obtain data
Please refer to the following link for the Intel D435i camera example programs: https://github.com/IntelRealSense/librealsense
For models with a Compute Board, if you want to use the camera, please stop the official camera image transmission program first
sudo systemctl stop noetix-video-capture.service # **Stop the image transmission service**
sudo systemctl disable noetix-video-capture.service # **Disable auto-start**
sudo systemctl start noetix-video-capture.service # **Start the image transmission service**
sudo systemctl enable noetix-video-capture.service # **Enable auto-start**
6. Rapid Development
All the following operations are performed on the Boomy Compute Board. The Compute Board comes pre-installed with the required compilation environment. You may also deploy the compilation environment via Docker.
For the external development environment setup, refer to: SDK Compilation Environment Deployment Process
Introduction to External Plugin Development
Since the Motion Control Board of humanoid products is not open to the public, user programs cannot be directly deployed and run on the Motion Control Board, so an additional computing carrier is required to execute user programs.
If the model purchased by the user is not equipped with a Compute Board, the "external plugin development" method shall be adopted: the user shall prepare the Compute Board or host by themselves, and connect to the Boomy rear interface via a network cable to communicate with the Motion Control Board.
Since the Motion Control Board only allows communication with the fixed IP of the original Compute Board, and the communication is based on the wired connection of the internal switch, the external plugin device needs to be configured with the same IP as the original Compute Board to simulate the original Compute Board and complete the access.
Wired SSH Login to the Computing Board
ssh noetix@192.168.55.101
Wireless SSH Login to the Computing Board (The wireless IP can be checked via the mobile App's WiFi connection page. The Compute Board uses the WiFi module of the Motion Control Board)
ssh -p 2222 noetix@[ROBOT_WIFI-IP]
(1) SDK Directory Structure
.
├── build.sh # Build script
├── CMakeLists.txt
├── config # Configuration files
├── dds # DDS library files
├── examples # Example codes
├── include
├── lib # Dynamic library files
├── LICENSE
├── policy # Model files
├── pybind # Pybind files
└── README.md
(2) Docker Development Environment Image Deployment
Note: This image package is for the aarch64 architecture and can only be deployed on computers with the aarch64 architecture
a. Extract the compressed package
tar -zxvf noetix_dds.tar.gz
After extraction, the compressed package contains the noetix_dds.tar development image file and the noetix_dds.yaml configuration deployment file
version: "3.8"
services:
noetix_dds:
image: noetix_dds:arm64
container_name: noetix_dds
network_mode: host
volumes:
- /home/noetix/work:/home/noetix/work:rw # Mount working directory
# /home/noetix/work The part before the colon should be changed to the user's working directory; create this directory in advance
stdin_open: true
tty: true
b. Deployment
docker load -i noetix_dds.tar
docker images # Confirm that the noetix_dds arm64 image exists
mkdir -p /home/noetix/work # Create a working directory; the user can choose it themselves
docker compose -f noetix_dds.yaml up -d # Deploy
docker ps # Check if noetix_dds exists
c. Enter the Docker container
docker exec -it noetix_dds /bin/bash
You can start development by simply placing the DDS SDK in the working directory afterwards.
(3) Compile the SDK
cd noetix_sdk_Boomy
./build.sh # One-click build script
After a successful compile, the executable files example_low, example_high, and example_policy_walk will be generated in the root directory of the SDK, and meanwhile the pybind libraries highcontrol_py and lowcontrol_py will be generated in the build directory for Python to call.
(4) Launch the Program
C++
cd noetix_sdk_Boomy
sudo ./build/example_high # or sudo ./build/example_low
Python
cd noetix_sdk_Boomy
python3 examples/py/test_low.py # python3 examples/py/test_high.py
Safety Tips
During the debugging process, please use the robot hanger throughout the whole process to avoid losses caused by out-of-control faulty programs
7. Common Issues and Notes
- DDS client demo: runtime error (1)
ARM architecture:
error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
libssl1.1_1.1.1f-1ubuntu2.16_arm64.deb
sudo dpkg -i libssl1.1_1.1.1f-1ubuntu2.16_arm64.deb
X86 architecture:
error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
libssl1.1_1.1.1f-1ubuntu2_amd64.deb
sudo dpkg -i libssl1.1_1.1.1f-1ubuntu2_amd64.deb

