Bumi Comprehensive Usage Document
I、About the Bumi
(I)Component Information
The Bumi robot has 21 degrees of freedom (DoF) in total. Each arm has 4 DoF, including (left-handed coordinate system):
Shoulder joint pitchShoulder joint rollUpper arm joint yawElbow joint pitch
Each leg has 6 DoF, including:
Hip joint pitchHip joint rollThigh joint yawKnee joint pitchAnkle joint pitchAnkle joint roll
The waist has 1 DoF, including:
Waist joint yaw
Equipped with multiple joint motor degrees of freedom, the Bumi robot is capable of achieving precise motion control and attitude control.
Details are as follows:
| Parameters | Bumi | Bumi - Education Edition |
|---|---|---|
| Height, Width and Thickness (Standing) | 98cm*35cm*20cm | |
| Weight with battery | Approximately 17kg | |
| Total DoF | 21 | |
| DoF per leg | 6 | |
| DoF per arm | 4 | |
| Waist DoF | 1 | |
| Computing module | / | NVIDIA Jetson Orin (Nano Super / NX) *Computing module not available for certain models |
| Joint communication | Ethercat & Canfd | |
| Maximum joint torque | 70N.m | |
| Lower-leg & thigh length | Approximately 513mm | |
| Arm span | Approximately 312mm | |
| Power supply | Lithium Battery | |
| Base computing power | 6 Tops | 6 + 67 / 117 Tops |
| Camera model | DC_V93 | Realsense D435ICB(EDU_Air except) |
| Perception sensors | IMU、cameras | |
| Microphone array | standard configuration | |
| Speaker | standard configuration | |
| Smart battery | Quick-release battery | |
| Image transmission | Available within the app | hardware can be directly accessed on the computing board during development |
| Charger | Yes | |
| Mobile app | Yes | |
| OTA upgrade | Yes | |
| Battery life | 2~3H | |
| Secondary development (HighController) | Yes | |
| Secondary development (LowController) | / | Yes |
| Component overview diagram | ||
(II)Exterior Component Dimensions

(III) Electrical Interface
The following electrical interfaces are provided on the back of Bumi:

| Serial No. | Type | Description | Quantity |
|---|---|---|---|
| 1 | RJ45 | Switch 1000 BASE-T | 1 |
| 2 | USB | Motion Control Board / Computing Board USB 3.0 | 2 |
| 3 | HDMI | Motion Control Board / Computing Board HD Display Cable | 1 |
For Bumi models with a computing board, the rear interfaces belong to the computing board. For models without a computing board, the rear interfaces belong to the motion control board.
(IV) Airborne Computer
| Type | Model number | User | Password | IP | computing power | Frequency | Memory |
|---|---|---|---|---|---|---|---|
| Motion Control Board | Rockchip RK3576 | The Motion Control Board is not open to users | The Motion Control Board is not open to users | 192.168.55.102 | 6 Tops | 2.4GHz | 8+64GB |
| Computing Board | NVIDIA Jetson orin nano super | noetix | bumi | 192.168.55.101 | 67 Tops | 1.7GHz | 8+128GB |
| Computing Board | NVIDIA Jetson orin NX | noetix | bumi | 192.168.55.101 | 117 Tops | 2.0GHz | 16+128GB |
The Motion Control Board (Rockchip RK3576) is not accessible to users. If the user accesses the Motion Control Board without authorization and this causes the robot’s motion control program to malfunction or lose control, our company shall not be responsible for any resulting damage or losses.
(V)Camera Information
Bumi-EDU:Realsense D435i
Intel® RealSense™ Depth Camera D435i - Product Specifications | Intel
Bumi-Standard: DC_V93

(VI)Battery Information
| Parameters | Information |
|---|---|
| Brand | Shenzhen Grepow |
| Model number | GRP6752097-13S1P |
| Capacity | 5100mAh |
| Voltage | 48V |
| Weight | 1360g |
(VII) Charger Specifications
| Parameters | Information |
|---|---|
| Product Model | TA300 Robot |
| Input | 100-240V~50/60Hz 4.5A |
| Battery Type | LiPo/LiFe/LiHV 13S-16S |
| Dimensions | 172*97*46mm |
| Product Name | Smart charger |
| Output | 35-72V ⎓ 7.5A 350W Max |
| Working Mode | Charge/charging |
| Weight | 510g±10g |
II. Instructions for Use
(I)Powering On and Precautions
1. Removing and Placing the Device
- Remove Bumi from its storage position.
- Steady the robot and place it in a seated position on the floor.
2. Installing the Battery
- Insert the battery into the battery compartment on the back of Bumi.
- Confirm that the battery is fully seated and the latch is locked.
3. Powering On
- Press the battery switch once briefly and confirm that the battery indicator lights up.
- Then press and hold the battery switch until the indicator shows a flowing-light effect.
- Wait for the robot to complete startup.
4. Initial Attitude Adjustment
- Place Bumiflat on its back facing upward on the floor,Make sure the floor is level and non-slip, there are no foreign objects on Bumi's foot plates, and there are no obstacles around it, so that surrounding objects do not interfere with the automatic stand-up sequence.
5. System Initialization Instructions
- After power-on, Bumi automatically enters full-body enabled mode::
- All joints automatically move to the zero position
- Do not interfere with or obstruct joint movement during this process
⚠️ CAUTION
- During startup, avoid touching the joints to prevent pinching injuries or interference with initialization.
- Ensure the ground is level to avoid abnormal attitude.
- Do not power on the robot unless the battery is correctly installed.
(II) Bumi Connection Method

After connecting to Bumi, do not select "Forget Device" in the system settings, as this will clear the other Bluetooth records and prevent the device from reconnecting.If this happens by mistake,please contact customer service.
(III) Connecting and Controlling Bumi with the Mobile App
Noetix Run App Download:
Android:

iOS:NOETIX Run App - App Store
Noetix Run App User Guide: Noetix Run App User Guide
(IV) Powering Off and Battery Charging
I. Preparing the Device for Storage
- Run the One-Tap Storageoperation in the App.
- After storage is complete, steady Bumi and place it in a seated positionon the floor.
II. Powering Off
- Press and hold the battery switch until Bumi is completely powered off.
III. Removing the Battery
- Press and hold the battery latch.
- Remove the batteryfrom the battery compartment on the back of Bumi.
IV. Charging the Battery
- Place the battery on the battery base and make sure the connection is secure.
- Connect the charging adapter to the battery base.
- Plug the adapter into a 220 V power outlet.
V. Confirming Charging Status
- Check the battery indicator:
- ndicator flashing: charging
- No indicator: battery fully charged
(V)Operation Tutorial Videos
(I)App Download
(II)Powering On
(III)Activation and Binding
After connecting to Bumi, do not select 'Forget Device' in the system settings, as this will clear the Bluetooth records and prevent the device from reconnecting. If this happens by mistake, please contact customer service.
(IV)One-Tap Stand-Up
When using Bumi's One-Tap Stand-Up function, make sure the floor is level and non-slip, there are no foreign objects on Bumi’s foot plates, and there are no obstacles around it, so that surrounding objects do not interfere with the automatic stand-up sequence.
(V)Movement
(VI)Basic Actions
(VII)Teaching Mode
(VIII)Dance Mode
(IX)Action Library
(X)Image Transmission Mode
(XI)One-Tap Storage - Powering Off - Battery Charging
Before executing One-Tap Storage in the App, make sure the floor is level and non-slip and that there are no obstacles around the robot.
(XII) Voice Interaction
(VI) Bumi Safety Manual
At present, humanoid robots are still in a stage of rapid development, and there is still room for improvement in both software and hardware technologies. To keep Bumi in good working condition and allow it to accompany you for a longer time, please read and follow this user guide carefully to avoid improper operation. Thank you for your understanding and support!
List of Hazardous Operations
- Do not use the Bumi robot in rainy weather
At present, to strike a balance between heat dissipation and protection, the Bumi robot does not feature an ultimate waterproof effect, and users should avoid operating it in humid, misty or rainy environments.
- Do not use Bumi on complex terrain for play
To balance material weight and motion-control performance, Bumi's motion-control performance has not reached an optimal level. Due to current industry limitations, most humanoid-robot motion-control algorithms are effectively “blind walking”: the built-in control algorithms cannot autonomously sense changes in terrain and adjust gait. Adaptability is low on uneven, soft, slippery surfaces, stairs, and slopes greater than 20 degrees, creating a risk of falls and damage. Do not use the robot in the above environments.
- Do not use Bumi's stand-up functions incorrectly
Bumi's One-Tap Stand-Up and Assisted Stand-Up functions both have explicit prerequisites.
Autonomous Stand-Up:
After Bumi powers on, it must be lying flat on the floor face-up, with all four limbs naturally positioned on both sides of the waist and both legs fully extended.
Do not power on with the robot's limb motors mechanically blocked at their limits. After startup, Bumi adjusts all motors to their initial positions. If a motor is already at a limit position, Bumi may be unable to reverse it, which may create a risk of motor burnout.
Assisted Stand-Up:
After Bumi powers on and enters enabled mode, the whole-body motors move to their initial positions. The user must assist Bumi into a standing posture and then use the Assisted Stand-Up function in the App.
Do not use Assisted Stand-Up when the robot is in any other posture, such as lying face-up or face-down. Otherwise, when Assisted Stand-Up is used, Bumi's motion-control program may interpret the IMU values as indicating that the robot has fallen and automatically enter protection mode.
- Do not use the stand-up function in an area with many surrounding obstacles
When using the stand-up function, note that the movement range is 3 m x 3 m. Do not place obstacles within this area. They may interfere with the motion-control program, causing stand-up failure or even a fall and damage.
- Do not use the stand-up function when Bumi's joint motors are placed at the wrong limit positions
When powering on Bumi, do not place the joint motors at their limit positions. Once Bumi's startup motion control program is fully launched, it will automatically adjust all the motors of the robot to their initial positions. If the joint motors are stuck due to an improper placement posture, there is a risk of motor burnout.
- Do not use the storage function in an area with many surrounding obstacles
Do not use the storage function where there are many surrounding obstacles, as this may affect the motion-control program and create a risk of loss of motion control, falling, and damage.
- Do not use the dance function in an area with many surrounding obstacles or pedestrians
Due to current industry limitations, most humanoid-robot motion-control algorithms are effectively “blind dancing.” In an area with many obstacles or pedestrians, there is a risk of falls, damage, or collisions with people.
- Do not intentionally switch Bumi between function modes rapidly and repeatedly
The motion-control program architecture has been highly optimized, so mode switching is very smooth and fast and can be performed between two modes without noticeable delay. However, physical inertia cannot be eliminated. Intentionally switching rapidly between two modes may generate excessive inertia and cause the robot to fall and be damaged.
- Do not intentionally interfere with Bumi while it is moving
Due to current industry limitations, most humanoid-robot motion-control algorithms are effectively “blind walking.” The motion-control program cannot truly sense the surrounding environment or actively resist disturbances to maintain stability, and Bumi is the same. Therefore, do not intentionally interfere with Bumi while it is moving, as this creates a risk of falling and damage.
- Do not lift the robot to keep its feet off the ground when Bumi is in a full-body walking gait.
When using the Bumi full-body walking gait, do not lift the robot to keep its feet suspended in the air. Since the current robot motion control program cannot accurately identify that the robot has been lifted off the ground under all circumstances, the robot may still continuously output joint torque according to the normal walking gait, which will cause abnormal stress on the joints and transmission components, and there are risks of motor damage as well as the robot falling and being damaged after landing back on the ground.
Video of Incorrect Operation
III. Bumi Development Guide
Introduction to SDK
This document describes Bumi development based on the DDS messaging middleware. The SDK adopts a client/server (C/S) architecture: under an isolation mechanism, the DDS client program deployed on the robot's secondary-development computing board sends control commands at a rate of 500 Hz to the motion control program running on the motion control board, which acts as the server, and continuously retrieves the robot's current state.
(I) System Architecture Diagram

(II) Developing Interfaces
DDS C++ Python
(III) Obtaining the SDK
https://github.com/Noetix-Robotics/noetix_sdk_bumi
(IV) URDF Files
(V) Application Development - HighController
Introduction to Application Development
HighController is developed for upper-layer scheduling based on the OEM motion control strategy, which enables the rapid construction of robot automated applications without modifying the underlying control logic, supports integration with large language models, and facilitates the creation of customized intelligent behavior systems.
Typical applications include:
- Combining visual algorithms with the OEM motion control system to realize autonomous perception and autonomous navigation
- Integrating with the voice interaction module to realize natural language control
- Integrating with large language models to dynamically generate and invoke original manufacturer's control actions based on inputs (this carries certain risks, requiring strict security policies)
- Implementing a positioning and navigation inspection system in coordination with the External Perception Radar
Simply put, HighController encapsulates the original factory's control capabilities through high-level scheduling, enabling automated and programmatic invocation of the controllable behaviors of remote controllers or APPs, and on this basis, extends the system with selected autonomous and intelligent capabilities.
I、HighController Architecture Diagram

(1) Status reporting mechanism
After the Client SDK establishes a connection with the robot motion control program, the motion control program will continuously send real-time robot status information to the Client, including:
- Whole-body motor parameter data (pos (position), vel (velocity), tau (torque), kp, kd, temperature, error(error message))
- IMU status data
- Current working mode (
workmode, indicating the currently running control strategy) - Aolion Controller Data (for Synchronized Manual Input or Auxiliary Control)
- Battery level and related system status information
(2) Client control instruction delivery
The Client, based on its own application scenarios, sends control commands to the operation and control program, which mainly include:
- (x, y, z) movement control data
is only valid in specific modes, such as walking mode, handshake, greeting, cheering, teaching mode, etc.
- action command
corresponds to workmode, which is used to switch or trigger the algorithm strategy currently running on the robot.
- data additional DDS data
PLAYTEACH (playback teaching mode), and is used to specify additional parameters such as the teaching file index.
(3) Internal processing flow of the operation control program
After receiving the client's instructions, the control program:
- Transfers the control parameters to the internal control algorithm
- The algorithm calculates the strategy based on the reinforcement learning model
- Generate the target expected values (target position, velocity or torque) of the motor and send them to the motor control interface
- Deliver the calculation results to the hardware control layer
(4) Hardware execution layer
After the hardware layer receives the target control data:
- Execute motor control commands
- Perform the corresponding expected action
Throughout the entire process, users only need to issue a portion of high-level control parameters.
The specific motion planning, dynamic balance and motor-level control calculations are all completed by the Black Box Algorithm in the original manufacturer's motion control program.
Therefore, in HighController mode, the range of capabilities that users can invoke and automate is, in principle, limited to the functional boundaries originally achievable by the APP or the controller, without granting underlying motor-level control access.
II、HighController Operation Flowchart

(1) Initialization phase (init)
After the system is started, the following initialization process shall be completed:
- Establish a communication connection between the Client and the robot operation control program
- Register a callback function to receive robot status data
(2) The DDS callback thread continuously receives postback status
The operation control program continuously pushes robot status data to the Client.
After receiving the data, the callback function encapsulates it and stores it in the buffer and the global state variable respectively:
motor_state_buffer_:Motor status bufferimu_buffer_:IMU status bufferjoy_buffer_:remote controller data buffercurmode:Current operation control strategy modeBmsState:Current battery information
III、Common Interfaces and Parameters of HighController
| HighController Interfaces | ||
|---|---|---|
| Interface Name | Interface Description | return type |
| instance | Obtain the singleton pointer of HighController | Object pointer |
| init | HighController Object Initialization | None |
| publish_cmd | Issue DDS control command data | None |
| subscribe_robot_hardware_status | Subscribe to robot hardware status information | None |
Obtain the singleton instance of the HighController object
| Function Name | instance |
|---|---|
| Function Prototype | static HighController *Instance() |
| Function Overview | Obtain the HighController singleton |
| Parameters | None |
| Return Value | Pointer to the HighController singleton object |
| Note |
C++ Example:
HighController *ctrl = HighController::Instance();
Python Example:
ctrl = HighController.instance()
HighController Object Initialization
| Function Name | init |
|---|---|
| Function Prototype | bool init() |
| Function Overview | HighController Object Initialization |
| Parameters | None |
| Return Value | bool |
| Note |
C++ Example:
ctrl->init();
Python Example:
ctrl.init()
Publish robot control commands
| Function Nam | publish_cmd |
|---|---|
| Function Prototype | void publish_cmd(double x, double y, double z, ControlCmd action, uint16_t index) |
| Function Overview | Use HighController to send DDS control commands |
| Parameters |
|
| Return Value | None |
| Note | This interface cannot be called continuously; otherwise, the robot will receive commands too fast to respond, and a delay of at least 2ms needs to be added after each command is sent. |
C++ Example:
ctrl->publish_cmd(0, 0, 0, ControlCmd.CHEER, 0);
Python Example:
ctrl.publish_cmd(0, 0, 0, ControlCmd.CHEER, 0)
actionThe parameter indicates switching the robot to the corresponding mode
| Parameter Name | Define Prototype | Serial No. | Example Value | Description |
|---|---|---|---|---|
ControlCmd | | 0 | WALK | Walking Mode |
| 1 | SWING | Factory-Preset Action: Wave | ||
| 2 | SHAKE | Factory-Preset Action: Handshake | ||
| 3 | CHEER | Factory-Preset action: Cheer | ||
| 4 | RUN | Reserved mode, not available in the current version | ||
| 5 | START | Enable/Disable | ||
| 6 | SWITCH | Preparation Mode | ||
| 7 | STARTTEACH | Start teaching | ||
| 8 | SAVETEACH | Save Teaching | ||
| 9 | ENDTEACH | End Teaching (Deprecated)) | ||
| 10 | PLAYTEACH | Play Demonstration | ||
| 11 | DANCE | Dance 1 Mode | ||
| 12 | FALLTOSTAND | Get up from the ground | ||
| 13 | STANDTOFALL | Get up and fall to the ground | ||
| 14 | DANCE1 | Dance 2 Mode | ||
| 15 | DANCE2 | Dance 3 Mode | ||
| 16 | TEAR | Preset Action: Wipe Tears | ||
| 17 | DEFAULT | Null Mode |
Purpose of the DEFAULT Empty Mode
And the process control function runs periodically. If teaching-related commands such as
action = ControlCmd::PLAYTEACH;
are continuously issued within this loop, the robot may repeatedly interrupt the teaching playback action before it has finished, causing the robot to appear stuck and unresponsive.
Correct Practice
PLAYTEACH is a one-time trigger instruction (event),and shall only be sent once;afterward, continuously send DEFAULT(a no-op command) in the loop to prevent repeated triggering.
Sample Code (Revised))
if (is_first) {
action = ControlCmd:: PLAYTEACH ; // Triggered only once`
is_first = false;
} else {
action = ControlCmd::DEFAULT;
}
Effect
- Received
PLAYTEACH→ Trigger and fully execute the teaching action - Subsequent
DEFAULT→ No longer interrupts the action - The robot behaves normally and predictably
Action commands shall be "edge-triggered" and must not be continuously issued within a control loop.
START Enable/Disable
workmode == 30,send START to put the robot into enabled mode
workmode != 30,send START to immediately put the robot into disabled mode
indexis used to specify the teaching file index in teaching-related modes
ctrl.publish_cmd(0, 0, ControlCmd.STARTTEACH, 0) # Begin teaching
sleep(10) # Record teaching actions
ctrl.publish_cmd(0, 0, ControlCmd.SAVETEACH, 1) # End and save the teaching action, specifying index 1.
sleep(2) # Allow the operation control program to complete the save process to prevent it from being interrupted by the next command.
ctrl.publish_cmd(0, 0, ControlCmd.PLAYTEACH, 1) # play teach file 1
Only when action is one of the following modes:
TEAR/PLAYTEACH/WALK/SWING/CHEER/SHAKE/DEFAULT
x and yaw will take effect to control the robot's movement; in other modes, x and yaw are invalid.
Subscribe to robot hardware status information
| Function Name | subscribe_robot_hardware_status |
|---|---|
| Function Prototype | void subscribe_robot_hardware_status(RobotHardwareStatusCallback callback); |
| Function Overview | Subscribe to robot hardware status information |
| Parameters | - callback: callback function for processing robot status data |
| Return Value | None |
| Note |
C++ Example:
ctrl->subscribe_robot_hardware_status(
[](const RobotHardwareStatus &status) {
latest_hw_status = status;
});
Python Example:
def _on_hw_status(status):
global latest_hw_status
latest_hw_status = status
ctrl.subscribe_robot_hardware_status(_on_hw_status)
| Parameter Name | Define Prototype | Parameters | Description |
|---|---|---|---|
| RobotHardwareStatus | | imu_data | IMU data |
| remote_data | Controller Data | ||
| motor_data | Motor Data | ||
| bms_data | Battery Data | ||
| workmode | Current operating mode |
workmode parameter correspondence table
The workmode value is only valid in highcontroller mode
| int值 | Description |
|---|---|
| 0 | Enable Mode |
| 1 | Preparation Mode |
| 2 | Walking Mode |
| 5 | Dance Mode |
| 8 | Factory Preset Action: Greeting |
| 9 | Factory Default Action: Handshake |
| 10 | Factory Default Action: Cheer |
| 11 | Start teaching |
| 12 | End Teaching |
| 14 | Save Teaching 1 |
| 23 | Play Demonstration |
| 26 | Protected Mode |
| 27 | Get up from the ground |
| 28 | Get up and fall to the ground |
| 29 | Save Teaching 2 |
| 30 | Disabled Mode |
| 31 | Dance Mode 1 |
| 32 | Dance Mode 2 |
| 33 | tear-wiping gesture |
Save Teaching Mode
After the SDK sends a SAVETEACH command to the robot, the robot will sequentially return three statuses: Exit Teaching Save Teaching 1 Save Teaching 2
Protected Mode
Under the standing control strategy (walking, teaching, dancing, etc.), the motion control program continuously monitors the IMU attitude data.
When the system detects that the posture deviates from the standing stability range and exceeds the preset threshold, it will immediately trigger the protection mechanism and forcibly switch to the safe mode.
In protected mode:
- All joints of the body enter a disabled state (the motor stops active output)
- Terminate the current motion control strategy
- Block the execution of further action instructions
The purpose of this mechanism is as follows:
- Prevent the robot from continuously outputting control torque during a fall or instability
- Avoid secondary injury caused by uncontrolled swinging of the limbs
- Mitigate potential damage to the equipment itself and the surrounding environment
This protection logic, as a underlying safety protection mechanism, enjoys the highest priority and will not be affected by upper-layer control instructions.
Handle Data Structure
| Parameter Name | Define Prototype | Example Value | Description |
|---|---|---|---|
| joydata | | axes | Remote control remote sensing linear value |
| button | Remote control button press mapping value: 1 means pressed, 0 means normal |
| Handle Joystick Mapping | Mapping Description | Scope |
|---|---|---|
| axes[0] | Right Stick Left-Right Push Value | [-1, 1] |
| axes[1] | Left Stick Forward/Backward Push Value | [-1, 1] |
The Bumi backplane with a computing board features an interface for the computing board, and the motion control board cannot accommodate the controller receiver, so the DDS cannot obtain controller data. Therefore, the controller data needs to be acquired directly from the computing board. For details, please refer to Direct Acquisition of Controller Data via computing Board
IMU State Data Structure
| Parameter Name | Define Prototype | Example Value | Description |
|---|---|---|---|
| NingImuData | | ori | Attitude Quaternion |
| ori_cov | attitude covariance | ||
| angular_vel | angular velocity | ||
| angular_vel_cov | angular velocity covariance | ||
| linear_acc | linear acceleration | ||
| linear_acc_cov | Linear acceleration covariance |
Motor status data structure
| Parameter Name | Define Prototype | Example Value | DescriptionDescription |
|---|---|---|---|
| MotorState | | pos | Motor Position |
| vel | Motor speed | ||
| tau | Motor Torque | ||
| motor_id | Motor ID | ||
| error | Motor Error Code: | ||
| temperature | Temperature |
Battery Status Data Structure
| Parameter Name | Define Prototype | Example Value | Description |
|---|---|---|---|
| RobotBmsData | | battery_temp_ | Temperature |
| battery_alarm_ | Alarm Information | ||
| battery_soc_ | Electric quantity | ||
| battery_soh_ | Health Level |
IV、Operation Control Status Switching Flowchart

Note: During the development process with HighController, please follow the above flow chart for mode switching to ensure rationality and safety.
V、Introduction to Demo Routines
In the SDK folder, examples/cpp/example_high.cpp 与 examples/py/test_high.py provide a sample demo, which is used to parse the input data of the Aolion Controller, and generate corresponding control instructions according to the controller operations to send to the robot.
(VI)Operation Control Development - LowController
Introduction to Operations Control Development
LowController exposes the motor control interface in the motion control program to the SDK, allowing users to directly adjust underlying motor parameters at a control frequency of 500Hz. Combined with the deployment of reinforcement learning algorithms, it enables in-depth customization of Bumi's motion capabilities, allowing it to complete complex behaviors such as diverse new movements and dances.
Note: Only the EDU version of Bumi supports LowController development
(I)Bumi Training Framework
https://github.com/Noetix-Robotics/noetix_bumi_lab
(II)LowController Architecture Diagram

(1) Status Data Reporting
After the client SDK establishes a connection with the robot motion control program, the motion control program continuously pushes real-time robot status information to the client, including:
- Whole-body motor parameter data(pos position, vel velocity, tau torque, kp, kd, temperature, error)
- IMU status data.
- Aolion controller data (for synchronizing manual input or auxiliary control).
(2) Client-side control calculation
The Client, in combination with its own algorithm (usually a reinforcement learning model), performs real-time policy calculation to generate the target expected values (such as target position, etc.) for each motor, writes the motor control instruction array into the buffer, and then delivers it to the motion control program.
(3) Forwarding Mechanism of Operation Control Program
After the operation control program receives the motor command array issued by the Client, it does not perform high-level strategy calculation, but directly forwards the control parameters to the hardware control layer.
(4) Hardware Execution Layer
The hardware layer receives the target parameters of the motor, executes the control commands, and drives the motor to complete the expected actions.
In LowController mode:
- The reinforcement learning model is deployed on the Client side
- Users can directly participate in motor-level control
- The operation control program is mainly responsible for communication and secure forwarding.
- The control frequency can reach
500Hz
This mode features higher control freedom and flexibility, but it also means:
- Higher requirements for algorithm stability.
- Stricter requirements for real-time performance and system security.
- A potential direct impact on robot motion safety if the control strategy is unstable.
(III)LowController Operation Flowchart

(1) Initialization phase (init)
After the system starts, complete the following initialization steps:
- Establish a communication connection between the Client and the robot operation control program
- Register robot status data receiving callback function
- Create
send_threaddata transmission thread
(2) Status Data Callback Mechanism
The operation control program continuously pushes robot status data to the Client.
After receiving the data, the callback function encapsulates it and stores it in the corresponding buffer or global variable:
motor_state_buffer_: Motor status bufferimu_buffer_: IMU status bufferjoy_buffer_: controller data buffer
This stage is only responsible for data reception and caching, and does not participate in control strategy calculation.
(3)send_thread(data transmission thread)
The data transmission thread also operates at a 2 ms cycle:
- If
motor_cmd_buffer_is not empty - Read the latest motor control command
- Send the motor target array to the motion control program
The operation control program is only responsible for forwarding data to the hardware layer and does not participate in high-level control decisions.
(4) Architectural Features
The core features of the LowController mode are as follows:
- Client directly participates in motor-level control
- Reinforcement learning or custom control algorithms run on the Client
- The operation control program is no longer responsible for high-level policy calculation, and only undertakes the responsibilities of communication and secure forwarding.
- The control frequency can reach up to 500Hz
This model offers higher control freedom and algorithmic plasticity, but at the same time, it imposes requirements on:
- Real-time performance
- Algorithm Stability
- Thread Safety and Scheduling Precision
set higher requirements.
4. Joint Motor Information
| ID | Name | Joint | Torque | Angle Range | Radian Range | ||
|---|---|---|---|---|---|---|---|
| 0 | arm_l1_joint | Left shoulder joint pitch | 5 N.m | -135 | 135 | -2.36 | 2.36 |
| 1 | arm_l2_joint | Left shoulder joint roll | 5 N.m | -8 | 111 | -0.14 | 1.94 |
| 2 | arm_l3_joint | Left arm joint yaw | 5 N.m | -90 | 90 | -1.57 | 1.57 |
| 3 | arm_l4_joint | Left elbow pitch | 5 N.m | -129 | 0 | -2.26 | 0.00 |
| 4 | leg_l1_joint | Left hip pitch | 60 N.m | -120 | 120 | -2.09 | 2.09 |
| 5 | leg_l2_joint | Left hip roll | 60 N.m | -38 | 90 | -0.66 | 1.57 |
| 6 | leg_l3_joint | Left thigh joint yaw | 15 N.m | -145 | 145 | -2.53 | 2.53 |
| 7 | leg_l4_joint | Left knee joint pitch | 60 N.m | 0 | 128 | 0.00 | 2.24 |
| 8 | leg_l5_joint | Left ankle pitch | 30 N.m | -55 | 25 | -0.96 | 0.44 |
| 9 | leg_l6_joint | Left ankle roll | 30 N.m | -10 | 10 | -0.17 | 0.17 |
| 10 | arm_r1_joint | Right shoulder pitch | 5 N.m | -135 | 135 | -2.36 | 2.36 |
| 11 | arm_r2_joint | Right shoulder joint roll | 5 N.m | -111 | 8 | -1.94 | 0.14 |
| 12 | arm_r3_joint | Right arm joint yaw | 5 N.m | -90 | 90 | -1.57 | 1.57 |
| 13 | arm_r4_joint | Right elbow joint pitch | 5 N.m | -129 | 0 | -2.26 | 0.00 |
| 14 | leg_r1_joint | Right hip pitch | 60 N.m | -120 | 120 | -2.09 | 2.09 |
| 15 | leg_r2_joint | Right hip joint roll | 60 N.m | -90 | 38 | -1.57 | 0.66 |
| 16 | leg_r3_joint | Right thigh joint yaw | 15 N.m | -145 | 145 | -2.53 | 2.53 |
| 17 | leg_r4_joint | Right knee joint pitch | 60 N.m | 0 | 128 | 0.00 | 2.24 |
| 18 | leg_r5_joint | Right ankle pitch | 30 N.m | -55 | 25 | -0.96 | 0.44 |
| 19 | leg_r6_joint | Right ankle joint roll | 30 N.m | -10 | 10 | -0.17 | 0.17 |
| 20 | waist_1_joint | Waist joint yaw | 27 N.m | -90 | 90 | -1.57 | 1.57 |
MotorCmd. pos corresponds to the radian information
(V)Aolion Controller Data Mapping
Aolion Controller. The button[] and key mapping table are as follows:
| Key Description | Button index |
|---|---|
| Right Shoulder Button | 1 |
| Left Shoulder Button | 2 |
| X | 5 |
| Y | 6 |
| B | 7 |
| A | 8 |
| + | 9 |
| - | 10 |
| None | 11 |
| Press the left joystick | 12 |
| Handle Joystick Mapping | Mapping Description |
|---|---|
| axes[0] | Right Stick Y-Axis Value |
| axes[1] | Left Stick X-axis Value |
Joystick Data Range
| Minimum Value | Maximum Value |
|---|---|
| -1 | 1 |
(VI)Common Interfaces and Usage Routines of LowController
| LowController Interface | ||
|---|---|---|
| Interface Name | Interface Description | return type |
| instance | Obtain the singleton pointer of LowController | Object pointer |
| init | LowController Object Initialization | None |
| set_joint | Write the motor control command into the buffer | None |
| getJointsIndex | Obtain the corresponding ID based on the joint name | int |
| subscribe_robot_hardware_status | Subscribe to robot hardware status information | None |
Obtain the singleton instance of the LowController object
| Function Name | instance |
|---|---|
| Function Prototype | static LowController *Instance() |
| Function Overview | Obtain the LowController singleton |
| Parameters | None |
| Return Value | Pointer to the LowController singleton object |
| Note |
C++ Example:
LowController *ctrl = LowController::Instance();
Python Example:
ctrl = LowController.instance()
LowController Object Initialization
| Function Name | init |
|---|---|
| Function Prototype | bool init() |
| Function Overview | LowController Object Initialization |
| Parameters | None |
| Return Value | bool |
| Note |
C++ Example:
ctrl->init();
Python Example:
ctrl.init()
The motor control command array is stored in the buffer
After being stored in the buffer, the data will be automatically sent by the send_thread thread.
| Function Name | set_joint |
|---|---|
| Function Prototype | void set_joint(std::array<MotorCmd, 21> motorcmd) |
| Function Overview | Set joint data |
| Parameters | - motorcmd motor command structure |
| Return Value | None |
| Note |
| Parameter Name | Define Prototype | Example Value | Description |
|---|---|---|---|
| MotorCmd | | pos | Motor Position |
| vel | Motor speed | ||
| tau | Motor Torque | ||
| kp | pid parameter | ||
| kd | pid parameter | ||
| motor_id | Motor ID |
When issuing motor control commands, please strictly refer to the specified parameter ranges in.
C++ Example:
std::array<MotorCmd, 21> motorcmd;
// Algorithm logic omitted...
ctrl->set_joint(motorcmd);
Python Example:
cmds = [MotorCmd() for _ in range(21)]
# Algorithm logic omitted...
ctrl.set_joint(cmds)
Obtain the corresponding joint motor ID according to the joint motor name
| Function Name | getJointsIndex |
|---|---|
| Function Prototype | int getJointsIndex(std::string jointname) |
| Function Overview | Obtain the corresponding joint motor ID according to the joint motor name |
| Parameters | jointname: Joint Motor Name |
| Return Value | int:Corresponding joint motor ID |
| Note |
C++ Example:
int id = ctrl->getJointsIndex("arm_l1_joint");
Python Example:
id = ctrl.getJointsIndex("arm_l1_joint")
Subscribe to robot hardware status information
| Function Name | subscribe_robot_hardware_status |
|---|---|
| Function Prototype | void subscribe_robot_hardware_status(RobotHardwareStatusCallback callback); |
| Function Overview | Subscribe to robot hardware status information |
| Parameters | - callback:callback function for processing robot status data |
| Return Value | None |
| Note | The usage is the same as that of HighController subscribe_robot_hardware_status |
Obtain the data of the Aolion controller (for Bumi models equipped with a computing board, the data can be directly obtained via the computing board)
AoLionDriver class
| AoLionDriver Class Interface | |
|---|---|
| init | Initialize |
| getremotedata | Obtain controller data |
| Function Name | init |
|---|---|
| Function Prototype | bool init(std::string port, int baudrate) |
| Function Overview | Initialize the handle device port |
| Parameters |
|
| Return Value | true:Initialization succeeded falese: Initialization failed |
| Note |
| Function Name | getremotedata |
|---|---|
| Function Prototype | const joydata getremotedata() |
| Function Overview | Obtain controller data |
| Parameters | None |
| Return Value | joydata:Handle data structure |
| Note |
Usage Example
C++
AoLionDriver aoliondriver;
joydata remote_data;
aoliondriver.init("/dev/input/js0", 115200);
const joydata jdata=aoliondriver.getremotedata();
for (int i = 0; i < 14; i++)
std::cout << jdata.button[i] << std::endl;
std::cout << jdata.axes[0] << std::endl;
std::cout << jdata.axes[1] << std::endl;
Python
al = AoLionDriver()
al.init("/dev/input/js0", 115200)
jdata = al.getremotedata()
for i in range(14):
print(jdata.button[i])
print(jdata.axes[0])
print(jdata.axes[1])
(VII)Introduction to Demo Routines
examples/cpp/example_policy_walk.cpp and examples/py/test_policy.py run a Bumi 21 degrees of freedom running strategy(policy/policy.onnx)of the sim2real (algorithm model real machine deployment) code, according to the policy provides some reinforcement learning deployment interface
| Function Name | loadModel |
|---|---|
| Function Prototype | bool loadModel(std::string modelpath) |
| Function Overview | Load custom models |
| Parameters |
|
| Return Value | true: Load succeeded; false: Load failed |
| Note |
| Function Name | onnxdatainit |
|---|---|
| Function Prototype | void onnxdatainit() |
| Function Overview | Initialize onnx data |
| Parameters | None |
| Return Value | None |
| Note |
| Function Name | getmodelparam |
|---|---|
| Function Prototype | bool getmodelparam() |
| Function Overview | Obtain model parameters from the yaml file |
| Parameters | None |
| Return Value | true: YAML file read successfully; false: Failed to read YAML file |
| Note |
| Function Name | setparameter |
|---|---|
| Function Prototype | void setparameter(Command &cmd,bool* isfirst) |
| Function Overview | Set the overall control parameters of the xyz unit |
| Parameters |
|
| Return Value | None |
| Note |
| Function Name | updatestateestimation |
|---|---|
| Function Prototype | bool updatestateestimation() |
| Function Overview | Update the status, including joint angles, IMU data, etc. , and obtain model parameters from the yaml file |
| Parameters | None |
| Return Value | true: YAML file read successfully; false: Failed to read YAML file |
| Note |
| Function Name | computeObservation |
|---|---|
| Function Prototype | void computeObservation() |
| Function Overview | Calculate the observation coefficients |
| Parameters | None |
| Return Value | None |
| Note |
| Function Name | computeActions |
|---|---|
| Function Prototype | void computeActions() |
| Function Overview | Calculate the final action |
| Parameters | None |
| Return Value | None |
| Note |
Joystick Operation Procedure
| Usage Instructions | |
|---|---|
+ | All joints return to zero position and enter damping mode |
LB & - | Keep your limbs slightly bent, and get into the pre-strategy preparation mode |
LB & X | Enter the policy-controlled running mode |
(VII) Audio and video Development
Introduction to Audio Development
MediaController establishes a communication link with the voice module on the motion control board via DDS topics, enabling control of Bumi audio peripherals (such as speakers and microphones) and providing capabilities for voice interaction agent configuration and data stream management.
Note
Only the EDU version of Bumi supports MediaController development
The open-source interfaces are as follows:
Status control and acquisition interface: supports functions including wakeup control, sleep control, module restart, operating status acquisition, and error information acquisition;
General configuration interface: an interface for configuring and retrieving general parameters, including non-dialogue sleep timeout duration, wake-up word, wake-up reply word, sleep reply word, as well as audio-visual data stream control;
Audio Interface: Audio stream push and retrieval, audio control commands (pause/resume, volume adjustment, etc.)
Video interface, video stream push and acquisition, video control instructions (pause and resume, etc.)
(I)MediaController Architecture Diagram


(1) Media service status and data reporting
After the Client SDK establishes a connection with the robot media service program, the media service program will continuously push real-time robot media status information to the Client, including:
- Continuous acquisition of internal audio and video data streams (only Bumi models without computing board can acquire internal video streams)
- Partial agent configuration reporting (actively obtained for the first time, and reported after subsequent changes)
- Acquisition of desensitized processed video stream (the version equipped with a computing board needs to continuously output external video streams to the audio-visual interaction system)
(2) Delivery of media data streams and configuration data
- Publish external audio stream data to the agent as input, or send it to the internal speaker for output
- Publish external video stream data for agent recognition
- Modify some Agent configuration settings, including volume, timeout, wake word, wake-up response, sleep response, and the data stream routing switch.
Currently, the input and output data collected for information acquisition only support the following specifications: channels= 8, sample_rate= 16000, format= 2 (16-bit little-endian)

Currently, the playback information input and output data only supports the following specifications: channels= 2, sample_rate= 16000, format= 2 (16-bit little-endian)

The captured data and playback data are interleaved.
(II)MediaController Startup Flowchart

III)Common Parameters of MediaController
DDS Packet Header Structure
| Parameter Name | Define Prototype | Member | Description |
|---|---|---|---|
| Header | | message_id | DDS Message ID |
| timestamp_us | us timestamp | ||
| sn | The current SN number of the Bumi device |
System Status Control
| SystemControlType | |
|---|---|
| enumeration | Description |
| TO_WAKEUP | Wake up the audio and video interaction system agent |
| TO_SLEEP | Dormant audio-video interaction system agent |
| TO_RESET | Restart the audio and video interaction system agent |
System Status
| Parameter Name | Define Prototype | Member | Description |
|---|---|---|---|
| SystemStatus | | header | DDS Header |
| value | agent status | ||
| reason | Status Reason |
| WorkStatus | |
|---|---|
| enumeration | Description |
| READY | Ready State |
| SLEEPED | Sleep state |
| WAKEUPED | Awake State |
| EXIT | Exit Status |
| StatusChangeReason | |
|---|---|
| Enumeration | Description |
| SYSTEM_LAUNCH | System Startup |
| CMD_RESET | Command Reboot |
| AUDIO_WAKEUPED | Voice Wake-up |
| CMD_WAKEUPED | Instruction Wake-up |
| AUDIO_SLEEPED | Voice Sleep |
| CMD_SLEEPED | Instruction Sleep |
| TIMEOUT_SLEEPED | Timeout Hibernation |
| ERROR_SLEEPED | False Hibernation |
Voice wake-up refers to triggering through actual human-computer voice interaction; command wake-up refers to triggering through DDS or other forms of commands;
System Error
| Parameter Name | Define Prototype | Member | Description |
|---|---|---|---|
| SystemError | | header | DDS Header |
| code | Error Code | ||
| message | Error Message |
Audio stream data
| Parameter Name | Define Prototype | Member | Description |
|---|---|---|---|
| AudioStream | | header | DDS Header |
| timestamp_us | Timestamp (us) | ||
| channels | Number of channels | ||
| sample_rate | sampling rate | ||
| format | Data Format | ||
| duration_ms | Time (ms)) | ||
| audio_data | audio data |
video stream data
| Parameter Name | Define Prototype | Example Value | Description |
|---|---|---|---|
| VideoStream | | header | DDS Header |
| timestamp_us | Timestamp (us) | ||
| format | Format: 1: YUYV Currently only the YUYV format is supported | ||
| width | Horizontal Pixels per Frame | ||
| height | Vertical pixels per frame | ||
| fps | frame rate | ||
| video_data | frame image data |
(IV)Common Interfaces of MediaController
| MediaController Interface | ||
|---|---|---|
| Interface Name | Interface Description | return type |
| Instance | Obtain the singleton pointer of MediaController | Object pointer |
| init | MediaController Object Initialization | bool |
| set_system_control | Set the control status of the voice interaction system | None |
| subscribe_system_status | Subscribe to status information of the audio-visual interactive system | None |
| subscribe_system_error | Subscribe to error status information of audio-visual interaction system | None |
| get_volume | Get the current volume of the audio-visual interaction system Range: 0–200 | int |
| set_volume | Set the volume of the current audio-visual interaction system Range: 0–200 | None |
| get_timeout | Get the inactivity timeout before entering sleep mode | int |
| set_timeout | Set the inactivity timeout before entering sleep mode | None |
| get_audio_cue_enable | get the enabled status of the prompt tone of the audio-visual interaction system | bool |
| set_audio_cue_enable | Set whether the prompt sound of the audio-visual interaction system is enabled | None |
| get_internal_capture_audio_data_to_agent_enable | Get the internal microphone audio input status for the Agent | bool |
| set_internal_capture_audio_data_to_agent_enable | Set the internal microphone audio input status for the Agent | None |
| get_external_custom_audio_data_to_agent_enable | Get the enabled status of external audio stream input for the agent | bool |
| set_external_custom_audio_data_to_agent_enable | Set the enabled status of external audio stream input for the agent | None |
| get_internal_agent_audio_data_to_playback_enable | Get the enabled status of agent audio stream output to the internal speaker. | bool |
| set_internal_agent_audio_data_to_playback_enable | Set the enabled status of agent audio stream output to the internal speaker. | None |
| get_external_custom_audio_data_to_playback_enable | Get the enabled status of external audio stream output to the internal speaker. | bool |
| set_external_custom_audio_data_to_playback_enable | Get the enabled status of external audio stream output to the internal speaker. | None |
| get_internal_capture_video_data_to_agent_enable | Get the enabled status of internal video stream input to the agent. | bool |
| set_internal_capture_video_data_to_agent_enable | Set the enabled status of internal video stream input to the agent. | None |
| get_external_custom_video_data_to_agent_enable | Get the enabled status of external video stream input to the agent. | bool |
| set_external_custom_video_data_to_agent_enable | Set the enabled status of external video stream input to the agent. | None |
| get_external_custom_audio_data_to_agent_use_internal_3a | Get the enabled status of using the internal 3A algorithm for external audio stream input to the agent. | bool |
| set_external_custom_audio_data_to_agent_use_internal_3a | Set the enabled status of using the internal 3A algorithm for external audio stream input to the agent. | None |
| get_wakeup_response_words | Get wake-up response words | string |
| set_wakeup_response_words | Set wake-up response words | None |
| get_sleep_response_words | Get dormant response words | string |
| set_sleep_response_words | Set sleep response words | None |
| get_wakeup_words | Get all wake words | string |
| add_wakeup_words | Add wake-up words | bool |
| reset_wakeup_words | Reset factory default wake-up word | None |
| subscribe_internal_audio_capture | Subscribe to the audio stream captured by the internal microphone | None |
| subscribe_internal_audio_playback | Subscribe to the audio stream played by the robot's internal speaker | None |
| subscribe_internal_video_capture | Subscribe to the video stream collected by the camera inside the robot (only available for Bumi without computing power board) | VideoStream struct |
| subscribe_internal_video_desensed | Subscribe to the audio-visual interaction system. The processed desensitized video stream (the Bumi equipped with a computing power board needs to continuously publish the availability of the external video stream) | VideoStream struct |
| pause_video_capture | Pause the internal camera Data Acquisition of the robot (only available for Bumi without computing power board) | None |
| resume_video_capture | Resume the internal camera Data Acquisition of the robot (only available for Bumi without computing power board) | None |
| publish_external_video_stream | Publish external video streams to the audio-visual interaction system | None |
| publish_external_audio_stream | Publish external audio stream input to the agent | None |
| publish_external_audio_playback_stream | Publish external audio stream output to the internal speaker | None |
| pause_audio_capture | Pause internal microphone audio stream capture | None |
| resume_audio_capture | Resume internal microphone audio stream capture | None |
| pause_audio_playback | Pause the audio stream output of the internal speaker | None |
| resume_audio_playback | Resume the audio stream output of the internal speaker | None |
All Set Agent configuration APIs (e.g., changing the volume, changing data switches, etc.) must not be called more frequently than once every 500 ms; otherwise, the voice interaction system may crash. If this issue occurs, please contact Noetix After-sales Service.
Get the singleton of the MediaController object
| Function Name | instance |
|---|---|
| Function Prototype | static MediaController *Instance() |
| Function Overview | Get the MediaController singleton |
| Parameters | None |
| Return Value | Singleton pointer to the MediaController object |
| Note |
C++ Example:
MediaController *ctrl = MediaController::Instance();
Python Example:
ctrl = MediaController.instance()
MediaController Object Initialization
| Function Name | init |
|---|---|
| Function Prototype | bool init() |
| Function Overview | MediaController Object Initialization |
| Parameters | None |
| Return Value | bool |
| Note |
C++ Example::
ctrl->init();
Python Example:
ctrl.init()
Set the status of the audio-visual interaction system
| Function Name | set_system_control |
|---|---|
| Function Prototype | void set_system_control(media::SystemControlType type, bool is_need_reply); |
| Function Overview | Set the status of the audio-video interaction system (wake-up, sleep, restart) |
| Parameters | - type:System Status - is_need_reply:whether to trigger the wake-up/sleep reply words |
| Return Value | None |
| Note |
C++ Example:
ctrl->set_system_control(media::SystemControlType::TO_WAKEUP, true);
Python Example:
ctrl.set_system_control(media::SystemControlType::TO_WAKEUP, true)
Subscribe to the status information of the audio-visual interaction system
| Function Name | subscribe_system_status |
|---|---|
| Function Prototype | void subscribe_system_status(SystemstatusCallback callback); |
| Function Overview | Subscribe to status information of the audio-visual interactive system |
| Parameters | - callback:The callback function that needs to be triggered when the state changes |
| Return Value | None |
| Note |
C++ Example:
ctrl->subscribe_system_status([&](const media::SystemStatus &status){
printf("status is %d \r\n" , static_cast<int>(status.value));
});
Python Example:
def on_status_switch(status):
print(status.value)
ctrl.subscribe_system_status(on_status_switch)
Subscribe to system error messages of the audio-visual interaction system
| Function Name | subscribe_system_error |
|---|---|
| Function Prototype | void subscribe_system_error(SystemerrorCallback callback); |
| Function Overview | Subscribe to system error messages of the audio-visual interaction system |
| Parameters | - callback: the callback triggered by a DDS message containing error information |
| Return Value | None |
| Note |
C++ Example:
ctrl->subscribe_system_error([&](const media::SystemError &error){
printf("status is %s \r\n", error.message.c_str());
});
Python Example:
def on_error(error):
print(error.message)
ctrl.subscribe_system_error(on_error)
Get the current volume
| Function Name | get_volume |
|---|---|
| Function Prototype | int get_volume() |
| Function Overview | Get the current volume of the audio-visual interaction system |
| Parameters | None |
| Return Value | int, volume value, range: 0~200 |
| Note |
C++ Example:
int volume = ctrl->get_volume();
std::cout << volume << std::endl;
Python Example:
volume = ctrl.get_volume()
print(volume)
Set the current volume
| Function Name | set_volume |
|---|---|
| Function Prototype | void set_volume(int volume) |
| Function Overview | Set the volume of the current audio-visual interaction system |
| Parameters | volume: volume value, range: 0~200 |
| Return Value | None |
| Note |
C++ Example:
ctrl->set_volume(100);
Python Example:
ctrl.set_volume(100)
Get the inactivity timeout before entering sleep
| Function Name | get_timeout |
|---|---|
| Function Prototype | int get_timeout() |
| Function Overview | Get the inactivity timeout before entering sleep |
| Parameters | None |
| Return Value | int, timeout (ms) |
| Note |
C++ Example:
int timeout = ctrl->get_timeout();
std::cout << timeout << std::endl;
Python Example:
timeout = ctrl.get_timeout()
print(timeout)
Set the timeout for entering sleep mode when there is no response
| Function Name | set_timeout |
|---|---|
| Function Prototype | void set_timeout(int timeout) |
| Function Overview | Set the timeout for entering sleep mode when there is no response |
| Parameters | timeout: Timeout duration (in ms) |
| Return Value | None |
| Note |
C++ Example:
ctrl->set_timeout(3000);
Python Example:
ctrl.set_timeout(3000)
Check whether the prompt tone is enabled
| Function Name | get_audio_cue_enable |
|---|---|
| Function Prototype | bool get_audio_cue_enable() |
| Function Overview | Obtain the enabled status of the prompt tone of the audio and video interaction system |
| Parameters | None |
| Return Value | bool |
| Note |
C++ Example:
bool enabled = ctrl->get_audio_cue_enable();
std::cout << (enabled ? "true" : "false") << std::endl;
Python Example:
enabled = ctrl.get_audio_cue_enable()
print(enabled)
Set whether the prompt tone is enabled
| Function Name | set_audio_cue_enable |
|---|---|
| Function Prototype | void set_audio_cue_enable(bool enable) |
| Function Overview | Set whether the prompt sound of the audio-visual interaction system is enabled |
| Parameters | Enable: true on, false off |
| Return Value | None |
| Note |
C++ Example:
ctrl->set_audio_cue_enable(true);
Python Example:
ctrl.set_audio_cue_enable(True)
Get the enabled status of internal microphone audio stream input to the agent.
| Function Name | get_internal_capture_audio_data_to_agent_enable |
|---|---|
| Function Prototype | bool get_internal_capture_audio_data_to_agent_enable() |
| Function Overview | Get the enabled status of internal microphone audio stream input to the agent. |
| Parameters | None |
| Return Value | bool |
| Note |
C++ Example:
bool enabled = ctrl->get_internal_capture_audio_data_to_agent_enable();
std::cout << (enabled ? "true" : "false") << std::endl;
Python Example:
enabled = ctrl.get_internal_capture_audio_data_to_agent_enable()
print(enabled)
Set the enabled status of internal microphone audio stream input to the agent.
| Function Name | set_internal_capture_audio_data_to_agent_enable |
|---|---|
| Function Prototype | void set_internal_capture_audio_data_to_agent_enable(bool enable) |
| Function Overview | Set whether the internal microphone is enabled to collect audio stream and input it to the agent |
| Parameters | Enable: true on, false off |
| Return Value | None |
| Note |
C++ Example:
ctrl->set_internal_capture_audio_data_to_agent_enable(true);
Python Example:
ctrl.set_internal_capture_audio_data_to_agent_enable(True)
Get the enabled status of external audio stream input to the agent.
| Function Name | get_external_custom_audio_data_to_agent_enable |
|---|---|
| Function Prototype | bool get_external_custom_audio_data_to_agent_enable() |
| Function Overview | Get the enabled status of external audio stream input for the agent |
| Parameters | None |
| Return Value | bool: true indicates enabled, false indicates disabled |
| Note |
C++ Example:
bool enabled = ctrl->get_external_custom_audio_data_to_agent_enable();
std::cout << (enabled ? "true" : "false") << std::endl;
Python Example:
enabled = ctrl.get_external_custom_audio_data_to_agent_enable()
print(enabled)
Set the enabled status of external audio stream input to the agent.
| Function Name | set_external_custom_audio_data_to_agent_enable |
|---|---|
| Function Prototype | void set_external_custom_audio_data_to_agent_enable(bool enable) |
| Function Overview | Set whether the external audio stream input to the agent is enabled |
| Parameters | Enable: true on, false off |
| Return Value | None |
| Note |
C++ Example:
ctrl->set_external_custom_audio_data_to_agent_enable(true);
Python Example:
ctrl.set_external_custom_audio_data_to_agent_enable(True)
Get the enabled status of agent audio stream output to the internal speaker.
| Function Name | get_internal_agent_audio_data_to_playback_enable |
|---|---|
| Function Prototype | bool get_internal_agent_audio_data_to_playback_enable() |
| Function Overview | Get whether the agent audio stream output to the internal speaker is enabled |
| Parameters | None |
| Return Value | bool: true indicates enabled, false indicates disabled |
| Note |
C++ Example:
bool enabled = ctrl->get_internal_agent_audio_data_to_playback_enable();
std::cout << (enabled ? "true" : "false") << std::endl;
Python Example:
enabled = ctrl.get_internal_agent_audio_data_to_playback_enable()
print(enabled)
Set the enabled status of agent audio stream output to the internal speaker.
| Function Name | set_internal_agent_audio_data_to_playback_enable |
|---|---|
| Function Prototype | void set_internal_agent_audio_data_to_playback_enable(bool enable) |
| Function Overview | Set whether the agent audio stream output to the internal speaker is enabled |
| Parameters | Enable: true on, false off |
| Return Value | None |
| Note |
C++ Example:
ctrl->set_internal_agent_audio_data_to_playback_enable(true);
Python Example:
ctrl.set_internal_agent_audio_data_to_playback_enable(True)
Get whether the external audio stream output to the internal speaker is enabled
| Function Name | get_external_custom_audio_data_to_playback_enable |
|---|---|
| Function Prototype | bool get_external_custom_audio_data_to_playback_enable() |
| Function Overview | Get the enabled status of outputting external audio stream to the internal speaker |
| Parameters | None |
| Return Value | bool type: true to enable, false to disable |
| Note |
C++ Example:
bool enabled = ctrl->get_external_custom_audio_data_to_playback_enable();
std::cout << (enabled ? "true" : "false") << std::endl;
Python Example:
enabled = ctrl.get_external_custom_audio_data_to_playback_enable()
print(enabled)
Set the enabled status of external audio stream output to the internal speaker.
| Function Name | set_external_custom_audio_data_to_playback_enable |
|---|---|
| Function Prototype | void set_external_custom_audio_data_to_playback_enable(bool enable) |
| Function Overview | Set whether the external audio stream is output to the internal speaker |
| Parameters | Enable: true on, false off |
| Return Value | None |
| Note |
C++ Example:
ctrl->set_external_custom_audio_data_to_playback_enable(true);
Python Example:
ctrl.set_external_custom_audio_data_to_playback_enable(True)
Get the enabled status of internal video stream input to the agent.
| Function Name | get_internal_capture_video_data_to_agent_enable |
|---|---|
| Function Prototype | bool get_internal_capture_video_data_to_agent_enable() |
| Function Overview | Obtain the enabled status of the internal video stream input to the agent |
| Parameters | None |
| Return Value | bool: true indicates enabled, false indicates disabled |
| Note |
C++ Example:
bool enabled = ctrl->get_internal_capture_video_data_to_agent_enable();
std::cout << (enabled ? "true" : "false") << std::endl;
Python Example:
enabled = ctrl.get_internal_capture_video_data_to_agent_enable()
print(enabled)
Set the enabled status of internal video stream input to the agent.
| Function Name | set_internal_capture_video_data_to_agent_enable |
|---|---|
| Function Prototype | void set_internal_capture_video_data_to_agent_enable(bool enable) |
| Function Overview | Set whether to enable the internal video stream input for the agent |
| Parameters | Enable: true on, false off |
| Return Value | None |
| Note |
C++ Example:
ctrl->set_internal_capture_video_data_to_agent_enable(true);
Python Example:
ctrl.set_internal_capture_video_data_to_agent_enable(True)
Get the enabled status of external video stream input to the agent.
| Function Name | get_external_custom_video_data_to_agent_enable |
|---|---|
| Function Prototype | bool get_external_custom_video_data_to_agent_enable() |
| Function Overview | Obtain the enabled status of the external video stream input to the agent |
| Parameters | None |
| Return Value | bool: true indicates enabled, false indicates disabled |
| Note |
C++ Example:
bool enabled = ctrl->get_external_custom_video_data_to_agent_enable();
std::cout << (enabled ? "true" : "false") << std::endl;
Python Example:
enabled = ctrl.get_external_custom_video_data_to_agent_enable()
print(enabled)
Set the enabled status of external video stream input to the agent.
| Function Name | set_external_custom_video_data_to_agent_enable |
|---|---|
| Function Prototype | void set_external_custom_video_data_to_agent_enable(bool enable) |
| Function Overview | Set whether the external video stream input to the agent is enabled |
| Parameters | Enable: true on, false off |
| Return Value | None |
| Note |
C++ Example:
ctrl->set_external_custom_video_data_to_agent_enable(true);
Python Example:
ctrl.set_external_custom_video_data_to_agent_enable(True)
Get the enabled status of using the internal 3A algorithm for external audio stream input to the agent.
| Function Name | get_external_custom_audio_data_to_agent_use_internal_3a |
|---|---|
| Function Prototype | bool get_external_custom_audio_data_to_agent_use_internal_3a() |
| Function Overview | Obtain the status of whether the agent enables the internal 3A algorithm when receiving external audio stream input |
| Parameters | None |
| Return Value | bool: true indicates enabled, false indicates disabled |
| Note |
C++ Example:
bool enabled = ctrl->get_external_custom_audio_data_to_agent_use_internal_3a();
std::cout << (enabled ? "true" : "false") << std::endl;
Python Example:
enabled = ctrl.get_external_custom_audio_data_to_agent_use_internal_3a()
print(enabled)
Set the enabled status of using the internal 3A algorithm for external audio stream input to the agent.
| Function Name | set_external_custom_audio_data_to_agent_use_internal_3a |
|---|---|
| Function Prototype | void set_external_custom_audio_data_to_agent_use_internal_3a(bool enable) |
| Function Overview | Set whether the external audio stream input to the agent uses the internal 3A algorithm |
| Parameters | Enable: true on, false off |
| Return Value | None |
| Note |
C++ Example:
ctrl->set_external_custom_audio_data_to_agent_use_internal_3a(true);
Python Example:
ctrl.set_external_custom_audio_data_to_agent_use_internal_3a(True)
Get wake-up response words
| Function Name | get_wakeup_response_words |
|---|---|
| Function Prototype | string get_wakeup_response_words() |
| Function Overview | Get wake-up response words |
| Parameters | None |
| Return Value | string, the current wake-up reply word |
| Note |
C++ Example:
std::string words = ctrl->get_wakeup_response_words();
std::cout << words << std::endl;
Python Example:
words = ctrl.get_wakeup_response_words()
print(words)
Set wake-up response words
| Function Name | set_wakeup_response_words |
|---|---|
| Function Prototype | void set_wakeup_response_words(string words) |
| Function Overview | Set wake-up response words |
| Parameters | Words: Wake up the content of the reply message |
| Return Value | None |
| Note |
C++ Example:
ctrl->set_wakeup_response_words("我在");
Python Example:
ctrl.set_wakeup_response_words("我在")
Get dormant response words
| Function Name | get_sleep_response_words |
|---|---|
| Function Prototype | string get_sleep_response_words() |
| Function Overview | Get dormant response words |
| Parameters | None |
| Return Value | string,the current dormant response word |
| Note |
C++ Example:
std::string words = ctrl->get_sleep_response_words();
std::cout << words << std::endl;
Python Example:
words = ctrl.get_sleep_response_words()
print(words)
Set sleep response words
| Function Name | set_sleep_response_words |
|---|---|
| Function Prototype | void set_sleep_response_words(string words) |
| Function Overview | Set up auto-reply messages for idle status |
| Parameters | words: content of the auto-reply for inactivity |
| Return Value | None |
| Note |
C++ Example:
ctrl->set_sleep_response_words("Okay, I'm going to rest first.");
Python Example:
ctrl.set_sleep_response_words("Okay, I'm going to rest first.")
Get wake-up words
| Function Name | get_wakeup_words |
|---|---|
| Function Prototype | string get_wakeup_words() |
| Function Overview | Get all wake-up words |
| Parameters | None |
| Return Value | string,all current wake words |
| Note |
C++ Example:
std::string words = ctrl->get_wakeup_words();
std::cout << words << std::endl;
Python Example:
words = ctrl.get_wakeup_words()
print(words)
Add wake-up words
| Function Name | add_wakeup_words |
|---|---|
| Function Prototype | bool add_wakeup_words(const std::string &words); |
| Function Overview | Add a new wake word |
| Parameters | - words: newly-wake word string |
| Return Value | bool:whether the addition via internal Regular Expression succeeded |
| Note | Regular Expression:(^(((([a-z]+) )?([^\s]+) ))+@(.+)$) [Pinyin (including initials and finals] @Corresponding nameExample: b ù m ī @Bu Mi |
C++ Example:
if (ctrl->add_wakeup_words("b ù m ī @Bu Mi"))
std::cout << "Success" << std::endl;
else
std::cout << "Failed" << std::endl;
Python Example:
if ctrl.add_wakeup_words("b ù m ī @Bu Mi"):
print("Success")
else:
print("Failed")
Reset Wake-up words
| Function Name | reset_wakeup_words |
|---|---|
| Function Prototype | void reset_wakeup_words(); |
| Function Overview | Reset default factory wake-up words |
| Parameters | None |
| Return Value | None |
| Note |
C++ Example:
ctrl->reset_wakeup_words();
Python Example:
ctrl.reset_wakeup_words()
Subscribe to the audio stream collected by the robot's internal microphone
| Function Name | subscribe_internal_audio_capture |
|---|---|
| Function Prototype | void subscribe_internal_audio_capture(AudiostreamCallback callback); |
| Function Overview | Subscribe to the audio stream collected by the robot's internal microphone |
| Parameters | - callback:callback function for processing audio stream data |
| Return Value | None |
| Note | The collected data is 8-channel data: the first four channels correspond to the four microphones mounted on the top of Bumi, the fifth and sixth channels are reserved channels, and the seventh and eighth channels are for loudspeaker echo acquisition. |
C++ Example:
ctrl->subscribe_internal_audio_capture(
[&](const media::AudioStream &stream) {
std::lock_guard<std::mutex> lk(mtx);
if (channels == 0) {
channels = stream.channels;
sample_rate = stream.sample_rate;
}
pcm.insert(pcm.end(), stream.audio_data.begin(),
stream.audio_data.end());
});
Python Example:
def on_audio(stream):
with lock:
if channels_ref[0] == 0:
channels_ref[0] = stream.channels
sample_rate_ref[0] = stream.sample_rate
pcm_all.extend(stream.audio_data)
ctrl.subscribe_internal_audio_capture(on_audio)
Subscribe to the audio stream played by the robot's internal speaker
| Function Name | subscribe_internal_audio_playback |
|---|---|
| Function Prototype | void subscribe_internal_audio_playback(AudiostreamCallback callback); |
| Function Overview | Subscribe to the audio stream played by the robot's internal speaker |
| Parameters | - callback:callback function for processing audio stream data |
| Return Value | None |
| Note | The collected data is 2-channel data from the Bumi ontology speaker. |
C++ Example:
ctrl->subscribe_internal_audio_playback(
[&](const media::AudioStream &stream) {
std::lock_guard<std::mutex> lk(mtx);
if (channels == 0) {
channels = stream.channels;
sample_rate = stream.sample_rate;
}
pcm.insert(pcm.end(), stream.audio_data.begin(),
stream.audio_data.end());
});
Python Example:
def on_audio(stream):
with lock:
if channels_ref[0] == 0:
channels_ref[0] = stream.channels
sample_rate_ref[0] = stream.sample_rate
pcm_all.extend(stream.audio_data)
ctrl.subscribe_internal_audio_playback(on_audio)
Subscribe to the video stream collected by the camera in the robot (only available for Bumi without computing board)
| Function Name | subscribe_internal_video_capture |
|---|---|
| Function Prototype | void subscribe_internal_video_capture(VideostreamCallback callback); |
| Function Overview | Subscribe to the video stream collected by the camera in the robot |
| Parameters | - callback:Callback function for processing video stream data |
| Return Value | None |
| Note | Only Bumi models without computing power boards are available |
C++ Example:
ctrl->subscribe_internal_video_capture(
[&](const media::VideoStream &stream) {
printf(
"[DEBUG] video frame: %ux%u format=%u data=%zu bytes\n",
stream.width, stream.height, stream.format,
stream.video_data.size());
if (got.exchange(true))
return;
save_video_frame("internal_video_capture", stream);
});
Python Example:
def on_video(stream):
if got.is_set():
return
result[0] = stream
got.set()
ctrl.subscribe_internal_video_capture(on_video)
Subscribe to the desensitized video stream processed by the audio-visual interaction system
| Function Name | subscribe_internal_video_desensed |
|---|---|
| Function Prototype | void subscribe_internal_video_desensed(VideostreamCallback callback); |
| Function Overview | Subscribe to the desensitized video stream processed by the audio-visual interaction system (Bumi equipped with a computing board needs to continuously publish the availability of external video streams) |
| Parameters | - callback:Callback function for processing video stream data |
| Return Value | None |
| Note | Bumi equipped with a computing board needs to continuously stream external video first before it can be used |
C++ Example:
ctrl->subscribe_internal_video_desensed(
[&](const media::VideoStream &stream) {
printf(
"[DEBUG] video frame: %ux%u format=%u data=%zu bytes\n",
stream.width, stream.height, stream.format,
stream.video_data.size());
if (got.exchange(true))
return;
save_video_frame("internal_video_desensed", stream);
});
Python Example:
def on_video(stream):
if got.is_set():
return
result[0] = stream
got.set()
ctrl.subscribe_internal_video_desensed(on_video)
Suspend the internal camera data acquisition of the robot
| Function Name | pause_video_capture |
|---|---|
| Function Prototype | void pause_video_capture() |
| Function Overview | Pause the internal camera Data Acquisition of the robot (only applicable to Bumi models without a computing power board) |
| Parameters | None |
| Return Value | None |
| Note | Only Bumi models without computing boards are available |
C++ Example:
ctrl->pause_video_capture();
Python Example:
ctrl.pause_video_capture()
Resume internal camera Data Acquisition of the robot
| Function Name | resume_video_capture |
|---|---|
| Function Prototype | void resume_video_capture() |
| Function Overview | Resume the internal camera Data Acquisition of the robot (only applicable to Bumi without a computing board) |
| Parameters | None |
| Return Value | None |
| Note | Only Bumi models without computing boards are available |
C++ Example:
ctrl->resume_video_capture();
Python Example:
ctrl.resume_video_capture()
Publish external video streams to the Audio-visual interaction system
| Function Name | publish_external_video_stream |
|---|---|
| Function Prototype | void publish_external_video_stream(VideoStream stream) |
| Function Overview | Publish external video streams to the audio-visual interaction system |
| Parameters | stream: VideoStream struct, containing video frame data |
| Return Value | None |
| Note |
C++ Example:
VideoStream stream;
// Populate stream data...
ctrl->publish_external_video_stream(stream);
Python Example:
stream = VideoStream()
#Populate stream data...
ctrl.publish_external_video_stream(stream)
Publish external audio stream input to the agent
| Function Name | publish_external_audio_stream |
|---|---|
| Function Prototype | void publish_external_audio_stream(AudioStream stream) |
| Function Overview | Publish external audio stream input to the agent |
| Parameters | stream: AudioStream structure, which contains audio data |
| Return Value | None |
| Note |
C++ Example:
AudioStream stream;
// Populate stream data...
ctrl->publish_external_audio_stream(stream);
Python Example:
stream = AudioStream()
# Populate stream data...
ctrl.publish_external_audio_stream(stream)
Publish external audio stream output to the internal speaker
| Function Name | publish_external_audio_playback_stream |
|---|---|
| Function Prototype | void publish_external_audio_playback_stream(AudioStream stream) |
| Function Overview | Publish external audio stream output to the internal speaker |
| Parameters | stream: AudioStream structure, which contains audio data |
| Return Value | None |
| Note |
C++ Example:
AudioStream stream;
// Populate stream data...
ctrl->publish_external_audio_playback_stream(stream);
Python Example:
stream = AudioStream()
# Populate stream data...
ctrl.publish_external_audio_playback_stream(stream)
Pause internal microphone audio stream capture
| Function Name | pause_audio_capture |
|---|---|
| Function Prototype | void pause_audio_capture() |
| Function Overview | Pause internal microphone audio stream capture |
| Parameters | None |
| Return Value | None |
| Note |
C++ Example:
ctrl->pause_audio_capture();
Python Example:
ctrl.pause_audio_capture()
Resume internal microphone audio stream capture
| Function Name | resume_audio_capture |
|---|---|
| Function Prototype | void resume_audio_capture() |
| Function Overview | Resume internal microphone audio stream capture |
| Parameters | None |
| Return Value | None |
| Note |
C++ Example:
ctrl->resume_audio_capture();
Python Example:
ctrl.resume_audio_capture()
Pause the audio stream output of the internal speaker
| Function Name | pause_audio_playback |
|---|---|
| Function Prototype | void pause_audio_playback() |
| Function Overview | Pause the audio stream output of the internal speaker |
| Parameters | None |
| Return Value | None |
| Note |
C++ Example:
ctrl->pause_audio_playback();
Python Example:
ctrl.pause_audio_playback()
Resume the audio stream output of the internal speaker
| Function Name | resume_audio_playback |
|---|---|
| Function Prototype | void resume_audio_playback() |
| Function Overview | Resume the audio stream output of the internal speaker |
| Parameters | None |
| Return Value | None |
| Note |
C++ Example:
ctrl->resume_audio_playback();
Python Example:
ctrl.resume_audio_playback()
(V)Demo Test Case
The SDK provides two complete sample programs with identical functions, which are implemented in C++ and Python respectively:
- C++:
examples/cpp/example_media.cppcompile and run
./build/example_media <command> [args]
- Python:
examples/py/test_media.pyrun
python3 test_media.py <command> [args]
All output files are saved by default in the out/ folder of the SDK root directory.
Test Case List
- Obtain all common_config configuration information
python3 examples/py/test_media.py
./build/example_media
- Subscribe to the internal microphone to capture a 10-second audio stream
python3 examples/py/test_media.py internal_audio_capture
./build/example_media internal_audio_capture
Subscribe to the internal microphone, capture audio for 10 seconds, and save it to out/internal_audio_capture.wav
- Record the audio stream played by the internal speaker
python3 examples/py/test_media.py internal_audio_playback
./build/example_media internal_audio_playback
Subscribe to the internal speaker, collect data for 10 seconds, and save to out/internal_audio_playback.wav
- Input the wav audio file to the audio-visual interaction system agent
python3 examples/py/test_media.py external_audio_agent <wav>
./build/example_media external_audio_agent <wav>
Read the wav file, and based on the status of the switches set_internal_capture_audio_data_to_agent_enable and set_external_custom_audio_data_to_agent_enable then TO_WAKEUP to wake up the agent, then send the audio in 20ms chunks via publish_external_audio_stream , and resume the switch status after completion
- Publish external audio stream output to the internal speaker
python3 examples/py/test_media.py external_audio_playback <wav>
./build/example_media external_audio_playback <wav>
Read the wav file, convert it to 2-channel interleaved format, split it into 20ms chunks, and transmit them via publish_external_audio_playback_stream for playback
- Subscribe to the internal video stream to capture a frame (for models without computing boards)
python3 examples/py/test_media.py internal_video_capture
./build/example_media internal_video_capture
Subscribe to the internal video stream and capture a frame to save it toout/internal_video_capture.png
- Subscribe to the internal desensitized video stream to capture a frame of image
python3 examples/py/test_media.py internal_video_desensed
./build/example_media internal_video_desensed
Subscribe to the internal desensitized video stream and capture one frame to save it to out/internal_video_desensed.png
Models equipped with computing boards need to first send the external video stream to the agent, as shown in the example below
- Publish external video stream input to the audio-visual interaction program agent
python3 examples/py/test_media.py external_video_publish
./build/example_media external_video_publish
ffmpeg collect and publish /dev/video4(640x480 yuyv422 @30fps),through publish_external_video_stream
(VIII) Camera Information
The camera hardware of the Bumi EDU version (BumiEDU_Pro, BumiEDU_MAX, BumiEDU_Tailor) is directly connected to the computing board, which can directly access the hardware interfaces on the computing board to obtain data.
Please refer to the following link for the Intel D435i camera routines: https://github.com/IntelRealSense/librealsense
For models with a computing board, if you want to use the camera, please close the official camera image transmission program first.
sudo systemctl stop noetix-video-capture.service # Disable image transmission service
sudo systemctl disable noetix-video-capture.service # Disable auto-start
sudo systemctl start noetix-video-capture.service # Enable image transmission service
sudo systemctl enable noetix-video-capture.service # Enable auto-start
(IX) Rapid Development
All the following operations are performed on the Bumi computing board. The computing board already has the required compilation environment installed; alternatively, you can deploy the compilation environment using Docker.
For the development and deployment environment of the external plugin, please refer to: SDK compilation environment deployment process
Introduction to External Plugin Development
Since the motion control board of humanoid products is not open to the public, user programs cannot be directly deployed and run on the motion control board, so an additional computing carrier is required to execute user programs.
If the model purchased by the user is not equipped with a computing board, the "external plugin development" method shall be adopted: the user shall prepare the computing board or host by themselves, and connect to the Bumi rear interface via a network cable to communicate with the motion control board.
Since the motion control board only allows communication with the fixed IP of the original computing board, and the communication is based on the wired connection of the internal switch, the external plugin device needs to be configured with the same IP as the original computing board to simulate the original computing board and complete the access.
Wired connection computing board SSH login
ssh noetix@192.168.55.101
Wireless SSH login to the computing board (the wireless IP can be viewed in the Wi-Fi connection status on the mobile App, which corresponds to the Wi-Fi module of the motion control board used by the computing board)
ssh -p 2222 noetix@[robotWIFI-IP]
(I)SDK Directory Structure
.
├── build.sh # Compilation script
├── CMakeLists.txt
├── config # configuration file
├── dds # DDS library files
├── examples # Example code
├── include
├── lib # Dynamic library files
├── LICENSE
├── policy # Model file
├── pybind # pybind file
└── README.md
(II)Docker Development Environment Image Deployment
Note: This image package is for the aarch64 architecture and can only be deployed on computers with the aarch64 architecture.
- Extract the Compressed Packet
tar -zxvf noetix_dds.tar.gz
After extraction, the Compressed Packet contains noetix_dds. tar development mirror file noetix_dds. yaml configuration deployment file
version: "3.8"
services:
noetix_dds:
image: noetix_dds:arm64
container_name: noetix_dds
network_mode: host
volumes:
- /home/noetix/work:/home/noetix/work:rw # Mount working directory
# /home/noetix/work The part before the colon should be changed to the user's working directory; create a directory named /home/noetix/work/.
stdin_open: true
tty: true
- Deployment
docker load -i noetix_dds.tar
docker images # Confirm that the noetix_dds arm64 image exists.
mkdir -p /home/noetix/work # Create a working directory; the user can choose it
docker compose -f noetix_dds.yaml up -d # deploy
docker ps # Check if noetix_dds exists.
- Enter the dock container
docker exec -it noetix_dds /bin/bash
You can start development by simply placing the DDS SDK in the working directory afterwards.
(III)Compile the SDK
cd noetix_sdk_bumi
./build.sh # One-click compilation script
After a successful compile, executable files will be generated in the root directory of the SDK: example_low andexample_high and example_policy_walkmeanwhile, the pybind libraries highcontrol_py 以及 lowcontrol_py will be generated in the build directory for Python to call.
(IV)Launch the Program
C++
cd noetix_sdk_bumi
sudo ./build/example_high #or sudo ./build/example_low
Python
cd noetix_sdk_bumi
python3 examples/py/test_low.py # python3 examples/py/test_high.py
During the debugging process, please use the robot hanger throughout the whole process to avoid losses caused by out-of-control faulty programs.
(X) Common Issues and Notes
- DDS Client Demo: Runtime Error (1)
ARM architecture:
error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
libssl1.1_1.1.1f-1ubuntu2.16_arm64.deb
sudo dpkg -i libssl1.1_1.1.1f-1ubuntu2.16_arm64.deb
Architecture:
error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
libssl1.1_1.1.1f-1ubuntu2_amd64.deb
sudo dpkg -i libssl1.1_1.1.1f-1ubuntu2_amd64.deb