Unitree G1 — Hua Wang

Task setup

Predefined grasping scene

Unitree G1 standing behind a table with a bottle on the left, a paper cup near the center, and a medicine box on the right — The predefined tabletop scene and its three target objects: a bottle, a medicine box, and a paper cup.

The minimal setup includes three preset target objects: a bottle, a medicine box, and a paper cup. The robot first moves to a fixed task location, then executes a single-arm grasping task.

The current prototype uses object detection with predefined grasp motions on a Unitree G1-EDU equipped with an Inspire Robots dexterous hand.

End effector

Inspire RH56E2 dexterous hand

Product image showing the Inspire Robots RH56E2 five-finger dexterous hand from both sides — Inspire Robots RH56E2, shown from both sides.

Five-finger end effector

The G1 prototype uses an RH56E2 five-finger dexterous hand from Inspire Robots. It serves as the end effector for the predefined grasping task.

Manufacturer’s product page

Dexterous-hand desktop control application with a hand visualization, motion presets, parameter settings, and feedback panels — Desktop control interface. Select the image for a closer look.

Desktop control software

The desktop application provides a direct interface for configuring finger motion and inspecting the hand’s feedback during testing.

RH56E2 finger-motion test, shown at its original speed.

Finger motion in action

This short hardware test shows the fingers moving from an open hand toward a closed grasp configuration.

View the G1 grasping demo

Text-prompted detection

Find the target with text

Robot head-camera RGB image with a YOLO-World bounding box labeled bottle 0.88 around a bottle on the table — Head-camera detections for the text prompts “bottle” (top) and “cup” (bottom). Labels show detector confidence scores.

Robot head-camera RGB image with a YOLO-World bounding box labeled cup 0.73 around a paper cup on the table — Head-camera detections for the text prompts “bottle” (top) and “cup” (bottom). Labels show detector confidence scores.

The robot’s head-mounted camera captures the RGB scene. Given an object name such as bottle or cup, YOLO-World locates the requested object and returns a 2D bounding box.

The center of this box provides the image-space location used for depth sampling and 3D target construction.

RGB-D sampling

Sample depth near the box center

Schematic showing a central sampling patch spanning 35 percent of the detected box width and height, with its median depth used as Z — Depth-sampling schematic. The value comes from a central patch of the head camera’s depth map.

Depth is measured by the robot’s head camera. Rather than relying on one center pixel, the system takes the median depth of a small central patch inside the detection box. This reduces sensitivity to isolated noisy depth values.

The patch width and height are each set to 35% of the detection box dimensions. When RGB and depth resolutions differ, the implementation scales the pixel coordinates and patch dimensions to the depth-map resolution before sampling.

Depth sampling details

The implementation uses separate horizontal and vertical scale factors. The patch is centered at the mapped box center; inner_fraction = 0.35 controls its width and height.

\begin{aligned} s_x &= \frac{W_{\mathrm{depth}}}{W_{\mathrm{rgb}}} \\[0.6em] s_y &= \frac{H_{\mathrm{depth}}}{H_{\mathrm{rgb}}} \\[0.9em] u_{\mathrm{d}} &= u_{\mathrm{center}} \cdot s_x \\[0.3em] v_{\mathrm{d}} &= v_{\mathrm{center}} \cdot s_y \\[0.9em] w_{\mathrm{patch}} &= (x_2 - x_1) \cdot s_x \cdot 0.35 \\[0.3em] h_{\mathrm{patch}} &= (y_2 - y_1) \cdot s_y \cdot 0.35 \end{aligned}

Coordinate conversion

Turn a pixel into a grasp target

Annotated head-camera image and robot scene showing the bottle’s pixel location, camera frame, waist frame, and camera-to-waist 3D point transformation — From image coordinates to camera and waist frames. Select the figure to inspect the coordinate labels at full size.

The detected 2D location, sampled depth, and camera intrinsics recover a 3D point in the camera frame.

Arm control needs this point in the robot’s coordinate frame. A camera-to-torso extrinsic transform combines rotation and translation to express the same physical point in G1’s torso_link (waist) frame.

Applying the palm offset then produces the right-arm target position used by the predefined grasp motion.

Engineering iteration

During grasp closure, the thumb could catch against the index finger and prevent the hand from fully closing. Adjusting the closing angles alone did not reliably resolve the interference.

Before · finger interference

The thumb and index finger catch against each other during closure.

After · revised timing

The revised sequence lets the hand close around the bottle in this demonstration.

Use a staged closing sequence

Finish the thumb’s rotation.
Close the other four fingers.
Wait 0.20 s, then bend the thumb.

A separate thumb-closing delay parameter makes the final step’s timing adjustable. Both videos are shown at their original speed.

Force feedback

Stop on contact, then grasp

A small detection error can make a fixed approach continue past the object’s surface, pushing it across or off the table. Contact feedback lets the hand stop the approach and begin grasping when contact is detected.

Four photographs of manual presses at different locations on the dexterous hand to check its response to applied force — Manual press tests check whether the hand’s tactile sensors respond to applied force. Select the image for a closer look.

Contact-triggered bottle grasping, shown at its original speed.

How the contact trigger works

Set a baseline. Record force and motor-current readings before approaching.
Monitor the changes. Poll the hand over Modbus and compare live readings with the baseline.
Enable the trigger late. Allow contact-triggered stopping only in the latter half of the approach to reduce early false triggers.
Stop, then grasp. When the feedback change exceeds the contact threshold, stop the approach and execute the grasping sequence.

Box placement

Locate the box and place the bottle

After grasping the bottle, the robot lifts its arm and places the bottle into a cardboard box. The destination is located from the head camera’s RGB and depth observations.

Three head-camera views: the RGB scene, a YOLO-World cardboard-box detection with its image center, and the depth map with a sampling region — Left to right: RGB observation, box detection, and depth localization. Select the image for a closer look.

Lifting the arm and placing the bottle in the box, shown at its original speed.

Locate the box

Capture the scene. Read an RGB frame from the robot’s head-mounted RealSense camera.
Find the box. YOLO-World detects cardboard box, returning its bounding box and center pixel.
Add depth. Use the detected region for depth sampling and 3D localization of the box.

Time optimization

Less waiting, faster placement

Repeated model loading and extra motion delays slowed down the placement routine. Keeping detection running and adjusting the motion sequence reduced the reported full-routine time by about 47%.

Vision loading + detection: 10 s → 2 s; Load the model once and reuse it.
Full placement routine: 46.68 s → 24.70 s; 21.98 s saved in the reported repeat test.

Before Load the model on every placement run

Start placement
Capture RGB-D
Load model
Detect box
Compute placement target

After Keep a separate detection service running

Start once · independent service Launch detector Load model & keep running

Image in · detection result out

Start placement
Capture RGB-D
Request detection
Compute placement target

The placement program sends an image to the running detector and uses the returned result to compute its target. Model loading is no longer repeated for each run.

Trim motion overhead

Shorten motion segments and remove extra pauses at waypoints.
Raise the return-motion speed limit from 0.35 to 0.45 rad/s.
After reset, run the status announcement and control release in parallel.

Keep the existing checks

Retain the obstacle-avoidance waypoints and the waits required for finger motion. Inverse-kinematics and reachability checks remain in place; the return still follows the transition points.

View the timing records and stage breakdown

The repeat-test slide reports a total of 24.70 s; the optimization log lists 24.60 s. The individual stage values below also sum to 24.60 s. Both source records are retained here; the headline uses the reported repeat-test total.

Optimization log
Configuration	Time (s)	Reduction
Initial routine	46.68	—
Persistent detector + initial speedup	32.53	30.3%
Remove pauses + faster return	24.60	47.3%

Stages with the 0.45 rad/s return limit
Stage	Time (s)
Startup + planning initialization	3.06
Box detection	0.95
Lift arm	3.90
Move to pre-placement + release points	5.01
Open hand + protective closure	3.39
Withdraw + return to transition point	4.01
Lower arm + reset	3.05
Release control + exit	1.23
Reported repeat-test total	24.70