GEM-4： Gemma Embodied 4 Physical Assistance

GEM-4: Gemma Embodied 4 Physical Assistance is a research prototype developed during a Google hackathon.

GEM-4 explores how a Gemma-based embodied AI system could support physical assistance through a wearable robotic arm. The system uses voice commands, wrist and chest cameras, and onboard edge AI inference on an NVIDIA Jetson AGX Orin to understand user intent and generate robot actions.

This demo introduces the hardware setup, basic object manipulation tasks, the Vision-Language-Action architecture, and a continuous learning workflow using our self-developed web applications, MimicAnno and MimicRec.

MimicAnno converts human first-person hand videos into robot learning data by using Gemma to infer tasks and subtasks, generate task annotations, and transform hand motion into robot action data.

MimicRec collects robot learning data from different cameras and robot platforms. It supports hand teaching, teleoperation, and human demonstration data collection, while recording camera streams, trajectories, joint angles, and gripper actions. Collected datasets can be uploaded seamlessly to Hugging Face for future fine-tuning.

Important Note:
This project is a research prototype. It is not a medical device or a certified accessibility assistive device, and we do not make any claims regarding clinical effectiveness. The system has not obtained regulatory approval or accessibility device certification.

This project is intended for research purposes only, to explore future possibilities of physical assistance for users with visual or upper-limb impairments. It is not positioned as a product for real-world deployment.

Tech Stack:
Gemma, NVIDIA Jetson AGX Orin, Raspberry Pi 5, reBot Arm, Intel RealSense, GoPro, VLA, Bridge Attention, MimicAnno, MimicRec, Hugging Face

#machine #Robotics #robot