QuadroBot // Robot "Spider"

I built a 4-legged spider robot that learns to walk using reinforcement learning. The physics were simulated in Godot 4.5, where a PPO neural network (23 inputs, 256×256 hidden layers, 12 outputs) was trained via Stable Baselines3 on a 50-core Ubuntu server running up to 50 parallel headless instances. The robot first learned to stand stably (Phase 0), then automatically transitioned to walking once the mean reward exceeded 6.5 (Phase 1). The simulated joints behaved like real servos, receiving target angles and moving at a realistic speed matching the actual hardware. The reward function encouraged upright posture, correct body height, foot contact, joint symmetry, smooth movements, and penalized drifting and jittery behavior. The trained model was exported as ONNX and deployed to an Arduino UNO Q with a PCA9685 servo controller driving 12 servos (3 per leg: hip horizontal, hip vertical, knee), using an LSM6DSOXTR IMU for real-time balance feedback.

At the moment, I cannot release the neural network because I am still learning how to do it in the best way. But for everyone who is interested, here is a nice graphic ;)

As you can see, I tried a lot of different things. The data shown here indicates that my networks always reached a point where they failed. On the y-axis, you can see the reward. If the reward gets close to 8, the robot is able to stand properly. From that point on, it would start training to walk. As you can see, I have not reached the point yet where it can walk. But standing up from different positions worked quite well. I cannot wait to dive deeper into that.

If you want to try your own stuff feel free to share it on makerworld. Also just one Tipp: Use a 9g Servo with metal gears ;)

Happy hacking!!!