Scaling RL to Large Musculoskeletal Systems

DEP-RL

Coordinated exploration enables scaling to many muscles.

Learning motion with reinforcement learning on large musculoskeletal models often lags behind its biological counterparts. Inefficient exploration in large action spaces is a major problem. We propose differential extrinsic plasticity (DEP) from self-organization for faster state-space exploration. Integrating DEP into RL leads to improved learning in grasping and locomotion.

Developing exploration methods and reinforcement learning (RL) algorithms capable of scaling to simulated systems with many muscles was a major challenge we addressed. Our method was first applied to an ostrich and a simple humanoid model.

Publications

DEP-RL: Embodied Exploration for Reinforcement Learning in Overactuated and Musculoskeletal Systems

Pierre Schumacher, Daniel F.B. Haeufle, Dieter Büchler, Syn Schmitt, Georg Martius

Proceedings of the Eleventh International Conference on Learning Representations (ICLR) · 2023