The field of robot learning has made great advances in developing behaviour learning methodologies capable of learning policies for tasks ranging from manipulation to locomotion. However, the problem of combined learning of behaviour and robot structure, here called co-adaptation, is less studied. Most of the current co-adapting robot learning approaches rely on model-free algorithms or assume to have access to an a-priori known dynamics model, which requires considerable human engineering. In this work, we investigate the potential of combining model-free and model-based reinforcement learning algorithms for their application on co-adaptation problems with unknown dynamics functions. Classical model-based reinforcement learning is concerned with learning the forward dynamics of a specific agent or robot in its environment. However, in the case of jointly learning the behaviour and morphology of agents, each individual agent-design implies its own specific dynamics function. Here, the challenge is to learn a dynamics model capable of generalising between the different individual dynamics functions or designs. In other words, the learned dynamics model approximates a multi-dynamics function with the goal to generalise between different agent designs. We present a reinforcement learning algorithm that uses a learned multi-dynamics model for co-adapting robot’s behaviour and morphology using imagined rollouts. We show that using a multi-dynamics model for imagining transitions can lead to better performance for model-free co-adaptation, but open challenges remain.
@inproceedings{10.1007/978-3-031-53969-5_24,author={Sliacka, Maria and Mistry, Michael and Calandra, Roberto and Kyrki, Ville and Luck, Kevin Sebastian},editor={Nicosia, Giuseppe and Ojha, Varun and La Malfa, Emanuele and La Malfa, Gabriele and Pardalos, Panos M. and Umeton, Renato},title={Co-imagination of Behaviour and Morphology of Agents},booktitle={Machine Learning, Optimization, and Data Science},year={2024},publisher={Springer Nature Switzerland},address={Cham},pages={318--332},isbn={978-3-031-53969-5},}
@inproceedings{dunion2023conditional,title={Conditional Mutual Information for Disentangled Representations in Reinforcement Learning},author={Dunion, Mhairi and McInroe, Trevor and Luck, Kevin Sebastian and Hanna, Josiah P. and Albrecht, Stefano V},booktitle={Thirty-seventh Conference on Neural Information Processing Systems},year={2023},url={https://openreview.net/forum?id=EmYWJsyad4},}
@inproceedings{huang2023practical,title={Practical Equivariances via Relational Conditional Neural Processes},author={Huang, Daolang and Haussmann, Manuel and Remes, Ulpu and John, S. T. and Clart{\'e}, Gr{\'e}goire and Luck, Kevin Sebastian and Kaski, Samuel and Acerbi, Luigi},booktitle={Thirty-seventh Conference on Neural Information Processing Systems},year={2023},url={https://openreview.net/forum?id=xax5eWeObb},}
@inproceedings{rajani2023co,title={Co-imitation: learning design and behaviour by imitation},author={Rajani, Chang and Arndt, Karol and Blanco-Mulero, David and Luck, Kevin Sebastian and Kyrki, Ville},booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},volume={37},number={5},pages={6200--6208},year={2023},}
2022
Which language evolves between heterogeneous agents?-communicating movement instructions with widely different time scopes
Marie Ossenkopf , Kevin Sebastian Luck , and Kory Wallace Mathewson
In Emergent Communication Workshop at ICLR 2022 , 2022
@inproceedings{ossenkopf2022language,title={Which language evolves between heterogeneous agents?-communicating movement instructions with widely different time scopes},author={Ossenkopf, Marie and Luck, Kevin Sebastian and Mathewson, Kory Wallace},booktitle={Emergent Communication Workshop at ICLR 2022},year={2022},}
@article{davchev2022residual,title={Residual learning from demonstration: Adapting dmps for contact-rich manipulation},author={Davchev, Todor and Luck, Kevin Sebastian and Burke, Michael and Meier, Franziska and Schaal, Stefan and Ramamoorthy, Subramanian},journal={IEEE Robotics and Automation Letters},volume={7},number={2},pages={4488--4495},year={2022},publisher={IEEE},}
@inproceedings{dunion2022temporal,title={Temporal Disentanglement of Representations for Improved Generalisation in Reinforcement Learning},author={Dunion, Mhairi and McInroe, Trevor and Luck, Kevin Sebastian and Hanna, Josiah P and Albrecht, Stefano V},booktitle={The Eleventh International Conference on Learning Representations},year={2022},}
Accelerating Laboratory Automation Through Robot Skill Learning For Sample Scraping
Gabriella Pizzuto , Hetong Wang , Hatem Fakhruldeen , Bei Peng , Kevin Sebastian Luck , and Andrew I Cooper
@article{pizzuto2022accelerating,title={Accelerating Laboratory Automation Through Robot Skill Learning For Sample Scraping},author={Pizzuto, Gabriella and Wang, Hetong and Fakhruldeen, Hatem and Peng, Bei and Luck, Kevin Sebastian and Cooper, Andrew I},journal={arXiv preprint arXiv:2209.14875},year={2022},}
2021
What robot do i need? Fast co-adaptation of morphology and control using graph neural networks
Kevin Sebastian Luck , Roberto Calandra , and Michael Mistry
@article{luck2021robot,title={What robot do i need? Fast co-adaptation of morphology and control using graph neural networks},author={Luck, Kevin Sebastian and Calandra, Roberto and Mistry, Michael},journal={arXiv preprint arXiv:2111.02371},year={2021},}
Humans and animals are capable of quickly learning new behaviours to solve new tasks. Yet, we often forget that they also rely on a highly specialized morphology that co-adapted with motor control throughout thousands of years. Although compelling, the idea of co-adapting morphology and behaviours in robots is often unfeasible because of the long manufacturing times, and the need to redesign an appropriate controller for each morphology. In this paper, we propose a novel approach to automatically and efficiently co-adapt a robot morphology and its controller. Our approach is based on recent advances in deep reinforcement learning, and specifically the soft actor critic algorithm. Key to our approach is the possibility of leveraging previously tested morphologies and behaviors to estimate the performance of new candidate morphologies. As such, we can make full use of the information available for making more informed decisions, with the ultimate goal of achieving a more data-efficient co-adaptation (i.e., reducing the number of morphologies and behaviors tested). Simulated experiments show that our approach requires drastically less design prototypes to find good morphology-behaviour combinations, making this method particularly suitable for future co-adaptation of robot designs in the real world.
@inproceedings{pmlr-v100-luck20a,title={Data-efficient Co-Adaptation of Morphology and Behaviour with Deep Reinforcement Learning},author={Luck, Kevin Sebastian and Amor, Heni Ben and Calandra, Roberto},booktitle={Proceedings of the Conference on Robot Learning},pages={854--869},year={2020},editor={Kaelbling, Leslie Pack and Kragic, Danica and Sugiura, Komei},volume={100},series={Proceedings of Machine Learning Research},month={30 Oct--01 Nov},publisher={PMLR},url={https://proceedings.mlr.press/v100/luck20a.html},}
@inproceedings{luck2019improved,title={Improved exploration through latent trajectory optimization in deep deterministic policy gradient},author={Luck, Kevin Sebastian and Vecerik, Mel and Stepputtis, Simon and Amor, Heni Ben and Scholz, Jonathan},booktitle={2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},pages={3704--3711},year={2019},organization={IEEE},}
Bio-inspired robot design considering load-bearing and kinematic ontogeny of chelonioidea sea turtles
Andrew Jansen , Kevin Sebastian Luck , Joseph Campbell , Heni Ben Amor , and Daniel M Aukes
In Biomimetic and Biohybrid Systems: 6th International Conference, Living Machines 2017, Stanford, CA, USA, July 26–28, 2017, Proceedings 6 , 30 oct–01 nov 2017
@inproceedings{jansen2017bio,title={Bio-inspired robot design considering load-bearing and kinematic ontogeny of chelonioidea sea turtles},author={Jansen, Andrew and Luck, Kevin Sebastian and Campbell, Joseph and Amor, Heni Ben and Aukes, Daniel M},booktitle={Biomimetic and Biohybrid Systems: 6th International Conference, Living Machines 2017, Stanford, CA, USA, July 26--28, 2017, Proceedings 6},pages={216--229},year={2017},organization={Springer},}
@inproceedings{Luck-RSS-17,author={Luck, Kevin Sebastian and Campbell, Joseph and Jansen, Michael and Aukes, Daniel and Amor, Heni Ben},title={From the Lab to the Desert: Fast Prototyping and Learning of Robot Locomotion},booktitle={Proceedings of Robotics: Science and Systems},year={2017},address={Cambridge, Massachusetts},month=jul,doi={10.15607/RSS.2017.XIII.075},}
@inproceedings{luck2017extracting,title={Extracting bimanual synergies with reinforcement learning},author={Luck, Kevin Sebastian and Amor, Heni Ben},booktitle={2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},pages={4805--4812},year={2017},organization={IEEE},}
@inproceedings{luck2016sparse,title={Sparse latent space policy search},author={Luck, Kevin Sebastian and Pajarinen, Joni and Berger, Erik and Kyrki, Ville and Amor, Heni Ben},booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},volume={30},number={1},year={2016},}
@inproceedings{luck2014latent,title={Latent space policy search for robotics},author={Luck, Kevin Sebastian and Neumann, Gerhard and Berger, Erik and Peters, Jan and Amor, Heni Ben},booktitle={2014 IEEE/RSJ International Conference on Intelligent Robots and Systems},pages={1434--1440},year={2014},organization={IEEE},}
