Robotoddler

Autonomous Robotic Assemblies

Started

June 1, 2023

Status

In Progress

Share this project

Industrial robots are widely used for repetitive automation tasks in many industries. For architecture however, the ever-changing form, environment or material of the constructions hinders their wide application, and existing use cases are mostly limited to linear design-plan-fabricate processes. Robotoddler investigates the use of reinforcement learning for the task of multi-robotic assembly of building elements into architectural structures. In a virtual simulation environment, one or multiple robots will learn to pick, move, place or hold different elements (building blocks) through trial- and-error and positive or negative reinforcement. This involves not only learning the robotic movements but also the design decisions, as the location, orientation and sequence of placement are part of the reinforcement learning (RL) parameters.

Once trained, the learned policy from the simulation RL model will be transferred to the real robot setup at the CRCL. A series of experiments with increasing complexity will be conducted, from one to two or more robots, from simple stacking to structures spanning between supports.

People

Collaborators

SDSC Team:

Paul Rolland

Senior Data Scientist

Paul joined the SCSC in July 2023. He received a PhD in Computer Science from EPFL, focusing on sampling and optimisation algorithms, Causality, robust Machine Learning and Reinforcement Learning. Prior to his PhD, he obtained a Bachelor degree in Physics and a Master in Computational Science and Engineering at EPFL. During his studies, he focused on the numerical integration of Ordinary and Stochastic differential equations, in addition to Machine Learning.

Paul Rolland

Johannes Kirschner

Senior Data Scientist

Johannes’ research focuses on developing practical andprincipled algorithms for sequential decision-making. His expertise spans awide range of topics from reinforcement learning theory and control, Bayesian optimization, safety and robustness to modern deep learning. He worked on challenging application domains, including deploying state-of-the-art data-driven optimization algorithms on two particle accelerators at the Paul Scherrer Institute. Before joining the SDSC in August 2023, Johannes was a postdoctoral researcher at the University of Alberta and completed an internship at Google DeepMind. Johannes earned his PhD in Computer Science in 2016 at ETH Zurich with Prof. Andreas Krause, and he holds a Master in Mathematics from ETH Zurich.

Johannes Kirschner

PI | Partners:

EPFL, Lab for Creative Computation:

Prof. Stefana Parascho

More info

description

Motivation

‍Construction and architectural design rely extensively on human expertise throughout planning and execution. Developing assistive, learning-based methods is challenging due to the rigid, multi-stage nature of the design process. End-to-end trained policies hold the promise to capture the entirety of the design and construction process, offering adaptability to unforeseen changes and discovering innovative designs. The main challenge is that the space of possible structures is enormous, and developing learning algorithms to navigate this space efficiently requires state-of-the-art reinforcement and planning methods and possibly novel solutions.

Proposed Approach / Solution

The SDSC partners are working on training reinforcement learning agents for construction tasks in a simulated environment. We investigated several distinct modelling approaches with the goal to master increasingly difficult construction tasks. Our current learning procedure, based on a visual perception of the scene, yields agents capable of building various kinds of structures (e.g., simple towers, bridges of various heights and widths, arches) with either one or two arms. As an intermediate step towards the sim-to-real transition, we introduced a constructibility constraint within the environment, in addition to the stability and collision constraint, to evaluate whether a robotic arm is able to place the desired block, with the goal of preventing the agent to design non-constructible structures. In the final step, we aim to deploy the trained policies on the robotic setup at the Lab for Creative Computation, EPFL, which will be done in collaboration with the project partners, which already have facilities for collaborative robotic construction (see Figure 1).

Impact

‍The outcomes of this project may lead to innovations in robotic construction, that allow for novel construction solutions and designs. We aim to demonstrate feasibility of learned policies for construction on the existing robotic platforms. We further expect to improve current reinforcement learning algorithms to handle the challenging construction domain.

***Figure 1:*** *Robotic Assembly at the Lab for Creative Computation, EPFL (Photo Credit: Stefana Parascho).*

Presentation

Download Presentation



Gallery

Annexe

Additional resources

—



Bibliography

Funk, N., Chalvatzaki, G., Belousov, B., & Peters, J. (2022, January). Learn2assemble with structured representations and search for robotic architectural construction. In Conference on Robot Learning (pp. 1401-1411). PMLR.
Bapst, V., Sanchez-Gonzalez, A., Doersch, C., Stachenfeld, K., Kohli, P., Battaglia, P., & Hamrick, J. (2019, May). Structured agents for physical construction. In International conference on machine learning (pp. 464-474). PMLR.
Vallat, G., Wang, J., Maddux, A., Kamgarpour, M., & Parascho, S. (2023, October). Reinforcement learning for scaffold-free construction of spanning structures. In Proceedings of the 8th ACM Symposium on Computational Fabrication (pp. 1-12).