Anchor-bolt insertion is a peg-in-hole task performed in the construction
field for holes in concrete. Efforts have been made to automate this task, but
the variable lighting and hole surface conditions, as well as the requirements
for short setup and task execution time make the automation challenging. In
this study, we introduce a vision and proprioceptive data-driven robot control
model for this task that is robust to challenging lighting and hole surface
conditions. This model consists of a spatial attention point network (SAP) and
a deep reinforcement learning (DRL) policy that are trained jointly end-to-end
to control the robot. The model is trained in an offline manner, with a
sample-efficient framework designed to reduce training time and minimize the
reality gap when transferring the model to the physical world. Through
evaluations with an industrial robot performing the task in 12 unknown holes,
starting from 16 different initial positions, and under three different
lighting conditions (two with misleading shadows), we demonstrate that SAP can
generate relevant attention points of the image even in challenging lighting
conditions. We also show that the proposed model enables task execution with
higher success rate and shorter task completion time than various baselines.
Due to the proposed model’s high effectiveness even in severe lighting, initial
positions, and hole conditions, and the offline training framework’s high
sample-efficiency and short training time, this approach can be easily applied
to construction.
Commentary:
In this study, the authors present a novel approach to automate a peg-in-hole task in the construction field using vision and proprioceptive data-driven robot control. This task involves inserting anchor bolts into holes in concrete, which is challenging due to variable lighting and hole surface conditions, as well as the need for short setup and task execution time. The authors propose a model that combines a spatial attention point network (SAP) and deep reinforcement learning (DRL) policy to control the robot.
The multi-disciplinary nature of this research is evident in the integration of computer vision, robotics, and machine learning techniques. By training the model in an offline manner, the authors aim to reduce training time and minimize the reality gap when transferring the model to the physical world. This offline training framework is designed to be sample-efficient, meaning it can utilize limited training data effectively.
One key contribution of this research is the SAP, which is able to generate relevant attention points even in challenging lighting conditions. This is crucial for successful task execution since the robot needs to accurately perceive the hole location. The authors demonstrate the effectiveness of their proposed model by evaluating its performance in various unknown hole configurations, starting from different initial positions, and under different lighting conditions including misleading shadows.
The results show that the proposed model outperforms various baselines in terms of both success rate and task completion time. This suggests that the model is robust and efficient in handling challenging real-world conditions.
The findings of this study have important implications for the construction industry. Automating tasks like anchor bolt insertion can improve efficiency, reduce human error, and enhance overall productivity on construction sites. By demonstrating the effectiveness of their approach in severe lighting conditions, initial positions, and hole conditions, the authors highlight the potential applicability of their model in real-world construction scenarios.
This study aligns with related research efforts on automating construction tasks using robotics and AI. Similar research has been conducted on tasks like bricklaying (Reference: https://ieeexplore.ieee.org/document/8803668) and rebar bending (Reference: https://www.sciencedirect.com/science/article/pii/S1877705815025686). These studies collectively contribute to the advancement of the construction industry through the implementation of cutting-edge technologies.
In conclusion, the vision and proprioceptive data-driven robot control model proposed in this study shows promise for automating the peg-in-hole task in construction. The combination of the SAP and DRL policy enables accurate perception and effective control, even in challenging real-world conditions. The multi-disciplinary nature of this research, incorporating computer vision, robotics, and machine learning, highlights the potential for interdisciplinary approaches to solve complex problems in various domains.
Read the original article