A study conducted by a team of researchers from Huazhong University of Science and Technology in Wuhan, China, focuses on developing a computer vision-based autonomous robotic arm to repair cracks in concrete structures. Concrete cracks are significant indicators of structural deterioration, often resulting from cyclic loading and fatigue stress. They create interconnected passageways that compromise the integrity and safety of the structure.
Identifying these cracks poses considerable challenges, underscoring the necessity for an unmanned detection and repair system. Various studies in this domain have demonstrated that recent advancements in computational techniques have facilitated the application of computer vision in detecting and assessing structural issues. Consequently, computer vision-based concrete crack detection is increasingly recognized as a form of nondestructive testing, employing a range of methodologies to ascertain the presence and location of cracks.
While some research has concentrated on extracting fundamental parameters such as crack length, width, and depth, this information alone is insufficient for making informed decisions regarding crack repair strategies. As a result, researchers are increasingly interested in exploring the use of bacteria to facilitate the repair of cracks in concrete.
Recent advancements in digital technologies have significantly enhanced the recognition of concrete cracks. Essential parameters for effective crack inspection include length, width, depth, and direction. Computer vision techniques provide more accurate results compared to traditional manual methods; however, challenges such as irregular shapes and the presence of noise continue to exist. A variety of image processing methods have been developed for crack detection, encompassing integrated algorithms and morphological approaches. Furthermore, advanced techniques, including spatial wavelet transformation and convolutional neural networks, have emerged, leading to improved crack localization and segmentation. These innovations have further augmented real-time detection capabilities, contributing to more efficient monitoring and maintenance efforts.
The architecture of the autonomous crack repair system is structured around four fundamental steps:
- Crack Feature Extraction via Computer Vision
This step employs advanced image processing techniques to accurately determine the size, position, and characteristics of cracks. The extracted features are then stored in a comprehensive database to facilitate informed decision-making.
- Trajectory Generation and Robot Calibration
Crack coordinates are effectively extracted through hand-eye calibration. The system utilizes a pinhole camera model to translate the identified crack locations into precise robot coordinates.
- Knowledge Base for Repair Decision Making
A well-organized database containing various crack types, appropriate repair materials, necessary tools, and execution methodologies is maintained. Human expertise plays a vital role in refining and optimizing repair strategies.
- Design and Control of the Robotic Arm
A 6-axis robotic manipulator, specifically the ABB IRB 6700-235, is employed to perform repair tasks. This includes functionalities such as grouting, sealing, and surface finishing through the use of convertible tools.
The study involves the implementation and testing of a semi-autonomous robotic system for concrete crack repair in a controlled laboratory environment as follows:
Crack Feature Database:
The database was developed with a focus on key crack parameters, including length, width, depth, material, and location. The system captures images, which are subsequently preprocessed and segmented to accurately detect these crack parameters using advanced computer vision techniques. The extracted data is organized in an Excel database and later converted to SQL format for seamless integration into the overall system.
The final database comprises over 200 samples, reflecting eight distinct crack features: component position, crack location, crack material, crack properties, crack length, crack width, crack depth, and crack direction. Both the crack images and processed data have been structured into an SQL table to facilitate efficient access and retrieval.
Knowledge Base of Repair Methods:
A knowledge base has been developed to ensure that each crack is repaired using the most suitable method. The data consists of eight features: repair technology, method name, scope of application, repair materials, repair tools, repair process, execution code, and reference standards. This database is connected to the crack feature database.
Simulation System and Execution Device:
To test the developed model, researchers created a simulation environment using RobotStudio. The simulation consists of a robot arm model, which includes a 6-axis robotic manipulator, collision detection, and execution code.
Physical Execution:
The developed semi-autonomous robotic arm is equipped with seven tools with different functions discussed as follows:
- Slotting machine – Surface preparation
- polisher – Removes debris
- Air-blowing device – Removes dust from cracks
- Scraper – Applies sealant
- Brush roller – Applies protective coating
- Injection Device – Fills deep cracks
- Spatula – Smoothens repaired surface
The precision of the robotic execution was rigorously assessed by performing a detailed comparison between the simulated repair paths generated by the system and the actual repair paths achieved in real-world conditions. To quantify the level of similarity between these two sets of data, a sophisticated mathematical technique was employed, which allowed for a comprehensive evaluation of the divergence between planned and executed repairs. Remarkably, the resulting similarity score was calculated at an impressive 99.23%, highlighting an exceptionally high degree of accuracy and consistency between the simulations and real-world implementations of the repair tasks.
This study significantly demonstrates the feasibility of using computer vision technology to guide robotic systems in effectively performing crack repairs in infrastructure. This advancement has profound implications, as it can substantially enhance safety standards, improve operational efficiency, and reduce overall maintenance costs associated with infrastructure upkeep.
Future research initiatives will aim to further develop this system, with a strong focus on achieving full autonomy and incorporating self-learning capabilities. By doing so, the technology can adapt and improve over time, ultimately leading to even more effective maintenance solutions in infrastructure repair.
Reference
Chen, R., Zhou, C. & Cheng, Ll. Computer-vision-guided semi-autonomous concrete crack repair for infrastructure maintenance using a robotic arm. AI Civ. Eng. 1, 9 (2022).https://doi.org/10.1007/s43503-022-00007-7
Kim, H., Sim, S.-H., & Spencer, B. F. (2022). Automated concrete crack evaluation using stereo vision with two different focal lengths. Automation in Construction, 135, 104136.
Shamsabadi, E. A., Xu, C., & Rao, A. S. (2022). Vision transformer-based autonomous crack detection on asphalt and concrete surfaces. Automation in Construction, 140, 104316.
Liu, L., Yan, R.-J., Maruvanchery, V., Kayacan, E., Chen, I.-M., & Tiong, L. K. (2017). Transfer learning on convolutional activation feature as applied to a building quality assessment robot. International Journal of Advanced Robotic Systems., 14(3), 1729881417712620. https://doi.org/10.1177/1729881417712620