ML-Spock

Motivation

‍This project focuses on steel structures commonly used in building and bridge construction, renowned for their ductility. Exposure to extreme conditions such as snowstorms, heavy traffic, or earthquakes results in inelastic deformation in these structures, diminishing their residual life. Traditional condition assessments heavily depend on periodic (human) visual inspections, a process prone to unreliability and risks for inspectors. The collaborative goal is to develop a novel, data-driven methodology for automated and reliable residual life (reserve capacity) prognosis of steel members in infrastructure systems based on visual data.

Proposed Approach / Solution

‍In this project, we explored various machine learning algorithms that rely solely on observed 3D deformations of steel members to assess their damage state, given that loading history in realistic settings is typically unavailable. To achieve this, we investigated different representations of a 3D digital object and fed the processed 3D point cloud of the deformed shape to the underlying machine learning model, as illustrated in Figure 1. The ground truth data was generated through high-fidelity numerical simulations of steel members, with a major focus on generalizing from the experimental laboratory environment to real-world settings. The simulations captured key behavioral transitions such as buckling-induced reductions in reserve capacity, as shown  in Figure 2.

Our proposed approach involved slicing the digital 3D objects into structured grids, which can be used in combination with pointwise MLP methods. Using a PointTransformer model, which applies attention within local neighborhoods around centroids in the point cloud, we demonstrated that this deep learning model can robustly estimate the reserve capacity of structural elements from their point cloud. Although including information beyond visual traces of deformation is not essential for robust quantification of the reserve capacity, incorporating the axial load ratio, member boundary conditions, and column dimensions as predictive features further reduced the error by approximately 20% to 25%. This is a practical finding given that these features can be estimated by engineers in a straightforward way if such additional accuracy is merited. Alternatively, integrating an auxiliary classification task—classifying the loading protocol, boundary condition, and loading ratio—enabled the model to learn a more discriminative feature space for point cloud embeddings, ultimately enhancing reserve capacity prediction accuracy. Given that the loading history of a structural member is typically unknown and cannot be inferred as accurately, these results highlight the primary importance of the deformed shape (rather than how the structural member came to be so deformed) on the associated reserve capacity.

Impact

‍This project's scientific breakthrough is quantifying the remaining lifespan of steel structures based on visual data alone. This can greatly simplify costly inspections, support expert training, and reduce reliance on time-consuming manual inspections that pose risks to humans. Moreover, the framework developed in this project can be used in drones or robots for automated inspection in hazardous environments. Finally, early damage identification enables proper remediation, making the infrastructure more sustainable and decreases its carbon footprint.