The dacl10k: Benchmark for Semantic Bridge Damage Segmentation is the first large-scale dataset for semantic bridge damage segmentation. It includes images collected during concrete bridge inspections acquired from databases at authorities and engineering offices, thus, it represents real-world scenarios. Concrete bridges represent the most common building type, besides steel, steel composite, and wooden bridges.

Civil engineering structures such as power plants, sewers, and bridges form essential components of the public infrastructure. It is mandatory to keep these structures in a safe and operational state. In order to ensure this, the authors of the dataset frequently inspected where the current recognition and documentation of defects and building components is mostly carried out manually.

A failure of individual structures results in enormous costs. For example, the economic costs caused by the closure of a bridge due to congestion is many times the cost of the bridge itself and its maintenance. The authors primary objective was to develop a dataset that enables the training of models which later support the inspector during damage recognition and documentation to a maximum.

Dataset acquisition

Approximately one half of the images originate from databases of engineering offices, while the other half was provided by local authorities from Germany. The images were taken between 2000 and 2020. Both data sources supplied highly heterogeneous images regarding camera type, pose, lighting condition, and resolution.

Dataset description

The authors recognize 13 frequently occurring defects on reinforced concrete bridges (e.g., crack, spalling, efflorescence) and 6 important building parts (e.g., exposed reinforcement bar exposed rebar, bearing, expansion joint, protective equipment). All these classes play an important role for determining the building’s structural integrity, traffic safety and durability. Dacl10k includes 9,920 images from more than 100 different bridges, specifically designed for practical use, as it comprises all visually unique damage types defined by bridge inspection standards. Every object class in dataset can be labeled with:

• concrete defect tag: cavity,efflorescence,alligator crack,spalling,restformwork,exposed rebars, hollowareas, crack, rockpocket classes,
• general defect tag: rust, wetspot, weathering, graffiti classes.
• object part tag: bearing, expansion joint, drainage, protective equipment, joint tape, washouts/concrete corrosion classes.

Example annotations from dacl10k. Top row: original image. Middle row: polygonal annotations. Bottom row: stacked masks. The following classes are abbreviated: Alligator Crack (ACrack), Washouts/Concrete corrosion (WConccor), Expansion Joint (EJoint), Protective Equipment (PEquipment) and Joint Tape (JTape). From left to right, the images display the individual classes: 1. Weathering, Spalling, Exposed Rebars, Rust; 2. Weathering, Crack; 3. Alligator Crack, Restformwork, Efflorescence; 4. Weathering, Crack, Spalling, Rockpocket; 5. Crack, Rust, Expansion Joint, Spalling; 6. Weathering, Rockpocket, Spalling, Efflorescence, Crack, Rust, Restformwork, Joint Tape; 7. Weathering, Protective Eq.

The 19 classes considered within dacl10k are separated into three groups: concrete defects, general defects and objects. The concrete defects appear only on building parts made of (reinforced) concrete, while general defects may be present on all materials (e.g., concrete or steel). The only defect within dacl10k that is not visually recognizable is hollowareas. This damage is usually identified by hammering on the concrete surface, thus, it can only be detected acoustically (not visually) but, as it is bordered with chalk during hands-on inspections, the authors annotated its markings. The objects group includes all components of a bridge that are not made of concrete, such as joint tapes, railings or impact attenuation devices (protective equipment). The objects often show defects such as geometrical irregularities or deficits in structural capacity. Geometrical irregularities can arise from wrong distances between the railing rods or if the railing height is less than the minimum according to the given national standard. These visually challenging recognizable issues are not part of the dataset.

Dataset comprises coarse pixel-level annotations. The authors border each defect and object on a given image with one polygon (shape) and assign its label. Furthermore, they include polygons of the same class that overlap with each other in one shape. With respect to inspection standards and application, for all the defect classes, it is not important to differentiate between instances of a given class. Instead, their size and localization on a class-basis is important. Thereby, the authors utilize the open-source labeling tool LabelMe.

It often appears that shapes of different damage or object classes overlap with each other. Consequently, the underlying task can be described as multi-label semantic segmentation because one pixel can be part of multiple defects and objects. In other words, the labels are not assigned mutually exclusive to the pixels. The authors labeling process consisted of two consecutive steps: annotation of data received from engineering offices by civil engineering students (in-house) and annotation data from the authorities by an external annotation team, previously filtered for relevant image content. The students labeled approximately 7,000 images in accordance to our guidelines. Nearly 30% of the images had to be rejected due to flags indicating bad quality (blurring, overexposure) or personal data.

The pipeline during in-house labeling was separated into three parts. The first part consisted of the regular annotation which comprises annotating a batch of 100 images, getting feedback by a domain expert and correcting the failures accordingly. The second part included an extensive analysis of the dataset to find structural failures in the annotations. Thirdly, the dataset was divided into subtasks with respect to the failures most commonly made. Then, one student corrected each failure type consecutively. The quality assessment of the data annotated by the external team was divided into four quality checks for each data batch. In average, one batch included 250 images. Each check included one iteration over the annotated data by experts. Based on the analysis, the error rate was determined which is the ratio of false-labeled images and total amount of frames in the according batch. Starting with an error rate of 60%, the rate could be lowered to a final value of 1%.

Statistical analysis

Regarding the pixel counts over the whole dataset, it can be stated that weathering, followed by protective equipment, are the most dominant classes with
nearly four and 1.5 billion pixels. Within the range of 0.1 billion and 1 billion pixels, the majority of defects and objects with respect to the number of pixels can be found. Clearly underrepresented are the defects restformwork (66 million) and the objects joint tape (68 million) and exposed rebars (65 million).

Pixel counts with respect to each class in dacl10k based on the original image sizes. The bars are arranged according to the group affiliation.

The average image size is 1581 px in height and 1950 px in width. The mean image area is approximately 4 megapixels while the total pixel area in the dataset is approx. 43 billion px. Table below provides an overview of statistics describing the classwise density of polygons, size of polygons, density of pixels, share of polygons and share of pixels over the whole dataset. In average, 1.8 crack shapes are present on a crack image, whereby, one polygon includes 27,467 px. The average crack image shows approx. 50,000 px labeled as crack. With respect to the displayed shares, four out of 100 polygons are labeled as crack and
0.3% of the total pixel area received the label crack.

Furthermore, table below reveals the cause of the overrepresented classes weathering and protective equipment. They display a share regarding the number of polygons of 5.31% (top 20%) and 2.09% (exactly the median). This, in combination with the fact that an according image shows 900,000 px or rather 786,000 px of that class, leads to their dominant role. The overrepresentation of weathering can be more fatal than the one of protective equipment with respect to the model performance. This is due to the fact thatthe features (shape and texture) of weathering are similar to the ones from wetspot. Both are of round shape and represented by a darker area surrounded by a brighter “rest”. They vary slightly with regards to their texture. weathering is more noisy and more matt than wetspot which is smooth and sometimes mirroring.

In addition, wetspot and weathering often overlap which makes it difficult to distinguish between them from the model’s perspective. The features of the object protective equipment, in contrast, are unique and therefore shouldn’t interfere with other classes during learning. The lack of pixels representing restformwork, exposed rebars and joint tape, as mentioned before, originates from their relatively rare occurrences but mostly from their small shapes. In average, polygons bordering restformwork or joint tape have a size of approx. 50,000 px while polygons labeled as exposed rebars include 26,000 px. The average size of their polygons is equal to or less than the lower quartile (50,367 px). Additionally, exposed rebars shows the smallest average polygon size of all classes.

Overall statistics of the dataset regarding average number of polygons per image, number of pixels per polygon, number of pixels per image, share of polygons and share of pixels. Midrules separate the classes according to their group affiliation.