The authors present the BSTLD: Bosch Small Traffic Lights Dataset, an accurate dataset for vision-based traffic light detection. Vision-only based traffic light detection and tracking is a vital step on the way to fully automated driving in urban environments. The authors hope that this dataset allows for easy testing of objection detection approaches, especially for small objects in larger images.

Note, similar BSTLD: Bosch Small Traffic Lights Dataset datasets are also available on the DatasetNinja.com:

• S2TLD: Small Traffic Light Dataset

Motivation

Accurate detection and classification of traffic lights are vital for the success of automated driving in urban settings. Presently, there is a lack of systems capable of reliably perceiving traffic lights in real-time, without relying on map-based information, and from adequate distances necessary for seamless urban driving. While automated driving on highways has garnered considerable research attention, urban environments present a fresh set of challenges demanding more sophisticated algorithms across various domains, including perception, behavioral planning, and collision avoidance systems.

Within the realm of perception, identifying and categorizing traffic signs and lights stands out as a critical task. Traffic lights, in particular, pose a formidable challenge due to their small size and potential confusion with other urban elements such as lamps, decorations, and reflections. Recent advancements in deep neural networks have significantly enhanced various fields of machine learning, particularly computer vision. Deep learning techniques have been successfully employed for tasks such as image classification, end-to-end object detection, pixel-precise object segmentation, and numerous other applications.

Sample detections of small traffic lights in an image. The top image is taken at the full resolution of 1280 × 720. At the bottom, the enlarged crop shows detected traffic lights of size about 6 × 12 pixels. All lights are correctly classified as yellow.

Dataset description

The authors publish the Bosch Small Traffic Lights Dataset, an accurately labeled dataset for detecting, classifying, and tracking traffic lights. Dataset contains RGB color images at the resolution of 1280 × 720 pixels. For training, the authors collected more than 5000 images, mainly along El Camino Real in the San Francisco Bay Area in California. Overall, 10,756 traffic lights are labeled within those images. The different traffic light sizes within the training set vary between approximately 1 and 85 pixels in width with the mean of 11.3 pixels.

Training set bounding box sizes in pixels.

The label distribution of the different traffic light states is heavily skewed towards the three most common types which are green, red, and yellow. Also, because of the difference between the sampling frequency of our camera and the traffic light refresh rate, a lot of traffic lights frequently appear to be off. Whenever this is the case, the authors opted for labeling them as off instead of their current state.

Label distribution over the training and test set.

For testing, the authors collected a stereo video sequence with odometry along University Avenue in Palo Alto, California. In contrast to the training images, those images are taken and labeled consecutively at a frequency of 15.6 frames per second. Since the bounding boxes are labeled consecutively, it is possible to evaluate a tracker on them. In total, the test set comprises 8,334 labeled image files, encompassing 13,493 traffic lights. Notably, 2,094 of these lights are labeled despite being obscured by various objects. Remarkably, our test set exclusively includes labels for traffic lights in off, green, red, and yellow states. Table III offers a comprehensive breakdown of the diverse sizes of traffic lights within our test set. This dataset is intended to serve as a rigorous benchmark for the detection of traffic lights, or more broadly, small objects within images. Additionally, the authors present our findings as a baseline using this dataset.

Test set traffic light bounding box sizes in pixels.

Identifying objects within images presents significant challenges, particularly when aiming for consistent performance in real-time across a spectrum of object sizes. In the case of traffic lights, achieving real-time detection with minimal false negatives and false positives is imperative. A high false negative rate can result in missed traffic lights, an outcome deemed unacceptable for autonomous vehicles. Conversely, false positive detections can induce erratic behavior in automated vehicles, such as halting for non-existent red traffic lights.

Proposed neural network

The YOLO architecture has demonstrated impressive performance on datasets like PASCAL VOC 2007 and 2012, achieving processing speeds of up to 45 frames per second. However, a grid-based approach to detection presents limitations, particularly in suggesting bounding boxes per cell. Authors have noted difficulties in detecting small objects and clusters of objects. Additionally, there’s a challenge in balancing image input sizes, processing speed, and memory usage. Given that the average width of traffic lights in our datasets is only 10 pixels, adjustments are necessary to enable the network to detect small objects effectively. To address this, the authors have implemented a multi-step approach. Instead of feeding the entire image into the network, they partition the image into different patches and evaluate each patch separately. During training, random crops of the image are utilized as inputs to the network, leading to improved convergence speed and accuracy. For the final detection phase, they focus on three specific crops in the upper part of the image, as this is where most traffic lights are typically located. The authors have also developed a heatmap to visualize the occurrences of traffic lights. Each network has a receptive field of 448 × 448 pixels, allowing for comprehensive analysis of the image data.

Sample image from test-set. The authors network is evaluated at the positions within the image, shown in red, teal, and yellow. Two green traffic lights are detected, one by a cell of the yellow network, the other by red.