The authors introduce a synthetic dataset, SYNTHIA, The SYNTHetic collection of Imagery and Annotations, specially designed to evaluate active learning for video object detection in road scenes. Autonomous driving systems require huge amounts of data to train. Manual annotation of this data is timeconsuming and prohibitively expensive since it involves human resources. Therefore, active learning emerged as an alternative to ease this effort and to make data annotation more manageable. For autonomous driving systems, the quality of object detection is of key importance. Its progress in recent years has been notable, partially due to the presence of large datasets. However, pushing detectors to further improve and finally be close to flawless, requires the collection of ever larger labeled datasets, which is both time and labor expensive. In their work the authors focused on active learning for object detection in videos.

Dataset description

The authors have created a new synthetic dataset to evaluate active learning for object detection in road scenes. In particular, they modified the SYNTHIA environment to generate the SYNTHIA-AL dataset using Unity Pro game engine. The aim is having an unbalanced foreground/background distribution, simulating the real collection scenario of a driving car. Moreover, a set of object classes and conditions should be predominantly present, while other classes and conditions must appear less frequent. The data is generated by driving a car in a virtual world consisting of three different areas, namely town, city, and highway. These areas are populated with a variety of pedestrians, cars, cyclists, and wheelchairs, except for the highway which is limited to cars. These dynamic objects are arbitrarily spawned at predefined positions with a given probability and follow randomly predefined paths without leaving each area. Several environmental conditions can be set: season (winter, fall, spring), day time (day or night), and weather (clear or rainy). By default, the authors always use spring
and clear during the day, and only change one condition at a time. Objects with no lights can be hard to visualize during the night, so they only use cars for the night condition.

Examples of errors detected by temporal coherence approach on SYNTHIA-AL (top, middle) and ImageNet-VID(bottom). The authors show ground-truth boxes in yellow and output detections in red. After solving their graphical model based on temporal coherence, some of the detections are considered as false positives (purple), while other boxes are added as false negatives (green).

The video sequences are captured at 25 fps with a random length between 10 and 30 seconds. The authors have generated one subset with the default parameters and three smaller subsets with altered conditions. The first subset consists of 150 sequences, which amounts to 75% of all the data, with the default settings, i.e. containing cars, pedestrians, and cyclists, under different daily conditions, but only in the city and highway areas. The second subset contains 36 sequences
(20% of the dataset) captured in the town area instead. The night condition only represents 3% of the whole data (6 sequences) and it is fully contained in the third subset. Finally, they have added wheelchairs and removed cars in the fourth subset, which represents the 2% of the dataset with only 5 sequences. The test set contains 85 sequences with balanced distributions on areas and conditions (except winter) on the three main classes plus another 12 sequences including wheelchairs. All images are automatically annotated with 2D bounding boxes and class labels for every object that can be reasonably seen (more than 50 pixels).

SYNTHIA-AL data distribution. Seq. indicates the number of videos. Environment conditions are Fall (F), Winter (W), Spring (S), Rain ®, and Night (N). Areas are City ©, Town (T), and Highway (H). The spawning probabilities are given for pedestrians (Pe),cyclists (Cy), cars (Ca), and wheelchairs (Wh).

Dataset features

• XLarge Volume of Data & Groundtruth: +200,000 HD images from video streams, +20,000 HD images from independent snapshots,
• Scene Diversity: european style town, modern city, highway, green areas,
• Variety of Dynamic Objects: cars, pedestrians, cyclists,
• Multiple Seasons: dedicated themes for winter, fall, spring, and summer,
• Lighting Conditions and Weather: dynamic lights and shadows, several day-time modes, rain mode, night mode,
• Sensor Simulation: 8 RGB cameras forming a binocular 360º camera, 8 depth sensors
• Automatic Groundtruth: individual instances for: semantic segmentation (pixelwise annotations), depth, car ego-motion.