The authors created the Expo Markers Dataset - new public synthetic dataset intended for detection and segmentation tasks. The dataset comprises 5,000 synthetic photorealistic images along with 200 real images, each accompanied by its corresponding pixel-perfect segmentation ground truth. The authors aimed to achieve superior performance on manually gathered and annotated real-world data featuring custom objects. To accomplish this, they generated 3D models of the target objects as well as potential distraction objects, situating them within a simulated environment.

Motivation

In contemporary times, supervised deep learning algorithms, particularly Convolutional Neural Networks (CNNs), have surpassed classical machine learning and computer vision algorithms in performance across standard tasks. While these algorithms achieve state-of-the-art results, they demand large volumes of labeled data to deliver robust solutions for real-world scenarios. However, the process of data collection and labeling is both costly and time-consuming, and the quality of available data varies significantly among providers, especially in tasks requiring pixel-wise accuracy such as segmentation. In some instances, extracting labels manually becomes nearly impossible. Thus, labeled data continues to present a bottleneck in enhancing outcomes. In numerous computer vision applications like autonomous driving, security systems, smart stores, and interactive robotics, end-users anticipate high-quality, dependable results. Therefore, employing well-established and superior quality supervised machine learning algorithms trained on extensive datasets remains the preferred approach. Although techniques like few-shot learning, weakly supervised learning, unsupervised learning, and deep feature extraction are gaining traction and displaying promising results, supervised learning remains straightforward, efficient, and easily trainable.

The potential of synthetic data has been apparent since its inception: datasets generated through computational algorithms capable of replicating semantic and visual patterns observed in the real world. Synthetic data facilitates the training of machine learning algorithms without compromising privacy (e.g., facial data) or succumbing to inherent biases in the data (e.g., ensuring equal representation of genders in the dataset). Furthermore, synthetic data offers scalability, as additional data can always be generated, and it addresses edge cases—instances where manually gathering data proves arduous or impractical. Over recent years, synthetic data has shown promise across various domains, from medical research, where patient privacy is paramount, to fraud detection, where synthetic datasets aid in testing and bolstering the robustness of security solutions. Within the computer vision field, synthetic data generation has gained significant traction as a solution to the data bottleneck challenge.

Dataset description

The Expo Markers Dataset contains 5,000 synthetic photorealistic images and 200 real images with their corresponding pixel-perfect segmentation ground truth. Image resolution is 1024x1024 and each image contains, on average, 13 marker object instances and 80 distractor object instances.

Samples from Synthetic dataset.

To assess and compare the performance of the synthetic data generated by the authors, they collected two manually gathered datasets named Real A and Real B. These datasets were acquired using different devices and in distinct environments, which the authors perceive as representing two distinct domains. Both datasets exhibit a mix of simplicity and complexity, featuring images with varying levels of occlusions and objects resembling markers, such as pens, pencils, and other brands of markers. The majority of images in the manually-gathered datasets contain four markers, each representing a different marker color. Real A comprises 250 images captured with a Samsung S10+ camera and is specifically utilized for evaluation purposes. On the other hand, Real B consists of 1,000 images captured with an Apple iPhone 7 camera, serving dual purposes for both training and evaluation.

Note: The authors provided only 200 real images for download, without division into Real A and Real B.

Expo Markers Dataset.

Initially, the authors developed a highly realistic 3D representation of the target class, focusing on Expo markers. Skilled 3D artists meticulously crafted these representations, ensuring precise modeling for authenticity. Additionally, a collection of photorealistic items served as distractors for the algorithm.
Within the 3D environment, the target markers were strategically positioned to be visible to the simulated camera lens. Concurrently, the distractor items were incorporated into the same scene, intensifying the visual complexity. Notably, the composition of objects, backgrounds, scene lighting, occlusions, and the randomness of orientation deliberately deviated from real-world distributions. Instead, they aimed to challenge the algorithm by providing a more demanding dataset for training. The primary objective was to equip the algorithm with the capability to learn a resilient representation capable of addressing extreme scenarios encountered in real-world settings.

The images were rendered using the Cycles rendering engine, accompanied by the creation of segmentation maps. This approach ensures the generation of photorealistic images coupled with pixel-level annotations. Several calibration iterations were undertaken to establish appropriate parameter values governing the placement of objects, lighting conditions, and frequency of appearances. Leveraging the simplicity of synthetic data generation, subsequent iterations were swiftly executed, allowing for experimentation with various parameters until desirable outcomes were attained.