The authors have made available the VisDrone2019-DET Dataset, a comprehensive collection of drone-captured images tailored for object detection tasks. Spanning various urban and suburban locales across 14 different cities in China, from north to south, the VisDrone dataset stands as the most extensive of its kind ever published. This dataset facilitates thorough evaluation and exploration of visual analysis algorithms specifically designed for drone platforms.

Motivation

In recent years, drones (or UAVs) equipped with cameras have garnered significant attention. It is projected that the global commercial drone market size will reach $501.4 billion by 2028, with a compound annual growth rate of 57.5% from 2021 to 2028. Utilizing embedded devices, drones have the capability to analyze captured data and give rise to a multitude of new application scenarios:

• Agriculture. Drones can provide valuable insights to help farmers or ranchers optimize agriculture operations, monitor crop growth and keep herds safe, etc.
• Aerial photography. Drones are used to extract aerial photography images instead of expensive cranes and helicopters.
• Shipping and delivery. Drones can efficiently send packages such as medical supplies, food, or other goods to the designated places.
• Security and surveillance. Drones can provide realtime visibility into security threats and emergency situations by monitoring large regions.
• Search and rescue. Drones are useful to help search missing persons, fugitives, or rescue survivors and drop supplies in difficult terrains and harsh conditions.

As a result, there is a growing demand for the automatic interpretation of visual data gathered from drones, drawing computer vision technology closer to these aerial platforms. Object detection and object tracking, being two fundamental challenges in computer vision, are currently undergoing extensive research and development. However, despite significant advancements in various application domains such as internet and security surveillance, existing methods are often suboptimal when applied to sequences or images captured by drones. Notably, drone-captured video sequences present several unique challenges not typically encountered in traditional computer vision datasets:

• Viewpoint variations: comparing to surveillance cameras with fixed viewpoints, drone-equipped cameras monitor the objects in arbitrary viewpoints.
• Scale variations: drone-equipped cameras monitor the objects at different altitudes, resulting in large variations of scales of objects.
• Motion blur: videos are generally recorded by the droneequipped cameras in the moving process, bringing in considerable motion blurs of the recorded videos.

Hence, there is a critical need to advance the development and assessment of vision algorithms tailored for visual data captured by drones. However, progress toward this objective is significantly hindered by the absence of publicly available large-scale benchmarks or datasets. While some recent endeavors have been dedicated to constructing datasets captured by drones, with a particular focus on object detection or tracking, these datasets remain constrained in size and scope due to the challenges associated with data collection and annotation. Consequently, thorough evaluations of existing or newly developed algorithms remain an ongoing challenge. There is a pressing need for a more inclusive and comprehensive benchmark to further advance research in this field.

Dataset description

A foundational element crucial for the effective evaluation of algorithms is the availability of a comprehensive dataset. In this regard, the authors of VisDrone have meticulously curated the most extensive dataset to date, aiming to propel research in object detection and tracking on drones forward. This dataset comprises 263 video clips encompassing 179,264 frames, along with an additional 10,209 static images. These videos and images were captured using various drone platforms, including the DJI Mavic and Phantom series (3, 3A, 3SE, 3P, 4, 4A, 4P), across a diverse range of scenarios spanning 14 different cities in China. These cities include Tianjin, Hongkong, Daqing, Ganzhou, Guangzhou, Jincang, Liuzhou, Nanjing, Shaoxing, Shenyang, Nanyang, Zhangjiakou, Suzhou, and Xuzhou. The dataset encompasses a wide array of weather and lighting conditions, thus representing a myriad of scenarios encountered in daily life. The video clips boast a maximal resolution of 3840 × 2160, while the static images reach a resolution of 2000 × 1500, ensuring high-quality data for research purposes. The VisDrone benchmark focuses on the following four tasks: Object Detection in Images (DET), Object Detection in Videos (VID), Single-Object Tracking (SOT) and Multi-Object Tracking (MOT).

Annotated example images in the proposed datasets. The dashed bounding box indicates the object is occluded. Different colors indicate different classes of objects. For better visualization, only a few attributes are displayed.

The VisDrone-DET dataset tackles the problem of localizing multiple object categories in the image. For each image, algorithms are required to predict the bounding boxes of all the object instances of predefined object categories, with a real valued confidence. The dataset consists of 10, 209 images in unconstrained challenging scenes, including 6, 471 images in the training subset, 548 in the validation subset, 1, 580 in the test-challenge subset, and 1, 610 in the test-dev subset. Notably, the class imbalance issue significantly affect the detection performance. For example, the number of the awning-tricycle instances is more than 40× less than the car instances.

The authors mainly focus on people and vehicles in daily life and define ten object categories, including pedestrian, person(if a human maintains standing pose or walking, they classify it as pedestrian; otherwise, it is classified as a person), car, van, bus, truck, motor, bicycle, awning tricycle, and tricycle. Some rarely occurring vehicles are ignored, e.g., machineshop truck, forklift truck, and tanker. The authors also provide two attributes of each annotated bounding box to analyze the algorithms thoroughly, i.e., the occlusion* and truncation ratios. Specifically, occlusion ratio α denotes the fraction of objects being occluded by other objects or background, including no occlusion (δ = 0%), partial occlusion (δ ∈ (0%, 50%]), and heavy occlusion (δ ∈ (50%, 100%]). The truncation ratio denotes the degree of object parts appearing outside a frame when the object is captured near the frame boundary. It is estimated based on the region outside the frame by the human.

The number of objects with different occlusion degrees of different object categories in the subsets of the dataset.