Dataset Ninja LogoDataset Ninja:

COCO-Stuff 10k Dataset

100001831923
Taggeneral, benchmark
Taskinstance segmentation
Release YearMade in 2017
Licensecustom
Download2 GB

Introduction #

Released 2017-04-06 ·Holger Caesar, Jasper Uijlings, Vittorio Ferrari

The authors of the COCO-Stuff 10k dataset address the distinction between semantic classes, categorizing them as either thing (an object with well-defined shapes such as cars and people) or stuff (amorphous background regions like grass and sky). They noted that while a significant amount of research has focused on “thing” classes, relatively less attention has been devoted to “stuff” classes. They emphasized the importance of “stuff” classes in image understanding, as they play a crucial role in defining scene types, contextual reasoning, and describing physical attributes and geometric properties of scenes.

To promote the understanding of “stuff” and “things” within context, the authors introduced COCO-Stuff, an extension of the COCO 2015 dataset. COCO-Stuff augmented 10K images from COCO 2014. To be compatible with COCO, version 1.1 of COCO-Stuff has 91 thing classes (1-91), 91 stuff classes (92-182) and 1 class “unlabeled” (0). Note that 11 of the thing classes from COCO 2015 do not have any segmentation annotations. The classes desk, door and mirror could be either stuff or things and therefore occur in both COCO and COCO-Stuff. To avoid confusion we add the suffix “-stuff” to those classes in COCO-Stuff.

Furthermore, the authors used COCO-Stuff to analyze various aspects, including the importance of “stuff” and “thing” classes in terms of surface coverage and frequency in image captions, the spatial relationships between “stuff” and “things,” and the performance of modern semantic segmentation methods on these classes.

They underscored the significance of “stuff” classes, emphasizing that they constitute the majority of visual surroundings and provide critical context for recognizing and understanding “things.” “Stuff” classes influence the type of scene and constrain the possible locations of “things.” Additionally, they help determine depth ordering and relative positions of “things” and support the interpretation of relationships between them. The context provided by “stuff” is instrumental in recognizing smaller or less common “things” in images.

The hierarchy of labels:

image

COCO-Stuff was introduced as a valuable addition to COCO, enabling the exploration of rich relationships between “stuff” and “things” in complex scenes. It was noted that COCO-Stuff offered a significant contribution to complete scene understanding.

ExpandExpand
Dataset LinkHomepageDataset LinkResearch Paper 1 (main)Dataset LinkResearch Paper 2

Summary #

COCO-Stuff 10K Dataset: Common Objects in Context Stuff 10k v1.1 is a dataset for instance segmentation, semantic segmentation, and object detection tasks. It is applicable or relevant across various domains.

The dataset consists of 10000 images with 228313 labeled objects belonging to 183 different classes including unlabeled, person, tree, and other: wall-other, sky-other, grass, building-other, clouds, road, pavement, chair, car, structural-other, dining table, fence, window-other, ground-other, cup, plant-other, bottle, bush, ceiling-other, furniture-other, light, bowl, table, dirt, door-stuff, and 155 more.

Images in the COCO-Stuff 10k dataset have pixel-level instance segmentation annotations. Due to the nature of the instance segmentation task, it can be automatically transformed into a semantic segmentation (only one mask for every class) or object detection (bounding boxes for every object) tasks. All images are labeled (i.e. with annotations). There are 2 splits in the dataset: train (9000 images) and test (1000 images). Additionally, images have caption tags, while objects contain category tags with information about labels hierarchy. Explore them in supervisely. The dataset was released in 2017 by the University of Edinburgh, United Kingdom and Google AI Perception.

Here is a visualized example for randomly selected sample classes:

Explore #

COCO-Stuff 10k dataset has 10000 images. Click on one of the examples below or open "Explore" tool anytime you need to view dataset images with annotations. This tool has extended visualization capabilities like zoom, translation, objects table, custom filters and more. Hover the mouse over the images to hide or show annotations.

OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
OpenSample annotation mask from COCO-Stuff 10kSample image from COCO-Stuff 10k
👀
Have a look at 10000 images
View images along with annotations and tags, search and filter by various parameters

Class balance #

There are 183 annotation classes in the dataset. Find the general statistics and balances for every class in the table below. Click any row to preview images that have labels of the selected class. Sort by column to find the most rare or prevalent classes.

Search
Rows 1-10 of 183
Class
Images
Objects
Count on image
average
Area on image
average
unlabeled
mask
8759
8759
1
13.7%
person
mask
5492
15805
2.88
16.34%
tree
mask
3008
8101
2.69
17.43%
wall-other
mask
2450
7170
2.93
19.88%
sky-other
mask
2230
5770
2.59
22.36%
grass
mask
1761
5616
3.19
22.26%
building-other
mask
1610
4381
2.72
17.03%
clouds
mask
1524
4115
2.7
22.07%
road
mask
1313
3315
2.52
18.18%
pavement
mask
1054
2876
2.73
13.51%

Images #

Explore every single image in the dataset with respect to the number of annotations of each class it has. Click a row to preview selected image. Sort by any column to find anomalies and edge cases. Use horizontal scroll if the table has many columns for a large number of classes in the dataset.

Class sizes #

The table below gives various size properties of objects for every class. Click a row to see the image with annotations of the selected class. Sort columns to find classes with the smallest or largest objects or understand the size differences between classes.

Search
Rows 1-10 of 172
Class
Object count
Avg area
Max area
Min area
Min height
Min height
Max height
Max height
Avg height
Avg height
Min width
Min width
Max width
Max width
unlabeled
mask
76367
1.57%
100%
0.01%
1px
0.16%
640px
100%
65px
13.62%
1px
0.16%
640px
100%
person
mask
15805
5.68%
94.64%
0.01%
3px
0.75%
640px
100%
139px
29.13%
3px
0.47%
640px
100%
tree
mask
8101
6.47%
98.7%
0.02%
1px
0.16%
640px
100%
111px
23.43%
1px
0.16%
640px
100%
wall-other
mask
7170
6.79%
100%
0.02%
1px
0.21%
640px
100%
144px
29.82%
1px
0.16%
640px
100%
sky-other
mask
5770
8.64%
99.79%
0.01%
1px
0.21%
640px
100%
99px
20.88%
1px
0.16%
640px
100%
grass
mask
5616
6.98%
98.64%
0.01%
2px
0.42%
640px
100%
91px
19.45%
1px
0.16%
640px
100%
building-other
mask
4381
6.26%
99.39%
0.01%
3px
0.47%
640px
100%
115px
23.98%
1px
0.16%
640px
100%
clouds
mask
4115
8.17%
99.81%
0.02%
2px
0.55%
640px
100%
96px
20.28%
1px
0.16%
640px
100%
road
mask
3315
7.2%
78.55%
0.01%
1px
0.2%
612px
100%
98px
21.01%
1px
0.16%
640px
100%
chair
mask
2912
2.2%
87.91%
0.02%
2px
0.53%
569px
100%
87px
18.65%
2px
0.31%
640px
100%

Spatial Heatmap #

The heatmaps below give the spatial distributions of all objects for every class. These visualizations provide insights into the most probable and rare object locations on the image. It helps analyze objects' placements in a dataset.

Spatial Heatmap

Objects #

Table contains all 228313 objects. Click a row to preview an image with annotations, and use search or pagination to navigate. Sort columns to find outliers in the dataset.

Search
Rows 1-10 of 98735
Object ID
Class
Image name
click row to open
Image size
height x width
Height
Height
Width
Width
Area
1
unlabeled
mask
COCO_train2014_000000412281.jpg
480 x 640
11px
2.29%
54px
8.44%
0.03%
2
unlabeled
mask
COCO_train2014_000000412281.jpg
480 x 640
11px
2.29%
28px
4.38%
0.03%
3
unlabeled
mask
COCO_train2014_000000412281.jpg
480 x 640
62px
12.92%
26px
4.06%
0.05%
4
unlabeled
mask
COCO_train2014_000000412281.jpg
480 x 640
46px
9.58%
14px
2.19%
0.08%
5
unlabeled
mask
COCO_train2014_000000412281.jpg
480 x 640
20px
4.17%
15px
2.34%
0.03%
6
unlabeled
mask
COCO_train2014_000000412281.jpg
480 x 640
56px
11.67%
41px
6.41%
0.38%
7
unlabeled
mask
COCO_train2014_000000412281.jpg
480 x 640
51px
10.62%
76px
11.88%
0.27%
8
unlabeled
mask
COCO_train2014_000000412281.jpg
480 x 640
24px
5%
26px
4.06%
0.13%
9
unlabeled
mask
COCO_train2014_000000412281.jpg
480 x 640
21px
4.38%
23px
3.59%
0.08%
10
unlabeled
mask
COCO_train2014_000000412281.jpg
480 x 640
180px
37.5%
155px
24.22%
0.5%

License #

COCO-Stuff is a derivative work of the COCO dataset. The authors of COCO do not in any form endorse this work. Different licenses apply:

Source

Citation #

If you make use of the COCO-Stuff 10k data, please cite the following reference:

@misc{caesar2018cocostuff,
  title={COCO-Stuff: Thing and Stuff Classes in Context}, 
  author={Holger Caesar and Jasper Uijlings and Vittorio Ferrari},
  year={2018},
  eprint={1612.03716},
  archivePrefix={arXiv},
  primaryClass={cs.CV}
}

Source

If you are happy with Dataset Ninja and use provided visualizations and tools in your work, please cite us:

@misc{ visualization-tools-for-cocostuff10k-dataset,
  title = { Visualization Tools for COCO-Stuff 10k Dataset },
  type = { Computer Vision Tools },
  author = { Dataset Ninja },
  howpublished = { \url{ https://datasetninja.com/cocostuff10k } },
  url = { https://datasetninja.com/cocostuff10k },
  journal = { Dataset Ninja },
  publisher = { Dataset Ninja },
  year = { 2024 },
  month = { feb },
  note = { visited on 2024-02-24 },
}

Download #

Dataset COCO-Stuff 10k can be downloaded in Supervisely format:

As an alternative, it can be downloaded with dataset-tools package:

pip install --upgrade dataset-tools

… using following python code:

import dataset_tools as dtools

dtools.download(dataset='COCO-Stuff 10k', dst_dir='~/dataset-ninja/')

Make sure not to overlook the python code example available on the Supervisely Developer Portal. It will give you a clear idea of how to effortlessly work with the downloaded dataset.

The data in original format can be downloaded here:

. . .

Disclaimer #

Our gal from the legal dep told us we need to post this:

Dataset Ninja provides visualizations and statistics for some datasets that can be found online and can be downloaded by general audience. Dataset Ninja is not a dataset hosting platform and can only be used for informational purposes. The platform does not claim any rights for the original content, including images, videos, annotations and descriptions. Joint publishing is prohibited.

You take full responsibility when you use datasets presented at Dataset Ninja, as well as other information, including visualizations and statistics we provide. You are in charge of compliance with any dataset license and all other permissions. You are required to navigate datasets homepage and make sure that you can use it. In case of any questions, get in touch with us at hello@datasetninja.com.