Person in Context - Dataset Ninja

Introduction #

Released 2021-05-24 ·Si Liu, Zitian Wang, Yulu Gaoet al.

The authors collect a PIC: Person in Context Dataset v2.0, which contains 17, 605 high-resolution images and densely annotated entity segmentation and relations, including 141 object categories, 23 relation categories. The authors introduced a new task named human-centric relation segmentation (HRS). HRS aims to predict the relations between the human and surrounding entities and identify the relation-correlated human parts, which are represented as pixel-level masks.

Note: the authors did not furnish the capability to compare a particular object with its surrounding entities in detail. The annotations merely offer broad descriptions of the interactions within the image. Consequently, in our scenario, the dataset is solely applicable for addressing instance segmentation challenges.

Dataset description

Data collection was done with three steps:

Data crawling: PIC contains both indoor and outdoor images, which are crawled from Flickr website with copyrights. Query words for retrieving indoor pictures include cook, party, drink, watch tv, eat, etc., and those for outdoor ones include play ball, run, ride, outing, picnic, etc. In this way, the collected data enjoy great diversities in terms of scenario, appearance, viewpoint, light condition and occlusion.
Data filtering: The authors filter out the images with low resolution or without human. Then they calculate the distributions of the relations.
Data balancing: The authors recollect the data for relations with lower frequency to balance the data distribution.

The authors first annotate 141 kinds of things and stuff in the images. The entity categories cover a wide range of indoor and outdoor scenes, including office,
restaurant, seaside, snowfield, etc. For each entity falling into predefined categories, they label it with its class and pixel-level mask segment. The disconnected regions of stuff are viewed as different entities.

An example of the original image and entity segmentation.

Expand

Homepage

Research Paper

GitHub

Summary #

PIC: Person in Context Dataset v2.0 is a dataset for instance segmentation, semantic segmentation, and object detection tasks. It is applicable or relevant across various domains.

The dataset consists of 17605 images with 191150 labeled objects belonging to 144 different classes including human, table, bag, and other: hat, ground, chair, door, painting/poster, sofa, building, shelf, window, grass, cabinet, vegetation, floor, guardrail, ball, book, curtain, cup, phone, bottle, toy, tree, plant, stick, instrument, and 116 more.

Images in the Person in Context dataset have pixel-level instance segmentation annotations. Due to the nature of the instance segmentation task, it can be automatically transformed into a semantic segmentation (only one mask for every class) or object detection (bounding boxes for every object) tasks. There are 2975 (17% of the total) unlabeled images (i.e. without annotations). There are 3 splits in the dataset: train (12653 images), test (2975 images), and val (1977 images). Alternatively, the dataset could be split into 2 location: indoor (7458 images) and outdoor (6539 images). The dataset was released in 2021 by the Beihang University, China, Academy of Science, China, and Sea AI Lab, China.

Here is a visualized example for randomly selected sample classes:

Explore #

Person in Context dataset has 17605 images. Click on one of the examples below or open "Explore" tool anytime you need to view dataset images with annotations. This tool has extended visualization capabilities like zoom, translation, objects table, custom filters and more. Hover the mouse over the images to hide or show annotations.

Sample annotation mask from Person in Context

👀

Have a look at 17605 images

View images along with annotations and tags, search and filter by various parameters

Class balance #

There are 144 annotation classes in the dataset. Find the general statistics and balances for every class in the table below. Click any row to preview images that have labels of the selected class. Sort by column to find the most rare or prevalent classes.

Rows 1-10 of 144

Class ㅤ	Images ㅤ	Objects ㅤ	Count on image average	Area on image average
human➔ mask	14629	38536	2.63	32.01%
table➔ mask	3268	6834	2.09	9.2%
bag➔ mask	3235	7222	2.23	1.95%
hat➔ mask	2799	5250	1.88	1.62%
ground➔ mask	2412	11693	4.85	30.04%
chair➔ mask	2288	7167	3.13	4.08%
door➔ mask	1683	3665	2.18	7.94%
painting/poster➔ mask	1652	3191	1.93	6.92%
sofa➔ mask	1637	9124	5.57	9.76%
building➔ mask	1337	5679	4.25	29.24%

Images #

Explore every single image in the dataset with respect to the number of annotations of each class it has. Click a row to preview selected image. Sort by any column to find anomalies and edge cases. Use horizontal scroll if the table has many columns for a large number of classes in the dataset.

Class sizes #

The table below gives various size properties of objects for every class. Click a row to see the image with annotations of the selected class. Sort columns to find classes with the smallest or largest objects or understand the size differences between classes.

Rows 1-10 of 141

Class	Object count	Avg area	Max area	Min area	Min height	Min height	Max height	Max height	Avg height	Avg height	Min width	Min width	Max width	Max width
human mask	38536	12.15%	94.94%	0%	1px	0.04%	5194px	100%	654px	47.94%	1px	0.05%	5566px	100%
ground mask	11693	6.2%	96.8%	0%	1px	0.02%	5503px	100%	323px	18%	1px	0.02%	8212px	100%
sofa mask	9124	1.75%	57.43%	0%	1px	0.04%	4559px	99.98%	210px	18.23%	1px	0.02%	6839px	99.99%
floor mask	7544	1.97%	85.87%	0%	1px	0.03%	3979px	99.95%	160px	14.55%	1px	0.07%	4287px	99.98%
bag mask	7222	0.88%	30.9%	0%	2px	0.14%	2818px	98.19%	215px	16.95%	2px	0.12%	1910px	99.93%
chair mask	7167	1.3%	31.12%	0%	1px	0.1%	4579px	99.9%	217px	17.87%	1px	0.04%	2297px	99.95%
table mask	6834	4.4%	56.26%	0%	1px	0.04%	2852px	99.93%	220px	20.08%	1px	0.05%	4751px	100%
grass mask	5886	6.9%	96.2%	0%	1px	0.04%	4869px	100%	322px	18.5%	1px	0.02%	5615px	100%
building mask	5679	6.88%	90.95%	0%	1px	0.03%	4188px	100%	365px	21.15%	1px	0.02%	6622px	100%
hat mask	5250	0.86%	23.3%	0%	5px	0.3%	1967px	87.22%	138px	11.63%	1px	0.08%	2011px	79.84%

Spatial Heatmap #

The heatmaps below give the spatial distributions of all objects for every class. These visualizations provide insights into the most probable and rare object locations on the image. It helps analyze objects' placements in a dataset.

Objects #

Table contains all 101181 objects. Click a row to preview an image with annotations, and use search or pagination to navigate. Sort columns to find outliers in the dataset.

Rows 1-10 of 101181

Object ID ㅤ	Class ㅤ	Image name click row to open	Image size height x width	Height ㅤ	Height ㅤ	Width ㅤ	Width ㅤ	Area ㅤ
1➔	human mask	indoor_00426.jpg	960 x 1452	729px	75.94%	318px	21.9%	7.56%
2➔	human mask	indoor_00426.jpg	960 x 1452	248px	25.83%	156px	10.74%	1.46%
3➔	human mask	indoor_00426.jpg	960 x 1452	645px	67.19%	459px	31.61%	8.98%
4➔	human mask	indoor_00426.jpg	960 x 1452	65px	6.77%	20px	1.38%	0.07%
5➔	human mask	indoor_00426.jpg	960 x 1452	226px	23.54%	94px	6.47%	0.64%
6➔	electronics mask	indoor_00426.jpg	960 x 1452	142px	14.79%	143px	9.85%	1.01%
7➔	kitchen_island mask	indoor_00426.jpg	960 x 1452	321px	33.44%	380px	26.17%	4.36%
8➔	kitchen_island mask	indoor_00426.jpg	960 x 1452	194px	20.21%	417px	28.72%	1.99%
9➔	kitchen_island mask	indoor_00426.jpg	960 x 1452	297px	30.94%	49px	3.37%	0.38%
10➔	kitchen_island mask	indoor_00426.jpg	960 x 1452	15px	1.56%	19px	1.31%	0.01%

License #

This PIC 2.0 API is made freely available to academic and non-academic entities for non-commercial purposes such as academic research, teaching, scientific publications, or personal experimentation. Permission is granted to use the code given that you agree to our license terms.

Source

Citation #

If you make use of the Person in Context data, please cite the following reference:

@dataset{Person in Context,
  author={Si Liu and Zitian Wang and Yulu Gao and Lejian Ren and Yue Liao and Guanghui Ren and Bo Li and Shuicheng Yan},
  title={Person in Context Dataset},
  year={2021},
  url={https://picdataset.com/challenge/task/download/}
}

Source

If you are happy with Dataset Ninja and use provided visualizations and tools in your work, please cite us:

@misc{ visualization-tools-for-person-in-context-dataset,
  title = { Visualization Tools for Person in Context Dataset },
  type = { Computer Vision Tools },
  author = { Dataset Ninja },
  howpublished = { \url{ https://datasetninja.com/person-in-context-2 } },
  url = { https://datasetninja.com/person-in-context-2 },
  journal = { Dataset Ninja },
  publisher = { Dataset Ninja },
  year = { 2024 },
  month = { jul },
  note = { visited on 2024-07-27 },
}

Download #

Dataset Person in Context can be downloaded in Supervisely format:

As an alternative, it can be downloaded with dataset-tools package:

pip install --upgrade dataset-tools

… using following python code:

import dataset_tools as dtools

dtools.download(dataset='Person in Context', dst_dir='~/dataset-ninja/')

Make sure not to overlook the python code example available on the Supervisely Developer Portal. It will give you a clear idea of how to effortlessly work with the downloaded dataset.

The data in original format can be downloaded here.

. . .

Disclaimer #

Our gal from the legal dep told us we need to post this:

Dataset Ninja provides visualizations and statistics for some datasets that can be found online and can be downloaded by general audience. Dataset Ninja is not a dataset hosting platform and can only be used for informational purposes. The platform does not claim any rights for the original content, including images, videos, annotations and descriptions. Joint publishing is prohibited.

You take full responsibility when you use datasets presented at Dataset Ninja, as well as other information, including visualizations and statistics we provide. You are in charge of compliance with any dataset license and all other permissions. You are required to navigate datasets homepage and make sure that you can use it. In case of any questions, get in touch with us at hello@datasetninja.com.