New datasets for human pose estimation

[carandraug]

🚀 The feature

We propose to add the following datasets for human pose estimation to torchvision:

• https://www.robots.ox.ac.uk/~vgg/data/pose/
• https://www.robots.ox.ac.uk/~vgg/data/pose_evaluation/
• https://www.robots.ox.ac.uk/~vgg/data/tv_human_interactions/

Motivation, pitch

These are some of our most downloaded datasets and we're trying to make them more useful by providing DataLoaders built into torchvision. Torchvision already includes some of our datasets (OxfordIIITPet, VOCDetection, VOCSegmentation, Flowers102, DTD, and FGVCAircraft) so we hope that the proposed new datastes would be accepted.

Alternatives

No response

Additional context

No response

cc @pmeier @NicolasHug

[carandraug]

To make it clear, I'm proposing to do this work (and I've actually already started). The question here is if there's interested on the torchvision side to include it (and if that comes with any conditions).

[bjuncek]

Hi @pmeier , do you have input on these?

Usually, the requirements for datasets are number of citations and papers used, combined with the most recent usage (vaguely).
Having said this, I believe we might be pausing the implementation until we get the prototype datasets are ready to replace the current implementations?

Some of these (namely VOC) are a staple, but they are also quite old...

[pmeier]

Usually, the requirements for datasets are number of citations and papers used, combined with the most recent usage (vaguely). Having said this, I believe we might be pausing the implementation until we get the prototype datasets are ready to replace the current implementations?

Correct. Unfortunately, datasets v2 takes longer than we anticipated. It makes little sense to implement wrappers for these datasets now just to supersede them in the near future.

Still, our original plan was to support quite a few more datasets than we had before. Since we currently don't have any datasets for human pose estimation, we could start with these. cc @NicolasHug

@carandraug is it ok to ping you if we finally managed to bring datasets v2 to a stable-ish state? If yes, could you provide a link to the paper than introduced the dataset as well as a citation count, for example from Google Scholar.

Plus, since I'm not familiar with the task, could either @carandraug or @bjuncek explain what the output of the dataset would look like?

[carandraug]

@carandraug is it ok to ping you if we finally managed to bring datasets v2 to a stable-ish state? If yes, could you provide a link to the paper than introduced the dataset as well as a citation count, for example from Google Scholar.

Yes, that is fine.

• TV Human Interactions Dataset:

  • dataset webpage
  • paper introducing dataset
  • Number of citations (according to Google Scholar): 215

• Human Pose Evaluator dataset (this is actually two datasets in one):

  • dataset webpage
  • paper introducing dataset
  • Number of citations (according to Google Scholar): 47

• Pose estimation datasets (these are 5 different datasets):

  • dataset webpage
  • This is a bit tricky because it's split over a few papers but I can sort this up once I get to these datasets)
  • Number of citations: 100 + 573 + 173 + 79 + 40

From the number of citations of the paper it doesn't look very impressive. However, we're basing the decision of doing this for these datasets on the number of downloads from our servers. We were surprised by the high number of downloads in certain old datasets which we assumed were no longer relevant. We believe that they're mainly used in teaching because when we take them down temporarily for maintenance we mainly received emails from undergrad students doing assignments and TA preparing lectures.

[datumbox]

One additional thing to keep in mind while discussing Pose estimation is that our Transforms V2 don't currently offer support for Keypoints. This is something we should factor in while prioritizing work.

[pmeier]

There is #5326 that I could revisit. #5326 (comment) states

we'll put this PR on hold until we have full support for bounding boxes and segmentation masks.

We do have support for this now, but I think priorities have changed in the mean time.