Examples¶
In this section, you will find the data loading implementations (using DataPipes) of various popular datasets across different research domains.
Audio¶
LibriSpeech¶
LibriSpeech dataset is corpus of approximately 1000 hours of 16kHz read English speech. Here is the DataPipe implementation of LibriSpeech to load the data.
Text¶
Amazon Review Polarity¶
The Amazon reviews dataset contains reviews from Amazon. Its purpose is to train text/sentiment classification models. In our DataPipe implementation of the dataset, we described every step with detailed comments to help you understand what each DataPipe is doing. We recommend having a look at this example.
IMDB¶
This is a large movie review dataset for binary sentiment classification containing 25,000 highly polar movie reviews for training and 25,00 for testing. Here is the DataPipe implementation to load the data.
SQuAD¶
SQuAD (Stanford Question Answering Dataset) is a dataset for reading comprehension. It consists of a list of questions by crowdworkers on a set of Wikipedia articles. Here are the DataPipe implementations for version 1.1 is here and version 2.0.
Additional Datasets in TorchText¶
In a separate PyTorch domain library TorchText, you will find some of the most popular datasets in the NLP field implemented as loadable datasets using DataPipes. You can find all of those NLP datasets here.
Vision¶
Caltech 101¶
The Caltech 101 dataset contains pictures of objects belonging to 101 categories. Here is the DataPipe implementation of Caltech 101.
Caltech 256¶
The Caltech 256 dataset contains 30607 images from 256 categories. Here is the DataPipe implementation of Caltech 256.
Additional Datasets in TorchVision¶
In a separate PyTorch domain library TorchVision, you will find some of the most popular datasets in the computer vision field implemented as loadable datasets using DataPipes. You can find all of those vision datasets here.
Note that these implementations are currently in the prototype phase, but they should be fully supported in the coming months. Nonetheless, they demonstrate the different ways DataPipes can be used for data loading.
Recommender System¶
Criteo 1TB Click Logs¶
The Criteo dataset contains feature values and click feedback for millions of display advertisements. It aims to benchmark algorithms for click through rate (CTR) prediction. You can find a prototype stage implementation of the dataset with DataPipes in TorchRec.
