In this section, you will find the data loading implementations (using DataPipes) of various popular datasets across different research domains.
Amazon Review Polarity¶
The Amazon reviews dataset contains reviews from Amazon. Its purpose is to train text/sentiment classification models. In our DataPipe implementation of the dataset, we described every step with detailed comments to help you understand what each DataPipe is doing. We recommend having a look at this example.
This is a large movie review dataset for binary sentiment classification containing 25,000 highly polar movie reviews for training and 25,00 for testing. Here is the DataPipe implementation to load the data.
SQuAD (Stanford Question Answering Dataset) is a dataset for reading comprehension. It consists of a list of questions by crowdworkers on a set of Wikipedia articles. Here are the DataPipe implementations for version 1.1 is here and version 2.0.
Additional Datasets in TorchText¶
In a separate PyTorch domain library TorchText, you will find some of the most popular datasets in the NLP field implemented as loadable datasets using DataPipes. You can find all of those NLP datasets here.
Additional Datasets in TorchVision¶
In a separate PyTorch domain library TorchVision, you will find some of the most popular datasets in the computer vision field implemented as loadable datasets using DataPipes. You can find all of those vision datasets here.
Note that these implementations are currently in the prototype phase, but they should be fully supported in the coming months. Nonetheless, they demonstrate the different ways DataPipes can be used for data loading.
Criteo 1TB Click Logs¶
The Criteo dataset contains feature values and click feedback for millions of display advertisements. It aims to benchmark algorithms for click through rate (CTR) prediction. You can find a prototype stage implementation of the dataset with DataPipes in TorchRec.