Filter

class torchdata.datapipes.iter.Filter(datapipe: IterDataPipe, filter_fn: Callable, drop_empty_batches: Optional[bool] = None, input_col=None)

Filters out elements from the source datapipe according to input filter_fn (functional name: filter).

Parameters:

• datapipe – Iterable DataPipe being filtered

• filter_fn – Customized function mapping an element to a boolean.

• drop_empty_batches (Deprecated) – By default, drops a batch if it is empty after filtering instead of keeping an empty list

• input_col –

  Index or indices of data which filter_fn is applied, such as:

  • None as default to apply filter_fn to the data directly.

  • Integer(s) is used for list/tuple.

  • Key(s) is used for dict.

Example

>>> from torchdata.datapipes.iter import IterableWrapper
>>> def is_even(n):
...     return n % 2 == 0
>>> dp = IterableWrapper(range(5))
>>> filter_dp = dp.filter(filter_fn=is_even)
>>> list(filter_dp)
[0, 2, 4]