left_pad_sequence

torchtune.data.left_pad_sequence(sequences: List[Tensor], batch_first: bool = False, padding_value: float = 0) → Tensor

This function is identical to torch.nn.utils.rnn.pad_sequence(), but instead pads a list of variable length Tensors from the left to the length of the longest sequence.

Note

This function returns a Tensor of size T x B x * or B x T x * where T is the length of the longest sequence. This function assumes trailing dimensions and type of all the Tensors in sequences are same.

Parameters:

• sequences (List[torch.Tensor]) – list of variable length sequences.

• batch_first (bool) – if True, the output will be in B x T x * format, T x B x * otherwise. Default False.

• padding_value (float) – value for padded elements. Default: 0.

Returns:

Tensor of size T x B x * if batch_first is False. Tensor of size B x T x * otherwise

Example

>>> a = torch.tensor([1, 2, 3])
>>> b = torch.tensor([4, 5, 6, 7])
>>> c = torch.tensor([8, 9, 10, 11, 12])
>>> left_pad_sequence([a, b, c], batch_first=True, padding_value=0)
tensor([[ 0,  0,  1,  2,  3],
        [ 0,  4,  5,  6,  7],
        [ 8,  9, 10, 11, 12]])