padded_collate_sft¶
- torchtune.data.padded_collate_sft(batch: List[Dict[str, List[int]]], padding_idx: int = 0, ignore_idx: int = - 100) Dict[str, Tensor] [source]¶
Pad a batch of sequences to the longest sequence length in the batch, and convert integer lists to tensors.
- Parameters:
- Returns:
Collated input and label tensors.
- Return type:
Dict[str, torch.Tensor]
Example
>>> token_pairs = [ >>> {"tokens": [1, 2, 3], "labels": [4, 5, 6]}, >>> {"tokens": [7,], "labels": [10,]}, >>> ] >>> collated = padded_collate( >>> batch=token_pairs, >>> padding_idx=padding_idx, >>> ignore_idx=ignore_idx, >>> ) >>> collated["tokens"] >>> tensor([[1, 2, 3], [7, 0, 0]]) >>> collated["labels"] >>> tensor([[4, 5, 6], [10, -100, -100]])