def collate_batch(batch: list[dict], tokenizer: "PreTrainedTokenizer") -> dict[str, "torch.Tensor"]:
        """Collate a batch of samples into a padded batch of tensors.

        Add padding to the input_ids, attention_mask and labels.
        Each example can be a dictionary with inputs (and optionally labels).
        """
        from torch.nn.utils.rnn import pad_sequence

        input_ids = [sample["input_ids"] for sample in batch]
        attention_mask = None
        if "attention_mask" in batch[0]:
            attention_mask = [sample["attention_mask"] for sample in batch]
        label_col = "labels" if "labels" in batch[0] else "label"
        if label_col not in batch[0]:
            # labels is the same as input_ids, the trainer left shifts the labels when computing loss
            labels = [input_id.clone() for input_id in input_ids]
        else:
            labels = [sample[label_col] for sample in batch]

        # apply padding and add to batch
        # need to worry about left or right padding?
        new_batch = {
            "input_ids": pad_sequence(input_ids, batch_first=True, padding_value=tokenizer.pad_token_id),
            # pad the labels with IGNORE_INDEX so that they are not used in loss computation
            # don't want to clone batched input_ids and ignore padding tokens in case eos token is used
            # as padding token
            "labels": pad_sequence(labels, batch_first=True, padding_value=IGNORE_INDEX),
        }
        if attention_mask:
            new_batch["attention_mask"] = pad_sequence(attention_mask, batch_first=True, padding_value=0)
        return new_batch
Collate a batch of samples into a padded batch of tensors.

        Add padding to the input_ids, attention_mask and labels.
        Each example can be a dictionary with inputs (and optionally labels).
        
olive/passes/pytorch/lora.py