def rotate_half(self, x):
        """Rotates half the hidden dims of the input."""
        x1 = x[..., : x.shape[-1] // 2]
        x2 = x[..., x.shape[-1] // 2 :]
        return torch.cat((-x2, x1), dim=-1)
Rotates half the hidden dims of the input.
intel_extension_for_pytorch/transformers/models/reference/fusions/mha_fusion.py