def rotate_half(x):
    """Rotates half the hidden dims of the input."""
    x1 = x[..., : x.shape[-1] // 2]
    x2 = x[..., x.shape[-1] // 2:]
    return torch.cat((-x2, x1), dim=-1)
Rotates half the hidden dims of the input.
internvl_chat_llava/llava/model/multimodal_encoder/internvl_14b/modeling_qllama.py