def entropy(p):
    """Compute the entropy of a probability distribution"""
    plogp = p * torch.log(p)
    plogp[p == 0] = 0
    return -plogp.sum(dim=-1)
Compute the entropy of a probability distribution
dpkd/transformers/examples/research_projects/bertology/run_bertology.py