def zero_grad(self):
        """Clears the gradients of all optimized parameters."""
        for p in self.params:
            p.grad = None
        self.optimizer.zero_grad()
Clears the gradients of all optimized parameters.
understand_icl/fairseq/fairseq/optim/fairseq_optimizer.py