def step_update(self, num_updates):
        """Update the learning rate after each update."""
        if num_updates < self.cfg.warmup_updates:
            self.lr = self.cfg.warmup_init_lr + num_updates * self.lr_step
        else:
            self.lr = self.decay_factor * num_updates ** -0.5
        self.optimizer.set_lr(self.lr)
        return self.lr
Update the learning rate after each update.
structured_prompting/fairseq-version/fairseq/fairseq/optim/lr_scheduler/inverse_square_root_schedule.py