def save_checkpoint(self, filename, extra_state):
        """Save all training state in a checkpoint file."""
        extra_state['rng_tracker_states'] \
            = get_cuda_rng_tracker().get_states()
        super().save_checkpoint(filename, extra_state)
Save all training state in a checkpoint file.
decoding/IAD/fairseq/fairseq/model_parallel/megatron_trainer.py