def __init__(self, initial_scale):
        """Initialize scale value with the input initial scale."""
        assert initial_scale > 0.0
        self._scale = torch.cuda.FloatTensor([initial_scale])
Initialize scale value with the input initial scale.
training/DeepSpeed-Domino/domino/optimizer/grad_scaler.py