def get_data_parallel_world_size():
    """Return world size for the data parallel group."""
    if torch.distributed.is_available() and torch.distributed.is_initialized():
        return torch.distributed.get_world_size(group=get_data_parallel_group())
    else:
        return 0
Return world size for the data parallel group.
training/DeepSpeed-Domino/domino/parallel_state.py