def get_tokens(self, idxs):
        """Normalize tokens by handling CTC blank, ASG replabels, etc."""
        idxs = (g[0] for g in it.groupby(idxs))
        idxs = filter(lambda x: x != self.blank, idxs)
        return torch.LongTensor(list(idxs))
Normalize tokens by handling CTC blank, ASG replabels, etc.
understand_icl/fairseq/examples/speech_recognition/w2l_decoder.py