def discard_equivalent_examples(self, split="test"):
    """Faster version to discard equivalent examples that already in training set."""
    to_delete = self.get_equivalent_ids(split_1="train", split_2=split)
    equivalent_examples = len(to_delete)
    for i_to_delete in sorted(to_delete, reverse=True):
      del self._data_pairs[split][i_to_delete]
      del self._template_identifiers[split][i_to_delete]
    return equivalent_examples
Faster version to discard equivalent examples that already in training set.
language/gscan/data/dataset.py