def calculate_metrics(
    scored_annotations: List[ScoredPair],
    subsets: Optional[Dict[str, Callable[[ScoredPair], bool]]] = None,
) -> Dict[str, float]:
  """Computes metrics for the scored_annotations for each subset.

  Standard subsets can include all of the data, only the true positives labeled
  by the system, all positive examples, or only the human portion of the data.

  Args:
    scored_annotations: The list of scored annotations to calculate metrics
      over.
    subsets: A dictionary of subset names to filter functions that take a
      ScoredAnnotation and return true or false. If None, will run the single
      set consisting of all of the annotations.

  Returns:
    Dictionary of metric names to metric values, with metric names having the
      subset appended.
  """
  metrics = {}
  if not subsets:
    # Just process the entire set!
    subsets = {"all": lambda _: True}
  for subset_name, subset_filter in subsets.items():
    subset = [anno.score for anno in scored_annotations if subset_filter(anno)]
    metrics.update(_aggregate_subset_scores(subset, subset_name))
  metrics.update(
      compute_f1_from_labels([anno.score.label for anno in scored_annotations])
  )
  return metrics
Computes metrics for the scored_annotations for each subset.

  Standard subsets can include all of the data, only the true positives labeled
  by the system, all positive examples, or only the human portion of the data.

  Args:
    scored_annotations: The list of scored annotations to calculate metrics
      over.
    subsets: A dictionary of subset names to filter functions that take a
      ScoredAnnotation and return true or false. If None, will run the single
      set consisting of all of the annotations.

  Returns:
    Dictionary of metric names to metric values, with metric names having the
      subset appended.
  
language/diffqg/metrics.py