def marginal_log_loss(logits, is_correct):
  """Loss based on the negative marginal log-likelihood."""
  # []
  log_numerator = tf.reduce_logsumexp(logits + mask_to_score(is_correct), -1)
  log_denominator = tf.reduce_logsumexp(logits, -1)
  return log_denominator - log_numerator
Loss based on the negative marginal log-likelihood.
language/orqa/models/orqa_model.py