def string_to_int_mapper(keys_to_map, mapping, num_oov_buckets=1, suffix="_id"):
  """Creates a mapping function to convert strings to ints in a tf.data.Dataset.

  For `dataset` outputs of type `str`, uses the list of strings in the given
  input `mapping` to look up the strings using tf.contrib.lookup and convert
  them to same-shape tensors of size tf.int32.

  Example:
    vocab = ['the', 'fox', 'jumped']
    dataset = dataset.map(string_to_int_mapper(['words'], mapping=vocab))
    dataset['words_id']  # <-- 'the' is mapped to 0, 'fox' to 1, etc...

  Args:
    keys_to_map: List of strings that are keys for tf.string Tensors to lookup.
    mapping: List of strings (or string tensors) to do the lookup. If the
        mapping is already a lookup table, then we directly use it.
    num_oov_buckets: Number of OOV buckets to use (default = 1).
    suffix: String to append to the given keys to indicate the mapped Tensors.

  Returns:
    _mapper: A mapping function that can be used with the tf.data.Dataset API.
  """
  if isinstance(mapping, LookupInterface):
    table = mapping
  else:
    table = contrib_lookup.index_table_from_tensor(
        mapping=mapping, num_oov_buckets=num_oov_buckets)

  def _mapper(dataset):
    for k in keys_to_map:
      dataset[k + suffix] = tf.to_int32(table.lookup(dataset[k]))
    return dataset
  return _mapper
Creates a mapping function to convert strings to ints in a tf.data.Dataset.

  For `dataset` outputs of type `str`, uses the list of strings in the given
  input `mapping` to look up the strings using tf.contrib.lookup and convert
  them to same-shape tensors of size tf.int32.

  Example:
    vocab = ['the', 'fox', 'jumped']
    dataset = dataset.map(string_to_int_mapper(['words'], mapping=vocab))
    dataset['words_id']  # <-- 'the' is mapped to 0, 'fox' to 1, etc...

  Args:
    keys_to_map: List of strings that are keys for tf.string Tensors to lookup.
    mapping: List of strings (or string tensors) to do the lookup. If the
        mapping is already a lookup table, then we directly use it.
    num_oov_buckets: Number of OOV buckets to use (default = 1).
    suffix: String to append to the given keys to indicate the mapped Tensors.

  Returns:
    _mapper: A mapping function that can be used with the tf.data.Dataset API.
  
language/common/inputs/dataset_utils.py