def batch_word_to_char_ids(words, word_length):
  """Batched version of word_to_char_ids.

  This is a deterministic function that should be computed during preprocessing.
  We pin this op to the CPU anyways to be safe, since it is slower on GPUs.

  Args:
    words: <string> [...]
    word_length: Number of bytes to include per word.

  Returns:
    char_ids: <int32> [..., word_length]
  """
  with tf.device("/cpu:0"):
    flat_words = tf.reshape(words, [-1])
    flat_char_ids = tf.map_fn(
        fn=partial(word_to_char_ids, word_length=word_length),
        elems=flat_words,
        dtype=tf.int32,
        back_prop=False)

  char_ids = tf.reshape(flat_char_ids,
                        tensor_utils.shape(words) + [word_length])
  return char_ids
Batched version of word_to_char_ids.

  This is a deterministic function that should be computed during preprocessing.
  We pin this op to the CPU anyways to be safe, since it is slower on GPUs.

  Args:
    words: <string> [...]
    word_length: Number of bytes to include per word.

  Returns:
    char_ids: <int32> [..., word_length]
  
language/common/inputs/char_utils.py