def construct_input_ids(inputs,
                        tokenize_fn,
                        cls_token=_CLS_TOKEN,
                        sep_token=_SEP_TOKEN,
                        pad_token=_PAD_TOKEN):
  """Perform the transformations to produce input_ids given untokenized strings.

  Given a Tensor of untokenized strings and a tokenization function, perform all
  the necessary transformation for producing the input_ids input used in BERT.
  This includes tokenization (token and wordpiece), and concat of special
  tokens.

  Args:
   inputs: [<tf.string>] a list of string Tensors each containing an untokenized
     string segment.
   tokenize_fn: a function that takes a tensor of strings and returns a
     RaggedTensor (w/ ragged_rank=2 of tokenized wordpieces).
   cls_token: string, the string used to represent the CLS token, default:
     '[CLS]'
   sep_token: string, the string used to represent the SEP token, default:
     '[SEP]'
   pad_token: string or int, the value used to pad tensors. Default: '[PAD]'

  Returns:
    a Tensor of input_ids w/ necessary padding.
  """
  assert isinstance(inputs, list) and inputs
  assert isinstance(cls_token, str)
  assert isinstance(sep_token, str)

  # TODO(hterry): add truncate budget/logic

  wordpieces = [tokenize_fn(segment) for segment in inputs]

  # Collapse the wordpiece dimension
  collapsed_wordpieces = [collapse_dims(rt) for rt in wordpieces]

  # Concat the [CLS], [SEP] tokens to get the input to BERT model_fn
  cls_tokens = tf.reshape(
      tf.tile([cls_token], [collapsed_wordpieces[0].nrows()]),
      [collapsed_wordpieces[0].nrows(), 1])
  sep_tokens = tf.reshape(
      tf.tile([sep_token], [collapsed_wordpieces[0].nrows()]),
      [collapsed_wordpieces[0].nrows(), 1])
  result = [cls_tokens]
  for rt in collapsed_wordpieces:
    result.append(rt)
    result.append(sep_tokens)
  return tf.concat(result, axis=1).to_tensor(pad_token)
Perform the transformations to produce input_ids given untokenized strings.

  Given a Tensor of untokenized strings and a tokenization function, perform all
  the necessary transformation for producing the input_ids input used in BERT.
  This includes tokenization (token and wordpiece), and concat of special
  tokens.

  Args:
   inputs: [<tf.string>] a list of string Tensors each containing an untokenized
     string segment.
   tokenize_fn: a function that takes a tensor of strings and returns a
     RaggedTensor (w/ ragged_rank=2 of tokenized wordpieces).
   cls_token: string, the string used to represent the CLS token, default:
     '[CLS]'
   sep_token: string, the string used to represent the SEP token, default:
     '[SEP]'
   pad_token: string or int, the value used to pad tensors. Default: '[PAD]'

  Returns:
    a Tensor of input_ids w/ necessary padding.
  
language/conpono/reconstruct/preprocess.py