def data_streams(dataset_name,
                 data_dir=None,
                 preprocess_fn=no_preprocess,
                 bare_preprocess_fn=None,
                 shuffle_buffer_size=1024,
                 eval_holdout_size=0,
                 input_name=None,
                 target_name=None):
  """Creates `(train, eval)` data sources from ``dataset_name``.

  Args:
    dataset_name: Name of dataset belonging to TFDS or T2T. T2T dataset names
      must start with ``'t2t_'``.
    data_dir: Directory where the data is located.
    preprocess_fn: Function to use for pre-processing after appending targets to
      inputs.
    bare_preprocess_fn: Function to use for pre-processing before appending
      targets to inputs.
    shuffle_buffer_size: Size of the shuffle buffer.
    eval_holdout_size: If greater than 0, specifies a fraction of training data
      to siphon off and use as eval data, in place of an separate eval split.
    input_name: Name of the inputs from the dictionary.
    target_name: Name of the outputs either from the dictionary or as a result
      of post-processing.

  Returns:
    A pair of functions, `(f, g)` for use as data sources; call `f()` to get an
    iterator of training data samples, and call `g()` to get an iterator of eval
    data samples.
  """
  data_dir = download_and_prepare(dataset_name, data_dir)

  cache = []

  def stream(which):
    """Create the stream, cache TF streams if needed."""
    if not cache:
      cache.append(
          _train_and_eval_streams(dataset_name, data_dir, preprocess_fn,
                                  bare_preprocess_fn, shuffle_buffer_size,
                                  eval_holdout_size, input_name, target_name))

    (train_ds, eval_ds, input_name_c) = cache[0]
    dataset = eval_ds if which == 'eval' else train_ds
    return dataset_to_stream(dataset, input_name_c)

  train_stream = lambda: stream('train')
  eval_stream = lambda: stream('eval')
  return train_stream, eval_stream
Creates `(train, eval)` data sources from ``dataset_name``.

  Args:
    dataset_name: Name of dataset belonging to TFDS or T2T. T2T dataset names
      must start with ``'t2t_'``.
    data_dir: Directory where the data is located.
    preprocess_fn: Function to use for pre-processing after appending targets to
      inputs.
    bare_preprocess_fn: Function to use for pre-processing before appending
      targets to inputs.
    shuffle_buffer_size: Size of the shuffle buffer.
    eval_holdout_size: If greater than 0, specifies a fraction of training data
      to siphon off and use as eval data, in place of an separate eval split.
    input_name: Name of the inputs from the dictionary.
    target_name: Name of the outputs either from the dictionary or as a result
      of post-processing.

  Returns:
    A pair of functions, `(f, g)` for use as data sources; call `f()` to get an
    iterator of training data samples, and call `g()` to get an iterator of eval
    data samples.
  
trax/data/tf_inputs.py