def count_and_skip(generator, name):
  """Count the number of items in the generator, skip already counted ones.

  This function counts the number of processed examples and puts it into
  the global variable `counters`. This variable can be saved and restored,
  and if restored, this function will skip examples until the restored counter
  is reached. When the data generator is deterministic, this allows to restore
  the data reading process from a checkpoint.

  Args:
    generator: generator for examples in the dataset.
    name: string, a unique id that we use to count the examples

  Yields:
    The examples from generator but first skip the number specified in the
    global variable counters[name] and next increment this variable every
    time a new example appears.
  """
  global data_counters
  local_counter = 0
  for example in generator:
    local_counter += 1
    # This check must be inside the loop due to asynchronous initializations.
    if name not in data_counters:
      data_counters[name] = 0
    if local_counter > data_counters[name]:
      data_counters[name] += 1
      yield example
Count the number of items in the generator, skip already counted ones.

  This function counts the number of processed examples and puts it into
  the global variable `counters`. This variable can be saved and restored,
  and if restored, this function will skip examples until the restored counter
  is reached. When the data generator is deterministic, this allows to restore
  the data reading process from a checkpoint.

  Args:
    generator: generator for examples in the dataset.
    name: string, a unique id that we use to count the examples

  Yields:
    The examples from generator but first skip the number specified in the
    global variable counters[name] and next increment this variable every
    time a new example appears.
  
trax/data/inputs.py