def reset_with_noops(env, index):
      """Resets environment and applies random number of NOOP actions on it."""
      obs = reset_with_initial_state(env, index)
      try:
        num_noops = random.randint(1, self.max_num_noops)
      except ValueError:
        num_noops = 0

      for _ in range(num_noops):
        (obs, _, done, _) = env.step(self.noop_action)
        if done:
          obs = reset_with_initial_state(env, index)

      return obs
Resets environment and applies random number of NOOP actions on it.
tensor2tensor/data_generators/gym_env.py