def multistep_forward_view(rewards, pcontinues, state_values, lambda_,
                           back_prop=True, sequence_lengths=None,
                           name="multistep_forward_view_op"):
  """Evaluates complex backups (forward view of eligibility traces).

    ```python
    result[t] = rewards[t] +
        pcontinues[t]*(lambda_[t]*result[t+1] + (1-lambda_[t])*state_values[t])
    result[last] = rewards[last] + pcontinues[last]*state_values[last]
    ```

    This operation evaluates multistep returns where lambda_ parameter controls
    mixing between full returns and boostrapping. It is users responsibility
    to provide state_values. Depending on how state_values are evaluated this
    function can evaluate targets for Q(lambda), Sarsa(lambda) or some other
    multistep boostrapping algorithm.

    More information about a forward view is given here:
      http://incompleteideas.net/sutton/book/ebook/node74.html

    Please note that instead of evaluating traces and then explicitly summing
    them we instead evaluate mixed returns in the reverse temporal order
    by using the recurrent relationship given above.

    The parameter lambda_ can either be a constant value (e.g for Peng's
    Q(lambda) and Sarsa(_lambda)) or alternatively it can be a tensor containing
    arbitrary values (Watkins' Q(lambda), Munos' Retrace, etc).

    The result of evaluating this recurrence relation is a weighted sum of
    n-step returns, as depicted in the diagram below. One strategy to prove this
    equivalence notes that many of the terms in adjacent n-step returns
    "telescope", or cancel out, when the returns are summed.

    Below L3 is lambda at time step 3 (important: this diagram is 1-indexed, not
    0-indexed like Python). If lambda is scalar then L1=L2=...=Ln.
    g1,...,gn are discounts.

    ```
    Weights:  (1-L1)        (1-L2)*l1      (1-L3)*l1*l2  ...  L1*L2*...*L{n-1}
    Returns:    |r1*(g1)+     |r1*(g1)+      |r1*(g1)+          |r1*(g1)+
              v1*(g1)         |r2*(g1*g2)+   |r2*(g1*g2)+       |r2*(g1*g2)+
                            v2*(g1*g2)       |r3*(g1*g2*g3)+    |r3*(g1*g2*g3)+
                                           v3*(g1*g2*g3)               ...
                                                                |rn*(g1*...*gn)+
                                                              vn*(g1*...*gn)
    ```

  Args:
    rewards: Tensor of shape `[T, B]` containing rewards.
    pcontinues: Tensor of shape `[T, B]` containing discounts.
    state_values: Tensor of shape `[T, B]` containing state values.
    lambda_: Mixing parameter lambda.
        The parameter can either be a scalar or a Tensor of shape `[T, B]`
        if mixing is a function of state.
    back_prop: Whether to backpropagate.
    sequence_lengths: Tensor of shape `[B]` containing sequence lengths to be
      (reversed and then) summed, same as in `scan_discounted_sum`.
    name: Sets the name_scope for this op.

  Returns:
      Tensor of shape `[T, B]` containing multistep returns.
  """
  with tf.name_scope(name, values=[rewards, pcontinues, state_values]):
    # Regroup:
    #   result[t] = (rewards[t] + pcontinues[t]*(1-lambda_)*state_values[t]) +
    #               pcontinues[t]*lambda_*result[t + 1]
    # Define:
    #   sequence[t] = rewards[t] + pcontinues[t]*(1-lambda_)*state_values[t]
    #   discount[t] = pcontinues[t]*lambda_
    # Substitute:
    #   result[t] = sequence[t] + discount[t]*result[t + 1]
    # Boundary condition:
    #   result[last] = rewards[last] + pcontinues[last]*state_values[last]
    # Add and subtract the same quantity at BC:
    #   state_values[last] =
    #       lambda_*state_values[last] + (1-lambda_)*state_values[last]
    # This makes:
    #   result[last] =
    #       (rewards[last] + pcontinues[last]*(1-lambda_)*state_values[last]) +
    #       pcontinues[last]*lambda_*state_values[last]
    # Substitute in definitions for sequence and discount:
    #   result[last] = sequence[last] + discount[last]*state_values[last]
    # Define:
    #   initial_value=state_values[last]
    # We get the following recurrent relationship:
    #   result[last] = sequence[last] + decay[last]*initial_value
    #   result[k] = sequence[k] + decay[k] * result[k + 1]
    # This matches the form of scan_discounted_sum:
    #   result = scan_sum_with_discount(sequence, discount,
    #                                   initial_value = state_values[last])
    sequence = rewards + pcontinues * state_values * (1 - lambda_)
    discount = pcontinues * lambda_
    return scan_discounted_sum(sequence, discount, state_values[-1],
                               reverse=True, sequence_lengths=sequence_lengths,
                               back_prop=back_prop)
Evaluates complex backups (forward view of eligibility traces).

    ```python
    result[t] = rewards[t] +
        pcontinues[t]*(lambda_[t]*result[t+1] + (1-lambda_[t])*state_values[t])
    result[last] = rewards[last] + pcontinues[last]*state_values[last]
    ```

    This operation evaluates multistep returns where lambda_ parameter controls
    mixing between full returns and boostrapping. It is users responsibility
    to provide state_values. Depending on how state_values are evaluated this
    function can evaluate targets for Q(lambda), Sarsa(lambda) or some other
    multistep boostrapping algorithm.

    More information about a forward view is given here:
      http://incompleteideas.net/sutton/book/ebook/node74.html

    Please note that instead of evaluating traces and then explicitly summing
    them we instead evaluate mixed returns in the reverse temporal order
    by using the recurrent relationship given above.

    The parameter lambda_ can either be a constant value (e.g for Peng's
    Q(lambda) and Sarsa(_lambda)) or alternatively it can be a tensor containing
    arbitrary values (Watkins' Q(lambda), Munos' Retrace, etc).

    The result of evaluating this recurrence relation is a weighted sum of
    n-step returns, as depicted in the diagram below. One strategy to prove this
    equivalence notes that many of the terms in adjacent n-step returns
    "telescope", or cancel out, when the returns are summed.

    Below L3 is lambda at time step 3 (important: this diagram is 1-indexed, not
    0-indexed like Python). If lambda is scalar then L1=L2=...=Ln.
    g1,...,gn are discounts.

    ```
    Weights:  (1-L1)        (1-L2)*l1      (1-L3)*l1*l2  ...  L1*L2*...*L{n-1}
    Returns:    |r1*(g1)+     |r1*(g1)+      |r1*(g1)+          |r1*(g1)+
              v1*(g1)         |r2*(g1*g2)+   |r2*(g1*g2)+       |r2*(g1*g2)+
                            v2*(g1*g2)       |r3*(g1*g2*g3)+    |r3*(g1*g2*g3)+
                                           v3*(g1*g2*g3)               ...
                                                                |rn*(g1*...*gn)+
                                                              vn*(g1*...*gn)
    ```

  Args:
    rewards: Tensor of shape `[T, B]` containing rewards.
    pcontinues: Tensor of shape `[T, B]` containing discounts.
    state_values: Tensor of shape `[T, B]` containing state values.
    lambda_: Mixing parameter lambda.
        The parameter can either be a scalar or a Tensor of shape `[T, B]`
        if mixing is a function of state.
    back_prop: Whether to backpropagate.
    sequence_lengths: Tensor of shape `[B]` containing sequence lengths to be
      (reversed and then) summed, same as in `scan_discounted_sum`.
    name: Sets the name_scope for this op.

  Returns:
      Tensor of shape `[T, B]` containing multistep returns.
  
trfl/sequence_ops.py