def calculateTrueGradient(self, value_w, value_s_tm1):
    """Calculates the true gradient over the batch.

    sum_k dq/dw_s = sum_k dq/da * da/dw_s
                  = w * sum_k da/dw_s

    Args:
      value_w: numpy.ndarray containing weights of the linear layer.
      value_s_tm1: state representation.

    Returns:
      The true_gradient of the test case.
    """
    dadws = np.zeros((value_w.shape[0],
                      np.product(self.w_s.get_shape().as_list())))
    for i in range(self.batch_size):
      dadws += np.vstack((np.hstack((value_s_tm1[i], np.zeros(3))),
                          np.hstack((np.zeros(3), value_s_tm1[i]))))
    true_grad = np.dot(np.transpose(value_w), dadws)
    true_grad = -np.transpose(np.reshape(
        true_grad, self.w_s.get_shape().as_list()[::-1]))
    return true_grad
Calculates the true gradient over the batch.

    sum_k dq/dw_s = sum_k dq/da * da/dw_s
                  = w * sum_k da/dw_s

    Args:
      value_w: numpy.ndarray containing weights of the linear layer.
      value_s_tm1: state representation.

    Returns:
      The true_gradient of the test case.
    
trfl/dpg_ops_test.py