ray.rllib.env.single_agent_episode.SingleAgentEpisode.set_rewards#

SingleAgentEpisode.set_rewards(*, new_data, at_indices: int | slice | List[int] | None = None, neg_index_as_lookback: bool = False) → None[source]#

Overwrites all or some of this Episode’s rewards with the provided data.

Note that an episode’s reward data cannot be written to directly as it is managed by a InfiniteLookbackBuffer object. Normally, individual, current rewards are added to the episode either by calling self.add_env_step or more directly (and manually) via self.rewards.append|extend(). However, for certain postprocessing steps, the entirety (or a slice) of an episode’s rewards might have to be rewritten, which is when self.set_rewards() should be used.

Parameters:

  • new_data – The new reward data to overwrite existing data with. This may be a list of individual reward(s) in case this episode is still not numpy’ized yet. In case this episode has already been numpy’ized, this should be a np.ndarray with a length exactly the size of the to-be-overwritten slice or segment (provided by at_indices).

  • at_indices – A single int is interpreted as one index, which to overwrite with new_data (which is expected to be a single reward). A list of ints is interpreted as a list of indices, all of which to overwrite with new_data (which is expected to be of the same size as len(at_indices)). A slice object is interpreted as a range of indices to be overwritten with new_data (which is expected to be of the same size as the provided slice). Thereby, negative indices by default are interpreted as “before the end” unless the neg_index_as_lookback=True option is used, in which case negative indices are interpreted as “before ts=0”, meaning going back into the lookback buffer.

  • neg_index_as_lookback – If True, negative values in at_indices are interpreted as “before ts=0”, meaning going back into the lookback buffer. For example, an episode with rewards = [4, 5, 6, 7, 8, 9], where [4, 5, 6] is the lookback buffer range (ts=0 item is 7), will handle a call to set_rewards(individual_reward, -1, neg_index_as_lookback=True) by overwriting the value of 6 in our rewards buffer with the provided “individual_reward”.

Raises:

IndexError – If the provided at_indices do not match the size of new_data.