ray.data.preprocessor.Preprocessor.transform#

Preprocessor.transform(ds: Dataset, *, batch_size: int | None = None, num_cpus: float | None = None, memory: float | None = None, concurrency: int | None = None) → Dataset[source]#

Transform the given dataset.

Parameters:

  • ds – Input Dataset.

  • batch_size – [experimental] Advanced configuration for adjusting input size for each worker.

  • num_cpus – [experimental] The number of CPUs to reserve for each parallel map worker.

  • memory – [experimental] The heap memory in bytes to reserve for each parallel map worker.

  • concurrency – [experimental] The maximum number of Ray workers to use concurrently.

Returns:

The transformed Dataset.

Return type:

ray.data.Dataset

Raises:

PreprocessorNotFittedException – if fit is not called yet.