Skip to content

ProcessingS3Output's s3_uri to be an optional field #5559

@rauldiaz

Description

@rauldiaz

Describe the feature you'd like
In the SDK v2, ProcessingOutput has a destination parameter to be either an S3 path where users want to write their results, but they can set it to None when they want to delegate the storage to SageMaker. This option stores the results in an S3 prefix partitioned by job_name, step_name, and output_name, which is quite convenient for traceability and to store intermediate results of a pipeline.

How would this feature be used? Please describe.
Much like v2, v3 should also support this feature. Users may not want to fix the destination path for intermediate results, so ideally the same behaviour should apply to v3 by leaving ProcessingS3Output's s3_uri field as None. Currently this field is mandatory and it's unclear whether users can reproduce the v2 None behaviour.

Describe alternatives you've considered
No alternatives considered so far other than sticking to v2.

Additional context
N/A

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions