ProcessingStep serializes pipeline parameters as strings in SDK v2.243.2 #5137
Unanswered
Srinivasaraonarlajarla
asked this question in
Help
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi team,
I'm encountering a regression in the SageMaker Python SDK when using ProcessingStep with pipeline parameters as job_arguments.
SDK Version Comparison :-
This works as expected in version 2.229.0, but breaks in 2.243.2.
Working (v2.229.0)
`
month = ParameterString(name="CurrentMonth", default_value=curr_month)
year = ParameterString(name= "CurrentYear", default_value =curr_year)
preprocess_step = ProcessingStep(
processor=preprocess_processor,
name='DataPreprocessing',
code=os.path.join(PROJECT_DIR, 'manifest_file_creation.py'),
job_arguments=['--year', year, '--month', month],
)
`
Relevant part of the generated pipeline definition:
"ContainerArguments": [ "--year", { "Get": "Parameters.CurrentYear" }, "--month", { "Get": "Parameters.CurrentMonth" } ]
Broken (v2.243.2):-
"ContainerArguments": [ "--year", "{\"Get\": \"Parameters.CurrentYear\"}", "--month", "{\"Get\": \"Parameters.CurrentMonth\"}" ]
Here, the parameter references are serialized as strings, rather than objects, causing the pipeline to treat them as literal strings instead of dynamically resolving them during execution.
Expected Behavior:-
{ "Get": "Parameters.CurrentYear" }
Am I missing something in v2.243.2 ?
Beta Was this translation helpful? Give feedback.
All reactions