-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Description
PySDK Version
- PySDK V2 (2.x)
- PySDK V3 (3.x)
Describe the bug
The _serialize_dict function in sagemaker-core/src/sagemaker/core/utils/utils.py uses a walrus operator with a truthiness check that drops empty string values from dictionaries:
def _serialize_dict(value: Dict) -> dict:
serialized_dict = {}
for k, v in value.items():
if serialize_result := serialize(v): # "" is falsy → key-value pair is dropped
serialized_dict.update({k: serialize_result})
return serialized_dict
Since "" is false in Python, any environment variable set to an empty string is silently removed from the serialized request. This makes it impossible to pass boolean CLI flags to containers that use empty-string env vars as a signal (e.g., the SageMaker vLLM DLC).
The same bug exists in _serialize_list (line 543) and _serialize_shape (line 560), where valid falsy values like 0, False, and "" are incorrectly dropped.
To reproduce
from sagemaker.core.utils.utils import serialize
# Simulates environment dict for a vLLM endpoint
env = {
"SM_VLLM_MODEL": "my-model",
"SM_VLLM_ENABLE_AUTO_TOOL_CHOICE": "", # boolean flag, must be empty string
"SM_VLLM_TOOL_CALL_PARSER": "hermes",
}
result = serialize(env)
print(result)
# Output: {'SM_VLLM_MODEL': 'my-model', 'SM_VLLM_TOOL_CALL_PARSER': 'hermes'}
# SM_VLLM_ENABLE_AUTO_TOOL_CHOICE is silently dropped!
Expected behavior
Empty strings are valid values for environment variables and should be preserved in the serialized output:
# Expected output:
{
'SM_VLLM_MODEL': 'my-model',
'SM_VLLM_ENABLE_AUTO_TOOL_CHOICE': '',
'SM_VLLM_TOOL_CALL_PARSER': 'hermes',
}
The SageMaker vLLM DLC entrypoint script (sagemaker_entrypoint.sh) relies on empty-string env vars for boolean flags like --enable-auto-tool-choice. The script checks if [ -n "$value" ] and only appends the value if non-empty, producing just the flag name. Setting the value to "true" instead produces --enable-auto-tool-choice true, which argparse rejects with unrecognized arguments: true.
Screenshots or logs
When using the SDK (empty string dropped, flag never reaches the container):
No --enable-auto-tool-choice flag in the vLLM command line
When setting the value to "true" as a workaround (flag reaches container but with invalid value):
api_server.py: error: unrecognized arguments: true
System information
A description of your system. Please provide:
- SageMaker Python SDK version: 3.4.1
- Framework name (eg. PyTorch) or algorithm (eg. KMeans): vLLM
- Framework version: 0.14.0
- Python version: 3.12.12
- CPU or GPU: GPU
- Custom Docker image (Y/N): N
Additional context
Suggested fix in sagemaker-core/src/sagemaker/core/utils/utils.py — use explicit None checks instead of truthiness: