When running RAGChecker 0.1.9 using ragchecker.cli in an EC2 instance, I ran into an issue where I got a rather non-informative error from botocore via litellm, and the process gets stuck in an infinite loop waiting for 10 seconds and then retrying, and my logs contain this over and over:
[sleep 10 seconds]
litellm.APIConnectionError: 'NoneType' object has no attribute 'split'
Traceback (most recent call last):
File "/usr/local/lib/python3.12/site-packages/litellm/main.py", line 2788, in completion
response = bedrock_converse_chat_completion.completion(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/litellm/llms/bedrock/chat/converse_handler.py", line 385, in completion
prepped = self.get_request_headers(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/litellm/llms/bedrock/base_aws_llm.py", line 677, in get_request_headers
sigv4.add_auth(request)
File "/usr/local/lib/python3.12/site-packages/botocore/auth.py", line 430, in add_auth
canonical_request = self.canonical_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/botocore/auth.py", line 371, in canonical_request
cr.append(self.canonical_headers(headers_to_sign) + '\n')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/botocore/auth.py", line 306, in canonical_headers
value = ','.join(
^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/botocore/auth.py", line 307, in <genexpr>
self._header_value(v) for v in headers_to_sign.get_all(key)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/botocore/auth.py", line 318, in _header_value
return ' '.join(value.split())
^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'split'
[sleep 10 seconds]
litellm.APIConnectionError: 'NoneType' object has no attribute 'split'
I tracked the issue down, it's the "Host" header which is None and this is because RAGChecker passes bedrock/meta.llama3-70b-instruct-v1:0 as the api_base to RefChecker by default:
|
parser.add_argument( |
|
'--extractor_api_base', type=str, default="bedrock/meta.llama3-70b-instruct-v1:0", |
|
help='API base URL for the extractor if using vllm deployed open source LLMs. Default: bedrock/meta.llama3-70b-instruct-v1:0' |
|
) |
It seems that this is not the intended behavior, and was likely added by mistake in commit 671b71b where a default value was added to --extractor_api_base when it was meant for --checker_name.
I can work-around it by by passing in --extractor_api_base explicitly as https://bedrock-runtime.<region>.amazonaws.com. I would prefer to not have to do this, however, I would rather rely on the default AWS SDK behavior which uses the default region and api base.
When running RAGChecker 0.1.9 using
ragchecker.cliin an EC2 instance, I ran into an issue where I got a rather non-informative error from botocore via litellm, and the process gets stuck in an infinite loop waiting for 10 seconds and then retrying, and my logs contain this over and over:I tracked the issue down, it's the "Host" header which is None and this is because RAGChecker passes
bedrock/meta.llama3-70b-instruct-v1:0as theapi_baseto RefChecker by default:RAGChecker/ragchecker/cli.py
Lines 24 to 27 in 6091f08
It seems that this is not the intended behavior, and was likely added by mistake in commit 671b71b where a default value was added to
--extractor_api_basewhen it was meant for--checker_name.I can work-around it by by passing in
--extractor_api_baseexplicitly ashttps://bedrock-runtime.<region>.amazonaws.com. I would prefer to not have to do this, however, I would rather rely on the default AWS SDK behavior which uses the default region and api base.