max_tokens=200_000, since max_tokens parameter was required so we decided to set it to a big number.Error code: 400 - {\'error\': {\'message\': \'bedrock error: The maximum tokens you requested exceeds the model limit of 4096. Try again with a maximum tokens value that is lower than 4096.\', \'type\': None, \'param\': None, \'code\': None}, \'provider\': \'bedrock\'}max_tokens param or set it to 4096 or lower, the request is successful.max_tokens=200_000 the request passes successfully.max_tokens_val > max_tokens_max_allowed_val.max_tokens took place.production, staging etc). How do I use it over the prompt render api?Auto-Instrumentation heading. The image is broken and the instrumentation field does not exist in the SDK. What am I missing? 😄 {
"strategy": {
"mode": "fallback",
"on_status_codes": [
401,
500,
503,
520,
524
]
},
"request_timeout": 360000,
"targets": [
{
"virtual_key": "open-ai-virtual-xxxx",
"override_params": {
"model": "gpt-4o-2024-08-06"
},
"request_timeout": 12,
"retry": {
"attempts": 1,
"on_status_codes": [
429,
408
]
}
},
{
"virtual_key": "anthropic-api-k-xxxx",
"override_params": {
"model": "claude-3-7-sonnet-20250219"
},
"request_timeout": 120000
},
{
"virtual_key": "anthropic-api-k-xxxx",
"override_params": {
"model": "claude-3-5-sonnet-20241022"
},
"request_timeout": 120000
}
],
"cache": {
"mode": "simple",
"max_age": 6
}
}
....{
"status": 408,
"headers": {
"content-type": "application/json",
"x-portkey-cache-status": "MISS",
"x-portkey-last-used-option-index": "config.targets[0]",
"x-portkey-provider": "openai",
"x-portkey-retry-attempt-count": "0",
"x-portkey-trace-id": "9ac0fc87-562c-4b42-92e6-ad3cdb100880"
},
"body": {
"error": {
"message": "Request exceeded the timeout sent in the request: 12ms",
"type": "timeout_error",
"param": null,
"code": null
}
},
"responseTime": 1851,
"lastUsedOptionJsonPath": "config.targets[0]"
}2025-02-27 08:09:14.751
stream=stdout
2025-02-27 13:09:14.737 | ERROR | ai_core.llms.open_ai_wrapper:generate_text_response_async:436 | [Portkey Gateway] Unexpected error while calling Portkey Gateway with config pc-opeai-5393d5: 'ConnectTimeout' object has no attribute 'response'
2025-02-27 08:09:14.730
stream=stdout
AttributeError: 'ConnectTimeout' object has no attribute 'response'
2025-02-27 08:09:14.730
stream=stdout
└ 1
2025-02-27 08:09:14.730
stream=stdout
│ └
2025-02-27 08:09:14.730
stream=stdout
│ │ └
2025-02-27 08:09:14.730
stream=stdout
if remaining_retries > 0 and self._should_retry(err.response):
2025-02-27 08:09:14.730
stream=stdout
File "/app/.venv/lib/python3.12/site-packages/portkey_ai/_vendor/openai/_base_client.py", line 1601, in _request
2025-02-27 08:09:14.730
stream=stdout
└
2025-02-27 08:09:14.730
stream=stdout
│ └
2025-02-27 08:09:14.730
stream=stdout
return await self._request(
2025-02-27 08:09:14.730
stream=stdout
File "/app/.venv/lib/python3.12/site-packages/portkey_ai/_vendor/openai/_base_client.py", line 1554, in request
2025-02-27 08:09:14.730
stream=stdout
└
2025-02-27 08:09:14.730
stream=stdout
│ └
2025-02-27 08:09:14.730
stream=stdout
│ │ └
2025-02-27 08:09:14.730
stream=stdout
│ │ │ └ FinalRequestOptions(method='post', url='/chat/completions', params={}, headers={'X-Stainless-Raw-Response': 'true'}, max_retr...
2025-02-27 08:09:14.730
stream=stdout
│ │ │ │ └ False
2025-02-27 08:09:14.730
stream=stdout
│ │ │ │ │ └ openai.AsyncStream[portkey_ai._vendor.openai.types.chat.chat_completion_chunk.ChatCompletionChunk]
2025-02-27 08:09:14.730
stream=stdout
return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
2025-02-27 08:09:14.730
stream=stdout
File "/app/.venv/lib/python3.12/site-packages/portkey_ai/_vendor/openai/_base_client.py", line 1860, in post