Welcome to Portkey Forum

Updated 3 days ago

Troubleshooting claude-3-7-sonnet-latest configuration

Hey friends, trying to get Claude 3.7 Sonnet to think, not working:. Any mistake in this, or Portkey limitation?

"body": { "model": "claude-3-7-sonnet-latest", "max_tokens": 8192, "thinking": { "type": "enabled", "budget_tokens": 16000 }, "stream": true, "stream_options": { "include_usage": true }, "tools": [ {tool definitions} ], "tool_choice": "auto", "messages": [ { "role": "system", "content": "You are.....
1
s
G
A
20 comments
we'll add support for this today
Awesome, will you stream back in the same way as antrhopic? Then I can already write the handling for it:

or await (const event of stream) { if (event.type === 'content_block_start') { console.log(\nStarting ${event.content_block.type} block...); } else if (event.type === 'content_block_delta') { if (event.delta.type === 'thinking_delta') { console.log(Thinking: ${event.delta.thinking}); } else if (event.delta.type === 'text_delta') { console.log(Response: ${event.delta.text});
{
"provider": "anthropic",
"api_key": " ",
"override_params": {
"thinking": {
"type": "enabled",
"budget_tokens": 2000
}
},
"retry": {
"attempts": 5
}
}

Just want to confirm that this Config set up will also work, whenever the new update is out? I guess Portkey needs to support the 'thinking' parameter?
I've commented out only my 'reasoning' button, I am eagerly awaiting this
Yes this would work
Sorry for the delay, we were discussing on how to standardise the transformer while being OpenAI compliant
Will this work with bedrock too? And when do you think this would be live
Hey I'm still not getting any reasoning out of Claude 3.7 at all @sega will you ping us when it's live? Or am I now missing something?
Hey @Gijs it's not live yet
we're adding support for it, but the reasoning chunks will be in the raw_response field because there is no corresponding openai field to map the response to,
more here: https://github.com/Portkey-AI/gateway/issues/964
I'll add documentation for this as well, this is a change from the way we've been doing thigs
Ok, whatever works as long as it comes in. I understand it's a pain in the ass.

*resumes nervous pacing
Do you have an ETA though? Because I could have already shipped it probably if I'd just gone direct to Anthropic for this one
it should be live by tomorrow at latest, the changes are raised, doing a round of sanity
looks like its still under review :/ waiting for this myself, but im using openrouter in the meanwhile
@sega how is it looking?
we're deciding on a unified api signature instead of sending raw response because raw response could be a bad case of API signature getting misused
Any idea on timeline?
Add a reply
Sign up and join the conversation on Discord