Configuring Portkey with Sonnet v2 and AWS Bedrock

Question

I’m using Sonnet v2 with AWS Bedrock in the us-east-1 region. It seems that invocation is only supported with an inference profile for Sonnet v2, and I need to set this as the model ID: us.anthropic.claude-3-5-sonnet-20241022-v2:0.

However, in Portkey, all the model IDs start with anthropic.XXX instead of us.XXX. How do I configure portkey with inference profile.

Vrushank | Portkey · Answer

Hi @Het007 - looking into this. cc @visarg

Het007 · Answer

Thanks

Het007 · Answer

Attaching one link. I think this will be helpfullhttps://github.com/danny-avila/LibreChat/discussions/4571

Vaibhav · Answer

@Het007 @Vrushank | Portkey facing the same issue, how can we use this?

Het007 · Answer

@Vaibhav @Vrushank | Portkey any update on this issue. Thanks

Vrushank | Portkey · Answer

Sorry for the delay! Updating you today

Het007 · Answer

@Vrushank | Portkey any update on this..it is a blocker for me. Anthropic apis are not reliable so need bedrock integration for this.

Vrushank | Portkey · Answer

Got it - I am confirming asap

Vrushank | Portkey · Answer

@Het007 some context:

Vrushank | Portkey · Answer

We’ll add it in prompt management UI too

Het007 · Answer

Thanks @Vrushank | Portkey Let me try this.

Het007 · Answer

Sorry forgot to reply but this worked 🔥

Vrushank | Portkey · Answer

Awesome!

Welcome to Portkey Forum

Configuring Portkey with Sonnet v2 and AWS Bedrock