Should caching work with streaming? Or may I have stumbled on a Vercel SDK + Portkey bug?

Hi! I am noticing that the system prompt is missing in the logging when using Anthropic and Portkey as a proxy. Anthropic uses the parameter system rather than a system message like OpenAI does. I think you might not be picking up any system parameter when working as proxy mode. This also affects reported token count

Additionally, the response is JSON-only

@Vrushank | Portkey How do I cancel my Portkey account?

Thanks for the quick reply 🙂

Yeh fair enough. Just note that @ai-sdk/openai does some zod schema checking and Portkey's response does not match the expected format. That's why I use createAnthropic rather than createOpenAI

Also, it should state "received" instead of "recieved"

Okay, good to know. Bypassing your Gateway for now in dev

When I copy from the UI to anywhere else (variables section, PyCharm, etc..), new lines are added. This prevents me from using the Playground properly, as I would like to copy-paste between my IDE and this UI.

I am using Chromium Version 123.0.6312.105 (Official Build) snap (64-bit)

Just a suggestion; I used Azure CLI to create capacity for all relevant models to me, in many different regions. We do not consume that much quota, but I just did not want to worry about rate limits in burst scenario's.

I ended up with 50ish Azure deployments using terraform, which translates to 50 virtual keys. I did not want enter these by-hand, therefore I recreated the POST request from the front-end.

My suggestion: An API to manage Portkey virtual keys / configs / prompts

I do not seem to be able to create a virtual key for Azure - Dall-E 3.0. It reports You do not have enough permissions to execute this request across all regions (swedencentral, eastus, australiaeast)

It works for embedding and chat completion models across many regions, but not for image. Am I missing something?