Interesting thanks i really want to setup the fallback from openai to anthropic

@Vrushank | Portkey still isn't working:

Attachment

And with openAI I get:

Attachment

ok wait, I think I found it

happy to get on a call and debug if needed!

Got this working with both openAI and anthropic 🎉

Awesome!!

Fallback worked!

Attachment

Awesome!! The timeout issue with streaming will be fixed very soon as well!

Team, sometimes I am getting errors like completion.create error: TypeError: t.content.map is not a function , while calling portkey.chat.completions.create
Can you please check.

Last was at 2024-07-19 17:58:44.201 IST

You're using the Portkey TS SDK?

yes

Think this is an issue i've seen before - @chandypaaji who can take a look!

For now, something like //@ts-ignore should work, but we'll push a fix for it

This is not a compile-time issue. I'm getting this at runtime.

Ok, would it be possible to share trace id for the call where you got this error? @visarg who can confirm if this could be due to some response payload issue

There are no logs on portkey! That is why I shared the time

Got it yeah sorry! It's failing before it can run

CChandeep | Portkey

Hey @vedu

Thanks for sharing this.

Can you share the code snippet on how are you implementing the portkey.chat.completions.create call?
That could be useful.

Thanks

Trace:

Plain Text

DEFAULT 2024-07-19T12:28:44.201583Z OpenAI completion.create error: TypeError: t.content.map is not a function
DEFAULT 2024-07-19T12:28:44.201602Z at /app/.next/server/chunks/434.js:4:2257
DEFAULT 2024-07-19T12:28:44.201607Z at Array.map (<anonymous>)
DEFAULT 2024-07-19T12:28:44.201613Z at c (/app/.next/server/chunks/434.js:4:2201)
DEFAULT 2024-07-19T12:28:44.201618Z at E (/app/.next/server/chunks/434.js:45:2011)
DEFAULT 2024-07-19T12:28:44.201622Z at P (/app/.next/server/chunks/434.js:45:1307)
DEFAULT 2024-07-19T12:28:44.201626Z at g (/app/.next/server/app/api/threads/[id]/route.js:1:1974)
DEFAULT 2024-07-19T12:28:44.201631Z at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
DEFAULT 2024-07-19T12:28:44.201635Z at async /app/.next/server/app/api/auth/[...nextauth]/route.js:1:12602
DEFAULT 2024-07-19T12:28:44.201640Z at async 
/app/.next/server/app/api/auth/[...nextauth]/route.js:1:12477
DEFAULT 2024-07-19T12:28:44.201645Z at async /app/node_modules/next/dist/compiled/next-server/app-route.runtime.prod.js:6:36258 6673541460d9ff285f6e9d69 new ObjectId('66984cfc77093749e290d6c1')

Okay give me a few minutes. May not be from portkey.chat.completions.create call

Hey folks. What is the update on the 2 bugs that I have reported?

cc @visarg who can share

Guys is this a priority or not? 😅

Hey! The request_timeout issue that you were facing for streaming call is fixed now. Please check and let us know.

Thanks @visarg. Will do! I will need to take a few days out to test it out, and then release it to production (current setup, with my BE doing the fallback, is working well so far)

Also, what about the error logs? Are you working on that?

Are you referring to the trace sent above?

No. When the timeout error was occuring, I wouldn't see any logs on portkey. Turns out, in case of portkey actions/errors (not downstream LLM providers'), nothing is logged

@visarg

This should also be fixed with the previous release.

It was related to the request_timeout bug for streaming calls.