Error code: 404 - the model gpt-4-turbo-preview does not exist or you do not have access to it

Hey. Were you using this model earlier or you tried for the first time? And what other models are failing for you?

Mistral was also failing and yes - so, I don't think the problem is with the model

Are you able to see the failed logs in portkey dashboard?

Yes, I copied above-mentioned error message from there

@ekevu. On the Mistral errors - are you calling Mistral by overriding the model param while making your prompt completions call? If the attached loadbalance config works differently, that would be given preference over everything else, including what you set inside the prompt template. Additionally, Mistral also does not have response_format param.

If you can share the relevant code snippet in DM I can verify if there are any syntax errors that have seeped in.

I am changing the model inside Portkey and whatever I choose there, I get "model not found" for exactly that when I run it in code.

Deactivating the load balancer seems to solve the problem, can you check if there is an issue with this structure? {
"retry": {
"attempts": 3
},
"strategy": {
"mode": "loadbalance"
},
"targets": [
{
"provider": "openai",
"virtual_key": "KEY",
"weight": 0.0526

Correct - there is no issue with either the prompt template or the loadbalancer config. The provider set in Config is given preference over what's in the prompt template. Hence, setting model to Mistral in the prompt template will give model not found error since Portkey will try to call that model on OpenAI (by picking it up from the Config).

Still not sure why it is happening for OpenAI models though - I'd suggest to update the SDK once and try it again since I see that some requests are coming from outdated SDK. Also checking further on this.

Did you make any changes in your openAI key that might not have gpt-4 turbo access?

Yeah this is my hunch as well @ekevu. - is it possible that the OpenAI API keys are access controlled?

@ekevu. - This looks like an issue on OpenAI's end. Here is one of the messages on openai discord where other user has reported this as well.
https://discord.com/channels/974519864045756446/1037561178286739466/1217253564397781064

But why does it work well if I deactivate the load balancer config file from Portkey?

It might be because one of the loadbalance provider keys might be facing the issue and the other works fine. Can you check one by one if the issue happens on all the loadbalance keys that you are passing

I have simply deactivated the load balancer for now. If it is an OpenAI issue, I hope it will go away soon

Hey. Was this solved?

No. I didn't have time to look into this, so I only deactivated the load balancer for now.