In such cases, instead of passing configID, you can pass raw config in the x-portkey-config header. Example curl header:

It should be on the far right in the list item. I think your UI might not be on latest version. Can you please click on update available popup that shows up in bottom left of the screen. Or if thats not visible, you can also do a hard refresh and try again.

Hey @susa . Have you tried adding the messages array in override_params in the config object? So your fallback config should look something like this:



When you do this, Portkey will first try the gpt-4 model with the messages sent in the request body. And if that fails, then it will try gpt-3.5-turbo but with the new messages array that you add in the override_params object for that target. Please let me know if this works for you.

Hey @kaushikbokka - This timeout is not happening on our end. Its happening on the LLM provider's end. If we were timing out, then we would not have got the response and logged it. I will check if others are facing this issue. One thing that you can do is add a request_timeout setting in your config along with fallback so that your requests are not stuck.

You can pass a map where key will be the tokenized word and value will be the bias which controls the likelihood of that token appearing in your generated response. Example:


For example, here 19045 is the tokenized id for good and 58234 is tokenized id for better. The above logit_bias will reduce the chances of the model generating the word good in the completion as its value is negative 10 and vice versa for the word better as its value is positive 10.

Reference to a simple article that explains it well: https://help.openai.com/en/articles/5247780-using-logit-bias-to-define-token-probability

You can use this to generate tokenized ids for words (for openai models): https://platform.openai.com/tokenizer

Looking into this asap.