Hello,

I am the dev of an organization and created an account to test it out. We liked the possibilities of the platform and want to upgrade now. I invited someone who can upgrade the account, but i cant make him to an owner of the organization and in a result of this, he can not upgrade the account.
How can i switch the owner of an orga?

Nope, using open ai with a virtual key

uhhhhm i think i found a bug

just added openrouter and used Run Test Request on the Getting Started Area

How do I get portkey to be the model provider in dify.ai

How can I suppoert custom models w/ Portkey?

Hello everyone, i am testing Portkey atm

it looks pretty Promising

i use Rust with the async-openai crate and now i get some errors, and i dont know how to fix them

with this setup:


i am getting this error:


what do i need exactly?
i could create a custom http client, but what does this headers mean? cant find them in the documentation

Hi, I followed the instructions for using Portkey with CrewAI and Python here https://portkey.ai/docs/integrations/agents/crewai#crewai, but the events aren't coming through. Any debugging tips?

hey just had a question. if I am self-hosting portkey using a docker container, is there a way to still get all of the logs/metrics?

Why isn't it possible to define a config (fallbacks, load balancing, etc.) or link to a config id in a prompt template? If you can select a single virtual key you should also be able to select a config that is used. In my opinion both features make only sense if combined, else you still have to override some stuff in code and cannot use the UI-based prompt management to iterate on prompt configurations properly. Or am I missing something?

Hi - Perplexity.ai now allows citations to come through the API - I tested it through Portkey and they aren't showing up - curious if you are aware of this update and when it may be available? See attached image

How to use portkey in cline?

Does Portkey have any way to enable Cache Control for all requests sent for Anthropic API? Like there's now a simple button in the playground, so maybe some config option that will add all input to the cache control. Basically to try and make it work like OpenAI's caching; automatic and always on.

The way we're using Portkey with HARPA AI, we don't really have a way to set cache control otherwise, so we can't use caching.

Hey guys, let's say i want to serve self hosted LLAMA 3.1 405B on 2 different VMs with H100s, I use vLLM + ngrok alternative btw.

I have tested Loadbalancing mode, works fine. with this config:

{ "strategy": { "mode": "loadbalance" }, "targets": [ { "provider": "openai", "custom_host": "https://llama-1.tunnels-dev.io.systems/v1", "api_key": "Bearer dummy-api-key-vm1k", "weight": 0.4, "override_params": { "model": "meta-llama/Llama-3.1-405B-FP8" } }, { "provider": "openai", "custom_host": "https://llama-2.tunnels-dev.io.systems/v1", "api_key": "Bearer dummy-api-key-vm2", "weight": 0.6, "override_params": { "model": "meta-llama/Llama-3.1-405B-FP8" } } ], "cache": { "mode": "simple", "max_age": 60000 }, "retry": { "attempts": 3, "on_status_codes": [ 404, 429, 500, 520 ] } }

But i want to achieve one more goal here, in case all retries to llama-1 (target-0) fails, i need to fallback to target-1.

basically i want a mode called loadbalance-fallback from the Docs, looks like there is such hack like this, but are there any better ways ?

@Vrushank | Portkey -Hi Im trying to connect llamaindex LLM's through portkey, but getting "Connection Error". Im able to connect the llamaindex LLM's succesfully without portkey.
Can you help me fix this?

🥹 any input on this guys

Hey is it possible to create a config to load balance, canary test, with virtual keys specifcally for a prompt id?

Hi there, I'm trying to figure out how the fallback strategy works with vision on gemini and other LLMs? I've posted a github issue but no response (it seems the issues are not very actively monitored)

https://github.com/Portkey-AI/gateway/issues/721

I think your config is fantastic, but both fallback, loadbalance or really any strategy which uses multi LLMs is incompatible with gemini + others (say gemini + openai).

The reason being is that gemini requires image URLs to be uploaded to a google storage bucket and you to provide the URLs gs://..... (or googles internal file manager, but same issue) in either case those urls are not accessable by other LLMs since they are not public links, just internal google bucket links.

So, how could we use one call which passes gemini compatible urls + public urls for other LLMs ? right now this seems impossible. We need a way to pass both kinds of urls in a special way in the "image_url" param.

We can slightly workaround it if we only send to one LLM and don't use any config (which is sad since it's a key selling point of portkey). We know the LLM we are sending to and we can prepare the data correctly.

However, if we want to use configs the problem gets worse because if we are using a config id, then we don't even know if portkey backend will send to non-google LLMs, we don't really know anything since that's handled completely transparently by Portkey and only admin level clients could even see the config settings.

Is it possible to set usage limits on a per user basis? Specifically, using the user attribute from "x-portkey-metadata: {"_user"

Hey
we were trying to store the chat completions in open ai by making use of openai store params. But seems like it is not working when we try calling open ai using portkey. The same is working when we try open ai sdk directly. can someone help me with this @Vrushank | Portkey

How should I proceed?

I've been facing an issue from the past few weeks where some of my requests end up in a connection error. But I see those requests in https://app.portkey.ai/organisation/org-uuid/logs

I see isQuotaExceeded : false in one of portkey's api calls.

Any suggestions?

But how do i acces to the local dashboard ?

I am using the self-hosted verison of the gateway and I want to add a layer of security and logging in front of the gateway probably with NodeJS
What I have in mind is sending requests to my NodeJS APP, do some security checks and redirect the request to my portkey gateway instance. Wdyt?

I want to add image generation feature of cloudflare for workers-ai provider, but their API response is different from portkey types

Could be an edge case, but the ability to choose models for prompt templates based on which API key is used. ie. I have 3 keys (dev, staging and prod). In staging I want to use llama 8b to save money while in prod I want 70b. Right now I need to implement this logic in my backend, but I could see value for something like dynamic model choice based on X (not necessarily API key but could be anything like user ID or something like that?)