Welcome to Portkey Forum

Updated 4 weeks ago

Creating Virtual Keys with Custom Host Providers

is it possible to create virtual key with custom host provider ?

as we use a not known cloud provider, with llama3 hosted, we have an custom url to reach the servise, but i dont find a way to create virtual key in this contexte
G
c
V
11 comments
nope, this currently not possible, your requirement makes sense, I'll let the product team know
thanks a lot
I think the most simple way to integrate it in the first time, is not to add a new name provider but just allow to setup a custom_host when we create a virtual key, it is not a problem to set up openain as provider as we use their api interface
because it already possible to set an custom host on portkey object, when we put a virtual key, with open ai en the regular url, it override the url when we run chat compeltion
Plain Text
from portkey_ai import Portkey

portkey = Portkey(
    api_key="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
    provider="openai", 
    virtual_key="test-xxxx-b9xxxb",
    custom_host="https://chat.XXXXXXXX.com/v1"
)

completion = portkey.chat.completions.create(
    messages= [{ "role": 'user', "content": 'Why is open-source software important?' }],
    model='meta-llama-31-8b-it'
)

print(completion)

it is working perfectly, the problem , is as we can't set the custom_url directly when we are creating the virtual_key, we can't use this virtual key alone, to associate it with user and workspace
------
and when we create the virtual key with custom_url field , as it is not a default field , it is not reconize as it should
@Alastar | Portkey to make sur you saw it , this case avoid to make a migration on the database
got it @cpubird tagging @Vrushank | Portkey, @Siddharth | Portkey from product team for visibility
Thanks for the suggestions @cpubird! It is on the roadmap to allow you to put a custom host while creating the virtual key directly
We'll let you know once this feature goes live!
Add a reply
Sign up and join the conversation on Discord