Portkey
Log in

Welcome to Portkey Forum

Updated 8 months ago

Self-hosting the gateway and using the portkey python sdk for chat completions

ssusa
·
Just realised I posted my other message on the wrong channel so posting back here.
I'm trying to self host the gateway (currently for testing running the server on a local port) and use the Portkey Python SDK to make the chat completions queries.
However while doing this running into an issue :
  1. As suggested here I am passing the base URL as a parameter (in my case http://127.0.0.1:8787) . However to instantiate the SDK object a Portkey API key is required, so I passed a dummy API key
When doing this, I get an error
Plain Text
portkey_ai.api_resources.exceptions.NotFoundError: {"message":"Not Found","ok":false}

Is there something I am doing wrong here?
V
s
4 comments
VVrushank | Portkey
Hi! Looking into this
VVrushank | Portkey
You can pass a dummy key for the API Key and it should work.

Here's the code snippet that you can use:

Plain Text
from portkey_ai import Portkey

portkey = Portkey(
    base_url="http://localhost:8787/v1",
    provider="openai",
    Authorization="Bearer YOUR_OPENAI_API_KEY",
    api_key="xxx"
)

resposne = portkey.chat.completions.create(
    messages = [{ "role": 'user', "content": '42' }],
    model = 'gpt-4'
)


As you can see, api_key refers to the Portkey API Key generally, where we can pass any dummy value, while Authorization is where you'd pass your whole OpenAI key string, including Bearer in the front.

Please let me know if this solves it for you!
ssusa
Ah that worked! Had been passing the base URL as
Plain Text
http://localhost:8787
instead of
Plain Text
http://localhost:8787/v1

Thanks a lot Vrushank!
VVrushank | Portkey
Perfect! I think time we update the docs for this! Thank you!
Add a reply
Sign up and join the conversation on Discord