Welcome to Portkey Forum

Updated 3 months ago

@Vrushank | Portkey -Hi Im trying to

At a glance
@Vrushank | Portkey -Hi Im trying to connect llamaindex LLM's through portkey, but getting "Connection Error". Im able to connect the llamaindex LLM's succesfully without portkey.
Can you help me fix this?
Attachment
image.png
1
V
v
C
12 comments
Hey! Can you please share which integration method you're using?
Possible to share what's in your Config? And actually the code snippet itself?
Nothing much i just tried the code given in your documentation
from llama_index.llms.openai import OpenAI
from llama_index.core.llms import ChatMessage
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

config = {
"provider":"anthropic",
"api_key":"<Replaced with my anthropic api key>",
"override_params": {
"model":"claude-3-5-sonnet-20240620",
"max_tokens":8192
}
}

#### You can also reference a saved Config ####
#### config = "pc-anthropic-xx"

portkey = OpenAI(
api_base=PORTKEY_GATEWAY_URL,
default_headers=createHeaders(
api_key="<Replaced with my portkey api key>",
config=config
)
)

messages = [
ChatMessage(role="system", content="You are a pirate with a colorful personality"),
ChatMessage(role="user", content="What is your name"),
]

resp = portkey.chat(messages)
print(resp)
Ok, thanks for reporting @vikash pratheepan - we're looking into this and will update you asap
Thanks @Vrushank | Portkey
Hey!

It is just a one line addition to the current implementation and it will be working for you.

In the createHeaders() you need pass Authorization="<anthropic-api--key>"

This should fix the error.

Sample implementation:

Plain Text
portkey = OpenAI(
    api_base=PORTKEY_GATEWAY_URL,
    default_headers=createHeaders(
        api_key="<portkey-api-key>",
        config=config,
        Authorization="<anthropic-api-key>",
    )
)
or alternatively, if you have the anthropic_key in the config object, while intializing the OpenAI client, you can simply pass a placeholder for api_key like so
Plain Text
portkey = OpenAI(
    api_base=PORTKEY_GATEWAY_URL,
api_key="xx" #placeholder do not remove
    default_headers=createHeaders(
        api_key="<Replaced with my portkey api key>",
        config=config
    )
)
we've updated the docs as well, now the code snippets should work, sorry for the inconvenience
No Problem @Severus | Portkey .

Thanks for the solution.It Works!
Add a reply
Sign up and join the conversation on Discord