Hey @hebertrfreitas, welcome!

VVrushank | Portkey

Hey @hebertrfreitas, welcome!

This is interesting. We'll investigate and get back. Currently Portkey only supports the standard Azure OpenAI URLs, but I wonder if using the custom_host param we can make this work in some way.

3 comments

hhebertrfreitas

Hey @Vrushank | Portkey , I found this in docs https://portkey.ai/docs/welcome/integration-guides/azure-openai#key-mapping and I suspect that internally portkey try to replace the variables like this:

Plain Text

curl https://{YOUR_RESOURCE_NAME}.openai.azure.com/openai/deployments/{YOUR_DEPLOYMENT_NAME}/chat/completions?api-version={API_VERSION} \
  -H "Content-Type: application/json" \
  -H "api-key: {YOUR_API_KEY}" \
  -d '{"messages": [{"role": "system", "content": "Hello"}] }'

So, using the AzureOpenAI client you can replace this https://{YOUR_RESOURCE_NAME}.openai.azure.com/ with the param azure_endpoint.

vvisarg

Hey! You should be able to make this work with the help of custom_host setting. Can you please try using the following snippet and see if it works for you?

Plain Text

client = Portkey(
  base_url="http://localhost:8787/v1",
  api_key="some value",
  provider="azure-openai",
  Authorization="AZURE_API_KEY",
  custom_host="<my_private_link_endpoint>",
  azure_api_version="2024-02-01"
)

The custom_host setting used here will build the final url like this (example):
custom_host: https://api.myprivateendpoint1234.xyz

final url (for chat completions): https://api.myprivateendpoint1234.xyz/chat/completions?api-version=2024-02-01

hhebertrfreitas

Thanks @visarg
Works like a charm, but with a detail, the final url for an AzureOpenAI service is something like https://{YOUR_RESOURCE_NAME}.openai.azure.com/openai/deployments/{YOUR_DEPLOYMENT_NAME} so in my case I concatenate /deployment/{YOUR_DEPLOYMENT} in param custom_host

Add a reply

Welcome to Portkey Forum

Hey @hebertrfreitas, welcome!