Thanks @Vrushank | Portkey! Tried this already by extrapolating a little bit from your docs but it doesnt seem to accept the default_headers arg:
Completions.create() got an unexpected keyword argument 'default_headers'
This is the code:
import os
from portkey_ai import createHeaders, PORTKEY_GATEWAY_URL
model = OpenAIServerModel(
model_id=AllModels().get_provider("deepinfra").llama_70b,
api_base=PORTKEY_GATEWAY_URL,
api_key="dummy",
default_headers=createHeaders(
api_key=settings.PORTKEY_API_KEY,
virtual_key=settings.DEEPINFRA_VIRTUAL_KEY
)
)
agent = CodeAgent(tools=[DuckDuckGoSearchTool()], model=model)
agent.run("How many seconds would it take for a leopard at full speed to run through Pont des Arts?")