Welcome to Portkey Forum

Updated 2 weeks ago

Portkey and Smolagents

heyo πŸ™‚ Wondering if theres any way to use Portkey with smolagents?
V
M
12 comments
Yep this will work! @Siddharth | Portkey let's write a cookbook on this?

We also raised a PR with direct Portkey integration that they unfortunately closed 😦

https://github.com/huggingface/smolagents/pull/144/files
For now, just doing this should suffice:

Plain Text
import os
from smolagents import OpenAIServerModel
from portkey_ai import createHeaders, PORTKEY_GATEWAY_URL

model = OpenAIServerModel(
    model_id="deepseek-ai/DeepSeek-R1",
    api_base=PORTKEY_GATEWAY_URL,
    api_key="dummy",
    default_headers=createHeaders(
        api_key=PORTKEY_API_KEY,
        virtual_key=PROVIDER_VIRTUAL_KEY
)
I'm just going based on vibes here, but if not the OpenAIServerModel, the AzureOpenAIServerModel should def work
Thanks @Vrushank | Portkey! Tried this already by extrapolating a little bit from your docs but it doesnt seem to accept the default_headers arg:

Completions.create() got an unexpected keyword argument 'default_headers'

This is the code:

Plain Text
import os
from portkey_ai import createHeaders, PORTKEY_GATEWAY_URL

model = OpenAIServerModel(
    model_id=AllModels().get_provider("deepinfra").llama_70b,
    api_base=PORTKEY_GATEWAY_URL,
    api_key="dummy",
    default_headers=createHeaders(
        api_key=settings.PORTKEY_API_KEY,
        virtual_key=settings.DEEPINFRA_VIRTUAL_KEY
)
)


agent = CodeAgent(tools=[DuckDuckGoSearchTool()], model=model)
agent.run("How many seconds would it take for a leopard at full speed to run through Pont des Arts?")
Got it working, here's the code @MolBioBoy:

Plain Text
from smolagents import OpenAIServerModel, CodeAgent, DuckDuckGoSearchTool
from portkey_ai import createHeaders, PORTKEY_GATEWAY_URL

model = OpenAIServerModel(
    model_id="MODEL_NAME",
    api_base=PORTKEY_GATEWAY_URL,
    api_key="dummy",
    extra_headers=createHeaders(
        api_key="PORTKEY_API_KEY",
        virtual_key="PROVIDER_VIRTUAL_KEY",
        trace_id="smolagents"
    )
)

agent = CodeAgent(tools=[DuckDuckGoSearchTool()], model=model)
agent.run("How many seconds would it take for a leopard at full speed to run through Pont des Arts?")
@Vrushank | Portkey Love it, thanks πŸ™‚ How'd you find the solution? Looked through so many docs lol.

Just out of curiosity, which model/provider did you use? It seems not every provider works so well for this lib. deepinfra for example does not seem to support tool-call and tool-response roles and so on that are sort of needed for this lib. Tested on fireworks and getting the same error for several models.
Haha, saw this on the OpenAI SDK repo. extra_headers is kwargs, whereas default_headers is not
I actually used deepseek too which did not work lol
@sega did you get smolagent to work on any llm?
Great find! I tried like 5 different providers haha. I tried the HfModel and it works for a single turn then it gives me a rate limit error. Probably would work better if I was a pro user but unsure. Either way I think this lib is in its infancy and maybe shouldnt be used for any real applications yet tbh.
I think deepseeks models dont support any sort of tool calling yet but i could be wrong
Yeah I glossed over that
Add a reply
Sign up and join the conversation on Discord