Welcome to Portkey Forum

Updated 4 months ago

Gemini tool configuration and usage

I've verified that tool_config in gemini works as intended @artyabra ,
If you require gemini to always use a tool, you can pass "tool_choice": "required" in the SDK or the API as per OpenAI format
Plain Text
{
    "model": "gemini-1.5-pro",
    "max_tokens": 200,
    "stream": false,
    "messages": [
        {
            "role": "system",
            "content": "Say Hi"
        },
        {
            "role": "user",
            "content": "Hey, what is the temperature in San Francisco in Celsius? Also tell me the current time"
        }
    ],
    "tool_choice": "required",
    "tools": [
      ....    
    ]
}
a
S
V
17 comments
@sega , @Vrushank | Portkey

I'm sending you the raw request I'm sending and the raw response I got.
Can you please try to reproduce on your end?
the api_key and the project_id I replaced with "___"
Will take a look today
When you're checking, try sending it multiple times
@artyabra just confirming, you are using the SDK or the API?
@sega , the openai client, not portkey sdk
@sega but I also see the same problem with Portkey-sdk
@sega - I see you pushed some changes today. I cloned the porkey gateway repo and run it, I think your commits solved the issue
I sent requests that returned without using a tool before, but now they use the tool each time 🙂
I think it’s because vertex behaves differently when we’re passing requiredFunctionNames as an empty array vs when the key itself is absent
It works differently on Gemini and Gemini hosted on Vertex
Man Google really be putting devs through it
I wasn’t sure that was the fix @artyabra that’s why I didn’t communicate it to you, I wanted to do some further testing to see what the problem was
@artyabra this will become part of a new stable release soon! Thank you so much for your patience!
Add a reply
Sign up and join the conversation on Discord