How Could I Use Tool_choice In Prompts Completion With ...

At a glance

Hey! - How could I use tool_choice in prompts completion with anthopic models? I dont see any documentation around this topic

sql_completion = portkey.prompts.completions.create(
prompt_id="...",
variables={"user_query":". user_id=xxx"},
max_tokens=600,
temperature=0,
tools=sql_tool,
tool_choice={"type":"tool", "name": "sql_generator"}
)

15 comments

ssega

hey @shockdav , you can pass tool choice in the openai format itself, portkey will take care of the transformations

ssega

https://platform.openai.com/docs/api-reference/chat/create#chat-create-tool_choice

sshockdav

thank you very much, that did the trick

eendyb

Hey @sega look at the error:
Whe we use the tool_choice parameter like OpenAI we get this error:

Plain Text

BadRequestError: Error code: 400 - anthropic error: tool_choice may only be specified while providing tools.

This is the code:

Plain Text

prompt_completion = portkey.prompts.completions.create(
  prompt_id="pp-sql-21c0ac",
  variables={"user_query":f"{query}"},
  tools=tool_sql,
  tool_choice="required",
  max_tokens=1660
)
prompt_completion

How ever when whe change the format to Anthropic:

Plain Text

tool_choice={"type":"tool", "name":"sql_generator"},

We get:

Plain Text

InternalServerError: {"status":"failure","message":"Something went wrong"}

ssega

your error might be from your tools object being empty

ssega

are you passing an array of tools?

eendyb

Yes I am passing an array, it is not empty

ssega

can you share your tools object value? is it openai compliant?

eendyb

Give me a second.

eendyb

tool_sql = [
{
"type": "function",
"function": {
"name": "sql_generator",
"description": "Generate a valid SQL query to fulfill the user's request accurately. Avoid adding any extra information, commentary, or explanations in the query.",
"strict": True,
"parameters": {
"type": "object",
"required": [
"sql_query"
],
"properties": {
"sql_query": {
"type": "string",
"description": "A syntactically valid SQL query that fully satisfies the user's request. The query must be concise, precise, and free of additional information, comments, or explanations."
}
},
"additionalProperties": False
}
}
}
]

eendyb

@sega it is already working.
I was passing the tool definition like Anthropic API.
Your last question clarified the problem to me.

ssega

tool definition should be in openai format

every request you send should be in openai format only, you dont need to modify anything apart from the model id

eendyb

Yes got it.

eendyb

Thank you very much!!

ssega

essentially portkey makes its so that you only have to write code once and use it across differnt providers

Add a reply

Welcome to Portkey Forum

How Could I Use Tool_choice In Prompts Completion With Anthropic Models?