Portkey
Log in

Welcome to Portkey Forum

Updated 4 days ago

Enabling JSON Mode on TogetherAI's Models

HHarold Senzei
·
How do I enable json mode on togetherai's models? They have some models that support it but portkey's prompt ui does not allow adding json mode.
S
H
3 comments
SSiddharth | Portkey
Hey @Harold Senzei you are right, Together AI supports JSON object for

meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo & meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo

Looks like we missed it on frontend, let me get back to you on this.

Secondly for models that do have JSON mode enabled. you simply access this by choosing the JSON mode in parameters settings:
Attachment
image.png
HHarold Senzei
The option is missing for togetherai's models.
SSiddharth | Portkey
Yes I just saw that. We need to enable JSON mode for together ai and vertex. Let me update this and get back
Add a reply
Sign up and join the conversation on Discord