Accessing Llama 3.1 API Service on Vertex AI via Portkey

Question

I am trying to access Llama 3.1 API service on vertex AI via Portkey. followed this documentation - https://docs.portkey.ai/docs/welcome/integration-guides/vertex-ai to make a virtual key but i don't see llama on the portkey playground. have enabled the llama api service too. could someone help me debug this? cc

Vrushank | Portkey · Answer

Hey @thismlguy we'll add this and let you know!

Vrushank | Portkey · Answer

cc @visarg @Sabbyasachi

thismlguy · Answer

thanks, by when do u think this will be in?

Vrushank | Portkey · Answer

We'll add it today

thismlguy · Answer

also, is this a typical process to request for a model on vertex ai?

Vrushank | Portkey · Answer

For now, yes - we can open up a separate channel for you guys to make the requesting streamlined

Vrushank | Portkey · Answer

This is until we ship a "model garden" type feature where you can add these details yourself... that's on the roadmap, but we don't have a strong deadline for it yet

thismlguy · Answer

got it, let me know once its added

Welcome to Portkey Forum

Accessing Llama 3.1 API Service on Vertex AI via Portkey