Ability to choose models for prompt templates based on api key
Ability to choose models for prompt templates based on api key
At a glance
The community member suggests the ability to choose models for prompt templates based on the API key used, such as using a smaller model like Llama 8B in a staging environment to save money, and a larger 70B model in production. Currently, this logic needs to be implemented in the backend, but the community member sees value in having a dynamic model choice feature based on factors like user ID. Other community members agree this would be a useful feature, especially for those who use templates and have to create multiple versions with different models. One community member mentions that conditional routing based on metadata is currently possible, and provides a link to relevant documentation.
Could be an edge case, but the ability to choose models for prompt templates based on which API key is used. ie. I have 3 keys (dev, staging and prod). In staging I want to use llama 8b to save money while in prod I want 70b. Right now I need to implement this logic in my backend, but I could see value for something like dynamic model choice based on X (not necessarily API key but could be anything like user ID or something like that?)
I mostly use templates, and this feature doesnt seem to be there for templates so I end up having to make the same templates with different models which is a bit of a pain
