Welcome to Portkey Forum

Updated 8 months ago

prompt completions: pass an array of assistant and user messages

Hi team, I have a prompt on Portkey with one system message (and possibly variables). I want to use the prompt completion api to call this prompt but also additionally pass an array of assistant and user messages (think of passing chat history). How can I do this with the portkey api? I am using the python SDK. I looked around the portkey.prompts.completions.create method, but couldn't find anything that could help.

6 comments

SSaif

As far as I know, the prompt endpoints are stateless, so you might have to maintain chat history within your app. Let's also see if Portkey team could add.

bbhuvansingla

I am indeed maintaining the chat history myself. The question is about passing it in a chat completion call with a Portkey prompt.

bbhuvansingla

I figured that if I used:

portkey.chat.completions.create : I could pass the array of messages but can't pass a Portkey prompt ID for the top-level system message that I want to maintain in Portkey.
portkey.prompts.completions.create: I could use a prompt in Portkey, but can't pass the array of conversation.

bbhuvansingla

Okay, with more tinkering around, I self-figured it out. One can use Raw on the UI in the prompt. Not sure if this was covered in the documentation somewhere.

VVrushank | Portkey

You're right! Raw mode is exactly to support this use case. We covered some of that functionality in this cookbook: https://portkey.ai/docs/cookbooks/few-shot-prompting

Let me update the docs as well. Curious if this Raw mode adequately serves your use case or if you have any feedback on it

bbhuvansingla

Thanks Vrushank, it seems to solve for now.

Add a reply