A/B Testing Cookbook: Portkey AI Integration

I picked up the following snippet form A/B testing cookbook:

Plain Text

import Portkey from 'portkey-ai'

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    config: "pc-blog-o-0e83d2" // replace with your config ID
})

// We can also override the hyperparameters
const pcompletion = await portkey.prompts.completions.create({
    promptID: "pp-blog-outli-840877", // Use any prompt ID
    variables: {
       "title": "Should colleges permit the use of AI in assignments?",
       "num_sections": "5"
    },
});

console.log(pcompletion.choices)

The configs split the traffic into two prompts at 50% traffic each.

I wonder why chat completions call would need promptID given these are part of config already?

3 comments

VVrushank | Portkey

This is because of how we do prompt completions currently, where a call can't take an empty prompt ID. So you can put up any string there, and it will be ignored in favor of the prompt IDs set inside the Config

SSaif

I suppose any string is not ignored, but they could be any valid prompt ID that will be ignored:

Plain Text

:95:5) {
  status: 400,
  headers: {
    'alt-svc': 'h3=":443"; ma=86400',
    'cf-ray': '857dd11dccc485ea-BOM',
    connection: 'keep-alive',
    'content-length': '75',
    'content-type': 'application/json',
    date: 'Mon, 19 Feb 2024 10:27:23 GMT',
    nel: '{"success_fraction":0,"report_to":"cf-nel","max_age":604800}',
    'report-to': '{"endpoints":[{"url":"https:\\/\\/a.nel.cloudflare.com\\/report\\/v3?s=tnfmhJb1uJCT10qruldYqA05px54hsjvQeavozPA62nChckqmoW8cE%2Bmz%2BAP%2BOj1BfWUhe7OWIMG7QEqUguXXaCJcJg7%2BJ7hsAH%2B8RbjiV4h54b5mE13j8gde4CZQFAl"}],"group":"cf-nel","max_age":604800}',
    server: 'cloudflare',
    vary: 'Accept-Encoding'
  },
  error: {
    status: 'failure',
    message: 'Following prompt_id are not valid: ignored'
  }
}

VVrushank | Portkey

Ah thanks! You're right

Add a reply

Welcome to Portkey Forum

A/B Testing Cookbook: Portkey AI Integration