Force Refresh Cache for Chat Completions Request Using ...

SSaif

I wanted to force refresh the cache for a chat completions request using portkey sdk. I don't want to hit the cache for specific requests.

Plain Text

import { Portkey, createHeaders } from "portkey-ai";

let reqHeaders = createHeaders({
  "cache-force-refresh": true,
});
// rest
try {
  var response = await portkey.chat.completions.create(
    {
      messages,
      model: "gpt-3.5-turbo",
      stream: true,
    },
    { headers: reqHeaders }
  );
  for await (const chunk of response) {
    process.stdout.write(chunk.choices[0]?.delta?.content || "");
  }
} catch (error) {
  console.error("Errors usually happen:", error);
}

This is my logs (request):

Plain Text

  "headers": {
    "user-agent": "Bun/1.0.1",
    "x-forwarded-proto": "https",
    "x-portkey-headers": "{\"x-portkey-cache-force-refresh\":true}",
  },

Nevertheless, I see cache HITs. Any suggestions could help....