Comparing Prices and Running Quality Checks in Prompt Playground

hey @Luke Pighetti you can calculate the cost by looking at the logs of each request made on Prompt Playground.

The logs reflect after 1-2 min delay to get you the accurate pricing with other information about your request.

1. what checks do you mean here?

things like: given this image expect this json value to be between 10 and 20 at least 95% of the time

if you are looking to have simple checks on your requests you can use Portkey's guradrails that allow you to run async checks on your query and reponses LLM. you can find these in the guardrails section of the Portkey APP. This case for example could maybe use JSON validator and regex guardrails on your response.

here are the docs on to use guardrails: https://portkey.ai/docs/product/guardrails

you can run multiple calls using the API and check the output on the logs dashboard/track the status code of each request manually