Skip to main content

Cost

The cost metric is another performance metric offered by deepeval, and measures whether the token cost of your LLM (application) is economically acceptable.

Required Arguments

To use the CostMetric, you'll have to provide the following arguments when creating an LLMTestCase:

  • input
  • actual_output
  • cost

Example

from deepeval import evaluate
from deepeval.metrics import CostMetric
from deepeval.test_case import LLMTestCase

metric = CostMetric(max_cost=0.4)
test_case = LLMTestCase(
input="...",
actual_output="...",
cost=0.34
)

metric.measure(test_case)
# True if cost <= max_cost
print(metric.is_successful())
note

Similar to LatencyMetric, the CostMetric threshold, max_cost, does NOT have any standard units. However, you need to make sure the monetary units you provide in the cost argument when creating an LLMTestCase matches that of the cost max_cost.