Skip to content
NotHolyDocs

Guides

Cost practices

How a request is priced and how to spend less for the same work.

View as Markdown

A request costs input tokens times the input rate, plus cached input tokens times the cached rate, plus output tokens times the output rate. Reasoning tokens count as output. Rates are per million tokens and published on the models page.

Ways to spend less:

  1. Use the cheapest model that does the job for bulk triage, and a larger one only where long context or depth matters. Models lists the rates side by side.
  2. Set max_tokens so a runaway answer cannot cost more than you expect.
  3. Use reasoning_effort: "low" when the task is lookup rather than analysis.
  4. Keep long, stable instructions at the start of the prompt so they can be cached.
  5. Give each script its own key with a daily budget, so a loop stops at the budget.
Python
import os
from openai import OpenAI

client = OpenAI(base_url="https://api.notholy.com/v1", api_key=os.environ["NOTHOLY_API_KEY"])

r = client.chat.completions.create(
    model="notholy-specter",
    max_tokens=120,
    reasoning_effort="low",
    messages=[{"role": "user", "content": "Classify this finding as low, medium or high: verbose stack trace on 500."}],
)
u = r.usage
print(u.prompt_tokens, u.completion_tokens)

The console shows the cost of every request under Activity, with the price version that was applied.

Next