Integrazioni

LiteLLM

Chiama NotHoly tramite LiteLLM con il prefisso openai/ e api_base.

Python

import os
import litellm

r = litellm.completion(
    model="openai/notholy-specter",
    api_base="https://api.notholy.com/v1",
    api_key=os.environ["NOTHOLY_API_KEY"],
    messages=[{"role": "user", "content": "What does EPSS estimate?"}],
)
print(r.choices[0].message.content)

Per il proxy di LiteLLM, aggiungi a config.yaml:

YAML

model_list:
  - model_name: notholy-specter
    litellm_params:
      model: openai/notholy-specter
      api_base: https://api.notholy.com/v1
      api_key: os.environ/NOTHOLY_API_KEY
  - model_name: notholy-pulse
    litellm_params:
      model: openai/notholy-pulse
      api_base: https://api.notholy.com/v1
      api_key: os.environ/NOTHOLY_API_KEY
  - model_name: notholy-vector
    litellm_params:
      model: openai/notholy-vector
      api_base: https://api.notholy.com/v1
      api_key: os.environ/NOTHOLY_API_KEY
  - model_name: notholy-wisp
    litellm_params:
      model: openai/notholy-wisp
      api_base: https://api.notholy.com/v1
      api_key: os.environ/NOTHOLY_API_KEY