Token-level likelihoods

Log Probabilities

Log probabilities expose token-level likelihood metadata for supported models and providers.

They can help with token inspection, ranking alternative tokens, debugging generation behavior, uncertainty heuristics, and custom sampling analysis.

Important: Log probabilities are model-generated likelihood values and should not be treated as calibrated confidence scores. A highly probable token can still be factually wrong, biased, or inappropriate for the application.

Request controls

Pass the following parameters to the OpenAI-compatible Chat Completions endpoint to request token metadata.

logprobs

boolean

Default: false

Purpose: Enables the return of log probability metadata for each generated token.

Model dependency: Must be supported by the upstream provider.

top_logprobs

integer

Requirements: Requires logprobs: true

Purpose: Requests the most likely alternative tokens and their probabilities at each position.

Range: Provider-specific (often restricted to a small integer to limit payload size).

OpenAI-compatible Chat Completions

Request log probabilities by setting the appropriate fields when calling POST https://api.devupai.com/v1/chat/completions.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["DEVUP_API_KEY"],
    base_url="https://api.devupai.com/v1",
)

response = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V4-Pro",
    messages=[{"role": "user", "content": "Say hello in one word"}],
    logprobs=True,
    top_logprobs=3,
)

for token in response.choices[0].logprobs.content:
    print(f"{token.token!r}: {token.logprob:.4f}")
    if token.top_logprobs:
        for alt in token.top_logprobs:
            print(f"  alt {alt.token!r}: {alt.logprob:.4f}")

Response anatomy

Generated token

Log probability

Top alternatives

Application analysis

Interpretation

Log probabilities are usually natural logarithms (base e).
Values closer to zero represent higher token likelihood.
More negative values represent lower likelihood.
Token probability may be derived with exp(logprob).

Token Anatomy

Token"Hello"

Log probability-0.0023

Bytes[72, 101, 108, 108, 111]

Top alternatives

"Hi" (-1.42)"Hey" (-3.87)

Illustrative response onlyThe generated IDs, token values, and probabilities shown are strictly illustrative. Always inspect the returned payload safely.

json

{
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello"
      },
      "logprobs": {
        "content": [
          {
            "token": "Hello",
            "logprob": -0.0023,
            "bytes": [72, 101, 108, 108, 111],
            "top_logprobs": [
              {
                "token": "Hello",
                "logprob": -0.0023,
                "bytes": [72, 101, 108, 108, 111]
              },
              {
                "token": "Hi",
                "logprob": -1.42,
                "bytes": [72, 105]
              }
            ]
          }
        ]
      }
    }
  ]
}

Streaming behavior

When stream: true is provided, log probabilities (if supported) are typically returned per chunk inside choices[].logprobs. However, streaming metadata formatting may differ by provider. Some models may omit logprobs entirely during streaming, or only return them on the final chunk. Always handle streaming fields defensively.

Model support

Check the selected model page or live catalog for current log-probability support. Not all models or providers support logprobs or top_logprobs. The DEVUP AI transparent proxy forwards your request, but if the upstream model does not support token metadata, it may ignore the parameters or return an error.

Use cases and limitations

Practical use cases

Inspect token alternatives
Build ranking heuristics
Compare generation uncertainty
Debug constrained outputs
Analyze token selection

Limitations

Not calibrated confidence scores
Provider and model-dependent availability
Tokenization boundaries vary by model
Top alternatives may drastically increase response payload size
Unavailable fields (like bytes) must be handled safely