Skip to content

Observability

What it is

Observability is the record of what happened: every model call, every change, and every connected client. It answers "how often does this run, how well, and who is using it?"

Everything below lives on the Observability page in the UI.

Run log

Every model invocation — from the Playground, Agents or Workflows — is logged with:

  • timestamp and source
  • provider and model
  • token counts (in / out, estimated)
  • latency in milliseconds
  • status (ok / error)

This is your record of what the AI actually did.

Stats

The run log is aggregated into summary cards:

  • Total runs
  • Success / failure counts
  • Estimated total tokens
  • Average latency
  • a breakdown by provider

Audit log

Every mutation is recorded separately: who changed what, and when. Creating, updating, deleting, publishing and rolling back a prompt all leave an audit entry with the action, resource, key and a short summary.

The run log tells you what the AI did; the audit log tells you what people did.

Connected clients

The Connected clients tab lists every live hot-reload connection — SDK instances and browser sessions currently watching for updates. For each one:

ColumnDescription
Typenode-sdk, python-sdk, java-sdk or browser
AppThe appName the SDK was created with
NamespaceThe environment it is watching
IP addressWhere it connected from
Connected atWhen the connection opened

This is how you see, at a glance, which applications are currently bound to PromptOps. See Hot Reload for how clients connect.

How it maps to the API

DataEndpoint
Run logGET /api/runs
Run statsGET /api/runs/stats
Audit logGET /api/audit
Connected clientsGET /api/clients

Next steps

Open-source runtime platform for AI prompts, agents, and workflows.