Platform

One place to see, score and steer your agents

Tracing shows what happened. Evals tell you if it was right. Guardrails stop it when it isn’t. Replay proves your fix works before you ship it.

Start freeRead the docs
Tracing & sessions

Read a conversation the way your agent lived it

Sessions group every trace in a conversation, so you can follow a user from first message to resolution and see which step went sideways.

  • Auto-captureModel calls, tool calls, retrievals, streaming tokens and errors.
  • RedactionPII scrubbed in the SDK, before data leaves your network.
  • Retention30 days on Team, up to 400 days on Enterprise.
Session · ses_41c9 · 3 turns · kb-answerer
Daniel K. · customerMy export to CSV keeps timing out on the Reports page.
kb-answererExports over 50,000 rows run in the background. You’ll get an email with a download link within 10 minutes.
kb.search 212msclaude-haiku-4-5 640msgrounded 0.97
Daniel K. · customerI never got the email. It’s been an hour.
kb-answererSorry about that. I’ve re-queued the export and sent it to d.k••••@••••.com.
jobs.requeue 88msemail.send 502grounded 0.52
Flagged: the agent confirmed an email was sent, but email.send returned 502. Added to dataset false-confirmations.
Evals

An LLM judge is only useful if it agrees with you

Label 50 traces by hand. Sightline measures how often each judge agrees with you and shows exactly where it doesn’t, so you know which scores to trust.

  • 30+ built-in judgesGroundedness, policy adherence, tone, task completion, tool correctness.
  • Code evaluatorsAny Python or TypeScript function that returns a score.
  • Online + offlineSample live traffic or run on a fixed dataset in CI.
Judge calibration · policy_adherence v4
Agreement with humans
94%+11 vs v3

Measured on 212 traces labelled by 3 reviewers. Cohen’s κ = 0.87.

v1v4 · today
Confusion matrix
Human: passHuman: fail Judge: pass1517 Judge: fail648

7 false passes, mostly partial refunds. Review them →

Guardrails

Rules that run in the request path

Guardrails check each step before it executes. The median check adds 6ms. If Sightline is unreachable, your agent keeps running with the last rules it fetched.

  • ActionsBlock, rewrite, ask for approval, or hand off to a human.
  • ScopePer agent, per tool, per customer tier or per environment.
  • AuditEvery intervention is stored on the trace with the rule version.
Guardrails · production
Refund limitBlock stripe.create_refund above $500, ask for approval
1,204 checks/day
Loop breakerStop after 5 identical tool calls within 30s
38 trips/day
PII redactionMask cards, SSNs, IBANs in inputs and outputs
all traffic
No legal promisesRewrite “guarantee”, “always”, “never” in replies
shadow mode
Hourly spend capPause agent if model spend exceeds $120 per hour
0 trips
Replay

Test the fix on the exact traffic that broke

Replay re-runs real production traces with your new prompt or model. Tool responses are served from the recording, so nothing touches Stripe, your CRM or your customers.

  • DeterministicRecorded tool outputs, fixed seeds, pinned model versions.
  • Side by sideEvery evaluator scored on both versions.
  • One clickPromote the winner to a percentage of live traffic.
Replay · refund-agent/system.md · v12 → v13
14When a customer reports a duplicate charge:
15- Refund the most recent charge.
15+ Call stripe.list_refunds first. If a refund already
16+ exists, tell the customer when it will arrive.
17Never refund more than the original amount.
412traces replayed
0.91policy score, was 0.72
$3.18replay cost

From first trace to first fix in an afternoon

Most teams find their first real bug within an hour of installing the SDK.

Step 1 · 5 minutes

Instrument

Install the SDK and call sightline.init(). Model and tool calls are captured with no other changes.

Step 2 · First hour

Observe

Turn on the default evaluators. Sightline groups failures by cause and ranks them by how many users they affect.

Step 3 · Same day

Fix and prove it

Change the prompt, replay the failing traces, and promote the new version once the scores improve.

Security

Your prompts are your product. We treat them that way.

Built for teams in finance, healthcare and the public sector.

SOC 2 Type II

Audited annually. Report available under NDA.

Self-hosted option

Run the full platform in your own AWS, GCP or Azure account.

Data residency

Store data in the US, EU (Frankfurt) or Australia (Sydney).

SSO & SCIM

Okta, Entra ID and Google. Role-based access down to the project.

Start with one agent

Free up to 50,000 spans a month. Upgrade when you need longer retention and more seats.

Start freeSee pricing