Sandbox

The same incident you've been reading about.

A read-only Sondar instance seeded with a checkout-latency incident: a spike in the histogram, a failing service, a connection pool that ran out. No account, no email, no sales call.

Status

The live sandbox is the most expensive thing on this site and the highest-leverage one, so it is being built first rather than last. Until the session broker ships, what's below is the product surface rendered from the same seed data — real markup, real numbers, not a picture of a UI. When the live instance lands, this page becomes it, and we'll say so here rather than quietly swapping them.

sondar · index=main

21 Jul 2026 14:03:00 UTC

SonQL Press ⌘↵ to execute

index=main sourcetype=checkout-api status>=500

| stats p99(duration_ms) by endpoint, service

14,209 matches · 42ms query time · 1.2M events (44 GB direct) Live tail off

14:03 Incident Start

endpoint	service	p99 latency	events
POST /api/checkout	payments-gateway	4,218ms	6,104
POST /api/checkout	checkout-api	4,106ms	2,988
GET /api/cart	checkout-api	388ms	121
POST /api/payment/authorize	payments-gateway	291ms	44
GET /api/inventory	inventory-svc	62ms	5

pool.name	service	avg wait	count
ledger-db	payments-gateway	4,188ms	8,412
ledger-db-replica	payments-gateway	912ms	406
sessions	checkout-api	38ms	96
catalog	inventory-svc	11ms	26

span	service	duration	depth
POST /api/checkout	checkout-api	4,218ms	0
authorize	payments-gateway	4,196ms	1
pool.acquire	payments-gateway	4,188ms	2
ledger.write	ledger-db	7ms	3

Demo World Checkout-latency incident. The same dataset seeds the sandbox.

What the seeded scenario walks through

14:03 UTC. p99 on POST /api/checkout goes from 240ms to 4.2s. The histogram shows it before anyone pages you.
Filter to the failing service. payments-gateway is the one whose latency moved; checkout-api is just waiting on it.
Extract a field you never mapped. The raw JSON has pool.wait_ms inside it. Pull it out at search time — no reindex, no pipeline change, no schema decision made six months ago.
Aggregate. stats p99(pool.wait_ms) by pool.name and the answer is ledger-db, at 32 of 32 connections in use.
Confirm the cause. Join to the deploy annotation four minutes earlier that widened a transaction.

Roughly ninety seconds end to end. If it takes you longer than that, that's a product problem and we'd like to hear about it.

Or skip the sandbox and run the real thing.

It's one container and it takes about as long as reading this page.

bash

docker run -v ./data:/data -p 8080:8080 sondardev/sondar