Receiving webhooks

A webhook is the alternative to asking. Polling a market every five minutes to catch a change that happens twice a day is the single most common source of wasted quota, and a delivery costs no quota at all.

Creating an endpoint

curl -sS -X POST "https://api.skautik.com/v1/api/webhooks" \
  -H "Authorization: Bearer $SKAUTIK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/hooks/skautik",
    "events": ["property.created", "property.updated", "property.withdrawn"]
  }'

The response carries the signing secret once:

{
  "data": {
    "webhook": { "id": "whk_1b7e6c19a1", "url": "…", "active": true },
    "secret": "whsec_uEw7ugUA3saVyNo087Sg3XvhBnWI0txICtik-ySKD2k"
  }
}

Store it before you close the response. It is never shown again, and the only recovery is rotating it, which invalidates the old one.

Events

Subscribe to what you will act on. An endpoint subscribed to everything and filtering in code still pays the cost of receiving everything, and a busy import makes property.updated the noisiest event on the platform.

Verifying the signature

Do this before you read the body. An unverified webhook endpoint is a public API that writes to your database on request from anybody who guesses the URL.

Every delivery carries three headers:

Skautik-Signature: t=1786680348,v1=8f2a41c9d0b7e6c19a139404cb173d23fcb3331c4e…
Skautik-Event: property.updated
Skautik-Delivery: dlv_9c4e1f2308

The signature is HMAC-SHA256(secret, timestamp + "." + body), hex encoded. The timestamp is inside the signature rather than only beside it, which is the part that matters: if only the body were signed, a captured delivery could be replayed at your endpoint forever and the signature would still match.

import { createHmac, timingSafeEqual } from "node:crypto";

export function verify(rawBody: string, header: string, secret: string): boolean {
    const parts = Object.fromEntries(
        header.split(",").map((piece) => piece.split("=") as [string, string]),
    );

    const timestamp = parts.t;
    const signature = parts.v1;
    if (!timestamp || !signature) {
        return false;
    }

    // Refuse anything older than five minutes. Without this the signature is
    // valid for ever and a captured request never stops working.
    const age = Math.abs(Date.now() / 1000 - Number(timestamp));
    if (!Number.isFinite(age) || age > 300) {
        return false;
    }

    const expected = createHmac("sha256", secret)
        .update(`${timestamp}.${rawBody}`)
        .digest("hex");

    // Constant time. A plain === leaks the position of the first wrong byte,
    // which is enough to forge a signature given enough attempts.
    const a = Buffer.from(expected, "hex");
    const b = Buffer.from(signature, "hex");
    return a.length === b.length && timingSafeEqual(a, b);
}

import hashlib
import hmac
import time


def verify(raw_body: bytes, header: str, secret: str) -> bool:
    parts = dict(piece.split("=", 1) for piece in header.split(","))

    timestamp = parts.get("t")
    signature = parts.get("v1")
    if not timestamp or not signature:
        return False

    if abs(time.time() - int(timestamp)) > 300:
        return False

    expected = hmac.new(
        secret.encode(),
        f"{timestamp}.".encode() + raw_body,
        hashlib.sha256,
    ).hexdigest()

    return hmac.compare_digest(expected, signature)

Sign the raw bytes, not a re-serialised object. Parsing JSON and dumping it again changes key order and whitespace, and the signature will never match. Most frameworks need telling to keep the raw body around.

Answering

Answer 2xx as soon as you have verified the signature and written the payload somewhere durable. Do the actual work afterwards, on a queue.

An endpoint that geocodes an address, writes to three tables and sends an email before answering is an endpoint that times out under load, which turns one delivery into six as we retry, which makes the load worse.

Anything other than a 2xx is a failure and will be retried.

Retries

A failed delivery is retried up to six attempts with exponential backoff. next_retry_at on the delivery record says when the next one is due.

Deliveries are at least once, not exactly once. A timeout on your side after you committed still counts as a failure to us, and you will see the event again. Make handling idempotent: key on Skautik-Delivery, or on the record id and its updated_at, and treat a repeat as a no-op.

Order is not guaranteed either. Two updates to one property can arrive the wrong way round, so compare updated_at before overwriting rather than trusting arrival order.

When something is wrong

GET /v1/api/webhooks/{webhook_id}/deliveries lists what we tried, with the status code you returned and how many attempts it took. It is the first place to look when data has stopped arriving, and usually answers the question before you open your own logs.

POST /v1/api/webhooks/{webhook_id}/test sends a synthetic delivery, which is the quick way to check a new endpoint is reachable and verifying correctly before you depend on it.

A checklist

• Verify the signature before reading the body.
• Reject deliveries older than a few minutes.
• Compare signatures in constant time.
• Answer quickly; work afterwards.
• Handle repeats and out-of-order arrivals.
• Subscribe only to events you act on.
• Keep the secret out of your repository, like any other credential.