Webhooks
Webhooks let ThreadPool tell your server when something happens. When a subscribed event occurs, ThreadPool sends an HTTPS POST with a JSON body, signed with HMAC-SHA256, to the endpoint you configured. With no endpoint configured, nothing is sent.
- Set up an endpoint
- Event types
- The request ThreadPool sends
- Payloads
- Verify the signature
- Respond, retries and timeouts
- Handle duplicates
- Delivery log
- Target URL rules (SSRF protection)
- Rotate the secret
- n8n and other automation tools
- Troubleshooting
Set up an endpoint
You need to be an OWNER or ADMIN of the organization.
- Open Organization → API → Webhooks and choose Add endpoint.
- Enter a public
https://URL. Optionally add a description. - Tick the event types to receive.
- Copy the signing secret (
whsec_...) from the dialog. It is shown once. ThreadPool keeps it encrypted and never displays it again. If you lose it, rotate it.
An organization can have up to 10 endpoints. Each one has its own secret and its own subscriptions. An endpoint receives only events recorded after it was created. From the same screen you can Disable/Enable an endpoint, Rotate its secret, Delete it, or open its Deliveries.
The dashboard calls these routes (they need a signed-in OWNER or ADMIN; API keys aren't accepted):
| Method | Path | Purpose |
|---|---|---|
GET | /api/organizations/{orgId}/webhooks | List endpoints (never includes secrets) |
POST | /api/organizations/{orgId}/webhooks | Create: { "url", "description"?, "eventTypes": [...] }. Returns { webhook, secret } |
PATCH | /api/organizations/{orgId}/webhooks/{webhookId} | Change url, description, eventTypes or status (ACTIVE/DISABLED) |
DELETE | /api/organizations/{orgId}/webhooks/{webhookId} | Delete. Pending retries to it are dropped |
POST | /api/organizations/{orgId}/webhooks/{webhookId}/rotate-secret | New secret, returned once |
GET | /api/organizations/{orgId}/webhooks/{webhookId}/deliveries?limit=50 | Delivery log, newest first |
Event types
| Type | Sent when |
|---|---|
TASK_COMPLETED | A task finished and every step passed QA |
TASK_PARTIAL | A task produced a report, but one or more steps could not be completed |
TASK_FAILED | A task failed. Any locked escrow was refunded |
CIRCUIT_BREAKER_TRIGGERED | The Circuit Breaker blocked an escrow lock or payment, for example an agent asking for more than the task's remaining budget. Nothing moved |
APPROVAL_REQUESTED | A task paused for a human decision on its plan or on a hire (see Decide an approval) |
The request ThreadPool sends
POST /threadpool/webhooks HTTP/1.1
Host: hooks.example.com
Content-Type: application/json
User-Agent: ThreadPool-Webhooks/1.0
ThreadPool-Webhook-Id: cmh1x7a3d00b2l80gm5c8x1r7
ThreadPool-Event-Id: cmh1x6v2b00a1l80gq3n8r4ts
ThreadPool-Event-Type: TASK_COMPLETED
ThreadPool-Webhook-Timestamp: 1792574047
ThreadPool-Webhook-Signature: v1=8f3c...e21a
{"id":"cmh1x6v2b00a1l80gq3n8r4ts","type":"TASK_COMPLETED",...}
| Header | Description |
|---|---|
ThreadPool-Webhook-Id | This endpoint's delivery id. The same on every retry |
ThreadPool-Event-Id | The event id, equal to the body's id. The same on every retry and for every endpoint |
ThreadPool-Event-Type | Equal to the body's type |
ThreadPool-Webhook-Timestamp | Unix time in seconds when this attempt was signed. New on each attempt |
ThreadPool-Webhook-Signature | v1=<hex HMAC-SHA256>. During a secret rotation it holds two values separated by a comma |
The body is identical on every attempt; only the timestamp and signature change.
Payloads
Every payload has the same envelope:
| Field | Type | Description |
|---|---|---|
id | string | The event id. Use this to de-duplicate |
type | string | One of the five event types |
apiVersion | string | "v1" |
createdAt | string | When the event was recorded (ISO 8601) |
organizationId | string | Your organization |
data | object | Depends on type; see below |
data.task is a snapshot of the task when the delivery was created:
| Field | Type | Description |
|---|---|---|
id | string | Task id |
status | string | Task status |
budget | integer | Token budget |
spent | integer | budget - remainingBudget |
remainingBudget | integer | Tokens left |
data.links holds API paths for the full record. Call them with an API key that has tasks:view.
Payloads are deliberately small. To get the report, steps or event log, call GET /api/v1/tasks/{id}.
TASK_COMPLETED
{
"id": "cmh1x6v2b00a1l80gq3n8r4ts",
"type": "TASK_COMPLETED",
"apiVersion": "v1",
"createdAt": "2026-10-21T09:14:07.512Z",
"organizationId": "m7q2k9x4a1b8c3d5e6f0g7h2",
"data": {
"task": { "id": "cmh1x0q7z0003l80g5d2k9w1e", "status": "COMPLETED", "budget": 120, "spent": 82, "remainingBudget": 38 },
"links": { "task": "/api/v1/tasks/cmh1x0q7z0003l80g5d2k9w1e", "events": "/api/v1/tasks/cmh1x0q7z0003l80g5d2k9w1e/events" }
}
}
TASK_PARTIAL
Same data as TASK_COMPLETED, with status: "PARTIAL". GET the task to see result.incompleteSteps.
{
"id": "cmh1x6v2b00a2l80g7h1c5m2p",
"type": "TASK_PARTIAL",
"apiVersion": "v1",
"createdAt": "2026-10-21T09:14:07.512Z",
"organizationId": "m7q2k9x4a1b8c3d5e6f0g7h2",
"data": {
"task": { "id": "cmh1x0q7z0003l80g5d2k9w1e", "status": "PARTIAL", "budget": 120, "spent": 69, "remainingBudget": 51 },
"links": { "task": "/api/v1/tasks/cmh1x0q7z0003l80g5d2k9w1e", "events": "/api/v1/tasks/cmh1x0q7z0003l80g5d2k9w1e/events" }
}
}
TASK_FAILED
Adds data.reason, a human-readable explanation.
{
"id": "cmh1x6v2b00a3l80g2j6d9k4q",
"type": "TASK_FAILED",
"apiVersion": "v1",
"createdAt": "2026-10-21T09:14:07.512Z",
"organizationId": "m7q2k9x4a1b8c3d5e6f0g7h2",
"data": {
"task": { "id": "cmh1x0q7z0003l80g5d2k9w1e", "status": "FAILED", "budget": 40, "spent": 0, "remainingBudget": 40 },
"reason": "The workflow did not finish within 10 execution segments. All escrowed funds have been refunded; please retry the task.",
"links": { "task": "/api/v1/tasks/cmh1x0q7z0003l80g5d2k9w1e", "events": "/api/v1/tasks/cmh1x0q7z0003l80g5d2k9w1e/events" }
}
}
CIRCUIT_BREAKER_TRIGGERED
Adds the agent that asked, the amount it requested (tokens) and the Circuit Breaker's reason. A blocked request moves no money: spent and remainingBudget are unchanged by it. An agent that makes a severe request can also be revoked permanently.
{
"id": "cmh1x6v2b00a4l80g8f3s1v7w",
"type": "CIRCUIT_BREAKER_TRIGGERED",
"apiVersion": "v1",
"createdAt": "2026-10-21T09:14:07.512Z",
"organizationId": "m7q2k9x4a1b8c3d5e6f0g7h2",
"data": {
"task": { "id": "cmh1x0q7z0003l80g5d2k9w1e", "status": "IN_PROGRESS", "budget": 50, "spent": 8, "remainingBudget": 42 },
"agentId": "rogue-agent",
"requestedAmount": 10000,
"reason": "budget exceeded",
"links": { "task": "/api/v1/tasks/cmh1x0q7z0003l80g5d2k9w1e", "events": "/api/v1/tasks/cmh1x0q7z0003l80g5d2k9w1e/events" }
}
}
Reasons you may see include budget exceeded, member spend cap exceeded, member suspended, insufficient escrow, unauthorized purpose, invalid amount and agent is revoked. Treat the text as informational.
APPROVAL_REQUESTED
Adds the approval request and links.decide, the endpoint to approve or reject it.
{
"id": "cmh1x6v2b00a5l80g4r9t2y6u",
"type": "APPROVAL_REQUESTED",
"apiVersion": "v1",
"createdAt": "2026-10-21T09:11:30.004Z",
"organizationId": "m7q2k9x4a1b8c3d5e6f0g7h2",
"data": {
"task": { "id": "cmh1x0q7z0003l80g5d2k9w1e", "status": "IN_PROGRESS", "budget": 120, "spent": 0, "remainingBudget": 120 },
"approval": {
"id": "cmh1x5k8e0007l80gw2q4z8n1",
"kind": "PLAN",
"amount": null,
"subtaskId": null,
"agentId": null,
"reasons": ["plan_policy"],
"expiresAt": "2026-10-21T10:11:30.000Z",
"details": {
"budget": 120,
"qualityThreshold": 75,
"dataSensitivity": "PUBLIC",
"estimatedMinCost": 24,
"steps": [
{ "subtaskId": "cmh1x3p1c0004l80gq8t6v1n2", "sequence": 1, "type": "market_research", "capability": "web_research", "description": "Collect pricing, fleet size and funding for the top European operators.", "dependsOnSequence": [], "minPrice": 4 }
],
"workflow": { "label": "Balanced", "assignments": [{ "subtaskId": "cmh1x3p1c0004l80gq8t6v1n2", "agentId": "agent-research-standard", "agentName": "Scout" }] }
}
},
"links": {
"task": "/api/v1/tasks/cmh1x0q7z0003l80g5d2k9w1e",
"events": "/api/v1/tasks/cmh1x0q7z0003l80g5d2k9w1e/events",
"decide": "/api/v1/approvals/cmh1x5k8e0007l80gw2q4z8n1"
}
}
}
details holds what a human approver sees in the dashboard:
PLAN: the budget, the minimum estimated cost, the steps, and the planned agent for each step.HIRE: the step, the agent (id,name,isExternal,providerName), the negotiatedamount, the remaining budget, the ranking score with its breakdown and explanation, and up to three alternatives.
Treat details as informational; its fields may grow. If nobody decides before expiresAt, the request counts as rejected.
The task snapshot in this event shows IN_PROGRESS or AWAITING_APPROVAL, depending on whether the task had finished parking when the delivery was created. Either way, the task is waiting for this decision.
Verify the signature
Verify every request before you trust it. The signature is:
hex( HMAC-SHA256( key = <your whsec_ secret, as UTF-8>, message = <timestamp> + "." + <raw request body> ) )
To verify a request:
- Read the raw body bytes before any JSON parsing. Re-serialized JSON won't match.
- Read
ThreadPool-Webhook-Timestamp. Reject the request if it is more than 5 minutes from your clock. This rejects replayed requests: the timestamp is inside the signed message, so an attacker can't refresh it without the secret. - Compute the expected signature with your secret.
- Split
ThreadPool-Webhook-Signatureon,. Accept the request if anyv1=value equals your signature, compared in constant time. - Only then parse the JSON.
Worked example (use it to test your code):
| Input | Value |
|---|---|
| secret | whsec_test_secret |
| timestamp | 1792584000 |
| raw body | {"id":"evt_1"} |
| signed message | 1792584000.{"id":"evt_1"} |
| signature | 559174633c32e7be7058ec24a42bc32a96a7ad851edce970d94cc39bcdf353a8 |
So that request carries ThreadPool-Webhook-Signature: v1=559174633c32e7be7058ec24a42bc32a96a7ad851edce970d94cc39bcdf353a8.
Node.js (Express)
import crypto from "node:crypto";
import express from "express";
const SECRET = process.env.THREADPOOL_WEBHOOK_SECRET; // whsec_...
const TOLERANCE_SECONDS = 300;
function verify(rawBody, timestamp, signatureHeader) {
if (!timestamp || !signatureHeader || !/^\d+$/.test(timestamp)) return false;
if (Math.abs(Date.now() / 1000 - Number(timestamp)) > TOLERANCE_SECONDS) return false;
const expected = crypto.createHmac("sha256", SECRET).update(`${timestamp}.${rawBody}`).digest();
return signatureHeader.split(",").some((part) => {
const [version, hex] = part.trim().split("=");
if (version !== "v1" || !/^[0-9a-f]{64}$/.test(hex ?? "")) return false;
return crypto.timingSafeEqual(Buffer.from(hex, "hex"), expected);
});
}
const app = express();
// express.raw keeps the exact bytes; express.json() would lose them.
app.post("/threadpool/webhooks", express.raw({ type: "application/json" }), (req, res) => {
const raw = req.body.toString("utf8");
if (!verify(raw, req.get("ThreadPool-Webhook-Timestamp"), req.get("ThreadPool-Webhook-Signature"))) {
return res.status(400).send("invalid signature");
}
const event = JSON.parse(raw);
res.status(204).end(); // acknowledge fast; do the work after
queueForProcessing(event); // your code: de-duplicate on event.id
});
Python (Flask)
import hashlib, hmac, os, time
from flask import Flask, request, abort
SECRET = os.environ["THREADPOOL_WEBHOOK_SECRET"].encode()
TOLERANCE_SECONDS = 300
app = Flask(__name__)
def verify(raw: bytes, timestamp: str | None, signature_header: str | None) -> bool:
if not timestamp or not signature_header or not timestamp.isdigit():
return False
if abs(time.time() - int(timestamp)) > TOLERANCE_SECONDS:
return False
expected = hmac.new(SECRET, timestamp.encode() + b"." + raw, hashlib.sha256).hexdigest()
return any(
part.strip().startswith("v1=") and hmac.compare_digest(part.strip()[3:], expected)
for part in signature_header.split(",")
)
@app.post("/threadpool/webhooks")
def threadpool_webhook():
raw = request.get_data() # raw bytes, before parsing
if not verify(raw, request.headers.get("ThreadPool-Webhook-Timestamp"), request.headers.get("ThreadPool-Webhook-Signature")):
abort(400)
event = request.get_json()
# de-duplicate on event["id"], then process (ideally on a queue)
return "", 204
A complete TypeScript receiver with no dependencies is in examples/quickstart.ts.
Respond, retries and timeouts
-
Respond with any
2xxwithin 10 seconds. Acknowledge first and do slow work afterwards, for example on a queue. -
What counts as a failure: any other status, a connection error, a TLS error, or no response within 10 seconds. Redirects (
3xx) are not followed and also count as failures, so configure the final URL. -
Retries. A failed delivery is retried with exponential backoff, up to 6 attempts in total:
Attempt Earliest time after the previous attempt 1 immediately when the event happens 2 30 seconds 3 2 minutes 4 8 minutes 5 32 minutes 6 2 hours 8 minutes After the 6th failure the delivery is marked FAILED and not retried again.
-
When retries actually run. Retries are carried out by the delivery sweep (
/api/webhooks/sweep) and whenever a new event for your organization is delivered. The times above are the earliest possible; actual times depend on how often your deployment runs the sweep. Operators: see Running the retry sweep. -
Blocked targets. A target that fails the URL rules at delivery time fails immediately, with no retries and no bytes sent.
-
Ordering. Not guaranteed. A retry can arrive after a later event. Use
createdAt, or read the task's current state withGET /api/v1/tasks/{id}.
Handle duplicates
The same event can arrive more than once, for example when your 2xx was lost on the way back. Make processing idempotent:
- De-duplicate on the body's
id(equal toThreadPool-Event-Id). Store processed ids, for example with a unique constraint, and skip repeats. - Act on current state. Prefer reading the task (
GET /api/v1/tasks/{id}) to trusting the snapshot indata.task. - Use the
CONFLICTcode. Deciding an approval twice is already safe: the second call gets409 CONFLICTand changes nothing.
Delivery log
Each delivery is recorded with its status (PENDING, DELIVERED, FAILED), the number of attempts, the last HTTP status or error, the duration, and the next retry time. View it under Organization → API → Webhooks → Deliveries, or with the .../deliveries route above.
Every attempt also writes to the organization's event log:
WEBHOOK_DELIVEREDwithwebhookId,deliveryId,eventId,eventType,taskId,attempt,statusCodeanddurationMs.WEBHOOK_FAILEDwith the same fields pluserror,willRetryandnextAttemptAt.
No webhook entries are written for organizations without endpoints.
Target URL rules (SSRF protection)
To stop webhooks being aimed at internal infrastructure, ThreadPool enforces these rules when you save a URL and again on every delivery:
- Scheme and form.
https://is required in production. No credentials in the URL (user:pass@) and no#fragment. - Ports. Only 443, 80, or 1024–65535.
- Hostnames. Must be fully qualified.
localhost, single-label names and internal suffixes (.local,.internal,.localhost,.lan,.home,.home.arpa,.corp,.intranet,.private) are refused. - IP addresses. Every address the hostname resolves to must be public. Loopback (
127.0.0.0/8,::1), private (10/8,172.16/12,192.168/16,fc00::/7), link-local and cloud metadata (169.254.0.0/16,fe80::/10), carrier-grade NAT, multicast, reserved and documentation ranges are blocked. So are IPv4-mapped, NAT64 and 6to4 forms, and alternative spellings such as2130706433or0x7f.1. - DNS rebinding. The address is checked at the moment of connection, and the connection goes to exactly that checked address. A hostname that resolved publicly when you saved it but points somewhere private later is refused.
- Redirects are never followed.
Local development. Outside production only, an operator can set WEBHOOK_ALLOW_PRIVATE_TARGETS=true to deliver to a receiver on the same machine or network. This setting is ignored when NODE_ENV=production.
Rotate the secret
Choose Rotate on the endpoint. You get a new secret, shown once. For the next 24 hours each delivery is signed with both secrets, so the signature header holds two v1= values. Rotate like this:
- Rotate in the dashboard.
- Deploy the new secret to your receiver. Either secret verifies during the 24 hours.
- After 24 hours, only the new secret is used.
n8n and other automation tools
ThreadPool sends plain signed HTTPS requests, so any tool that accepts a webhook works. For n8n:
- Add a Webhook node, set the method to POST, and copy its Production URL. n8n must be reachable from the internet over HTTPS.
- In ThreadPool, add that URL as an endpoint and subscribe to the events you want.
- In the Webhook node, turn on Raw Body. Then verify the signature in a Code node using the algorithm above. Self-hosted n8n needs
NODE_FUNCTION_ALLOW_BUILTIN=cryptofor this. - Respond quickly (the Webhook node's default "Immediately" mode does), then branch on
type. - To fetch the full report, call
GET /api/v1/tasks/{id}from an HTTP Request node with anAuthorization: Bearer <api key>header.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| Saving the URL fails: "Webhook URL rejected: …" | The URL breaks a URL rule: http in production, a private address, an internal name, a blocked port | Use a public HTTPS hostname. For local testing, expose your receiver with a tunnel that gives a public HTTPS URL |
| Nothing ever arrives | The endpoint is disabled, not subscribed to that event type, or was created after the event | Check the endpoint's events and status. Only events after creation are sent |
| Delivery log: "Redirect (301) not followed" | The URL redirects, e.g. http→https or a missing trailing slash | Save the final URL |
| Delivery log: "Timed out after 10000 ms" | The receiver does work before responding | Respond 2xx first, then process |
| Delivery log: "Blocked target: …" | The hostname now resolves to a private or reserved address | Fix DNS, or point at a public endpoint. These are not retried |
| Delivery log: "Endpoint responded with HTTP 401/403" | Your server or proxy rejected the request | Allow the path; authenticate with the signature, not with a header ThreadPool doesn't send |
| Your verification fails | The body was parsed and re-serialized, the wrong secret was used, or your clock is off | Verify the raw bytes, and check the secret (the dashboard shows its first characters). Keep your clock in sync (NTP) |
| The same event arrives twice | A retry after a lost acknowledgement | De-duplicate on id |
| Retries run much later than the schedule | The sweep runs rarely on this deployment | Operators: schedule the sweep more often (below) |
Running the retry sweep
These notes are for operators of a ThreadPool deployment.
GET or POST /api/webhooks/sweep retries due deliveries. It also creates deliveries that a crashed process missed, for events from the last 24 hours. It needs Authorization: Bearer <CRON_SECRET> or Bearer <ADMIN_API_TOKEN>, and is safe to call at any frequency.
vercel.json schedules it once a day, because some Vercel plans allow only daily cron jobs. For the backoff schedule above to hold, call it every minute from a scheduler you control, for example:
curl -fsS -X POST "https://threadpool.example.com/api/webhooks/sweep" -H "Authorization: Bearer $CRON_SECRET"
Two more settings:
WEBHOOKS_DISABLED=truestops all outbound deliveries and makes the sweep do nothing. When you remove it, the next sweep delivers events from the previous 24 hours that were missed.KRAVEN_SECRETS_KEY(32 bytes, hex or base64) encrypts webhook secrets at rest. Without it, a per-process key is used and existing endpoints can't sign after a restart. Rotate their secrets, or set the key.