Waafir
Cookbook

Cancel an in-flight import

This recipe shows how to cancel an async import while it is still processing. The worker checks for a cancellation request at every batch boundary; once it sees one, it stops at the end of the in-progress batch and flips the status to cancelled. Rows that completed before the cancellation stay landed — there are no partially-written batches to clean up after.

When to cancel

The cancel endpoint is the right tool when you realise mid-flight that the import is wrong: the wrong file, the wrong portfolio, a stale snapshot, the wrong day. Typical triggers:

  • A monitoring dashboard catches the wrong file pointed at the import.
  • You started a multi-million-row backfill against the production portfolio when you meant to dry-run it first.
  • An upstream system flagged the file as corrupted after the upload completed.

Cancellation is not a substitute for dryRun: true (see the dry-run recipe). Once a real import is running, rows that succeed before the cancel point are committed. You can rely on Waafir not to write partial batches but you cannot rewind already-committed rows; the right pattern is to follow up with a corrective import that adjusts what landed wrong.

The batch-boundary guarantee

The worker processes rows in batches of two hundred. After each batch it checks the import status; if it sees cancelled, it stops there and the status stays cancelled. The guarantee is:

  • Every batch the worker started before the cancel signal completes fully. Those rows are committed.
  • Every batch the worker had not yet started when the cancel signal arrived is skipped. Those rows do not land.
  • There is never an in-between state with half a batch landed. If a batch starts, it finishes.

The terminal cancelled status carries the row counts at the moment of cancellation — that is the count of what actually committed.

The recipe

# curl — cancel an in-flight import
set -euo pipefail

PAT="${WAAFIR_PAT}"
BASE_URL="${WAAFIR_BASE_URL:-https://app.waafir.io}"
IMPORT_ID="${1:?supply the import id}"

# Step 1 — DELETE the import. The endpoint returns immediately with the
# (eventual) cancellation in motion. A second DELETE is idempotent —
# already-cancelled imports return the same final status.
curl -fsS -X DELETE "${BASE_URL}/api/v1/monitoring/loan/imports/${IMPORT_ID}" \
  -H "Authorization: Bearer ${PAT}" \
  | jq '{importId, status}'

# Step 2 — wait for the worker to reach the next batch boundary and
# flip to a terminal status.
TERMINAL_PATTERN='succeeded|partial|failed|cancelled'
while :; do
  STATUS=$(curl -fsS "${BASE_URL}/api/v1/monitoring/loan/imports/${IMPORT_ID}" \
    -H "Authorization: Bearer ${PAT}" | jq -r .status)
  echo "import ${IMPORT_ID}: ${STATUS}"
  if echo "$STATUS" | grep -Eq "^(${TERMINAL_PATTERN})$"; then break; fi
  sleep 2
done

# Step 3 — read the final row counts to know how many rows actually committed.
curl -fsS "${BASE_URL}/api/v1/monitoring/loan/imports/${IMPORT_ID}" \
  -H "Authorization: Bearer ${PAT}" \
  | jq '{status, rowCountProcessed, rowCountSucceeded, rowCountFailed}'
// TypeScript — cancel an in-flight import
const PAT = process.env.WAAFIR_PAT!;
const BASE_URL = process.env.WAAFIR_BASE_URL ?? "https://app.waafir.io";
const importId = process.argv[2];
if (!importId) throw new Error("usage: ts-node cancel.ts <importId>");

const auth = { Authorization: `Bearer ${PAT}` };

// Step 1 — DELETE the import; the response confirms the signal was accepted.
const cancelRes = await fetch(`${BASE_URL}/api/v1/monitoring/loan/imports/${importId}`, {
    method: "DELETE",
    headers: auth,
});
if (!cancelRes.ok) throw new Error(`cancel failed: ${cancelRes.status} ${await cancelRes.text()}`);
console.log("cancel accepted:", await cancelRes.json());

// Step 2 — poll for terminal status.
const TERMINAL = new Set(["succeeded", "partial", "failed", "cancelled"]);
type Status = {
    status: string;
    rowCountProcessed: number;
    rowCountSucceeded: number;
    rowCountFailed: number;
};
let final: Status;
while (true) {
    final = (await (await fetch(
        `${BASE_URL}/api/v1/monitoring/loan/imports/${importId}`,
        { headers: auth },
    )).json()) as Status;
    console.log(`import ${importId}: ${final.status}`);
    if (TERMINAL.has(final.status)) break;
    await new Promise((r) => setTimeout(r, 2_000));
}

// Step 3 — what actually landed.
console.log({
    status: final.status,
    rowCountProcessed: final.rowCountProcessed,
    rowCountSucceeded: final.rowCountSucceeded,
    rowCountFailed: final.rowCountFailed,
});
# python — cancel an in-flight import
import os
import sys
import time
import requests

PAT = os.environ["WAAFIR_PAT"]
BASE_URL = os.environ.get("WAAFIR_BASE_URL", "https://app.waafir.io")
import_id = sys.argv[1] if len(sys.argv) > 1 else None
if not import_id:
    raise SystemExit("usage: python cancel.py <importId>")

auth = {"Authorization": f"Bearer {PAT}"}

# Step 1 — DELETE the import.
cancel = requests.delete(
    f"{BASE_URL}/api/v1/monitoring/loan/imports/{import_id}",
    headers=auth,
    timeout=30,
)
cancel.raise_for_status()
print("cancel accepted:", cancel.json())

# Step 2 — poll for terminal status.
terminal = {"succeeded", "partial", "failed", "cancelled"}
while True:
    final = requests.get(
        f"{BASE_URL}/api/v1/monitoring/loan/imports/{import_id}",
        headers=auth,
        timeout=30,
    ).json()
    print(f"import {import_id}: {final['status']}")
    if final["status"] in terminal:
        break
    time.sleep(2)

# Step 3 — what actually landed.
print({
    "status": final["status"],
    "rowCountProcessed": final["rowCountProcessed"],
    "rowCountSucceeded": final["rowCountSucceeded"],
    "rowCountFailed": final["rowCountFailed"],
})

What to verify

After the polling loop terminates, status is cancelled if the cancel signal reached the worker before it finished, or one of succeeded / partial / failed if the worker finished the entire file before the next batch boundary check (rare, but possible for small files).

The rowCountSucceeded field is the number of rows that committed before the cancel. Reconcile those rows by external ID (see the reconciliation recipe) if you need a definitive list of what landed.

A second DELETE on the same import returns the existing terminal status without re-issuing the cancel; the endpoint is idempotent. Cancelling an already-terminal import (one that already reached succeeded, partial, or failed) is a no-op that returns the existing status.

This recipe anchors the friction scenario import-cancellation-mid-flight — the catalog entry pins the contract that cancellation during an in-progress run stops the worker at the next batch boundary without leaving partially-written rows.

Inspect partial-failure errors after an import

Read the error summary and page through per-row errors when an async import lands as partial or failed — pinpoint the offending rows by row number, error code, and raw payload.

Reconcile your data by external ID

Confirm a loan landed under your own identifier, then verify daily counts per entity — the two reconciliation endpoints close the loop on every ingestion submission.