Skip to content

Developers

Keep your system in sync

Mirror your Demurly records in a TMS, ERP or AP system without missing a change or applying one twice.

On this page

The approach

Use the API as the record of truth, and two ways of learning that something changed:

  • Poll GET /api/v1/records with updated_since every few minutes. This alone is a complete sync, and it catches anything a webhook missed.
  • Webhooks tell you within seconds. Treat each as a nudge: check it's new, then read the record.

Store each record by its id with its updated_at, and only overwrite it with a copy whose updated_at is newer.

Poll for changes

updated_since returns every record whose updated_at is at or after that time. updated_at changes on anything: a status change, a new amount, a comment or a photo. The list is ordered newest-created first, not by updated_at, so read every page of a poll before you move your watermark.

Poll
// Run every few minutes. "store" is your database.
const OVERLAP_MS = 2 * 60_000; // re-read the last 2 minutes: see "Why the overlap"

export async function syncRecords(store) {
  const startedAt = new Date();
  const since = await store.getWatermark(); // ISO string, or null the first time
  let cursor = null;
  do {
    const query = new URLSearchParams({ limit: "100" });
    if (since) query.set("updated_since", since);
    if (cursor) query.set("cursor", cursor);
    const response = await fetch(`https://demurly.com/api/v1/records?${query}`, {
      headers: { Authorization: `Bearer ${process.env.DEMURLY_API_KEY}` },
    });
    const page = await response.json();
    if (!response.ok) throw new Error(`${page.error.code}: ${page.error.message} (${page.error.request_id})`);
    for (const record of page.data) {
      if (record.status === "withdrawn") {
        await store.closeCharge(record.id, record.withdrawn_reason); // a tombstone: see below
      } else {
        // Keep the newer copy: a record can come back on the next poll
        await store.upsertIfNewer(record.id, record.updated_at, record);
      }
    }
    cursor = page.next_cursor;
  } while (cursor);
  // Only after every page: next time, start a little before this run began
  await store.setWatermark(new Date(startedAt.getTime() - OVERLAP_MS).toISOString());
}

Why the overlap

A change is stamped with its updated_at as it starts and becomes visible when it finishes, a moment later. A poll that lands in between won't see it, and the next poll would skip it if it started exactly where the last one ended. Starting each poll a couple of minutes before the previous one began (and keeping the newer copy) catches those. Don't use the largest updated_at you saw as the watermark for the same reason.

The first run (no watermark) reads every record. If you ever lose your watermark or suspect a gap, run without it again: the upsert makes it safe.

Withdrawn charges

For a fuel buyer, a charge it was sent never vanishes. If the carrier cancels it, sends it to another buyer or takes it back, your next poll returns a tombstone in its place: status: "withdrawn", withdrawn_reason (cancelled, reassigned or recalled) and withdrawn_at, with an updated_at of when it was withdrawn so a poll sees it once. Close the payable on your side. If the carrier sends it to you again, it comes back as an ordinary record with a newer updated_at. Add status=withdrawn to list only tombstones.

Add webhooks

Deliveries can repeat and arrive out of order. Drop an id you've handled, ignore an event whose sequence is lower than the last you applied for that record, and read the current record rather than trusting the snapshot in the body:

Handle an event
// After verifying the signature (see the webhooks guide)
export async function onDemurlyEvent(event, store) {
  if (await store.seenEvent(event.id)) return; // at-least-once: drop repeats
  if (event.record_id) {
    const last = await store.lastSequence(event.record_id);
    if (event.sequence !== null && last !== null && event.sequence <= last) return; // older than what we have
    // The body's record is a snapshot from when it was first sent; read the current one
    const response = await fetch(`https://demurly.com/api/v1/records/${event.record_id}`, {
      headers: { Authorization: `Bearer ${process.env.DEMURLY_API_KEY}` },
    });
    if (response.ok) {
      const { data } = await response.json();
      if (data.status === "withdrawn") await store.closeCharge(data.id, data.withdrawn_reason);
      else await store.upsertIfNewer(data.id, data.updated_at, data);
    }
    await store.setLastSequence(event.record_id, event.sequence);
  }
  await store.markEventSeen(event.id);
}

Answer the webhook with a 2xx first and do this work afterwards (a queue), so slow work doesn't time out the delivery. Signatures and the delivery rules are in the webhooks guide.

Checklist

  • Upsert by id; keep the copy with the newer updated_at.
  • Read every page of a poll; move the watermark only after the last one, to the poll's start minus a couple of minutes.
  • Handle tombstones (status: "withdrawn") if you're a fuel buyer.
  • Ignore fields and statuses you don't know; new ones are added without notice.
  • Keep to the rate limit: one poll every few minutes with limit=100 is far inside it.
  • Log the X-Request-Id of failed calls. The error catalogue says which to retry.
NextConventions and policiesTimes and time zones, money, IDs, idempotency, versioning, testing and browser use.
Keep your system in sync