# santoku-http

> transport-agnostic HTTP client front end: retry with jittered backoff, cancellation, query params, and request/response hooks over a pluggable socket backend

This file is one tab of the santoku reference, at https://santoku.dev/santoku-http/llms.txt. https://santoku.dev/santoku-http is the same tab in a browser, https://santoku.dev/llms.txt indexes every tab, and https://santoku.dev/llms-small.txt summarizes them all on one page.

santoku-http is a transport-agnostic HTTP client front end. Calling the module with a backend table yields a client with a five-function surface, fetch, get, post, on, and off, that layers query-string building, retry with jittered backoff, cancellation, and request/response hooks over a backend that does all the network I/O. A backend supplies sleep(ms) plus either fetch(url, opts) returning ok, resp or request(url, opts) returning a { cancel, await } handle. The same client code runs over santoku.socket in native scripts, santoku.web.socket in the browser, and santoku.resty.socket under OpenResty. The examples below cover the request verbs and the shared response fields, the arithmetic of the retry loop, cancellation at every phase including mid-backoff, both hook forms and how they interact with retry, and three fuller worked examples. The examples use tiny in-memory backends, the same technique the test suite uses, so every printed count is exact; apart from one query-string interlude that runs live, they are shown for reading rather than running in the page.

### Building a client over a backend

The module is a factory: pass a backend with fetch(url, opts) and sleep(ms), get back { fetch, get, post, on, off }.

```lua
local http = require("santoku.http")
local client = http({
  fetch = function (url, opts)
    return true, { status = 200, body = "hello from " .. url }
  end,
  sleep = function (ms) end,
})
local ok, resp = client.get("http://example/api")
print(ok, resp.status)
return resp.body
```

### GET with query params

get sets opts.method and appends str.to_query(opts.params) to the url before handing it to the backend. Values are percent-encoded; only string, number, and boolean params are kept, anything else is silently dropped. This is a get feature: post and fetch forward params untouched and no transport reads it, so for other verbs encode the query into the url yourself, as in posting to "/sync?since=" .. cursor. The built string always starts with "?", so hand get a url without one.

```lua
local http = require("santoku.http")
local seen
local client = http({
  fetch = function (url, opts)
    seen = url
    return true, { status = 200 }
  end,
  sleep = function (ms) end,
})
client.get("http://example/search", { params = { q = "hello world" } })
print("get:", seen)
client.post("http://example/search", { params = { q = "hello world" } })
print("post:", seen)
return seen
```

### Query strings by hand

The encoding under get lives in santoku.string, so it runs right here: to_query builds the "?k=v" string the client appends, to_formdata is the same pairs minus the leading "?" (Stripe bodies below), and from_query reverses both, decoding percent escapes and reviving numbers and booleans.

```lua
local str = require("santoku.string")
print(str.to_query({ q = "hello world" }))
print(str.to_formdata({ mode = "subscription" }))
local parsed = str.from_query("?page=2&debug=true&q=hello%20world")
print(parsed.page + 1, parsed.debug, parsed.q)
return str.to_query({ n = 42 })
```

### POST with headers and a body

post writes method = "POST" into the opts table you pass and forwards everything else untouched: headers and body reach the backend exactly as given, so encoding is the caller's job (str.to_formdata and cjson.encode in practice).

```lua
local http = require("santoku.http")
local seen
local client = http({
  fetch = function (url, opts)
    seen = opts
    return true, { status = 201 }
  end,
  sleep = function (ms) end,
})
local ok, resp = client.post("http://example/items", {
  headers = { ["Content-Type"] = "application/json" },
  body = "{\"name\":\"widget\"}",
})
print(seen.method, seen.headers["Content-Type"])
print(ok, resp.status)
return seen.body
```

### Any method through fetch

fetch is the primitive under get and post: set opts.method yourself for DELETE, PUT, or PATCH, passing method = "DELETE" and the rest unchanged.

```lua
local http = require("santoku.http")
local seen
local client = http({
  fetch = function (url, opts)
    seen = opts.method
    return true, { status = 204 }
  end,
  sleep = function (ms) end,
})
local ok, resp = client.fetch("http://example/items/42", { method = "DELETE" })
print("method:", seen)
print(ok, resp.status)
return resp.status
```

### The response convention

The client passes responses through untouched; their fields are a convention kept by all three shipped transports: ok is true for 2xx, status is a number, headers are lowercased, resp.ok mirrors the first return, and body is a function so streaming transports can defer the read.

```lua
local http = require("santoku.http")
local client = http({
  fetch = function (url, opts)
    local body = "{\"n\":1}"
    return true, {
      status = 200,
      ok = true,
      headers = { ["content-type"] = "application/json" },
      body = function () return body end,
    }
  end,
  sleep = function (ms) end,
})
local ok, resp = client.get("http://example/api")
print(ok, resp.ok, resp.status)
print(resp.headers["content-type"])
return resp.body()
```

### Transport failure, the same fields everywhere

When the network itself fails, before any status arrives, all three transports return the same sentinel: ok false, status 0, empty headers, error carrying the transport's reason (the LuaSec message natively, err from lua-resty-http, the thrown value in the browser), and body() returning nil. Status 0 counts as transient for the default retry filter, so probes that must answer fast pair it with retry = false, and a status of 0 is the natural place to map "offline".

```lua
local http = require("santoku.http")
local client = http({
  fetch = function (url, opts)
    return false, {
      status = 0,
      ok = false,
      headers = {},
      error = "connection refused",
      body = function () return nil end,
    }
  end,
  sleep = function (ms) end,
})
local ok, resp = client.get("http://example/api", { retry = false })
print(ok, resp.status, resp.error)
print(resp.body())
return resp.status
```

### Retry with jittered backoff

Retry is on by default: transient failures are re-fetched with a growing jittered delay between attempts. Here the third attempt succeeds, so the backend saw 3 calls and slept twice.

```lua
local http = require("santoku.http")
local calls, backoffs = 0, 0
local client = http({
  fetch = function (url, opts)
    calls = calls + 1
    if calls < 3 then
      return false, { status = 503 }
    end
    return true, { status = 200, body = "recovered" }
  end,
  sleep = function (ms)
    backoffs = backoffs + 1
  end,
})
local ok, resp = client.get("http://example/flaky")
print("attempts:", calls, "backoffs:", backoffs)
print(ok, resp.status)
return resp.body
```

### What retries by default

The default filter retries only when status is missing or 0 (transport failure) or one of 429, 502, 503, 504. Everything else, including 500 and 404, is returned immediately: exactly one backend call here.

```lua
local http = require("santoku.http")
local calls = 0
local client = http({
  fetch = function (url, opts)
    calls = calls + 1
    return false, { status = 500 }
  end,
  sleep = function (ms) end,
})
local ok, resp = client.get("http://example/bug")
print("attempts:", calls)
print(ok, resp.status)
return resp.status
```

### Retry exhaustion, exactly counted

The default policy is times = 3: the retry loop makes 4 filtered attempts with backoff after the first 3, then one final unconditional attempt whose result is returned as-is. A permanently failing endpoint therefore sees 5 backend calls and 3 sleeps.

```lua
local http = require("santoku.http")
local calls, sleeps = 0, 0
local client = http({
  fetch = function (url, opts)
    calls = calls + 1
    return false, { status = 503 }
  end,
  sleep = function (ms)
    sleeps = sleeps + 1
  end,
})
local ok, resp = client.get("http://example/down")
print("calls:", calls, "sleeps:", sleeps)
print(ok, resp.status)
return calls
```

### Custom retry policies

opts.retry takes times, backoff (initial ms), multiplier, and filter(ok, resp) deciding what counts as transient. With times = 1 the loop retries once with backoff, then makes the final unconditional attempt: 3 calls, 1 sleep.

```lua
local http = require("santoku.http")
local calls, sleeps = 0, 0
local client = http({
  fetch = function (url, opts)
    calls = calls + 1
    return false, { status = 500 }
  end,
  sleep = function (ms)
    sleeps = sleeps + 1
  end,
})
local ok, resp = client.get("http://example/fivehundred", {
  retry = {
    times = 1,
    backoff = 50,
    multiplier = 2,
    filter = function (ok, resp)
      return resp and resp.status == 500
    end,
  },
})
print("calls:", calls, "sleeps:", sleeps)
print(ok, resp.status)
return calls
```

### Disabling retry

opts.retry = false makes exactly one attempt and returns whatever comes back. Connectivity probes work this way: a nonce fetch with retry = false answers "am I online" fast instead of backing off for seconds.

```lua
local http = require("santoku.http")
local calls = 0
local client = http({
  fetch = function (url, opts)
    calls = calls + 1
    return false, { status = 503 }
  end,
  sleep = function (ms) end,
})
local ok, resp = client.get("http://example/nonce", { retry = false })
print("attempts:", calls)
print(ok, resp.status)
return resp.status
```

### Watching the backoff schedule

Each delay is backoff + backoff * rand.num(), so between 1x and 2x the current backoff, which is then multiplied for the next round. With backoff = 100 and multiplier = 2 the recorded delays land in [100, 200) and [200, 400).

```lua
local http = require("santoku.http")
local delays = {}
local client = http({
  fetch = function (url, opts)
    return false, { status = 503 }
  end,
  sleep = function (ms)
    delays[#delays + 1] = ms
  end,
})
client.get("http://example/down", {
  retry = { times = 2, backoff = 100, multiplier = 2 },
})
print("sleeps:", #delays)
print(delays[1] >= 100 and delays[1] < 200)
print(delays[2] >= 200 and delays[2] < 400)
return #delays
```

### Cancelable requests

With opts.cancelable, fetch returns { cancel, await } instead of resolving inline. A request canceled before await never touches the backend at all and resolves to false plus the { status = 0, canceled = true } sentinel.

```lua
local http = require("santoku.http")
local calls = 0
local client = http({
  fetch = function (url, opts)
    calls = calls + 1
    return true, { status = 200 }
  end,
  sleep = function (ms) end,
})
local req = client.fetch("http://example/slow", { cancelable = true })
req.cancel()
local ok, resp = req.await()
print("backend calls:", calls)
print(ok, resp.status, resp.canceled)
return resp.canceled
```

### Canceling during backoff

The retry loop re-checks the cancel flag after every sleep, so a cancel that lands mid-backoff stops the request before the next attempt. Here the backend's sleep stands in for a timeout task that fires cancel: one failing call, one sleep, then the sentinel instead of retries two through five.

```lua
local http = require("santoku.http")
local calls, sleeps = 0, 0
local req
local client = http({
  fetch = function (url, opts)
    calls = calls + 1
    return false, { status = 503 }
  end,
  sleep = function (ms)
    sleeps = sleeps + 1
    req.cancel()
  end,
})
req = client.fetch("http://example/slow", { cancelable = true })
local ok, resp = req.await()
print("calls:", calls, "sleeps:", sleeps)
print(ok, resp.status, resp.canceled)
return resp.canceled
```

### Backends with a request handle

A backend may expose request(url, opts) returning { cancel, await }; when present it is preferred over fetch, and client-side cancel is forwarded to the live handle mid-flight. This is how the browser transport aborts through AbortController and the resty transport closes its cosocket. Fetch-only backends get a no-op cancel plus the sentinel checks between attempts.

```lua
local http = require("santoku.http")
local used = {}
local client = http({
  request = function (url, opts)
    used[#used + 1] = "request"
    return {
      cancel = function () end,
      await = function ()
        return true, { status = 200 }
      end,
    }
  end,
  fetch = function (url, opts)
    used[#used + 1] = "fetch"
    return true, { status = 200 }
  end,
  sleep = function (ms) end,
})
local ok, resp = client.get("http://example/api")
print("used:", used[1], "count:", #used)
print(ok, resp.status)
return used[1]
```

### Observing traffic with sync hooks

on(event, handler) without the async flag registers an observer: it sees the values but its return is ignored, so it cannot replace them. Hooks fire per backend attempt, so a retried request logs once per try.

```lua
local http = require("santoku.http")
local arr = require("santoku.array")
local log = {}
local client = http({
  fetch = function (url, opts)
    return true, { status = 200 }
  end,
  sleep = function (ms) end,
})
client.on("request", function (url, opts)
  log[#log + 1] = "-> " .. url
end)
client.on("response", function (ok, resp)
  log[#log + 1] = "<- " .. resp.status
end)
client.get("http://example/a")
client.get("http://example/b")
print(arr.concat(log, "\n"))
return #log
```

### Rewriting traffic with async hooks

Passing true as the third argument to on makes the handler a santoku.async stage: it receives a continuation k first and whatever it passes to k replaces the url and opts (request) or ok and resp (response) for everything downstream.

```lua
local http = require("santoku.http")
local seen
local client = http({
  fetch = function (url, opts)
    seen = url
    return true, { status = 200, body = "raw" }
  end,
  sleep = function (ms) end,
})
client.on("request", function (k, url, opts)
  return k(url .. "?traced=1", opts)
end, true)
client.on("response", function (k, ok, resp)
  resp.body = resp.body .. " (rewritten)"
  return k(ok, resp)
end, true)
local ok, resp = client.get("http://example/api")
print("backend saw:", seen)
print(ok, resp.body)
return resp.body
```

### Hooks meet the retry filter

Response hooks run inside each attempt, before the retry filter sees the result, so an async hook can rewrite a transient failure into a success and the loop never retries at all. A hook serving a cached fallback for 503 turns a would-be 5-call exhaustion into a single attempt.

```lua
local http = require("santoku.http")
local calls = 0
local client = http({
  fetch = function (url, opts)
    calls = calls + 1
    return false, { status = 503 }
  end,
  sleep = function (ms) end,
})
client.on("response", function (k, ok, resp)
  if resp and resp.status == 503 then
    return k(true, { status = 200, ok = true, body = "served from cache" })
  end
  return k(ok, resp)
end, true)
local ok, resp = client.get("http://example/flaky")
print("attempts:", calls)
print(ok, resp.status, resp.body)
return resp.body
```

### Removing hooks

off(event, handler) unregisters by identity, so keep a reference to the function you passed to on.

```lua
local http = require("santoku.http")
local count = 0
local client = http({
  fetch = function (url, opts)
    return true, { status = 200 }
  end,
  sleep = function (ms) end,
})
local tap = function (url, opts)
  count = count + 1
end
client.on("request", tap)
client.get("http://example/one")
client.off("request", tap)
client.get("http://example/two")
print("hook fired:", count)
return count
```

### One client, three transports

Everything above is transport-independent. santoku.web.socket wraps browser fetch with AbortController; santoku.socket wraps LuaSec's ssl.https with ltn12 for scripts and CLIs; santoku.resty.socket wraps lua-resty-http cosockets with ngx.sleep. A client-side platform module is usually just the single web line.

```lua
local http = require("santoku.http")
local in_browser = http(require("santoku.web.socket"))
local in_scripts = http(require("santoku.socket"))
local under_nginx = http(require("santoku.resty.socket"))
return in_browser, in_scripts, under_nginx
```

### Version pinning through hooks

santoku.web.version.install_hooks is a real hook consumer: an async request hook stamps x-client-version on same-origin requests, and a response hook compares the server's x-app-version header, firing on_mismatch once when the deploy has moved ahead. Server-side, version.check under OpenResty answers a stale client with 409.

```lua
local http = require("santoku.http")(require("santoku.web.socket"))
local version = require("santoku.web.version")
version.install_hooks(http, "42", function ()
  print("server is ahead, reload for the new bundle")
end)
local ok, resp = http.get("/api/items")
print(ok, resp.status)
return resp.body()
```

### Stripe from OpenResty

A billing integration: form-encoded bodies via str.to_formdata, bearer auth in headers, and body() read once up front so both the error path and the success path can use it.

```lua
local json = require("cjson")
local str = require("santoku.string")
local http = require("santoku.http")(require("santoku.resty.socket"))
local secret_key = "sk_test_..."
local ok, resp = http.fetch("https://api.stripe.com/v1/checkout/sessions", {
  method = "POST",
  headers = {
    ["Authorization"] = "Bearer " .. secret_key,
    ["Content-Type"] = "application/x-www-form-urlencoded",
  },
  body = str.to_formdata({
    mode = "subscription",
    success_url = "https://example.com/ok",
    ["line_items[0][quantity]"] = "1",
  }),
})
local body = resp.body()
if not ok then
  return nil, "request failed: " .. (resp.error or body or "unknown")
end
if resp.status < 200 or resp.status >= 300 then
  return nil, "stripe error " .. resp.status .. ": " .. (body or "")
end
return json.decode(body).url
```

### AWS APIs: SigV4 and instance-role credentials

santoku.http.aws signs requests with AWS Signature Version 4 and needs no SDK. sign(r) takes method, host, path, query (the raw query string), headers, body, region, service, time (epoch seconds) and credentials (access_key_id, secret_access_key, session_token), and returns the headers to add: Authorization, X-Amz-Date, and X-Amz-Security-Token when there is a token. sign_body = true also signs and returns X-Amz-Content-Sha256. It is pure Lua over santoku.string, so it runs natively, in the browser and under OpenResty, and its spec passes AWS's published SigV4 test suite. instance_credentials(client[, opts]) returns a function that reads an EC2 instance role's credentials through IMDSv2 with the client you pass, and caches them until opts.margin seconds (300 by default) before they expire. The example calls Cognito's JSON 1.1 API.

```lua
local json = require("cjson")
local utc = require("santoku.utc")
local http = require("santoku.http")(require("santoku.resty.socket"))
local aws = require("santoku.http.aws")
local creds = aws.instance_credentials(http)
local host = "cognito-idp.us-east-1.amazonaws.com"
local body = json.encode({ UserPoolId = "us-east-1_EXAMPLE", Username = "a@example.com" })
local headers = {
  ["Content-Type"] = "application/x-amz-json-1.1",
  ["X-Amz-Target"] = "AWSCognitoIdentityProviderService.AdminGetUser",
}
local add = aws.sign({
  method = "POST", host = host, path = "/", headers = headers, body = body,
  region = "us-east-1", service = "cognito-idp", time = utc.time(), credentials = creds(),
})
for k, v in pairs(add) do
  headers[k] = v
end
local ok, resp = http.post("https://" .. host .. "/", { headers = headers, body = body })
return ok and json.decode(resp.body())
```

### A cancelable sync loop

A sync orchestrator: the in-flight post is held in a module-local so lock or teardown can abort it mid-flight; the canceled sentinel, offline (status 0), and protocol statuses each route differently, and the connectivity probe runs with retry = false while the sync post keeps the default retry.

```lua
local http = require("santoku.http")(require("santoku.web.socket"))
local active
local function sync (cursor, body, headers)
  active = http.post("/sync?since=" .. cursor, {
    cancelable = true,
    headers = headers,
    body = body,
  })
  local _, response = active.await()
  active = nil
  if not response then return "offline" end
  if response.canceled then return nil end
  local status = tonumber(response.status) or 0
  if status == 0 then return "offline" end
  if status == 401 then return "auth-required" end
  if status == 409 then return "resync" end
  if not response.ok then return false end
  return response.body()
end
local function stop ()
  if active then active.cancel() end
end
return sync, stop
```