santoku

the base library: arrays, strings, tables, functions, numbers, and fractional order keys

santoku is the base library of the framework: plain Lua 5.1 modules for arrays, strings, tables, functions, numbers, ordering keys, errors, validation, time, randomness, and async control flow, with no dependencies beyond Lua itself and a few small C helpers. The examples below run from basics to advanced, and every example is live: edit the code and press Run to execute it right here, in a Lua interpreter compiled to WebAssembly.

santoku.array: pipelines

In-place array pipelines: build a range, filter it, map it, fold it.

local arr = require("santoku.array")
local t = arr.range(1, 10)
arr.filter(t, function (v)
  return v % 2 == 0
end)
arr.map(t, function (v)
  return v * v
end)
print("squares:", arr.concat(t, ", "))
return arr.sum(t)

santoku.array: in-place or copying, your choice

Most transforms come in pairs: sort/sorted, map/mapped, filter/filtered. Each takes the array first and the function second, as in arr.mapped(t, fn, ...), and extra arguments pass through to fn after the element. The bare name mutates its argument, the -ed form returns a fresh table. sort takes either a comparator or an options table with unique = true.

local arr = require("santoku.array")
local str = require("santoku.string")
local words = { "pear", "fig", "apple", "banana" }
local by_len = arr.sorted(words, function (a, b)
  return #a < #b
end)
print("shortest first:", arr.concat(by_len, " "))
print("original intact:", arr.concat(words, " "))
local dedup = arr.sort({ 3, 1, 2, 3, 1 }, { unique = true })
print("sorted unique:", arr.concat(dedup, " "))
local caps = arr.mapped(words, str.upper)
print("mapped copy:", arr.concat(caps, " "))
print("middle:", arr.concat(arr.slice(words, 2, 3), " "))
print("last two:", arr.concat(arr.takelast(words, 2), " "))
return arr.concat(arr.reverse(arr.filtered(words, function (w)
  return #w > 3
end)), " ")

santoku.array: stacks, queues, and edits

push/pop and shift make stacks and queues; insert, remove, find, and includes edit and query; tabulate turns a positional row into a named record, with a rest key for the tail.

local arr = require("santoku.array")
local stack = {}
arr.push(stack, "a", "b", "c")
local _, top = arr.pop(stack)
print("popped:", top)
local queue = { "first", "second", "third" }
local _, head = arr.shift(queue)
print("shifted:", head)
arr.insert(queue, 1, "zeroth")
arr.remove(queue, 2, 2)
print("queue:", arr.concat(queue, " "))
local v, i = arr.find(queue, function (x)
  return x == "third"
end)
print("found:", v, "at index", i)
print("includes:", arr.includes(queue, "third", "missing"))
local row = { "Ada", "Lovelace", 1815, "London", "mathematics" }
local person = arr.tabulate(row, { rest = "tags" }, "first", "last", "born")
print("record:", person.first, person.last, person.born)
return arr.concat(person.tags, ", ")

santoku.array: grouping and reshaping

group buckets by a key function, partition splits on a predicate, zip/unzip pair and unpair, chunked windows, flatten unnests, and uniqued dedups without sorting.

local arr = require("santoku.array")
local nums = arr.range(1, 10)
local by_parity = arr.group(nums, function (v)
  return v % 2 == 0 and "even" or "odd"
end)
print("evens:", arr.concat(by_parity.even, " "))
local small, large = arr.partition(nums, function (v)
  return v <= 5
end)
print("small:", arr.concat(small, " "))
print("large:", arr.concat(large, " "))
local zipped = arr.zip({ "x", "y", "z" }, { 1, 2, 3 })
print("second pair:", zipped[2][1], zipped[2][2])
local letters, values = arr.unzip(zipped)
print("unzipped:", arr.concat(letters, ""), arr.concat(values, ""))
local windows = arr.chunked(nums, 4)
print("chunks:", #windows, "last:", arr.concat(windows[#windows], " "))
local flat = arr.flatten({ 1, { 2, 3 }, { 4, { 5 } } })
print("flattened one level:", #flat, "items")
return arr.concat(arr.uniqued({ 1, 2, 2, 3, 3, 3 }), " ")

santoku.array: numbers in tables

Arrays of numbers get sum, mean, max/min (value and index), plus small vector math: dot products, magnitudes, scalar and elementwise arithmetic. Range arguments let every one of these work on a slice without copying.

local arr = require("santoku.array")
local xs = { 3, 1, 4, 1, 5, 9, 2, 6 }
print("sum:", arr.sum(xs), "mean:", arr.mean(xs))
local m, mi = arr.max(xs)
print("max:", m, "at index", mi)
print("mean of first four:", arr.mean(xs, 1, 4))
local a = { 1, 2, 3 }
local b = { 4, 5, 6 }
print("dot:", arr.dot(a, b))
print("magnitude:", arr.magnitude({ 3, 4 }))
arr.scale(a, 10)
print("scaled:", arr.concat(a, " "))
arr.addv(a, b)
print("elementwise sum:", arr.concat(a, " "))
arr.add(b, 100)
return arr.concat(b, " ")

santoku.array: lifting iterators

The i-prefixed functions consume any Lua iterator (gmatch, ipairs, your own) without building an intermediate table first: imap, ifilter, ireduce, and icollect with an optional limit.

local arr = require("santoku.array")
local str = require("santoku.string")
local words = arr.imap(str.upper, ("lift map over any iterator"):gmatch("%a+"))
print("mapped:", arr.concat(words, " "))
local long = arr.ifilter(function (w)
  return #w > 3
end, ("one two three four"):gmatch("%a+"))
print("filtered:", arr.concat(long, " "))
local first3 = arr.icollect(3, ("a b c d e"):gmatch("%a+"))
print("first three:", arr.concat(first3, " "))
return arr.ireduce(function (acc, field)
  return acc + #field
end, 0, ("ab,cde,f"):gmatch("[^,]+"))

santoku.string: splitting and matching

santoku.string is a superset of Lua's string table: find, sub, match, gmatch, gsub, lower, upper, rep, byte, char and format are all there, so code never needs string.* directly. splits cuts on a Lua pattern; pass true as the third argument to keep the delimiters as their own elements. matches collects every match of a pattern into an array.

local str = require("santoku.string")
local arr = require("santoku.array")
local parts = str.splits("one,two,three", ",")
print("fields:", #parts, arr.concat(parts, "|"))
local kept = str.splits("key=value", "=", true)
print("with delimiter:", arr.concat(kept, " "))
local nums = arr.map(str.matches("10 39.5 46.8", "%S+"), tonumber)
print("parsed sum:", nums[1] + nums[2] + nums[3])
return arr.concat(str.splits("path/to/some/file.lua", "/"), " > ")

santoku.string: interp and parse

interp interpolates by name, position, or with printf formats via %fmt#(key); parse is its inverse, mapping capture groups tagged with #(key) into a named table.

local str = require("santoku.string")
local num = require("santoku.num")
print(str.interp("Hello %who, %adj to meet you!", {
  who = "World",
  adj = "nice",
}))
print(str.interp("%1 %3 %2", { "a", "b", "c" }))
print(str.interp("pi is %.3f#(pi)", { pi = num.pi }))
print(str.interp("%(num_to_display)", { num_to_display = 7 }))
local t = str.parse("2023-10-26 09:10:26",
  "(%d+)#(year)-(%d+)#(month)-(%d+)#(day) (%d+)#(hour):(%d+)#(min):(%d+)#(sec)")
print("year:", t.year, "hour:", t.hour)
return t.month

santoku.string: predicates and trimming

Everyday string chores: prefix and suffix tests, trimming with custom patterns, common prefixes across many strings, occurrence counts, human-readable number formatting, and quoting. quote(s[, q[, e]]) wraps s in q, a double quote by default, escaping e (a backslash by default) and q inside it with e; unquote reverses it and returns s unchanged when it isn't wrapped in q.

local str = require("santoku.string")
print("starts:", str.startswith("santoku.array", "santoku."))
print("ends:", str.endswith("init.lua", ".lua"))
print("trimmed:", "[" .. str.trim("  padded  ") .. "]")
print("custom trim:", str.trim("__flag__", "_*"))
print("stripped:", str.stripprefix("santoku.array", "santoku."))
print("common:", str.commonprefix("santoku.array", "santoku.string", "santoku.table"))
print("count:", str.count("banana", "an"))
print("empty:", str.isempty("   "), str.isempty("x"))
print("quoted:", str.quote("say \"hi\""))
print("unquoted:", str.unquote(str.quote("say \"hi\"")))
return str.format_number(1234567.89)

santoku.string: hashing and encodings

The C core merged into santoku.string covers SHA-256 digests, hex, base64 (plain and URL-safe, both padded unless you pass false as the last argument: to_base64(s, url, pad), to_base64_url(s, pad)), SHA-1 for protocols that require it, percent-encoding, parsing a number out of the middle of a string, and utf8_char, which encodes codepoints as UTF-8 and raises on a surrogate or anything past U+10FFFF. utf8_next(s, i) returns the codepoint at byte i and its width in bytes, utf8_len(s) counts codepoints, and utf8_lower and utf8_fold lowercase and case-fold with the Unicode tables; each returns nothing on invalid UTF-8. sha1(s) and sha256(s) return the raw digest bytes (20 and 32), so wrap them in to_hex or to_base64 to print. xor(data, key) XORs data with key repeated to its length and raises "empty key" on an empty key.

local str = require("santoku.string")
local digest = str.sha256("hello")
print("sha256:", str.to_hex(digest))
print("sha1:", str.to_hex(str.sha1("hello")))
local masked = str.xor("secret", "k")
print("xor round trip:", str.xor(masked, "k"))
local b64 = str.to_base64("hello, world")
print("base64:", b64)
print("decoded:", str.from_base64(b64))
print("url-safe:", str.to_base64_url("data?with=symbols"))
local enc = str.to_url("a value & more")
print("percent-encoded:", enc)
print("round trip:", str.from_url(enc))
print("number at offset 2:", str.number("v1.25-beta", 2))
print("utf8_char:", str.utf8_char(99, 97, 102, 233, 8364))
return str.to_hex("\1\2\255")

santoku.string: URLs and query strings

parse_url decomposes a URL into scheme, host, port, path segments, typed params (numbers and booleans are converted), and fragment; encode_url rebuilds one, and to_query/from_query handle bare query strings.

local str = require("santoku.string")
local arr = require("santoku.array")
local u = str.parse_url("https://example.com:8080/docs/intro?page=2&dark=true#usage")
print("scheme:", u.scheme, "host:", u.host, "port:", u.port)
print("path:", arr.concat(u.path, "/"))
print("params:", u.params.page, u.params.dark)
print("fragment:", u.fragment)
u.params.page = u.params.page + 1
u.search = nil
print("rebuilt:", str.encode_url(u))
local q = str.from_query("?limit=10&q=lua")
print("typed:", q.limit + 5, q.q)
return str.to_query({ sort = "asc" })

santoku.string: natural sort order

compare_natural(a, b) is a less-than for arr.sort that ignores case and compares each run of digits by its numeric value, so item 2 sorts before item 10 and v1.2 before v1.10. When two digit runs have the same value, the shorter run sorts first, so x9 comes before x09. Case folding is ASCII only; accented letters compare by their bytes.

local str = require("santoku.string")
local arr = require("santoku.array")
local names = { "item 10", "Item 2", "item 1", "v1.10", "v1.2", "x09", "x9" }
arr.sort(names, str.compare_natural)
print(arr.concat(names, ", "))
return names[1]

santoku.table: paths into nested tables

get, set, and update address deep keys as a path array, creating intermediate tables on the way down; merge fills in missing keys recursively without clobbering what is already there.

local tbl = require("santoku.table")
local cfg = { server = { host = "localhost" } }
tbl.set(cfg, { "server", "ports", 1 }, 8080)
tbl.update(cfg, { "server", "ports", 1 }, function (p)
  return p + 1
end)
tbl.merge(cfg, {
  server = { host = "ignored", scheme = "https" },
})
print("host:", tbl.get(cfg, { "server", "host" }))
print("scheme:", tbl.get(cfg, { "server", "scheme" }))
print("missing is nil:", tbl.get(cfg, { "server", "tls", "cert" }))
return tbl.get(cfg, { "server", "ports", 1 })

santoku.table: dictionaries as data

keys, vals, and entries project a map into arrays; from indexes an array by a key function; invert flips keys and values; equals compares deeply and returns a reason on mismatch.

local tbl = require("santoku.table")
local arr = require("santoku.array")
local ages = { ada = 36, alan = 41, grace = 85 }
print("keys:", arr.concat(arr.sort(tbl.keys(ages)), " "))
print("total:", arr.sum(tbl.vals(ages)))
local by_letter = tbl.from({ "lua", "santoku" }, function (s)
  return s:sub(1, 1)
end)
print("l:", by_letter.l, "s:", by_letter.s)
local flipped = tbl.invert({ a = 1, b = 2 })
print("flipped:", flipped[1], flipped[2])
print("deep equal:", tbl.equals({ x = { 1, 2 } }, { x = { 1, 2 } }))
local _, why = tbl.equals({ x = 1 }, { x = 2 })
return why

santoku.functional

Function combinators: bind for partial application, compose, const, sel and take to reshape argument lists, choose as an expression-level branch, and get to turn a field name into an accessor.

local fun = require("santoku.functional")
local arr = require("santoku.array")
local function add (a, b)
  return a + b
end
local inc = fun.bind(add, 1)
local function double (x)
  return x * 2
end
print("compose:", fun.compose(double, inc)(10))
local from_second = fun.sel(fun.id, 2)
print("sel:", from_second("a", "b", "c"))
local print1 = fun.take(print, 1)
print1("kept", "dropped", "dropped")
print("choose:", fun.choose(1 > 2, "yes", "no"))
local names = arr.mapped({ { name = "ada" }, { name = "alan" } }, fun.get("name"))
print("names:", arr.concat(names, " "))
return fun.const(42)()

santoku.num

Rounding, decimal truncation, exponential moving averages, and a C one-dimensional minimizer; the module also re-exports all of math. round(n) rounds half up to an integer; round(n, step) rounds up to the next multiple of step. bracket(f, x0, h[, maxit]) walks downhill from x0 in steps of h (40 at most by default) and returns xa, xb, xc and f(xb), with xb lower than its neighbours when it succeeds. brent(f, ax, bx, cx, fbx, tol, itmax) refines that bracket and returns the minimizing x and f(x). Both raise "objective must return a number" when f returns anything else.

local num = require("santoku.num")
print("round:", num.round(2.5), num.round(2.4))
print("round up to 0.25 steps:", num.round(0.30, 0.25))
local f = function (x) return (x - 2) ^ 2 + 1 end
local xa, xb, xc, fb = num.bracket(f, 0, 0.5)
local x, fx = num.brent(f, xa, xb, xc, fb, 1e-8, 100)
print("minimum:", num.trunc(x, 4), num.trunc(fx, 4))
print("trunc to 3 decimals:", num.trunc(num.pi, 3))
local avg = num.mavg(0.5)
print("mavg:", avg(10), avg(20), avg(30))
print("math included:", num.floor(num.pi), num.max(3, 7))
return num.round(num.pi * 100) / 100

santoku.fracidx: fractional order keys

Order keys that sort lexically: insert between any two items without renumbering the rest. between_n generates a balanced batch, and validate rejects malformed keys. This is how to order list items under concurrent edits. For a block of derived keys, gap(prev, next) returns a prefix that sorts after prev and before next (next may be nil), and prefix .. suffix(i, width) numbers the block in order, while suffix_desc numbers it in reverse. width is 1 to 8 digits and must fit i.

local fracidx = require("santoku.fracidx")
local arr = require("santoku.array")
local err = require("santoku.error")
local first = fracidx.between(nil, nil)
local second = fracidx.between(first, nil)
local middle = fracidx.between(first, second)
print("first:", first)
print("second:", second)
print("middle:", middle)
print("ordered:", first < middle and middle < second)
local batch = fracidx.between_n(first, second, 3)
print("batch of 3:", arr.concat(batch, " "))
print("bad key rejected:", err.pcall(fracidx.validate, "0oops"))
local prefix = fracidx.gap(first, middle)
local block = { prefix .. fracidx.suffix(0, 2), prefix .. fracidx.suffix(1, 2), prefix .. fracidx.suffix(2, 2) }
print("block:", arr.concat(block, " "))
print("inside:", first < block[1] and block[3] < middle)
print("desc:", fracidx.suffix_desc(0, 2), fracidx.suffix_desc(1, 2))
return fracidx.between(middle, second)

santoku.error: errors with structure

error and assert carry multiple values instead of one string; pcall returns them all, xpcall hands them to your handler, and wrapnil converts nil-plus-message APIs into throwing ones. checknil(v, ...) does the same for one call already made: it raises the rest of the values when v is nil and returns them all otherwise, as in err.checknil(ws:recv_frame()). Outside any pcall the values collapse to a readable colon-joined message.

local err = require("santoku.error")
local ok, tag, detail, code = err.pcall(function ()
  err.error("db", "connection refused", 111)
end)
print("caught:", ok, tag, detail, code)
local _, val = err.pcall(function ()
  return err.assert(tonumber("42"), "not a number")
end)
print("assert passes values through:", val)
local open = err.wrapnil(function (path)
  return nil, "no such file: " .. path
end)
print("wrapnil:", err.pcall(open, "missing.txt"))
local function div (a, b)
  if b == 0 then
    err.error("divide by zero", a)
  end
  return a / b
end
return err.xpcall(function ()
  return div(1, 0)
end, function (msg)
  return "handled: " .. msg
end)

santoku.op: operators as functions

Every Lua operator as a named function, ready to hand to reduce, sort, and mapped without writing a wrapper lambda each time.

local op = require("santoku.op")
local arr = require("santoku.array")
print("sum:", arr.reduce({ 1, 2, 3, 4 }, op.add))
print("product:", arr.reduce({ 1, 2, 3, 4 }, op.mul))
local words = arr.sort({ "pear", "fig", "apple" }, op.gt)
print("descending:", arr.concat(words, " "))
print("lengths:", arr.concat(arr.mapped({ "ab", "cdef" }, op.len), " "))
print("logic:", op["not"](nil), op.neg(5))
return op.cat("santoku", ".op")

santoku.validate: checks that return a reason

Every validator returns true, or false plus a reason and the offending values, which slots straight into error.assert: a failed check throws a structured error with the full context attached.

local validate = require("santoku.validate")
local err = require("santoku.error")
print(validate.isnumber(42))
print(validate.isnumber("42"))
print(validate.between(5, 1, 10))
print(validate.isarray({ 1, 2, 3 }))
print(validate.isarray({ 1, 2, x = 3 }))
local function set_port (cfg, port)
  err.assert(validate.istable(cfg))
  err.assert(validate.between(port, 1, 65535))
  cfg.port = port
  return cfg
end
local ok, msg, bad = err.pcall(set_port, {}, 99999)
print(ok, msg, bad)
return msg

santoku.utc: time without timezones

Epoch seconds in and out (optionally sub-second), broken-down UTC dates, strftime formatting, calendar-aware shift and trunc, and a stopwatch closure for quick timings. When local time is needed, local_time(fields) reads broken-down local time as epoch seconds and local_offset(t) gives the local UTC offset in seconds. days_from_civil, civil_from_days and weekday convert between dates and day numbers.

local utc = require("santoku.utc")
local now = utc.time()
print("epoch:", now)
print("precise:", utc.time(true))
local d = utc.date(now)
print("today:", d.year, d.month, d.day)
print("iso:", utc.format(now, "%Y-%m-%dT%H:%M:%SZ"))
print("tomorrow:", utc.format(utc.shift(now, 1, "day"), "%Y-%m-%d"))
print("midnight:", utc.format(utc.trunc(now, "day"), "%H:%M:%S"))
local lap = utc.stopwatch()
local duration, total = lap()
print("elapsed:", duration, total)

santoku.random

Seeding, random strings and alphanumerics, a clamped normal variate, and a fast seedable PRNG from C (fast_seed, fast_random, fast_max) for when math.random is the bottleneck. Requiring the module seeds both generators from process entropy, so distinct processes produce distinct streams without any setup; call seed or fast_seed with an explicit value only when you want a reproducible run. alnum draws from the 62 ASCII alphanumerics, while str draws from a raw byte range and will include punctuation. The fast generator is one state per Lua state, shared by every C extension, so fast_seed also fixes C draws such as an unseeded vec:shuffle. A C extension that includes <santoku/lua/utils.h> calls tk_fast_bind(L) in each entry point before tk_fast_random, tk_fast_normal or tk_fast_seed; a draw before any bind aborts. seed(n) fixes math.random draws such as arr.shuffle, and vec:shuffle(seed) seeds one call alone. fast_uniform() draws from [0, 1) and fast_normal(mean, variance) a normal variate, both from the fast generator. hash_unit(seed, s) maps a seed and a string to a fixed number in (0, 1), the same on every run. bytes(n) returns n bytes from the operating system's secure source (getrandom, arc4random or crypto.getRandomValues under wasm) and raises "negative length" below zero; use it for keys, salts and tokens.

local random = require("santoku.random")
local str = require("santoku.string")
random.seed()
print("die roll:", random.num(1, 6))
print("string:", random.str(8))
print("alnum:", random.alnum(12))
print("normal-ish:", random.norm())
random.fast_seed(42)
print("fast:", random.fast_random())
print("fast max:", random.fast_max)
print("uniform:", random.fast_uniform())
print("normal:", random.fast_normal(0, 1))
print("hash_unit:", random.hash_unit(7, "user-42") == random.hash_unit(7, "user-42"))
print("token:", str.to_hex(random.bytes(16)))

santoku.async: callbacks without deep nesting

Continuation-passing helpers that flatten callback code: pipe chains steps and short-circuits on failure, events is a tiny synchronous emitter, and each/map/all walk collections through async functions.

local async = require("santoku.async")
async.pipe(
  function (done)
    done(true, 2)
  end,
  function (done, x)
    done(true, x * 10)
  end,
  function (ok, result)
    print("pipe:", ok, result)
  end)
local events = async.events()
events.on("save", function (id)
  print("saved:", id)
end)
events.emit("save", 42)
async.each({ "a", "b" }, function (done, v, i)
  print("item:", i, v)
  done(true)
end, function (ok)
  print("all done:", ok)
end)

santoku.serialize: tables to Lua source

Calling the module serializes any value, tables included, to Lua source that round-trips through loadstring. Requiring santoku.autoserialize patches the global print so every printed table shows its contents instead of an address; toku lua --serialize loads it for you. This page runs each example with its own print, so the patch is shown for reading only.

local serialize = require("santoku.serialize")
local src = serialize({
  name = "example",
  tags = { "lua", "pwa" },
  nested = { deep = true },
})
print(src)
local restored = loadstring("return " .. src)()
print("round trip:", restored.name, restored.tags[2])
return restored.name

Smaller modules: co, inherit, geo, env

santoku.co builds tagged coroutine sets that nest without stealing each other's yields, inherit manages __index chains, geo does planar and great-circle math, and env reads variables. env.var(name, default) returns the default when the variable is unset, and an explicit nil default returns nil. With no default it raises Missing environment variable: name. Under err.pcall it returns false, Missing environment variable and the name. env.with_paths(path, cpath, fn, ...) runs fn with package.path and package.cpath swapped, then restores both even when fn raises; env.path() and env.cpath() read the current values. bench, test, and tracer cover timing and test scripts.

local inherit = require("santoku.inherit")
local base = { greet = function () return "hello" end }
local child = inherit.pushindex({}, base)
print("inherited:", child.greet())
local geo = require("santoku.geo")
local tampa = { lat = 27.95, lon = -82.46 }
local albany = { lat = 42.65, lon = -73.75 }
print("km apart:", geo.earth_distance(tampa, albany))
print("angle:", geo.angle({ x = 0, y = 0 }, { x = 1, y = 1 }))
local env = require("santoku.env")
print("editor:", env.var("EDITOR", "vi"))
print("optional:", env.var("SANTOKU_UNSET_EXAMPLE", nil))
local err = require("santoku.error")
print("required:", err.pcall(env.var, "SANTOKU_UNSET_EXAMPLE"))
local co = require("santoku.co")()
local gen = co.wrap(function ()
  co.yield(1)
  co.yield(2)
end)
print("yields:", gen(), gen())