santoku

santoku

A Lua framework for building applications end to end

Lua 5.1 from the build system to the server to the browser. Every example on this site runs in your browser. Edit one and run it.

santoku targets Lua 5.1, because 5.1 is the version its platforms actually share. LuaJIT implements 5.1 and OpenResty embeds LuaJIT, so server code, client code and the build harness all run the same language. Compiled to WebAssembly, the 5.1 core is smaller to embed, and its single number type sidesteps the integer and float split introduced in 5.3, which keeps numeric values crossing the JavaScript boundary simple. Support for newer versions is meant to come from compatibility shims layered over the 5.1 core rather than version branches in the tree; contributions and design ideas for that layer are welcome.

Running the interactive examples needs JavaScript and WebAssembly; the documentation itself is fully readable without them.

santoku-cdp

View on LuaRocks

Chrome DevTools Protocol client: attach to a page, navigate, evaluate JavaScript, wait for elements and take screenshots, plus a toku-browser command

santoku-cdp drives a Chromium browser from Lua over the Chrome DevTools Protocol: attach to a page, navigate, evaluate JavaScript, wait for an element and take screenshots. It speaks the protocol through santoku-socket's WebSocket client, so it runs in CLIs and scripts with native sockets rather than in this page, and the examples are shown for reading. The toku-browser command exposes the same operations from a shell, and toku browser reaches it.

A browser with a DevTools port

santoku-cdp attaches to a browser started with --remote-debugging-port. targets(opts) and version(opts) read the browser's /json/list and /json/version; opts.host defaults to 127.0.0.1 and opts.port to 9222. A failed request raises "cdp: GET failed" with the path and status.

$ chromium --headless=new --remote-debugging-port=9222 &
$ toku browser targets
3F2A...	page	about:blank	about:blank

Illustrative: runs outside the browser.

santoku.cdp.page: open, navigate, eval

page.open(opts) attaches to the first page target, or the one opts.id names, enables page events, and returns the same session cdp.page returns, so call, on and wait from the session card below work on it. type(s, selector, text) focuses the element and inserts the text, and raises "cdp: no focusable element matches" when it can't focus one. wait_until(s, expr[, ms]) evaluates expr every 200 ms and returns its first value that is truthy in JavaScript, so false, null, "" and 0 keep it waiting; at the deadline, 10000 ms by default, it raises "cdp: condition not met". navigate(s, url[, ms]) waits for the load event, 30000 ms by default, and raises "cdp: navigate failed" with the URL and Chrome's errorText when the load fails. eval(s, expr[, ms]) returns the expression's value and awaits it first when it is a promise; a thrown exception raises "cdp: eval failed" with its description.

local page = require("santoku.cdp.page")
local s = page.open({ port = 9222 })
page.navigate(s, "https://santoku.dev/")
print(page.eval(s, "document.title"))
print(page.eval(s, "fetch('/llms.txt').then(function (r) { return r.status })"))
page.type(s, "input", "lpeg")
print(page.wait_until(s, "document.querySelector('h1') && document.querySelector('h1').textContent", 5000))
s.close()

Illustrative: runs outside the browser.

Screenshots and waiting for an element

screenshot(s[, opts]) returns PNG bytes of the viewport. With opts.full it sizes the capture to the page's content, so the whole page comes back in one image; opts.ms bounds the capture, 30000 by default. wait_for(s, selector[, ms]) checks for a CSS match every 200 ms while it keeps reading events, and raises "cdp: no element matches" with the selector after ms, 10000 by default.

local fs = require("santoku.fs")
local page = require("santoku.cdp.page")
local s = page.open({ port = 9222 })
page.navigate(s, "https://santoku.dev/")
page.wait_for(s, "nav a", 5000)
fs.writefile("site.png", page.screenshot(s, { full = true }))
s.close()

Illustrative: runs outside the browser.

Passkeys and console capture

santoku.cdp.webauthn gives a page a virtual authenticator, so a test can sign in with a passkey. authenticator(s[, opts]) adds one (CTAP2, internal, resident keys, user verification, PRF) and returns its id. credentials(s, id) returns what WebAuthn.getCredentials reports, and load(s, json[, opts]) adds a fresh authenticator and those credentials back in a later run, taking { credentials = [...] } or a bare list, and returns the id and the count. A reloaded passkey signs in, but its PRF output comes back null, so it can't stand in for a PRF-based unlock. santoku.cdp.auth.basic(s, url_pattern, user, pass) answers HTTP basic-auth challenges for URLs matching url_pattern, only from the request's own origin, declines proxy challenges, and returns a function that counts the challenges it answered. Event handlers run inside the socket read, so they send with s.send(method, params), which drops the reply, instead of the blocking s.call. santoku.cdp.log.watch(s) discards console entries buffered before it ran, enables Runtime and Network, and returns a drain function; each call returns the lines collected since the last one: console calls, exceptions, failed requests, HTTP statuses of 400 and up, and WebAuthn credential events. toku-browser takes --passkey-env VAR to load credentials JSON from an environment variable before any command, --logs to print those lines on stderr, and --basic-env USER_VAR,PASS_VAR with --basic-pattern URLPATTERN for basic auth.

local cjson = require("cjson")
local page = require("santoku.cdp.page")
local webauthn = require("santoku.cdp.webauthn")
local log = require("santoku.cdp.log")
local s = page.open({ port = 9222 })
local drain = log.watch(s)
local id = webauthn.authenticator(s)
page.navigate(s, "http://localhost:8080/")
page.wait_until(s, "document.body.dataset.signedIn", 10000)
local saved = cjson.encode(webauthn.credentials(s, id))
for _, line in ipairs(drain()) do
  print(line)
end
s.close()

Illustrative: runs outside the browser.

The session: protocol calls and events

cdp.page(opts) returns a session on the page's WebSocket, and cdp.connect(url, opts) opens one on any webSocketDebuggerUrl, such as the browser endpoint that version() reports; opts.connect_timeout_ms and opts.max_message pass through to the WebSocket client. call(method[, params[, ms]]) sends one command and returns its result, waiting 10000 ms by default. It raises "cdp: no reply" with the method and the transport's reason, or "cdp: call failed" with Chrome's message. on(method, fn) registers an event handler, called with the event's params and sessionId. wait(fn, ms) reads the socket and dispatches events until fn returns true or ms passes. cdp.session(conn) builds the same session over any object with send, wait and close, which is how its specs run without a browser.

local cdp = require("santoku.cdp")
local s = cdp.page({ port = 9222 })
s.on("Network.requestWillBeSent", function (params)
  print(params.request.url)
end)
s.call("Network.enable")
s.call("Page.navigate", { url = "https://santoku.dev/" })
s.wait(function () return false end, 2000)
s.close()

Illustrative: runs outside the browser.

A tab of your own on a shared browser

page.open attaches to a tab that already exists. cdp.new_page(opts) opens a new about:blank tab through the DevTools /json/new endpoint and returns a session on it. page.enable(s) does what page.open adds to a session: it sends Page.enable and counts load events, which navigate needs, and s.loads() returns the count. Close the tab with the raw Page.close method, then close the session. A reload is Page.reload plus a wait for the count to rise.

local cdp = require("santoku.cdp")
local page = require("santoku.cdp.page")
local s = page.enable(cdp.new_page({ port = 9222 }))
page.navigate(s, "https://santoku.dev/")
local n = s.loads()
s.call("Page.reload", { ignoreCache = true })
s.wait(function () return s.loads() > n end, 30000)
print(page.eval(s, "document.title"))
s.call("Page.close")
s.close()

Illustrative: runs outside the browser.

toku-browser from a shell

The rock installs a toku-browser command. --host, --port and --id pick the browser and the page. shot and eval load the URL first; pass - to act on the current page instead. eval prints the value as JSON and waits at most 10000 ms for it, with no --ms option, so a longer promise needs page.eval(s, expr, ms) from Lua. The rest act on the current page: wait for a selector, until for a JavaScript condition, type into a selector, click a selector or an x,y point (--right, --double), key, text, drag between two points as touch, and call for one raw DevTools method. toku browser works through toku's command dispatch: santoku-cli 3.3.0 and later run toku-<name> from PATH for any name that is not a built-in command.

$ toku browser shot https://santoku.dev/ site.png --full
site.png 1136328 bytes
$ toku browser eval https://santoku.dev/ 'document.links.length'
54
$ toku browser wait 'nav a' --ms 5000
$ toku browser type '#q' komodo
$ toku browser until "document.querySelector('td') && document.querySelector('td').textContent"
"Komodo Island"

Illustrative: runs outside the browser.

toku-browser run: one wasm spec in a fresh Chromium

run starts its own headless browser and ignores --host and --port. It runs $TOKU_CHROMIUM with a temporary profile and --remote-debugging-port=0, reads the port Chromium picked, and serves the spec through DevTools request interception at http://localhost/, so no server or fixed port is involved. The page relays console output to stdout and errors to stderr. The spec is done when emscripten has set its exit status and no timer is pending; an uncaught error or rejection fails it, and TOKU_BROWSER_TIMEOUT_MS (60000 by default) bounds it. The command exits with the spec's status, then stops only the Chromium it started and deletes the profile. TOKU_CHROMIUM_FLAGS adds browser flags. toku test --wasm-browser uses it as the spec interpreter; santoku.cdp.run.run(spec, opts) is the same thing from Lua.

$ TOKU_CHROMIUM=chromium-browser toku browser run build/default-wasm/test/test-browser/test/spec/my_lib.js
$ echo $?
0
$ TOKU_CHROMIUM=chromium TOKU_CHROMIUM_FLAGS=--no-sandbox toku browser run spec.js

Illustrative: runs outside the browser.