santoku

santoku

A Lua framework for building applications end to end

Lua 5.1 from the build system to the server to the browser. Every example on this site runs in your browser. Edit one and run it.

santoku targets Lua 5.1, because 5.1 is the version its platforms actually share. LuaJIT implements 5.1 and OpenResty embeds LuaJIT, so server code, client code and the build harness all run the same language. Compiled to WebAssembly, the 5.1 core is smaller to embed, and its single number type sidesteps the integer and float split introduced in 5.3, which keeps numeric values crossing the JavaScript boundary simple. Support for newer versions is meant to come from compatibility shims layered over the 5.1 core rather than version branches in the tree; contributions and design ideas for that layer are welcome.

Running the interactive examples needs JavaScript and WebAssembly; the documentation itself is fully readable without them.

Editor support

syntax highlighting for .tk templates in VS Code, JetBrains IDEs, Visual Studio, Sublime Text, Helix, Zed and Emacs, downloaded from releases.santoku.dev with no marketplace: per-editor archives, the tree-sitter grammar tarball, checksums, and install steps

toku-syntax highlights santoku .tk templates in VS Code, JetBrains IDEs, Visual Studio, Sublime Text, Helix, Zed and Emacs, installed from release archives with no marketplace or extension store. A template is host text with <% ... %> blocks of Lua. The host language comes from the file name with .tk removed: page.tk.html is HTML, db.tk.wasm.lua is Lua, klib.h.tk is C, Makefile.tk is a Makefile and a bare name.tk is plain text. Lua highlighting covers each block and the host language covers the rest. Release 0.1.0 is at https://releases.santoku.dev/toku-syntax/0.1.0/ with one archive per editor and a SHA256SUMS file. Helix, Zed and Emacs also need the tree-sitter grammar, tree-sitter-toku-0.1.0.tar.gz from the same directory.

Hosts and file names

Every editor matches *.tk.<ext> and *.<ext>.tk for the hosts below. VS Code, Helix and Emacs also match *.tk.<anything>.<ext>, such as db.tk.wasm.lua. Sublime Text and Zed match the *.tk.wasm.<ext> form only. TOML and Nginx hosts in VS Code and Sublime Text need an extension or package that provides the source.toml or source.nginx grammar.

HTML        *.tk.html  *.tk.htm
CSS         *.tk.css
JavaScript  *.tk.js  *.tk.mjs  *.tk.cjs
JSON        *.tk.json
Lua         *.tk.lua
Shell       *.tk.sh  *.tk.bash
SQL         *.tk.sql
Markdown    *.tk.md  *.tk.markdown
YAML        *.tk.yaml  *.tk.yml
TOML        *.tk.toml
Nginx       *.tk.conf  *.tk.nginx
Makefile    Makefile.tk  makefile.tk  GNUmakefile.tk  *.tk.mk
XML         *.tk.xml  *.tk.svg
C           *.tk.c  *.tk.h
Text        *.tk  *.tk.txt

Illustrative: runs outside the browser.

Download and verify

Download SHA256SUMS and the archives you need, then check them. sha256sum --ignore-missing checks only the files you downloaded. On macOS, use shasum -a 256 -c in place of sha256sum -c.

$ base=https://releases.santoku.dev/toku-syntax/0.1.0
$ curl -fSLO "$base/SHA256SUMS"
$ curl -fSLO "$base/toku-syntax-0.1.0.vsix"
$ curl -fSLO "$base/tree-sitter-toku-0.1.0.tar.gz"
$ sha256sum -c --ignore-missing SHA256SUMS
toku-syntax-0.1.0.vsix: OK
tree-sitter-toku-0.1.0.tar.gz: OK

Illustrative: runs outside the browser.

VS Code

Install toku-syntax-0.1.0.vsix from the command line, or with Extensions: Install from VSIX in the command palette. Each host gets its own language id, such as toku-html, so comment toggling uses the host's comment syntax. For TOML and Nginx hosts, also install Even Better TOML and NGINX Configuration.

$ curl -fSLO https://releases.santoku.dev/toku-syntax/0.1.0/toku-syntax-0.1.0.vsix
$ code --install-extension toku-syntax-0.1.0.vsix

Illustrative: runs outside the browser.

JetBrains IDEs

toku-syntax-textmate-0.1.0.zip holds the same TextMate bundle as the VSIX, in the VS Code extension layout JetBrains IDEs import. Unpack it, open Settings, Editor, TextMate Bundles, click Add and select the toku-syntax-textmate-0.1.0 directory.

$ curl -fSLO https://releases.santoku.dev/toku-syntax/0.1.0/toku-syntax-textmate-0.1.0.zip
$ unzip toku-syntax-textmate-0.1.0.zip -d ~/.local/share
# Settings > Editor > TextMate Bundles > Add > ~/.local/share/toku-syntax-textmate-0.1.0

Illustrative: runs outside the browser.

Visual Studio

Visual Studio loads TextMate grammars from %userprofile%\.vs\Extensions\<language>\Syntaxes. toku-syntax-visual-studio-0.1.0.zip unpacks to Toku\Syntaxes, so extract it into %userprofile%\.vs\Extensions, creating that directory if it is missing. Close and reopen any open .tk files afterwards.

PS> Invoke-WebRequest https://releases.santoku.dev/toku-syntax/0.1.0/toku-syntax-visual-studio-0.1.0.zip -OutFile toku-syntax-visual-studio-0.1.0.zip
PS> Expand-Archive toku-syntax-visual-studio-0.1.0.zip -DestinationPath "$env:USERPROFILE\.vs\Extensions"

Illustrative: runs outside the browser.

Sublime Text

toku-syntax-sublime-0.1.0.zip unpacks to a Toku package directory. Extract it into the directory Preferences, Browse Packages opens. Each syntax pushes the host's own syntax and embeds Lua in every block, ending the block at the first %>. The Nginx host needs the Nginx package.

$ curl -fSLO https://releases.santoku.dev/toku-syntax/0.1.0/toku-syntax-sublime-0.1.0.zip
$ unzip toku-syntax-sublime-0.1.0.zip -d ~/.config/sublime-text/Packages

Illustrative: runs outside the browser.

The tree-sitter grammar for Helix, Zed and Emacs

Helix, Zed and Emacs build the toku tree-sitter grammar from source. Unpack tree-sitter-toku-0.1.0.tar.gz to ~/.local/share/tree-sitter-toku, the path the Helix, Zed and Emacs files expect. The archive holds the generated src/parser.c, so building needs a C compiler and no tree-sitter CLI.

$ curl -fSLO https://releases.santoku.dev/toku-syntax/0.1.0/tree-sitter-toku-0.1.0.tar.gz
$ mkdir -p ~/.local/share/tree-sitter-toku
$ tar xzf tree-sitter-toku-0.1.0.tar.gz --strip-components=1 -C ~/.local/share/tree-sitter-toku

Illustrative: runs outside the browser.

Helix

languages.toml adds one language per host, such as toku-html, all on the toku grammar, whose source is a local path. Helix needs that path absolute, so the file ships with /home/you and the sed below writes your home directory in its place. hx --grammar fetch skips local grammars and fetches the host grammars, and hx --grammar build compiles them all. hx --health toku-html reports the parser and queries.

$ curl -fSLO https://releases.santoku.dev/toku-syntax/0.1.0/toku-syntax-helix-0.1.0.zip
$ unzip toku-syntax-helix-0.1.0.zip
$ mkdir -p ~/.config/helix/runtime/queries
$ sed "s|/home/you|$HOME|" toku-syntax-helix-0.1.0/languages.toml >> ~/.config/helix/languages.toml
$ cp -r toku-syntax-helix-0.1.0/runtime/queries/* ~/.config/helix/runtime/queries/
$ hx --grammar fetch
$ hx --grammar build
$ hx --health toku-html

Illustrative: runs outside the browser.

Zed

Zed fetches an extension's grammar with git: it runs git fetch on the repository URL in extension.toml and checks out its rev, and it accepts a file:// URL for a local repository. extension.toml names file:///home/you/.local/share/tree-sitter-toku and rev toku, so turn the unpacked grammar into a git repository with a toku branch, then write your home directory into extension.toml. Install the extension with zed: install dev extension from the command palette, selecting the toku-syntax-zed-0.1.0 directory. Zed compiles the grammar to WebAssembly itself. Install the Zed extensions for the hosts it doesn't ship: HTML, Lua, SQL, TOML, XML, Make and Nginx.

$ cd ~/.local/share/tree-sitter-toku
$ git init
$ git checkout -b toku
$ git add -A
$ git commit -m "tree-sitter-toku 0.1.0"
$ cd ~
$ curl -fSLO https://releases.santoku.dev/toku-syntax/0.1.0/toku-syntax-zed-0.1.0.zip
$ unzip toku-syntax-zed-0.1.0.zip -d ~/.local/share
$ cd ~/.local/share/toku-syntax-zed-0.1.0
$ sed "s|/home/you|$HOME|" extension.toml > extension.toml.new && mv extension.toml.new extension.toml
# Zed: command palette > zed: install dev extension > ~/.local/share/toku-syntax-zed-0.1.0

Illustrative: runs outside the browser.

Emacs

toku-ts-mode.el needs Emacs 31 or newer. It defines one mode per host, such as toku-html-ts-mode, binds them in auto-mode-alist, and registers the toku grammar recipe at toku-ts-mode-grammar-dir, which defaults to ~/.local/share/tree-sitter-toku; set it before the require to use another path. treesit-install-language-grammar then builds the grammar from that directory. Install the lua grammar and each host grammar you use the same way: html, css, javascript, json, bash, yaml, toml, c. SQL, Markdown, Nginx, Makefile and XML hosts have no Emacs tree-sitter mode, so those files get Lua highlighting only.

$ curl -fSLO https://releases.santoku.dev/toku-syntax/0.1.0/toku-syntax-emacs-0.1.0.zip
$ unzip toku-syntax-emacs-0.1.0.zip -d ~/.emacs.d

;; init.el
(add-to-list 'load-path (expand-file-name "toku-syntax-emacs-0.1.0" user-emacs-directory))
(require 'toku-ts-mode)

M-x treesit-install-language-grammar RET toku RET
M-x treesit-install-language-grammar RET lua RET

Illustrative: runs outside the browser.

Known gaps

In VS Code, JetBrains IDEs and Visual Studio, a %> inside a Lua string inside a block ends the highlighted block there, while toku ends the block at the first %> anywhere, and a block inside a Markdown inline link URL isn't highlighted. Emacs needs version 31 or newer. In Emacs, a .tk file whose first line is a #! shebang opens in the interpreter's mode, because Emacs checks interpreter-mode-alist before auto-mode-alist. The JetBrains and Visual Studio files have not been run in those IDEs, and the Zed extension has not been built in Zed.

<% return "a%>b" %>    VS Code, JetBrains, Visual Studio: the block ends inside the string
[link](<% return u %>) VS Code, JetBrains, Visual Studio: the Markdown link wins
#!/bin/sh in x.tk.sh   Emacs: opens in sh-mode

Illustrative: runs outside the browser.