santoku-zlib

gzip, zlib and raw deflate on zlib 1.3.2: gunzip, inflate_raw and gzip, with a max_out cap against decompression bombs

santoku-zlib binds zlib 1.3.2 into santoku.zlib, with three functions: gunzip decodes gzip or zlib-wrapped input, inflate_raw decodes raw deflate, and gzip compresses. santoku.zlib.png decodes PNG files and recompresses them losslessly. zlib is vendored and linked statically, so the rock needs no system zlib. Output grows in chunks with no fixed cap, and an optional max_out byte limit stops a decompression bomb. Every failure raises a structured error naming the function. santoku-socket's fetch decodes gzip and deflate responses through this module. This page's wasm bundle doesn't include it, so the examples are shown for reading.

gzip and gunzip

gzip(s[, level]) returns a gzip stream of s, starting with the magic bytes 1f 8b 08. level is 0 (stored) to 9 (smallest), or -1, the default, for zlib's default level. Any other level raises "level must be -1 or 0 to 9". gunzip(s[, max_out]) returns the decoded bytes.

local zlib = require("santoku.zlib")
local str = require("santoku.string")
local text = str.rep("a line of log output\n", 500)
local gz = zlib.gzip(text)
print("in:", #text, "gzip:", #gz)
print("magic:", str.byte(gz, 1, 3))
print("round trip:", zlib.gunzip(gz) == text)
print("level 0:", #zlib.gzip(text, 0), "level 9:", #zlib.gzip(text, 9))
return #gz

Illustrative: runs outside the browser.

Raw deflate, zlib-wrapped input and gzip members

gunzip reads the header and accepts gzip or zlib-wrapped input. A gzip input made of several concatenated members decodes to all of them joined, which is what gzip -c a b produces. inflate_raw(s[, max_out]) decodes deflate with no header or trailer, the form a zip entry or a raw deflate HTTP body carries. Raw deflate and gzip aren't interchangeable: each function raises on the other's input.

local zlib = require("santoku.zlib")
local str = require("santoku.string")
local gz = zlib.gzip("hello, deflate")
local raw = str.sub(gz, 11, #gz - 8)
print("raw:", zlib.inflate_raw(raw))
local wrapped = "\120\156\203\72\205\201\201\215\81\168\202\201\76\2\0\24\178\4\18"
print("zlib-wrapped:", zlib.gunzip(wrapped))
print("members:", zlib.gunzip(zlib.gzip("first ") .. zlib.gzip("") .. zlib.gzip("second")))
return zlib.inflate_raw(raw)

Illustrative: runs outside the browser.

max_out and errors

max_out caps the decoded size in bytes: output of exactly max_out passes, and one byte more raises "output exceeds max_out" without decoding the rest. max_out must be a non-negative integer. Every error is err.error(fn, msg), so err.pcall returns false, the function name and the message, and an uncaught error reads "gunzip: truncated input". Short input, including the empty string, is "truncated input". Bytes after the last gzip member, or after a zlib-wrapped or raw stream, are "trailing data after the compressed stream". Corrupt input raises zlib's own message, such as "incorrect header check" or "incorrect data check". The module stays usable after any error.

local zlib = require("santoku.zlib")
local err = require("santoku.error")
local str = require("santoku.string")
local bomb = zlib.gzip(str.rep("\0", 16 * 1024 * 1024), 9)
print("bomb bytes:", #bomb)
print(err.pcall(zlib.gunzip, bomb, 1024 * 1024))
local gz = zlib.gzip("abc")
print("exact cap:", zlib.gunzip(gz, 3))
print(err.pcall(zlib.gunzip, str.sub(gz, 1, #gz - 4)))
print(err.pcall(zlib.gunzip, gz .. "\0\0"))
print(err.pcall(zlib.gunzip, "not compressed at all"))
print(err.pcall(zlib.inflate_raw, gz))
print(err.pcall(zlib.gunzip, gz, 1.5))
print(err.pcall(zlib.gzip, "x", 10))
return #bomb

Illustrative: runs outside the browser.

santoku.zlib.png: decode and optimize

png.decode(data) returns width, height, channel depth (8 or 16) and the pixels as RGBA, 4 bytes a pixel at depth 8 and 8 at depth 16. png.optimize(data) returns the smallest lossless re-encoding it finds and a note naming the color type, bit depth, filter and deflate strategy it chose. It reduces color type and bit depth where the pixels allow, writes a palette when the colors fit, and drops interlacing. It drops tEXt, zTXt, iTXt, tIME and bKGD chunks and keeps gAMA, cHRM, sRGB, iCCP and pHYs, so an image with an ICC profile stays in color. Output that wouldn't be smaller comes back unchanged with "not smaller", and a file with any other chunk comes back unchanged with "unhandled chunk". Corrupt input raises an error naming the function, like "png.decode: not a PNG: bad signature". santoku-make runs optimize on every PNG a web build ships.

local png = require("santoku.zlib.png")
local fs = require("santoku.fs")
local err = require("santoku.error")
local data = fs.readfile("icon.png")
local w, h, depth, px = png.decode(data)
print(w, h, depth, #px)
local out, how = png.optimize(data)
print(#data, #out, how)
local _, _, _, again = png.decode(out)
print("same pixels:", again == px)
local twice, why = png.optimize(out)
print(twice == out, why)
print(err.pcall(png.decode, "GIF89a"))
return #out

Illustrative: runs outside the browser.