A signal of the future WebAssembly/emscripten is giving us: a database served on a static web page
Posted by jpluimers on 2025/04/17
From a quite a while back; apparently it ended up in the drafts in stead of the blog queue:
[Wayback/Archive] Ian Miell on Twitter: “A signal of the future WebAssembly/emscripten is giving us: a database served on a static web page: … The possibilities are dizzying. Also, I’ve thought for a while that if I had to put money on it, SQLite will outlast every other database out there.”
The combination of Emscripten and WebAssembly is cool as it allows you to run C/C++ based code in most Web Browsers at near-native speed (though the standard is open and can just as easily outside that realm).
[Wayback/Archive] Hosting SQLite databases on Github Pages – (or any static file hoster) – phiresky’s blog:
…
Note that this website is 100% hosted on a static file hoster (GitHub Pages).
So how do you use a database on a static file hoster? Firstly, SQLite (written in C) is compiled to WebAssembly. SQLite can be compiled with [Wayback/Archive] emscripten without any modifications, and the [Wayback/Archive] sql.js library is a thin JS wrapper around the wasm code.
sql.js only allows you to create and read from databases that are fully in memory though – so I implemented a virtual file system that fetches chunks of the database with HTTP Range requests when SQLite tries to read from the filesystem: [Wayback/Archive] sql.js-httpvfs. From SQLite’s perspective, it just looks like it’s living on a normal computer with an empty filesystem except for a file called
/wdi.sqlite3that it can read from. Of course it can’t write to this file, but a read-only database is still very useful.Since fetching data via HTTP has a pretty large overhead, we need to fetch data in chunks and find some balance between the number of requests and the used bandwidth. Thankfully, SQLite already organizes its database in “pages” with a user-defined [Wayback/Archive] page size (4 KiB by default). I’ve set the page size to 1 KiB for this database.
…
Of course, everything here is open source. The main implementation of the sqlite wrapper is in [Wayback/Archive] sql.js-httpvfs. The source code of this blog post is [Wayback/Archive] a pandoc markdown file, with the demos being a [Wayback/Archive] custom “fenced code block” React component.
Note, [Wayback/Archive] phiresky/sql.js-httpvfs:
This repo is a fork of and wrapper aroundsql.jsto provide a read-only HTTP-Range-request based virtual file system for SQLite. It allows hosting an SQLite database on a static file hoster and querying that database from the browser without fully downloading it.The virtual file system is an emscripten filesystem with some “smart” logic to accelerate fetching with virtual read heads that speed up when sequential data is fetched. It could also be useful to other applications, the code is in [Wayback/Archive] lazyFile.ts. It might also be useful to implement this lazy fetching as an [Wayback/Archive] SQLite VFS since then SQLite could be compiled with e.g. WASI SDK without relying on all the emscripten OS emulation.Note that this whole thing only works well if your database and indexes are structured well.sql.js-httpvfsalso provides a proof-of-concept level implementation of a DOM virtual table that allows interacting (read/write) with the browser DOM directly from within SQLite queries.
--jeroen
https://twitter.com/ianmiell/status/1389834618334060545
The Wiert Corner - irregular stream of stuff 
Leave a comment