“Towards Inserting One Billion Rows in SQLite Under A Minute”
Posted by jpluimers on 2025/04/22
From a long time ago, but the (impressive!) leaderboard still has the same order (the Rust implementation did get faster!).
[Wayback/Archive] Towards Inserting One Billion Rows in SQLite Under A Minute – blag
Recently, I ran into a situation where I needed a test database with lots of rows and needed it fast. So I did what any programmer would do: wrote a Python script to generate the DB. Unfortunately, it was slow. Really slow. So I did what any programmer would do: went down the rabbit hole of learning more about SQLite, Python, and eventually Rust… in my quest to get a 1B row database under a minute. This blog post is a summary of this fun and educational exercise.
Repository at [Wayback/Archive] avinassh/fast-sqlite3-inserts: Some bunch of test scripts to generate a SQLite DB with 1B rows in fastest possible way
Leaderboard
(for 100M insertions)
Variant Time Rust 23 seconds PyPy 126 seconds CPython 210 seconds
Via: [Wayback/Archive] Shawn Wildermuth #000000 { lives: matter; } 🇺🇦 on Twitter: “Towards Inserting One Billion Rows in SQLite Under A Minute””
--jeroen
https://twitter.com/ShawnWildermuth/status/1420475653741236225
Leave a Comment