A 16-year-old bug in SQLite broke Tailscale for months. Here is how they found it.

Tailscale spent the back half of 2025 watching its own network get flaky. The VPN provider's control plane, the routing layer that keeps customer "tailnets" (Tailscale's name for its private mesh networks) talking to each other, became unstable for months. On Wednesday the company published a post-mortem that lands the blame not on its own stack, and not on a recent regression, but on a defect in SQLite, the embedded database that ships inside much of the software industry, that had been sitting latent in the codebase for roughly 16 years.

SQLite is the embedded database that ships inside much of the software industry: small, single-file, serverless, and present in nearly every phone, browser, and desktop on the planet. Tailscale picked it in 2022 as "boring technology": well-known, widely used, and unlikely to surprise. The control plane was sharded, with each customer's tailnet living on one coordination-server shard at a time, and each shard backed by its own dedicated SQLite database. Each database had exactly one Go process writing to it, matching SQLite's intended single-writer model. From early 2023, every few minutes a full snapshot of each shard's SQLite file was uploaded to an S3 bucket. The setup ran without incident for more than two years.

Then, in August 2024, a data pipeline reading the S3 backups flagged an error. A PRAGMA integrity_check against the downloaded file confirmed the backup was corrupted, and the same shape of corruption kept showing up on shards across the fleet. Tailscale's post-mortem describes the months of work as a forensic investigation rather than an outage response, because the live control plane was still running; the backups, not the databases themselves, kept tripping the alarm.

The defect lived in SQLite's write-ahead log, the WAL, the journal SQLite uses to record changes before they are committed to the main database file. A race condition in the WAL-reset code path could leave a snapshot file in a state that looked valid to the writer but failed integrity checks once the writer had moved on. The condition was only reachable under Tailscale's specific pattern: one writer per file, an external process copying the file while writes were in flight, and a WAL reset happening at exactly the wrong moment. None of the pieces were exotic, and the combination was not what the SQLite developers had primarily designed or tested for.

Tailscale's response was to write a custom SQLite VFS shim, a thin layer that sits between the application and the file system, to reproduce the race deterministically and isolate the failing sequence. With the bug in hand, the company engaged the upstream SQLite developers under a professional support contract and funded the fix. The post does not give a SQLite release number, but the company says the latent defect has now been patched upstream and that the operational pattern that triggered it is documented for future operators.

The Hacker News thread reads the post as both a debugging war story and a case study in paying open-source maintainers for targeted work on infrastructure the rest of the industry depends on. Several commenters point to the VFS shim approach as a model for reproducing a race no test had caught, and to the paid-support angle as a template for unblocking an upstream fix when the maintainers are stretched thin.

For anyone running embedded SQLite at scale with file-based backups, the post-mortem is a reminder that the WAL reset path is now somewhere on the list of things to watch. The broader point is that "boring" technology choices still warrant deep operational skepticism when the deployment pattern strays from the textbook, and that a single backup-corruption alert is worth treating as a multi-month investigation rather than a one-off. The fix is upstream; the operational lesson is local.