System administration topics — Linux, OpenLiteSpeed, MySQL, monitoring.
Last spring I was cleaning out an old VPS and found three crontab entries pointing at scripts that didn’t exist anymore. Two had been deleted during a refactor in 2022; one was a project I’d rm -rf‘d a year …
You’ve installed a content blocker on every browser. The phone in your pocket has a half-dozen apps that show ads anyway. The smart TV in the living room shows ads on the home screen with no user-facing way to turn …
The first time you write a backup script, it’s six lines: restic backup /home. The first time it silently stops running, you discover that the only signal something’s wrong is “I went to restore something and the most recent …
The 3-2-1 backup rule is older than this whole industry: 3 copies of the data, on 2 different media types, with 1 of them offsite. It’s the conservative answer for a reason — ransomware can take out one local …
If you run more than one WordPress site on a single server and every wp-config.php has DB_USER = 'root', your eight sites are effectively one site as far as a compromise is concerned. One vulnerable plugin on any of …
Open /var/log/auth.log on a public-facing server and you’ll see thousands of lines per day — failed logins, accepted logins, sudo events, cron registrations. The signal you usually care about (who’s brute-forcing me, from where, against which users?) is buried in …
Pinboard is the bookmark service hardcore link-collectors swore by for over a decade. The thing was: it was always one person (Maciej Cegłowski), aging gracefully but increasingly out of touch with modern web platforms. Pinboard’s iOS extensions broke after iOS …
You have a backup script. You add it to /etc/crontab. It runs on schedule for a year. Then one day it stops, because the box was suspended for ten minutes during the run window and cron’s “minute X of …
A 4 TB ext4 partition I’d been using as bulk storage on a NAS started returning EIO on a handful of files last month. dmesg was full of EXT4-fs error messages about inode table corruption. The drive itself was healthy …
I had a single mysqldump file go partially-corrupted on S3 last year — exactly one byte flipped somewhere in the middle of a 1.2 GB nightly backup. The kind of thing that should never happen, but did, probably during transit …