tools

I still meet teams that get tripped up by scheduling math, even when the algorithm is as simple as first-come, first-served. The confusion usually shows up during incident reviews: a long batch job hogs the CPU, dashboards show latency spikes, and someone asks why a tiny task waited so long. FCFS is the baseline that […]

Last year I audited a Perl ingestion service that booted with a 60-item allowlist. The list was hand-quoted, and one missing quote pushed a whole tail of strings into a single element. The service started, but the allowlist was wrong and the bug only appeared in production. I replaced the literal list with qw(…) and

Every quarter I watch a team lose a week because someone assumed Windows and macOS are just different skins. The friction shows up in small places: a build script that calls sed -i, a deployment guide that assumes ~/Library, or an IT policy that locks down kernel extensions. As a senior developer I care less

At 2 a.m. last winter, I chased a CPU spike in a packet processor. Two threads on different cores kept updating a shared counter that sat on the hot path of the scheduler. The counter update itself was tiny, but the blocking mutex around it forced a sleep/wake cycle that cost far more than the

Years ago I broke a bootloader and stared at a black screen for hours. No logs, no stack trace, just a blinking cursor and the realization that every tool I trust sits on top of an operating system. That moment reshaped how I think about software. When you build even a tiny kernel, you get

Two years ago I was helping a payments team whose service looked fine in profiling, yet every peak caused timeouts. The CPU graphs were calm, but disks were busy and the latency histogram had long tails. The root cause was a classic paging problem: we were touching more pages than physical memory could hold, so

I once watched a fleet of retail kiosks go dark because a firmware update was pushed during a late-night power dip. The apps were healthy, the OS image was fine, yet the devices refused to boot. That incident taught me that firmware is not just ‘software on hardware‘. It is the thin, stubborn layer that

Last year I reviewed a Perl data pipeline that was quietly dropping lines. The bug wasn‘t in the regex or the file parsing; it was a special variable that got reused in a helper and overwrote the value a parent loop expected. That one character changed the meaning of five lines of code. If you

Two years ago I was on-call for a backend cluster that suddenly slowed to a crawl. Memory graphs looked stable, yet kernel logs showed a storm of page faults every few milliseconds. The culprit was tiny: a single page table entry had its present bit cleared during a failed hot-patch, and the CPU kept trapping

Two years ago I was on call for a storage service when a background job started reading archived invoices from the wrong tenant. The SQL query was correct and the job’s config looked fine. The real culprit was a permissions map that had drifted from reality after a reorg. We had role names in code,

Last year I reviewed a Rails service where a per-request counter was stored in a class variable. It looked harmless, but it leaked between tenants and every few hours someone’s invoice jumped by one. The fix was a single character: @@ to @. That tiny symbol is how Ruby communicates scope, and it’s still one

Last winter I reviewed a SaaS incident where a staging box leaked customer exports. The data was encrypted at rest, the app had SSO, and the perimeter looked solid. The leak came from a cron job that wrote CSVs into /tmp with default permissions. A diagnostics tool running under a different user had read access