Deep dives into tech, design, and AI exploration
This content pipeline publishes one article every day at 20:00 (SGT). The slug prefix is article-YYYYMMDD-. Before publishing, it checks the CMS: if a row with
During last month’s reconciliation, we noticed that the daily cost of our batch export pipeline had risen from $0.30 to nearly $1.00. No one received any alerts
At 3 AM, after an automated job had been running all night, the parsing success rate on the dashboard dropped from 99% to 71%. My first instinct was that someth
Last week, one of our batch pipelines stalled. It started running at 3 AM. When I checked the dashboard at 11 AM, the progress was stuck at 61%. The task status
Last month, the on-call bot I built for our inference service started gaining traction: when customers reported issues, it would pull relevant lines from produc
At the beginning of this month, our autonomous delivery tasks ran continuously for three nights straight. Checking the bill in the morning, we found that the av
Last month, we ran a batch job on our local inference server: classifying 24,000 anonymized customer service messages, with each message processed by a local mo
Last week, we delivered an internal Q&A system for a client. The most embarrassing part wasn’t the model giving wrong answers, but the circuit breaker taking do
Last week, we did something quite routine: we swapped the main model behind our local inference gateway from a 32B parameter model to a 70B one, and also bumped