Content Pipeline: September State of the Machine

The pipeline that publishes these daily articles has been through four rewrites. In September it stopped needing changes. That stability is worth documenting, because a pipeline that doesn't need changes is a pipeline that's done.

The Current Shape

notes file  ->  batch script  ->  POST to API  ->  published post
    |             |                  |
    |             |                  +->  static generation
    |             +->  markdown body                 |
    +->  topic queue                                 +->  CDN edge

Four steps. No queue, no worker, no cron. A human (me) runs the batch script when a month's worth of articles is ready.

What Each Rewrite Fixed

• Rewrite 1: stopped writing raw SQL and used the API
• Rewrite 2: batched by month instead of one-at-a-time
• Rewrite 3: moved bodies from separate files into the script
• Rewrite 4: added idempotent slugs so reruns are safe

Each rewrite removed a step I thought was necessary. The system kept shrinking until it stopped causing problems.

Why Monthly Batches Beat Daily Publishing

Daily publishing sounds disciplined. In practice it meant a tiny daily task that kept getting dropped. Monthly batches let me sit down once, write thirty pieces in focused mode, and then not think about content for a month. The system matches my attention pattern.

The Lesson

A pipeline is done when changes make it worse. September's pipeline passes that test. Every 'improvement' I've considered would add complexity without moving the metric that matters: articles out the door.