Skip to content

managed-datastore-write-cost-discipline

Managed Datastore Write-Cost Discipline — Aggregate Before You Write

Section titled “Managed Datastore Write-Cost Discipline — Aggregate Before You Write”

Managed row-billed stores (Cloudflare D1, Postgres-per-row insert paths, DynamoDB write units) bill per row or per operation written and scanned, not per byte stored. Raw event payloads written 1:1 are the single most expensive thing you can do with them.

  1. Aggregate before you write. Only aggregated, calculated, or otherwise reduced data goes into a row-billed datastore — never the raw event stream. If you cannot name the aggregation or reduction a row represents, it does not belong there.
  2. Raw data stays in object storage or is computed at query time. Object storage (R2-class, no egress) holds raw event payloads keyed by ID/timestamp; anything re-derivable at query time is not persisted at all.
  3. Watch the read-scan bill, not just the write bill. Row-billed stores charge rows scanned by the query planner, not rows returned. One unindexed WHERE over an N-row table bills N rows read per query.
  4. Index every filtered column, LIMIT every scan, avoid SELECT * over wide tables. Never COUNT(*) a growing table on a hot path — maintain a counter column/row instead.
  5. Warn before you write or scan, not after. Before any command that writes to or scans a managed datastore (D1 INSERT/UPDATE/batch, cf d1 query, bulk R2 ops, a migration), state the estimated row/operation count and its dollar cost, then proceed. Unknown volume is itself the warning — say so rather than staying silent. Pairs with modules/t1k-base/rules/preview-first-batch.md (cite its mechanics, don’t repeat them).
  6. Every new table declares its class — -- datastore-class: aggregate or -- datastore-class: raw reason="..." on the line above a new CREATE TABLE (exempt: CREATE TEMP TABLE; pre-existing tables are grandfathered). Add bun .claude/scripts/t1k-datastore-class-check.cjs --base origin/main to CI.
  7. Growing config is a table, not a Worker var. A value that grows with the product (consumers, allow-lists, flags, tenants) is read from D1 at request time, so adding one needs no redeploy: skills/t1k-cloudflare/references/config-as-data.md.

Pricing table, decision table + audit flag, worked example, the originating incident, sibling rule (docker-volume-discipline.md), the operational playbook (skills/t1k-datastore-cost/SKILL.md) and canonical agent: docs/datastore-cost-discipline.md.