Skip to content

Bonbon Dino — Debugging Lessons

A cross-project bug log. Each entry: Symptom → Root cause → Fix → File(s). Drawn from real sessions, ordered roughly by category (UI, deploy, auth, data, email).

This file is the most valuable one when a future agent is stuck on a "this is weird" bug — search before re-debugging.


UI / Layout

Mobile drawer appears transparent / see-through

  • Symptom: On phones, opening the hamburger menu shows a partially-transparent drawer constrained to the header bar instead of the full viewport.
  • Root cause: A CSS backdrop-filter: blur on the sticky <header> creates a new containing block for position: fixed descendants. The drawer's fixed positioning was being scoped to the header's bounds, not the viewport.
  • Fix: Render the drawer with createPortal(<Drawer />, document.body) so it lives outside the blurred header's stacking context. Add a mounted state guard (useEffect sets it true) so SSR doesn't try to access document.body.
  • File: rushroad_storage/components/mobile-menu.tsx
  • Commit: 2eeb3cf (mobile menu fix)

Eye-icon password toggle pushed outside the input

  • Symptom: The show/hide-password button visually sits next to the input instead of inside it.
  • Root cause: className="pr-10" was applied directly to the <input>. Padding-right on an input shrinks the text area but the absolutely-positioned eye button — which positions relative to its containing block, the wrapper <div> — sits in the wrapper's right edge, which is now wider than the input.
  • Fix: Move className from the <input> to the wrapper <div>. The input fills the div minus the padding; the button positions inside the padded area.
  • File: rushroad_storage/components/password-input.tsx
  • Commit: 3db2e8b

Pagination disappears when there's only one page

  • Symptom: New board with 8 posts shows no pagination bar, so users wonder if there's anything below.
  • Root cause: Component had if (totalPages <= 1) return null early return.
  • Fix: Always render the pagination component. Show a single "1" pill when total=1.
  • File: rushroad_storage/components/board-pagination.tsx
  • Commit: dfe52cd
  • Symptom: Open question on /board?sort=date&page=3 → close modal → land on /board (page 1).
  • Root cause: The detail link was just /board?id=<uuid> — it set id but dropped every other param. Closing did router.push("/board") instead of router.push(currentParamsWithoutId).
  • Fix: Always preserve the existing search params when building the modal-open link, and remove only id when closing.
  • File: rushroad_storage/app/board/page.tsx, components/board-detail-modal.tsx
  • Commit: 72e2b37
  • Symptom: Whitespace gap between the last section and the footer that doesn't match other pages.
  • Root cause: Default <footer> element styles + a mt-* Tailwind utility had stacked. Mixed default margin and explicit margin caused inconsistent spacing across pages.
  • Fix: Reset <footer> element margin to zero; rely on parent container spacing only.
  • File: rushroad_storage/components/footer.tsx
  • Commit: ee90ecd
  • Symptom: On iPad Pro (Safari) the footer never appears — the page looks like it ends at the last content section. Works fine in desktop Chrome, so it's easy to miss.
  • Root cause: A zoom: 0.9 style on the page-root <div> (a hack to shrink the desktop layout to fit). Safari's zoom support is partial and miscalculates document scroll height, so the footer is laid out below the reachable scroll area. Chrome computes scroll height correctly and hid the bug. Tell-tale: footer.offsetTop (3826) was greater than document.documentElement.scrollHeight (3753).
  • Fix: Remove zoom entirely and let the existing responsive clamp()/vw units carry the layout. After removal, offsetTop (3826) < scrollHeight (4169) and the footer is fully reachable on desktop, iPad, and phone.
  • Detect: const f=document.querySelector('footer'); f.offsetTop <= document.documentElement.scrollHeight must be true.
  • File: bonbon_dino_web/src/pages/Index.tsx (root <div>, ~line 494)

Page hero subtitle clipped on narrow viewports

  • Symptom: Long subtitle text gets cut off horizontally on phones.
  • Root cause: Subtitle used whitespace-nowrap (or inherited it).
  • Fix: Allow wrap; remove whitespace-nowrap. Add max-w-prose if you want a sensible line length.
  • File: rushroad_storage/components/page-hero.tsx
  • Commit: 69a0cca

Horizontal overflow on Android, not iOS

  • Symptom: Board page scrolls horizontally on Android Chrome; fine on iOS Safari.
  • Root cause: A long unbreakable word in a question title pushed the table wider than viewport. iOS Safari broke it; Android Chrome did not.
  • Fix: overflow-x-auto on the table wrapper + break-words on the title cell.
  • File: rushroad_storage/components/board-table.tsx (or board grid)
  • Commit: 7bff87b

Landing hero collapses on short viewports

  • Symptom: On a 480px-tall mobile viewport, the hero contents are cut by min-h-[640px] + overflow-hidden.
  • Root cause: Fixed min-height plus overflow-hidden.
  • Fix: Let the hero grow to fit content (min-h-[480px] floor, content drives the rest).
  • File: rushroad_storage/app/page.tsx
  • Commit: ce808e0

Mobile drawer too wide / admin sidebar visible on mobile

  • Symptom: Public drawer overflows the screen; admin sidebar takes 60% of the viewport on phones.
  • Root cause: Drawer was w-80 (320px) on all breakpoints; admin sidebar didn't hide at <md.
  • Fix: Public drawer: w-72 sm:w-80. Admin sidebar: hidden md:block + add a top-nav-bar dropdown for <md.
  • File: rushroad_storage/components/mobile-menu.tsx, app/admin/layout.tsx
  • Commit: 68170e7

Deploy / Build

wrangler pages deploy produces a half-broken site

  • Symptom: Site loads HTML but no CSS/JS/API routes work — clicks don't do anything, fonts unstyled.
  • Root cause: wrangler pages deploy only uploads static assets — the Worker bundle (which serves SSR + API) is ignored. The OpenNext output needs the worker bundle deployed too.
  • Fix: Use npx opennextjs-cloudflare deploy instead — that command uploads both the worker and the assets together. Switch wrangler.jsonc (or wrangler.toml) from Pages config (pages_build_output_dir) to Workers config (main = ".open-next/worker.js" + assets.binding = "ASSETS").
  • File: wrangler.jsonc, deploy command in package.json
  • Commit: 09dd7ef

Custom domain refuses to attach to Worker

  • Symptom: Adding <domain> as a custom domain on the Worker fails with a vague error.
  • Root cause: Cloudflare's "Add Site" flow auto-imports placeholder A records pointing to 127.0.0.1. These exist before the Worker custom domain step, and they block the attachment.
  • Fix: Delete all 3 placeholder A records under DNS → Records → A. Then add the custom domain to the Worker.
  • File: Cloudflare dashboard (no code)
  • Source: memory/2026-06-05_2030_domain_live_turnstile_https.md

Turnstile widget not rendering in production (or siteKey is undefined in the bundle)

  • Symptom: The Turnstile div is in the DOM but never initializes — no challenge appears, form submission fails siteverify. The Send button stays disabled because onSuccess never fires. Curl-ing the page's JS chunk shows the env var name (process.env.NEXT_PUBLIC_TURNSTILE_SITE_KEY) uninlined instead of the literal key.
  • Root cause: NEXT_PUBLIC_TURNSTILE_SITE_KEY was set as a Wrangler secret or only in wrangler.jsonc vars. Both are runtime-only. NEXT_PUBLIC_* vars are inlined into the client bundle at build time; if the build didn't have the value, the bundle ships the var name unresolved and the browser sees undefined.
  • Fix depends on deploy mechanism:
  • Workers Builds (default): commit a .env.production (NOT gitignored) at the repo root with the public var. Next.js reads it during opennextjs-cloudflare build and inlines the value. The Turnstile site key is public by design (it ships in HTML) so committing is fine. The secret key stays out of git via wrangler secret put.
  • GitHub Actions (alternative): add the var as a GitHub Actions secret + inject it into the npm run build:cf step's env.
  • Verify: after deploy, curl -s https://<site>/<page> | grep -oE '/_next/static/chunks/[A-Za-z0-9_./~%-]+\.js' | sort -u to find the JS chunks, then curl one and grep for the literal key value. If the literal value is in the bundle, you're set. If you see the env var name uninlined, the build didn't have it.
  • File: .env.production (Workers Builds) or .github/workflows/deploy.yml (GH Actions)
  • Source: keilen_family_services — Jun 15, 2026 (commit 120ef4e for .env.production)

npm run build:cf crashes with cryptic OpenNext error

  • Symptom: Build fails with errors mentioning proxyExternalRequest, edgeExternals, or middleware.
  • Root cause: open-next.config.ts is missing required keys. @opennextjs/cloudflare v1.19+ requires all three.
  • Fix:
    export default defineCloudflareConfig({
      proxyExternalRequest: "fetch",
      edgeExternals: [],
      middleware: { external: true },
    });
    
  • File: rushroad_storage/open-next.config.ts
  • Commit: f3137c2

Resend is not defined during build

  • Symptom: npm run build:cf fails with Cannot read properties of undefined or RESEND_API_KEY not set during static analysis.
  • Root cause: new Resend(process.env.RESEND_API_KEY!) at module top runs during build, when the key isn't set in the build environment.
  • Fix: Lazy-init the Resend client — only construct on first send. See email-checklist.md §2.
  • File: rushroad_storage/lib/email.ts
  • Commit: ce812ce

Cloudflare can't serve static files

  • Symptom: After deploy, all CSS/JS/image requests 404.
  • Root cause: wrangler.jsonc (or wrangler.toml) missing the assets binding section.
  • Fix:
    {
      "assets": {
        "directory": ".open-next/assets",
        "binding": "ASSETS"
      }
    }
    
  • File: wrangler.jsonc
  • Commit: 09dd7ef

Nameserver change doesn't propagate

  • Symptom: Updated NS records at the registrar, but dig <domain> NS still shows old nameservers a day later.
  • Root cause: Some registrars (whois.com observed) have two panels — "DNS Management" (which doesn't allow NS edits) and "Domain Registration → Name Servers" (which does). User updated NS records in the wrong panel; the change didn't actually apply.
  • Fix: Use the registrar's "Domain Registration → Name Servers" modal, not the DNS Management panel.
  • File: Registrar dashboard (no code)
  • Source: memory/2026-06-05_1900_domain_nameserver_update.md

Cloudflare custom domain won't attach to Worker — "externally managed DNS records"

  • Symptom: Adding a custom domain to a Worker via the Cloudflare dashboard fails with "This hostname already has externally managed DNS records."
  • Root cause: Cloudflare imported a DNS record for that hostname when the zone was added (e.g. a www CNAME from Namecheap, or a placeholder A record Cloudflare inserts automatically). That record blocks the custom domain attachment.
  • Fix: Go to DNS → Records, delete the conflicting record for that hostname (A or CNAME), then retry the custom domain attachment. The attachment flow will create its own internal record.
  • Key rule: Delete first, attach second. Never try to add a custom domain while any DNS record exists for that hostname.
  • Applies to: Both www subdomains and the apex — same dance for each.
  • File: Cloudflare dashboard (no code)
  • Source: keilen_family_services DNS setup — Jun 13, 2026

Apex HTTPS returns HTTP 522 (Connection Timed Out)

  • Symptom: https://<apex> hangs and eventually returns Cloudflare error 522. https://www.<domain> works fine.
  • Root cause: Cloudflare DNS has an A record for the apex pointing to a registrar parking IP (e.g. Namecheap's 192.64.119.207). Cloudflare proxies the request to that IP, which doesn't respond → 522. The Worker never sees the request.
  • Fix: Delete the apex A record in Cloudflare DNS. Then add the apex as a Worker custom domain (Workers & Pages → Worker → Domains → + Add Domain, leave subdomain field empty). Cloudflare issues a cert and creates the internal routing record.
  • Do NOT re-add the A record after deleting it — the custom domain attachment replaces it.
  • File: Cloudflare dashboard (no code)
  • Source: keilen_family_services DNS setup — Jun 13, 2026

wrangler.jsonc routes + custom_domain: true fails with error 10082

  • Symptom: wrangler deploy fails: "Can't infer zone from route" (code 10082).
  • Root cause: Adding a routes block with custom_domain: true to wrangler.jsonc requires the domain's zone to already be active in the Cloudflare account. If the domain's DNS is hosted at a third-party registrar (Namecheap etc.), no zone exists yet — the API can't resolve the zone and rejects the deploy.
  • Fix: Remove the routes block from wrangler.jsonc entirely. Attach custom domains via the Cloudflare dashboard (Workers & Pages → Worker → Domains → + Add Domain) after the zone is active. Dashboard-attached domains do not need any entry in wrangler.jsonc.
  • Side effect: A failed deploy with the routes block can mark the workers.dev URL as Inactive. After removing the block and re-deploying successfully, the workers.dev URL reactivates automatically.
  • File: wrangler.jsonc (remove routes block)
  • Source: keilen_family_services DNS setup — Jun 13, 2026

Cloudflare Redirect Rule redirect URL uses $1 — doesn't substitute

  • Symptom: Redirect Rule with target URL https://www.example.com/$1 fires but the path is literally /$1 instead of the matched path.
  • Root cause: Cloudflare Redirect Rules require ${1} (curly-brace syntax), not $1. Without braces the variable is not substituted.
  • Fix: Use https://www.<domain>/${1} in the redirect target.
  • File: Cloudflare dashboard → Rules → Redirect Rules
  • Source: keilen_family_services DNS setup — Jun 13, 2026

Full checklist: Namecheap domain → Cloudflare Workers (custom domain)

The shortest proven path — no client interaction required, everything managed in the Cloudflare account:

  1. Add zone to Cloudflare — Cloudflare dashboard → Add a Site → enter the bare domain → choose Free plan → copy the two nameserver hostnames.
  2. Update nameservers at Namecheap — Domain List → Manage → Nameservers → Custom DNS → paste both Cloudflare nameservers. Save.
  3. Wait for propagation — typically 5–30 min; up to 24h. Check with dig <domain> NS.
  4. Delete conflicting DNS records — Cloudflare auto-imports records from the old DNS. Delete any A or CNAME for the hostnames you'll attach as custom domains. (MX and TXT records are fine to keep.)
  5. Attach custom domains — Workers & Pages → Worker → Domains → + Add Domain.
  6. For www: type www in the Subdomain field.
  7. For the apex: leave the Subdomain field empty.
  8. Do each hostname separately; wait for "Active" before moving on.
  9. Enable Always Use HTTPS — SSL/TLS → Edge Certificates → toggle "Always Use HTTPS" on.
  10. Add apex → www Redirect Rule (optional, for canonical www) — Rules → Redirect Rules → + Create. Match: URI Full wildcard https://<domain>/*. Action: 301 to https://www.<domain>/${1}. Note: if the apex is attached as a Worker custom domain, the Worker intercepts HTTPS apex requests before the redirect fires. Handle canonical via NEXT_PUBLIC_SITE_URL / HTML canonical tags instead.
  11. Smoke test: curl -Ik https://www.<domain> → 200; curl -Ik http://<domain> → 301 → https.

  12. Source: keilen_family_services DNS setup — Jun 13, 2026; also rushroad_storage — Jun 5, 2026

Two apps with the same name in Cloudflare dashboard — one is a Pages orphan

  • Symptom: Workers & Pages shows two apps both named the same as your project. One has a Pages icon and a prominent GitHub badge. The other has a Worker icon and www.<domain> + 2 other routes underneath it. Only the Worker is actually serving the custom domain; the Pages project 404s on real routes.
  • Root cause: The repo was originally set up to deploy on Cloudflare Pages (Pages GitHub integration is the obvious default for Next.js). When production was later switched to OpenNext-on-Workers (because Pages' Next.js adapter couldn't support server actions + D1 + email bindings), the old Pages project was never deleted. It keeps rebuilding on every push, owns no domain, and serves nothing live.
  • Why this is confusing: the Pages project shows the GitHub connection in the dashboard listing (top row, prominent badge). The Worker's GitHub connection (a separate Cloudflare product: Workers Builds) lives on the Worker's own Settings → Build tab, not on the listing page. So the listing page makes it look like only the Pages project is connected to GitHub — but pushing to main actually deploys the Worker, and the Pages project is just dead weight.
  • Confirm which app is production:
  • The dashboard listing shows the Worker with www.<domain>.com underneath it. The custom domain is attached there, not on Pages.
  • The Pages project's Custom domains tab is empty (no custom domain attached).
  • curl -I https://www.<domain> response headers include x-opennext: 1 (added by @opennextjs/cloudflare Worker runtime — Pages would not add this header).
  • Fix: delete the Pages project from the dashboard. It owns no domain, so deletion cannot affect production. Verify after deletion with one more git push of a small change — the live site should still update, proving the Worker was the real auto-deploy target all along.
  • Anti-pattern: when migrating a project from Pages to Workers, leave the old Pages project in place "just in case." It serves nothing and confuses every future person who looks at the dashboard. Delete it as part of the migration.
  • File: Cloudflare dashboard (no code)
  • Source: keilen_family_services — Jun 15, 2026 (resolution confirmed by x-opennext: 1 header on live site + empty Custom domains tab on the Pages project)

Failed to fetch on /api/auth/login from a deployed SPA

  • Symptom: Sign-in toast shows "Sign in failed — Failed to fetch". DevTools Network tab shows the login request either missing entirely or with status (failed) and no response body. The frontend bundle is otherwise healthy — assets, JS, and HTML all 200.
  • Root cause: The frontend is calling an API URL it can't reach. Three flavors, in order of frequency:
  • The build-time VITE_API_BASE_URL env var is missing or wrong in the deploy platform's settings. The code falls back to http://localhost:8787, the browser tries the user's own machine, gets ERR_CONNECTION_REFUSED, and surfaces it as "Failed to fetch".
  • The API Worker literally has no code deployed (the Worker exists in the Cloudflare account but has zero deployments, so every request 404s at the edge).
  • The API exists but CORS is misconfigured — the response lacks the frontend's origin in Access-Control-Allow-Origin and the browser blocks it before any code runs.
  • Fix: Same-origin routing. Either consolidate the API into the same Cloudflare resource (functions/api/[[path]].js for Vite SPAs, or co-located app/api/ routes for Next.js — see db-and-deploy-checklist.md §15), or ensure the deployed build has a correct VITE_API_BASE_URL AND the API's CORS layer echoes the deployed origin with Access-Control-Allow-Credentials: true. Same-origin is the only option that can't be broken by a missing env var.
  • Lesson: Two repos + two deploy targets + one baked env var is a three-way coupling that breaks the first time any one of them drifts. Default to single repo, single deploy.
  • File: src/lib/api.ts (Vite SPA) or lib/api-client.ts (Next.js). Network tab → the failed request's URL tells you which of the three root causes applies.
  • Source: bonbon_dino_web/memory/2026-06-12_failed_to_fetch_admin_login.md

Auth

  • Symptom: Customer enters their question password, sees "unlocked!", but the modal re-fetches and shows locked metadata again.
  • Root cause: Unlock cookie set with path: "/board". The modal fetches /api/questions/<id> — a different path — so the cookie wasn't sent.
  • Fix: Set unlock cookie with path: "/".
  • File: rushroad_storage/app/api/questions/[id]/unlock/route.ts
  • Source: memory/2026-06-04_2046_board_modal_and_header_polish.md

"Incorrect password" on every seeded question

  • Symptom: Test seed of 50 questions with known password "secret123" all return "Incorrect password" when unlocking.
  • Root cause: Seed ran before .env.local was finalized. Stored access keys are HMACs of password using the OLD JWT_SECRET. New requests compute HMAC with current JWT_SECRET — they don't match.
  • Fix: Re-run the seed script with the current JWT_SECRET. Don't seed before env vars are finalized.
  • File: rushroad_storage/scripts/seed-board.ts
  • Source: memory/2026-06-04_2046_board_modal_and_header_polish.md

Admin login broken in production despite Vercel demo working

  • Symptom: Production Cloudflare site shows "Invalid credentials" on admin login even with correct password; local dev and Vercel demo both work.
  • Root cause: ADMIN_PASSWORD_HASH not set as a wrangler secret. Login routes fell through to the env-var seed path, which compared against undefined.
  • Fix: wrangler secret put ADMIN_PASSWORD_HASH with the bcrypt hash.
  • File: wrangler config (no code change)
  • Source: memory/2026-06-05_1238_cloudflare_production_deploy.md

Admin email changed in dashboard then user can't log in with env-var email

  • Symptom: Admin updated email via settings page to a new address. Tries to log in with the new email — fails. Tries the old env-var email — also fails.
  • Root cause: Login route checks DB first; DB has the new email. Old env-var email no longer accepted. Password may have been updated too — the bcrypt hash in env vars is stale.
  • Fix: Once an admin updates credentials, env-vars are dead. Either: (1) log in with the new credentials from settings, or (2) directly UPDATE the admin_credentials row in Turso with a fresh bcrypt hash:
    UPDATE admin_credentials SET password_hash = '<new_bcrypt_hash>' WHERE id = 1;
    
  • File: Turso DB (no code change)
  • Source: memory/2026-06-05_2030_domain_live_turnstile_https.md

getDb() returns undefined and surfaces as a vague TypeError

  • Symptom: A wrangler.jsonc rename or a missed D1 binding causes the first .prepare(...) call in a route to throw TypeError: Cannot read properties of undefined (reading 'prepare'). The stack trace points at a random route, not at the misconfiguration. The error is not actionable from the stack.
  • Root cause: getDb() returns env.DB directly with no guard. The D1 binding being undefined is a configuration error, but it surfaces as a runtime error far from the cause.
  • Fix: Add a missing-binding guard at the boundary:
    if (!env.DB) throw new Error('D1 binding "DB" is not configured')
    return env.DB
    
    Fails fast at the entry point with a clear, actionable message. The deploy is misconfigured, not the route.
  • Lesson: Every "returns a binding from getCloudflareContext()" helper should guard. The same pattern applies to env.RESEND_API_KEY, env.TURNSTILE_SECRET_KEY, etc. — fail at the boundary, not deep in a route.
  • File: keilen_family_services/lib/db.ts

Email-bombing a single-admin forgot-password endpoint

  • Symptom: With no Turnstile on POST /api/auth/forgot-password, an attacker can call the route hundreds of times per second. The admin's inbox is flooded, Resend quota burns down, and any in-progress password reset is invalidated on every call (the route DELETEs the existing token row before INSERTing a new one — see the code referenced below).
  • Root cause: Three missing things: no rate limit at the route or WAF, no Turnstile challenge, and a token-replacement strategy that lets attackers DoS legitimate users in progress.
  • Fix: Layer three controls:
  • Turnstile on the forgot-password form (and on login) — reuse the same lib/turnstile.ts helper as the contact form. The keilen pattern works for both routes.
  • Skip issuing a new token when an unexpired one already exists. Look up the most recent password_reset_tokens.expires_at; if it's still in the future, return { success: true } without re-issuing. Attackers can no longer invalidate a legitimate reset.
  • (Dashboard layer) WAF rate-limit on POST /api/auth/forgot-password and POST /api/auth/login — see db-and-deploy-checklist.md §11.
  • File: keilen_family_services/app/api/auth/forgot-password/route.ts, keilen_family_services/lib/turnstile.ts, keilen_family_services/components/admin/LoginForm.tsx

Data

"Invalid Date" on board listing after deploy

  • Symptom: Local dev shows dates correctly. Production Cloudflare shows "Invalid Date" everywhere.
  • Root cause: Drizzle integer-mode date columns serialize as Unix int. The int survives Node but corrupts across the Worker fetch boundary, deserializing as NaN. new Date(NaN) → "Invalid Date".
  • Fix: Switch all date columns to text storing ISO 8601 strings. new Date(isoString) works identically on every runtime.
    createdAt: text("created_at").notNull().default(sql`(datetime('now'))`),
    
  • File: rushroad_storage/lib/schema.ts
  • Source: Multiple session memories; foundational data-layer decision.

Trim script destroyed local dev data instead of prod

  • Symptom: Ran a script to trim production Turso to 20 posts. It trimmed local.db instead.
  • Root cause: Script used process.env.TURSO_DATABASE_URL, which .env.local had pointed at file:./local.db for dev. Production URL was never loaded.
  • Fix: Before any destructive script, log the target URL and prompt to confirm. For prod, use the Turso CLI directly (turso db shell <db-name>) with an explicit token, no env files.
  • File: rushroad_storage/scripts/trim-board.ts
  • Source: memory/2026-06-05_2325_qa_cleanup_and_form_fixes.md

Board crashes when DB is empty

  • Symptom: Brand new site, no questions yet, /board shows 500.
  • Root cause: No try/catch around the DB read. An empty result was fine, but a transient connection error in early-deploy state crashed the route.
  • Fix: Wrap the read in try/catch, fall back to empty array on error, render the "no questions yet" empty state.
  • File: rushroad_storage/app/board/page.tsx
  • Commit: 9e2b544

Pricing units missing in production after deploy

  • Symptom: Pricing page shows empty grid on first prod load.
  • Root cause: Seed-on-first-access logic worked locally but production DB hadn't been hit yet, AND the seed was inside an admin-only API route the first user wasn't hitting.
  • Fix: Move seed logic into the read path (getPricingUnits() itself), not the admin write path. Now any first read triggers the seed.
  • File: rushroad_storage/lib/pricing.ts
  • Commit: Patterns from lib/pricing.ts seed-on-read; production-only fix done directly in DB (commit 3331)

Post numbers out of order after manual prod insert

  • Symptom: Inserted a question directly via SQL to backfill — its post_number clashed with an existing row.
  • Root cause: post_number is computed in the API route via MAX(post_number) + 1, not by a DB constraint. Manual inserts bypassed it.
  • Fix: Manual SQL must SELECT MAX(post_number) + 1 first. Better: add a unique constraint on post_number so a clashing insert fails loudly instead of corrupting silently.
  • File: rushroad_storage/lib/schema.ts (consider unique constraint), app/api/questions/route.ts
  • Commit: Pattern surfaced during pricing-data prod fix; not a single commit

Email

  • Symptom: Admin receives reset email but the link starts with "undefined".
  • Root cause: process.env.DOMAIN not set. Template literal interpolation puts the string "undefined" in the URL. DB insert had already succeeded, leaving an orphan token row.
  • Fix: Validate DOMAIN before any DB write that produces a link. Throw 500 first; no row, no email, no orphan.
  • File: rushroad_storage/app/api/auth/forgot-password/route.ts
  • Commit: 74a408f

Admin notification subject too generic

  • Symptom: Admin receives "New question" — when they manage multiple Bonbon Dino sites, they don't know which one.
  • Root cause: Subject was hard-coded to "New question" without the business name.
  • Fix: Prefix subject with [<Business Name>]. See email-checklist.md §6.
  • File: rushroad_storage/lib/email.ts
  • Commit: 26e9ac8
  • Symptom: Admin clicks the notification email → lands on the dashboard → has to find the question manually.
  • Root cause: Email link was ${DOMAIN}/admin/dashboard. The admin then had to search/scroll.
  • Fix: Link to ${DOMAIN}/board?id=<uuid> — the public board with the question modal already open. Since the admin's session cookie is set, they see the reply form inline.
  • File: rushroad_storage/lib/email.ts
  • Commit: 3e4fadb

Working Habits / Process

"Don't commit yet" leaves changes adrift between sessions

  • Symptom: User says "don't commit, I want to fully test" → testing reveals issues → session ends → next session can't find what's on disk.
  • Fix: When the user holds commits, the session memory log must list every modified file explicitly so the next session can pick up. See memory/2026-06-04_2046_board_modal_and_header_polish.md for the format.
  • Lesson: Always write the session log at the end of held-changes sessions, even if no commits land.

Force-pushing to fix a stuck Vercel deploy

  • Symptom (Vercel-specific, historical): Vercel deploy stays BLOCKED forever. Logs show "could not be matched to a GitHub account".
  • Root cause: Local git config used a name/email that doesn't match any GitHub account. Vercel's deployment gate blocked it.
  • Fix: Set local git config to the user's actual GitHub email, amend or new-commit, force-push.
  • Note: This is Vercel-only. Cloudflare-only deploys (the current default) don't have this gate. Recorded here as historical context — if you ever stand up a Vercel demo, you'll need to know.
  • Source: memory/ observations on 2026-06-05 16:xx range (deploy 4:48–5:16 PM)

Forms / Validation

Phone validation rejects formatted numbers like (517) 555-0142

  • Symptom: Phone input passes the UI but the API returns a validation error for any number written with parentheses: (517) 555-0142, (800) 555-0100, etc.
  • Root cause: The regex was anchored to digits and dashes only (e.g. /^\d{3}-\d{3}-\d{4}$/). A leading ( is not a digit, so it fails immediately.
  • Fix: Accept common North American formats — strip non-digit characters first, or use a permissive regex like /^[\d\s()\-+.]{7,15}$/. Alternatively, normalize before validating: phone.replace(/\D/g, "") → check 10-digit length.
  • File: keilen_family_services/lib/invoices.ts — phone field in parseInvoiceForm
  • Commit: 5bb63d7

How to extend this file

This is a living document. Add new entries at the bottom of the relevant category (UI / Deploy / Auth / Data / Email / Process / Forms / Third-party API Integration / Migrations / Refactors), and rewrite existing entries in place when they go stale (e.g., when the default deploy mechanism changes and the "Fix" needs to cover both the new default and the old alternative). The old rule was append-only; it has been retired — outdated fixes are worse than no fix.

Format every entry:

### Short symptom line

- **Symptom:** What the user / monitoring saw.
- **Root cause:** What was actually broken.
- **Fix:** What changed.
- **File:** Path(s) in the reference impl.
- **Commit (optional):** Commit hash or source link.

If the bug spans multiple categories, put it in the one where the fix lived. Cross-reference from the other category with a one-liner: "See UI / Pagination disappears above."

The goal is: a future agent stuck on a bug greps this file by symptom keyword and lands on the root cause in under a minute.


Third-party API Integration

Vendor X uses dollars, vendor Y uses cents — Line[0].Amount is in dollars

  • Symptom: After migrating from Square to QuickBooks, every invoice is sent for 100× the intended amount. A $150 invoice is sent as $15,000.
  • Root cause: Square's invoice API takes integer cents (Amount: 15000 for $150). QuickBooks' API takes decimal dollars (Amount: 150.00 for $150). The cents value was passed through unchanged.
  • Fix: Convert at the boundary — (amountCents / 100).toFixed(2) → parse to number before passing to QBO. Or better: keep the canonical internal model in dollars and convert only when posting to vendors that use cents.
  • Lesson: Check the units of every money field in a vendor's API docs. The same Amount field name can mean cents (Square, Stripe) or dollars (QuickBooks, Xero, FreshBooks). Assumed parity is a 100×-or-1000× bug.
  • File: keilen_family_services/lib/quickbooks.tssendInvoice() body assembly.

OAuth refresh-token race condition revokes the refresh token

  • Symptom: The first request after a token expiry succeeds. Every subsequent request fails with "invalid_grant" / "refresh token expired", even though the app should be able to keep refreshing.
  • Root cause: The vendor (Intuit QuickBooks is the documented case) rotates the refresh token on every refresh. Two concurrent calls each POST the same refresh token; the second POST is treated as a replay and the new token is revoked.
  • Fix: Serialize refreshes with a module-level promise:
    let inFlightRefresh: Promise<string> | null = null
    async function getValidAccessToken() {
      if (tokenStillValid()) return currentToken
      if (!inFlightRefresh) {
        inFlightRefresh = doRefresh().finally(() => { inFlightRefresh = null })
      }
      return inFlightRefresh
    }
    
    Every caller awaits the same promise. Single POST, single new refresh token, single write to the token store.
  • Lesson: Any vendor that "rotates refresh tokens on every refresh" (Intuit, some OAuth providers) is also implicitly "revokes if you POST twice." A Promise-level mutex is the cheapest fix.
  • File: keilen_family_services/lib/quickbooks.tsinFlightRefresh module variable.

Multi-step external API flow leaks orphans on partial failure

  • Symptom: A 3-call external flow (e.g. Square order → invoice → publish) leaves a dangling draft record in the third party when the 2nd or 3rd call fails. The first call's side effect isn't rolled back; the user sees the original error and never knows the orphan exists. With Square, the result is a draft Order in the Square dashboard with no invoice attached.
  • Root cause: Each step was a separate await with no cleanup path. The natural "transaction" model doesn't exist in the third party's REST API, so the integration has to build its own.
  • Fix: Wrap the dependent steps in try { ... } catch (err) { await cleanup(); throw err; }. On failure, do a best-effort compensating action on the first resource — typically a state transition to a terminal state (POST /v2/orders/{id} with state: 'CANCELED' for Square orders). Don't try to DELETE; most external resources only transition state. Swallow the cleanup error so the original error reaches the caller. Test the failure path with a mocked client (set up the mock to fail on the 2nd or 3rd call, assert the cleanup was attempted).
  • Lesson: "Compensating action" is the API equivalent of a database rollback. Every multi-step external flow needs one for each earlier step that has a side effect. If the vendor doesn't support a terminal state transition, escalate to the vendor — there's no safe alternative.
  • File: keilen_family_services/lib/square.ts (sendInvoice + private cancelOrder)

Migrations / Refactors

Feature migration leaves stale user-facing copy two days later

  • Symptom: After migrating from Square to QuickBooks, two strings in app/admin/invoices/page.tsx still said "Square emails the customer" and "in the Square Dashboard" — visible to admins creating invoices.
  • Root cause: Migration commit (e.g., c222ffb) replaced the underlying library, the env vars, the spec, and most admin copy — but the find-and-replace missed the page-description block. The new entries on the same page said "QuickBooks"; only the description block was stale.
  • Fix: Grep the whole repo for the old vendor name across *.tsx / *.ts before considering the migration done. rg -l "Square|square" app/ components/ --type tsx --type ts (excluding node_modules, .next, .open-next).
  • Lesson: Migrations of a whole feature need a "vendor name in user-facing strings" sweep, not just a code-level find-and-replace. The new code paths are correct; the old strings are in copy, not imports, and survive by accident.
  • File: keilen_family_services/app/admin/invoices/page.tsx
  • Commit: 3fd922c (cleanup after c222ffb)

React 19 server action resets uncontrolled form inputs on every submission

  • Symptom: Contact form fields (text inputs, radio buttons, checkboxes) all clear after the server action returns — even when the action returns a validation error. Users have to re-type everything.
  • Root cause: <form action={formAction}> is React 19's progressive-enhancement form pattern. React resets all uncontrolled inputs after every action completion, mimicking native form reset behavior, regardless of whether the action returned success or an error.
  • Fix: Switch from action={formAction} to onSubmit + startTransition. The e.preventDefault() blocks the native reset; the server action still runs via the useActionState dispatch:
    <form
      onSubmit={(e) => {
        e.preventDefault();
        const formData = new FormData(e.currentTarget);
        startTransition(() => formAction(formData));
      }}
    >
    
  • Lesson: Any form using useActionState + <form action={...}> in React 19 will auto-reset on submission. If the form has fields the user should keep after a failed attempt, use onSubmit instead.
  • File: keilen_family_services/components/contact/ContactForm.tsx
  • Commit: 1407e33

Cloudflare Turnstile token consumed on first submit; retry always fails

  • Symptom: User submits the contact form with a missing field. Server returns a validation error. User fixes the missing field and resubmits — gets "Security check failed. Please try again." even though the Turnstile widget still shows its green checkmark.
  • Root cause: Cloudflare Turnstile tokens are single-use. The server-side siteverify call consumes the token on the first submission (even if Zod validation fails afterwards). The widget doesn't know the token was consumed server-side, so it keeps displaying "Success!" — but the turnstileToken state holds the stale, already-used token. The second submission sends that dead token → Cloudflare rejects it.
  • Fix: Use a ref to imperatively reset the Turnstile widget after any server action error, and clear the token state so the submit button re-disables until the widget re-verifies:
    const turnstileRef = useRef<TurnstileInstance>(null);
    
    useEffect(() => {
      if (state.error) {
        turnstileRef.current?.reset();
        setTurnstileToken('');
      }
    }, [state]);
    
    <Turnstile ref={turnstileRef} ... />
    
  • Lesson: Always reset Turnstile (and clear your token state) after a failed server action. The widget's visual state does not reflect server-side token consumption.
  • File: keilen_family_services/components/contact/ContactForm.tsx
  • Commit: 1407e33

Post-launch UI rollback leaves orphaned API routes

  • Symptom: A feature shipped with both UI (QuickBooksStatusCard.tsx) and API (/api/admin/quickbooks/* routes). A product decision weeks later removed the UI ("not the right surface"). The API routes were left in place, still serving 200, with no caller. The next dev who grepped for "QuickBooks" found live routes, assumed the feature was active, and went down a 30-minute rabbit hole before finding the deleted component.
  • Root cause: Rollback was scoped to the UI; the routes were not explicitly archived. There's no general "this route has no UI caller" lint to catch it.
  • Fix: When removing a UI surface that backs an API, choose one of three explicit paths:
  • Delete the routes too if the feature is permanently retracted (preferred when there's no near-term plan to re-surface).
  • Mark the routes deprecated with a // DEPRECATED: no UI caller as of <date> — see <issue> header, so a future grep finds the context.
  • Keep both if the feature is expected to come back; record the plan in a spec.
  • Lesson: "What UI does this API support?" is a question that has to be answerable from the repo, not from memory. Either pair the files in a directory, or add a one-line breadcrumb on the route.
  • File (this case): keilen_family_services/app/api/admin/quickbooks/* (routes live; components/admin/QuickBooksStatusCard.tsx deleted in 3fd922c)