PHOTOLIB — Proposal: an agent-native photo intelligence layer over 450k images

For: Byron · Status: PROPOSAL ONLY — no build, no mockups yet, per instruction Sources of truth: Lightroom Classic catalog (.lrcat, Windows PC) · Excire AI metadata · originals on NAS (read-only)

1. The core insight: you already paid for the index

450,000 photos sounds like an indexing mountain. It isn't — because two engines already climbed it:

So the strategy is: harvest, don't re-index. Build a lightweight, agent-native index from what exists; add new AI only incrementally, only where the harvest has gaps.

2. Recommended architecture ("Photolib")

Windows PC                      Mac Mini (Mia)                     Anywhere
┌─────────────┐   nightly copy  ┌──────────────────────────┐
│ .lrcat      │ ───────────────▶│ INGEST  lrcat + XMP      │
│ Excire DB   │  (Zia/robocopy  │   ↓                      │
│  → XMP once │   to NAS share) │ PHOTOLIB.db (SQLite)     │   MCP over Tailscale
└─────────────┘                 │  images/keywords FTS5    │◀── OpenClaw fleet
      NAS (read-only SMB) ─────▶│  faces·gps·ratings       │◀── Claude (me)
      │ originals (never write) │  vectors (sqlite-vec)*   │
      │ music library           │ PROXIES 1600px JPEGs     │
      └────────────────────────▶│ SLIDESHOW engine         │──▶ web player (any screen)
                                └──────────────────────────┘──▶ ffmpeg MP4 render
                                                     * Phase 3, optional

Photolib.db — one SQLite file with FTS5 full-text over keywords/captions/paths + structured columns (time, GPS, rating, faces-count, camera). 450k rows ≈ 300–600MB. Query latency: single-digit milliseconds. This is the whole "database size" fear dissolved: metadata at this scale is small; it's pixels that are big, and we never move them.

Proxies — browsing/slideshows never touch NAS RAWs. This is the true long pole, stated honestly: 450k RAWs × ~25MB ≈ 11TB of NAS reads; at a sustained 100–200MB/s that is 15–30 hours of pure I/O plus RAW decode at 0.5–2s/image ≈ 2.5–10 days of throttled background compute — a multi-day pipeline with a resume queue and failure reporting (at 450k, a 1% failure rate = 4,500 files someone must account for: exotic RAWs, CMYK TIFFs, broken EXIF, orientation chaos). Two mitigations change the game and get measured in Phase 0: (a) harvest Lightroom's Previews.lrdata pyramid instead — already-rendered JPEGs incl. your edits, no RAW decode at all; coverage/size audited in Phase 0; (b) LR batch-export on the Windows PC (renders with your edits applied, uses the PC's already-local disk bandwidth). Output ≈ 80GB of 1600px JPEGs either way. Proxies key on LR's id_global UUID (stable across renames/moves), with edit-invalidation (touch-time check) so re-edited photos re-proxy.

MCP server (photolib-mcp on Mia, Tailscale-only) — the fleet hook you asked for. Tools:

Your demo query — keyword ∈ {aurora, northern lights} AND faces ≥ 1 — is the Phase 0 acceptance test, not a promise: it works if Excire's fixed taxonomy emits an aurora concept and its face counts survive harvest as queryable fields (they may export as "one person"/"group" keywords rather than numbers). Phase 0 runs this exact query against a 500-photo sample harvest before anything else is built. If Excire lacks the concept, the query lands in Phase 3 (semantic) instead — still no LLM needed.

3. Semantic search (the careful, optional layer)

You're right to distrust Gemma for this — but the right tool isn't an LLM at all. CLIP-family embedding models (SigLIP-base ONNX) are the standard for photo semantic search: ~400MB, and they enable "moody beach sunset with silhouettes" queries beyond keywords. Honest numbers (to be measured in Phase 0 on Mia, not assumed): at an unverified 40–80 img/s from local proxies, 450k ≈ 2–4 hours of embedding compute — the gating dependency is that proxies must exist first (see the proxy long-pole above). Storage: 450k × 768-dim fp16 ≈ 700MB in sqlite-vec (fp32 would be 1.4GB). Query honesty: brute-force vector search over 450k is ~100–300ms per query, not the millisecond class of the FTS index — fine for interactive use, stated so it's never a surprise. Phase 3, only if the keyword harvest leaves you wanting — my bet is Excire's tags cover 80% of real queries.

4. Slideshow engine (the ProShow bar)

ProShow Producer's essence: timeline of slides → layered motion (Ken Burns pan/zoom), transition library, music-driven timing, render to video. Plan:

5. Phases

Phase What Effort You get
0 Discovery & measurement: catalog snapshot to Mia, keyword/faces/GPS inventory, Excire DB parse attempt, 500-photo sample harvest + the aurora acceptance query, NAS RO mount + throughput test, path-resolution audit, Previews.lrdata coverage check, SigLIP throughput on Mia 1–2 days Measured go/no-go facts replacing every assumption in this doc
1 Harvest + index: lrcat→photolib.db (structured tables, XMP as fallback), Excire harvest per chosen path, FTS, id_global keying, nightly snapshot+verify+swap pipeline 3–4 days Instant search over 450k by everything you've ever tagged
2 MCP + proxies + web slideshow: MCP server + constrained query DSL, proxy pipeline w/ resume+failure queue (multi-day background run), web player + exactly 2 templates + music picker 1–1.5 weeks + proxy runtime The demo: fleet-directed slideshows
3 Semantic layer: SigLIP embeddings over proxies (~hours once proxies exist), hybrid keyword+vector search 2 days "Describe it" search
4 Producer features: headless-WebGL video export, beat-sync, face-aware Ken Burns, template editor, generated music 2–3 weeks, gated on Phase 2 usage ProShow-class output

Estimates are the critique-doubled honest versions; Phase 0's measurements may pull them back down.

6. Things you didn't ask about (but will hit)

  1. The Excire harvest is the most dangerous step and gets a real protocol. Preference order: (a) parse Excire's own SQLite DB directly (%APPDATA%\Excire Foto) — read-only, zero writes anywhere, schema reverse-engineered in Phase 0 on a copy; (b) if unparseable, "Store Metadata" sidecar export to a SEPARATE write location (not the originals share — preserving the read-only invariant), noting that for JPEG/TIFF/HEIC Excire may embed (rewriting originals — if unavoidable: checksum-before/after protocol on a sample, then explicit go/no-go from you); and the Lightroom side-effect must be planned either way: LR will flag "metadata changed on disk" on affected images, so we script the harvest to avoid touching files LR watches, or accept and document the conflict-badge wave. This was a parenthesis in draft 1; the critique correctly promoted it to a first-class risk.
  2. Faces ≠ names. Excire counts faces; LR People view names them only if you've done that work. Phase 0 tells us whether "photos with Sophia" is queryable or only "photos with ≥1 person." Named-face search may become a Phase 3+ item (local face clustering) — flagging now to set expectations.
  3. Virtual copies, stacks, videos, smart previews — the catalog contains all of these; the ingester must handle them (dedupe virtual copies to master image, include/exclude video toggle).
  4. Write-back is a trap. Everything here is read-only by design — Photolib never writes to NAS originals or the catalog. Ratings/tags created by agents live in Photolib's own layer. If you ever want write-back into LR, that's a deliberate later decision (keyword import via LR plugin), not a default.
  5. Privacy perimeter. This index includes your family's faces and locations. MCP binds to Tailscale only — never a public endpoint, no exceptions, regardless of the fleet's no-auth aesthetic elsewhere.
  6. Derived-data backup: photolib.db + proxies are re-derivable but expensive (~days); include them in fleet-backup.
  7. The catalog copy is correct-by-construction, not hopeful. A live .lrcat is an open SQLite database; naive robocopy risks torn reads. The nightly job on the Windows PC uses sqlite3 .backup (or VSS snapshot, or runs only when Lightroom is closed — Phase 0 picks per your usage pattern), ships the snapshot + a PRAGMA integrity_check result, and Mia ingests only on a passing check with an atomic swap of photolib.db. And to be precise: there is no incremental "delta" mechanism for a copied SQLite file — each ingest is a full re-read of the relevant tables reconciled against the index (minutes at this scale; fine, and said plainly).
  8. Windows→POSIX path mapping + coverage audit. The catalog stores Windows absolute paths (D:\Photos\…); Mia sees /Volumes/…. Phase 0 builds the root-folder mapping table and runs a resolution audit: what % of 450k entries actually resolve to readable NAS files (every big catalog has offline drives and dead references). That coverage number gates proxy and embedding plans.
  9. The TV story, inside the privacy perimeter. "Any screen" must not break Tailscale-only. Resolution: the web player runs on YOUR devices (phone/trifold/laptop on the tailnet) and reaches the TV by casting/AirPlay/HDMI from that device — the TV never talks to Photolib directly. For a future ambient-display mode, a dedicated LAN-only render box (or a rendered MP4 on a USB stick) — never a public endpoint.

7. Alternatives considered (and why not as the core)

8. Open questions for you (Phase 0 inputs)

  1. NAS make/protocol (Synology/SMB?) and whether Mia may hold ~80GB of proxies locally or should keep them on-NAS.
  2. Excire product + version — Foto standalone or Search plugin inside LR? (Determines whether keywords are already in the catalog.)
  3. Any LR People/face-naming done, or Excire faces only?
  4. Music library location + formats.
  5. Comfort with a one-time overnight "Store Metadata (sidecar)" run on the Windows PC.

9. Self-critique — the Fable adversarial pass (what it caught, verbatim themes)

Draft 1 of this proposal was run through Claude Fable 5 as a hostile principal-engineer review. It found real errors, all now fixed above; preserved here so you can see the delta:

Next step on your word: Phase 0 discovery (needs the .lrcat copied to NAS/Mia + NAS read-only credentials), after which I return with measured facts, the visual mockups/wireframes, and the MCP tool schema.