PHOTOLIB — Proposal: an agent-native photo intelligence layer over 450k images

For: Byron · Status: PROPOSAL ONLY — no build, no mockups yet, per instruction

Sources of truth: Lightroom Classic catalog (.lrcat, Windows PC) · Excire AI metadata · originals on NAS (read-only)

---

Executive summary

Problem. 450k photos in a Lightroom Classic catalog that's slow to work in, hard to query, and invisible to your agent fleet — despite years of invested tagging (LR keywords) and a full AI pass (Excire) already sitting in databases you own.

Proposal. Don't re-index — harvest. Build Photolib: a lightweight metadata index (single SQLite file, ~500MB, millisecond search) extracted directly from the .lrcat and Excire's data, living on the Mac Mini, fronted by an MCP server on the tailnet so you — and any fleet agent — can query, curate, and compose. Browsing and slideshows run off ~80GB of one-time proxies, never touching NAS originals (read-only, always). A slideshow engine with JSON template specs and a single WebGL renderer (instant playback on your devices; the same engine headless-rendered to MP4) targets ProShow-class output: Ken Burns, transition templates, your music with beat-synced cuts. Semantic "describe it" search is a deliberately-gated Phase 3 — a small SigLIP embedding model, hours of compute, no LLM, only if the harvested keywords prove insufficient.

Cost & risk honestly. The metadata index is small and fast — that part is easy. The two real costs are one-time: proxy generation (days of throttled background I/O, or shortcut via Lightroom's own preview pyramid) and the Excire harvest (its data is siloed; prefer parsing its DB read-only, fall back to a carefully-controlled export). Everything is read-only against your originals and catalog; the whole thing is re-derivable; nothing gets a public endpoint.

Decision sought. Approve Phase 0 only — 1–2 days of measurement (catalog inventory, sample harvest, the aurora acceptance query, NAS throughput, path-resolution audit). It converts every assumption in this document into a measured fact, and you get mockups + the MCP schema with the results. Five inputs needed from you (§8).

Target-state architecture

Photolib target-state architecture — full diagram (SVG)

---

1. The core insight: you already paid for the index

450,000 photos sounds like an indexing mountain. It isn't — because two engines already climbed it:

• Lightroom Classic's catalog is a plain SQLite database (~116 tables). Your keywords (AgLibraryKeyword / AgLibraryKeywordImage), captions, ratings, picks, GPS, capture times, camera data, folder structure, collections, and a full XMP blob per image (Adobe_AdditionalMetadata) are all directly queryable — no Adobe APIs, no export, no Lightroom running. Community tooling (lrcat-extractor and others) has mapped the schema.
• Excire has already AI-tagged the library — but its results live in Excire's private database (%APPDATA%\Excire Foto), NOT in your files. Excire's sanctioned bridge is its "Store Metadata" command, which writes its AI keywords into XMP (embedded or sidecar). One one-time export run and that intelligence becomes portable forever. (If you used the Excire Search plugin inside LR and saved keywords back, much of it may already be in the catalog — Phase 0 verifies which.)

So the strategy is: harvest, don't re-index. Build a lightweight, agent-native index from what exists; add new AI only incrementally, only where the harvest has gaps.

2. Recommended architecture ("Photolib")

Windows PC                      Mac Mini (Mia)                     Anywhere
┌─────────────┐   nightly copy  ┌──────────────────────────┐
│ .lrcat      │ ───────────────▶│ INGEST  lrcat + XMP      │
│ Excire DB   │  (Zia/robocopy  │   ↓                      │
│  → XMP once │   to NAS share) │ PHOTOLIB.db (SQLite)     │   MCP over Tailscale
└─────────────┘                 │  images/keywords FTS5    │◀── OpenClaw fleet
      NAS (read-only SMB) ─────▶│  faces·gps·ratings       │◀── Claude (me)
      │ originals (never write) │  vectors (sqlite-vec)*   │
      │ music library           │ PROXIES 1600px JPEGs     │
      └────────────────────────▶│ SLIDESHOW engine         │──▶ web player (any screen)
                                └──────────────────────────┘──▶ ffmpeg MP4 render
                                                     * Phase 3, optional

Photolib.db — one SQLite file with FTS5 full-text over keywords/captions/paths + structured columns (time, GPS, rating, faces-count, camera). 450k rows ≈ 300–600MB. Query latency: single-digit milliseconds. This is the whole "database size" fear dissolved: metadata at this scale is small; it's pixels that are big, and we never move them.

Proxies — browsing/slideshows never touch NAS RAWs. This is the true long pole, stated honestly: 450k RAWs × ~25MB ≈ 11TB of NAS reads; at a sustained 100–200MB/s that is 15–30 hours of pure I/O plus RAW decode at 0.5–2s/image ≈ 2.5–10 days of throttled background compute — a multi-day pipeline with a resume queue and failure reporting (at 450k, a 1% failure rate = 4,500 files someone must account for: exotic RAWs, CMYK TIFFs, broken EXIF, orientation chaos). Two mitigations change the game and get measured in Phase 0: (a) harvest Lightroom's Previews.lrdata pyramid instead — already-rendered JPEGs incl. your edits, no RAW decode at all; coverage/size audited in Phase 0; (b) LR batch-export on the Windows PC (renders with your edits applied, uses the PC's already-local disk bandwidth). Output ≈ 80GB of 1600px JPEGs either way. Proxies key on LR's id_global UUID (stable across renames/moves), with edit-invalidation (touch-time check) so re-edited photos re-proxy.

MCP server (photolib-mcp on Mia, Tailscale-only) — the fleet hook you asked for. Tools:

• search(query) — structured DSL and natural language ("northern lights, at least one person, rating ≥3, 2019-2025") compiled to SQL
• sample(query, n, strategy) — curated picks for agents (spread across time/places)
• photo(id) / preview(id, size) — metadata + proxy bytes
• collections — create/list/append virtual sets
• slideshow(spec) — build + return player URL or rendered MP4
• stats() / keywords() / faces() — corpus introspection

Any OpenClaw agent — Gia, me, Command — can then direct photos: "pull 40 best aurora shots with people, make a 3-minute reel to this track."

Your demo query — keyword ∈ {aurora, northern lights} AND faces ≥ 1 — is the Phase 0 acceptance test, not a promise: it works if Excire's fixed taxonomy emits an aurora concept and its face counts survive harvest as queryable fields (they may export as "one person"/"group" keywords rather than numbers). Phase 0 runs this exact query against a 500-photo sample harvest before anything else is built. If Excire lacks the concept, the query lands in Phase 3 (semantic) instead — still no LLM needed.

3. Semantic search (the careful, optional layer)

You're right to distrust Gemma for this — but the right tool isn't an LLM at all. CLIP-family embedding models (SigLIP-base ONNX) are the standard for photo semantic search: ~400MB, and they enable "moody beach sunset with silhouettes" queries beyond keywords. Honest numbers (to be measured in Phase 0 on Mia, not assumed): at an unverified 40–80 img/s from local proxies, 450k ≈ 2–4 hours of embedding compute — the gating dependency is that proxies must exist first (see the proxy long-pole above). Storage: 450k × 768-dim fp16 ≈ 700MB in sqlite-vec (fp32 would be 1.4GB). Query honesty: brute-force vector search over 450k is ~100–300ms per query, not the millisecond class of the FTS index — fine for interactive use, stated so it's never a surprise. Phase 3, only if the keyword harvest leaves you wanting — my bet is Excire's tags cover 80% of real queries.

4. Slideshow engine (the ProShow bar)

ProShow Producer's essence: timeline of slides → layered motion (Ken Burns pan/zoom), transition library, music-driven timing, render to video. Plan:

• Two output paths, ONE rendering engine (decided now, per critique): the WebGL player is canonical, and video export headless-renders the same WebGL engine to frames piped into ffmpeg — identical motion/easing/color in both, and it sidesteps ffmpeg's notoriously steppy zoompan Ken Burns. (a) instant web player — crossfade/push/dissolve/parallax-zoom/dip-to-black, Ken Burns with face-aware framing where face data exists; (b) H.265 MP4 export via the same engine.
• Music: from your library on the NAS (picker via MCP), or generated on the fly by the fleet's audio pipeline. Beat-sync: onset/tempo detection (librosa) maps slide cuts and transition timing to the actual track — the single feature that makes slideshows feel produced rather than played.
• Transition templates as JSON specs — {sections:[{query|photos, duration, transition, motion, textOverlay?}], audio:{src|generate, sync:"beats|bars|off"}}. We define 3–4 signature templates together (e.g. Documentary slow-cross, Energetic beat-cut, Cinematic letterboxed parallax) and agents compose from them. Because it's a spec, "Gia, aurora reel, template Cinematic, use track X" is one MCP call.

5. Phases

Estimates are the critique-doubled honest versions; Phase 0's measurements may pull them back down.

6. Things you didn't ask about (but will hit)

1. The Excire harvest is the most dangerous step and gets a real protocol. Preference order: (a) parse Excire's own SQLite DB directly (%APPDATA%\Excire Foto) — read-only, zero writes anywhere, schema reverse-engineered in Phase 0 on a copy; (b) if unparseable, "Store Metadata" sidecar export to a SEPARATE write location (not the originals share — preserving the read-only invariant), noting that for JPEG/TIFF/HEIC Excire may embed (rewriting originals — if unavoidable: checksum-before/after protocol on a sample, then explicit go/no-go from you); and the Lightroom side-effect must be planned either way: LR will flag "metadata changed on disk" on affected images, so we script the harvest to avoid touching files LR watches, or accept and document the conflict-badge wave. This was a parenthesis in draft 1; the critique correctly promoted it to a first-class risk.
2. Faces ≠ names. Excire counts faces; LR People view names them only if you've done that work. Phase 0 tells us whether "photos with Sophia" is queryable or only "photos with ≥1 person." Named-face search may become a Phase 3+ item (local face clustering) — flagging now to set expectations.
3. Virtual copies, stacks, videos, smart previews — the catalog contains all of these; the ingester must handle them (dedupe virtual copies to master image, include/exclude video toggle).
4. Write-back is a trap. Everything here is read-only by design — Photolib never writes to NAS originals or the catalog. Ratings/tags created by agents live in Photolib's own layer. If you ever want write-back into LR, that's a deliberate later decision (keyword import via LR plugin), not a default.
5. Privacy perimeter. This index includes your family's faces and locations. MCP binds to Tailscale only — never a public endpoint, no exceptions, regardless of the fleet's no-auth aesthetic elsewhere.
6. Derived-data backup: photolib.db + proxies are re-derivable but expensive (~days); include them in fleet-backup.
7. The catalog copy is correct-by-construction, not hopeful. A live .lrcat is an open SQLite database; naive robocopy risks torn reads. The nightly job on the Windows PC uses sqlite3 .backup (or VSS snapshot, or runs only when Lightroom is closed — Phase 0 picks per your usage pattern), ships the snapshot + a PRAGMA integrity_check result, and Mia ingests only on a passing check with an atomic swap of photolib.db. And to be precise: there is no incremental "delta" mechanism for a copied SQLite file — each ingest is a full re-read of the relevant tables reconciled against the index (minutes at this scale; fine, and said plainly).
8. Windows→POSIX path mapping + coverage audit. The catalog stores Windows absolute paths (D:\Photos\…); Mia sees /Volumes/…. Phase 0 builds the root-folder mapping table and runs a resolution audit: what % of 450k entries actually resolve to readable NAS files (every big catalog has offline drives and dead references). That coverage number gates proxy and embedding plans.
9. The TV story, inside the privacy perimeter. "Any screen" must not break Tailscale-only. Resolution: the web player runs on YOUR devices (phone/trifold/laptop on the tailnet) and reaches the TV by casting/AirPlay/HDMI from that device — the TV never talks to Photolib directly. For a future ambient-display mode, a dedicated LAN-only render box (or a rendered MP4 on a USB stick) — never a public endpoint.

7. Alternatives considered (and why not as the core)

• Immich (self-hosted Google-Photos-alike): lovely UI, has ML + external read-only libraries — but documented web-client struggles at 65k+ photos (450k = 7× that), its ML would re-do what Excire already did, read-only libraries have trash/edit quirks, and its API wasn't designed for agent slideshow composition. Could be added later as a human-browsing UI beside Photolib; wrong foundation.
• PhotoPrism: similar shape, similar concerns at scale, Go/TensorFlow stack heavier to bend to MCP.
• Scripting inside Lightroom (plugin/SDK): keeps Adobe in the loop for every query — the slowness you're escaping.
• Full cloud re-index (Google/AWS Rekognition): $$$ at 450k, privacy hostile, unnecessary given Excire.

8. Open questions for you (Phase 0 inputs)

1. NAS make/protocol (Synology/SMB?) and whether Mia may hold ~80GB of proxies locally or should keep them on-NAS.
2. Excire product + version — Foto standalone or Search plugin inside LR? (Determines whether keywords are already in the catalog.)
3. Any LR People/face-naming done, or Excire faces only?
4. Music library location + formats.
5. Comfort with a one-time overnight "Store Metadata (sidecar)" run on the Windows PC.

8b. Access recon — what I can reach, what I need (read-only)

I ran a live, read-only reconnaissance from Mia. Findings, then the precise ask.

✅ Reachable & ready now

• Compute (Mia). SQLite 3.51, Python 3.14, ffmpeg 8.1, Node 25 — present. 1.4 TB free on Mia, so photolib.db (~500MB) and ~80GB of proxies fit locally, no NAS-side storage required. (Two small installs needed for the pipeline: libvips/pyvips for fast proxy generation and exiftool for XMP parsing — I can add these; they're read-only tools.)
• The two machines that hold everything are visible on the LAN from Mia:
• NAS — "sherpastor" (192.168.1.34, Synology, SMB/445 open). This is where originals + music presumably live.
• Windows PC / Zia — "DESKTOP-BCB4CHM" (192.168.1.201, SMB/445 open, WORKGROUP). This is where the .lrcat and the Excire database live.
• Fleet, Gitea, Vercel, GoDaddy — already reachable (I use them).

⛔ Blocked right now — the single gap is credentials

Both machines answer, but Mia holds no SMB credentials for either — guest/anonymous reads are refused (Authentication error), and nothing is stored in Mia's keychain or the fleet .env. So although you've enabled the network path, the authentication path isn't yet in Mia's hands. That's the whole blocker — everything else is ready.

🔑 What I need from you (each strictly read-only)

Give me one read-only SMB account (or open read-only shares) so Mia can mount these. Concretely, the files/paths I need to read:

Two equally-clean ways to grant this — your call:

• (A) Read-only SMB user. Create a DSM/Windows account with read-only rights to those shares and give me the username/password (I'll store it in Mia's keychain, mount every share with the :ro flag, and the account's own permissions enforce the invariant at the OS layer — belt-and-suspenders). This is my recommendation.
• (B) Zia pushes, Mia pulls. A scheduled task on the Windows PC copies a nightly .backup of the catalog + the Excire DB to a NAS share; Mia only ever reads that NAS folder. Zia never grants Mia direct access; the catalog is snapshotted safely (no torn reads). Best if you'd rather Mia never touch the PC directly.

Either way, Mia never gets write access to anything — that's designed in, and with (A) it's also enforced by the account's permissions.

One confirmation

You said you believe you've given bi-directional access — the honest finding is that the reachability is there but the credentials aren't yet on Mia. So the ask above isn't "open the network," it's "hand Mia a read-only key (and confirm the four exact paths)." The moment I have that, Phase 0 runs itself.

9. Self-critique — the Fable adversarial pass (what it caught, verbatim themes)

Draft 1 of this proposal was run through Claude Fable 5 as a hostile principal-engineer review. It found real errors, all now fixed above; preserved here so you can see the delta:

• "The long pole is proxy generation, not the Excire export — 'overnight' is fantasy: ~11TB of NAS reads + RAW decode ≈ 2.5–10 days." Correct; draft 1 was off by an order of magnitude. §2 now carries the honest math + the two mitigations (LR previews pyramid / PC-side batch export).
• "Your SigLIP numbers contradict themselves" (40–80 img/s ⇒ hours, not the claimed days; 768-d fp32 ≈ 1.4GB not 700MB; vector search is 100s of ms not single-digit). All corrected with the fp16 assumption stated; every remaining number is tagged as measure-in-Phase-0.
• "The Excire sidecar export violates your own read-only invariant" — sidecars land next to originals; JPEG/TIFF may get embedded (rewritten); LR then flags metadata conflicts on up to 450k images. Promoted from a parenthesis to the plan's #1 risk with a real protocol (direct Excire-DB parse preferred; separate-location sidecars; checksum gate on any embed path).
• "'Demo works day one' is unverified" — Excire's taxonomy may not contain "aurora"; face counts may not survive as numbers. Downgraded to the Phase 0 acceptance test it always should have been.
• "Hot-copying a live .lrcat risks torn reads" → sqlite .backup/VSS/LR-closed gate + integrity check + atomic swap, specified.
• "Windows→POSIX path mapping is entirely absent; no stable-ID strategy" → root mapping table, resolution-coverage audit, id_global keying + edit invalidation, added.
• "The web player breaks the Tailscale perimeter you just declared" → TV via casting from a tailnet device only; perimeter holds.
• "Two rendering engines will drift" → one engine: headless WebGL → ffmpeg.
• "Effort table ignores its own self-critique — double it, minimum." Done.
• What it endorsed: harvest-don't-reindex, read-only posture, metadata-small/pixels-big framing, SigLIP-over-LLM gated behind keyword insufficiency, JSON slideshow specs as the agent interface, Phase 0 as a measured gate.

---

Next step on your word: Phase 0 discovery (needs the .lrcat copied to NAS/Mia + NAS read-only credentials), after which I return with measured facts, the visual mockups/wireframes, and the MCP tool schema.