Indexing a year of video locally on a 2021 MacBook with Gemma4-31B (50GB swap)

NJ, who splits his time between managing a safari lodge in Maasai Mara and software development in Silicon Valley, found himself with an ever-growing, unindexed archive of video footage. Despite his proficiency with AI tools in his professional life, his personal video content remained unmanageable. This led him to tackle the problem of video indexing head-on, aiming to create a searchable, local archive.

• Initially, NJ considered commercial AI video editors but found them unsuitable due to concerns about generative AI compromising brand authenticity and their failure to address the fundamental issue of an unlabeled archive. He identified that the crucial first step was to build a comprehensive index.
• He designed a local-first indexing pipeline, favoring plain-text sidecar files next to each video clip over a central database. This ensures data portability and resilience.
• The pipeline integrates various tools: ffprobe for metadata, exiftool for GPS, Nominatim for reverse-geocoding, ffmpeg for frame extraction, WhisperX for transcription and diarization, and insightface for facial recognition.
• For the core vision processing, he used a multi-tiered approach: Claude CLI for zero marginal cost, Anthropic API for speed, and critically, a local Gemma 4 31B Q4 model running via LM Studio for bulk processing.
• The vision model processes extracted frames, transcripts, and folder context to generate YAML frontmatter and a prose description, capturing extensive details like lighting, time of day, color palette, people count, and potential use cases.
• A major revelation was the performance of his 2021 M1 Max MacBook Pro. Despite being "five years old" by the story's internal timestamp, it ran the 31B Gemma model and handled the indexing, utilizing over 50GB of swap memory, highlighting the unexpected longevity and capability of Apple Silicon.
• NJ shared valuable lessons from overcoming four significant bugs, emphasizing defensive coding for fast-moving AI libraries, vigilance against silent API failures, strict schema design over union types, and careful consideration of culling criteria for video memories versus professional portfolios.
• He concluded that structured prompts with enum constraints are vital for preventing model confabulation, local 31B models can largely match cloud services for bulk tasks, and the true "missing layer" in current AI video editors is a robust, underlying index.

NJ is now proceeding to build the editing layer, leveraging the newly created index with Claude Code and DaVinci Resolve. He views this as a "tooling problem" that he can solve, turning a long-standing personal frustration into an opportunity to apply his technical skills. He also opens the door to sharing his "skill" (code) for public use and collaboration.