An Obsidian AI video ingestion engine is a config-driven pipeline that takes videos from a cloud folder, transcribes them with Whisper, reads key frames with a vision model, classifies each video against your own taxonomy, and writes a rich markdown note into your Obsidian vault plus a row in Postgres. Kev’s Obsidian Ingestion Engine, an open-source project by Kevin (KevBuildsApps), does exactly that from a single config file. The source is at https://github.com/kevinbadi/kevs-obsidian-ingestion-engine.
Dropbox folder
|
v
[1] download video
|
v
[2] Whisper (local or OpenAI API) -----> timestamped transcript
|
v
[3] OpenCV scene detection -----------> key frames
|
v
[4] Claude vision on frames ----------> what is on screen
|
v
[5] taxonomy classification ----------> type, topics, people
|
+----> Obsidian vault: markdown note + thumbnail
|
+----> Postgres: indexed rows for search
|
v
[6] auto-link people, topics, locationsWhat is an Obsidian AI video ingestion engine?
It is the difference between footage you own and footage you can search. The engine treats every clip as a document: it reads the audio, looks at the frames, decides what the video is about using rules you wrote, and files it where you can find it later.
Three pieces make it work:
- A source folder. Dropbox is the default. Google Drive support is optional per the README.
- A taxonomy file. Your content types, topics, people and locations. The classifier is driven by this, not by hardcoded categories.
- Two outputs. A note in your Obsidian vault and a row in Postgres. The note is for reading. The database is for querying.
The README describes it plainly: point it at a Dropbox folder, define your taxonomy, and it downloads, transcribes, extracts frames, analyzes, classifies, writes notes, indexes and auto-links. That is the whole loop.
This is Kevin’s own project, and it pairs naturally with the rest of his open-source work. If you like this kind of tooling, the HyperEdit post on an AI video editor built with ffmpeg and Remotion covers the rendering side, and the KevBuildsApps YouTube channel has the walkthroughs.
How the pipeline works: Dropbox to Whisper to Claude to Obsidian
Each stage is separate, so you can swap one out without rewriting the rest. The engine runs the steps in a fixed order per video.
- Download. The engine pulls the video from the Dropbox folder you named in the config.
- Transcribe. Whisper runs locally or through the OpenAI API, depending on your setting. You get a timestamped transcript.
- Extract frames. OpenCV scene detection picks the moments where the picture changes, so you are not analyzing near-identical frames.
- Analyze frames. Claude vision looks at the extracted frames and describes what is on screen. This is what tells the engine whether the video is a talking head, a screen recording, b-roll or a location shot.
- Classify. Claude applies your taxonomy to the transcript plus the frame analysis and returns content type, topics, people, location, tone, quality and a confidence score.
- Write the note. A markdown file lands in your Obsidian vault with frontmatter, a title, a summary, a thumbnail, quotes, clip suggestions and the full transcript.
- Index. The same record goes into Postgres.
- Auto-link. People, topics and locations found in the video get linked to existing entities, which is how the graph builds itself over time.
Because step 5 is driven by your taxonomy, a folder of client interviews, a folder of product demos and a folder of travel footage can each produce different metadata without touching the code. You change a file, not a function.
What you get in each Obsidian note (frontmatter, thumbnail, quotes, clips)
Every video produces one note designed to be read in ten seconds and searched in one.
| Part of the note | What it contains |
|---|---|
| YAML frontmatter | Content type, topics, people, location, tone, quality, confidence |
| Title and summary | Auto-generated title plus a one-line summary of the video |
| Thumbnail | An embedded thumbnail image from the key frames |
| Key quotes | The strongest lines pulled from the transcript |
| Suggested clips | Timestamps for moments worth cutting |
| Transcript | The full timestamped transcript |
The suggested clips field is the practical one. If you are building shorts, you open the note, read the timestamps and go straight to the good part instead of scrubbing an hour of footage. That pairs well with the workflow in Fireship style video editing with AI, where the cut is the whole game.
How the config-driven taxonomy classification works
The taxonomy is the brain of the system, and it lives in configuration rather than code. You define the buckets. The classifier sorts each video into them.
# shape of the idea, not a literal file
content_types:
- interview
- tutorial
- vlog
- product_demo
topics:
- publishing
- video_editing
- ai_tools
people:
- Kevin
- guest_name
locations:
- home_studio
- conferenceWhen a video arrives, the classifier reads the transcript and the frame analysis, then returns the best match for each field along with a confidence value. Low confidence is your signal to review the note yourself. The quality and tone fields give you a second axis: a shaky handheld clip and a studio interview both get described, not just tagged.
Start small. Ten topics and five people will classify accurately. A taxonomy with two hundred entries will produce noise until you have enough videos to calibrate it. Grow the config as your library grows.
How to set it up: Dropbox folder, taxonomy, Whisper, Claude, Postgres
The order matters less than the completeness of each piece. Get all five configured before the first run.
- Dropbox folder. Create a dedicated folder for source footage and connect it. Keep unrelated files out of it so the engine is not transcribing noise.
- Taxonomy. Write your content types, topics, people and locations. Keep it short for the first pass.
- Whisper. Choose local or the OpenAI API. Local means no per-minute transcription bill but more CPU or GPU time. The API is faster to start.
- Claude. Supply your model credentials for frame analysis and classification.
- Postgres. Point the engine at a database. This is the index it queries later.
Then run it against a small test folder of three or four short videos and read the notes. If the frontmatter is wrong, fix the taxonomy before you process the archive. Reprocessing an entire library because your topics were vague is the expensive mistake.
The README is the source of truth for exact configuration keys and install steps. Check it at https://github.com/kevinbadi/kevs-obsidian-ingestion-engine before you build your own config.
Why Postgres plus Obsidian beats notes alone for search
Obsidian is excellent at reading and linking. It is not a query engine. Postgres answers questions Obsidian cannot.
With markdown only, “find every video where the confidence is below 0.6” means opening files or trusting a search plugin. With Postgres, it is one query. The same goes for “all interviews from last quarter with a person named in the frontmatter,” or “every clip suggestion tagged as a product demo.”
The two outputs serve different jobs:
- Obsidian is where you read, follow links and write.
- Postgres is where you filter, count and join across the whole library.
You do not have to choose. The engine writes both from the same record, so they never drift apart. If you later build a script or a dashboard on top, you query the database and link back to the note by name.
How auto-linking builds a knowledge graph of people, topics, and locations
The graph is a side effect of doing classification properly. Each video returns people, topics and locations. The engine links those to existing entries instead of creating a new one every time.
The payoff shows up after a few dozen videos. Open a person’s entry and you see every video they appear in. Open a topic and you get a chronological thread of everything you have said about it. Open a location and you see the sessions shot there. That is a graph, and you did not draw any of it by hand.
This is the same reason structured metadata beats free-form tags in publishing. If you want the other half of the workflow, scheduling the clips you cut from these notes, see how to schedule AI video to TikTok from one dashboard.
Can you run Whisper and Claude vision locally or on a budget?
Whisper has a clear local option, so transcription can run on your own machine with no API cost. Frame analysis through Claude vision is an API call, and OpenCV scene detection keeps that cost down because it only sends frames where the scene actually changes.
Practical ways to control spend:
- Run Whisper locally and reserve paid calls for vision and classification.
- Process a test folder first, then the archive, so you do not pay twice.
- Keep scene detection tight so a static talking head sends a handful of frames instead of hundreds.
I will not quote a dollar figure for your library, because it depends on video length, frame count and model choice. Estimate from your own test run: process ten videos, read the usage, and multiply.
For the broader picture of running models on your own hardware, read AI video generation locally with coding agents. If you want to see these tools being built, the launch video is a good place to start.
FAQ
How do I ingest videos from Dropbox into Obsidian automatically?
Point Kev’s Obsidian Ingestion Engine at a Dropbox folder and give it a taxonomy and an Obsidian vault path. It downloads each video, processes it through transcription, frame analysis and classification, then writes a markdown note into the vault. You run it on a schedule or on demand, and the notes appear without manual work.
What does an AI video ingestion engine do with Whisper and Claude?
Whisper turns the audio into a timestamped transcript. Claude reads that transcript plus descriptions of key frames and classifies the video against your taxonomy. Together they produce the frontmatter, summary, quotes and clip suggestions that end up in the note.
Can I use this Obsidian ingestion engine with Google Drive instead of Dropbox?
The README says Google Drive support is optional, while Dropbox is the documented path. Treat Google Drive as something to verify against the source before you depend on it. Check the repo at github.com/kevinbadi/kevs-obsidian-ingestion-engine for the current state.
Is Claude vision required, or can I use another model for frame analysis?
The documented pipeline uses Claude vision for frame analysis and classification. The stages are separate, so the vision step is a distinct piece, but swapping in a different model is your change to make and test. Start with the documented setup and check the README for configuration options.
How much does it cost to run Whisper and Claude on a folder of videos?
There is no fixed number, because it depends on video length, how many frames scene detection extracts and which model you choose. Run a small test folder first, read your usage, then extrapolate before processing the archive. Running Whisper locally removes the transcription cost entirely.
Get started with Creator OS
Ingestion solves the search problem. Publishing solves the distribution problem, and that is what Creator OS is for: one place to publish, schedule, reply and read analytics on 11 platforms plus Skool, with a REST API, a CLI and a hosted MCP server so an agent can run the accounts.
Creator OS plans start at $14.99/month for 11 platforms plus Skool, with the API, the MCP server and the CLI included. Start with a free trial at https://www.creatoros.ca/sign-up.
Then clone Kev’s engine from https://github.com/kevinbadi/kevs-obsidian-ingestion-engine, process a test folder, and watch your footage become something you can actually find. For the publishing side, the CLI docs at https://www.creatoros.ca/docs/cli show the commands, and the KevBuildsApps channel has the builds.
Get started for $0
Free trial for new subscribers. Cancel anytime.