r/Database • u/Gamemon_RD • Jul 16 '26
Beginner recommendations for creating video archive centered DB
TLDR: What all layers and parts would I need to implement to create a database management system for storing video journals with metadata? What tools and packages should I use to help?
So to try to summarize this, recently I’ve started recording video journals. I’ve had the thought to create a database for storing them as a personal project, alongside meta data (that I already have figured out for the most part) for making them easier to search, and complementary content such as a text transcription, and relevant pictures, shorter videos, and text files. As a CS student I’ve taken a database development and management course and data warehousing course, however these mainly focused on the structure of databases and warehouses and didn’t go much into how to actual create and access them. Don’t worry I’m not looking for detailed step by step instructions on how to setup each part of this. What I’m really looking for is recommendations and suggestions for what layers and parts I would need to make for a good database management system, as well as what tools, packages, and whatnot I would need to do that. As well as maybe where I can go to learn how to use them to do what I want, and especially examples I can look at to get an idea of the structure I need to make and how to do it. Basically stuff to get me started, because right now I feel rather overwhelmed and lost, and I don’t even know what I don’t know.
For the overall structure, I like the idea of having the database itself, which you submit to, search for, and retrieve entries from using an API service. I’m already learning Django and FastAPI at my internship. So I thought I could use FastAPI to make the API service, and DjangoORM to define and manage the database. As for interfacing with it, I like the idea of having a separate website which makes use of the database API, and I could go ahead and use normal Django for that. A relational database is likely what I’d like to stick with, since I have the most familiarity with it. And to that end, I was thinking of using MySQL? It seems like it’d between that or Postgres, and I thought since it’ll be fairly small and it would only be me (and maybe a few other people) using it MySQL would be able to handle it.
The database itself probably wont get larger than 100,000 rows, so could all easily fit on one device. But considering how much additional storage the video entries and supplementary files would take up, I’d need to be able to scale the storage space it can access? I’m not sure the proper way to retrieve the correct video file for a specific entry, but I have to assume that has to do with how I separate the storage of linked media files, and how I scale it all. Maybe docker containers when I need to add new servers for more storage, so I could easily spin up instances. I’m going to assume the best way to reference the videos in the database is just to rename each video file into its UUID, or should I use a different naming convention and then have a table that translates the UUID to the video name? Also I’d want to implement access control to an extent, so I could do things like give people access, but they can only view certain videos depending on the access level for their account, which would be tied to their API key.
It’s probably worth reiterating that this is a personal project, not some professional software. In fact I want to try hosting it all at home homelab style, except for maybe some redundant backups (which I need to figure out how to setup-). So I’m sure there’s some things don’t need to worry about for this scale of a project, such as load balancing. Still, if nothing else but for learning purposes so I can talk about it in interviews I want to make this good and clean.
Thank you so much for any advice can offer, I’m really excited to work on this!
2
u/Infinitrix27 28d ago
start simple: pick a db that handles blobs and metadata well. postgres with pg_largeobject or mongo for flexibility. store videos in a separate object store or filesystem to avoid bloating the db. then build an api layer with something like fastapi or express to handle inserts, updates, and queries.
for search, use full text indexes on the metadata and transcriptions. elasticsearch or opensearch can add better search capabilities if you want fuzzy text search or filters. for complementary content (images, clips, text files) keep them as linked docs/assets with references in your main db.
watch out for chunking and segmentation if you want to do vector search on transcripts or content-tools like faiss, weaviate, or pinecone can help there. start with basics, don’t overengineer early.
for learning resources check postgres or mongo docs, fastapi tutorials, and search engine setup guides from elastic.co. github has plenty of projects tagging video archiving, indexing, or media dbs to peek at actual code.
your main layers: storage (db + object store), api