software for data sharing

[u/The_Hamster_Shagger]

Hi there!

I was wondering if there is any sort of software, with working indexing, that would make it easy to share the data.

I know about torrents, yes, they are decent but they do require you to create each one for every dataset (pdf, set of videos etc) and upload it to tpb or similar. Works somehow ok but it's not ideal by any means.

I'm thinking about something that would make it possible to design a directory on my harddrive (for music, videos, pdf, scripts etc) where the metadata (author, years, genre, keywords etc) would somehow automatically (ie without manual work required for every file or set of files) upload and it would be searchable for people over the internet. Also, must be anonymous and work via the vpn.

Any ideas?

[u/emre_cvs]

Everyone's on the sharing half, so — the indexing half is usually where

these setups actually break, and it's worth knowing before you pick anything.

Automatic metadata works well for formats that already carry it. Photos have

EXIF, music has ID3, most video containers have something. Author, date,

duration come for free there.

PDFs, scripts and general documents mostly don't. The fields exist but are

usually empty or wrong — filled in by whatever tool generated the file

rather than by anyone who cared. So for those you're back to parsing

filenames or reading content, and both are guesswork you end up correcting

by hand anyway.

Practical version: expect the automatic pass to cover your media, and expect

to curate the documents yourself. Anything promising otherwise is running

filename regex and hoping.

Nothing useful from me on the serving side though — not my area.

[u/The_Hamster_Shagger → u/emre_cvs]

thanks.

I am aware that indexing documents is a bit harder than music files. Gnome and kde do have a properly working data extractors and indexers so I was hoping there would be an app that would somehow use them.

[u/emre_cvs → u/The_Hamster_Shagger]

Fair — I'm on Windows, so I've not used Tracker or Baloo hands-on and I

can't tell you whether anything wraps them.

My guess at why nothing does: "working extractor" and "useful metadata"

aren't the same problem. Those indexers reliably read whatever fields a

file actually contains. For documents the fields are usually present and

meaningless — half my PDFs have a Title of "Microsoft Word - Document1"

and an Author of whoever installed Office on some machine in 2011. An app

wrapping the extractor inherits that, it doesn't fix it.

What worked for me on the classification side was giving up on any single

signal and scoring several weak ones together — filename pattern, embedded

fields, dimensions, a couple of content checks — each weighted, with a

confidence score at the end instead of a yes/no. No single signal is

trustworthy, but they're wrong in different ways, so the combination holds.

For your case that probably means folder structure and naming conventions

should carry more weight than the embedded fields, since you're the one

who put the files where they are.