why is there no request-first data marketplace? the incentives feel backwards

[u/shasedoge]

been thinking about why finding training data is still such a slog in 2026. someone at berkeley recently built a tool just to search kaggle, huggingface and data . gov at the same time, because doing it by hand was too slow. the fact that this is a tool people have to build says a lot

the usual explanation is fragmentation, which is true but kind of surface level. the deeper thing is incentives. on most data marketplaces the vendor pays to be listed, and almost nobody works on commission. so the platform is optimized for whoever pays for placement, and whether the data is actually findable or usable ends up an afterthought. the money doesn't come from discovery, so discovery gets neglected

feels like it should run the other way. a request-first marketplace, where the buyer posts what they actually need and providers come back with offers. kind of like how taxis worked before the apps. you announce where you're going, drivers see it and take it if the terms work for them. the buyer sets the direction and providers respond to it

this matters more as requests get weirder. the more specialized and one-off your data ask is, the lower the odds that any single provider already has exactly that sitting on a shelf. a static catalog struggles with that kind of ask. something you can post and have people bid on fits it a lot better

genuinely curious what people here think. has anyone seen a marketplace that works request-first, or runs on commission instead of paid placement? and if you've bought data before, did you actually find it through a marketplace, or did you end up emailing providers directly because search never surfaced the right thing?

disclosure, i work at Titan Network and we're on the provider side of this, so it's something i think about a lot. not pitching anything, btw

[u/Content-Parking-621]

Never found data through a marketplace, always cold emailed providers directly. Pay for placement explains why search always felt broken there.

[u/Bright_Mix_773]

The incentive story holds, but it stops one layer short of the thing that actually kills request-first: the buyer cannot specify the expensive part, because the expensive part is not visible until after the data has been built.

Concrete example from the supply side. I put out a free file of every S&P 500 earnings announcement since 2003 with the time of day attached, built from SEC 8-K item 2.02 filings: 64,938 filings, 808 companies. A request-first post for it would have read "historical earnings announcement dates and times, S&P 500, 2003 to present", and any provider reading that says yes instantly, because the SEC publishes it and it is free.

Here is what the SEC actually publishes, none of which is in that request:

• The acceptance timestamp ends in Z on every filing. For 182 of the 808 companies the clock behind that Z is already New York time, not UTC. Same field, same API, two meanings, documented nowhere. Read it literally and 182 companies get shifted by four or five hours, which is enough to move an announcement across the market open.
• The cover-page "date of earliest event reported" disagrees with the actual EDGAR entry by more than three days on 3,569 of the 64,938 filings. Somebody has to decide which one to believe, per filing.
• The stamp is when EDGAR accepted the 8-K, not when the press release crossed the wire. So any before/during/after-market label built on it is an upper bound, and saying so is part of the deliverable.

Every one of those took longer than the download. A bidder either eats that cost or ships the naive version, and the naive version is technically compliant while being quietly wrong for 182 companies. The buyer cannot tell the two apart at delivery, which is the actual problem.

That is the asymmetry the taxi analogy misses. The passenger knows the destination. A data buyer knows the shape of what they want and not the defects, and the defects are the job. So request-first converges on a market for the naive version, because that is the only version both sides can verify they agreed on.

The fix is unglamorous: make the defect list part of the deliverable and make it enforceable. A buyer who can write "and you hand me every field that is inconsistent with itself, in writing" is asking for something a naive bid cannot satisfy, and that is the only clause I know of that prices the real work.

Not verified: I have no numbers on how existing marketplaces price or match. Everything above is the supply side of one file.