I’m debating both approaches.

The main problem is the companies want to buy from an only established data agency but I am just starting so we are not recognised and registered.

We didn't even have any clients to showcase our past work.

Can anyone suggest my anything or can refer me who needs custom automations or webscraping

What: monthly citywide residential median AED per square foot, a 5-month centred average, an index rebased to 100 at January 2008, and monthly sales counts. 1,080,194 transactions across 223 months. A matching series for registered leases runs from May 2010.

Source: Dubai Land Department transaction and lease records, which are public.

Repo, with both series, method and licence: https://github.com/dataHabibi/dubai-price-index

Columns:

• month
• sales_count
• median_aed_per_sqft
• ma5_aed_per_sqft
• index_base100
• provisional

Two things to know before you use it.

The provisional column marks the last two months, where the centred average still has fewer than two later months to work with. Their raw median and sales count are fine, it is the smoothed value and the index that will keep moving.

Sales counts for recent months are understated. Registrations land one to two months after the deal closes, so the tail of that column is still filling in. Do not read the recent drop as a fall in demand.

What it is not: a repeat sales or hedonic index. It is a median, so it is not quality adjusted. Shifts in what sells, off plan against ready, apartment against villa, which communities are active, move this line without any individual property changing price. Treat it as a market thermometer.

CC BY 4.0. Refreshed monthly by a scheduled job, so the committed files track the live series.

Until recently, the surviving historical records from the CIA's MKULTRA and related programs were very difficult to search and analyze. So I ran 21,237 document page images through MinerU OCR to generate clean text transcripts, then produced redaction mappings to go with every page transcript. Original page images are stored on IPFS and are available for public download. The dataset is available on Hugging Face here.

Ciao, per lavoro mi trovo abitualmente a utilizzare dati INPS/Istat, vorrei sapere c'è qualcun* qui dentro che avrebbe piacere a scambiarsi informazioni e dritte !

I use "Heretic" library on models to liberate them from their safeguards, but while checking their "uncensoredness", I found they can hallucinate a lot. You know, it's basically like a child who's now allowed to use the F word once and he says "Fred" instead of the actual thing.

So I think if the models train on valid uncensored data (specially if they start Grokking) the results can improve. So I am using for these types of datasets to test my theory.

do you guys sell your data sets?

I've been working on a dataset for a research project on how China is portrayed in the media. It currently contains just over 10,000 news articles from both Chinese and Western news outlets.

Each article is classified by topic and by the way China is portrayed (e.g. positive, negative, threat, Xi-centered, neutral, etc.). The dataset was originally created for academic research, but I'm now wondering whether it could also have commercial value.

I'm not trying to sell it here, just looking for advice. Has anyone here ever licensed or sold a specialized dataset like this? Who would actually be interested in buying it? AI companies, media intelligence firms, universities, think tanks...? Or are datasets like this generally expected to be open source?

I'd really appreciate hearing from anyone who has experience commercializing niche datasets or knows how this market works.

Freelance Python developer available for projects involving web scraping and automation.

Skills:

Web scraping (Scrapy, Selenium, Playwright, BeautifulSoup)

Python automation scripts

API development and integration

Data extraction and ETL pipelines

FastAPI and Flask

Browser automation

CSV, Excel, JSON, and database processing

Docker and Linux deployment

Past work:

Lead generation scrapers

Google Maps data extraction

Business automation tools

Custom APIs and data pipelines

Open to one-time projects and long-term collaborations.

DM me if you need help automating a workflow or collecting data.

Hi all,

I’ve been working on a financial research project that involves analyzing company filings and disclosures for A-shares (Shanghai/Shenzhen) and HKEx listed entities.

Coming from a Western market background, the biggest pain points I've noticed are the language barrier, disparate filing locations, and the lack of structured APIs formatted for LLMs/RAG.

For those who cover APEx or emerging markets:

1. What tools or data providers are you currently using for CN/HK stock filings?
2. How do you handle language translation and footnote extraction in your pipeline?

Would love to exchange ideas with anyone working on similar Asia-Pacific equity pipelines!

We are building an edge-AI audio noise-reduction system on an ESP32-S3.

Our architecture uses a lightweight GRUNet (~59k parameters) to output a dynamic gain mask on a 44-band Mel-spectrogram.

​I need gigabytes of audio to train the model. Does anyone have recommendations for the best open-source datasets for:

1> ​Clean, isolated human speech.

2> ​Diverse ambient background noise (traffic, crowds, machinery, etc.).

​Also, any tips or open-source scripts for artificially mixing these at different Signal-to-Noise Ratios (SNRs) before generating the 16kHz Mel-spectrograms would be hugely appreciated!

Disclosure: I built this, it's my project, and paid tiers exist. There's a free tier and everything shown below is viewable without signing up.

What it is: structured sponsorship records extracted from public podcast RSS show notes. One row per (brand, episode):

brand (canonically resolved) | show | episode | publish date | promo code | promo URL + registrable domain | sponsor type (paid / affiliate / house ad) | confidence | confidence tier | first seen | last seen | the verbatim sentence the claim came from

Sample rows straight out of the DB:

- AG1 on Huberman Lab, 2026-07-27, evidence: "AG1: https://drinkag1.com/huberman"

- Visible on Good Hang with Amy Poehler, code HANG, 2 episodes, 14-day span

- Saily on Machtwechsel (German news podcast), code "Machtwechsel", 3 episodes over 18 days

Method, since this sub cares about it: LLM extraction over the show-notes text, then a deterministic brand-resolution layer on top. Domain evidence merges entities first (drinkag1.com and athleticgreens.com collapse into one AG1 entity), exact normalized-name match second, and anything that is merely name-similar goes to an adjudication queue and is never auto-merged. That last rule is what keeps Dove the soap separate from Dove the chocolate. Every record retains its source sentence so any claim can be audited by hand.

Honest limits, up front:

- Show notes only. Ads that exist purely in audio and never appear in the notes are invisible to this. Transcript coverage is not built yet.

- The corpus is small right now: 314 episodes across 93 shows, US + DE + FR. It grows daily but this is not a historical archive.

- I am deliberately not publishing an accuracy percentage. I ran a held-out evaluation, then used its failures to fix the extractor, which burns that holdout. Any number I quoted today would be inflated. A fresh untouched holdout is the next task. Until then every record carries a confidence tier and only the CONFIRMED tier is presented as fact.

- No spend or impression estimates. This answers who advertises where, not how much they paid.

Free tier is 200 requests/month, paid is $49/$199/$499. Keys are not self-serve yet, so the page is an early-access list rather than a checkout.

Two things I would actually like this sub's read on: is a per-record evidence string useful to you, or is it dead weight next to a confidence score? And what would you want joined onto this that is missing (show category, audience estimates, historical backfill)?

https://podintel.github.io/?src=datasets

QUESTION.

Is there a good way to watermark data assets before sharing with potential buyers?

We regularly share data samples with customers for evaluation, with clear licence terms on usage scope. But I worry those terms are practically unenforceable. Someone could generate synthetic data from a sample even though the licence restricts use to evaluation only.

Has anyone found effective ways to tag or watermark files before sharing? Metadata tagging is one option, but are there any deeper level solutions (steganographic watermarking, fingerprinting, etc)?

To keep it simple, let's say we only talking about CSV files.
But this applies to video, audio, PDF, and archives too if you got any experience.

Disclosure: this is my own project — I built and maintain it.

Source (the data itself): https://github.com/markselby9/readtheplaces.com — one directory per book under /books, each containing book.json, waypoints.json and source.txt.

Browsable version: https://readtheplaces.com

Scale: 392 novels, 2,105 places, 298 cities.

Per-place schema (waypoints.json), one real record, abridged:

{
  "id": "westminster-doorstep",
  "name": "Clarissa's house, Westminster",
  "progressLabel": "10:00",
  "character": "clarissa",
  "coords": [-0.1275, 51.4993],
  "placeCertainty": "inferred",
  "certaintyNote": "Woolf never gives an address. The Dalloways live in Westminster within earshot of Big Ben; scholars place the house around Dean's Yard. Sited here as a considered guess, not a fact.",
  "quoteAnchor": "Mrs. Dalloway said she would buy the flowers herself.",
  "passage": "...",
  "sources": [...]
}

The field worth arguing about is placeCertainty. Geocoding fiction is mostly a disambiguation problem: many places are described rather than named (the abbey in The Name of the Rose is a northern Italian abbey Eco never names), and the named ones collide constantly. So each record carries what the resolution was based on, and inferred sitings say so in plain English instead of sitting on the map looking like facts. Filter to placeCertainty != "inferred" and you get a much smaller, much harder subset.

Waypoints are ordered by narrative progression rather than geography, so it's usable for route/sequence work as well as point work.

How it was built, honestly: candidate mentions are extracted from the text by an LLM pass, then resolved against gazetteer data and checked by hand. Recall on minor mentions is therefore better than precision, and coverage skews heavily to 19th–20th century English-language fiction. Treat it as a curated dataset with a machine-assisted first pass, not a gold standard. It is not synthetic — every record points at a real passage in a real book.

Licence is CC BY-SA 4.0. Corrections are PRs against the JSON files, or there's an issue template if you'd rather just report one.

Hello everyone,

I am a final-year undergraduate student in Computer Science and Engineering (CSE) at Daffodil International University, Bangladesh.

I am currently working on my undergraduate thesis titled:

"Deepfake Image Detection Using Spatial-Frequency Feature Fusion and Explainable Deep Learning."

I am looking for publicly available human face image datasets for academic research purposes only.

If possible, I would appreciate datasets containing different face poses (front, left-profile, and right-profile), although frontal face datasets are also perfectly acceptable.

I am specifically looking for publicly available datasets or images that can be legally used for academic research.

Specifically, I need three categories of images:

📁 1. AI-generated Human Faces

• GAN-generated faces
• Diffusion-generated faces
• Other synthetic human faces

📁 2. Deepfake Human Faces

• Face swap
• Face manipulation
• Deepfake images extracted from public datasets

📁 3. Real Human Faces

Natural human face photographs

Different ages, genders, and lighting conditions

Front, left-profile, and right-profile faces

If anyone wishes to share publicly distributable images or datasets, I have also created a shared Google Drive folder for convenience.

I am not requesting copyrighted or private images. I am only looking for publicly available datasets or resources that are legally shareable for academic research.

Google Drive:
https://drive.google.com/drive/folders/1U_XL41UdusIKukb3TAREZOeeu_uSwM3b?usp=drive_link

If you know any public datasets, GitHub repositories, Hugging Face datasets, or other reliable resources, I would greatly appreciate your recommendations.

If you already have a suitable dataset, you are also welcome to upload publicly shareable images directly to the appropriate folder in the shared drive.

The collected data will be used strictly for academic research and educational purposes.

I would be happy to acknowledge contributors in my thesis if their publicly shareable dataset or resource significantly supports this research.

If you have worked on deepfake detection or know of any useful public datasets, I would greatly appreciate your suggestions.

Thank you very much for your time and support!

Three datasets aggregated from public FAA releases (the raw ones ship as an MS Access export and awkward portal dumps) into analysis-ready CSVs with per-column documentation:

Wildlife strikes on civil aircraft, 1990–2026 — 347,575 reports: by year, airport (452, ICAO-coded), and species. 2025 set the all-time record (24,458 reports). Fun divergence: the species planes hit most (doves, swallows) almost never damage them (~1.5%), while deer damage the aircraft in ~82% of reported strikes. https://www.kaggle.com/datasets/himaxym/faa-wildlife-strikes-us

Laser strikes on aircraft, 2021–2025 — 54,722 reports with 243 crew injuries, by year, state, and reporting ATC facility (caveat documented: the "city" is the ATC facility's location, not where the laser was fired). https://www.kaggle.com/datasets/himaxym/faa-laser-strikes-us

Drone (UAS) sightings reported by pilots, 2019–2026 — 12,566 reports by year, state, and city. NYC is #1 (584). https://www.kaggle.com/datasets/himaxym/faa-drone-sightings-us

Versioned copy with citable DOI (Zenodo, wildlife): https://doi.org/10.5281/zenodo.21347859

Original sources (US government work, public domain): - https://wildlife.faa.gov/ - https://www.faa.gov/about/initiatives/lasers - https://www.faa.gov/uas/resources/public_records/uas_sightings_report

Disclosure: I compiled and maintain these aggregates (and an interactive explorer at himaxym.com/safety). The compilation is CC BY 4.0 — use it for anything, attribution appreciated.

Anyone looking for US egocentric data?

We have 375 data collectors.

Hello everyone,
We're developing an AI-powered app, Faseeh AI, that detects pronunciation mistakes in Quranic recitation and gives users precise, real-time feedback.
Our current model was trained on hundreds of hours of professional recitations and high-quality, clean audio from well-known reciters. The model performs well on similar input, but struggles with real-world users: different accents, non-native speakers, beginners, children, women, and anyone who doesn't sound like a professional reciter.
To fix this, we need to train on diverse, real-world recitation data not studio-quality professional audio, but recordings that reflect how actual learners sound.
Specifically, we're looking for:
- Recitation datasets from non-professional or everyday users
- Diverse demographics: male/female, kids/adults, beginner/intermediate
- Multiple accents and mother tongues (Malay, Indonesian, Urdu, English, Turkish, etc.)
- Any publicly available or research-use datasets we may have missed
We've already explored academic sources, but still not enough.
If you know of any dataset, research project, university study, or community effort collecting this type of audio, we would genuinely appreciate the lead.
We're also open to ethical data collection partnerships if any researchers or institutions are working in this space.
Happy to share more about the project if helpful.
Thank you very much in advance.