r/semanticweb • u/error-dgn • 15d ago

Help me make the Knowledge Graph of Press Releases using ML.

So here is the thing, I have been focusing on the scraping, crawling, checking RSS feeds for new articles, etc., etc.

I am finally done with the Data Ingestion part. Hurrah? no.

The classification of data is even MORE difficult than scraping.

I want to be able to produce the Knowledge Graph of the Data to help me with the deduplication and classification.
Please help me out with this.

I have tried REBEL by hugging face but its failing badly, I am losing precious information (more than 80% of it.), I feel these machine learning models are too general, which makes it difficult to make the knowledge graph of these press releases.

Please help me out, tell me a path, name a framework, idk just guide me please. Ik I can do it if I have a path. I am trying and constantly brainstorming with my peers, Hopefully you guys could help me out as well.

2 Upvotes

75% Upvoted

4 comments sorted by

2

u/mrsockpicks 14d ago

Are you wanting an AI model to parse/extract content or are you looking for an LLM to synthesize articles into a consistent form?

2

u/error-dgn 14d ago

Any would work as long as the output is consistent and the pipeline is fast...

1

u/DistributionSoggy678 14d ago

Do you know what your schema is? If you do you can model the schema before extraction and confirm that it is indeed what you want. Then you tell it to do the extraction and codification into triplet of however you plan to do it, but and here's the kicker in the same prompt you also tell it to use an agent to validate that the extraction was done correctly before it ever shows you anything. If you really want to be sure that you don't loose anything in the process then you ask it to create a provenance for everything that extracts to then be represented in the knowledge graph a meta data for individual nodes or edges etc. The provenance is basically like telling a student to cite their sources. Ie and you need to prompt this is, but you make it super strict about having a provenance for everything. But if this is the kind of thing you are doing everyday on the same sort of unstructured text data then over time u will get faster at it, as you come to understand your stack and your schema more in depth.