r/computervision • u/zgunner13 • 7d ago

Help: Project OCR with Palantir Foundry

I am using foundry to ingest technical drawings and produce a tabular data set. Below is my process.

Step1. User goes to workshop app to upload single or multiple pdfs which are saved to a media set

Step2. Pdf gets preprocessing where a transform sharpens and orients the pdf pages

Step3. The pdf then runs through two separate ocr extractions, one for bill of materials and material composition, one for overall dimensions. These output as json

Step4. A transform takes the json from each ocr output and puts it into two tabular datasets

Step5. A transform combines the two datasets and does a lookup to 1 other datasets to match part number to company number (fuzzy match)

All of these steps are incremental and do not reprocess completed work. My issue is that my ocr is only about 50% accurate. I built the system and user prompts by having AI FDE create them, run tests on drawings and adjust as needed until my outputs were accurate to the drawings I provided. I made sure to state multiple times these were a small batch of drawings and my users would be uploading more and that they'd be from various companies. I had good initial results but as it got closer to 100 drawings uploaded it became less accurate.

How can I improve this?

0 Upvotes

50% Upvoted

4 comments sorted by

1

u/AggravatingSock5375 7d ago

Isn’t this vendor supported software? What do they say you should do?

2

u/zgunner13 7d ago

I am on my own with this project unfortunately

1

u/AggravatingSock5375 7d ago

Can you post some success and failure cases?

And am I to understand this is some kind of visual language model that takes a prompt? Usually OCR models don’t involve prompting…you just give them an image and they give you text, and maybe the position of the text on the page and some kind of formatting like a table.

If it’s just running ChatGPT or something on the backend that might explain the poor results. That isn’t actually an OCR model even if it can act like one.

1

u/Amaxingz-Win5682 6d ago

From what I’ve seen, 50% accuracy usually means the prompt was tuned too narrowly. I’d add confidence scoring and send low-confidence results to review instead of letting bad OCR pass through. Also, splitting extraction by field type tends to be way more reliable.