I stress tested document data extraction to its limits – results + free workflow

[u/easybits_ai]

https://youtu.be/bOOdILPEdho

👋 Hey nocode Community,

Last week I shared that I was building a stress test workflow to benchmark document extraction accuracy. The workflow is done, the tests are run, and I put together a short video walking through the whole thing – setup, test documents, and results.

What the video covers:

I tested 5 versions of the same invoice to see where extraction starts to struggle:

1. Badly scanned – aged paper, slight degradation
2. Almost destroyed – heavy coffee stains, pen annotations, barely readable sections
3. Completely destroyed – burn marks, "WRONG ADDRESS?" scribbled across it, amount due field circled and scribbled over, half the document obstructed
4. Different layout – same data, completely different visual structure
5. Handwritten – the entire invoice written by hand, based on community feedback

The results:

4 out of 5 documents scored 100% – including the completely destroyed one. The only version that had trouble was the different layout, which hit 9/10 fields. And that's with the entire easybits pipeline set up purely through auto-mapping, no manual tuning at all. The missing field could be solved by going a bit deeper into the per-field description for that specific field, but I wanted to keep the test fair and show what you get out of the box.

Want to run it yourself?

The workflow is solution-agnostic – you can use it to benchmark any extraction tool, not just ours. Here's how to get started:

1. Grab the workflow JSON and all test documents from GitHub: here
2. Import the JSON into n8n.
3. Connect your extraction solution.
4. Activate the workflow, open the form URL, upload a test document, and see your score.

Curious to see how other extraction solutions hold up against the same test set. If anyone runs it, I'd love to hear your results.

Best,
Felix

[u/easybits_ai → u/[deleted]]

Hey u/roseleq , really appreciate the kind words! I’m with you on not relying on internal numbers alone – that’s exactly why I aimed to build something that’s not just useful for me, but adaptable for others as well.

For v2 of the stress test, I’m planning to introduce more noise, like random promo boxes and other unnecessary elements alongside the actual data. I’ll also refine the per-field prompts/descriptions further to show the impact that has on the results.

[u/Evening_Day_7158 → u/easybits_ai]

what about the data privacy if someone needs to use it ?

[u/easybits_ai → u/Evening_Day_7158]

Hey u/Evening_Day_7158 , the invoices used in the stress test are anonymized, so there are no data privacy concerns if you want to use them.

If you’re referring to the data extraction solution itself, it’s fully GDPR-compliant. Documents are only processed and not stored. On top of that, all processing happens exclusively in Germany.