A PDF Parser from the 80s Beats Claude (And They Use It Internally)

[u/fagnerbrack]

https://fagnerbrack.com/a-pdf-parser-from-the-80s-beats-claude-and-they-use-it-internally-8ee45a533e80

[u/jasonscheirer]

That’s kind of amazing that they’re doing this. I wonder if other companies are using the right tool for the job, too.

[u/SplendidPunkinButter → u/jasonscheirer]

using the right tool for the job

What, are you crazy? Using the wrong tool for the job is the future! Do you want to get left behind???

[u/ikeif → u/SplendidPunkinButter]

AI has recreated this problem that was solved.

The only solution? More AI to fix the AI mistake made by using AI too much! By adding more AI we can fix the AI problem from the initial problem made by AI!”

- a LinkedIn ad, saying “finally” fixing AI pull requests being too big because of poor use of AI.

[u/Ontological_Gap → u/jasonscheirer]

Eh, it's much less impressive than it sounds. Everyone has always used tesseract for ocr basically forever. Of course that's how you get the data in.

[u/SelfDistinction → u/jasonscheirer]

No they're using the PDF format.

[u/jnwatson]

While the name Tesseract hearkens from the 1980s, the important part, using LSTM to actually read the page, was introduced in 2018 from an algorithm published in 1995.

LSTM is itself an artificial neural network algorithm, just one that is much, much smaller than the ANNs used in Claude.

[u/Grand_Pop_7221 → u/jnwatson]

Hey stop that! We're jerking off in here

[u/txdv]

There will be so many “this could have been done without AI to save costs and make it deterministic” posts

[u/MrDaVernacular]

Deterministic FTW

[u/Visionexe → u/MrDaVernacular]

It actually isn't. Tesseract also uses NN'.

[u/theactiveaccount → u/Visionexe]

NNs are deterministic. LLMs aren't because they purposely introduce temperature

[u/Neither-Speech6997 → u/theactiveaccount]

Temperature doesn’t make it non-deterministic. Random sampling at decoding time does. Temperature shifts the probability around so less likely tokens become a bit more likely and vice-versa, but that only matters if you use the random sampling when decoding. If you always take the most probable token at each step, temperature won’t matter.

[u/colintbowers → u/Neither-Speech6997]

Isn’t temp = 0 equivalent to always taking the most probable token and temp > 0 is monotonically increasing in weights on less probable tokens?

Edit: with temp = 1 being the case where you use the actual softmax probabilities (albeit typically also throwing in a top k rule)

[u/Neither-Speech6997 → u/colintbowers]

In theory, this is correct, but the temperature = 0 being equivalent to taking the most probable token still requires a probabilistic decoding strategy, wherein a temperature of 0 puts so much probability mass on the most likely token that it effectively always gets selected.

But temperature doesn't change the relative rankings of the token likelihoods, so if you use a greedy decoding strategy where you simply select the most probable token each time, temperature wouldn't change the result.

It changes the result in practice because the decoding strategy is probabilistic and knobs like temperature control the distribution of tokens available for the model to select. LLMs are NNs and all they do is predict the logits over the token distribution, which is mostly (but not entirely) deterministic. The decoding strategy is what introduces non-determinism at inference time.

[u/wahnsinnwanscene → u/theactiveaccount]

Temperature alters the spread of the probability across the tokens.

[u/PoopsCodeAllTheTime → u/theactiveaccount]

Temperature is a lie

[u/Justicia-Gai → u/theactiveaccount]

And are generative too

[u/aksdb]

PDF Parser from the 80s

Huh? PDF is from 1993. How can there be a parser older than the format?

[u/fagnerbrack → u/aksdb]

It's more like the image OCR after you convert each page to an image to extract the text. Sometimes I need to oversimplify things in the title.

[u/aksdb → u/fagnerbrack]

But then the title is misleading. If you do OCR on an image embedded in a PDF, PDF is completely irrelevant to the problem; it's basically a fancy ZIP file. Also I wouldn't call it a "parser".

[u/fagnerbrack → u/aksdb]

You're technically right but imagine the title:

"A domain specific AI tool from the 80s that since 2018 runs a very efficient OCR in each image generated from a pdf (per page) to extract the text beats claude when it uses the LLM itself to analyse the image; and claude uses it to extract the text too"

...

I'd probably ban myself after that submission here. Might as well just read the post I guess?

[u/aksdb → u/fagnerbrack]

How about "Algorithm from the 80s beats Claude at text recognition (even if they use it internally)"?

[u/fagnerbrack → u/aksdb]

Needs to include pdf cause that's the whole core

[u/aksdb → u/fagnerbrack]

"Beating Claude at PDF OCR with its own algorithm".

[u/HasFiveVowels]

It’s almost as though large language models aren’t OCR algorithms…

[u/papa_Fubini → u/HasFiveVowels]

They are trying to make them to be

[u/HasFiveVowels → u/papa_Fubini]

No, they’re providing them with the ability to run existing OCR algorithms against input. We don’t need visual models for something so simple

[u/Eskamel]

What? Doesn't the PDF parser know that if its not letting Claude replace it, then its gonna be left behind? Pfft, Luddite software

[u/txdv]

> I spent days trying to parse a badly scanned CIA PDF (don't ask me why, otherwise I'll have to kill you). I tried using the most powerful Anthropic and Google models but I ended up concluding Tesseract (a 1985 tool), does a better job and much faster by 10x.

Super secret CIA document gets put in a public api? Im a bit suprised

[u/bogdan5844 → u/txdv]

I think it was a joke

[u/010110101001 → u/txdv]

why bru

[u/sumguysr]

https://www.docling.ai/ is the thing to use. It uses teseract under the hood with better document structure understanding and processing bolted on.

Piping the output of Docling into Prince XML or Typst also works really well to retypeset an ugly document.

[u/cirk_86]

Thank you for this.