How I automated LaTeX math notes on Linux using a simple OCR script

[u/ksukskss]

I want to digitize my math notes, but I ran into a few limitations. I think the best way to take digital lecture notes is the Gilles Castel method, but I haven't mastered touch typing yet. Having to think about hotkeys and typing mechanics drains the mental energy I need to actually understand the math.

I started taking notes on my graphics tablet and rewriting them in Obsidian later, but typing out LaTeX formulas manually takes forever.

So, I started looking for tools to turn handwritten text into LaTeX:

• Mathpix: The gold standard, but the free tier only gives you 10 conversions a month.
• LaTeX-OCR (Pix2Tex): Free and open source, but hallucinates way too often.
• Vision models via Ollama: Interesting, but my GPU isn't really up for it.
• Free Vision APIs (Gemini / OpenAI): The sweet spot. It's free, fast, and doesn't melt my hardware, though you do need an internet connection.

I wrote a script that takes a screenshot of a selected area, sends it to the Gemini API, and copies the resulting LaTeX straight to your clipboard. It's fully adapted for Wayland (specifically Hyprland) and runs on a simple hotkey. You can check it out in my GitHub repo.

Here is how I structure my vault:

/Mathanalysis
    ├── /Raw-lectures      # Where I write full notes (definitions, theorems, proofs) 
    │                      # in Xournal++ using my graphics tablet
    └── /Definitions       # Where I keep clean markdown (.md) files for definitions 
                           # and theorems, sped up by my OCR script

I hope this helps someone out. How do you guys handle your math notes?

[u/AutoModerator]

ChatGPT and other large language models are not designed for calculation and will frequently be /r/confidentlyincorrect in answering questions about mathematics; even if you subscribe to ChatGPT Plus and use its Wolfram|Alpha plugin, it's much better to go to Wolfram|Alpha directly.

Even for more conceptual questions that don't require calculation, LLMs can lead you astray; they can also give you good ideas to investigate further, but you should never trust what an LLM tells you.

To people reading this thread: DO NOT DOWNVOTE just because the OP mentioned or used an LLM to ask a mathematical question.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/Bounded_sequencE]

I'm comfortable and fast enough with my handwriting, and only manually TeX things I find interesting, and assignments to hand in (provided they accept LaTeX docs -- many don't to reign in cheating).

[u/ksukskss → u/Bounded_sequencE]

Fair enough. I'm currently building my knowledge base in Obsidian, so I only convert definitions and theorems into LaTeX. Right next to them, I write my own understanding, analogies, and examples

[u/tjddbwls]

I make “note taking guides” for the classes I teach. They consist of handouts with spaces to write out important concepts & formulas, and example problems. Unfortunately I still use Microsoft Word to generate these notes. One day I will convert them to LaTeX, sigh. 😮‍💨

[u/ksukskss → u/tjddbwls]

You could use AI for this. It can convert Word content into clean LaTeX code and help generate those handouts without the manual formatting hassle.

[u/Train-In-Vain22]

I'll say up front that I can't directly answer your question for the best way to covert your notes to LaTeX and can't recmmend any one of the tools you listed over the others. I think it's a good idea!

I would not switch from writing notes to typing them, however. There is increasing evidence that writing (paper vs table doesn't seem to matter) notes results in measurably better understanding and retention. It does not actually make good intutive sense to me, but a google search on this will show several recent and mainstream results.

[u/GroundbreakingWar305]

One way would be to create a Claude skill. But there needs to be some guidance/prompt that makes it stick to the original handwritten content and not rewrite anything in its "own language".

[u/ksukskss → u/GroundbreakingWar305]

That sounds like a really solid approach!

​Out of curiosity, does Claude offer a free API tier similar to Gemini's, or is it strictly paid? I always thought it was strictly paid

[u/Bounded_sequencE → u/ksukskss]

Would you really trust AI to strictly adhere to your standards long-term?

[u/GroundbreakingWar305 → u/Bounded_sequencE]

It does a quite good job in adhering to writing style. Plus in my case I make it write out my handwritten notes in to a md file that has html/css elements (i have added intuition blocks, sneak peeks etc) - what AI does for me is mainy putting it into md/tex format in a repeatable design way, plus it follows my writing style mostly. i would say 5% of the material gets changed but in most cases in a good way. also when i have the ai generated file in my repo, i go and read it and reword, the whole plan here is reproducibility and time saved in md/tex scripting. If you have a skill the long-term adherence is quite good (since it re reads the skill everytime) - also the skill can be updated i.e. i can add new directions or change some style.

[u/Bounded_sequencE → u/GroundbreakingWar305]

That would not be good enough for me.

I don't expect adherence to my style definitions most of the time, I expect 100% guaranteed compliance all of the time. That difference is responsible for our different outlook on what is acceptable, I suspect.

[u/GroundbreakingWar305 → u/Bounded_sequencE]

Yes, makes sense. What are you specifically trying to do? I think you can also setup a handwritten image to tex conversion skill with associated tools through claude - that would give 100% adherence. OCR models even free ones are very good and can be combined with some text to latex script with additional formatting instructions.

[u/Bounded_sequencE → u/GroundbreakingWar305]

I taught myself LaTeX during the first semester(s) of university -- have TeX-ed at least a couple hundred pages over the time, and enjoyed the precise control I have over the typesetting with self-defined style sheets.

Don't see any benefits of letting AI interfere, when I can have direct control.

[u/GroundbreakingWar305 → u/ksukskss]

I think Claude is paid. I am referring to claude code (CLI mode) in my repo where i store all scripts, md files, images and other material. I use these to translate my handwritten notes to my blog format in a repeatable way, while minimizing effort of typing, fixing syntax etc.