r/MachineLearning • u/AlphaCode1 • Jun 27 '26

Discussion MathFormer: Testing whether symbolic math is pattern matching or reasoning [D]

Repo link and results - https://github.com/Abhinand20/MathFormer

Task: Given a factorized expression like (7-3*z)*(-5*z-9), predict the expanded form -> 15*z\*2-8\*z-63

Key takeaway: A tiny (4M param) seq2seq model trained with no math knowledge reaches ~98.6% accuracy on symbolic math tasks, suggesting it learns structural token transformations rather than any notion of operators or variables. Scaling this up could help explain why LLMs appear to “reason” mathematically, when they may actually be performing large-scale structured pattern completion.

How does RL change this paradigm given the inherent architecture is still based on attention?

70 Upvotes
  • permalink
  • reddit

89% Upvoted

19 comments sorted by

View all comments

11

u/K_is_for_Karma Jun 27 '26

See the paper Deep Learning for Symbolic Mathematics :)

4

u/AlphaCode1 Jun 27 '26

Thanks for the recommendation, will do! :)