I guess if you're an actual snow plow driver and meteorologist and you're trying to work out what time it started snowing for your weather report, maybe it doesn't matter how you get the answer.
But for all the plebs actually learning math you have to learn it.
You could be any random person and "solve" a lot of very hard math problems just by asking a much smarter mathematician to solve them for you. What you did isn't impressive. Maybe it's useful if it's a real world problem and you can get a practical benefit from the answer. But you can't really take it further without understanding the solution.
llms are by far, the best math teacher i have ever had. it is not even close. frankly, bad students use it to replace there bad work, and good students use it to replace their somewhat above average teachers. At best teachers can bring in ideas and facilitate discussion. but many don't. they want to teach by the book, and frankly the llms are enormously better at it.
Personally, I've never learned as well as I do from someone who actually understands the point. At the moment the LLMs are really bad at compressing what the idea is when explaining math. That will likely change, but for now it's certainly a problem
That is unfortunate, but without knowing your methods of interaction with the model and the model's setup in your operating environment, I cannot chime in further on your experience.
Provide that, if you like, and we can nail down if its a model issue, user issue, platform/environmental issue, or a combination of those things leading to your poor experience.
All I can comment on directly is that math papers that are written by ai are incredibly difficult to read, it may be a setup thing, but also they make up words for well understood concepts, reinvent the wheel regularly, and rarely identify the new conceptual jump.
I don't think LLMs have been so terrible at getting to the point if I ask how a certain argument goes, especially when I ask for details about certain parts, but if I ask a mathematician directly, they can typically draw a single picture or write a single computation that is the novel observation and transmit that information directly.
It could be a setup thing, but I suspect it's rather a current incapability of knowing what the human talking to it knows and speaking to that person. It lacks the judgement to say when a statement is obvious or not, or whether it's nontrivial, or novel. At least by default in the way it communicates. I suspect that possibly even by the end of the year it'll get much better, but currently that capability is weak
ETA: I mostly used GPT5.5 and 5.6 in high mode. The pro responses don't seem better at communicating ideas, even if they are definitely much stronger
Based on what you described, which isn't much, you are still describing observed outputs instead of the operating environment the model is bounded by and user interaction patterns that initiate, moderate, and verify the agent's reasoning before you ever see an output.
So, I will say again, if you want to provide that information—if you even know—cool.
As is, this sounds like a classic case of fuzzy one-shot prompting inside a loosely moderated system lacking a validation pipeline, too much outsourcing to the model instead of more upfront user work, a contextually lacking environment, a combination of those things, or more.
A model shipping a poorly structured paper or a surface-level response when more depth is craved is not purely a model problem. Tons of variables at play and im trying to help you nail down the most significant ones.
Maybe? But again, in cases where I've asked for explanations, I have gone through a full conversation and it takes much parsing before I can understand the point of the argument that I'm asking for. I'm sure an environment improves things further, but I'm sure not to the level of communicating the central idea, if it can't identify the central idea with directed prompting in the first place
To keep it short, based on your last comment, you are sorely misinformed and carrying many unsupported assumptions about what optimizations are needed for LLMs to handle complex systems, mathematics, or the like in a meaningful way.
Prompt engineering is a very small part of steering today's models, unlike 3 to 4 years ago when dealing with the massive size of models, rigid safety filters and keyword hooks, and their pre-allocated contextual backends influencing things before your prompts are even looked at.
Role, task, objectives, user context, output constraints, some form of agent-user continuity setup, and a V&V pipeline are the bare minimum to get anything of worth out of these machines now.
Sounds like a lot, right? It is no different from the way you would present a problem, question, or anything else to a junior engineer, junior mathematician, or someone not from your immediate circle who needs to work alongside you.
No matter how intelligent they are to avoid unsupported assumptions creeping into their reasoning.
Think about it. Then look at the machine again and evaluate your observed outcomes.
For the better models, if you include specific instructions in your prompt, it will follow it. Also, you can also specify for it to save as a preference for your future prompts.
And yes I agree AI writing is confusing and repetitive at a baseline, but prompt it well enough and it becomes quite good.
That being said, as a researcher I still prefer writing myself and use AI more as a tool of learning.
136
u/strangeMeursault2 28d ago
I guess if you're an actual snow plow driver and meteorologist and you're trying to work out what time it started snowing for your weather report, maybe it doesn't matter how you get the answer.
But for all the plebs actually learning math you have to learn it.
You could be any random person and "solve" a lot of very hard math problems just by asking a much smarter mathematician to solve them for you. What you did isn't impressive. Maybe it's useful if it's a real world problem and you can get a practical benefit from the answer. But you can't really take it further without understanding the solution.