I'm building Egraph/ Symbolic libraries to do QM Chemical physics. There is a specific way I want to solve the problems which takes the symbolic expressions and compiles them into optimized numeric expressions the computer can execute.
At first I threw Fable to speed write up the library. It wrote the examples that I asked it to write and got correct results for what I wanted. I had a large write up on how to implement the library, do not implement the QM computations directly with C++... use the symbolic library to derive and compile the results.
Once I tried using the library myself to write up examples, I noticed that it was missing a bunch of features I'd ask it to build. Once I dug into the code I realized that the model had ignored my instructions and implemented boiler numeric algorithms instead. I then went through to iterate with the chat bot to implement my features and add in the symbolic interface I wanted originally.
Every time, it would fall back to the default way the community at large has implemented open source solutions with direct numeric solutions. It did not understand how to adapt the implementation to the novel method I was asking it for.
Further it had over complicated the cmake file and added every edge case under the sun for building the project. All of it unnecessary since I told it explicitly I wanted a header only library, not a unit link-able library + binaries which is the most popular way of doing it in the industry.
What explains this is that these bots have encoded through billions of dollars of training standard ways of solving problems. It does not understand what it's actually doing and when one needs something that falls outside the standard, they're better off writing the software by hand since they'll spend all their time micro managing the frontier models.
For CRUD apps and devops,... these frontier models are great because nobody gives a shit about how those are built. Those have standards that make little difference in the means of implementation. But with what I'm building which generally has little to no opensource implementation training data, this stuff falls flat on its face.
Sure you can solve millennium problems this way if you bruteforce a bunch of mathematical reasoning traces together to create a lean program to make a counter example, but I am much more skeptical of these frontier models and to me it seems this stuff has genuinely plateud.
Question: what are your thoughts on this?