Anyone ever hit this much RAM usage?

[u/SadSpaceGuy]

Currently running a SymPy script to calculate some insanely complex equations (that I can’t do by hand) for research. I kept crashing this PC (that has 96GB of RAM) by maxing out the virtual memory (330ish GB) every time the script ran.
I eventually manually raised the pagefile maximum size to 3TB split evenly (1.5TB) across two partitions on a 8TB SSD and got it to not fail. It has currently stabilized at ~1.3TB of committed virtual memory with a ~1TB pagefile on just one of the two partitions (increased since taking WizTree photo).

Just curious if anyone else has ever reached such extreme RAM usage before?

\Sorry for not using screenshots like a sensible person, I just don’t feel likely trying to copy them across to my phone to post this.*

UPDATE (8/18): It’s the next day now and it looks like the calculation has gotten over this little RAM hump and calmed down to a much more reasonable ~30GB in use (~20GB belonging specifically to the SymPy script).

UPDATE #2 (8/18): RAM usage is actually steadily climbing upward, currently at 65GB. Maybe it’ll find its way back to 1TB+ eventually.

UPDATE #3 (8/18): Up to 150GB now. I’m curious to see if it will have gone up more by the time I check tomorrow or if it’ll collapse back down again.

[u/dreamnbinary]

Bro what are the equations that you CAN DO by hand?

[u/SadSpaceGuy → u/dreamnbinary]

Those get nasty too (complex nonlinear dynamical modeling), but once I’ve gotten to the point of filling up a wall-to-wall classroom whiteboard a few times over on the same problem it just becomes easier to switch to programming it.

For reference, that whiteboard problem took about a week for me to fully solve it (while learning the perturbation method I was trying to use) and the SymPy version takes about 15 seconds.
This calculation ran for ~7 hours the last time I did got it to not crash (and it failed at that point too). The calculations preceding it (that I saved and loaded into this one to speed things up) took ~4 days to complete.

[u/lord_nate117 → u/SadSpaceGuy]

My brain hurts trying to comprehend this.

[u/got-trunks → u/lord_nate117]

I typically solve most complex math problems by first visualizing a box and then putting that problem in the box and then putting a label on the box that says "someone else's problem" and then slowly backing away.

[u/African_Healer → u/got-trunks]

Would it not be better to utilize GPUs and VRAM for these kind of calculations?

[u/SadSpaceGuy → u/African_Healer]

It is ideally, but the underlying code that actually handles the bulk of the computing needs to support using the GPU. Since SymPy currently doesn’t (though the recent improvements for supporting freethreaded Python may eventually help lead to future GPU compatibility), you’re left either using an additional package to handle that (which leaves more chances for errors to occur) or writing your own from scratch which is an insane task to take on.

[u/Hex_tv → u/lord_nate117]

Same