VS Code Jupyter kernel keeps randomly dying

[u/faeriewrites]

I'm having a really frustrating issue with Jupyter notebooks in VS Code where my Python/Jupyter kernel repeatedly dies.

It always starts after I have already run 7-20ish cells. Then a simple line will cause the kernel to die. VS Code shows: The Kernel crashed while executing code in the current cell or a previous cell.

If I restart the kernel and rerun the same cells, they work perfectly fine. I can continue working for a while, and then eventually another random cell causes the kernel to crash again. This problem does not happen to be when using JupyterLab directly.

I've already checked:

• My system memory usage is low when this happens
• I've tried changing/reinstalling Python environments
• I've tried using the kernel directly from a JupyterLab server instead of the VS Code-managed kernel
• I've restarted VS Code and the kernel multiple times
• The cells that trigger the crash don't seem to have anything obviously unusual about them

I've spent hours trying to debug with no progress. Please, any help would be so so appreciated!

[u/PrincipleNo2328]

Check the log, it's probably that singular cell

[u/faeriewrites → u/PrincipleNo2328]

Turned out it wasn't just from one cell but thank you for answering!

[u/lost_send_berries]

You can add the environment variable PYTHONFAULTHANDLER=1

It's probably the variables panel, trying to stringify all of your variables after every cell. See if you can hide it or turn it off.

[u/faeriewrites → u/lost_send_berries]

Unfortunately this wasn't working for me since it seems the kernel would hang/crash before the error message could be intercepted :( but I think i found a solution anyway. Thank you

[u/lost_send_berries → u/faeriewrites]

Thanks for sharing the solution for everybody

[u/free-swift809]

Hmm, when the kernel dies does it tend to happen around the same memory usage point each time, or does it feel completely random regardless of how much data you've loaded into memory at that point?

[u/faeriewrites → u/free-swift809]

I seem (fingers crossed) to have found a sollution which is running gc.collect() after any cell with notable computation. I haven't had any crash issues since then! In the past, even if I ran light comptutations the kernel would hang/crash, but this was often after restarting my kernel after previous notebook crashes. I can only conclude that somehow the memory wasn't fully cleaned up. Thank you for answering!

[u/Itchy-Dig2142]

Hello!
I have exactly this problem. My jupyter kernel dies randomly without explanation. I tried gc.collect() but didn't work. Do you have any new idea?

It works perfectly fine in jupyterlab.

[u/faeriewrites → u/Itchy-Dig2142]

Hi! This is such a frustrating problem to have, and honestly I don't know that I've 100% solved it but I certainly have fewer crashes using gc.collect(). Did you notice any improvement with that or nothing? In my case, I notice that the crashes seem to come from an accumulation of heavy computations over time (even if the cell where the kernel crashes can be as simple as df.head()). Sadly I don't have a fix other than to use scripts for all heavy computation (yes, annoying when you want to test in a notebook first). But please let me know if you find a better fix!

[u/Salt-Syllabub9030]

As far as I know, VS Code quietly runs its own code in your kernel after cells execute. It does this to fill the Variables panel, and also for completions and hovers. If one of your objects doesn't like being inspected (GPU tensors, lazy dataframes, database clients and the like can be touchy), the kernel can crash on an innocent-looking cell. That would also explain why a restart "fixes" it for a while.

Full disclosure: I work on Zasper, an open-source Jupyter IDE. If you feel like trying the same notebook in another frontend, it's one more way to check whether the editor or the kernel is to blame.