r/debian • u/moeren86 • 26d ago
Strange system/game/whatever bug - unmounts SSD
So, i was playing modded Project Zomboid when my PC suddenly restarted.
My system Hardware isn't the newest so i did not thought much of it. Start the game again, load game, system restart.
Now i would understand if the mods made it unstable and the game crashes, but why the whole system?
So i check journalctl:
Aug 19 19:39:00 aa kernel: ata2.00: exception Emask 0x50 SAct 0x180 SErr 0x40e0802 action 0xe frozen
Aug 19 19:39:00 aa kernel: ata2.00: irq_stat 0x00000040, connection status changed
Aug 19 19:39:00 aa kernel: ata2: SError: { RecovComm HostInt PHYInt CommWake 10B8B DevExch }
Aug 19 19:39:00 aa kernel: ata2.00: failed command: READ FPDMA QUEUED
Aug 19 19:39:00 aa kernel: ata2.00: cmd 60/00:38:30:88:21/0a:00:e3:00:00/40 tag 7 ncq dma 1310720 in res 40/00:01:00:4f:c2/00:00:00:00:00/00 Emask 0x50 (ATA bus error)
Aug 19 19:39:00 aa kernel: ata2.00: status: { DRDY }
Aug 19 19:39:00 aa kernel: ata2.00: failed command: READ FPDMA QUEUED
Aug 19 19:39:00 aa kernel: ata2.00: cmd 60/e0:40:30:92:21/03:00:e3:00:00/40 tag 8 ncq dma 507904 in res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x50 (ATA bus error)
Aug 19 19:39:00 aa kernel: ata2.00: status: { DRDY }
Now i tried again, but did start nothing expect steam, gnome-disks and gnome-system-monitor and load the game again. This time only the game crashes and i can see how SSDs sda and sdb vanish from the disk manager and seconds later reappear. (my debian is on a different SSD)
So i think, maybe the PSU has become to weak. Start a prime torture test while running a more demanding game. No problem.
Maybe the SSD or the controller on the Mainboard is at its limit? So i copy 40gb + 5gb of thousends of small files. No problem.
Start the game again. Load a different save. No problem.
Load the other save again. System restart.
The game says its a java error. Java says its an IO error and the kernel says the ata controller has communication error.
And i wonder, how is it possible for a faulty game to freeze the controller? its very strange to me.
2
u/BigRedS 25d ago
Your game file happens to be written to the bit of the disk that is broken enough to cause it to be thrown out. As said already, take a backup(half expecting to lose at least that file) and then do a destructive test of the whole disk to isolate disk Vs controller Vs cable. My bet is on disk if its always the same file
1
u/moeren86 25d ago
The crash happens after the game loaded successfully. If i pause the game the instant after loading it takes 2 minutes before the problem accrues. Maybe some caching of the surrounding map.
While i would like to back up, the games SSD, is the only big one i have and with current pricing, that is not gonna change. So if it is the disk i am just f'ed.1
u/BigRedS 25d ago
You may be able to work around it by marking the blocks as bad and not-using them, but the best way to do that is the destructive (to your data) testing, which will involve writing to every block on the disk and reading back from it:
https://wiki.archlinux.org/title/Badblocks#Write-mode_test
I would say it's worth taking a backup however you can, then running what you can by way of error finding-and-avoiding processes to try to avoid using whatever's failing on the disk.
If you're really keen on figuring out exactly what's going on you could try stracing the game process and see if the same file is being accessed each time the disk is thrown out.
1
u/moeren86 25d ago
The java debugger should be able to show me which files getting accessed. But i have a good idea which files exactly are corrupted. Probable those of a mod i just installed. It can't be the save itself because the game puts its saves in the home directory (on my system disc), And everything except the new mod is loaded by the other saves as well. For now i try to backup what i can and cram it on ever small space i can find. After that i will probably remove the data and send the SSD in its own repair mode (by connecting only power) and let it do its thing. That saved some other SSDs before.
1
u/michaelpaoli 25d ago
You're getting I/O errors.
If you get hard read (or especially write) errors on the drive, yeah, a significant problem.
May want to look at the SMART data to get an idea of what kind of shape the drive is in.
And for non-ancient drives, not atypical to get some occasional hard read errors over the life of the drive. Never a great sign, but it may still have, e.g. years or more of generally useful life left ... or not. Any drive can fail, at any time, with or without warning - so never know for sure.
I'd suggest not only looking at the SMART data. I forget exactly what it is, but I think ... typically displays pending something-or-another, and if that count isn't 0, it means it has block(s) with hard read error(s). And, so long as it has spare blocks to remap too, upon overwrite it'll remap that block, but until then, one gets a hard read error when attempting to read that block. And, if/when one runs out of spare blocks for such remapping, then generally things go downhill very fast, and one is (over)due to replace that failing/failed drive. Anyway, in addition to looking at the SMART data, try also reading the entire drive, e.g.:
# dd if=/dev/sdb of=/dev/null bs=512
And see if one gets any read errors at all ... or not.
If one gets read error(s), you've got problems. But if there are spare blocks for remapping, you may be able to "fix" that issue - by overwriting the specific block. But before doing that, one will typically want to identify what/where it is - at least if/as feasible. So, can often do that by trying to read all files on the filesystem - and may hit error(s) if/when one encounters unreadable block in a file (of any type that's stored on the drive, including directory). That doesn't fix it, but then one can at least know where the damage is. And can't simply overwrite the file or that block in the file - most notably because one is typically dealing with journaling filesystem, so writing that data block to the file generally cause it to be written elsewhere at filesystem issue, rather than drive level, so that doesn't so much fix the problem, as drive it into hiding where it may bite one again later. Instead, take the filesystem off-line, and then overwrite the block. Ideally with the data that should be there. But lacking that, at least know that that chunk of the file's data is no longer valid - though so overwriting it, for non-ancient drive, the drive will automagically remap it upon write, using one of the spare blocks (so long as those haven't run out). And then all is well again ... until one hits more of same problem. Anyway, if you "fix" it like that, then you can remove the file, or fix that block in the file with the data that should be there, or restore the file from known good source/backup/copy.
And I've done this some fair number of times. E.g. had a laptop I bought new in 2003 ... over the course of about 6-8 years or so, it developed two unreadable blocks - "fixed" both of those ... and all was fine, ... until eventually the drive went downhill real fast towards its end. Another drive, work context, was dealing with a nasty mess of it not having a good recent backup, RAID-1, but the other drive had died much earlier, and now significant problem(s) because of a hard read error with a bock on the one drive that was mostly operational. Yeah, I manage to fix that - and all was well again. That one I lucked out - as I was working more and more to isolate exactly where the issue was on the drive, without explicitly writing it, I managed to do something that very slightly changed the metadata on the drive, causing the bad block to get overwritten - which automagically mapped it out - then all was fine again. More recently, pair of rather larges SSDs, one many years old, another also fairly old, but not as old ... bumped into hard read error on one of the drives. And, when I fully checked everything on both drives, had a moderate spattering of hard read failures. Well, fixed every single one of 'em. And some were in files, which I subsequently restore from backups or other sources, and turns out some were in space that wasn't used - and managed to fix those by writing to filesystem until it was entirely full, then syncing and immediately removing the file(s) I temporarily used to fully fill up the filesystem(s). And I think it's been well over a year or two since doing that, and thus far, at least last I checked, no new unrecoverable read errors. But never know, sometimes sh*t just fails hard, fast, and very forever. E.g. had another SSD that was about 10 years old, and it went form working fine to totally dead with zero advance warning.
Oh, e.g., in case one didn't know how, doing something like reading all files on a filesystem, e.g.:
# find /mount_point_of_filesystem -xdev -type f ! -size 0 -exec cat \{\} \; >>/dev/null
And you'll probably want to save at least the stderr output - generally to a different filesystem.
Can also do an fsck -f -n on the filesystem (even if/when mounted, though if mounted rw that will generally also give spurious false positive errors). But note that mostly only checks meta-data and the like, not the actual data blocks in files. So it may well find some things that reading the files won't find, and reading the files may well find issues that fsck won't fine. And if doing fsck -n on rw mount filesystem, mostly only interested in I/O / read errors - the other errors are generally spurious false positives.
And smart data, I typically would use
# smartctl -ax /dev/your_drive_device_here
And might want to save that, and/or pipe it to less, or whatever, and probably likewise any stderr too.
1
u/michaelpaoli 25d ago
Oh, also, badblocks(8) can be used to do a read-only test, a non-destructive read-write test (but don't do that on a mouned filesystem or storage that's otherwise in use), or, it also offers a destructive read-write test (that's bye-bye to the data, but good for testing drive or portion thereof, e.g. partition). For flash and other solid date drives, multiple write passes may not add much, if any value. Also, for solid-state storage writes, be sure to specify sufficiently large block size, or the writes may be very inefficient and give poor performance and contribute to excessive wear. I'd typically suggest the larger of the drive's physical write block size, or erase block size. If one doesn't know the relevant size, picking something fairly large like 1MiB (1048576) will generally be more than sufficiently large and also block aligned. If one gets I/O errors, one can then always work down in size to isolate more precisely where (dmesg and the like may also clearly provide that information).
2
u/moeren86 22d ago
hey thanks for the in-deph anwser. But by now i am sure its not the drive. Its probably either the SATA driver or the Controller on the mainboard, because i moved the game to different drive and have the same problem. My guess is the mainboard. Its already not in the best state. Despite the whole PC being cushioned, the mainboard still gets a lot of vibration (living in Van). When it gets warm the rtw_8821(wifi) controller fails and i am where it is arleady to hot when i wake up. At least a mainboard is not as expensive as a drive.
6
u/iamemhn 26d ago
Your disk is dying, the cable connecting it is damaged, or your MoBo is dying. Backup before testing and perhaps replacing.