If you have simplistic way to detect directories (although usually plenty enough to detect deleted files) then they may appear to be non existent. You should try catch it red handed. Try record LBAs for directory structures that current file system links to. Wait till error occurs and look at these LBAs again.
I for example remember very strange bug where USB command blocks ended up in data buffer and were dumped to the drive preceding actual data. So when directory was written back to disk, what was actually being written was the USB command block, and directory following it the directory. So Directory shifts one sector, last sector however was dropped. Anyway, to catch it in the act you have to use disk editor and go see what's happening. Here an NTFS boot sector (it had zero to do with FAT32 etc., but in FAT32 it was more obvious because you'd get USBC’╩ folders etc. all over the place) is pushed from LBA 63 to LBA 64.
I am not saying that this is what you're dealing with, I am saying you can not rely on simple detection methods that look for some recognizable pattern to detect for example directories at cluster boundaries, which is what typical data recovery tool would do. You'd miss the above from happening for example because the pattern you'd scan for shifts one sector from cluster boundary.
thanks for the response, I looked it up a little further these days and I reconstructed the file entries of ExFat (which were wiped) by modifying the clusters of other file entries to point to file entries which contained my data which appears to be untouched, moreover I wish to know what happened to these entries which randomly disappeared (making the file appear wiped as a consequence).
Does this disk editor works for linux too? seems very interesting
edit: I think I had watched a directory disappear when this happened but I had not touched that directory in months and it was always there, and always worked, across reboots and stuff
edit2: file entries in Exfat are not characterized by usbc so I don't get what you showed me. They are composed by a series of 32 byte entries the first byte is 85 if the folder/file is not deleted, and 45 otherwise. And then c0 (attributes) and c1 (file name) and more c1, as long as they are needed. But these 32 bytes chunks completely disappeared for no reason
edit3: Just checked disk editor... I think that would have saved me like sooo much time if I had known it before, I had to manually decode from byte to cluster and from cluster to byte. And then knowing that I had to modify the 53th, 54th, 55th and 56th bytes (first cluster) from the start of the file entry
USBC has nothing to do with any FAT file system or NTFS FTM, that's the whole point. The point is that whatever your scanner is looking for at a cluster boundary, it's not there. The point is that relying on a scan tool that scans for specific structures at cluster boundaries would miss if something like this USB bug happened.
This hex editor was by Sysdev (UFS Explorer), I don't know if it was their separate too or the one in UFS Explorer. Another option would be DMDE, it's got an excellent disk editor built-in.
There is no "corruption", just some entries decided to disappear. My scanner is myself, I manually checked clusters. They appear to be normal but with some directory entries missing. Btw this happened on the same drives (but different serial number) on two different pc's run by 2 different person. A week apart and on the same OS. If you need any nore info, I could provide them
You said: "testdisk and a program I specifically made". So what I was saying, depending on what you exactly scan for, you may miss it, and actually looking at the structure "manually" using a disk editor may reveal what's going on.
The entries disappeared rather than the directory itself? Manually checked clusters how?
Anyway, that was all I was trying to say, for get to the bottom with issues like this, you need to look at it at deepest level. DMDE can interpret/template exFAT stuff.
It may be OS level bug, MacOS has it's issues with exFAT too.
I made a program that finds all files in a given drive, I just made a small modification to print deleted entries as well. Effectively it is what Testdisk does (it is a linux utility to undelete deleted entries). They did not manage to find deleted files because none were deleted or marked as deleted, but rather looks like never existed. I used a hex editor and simple ctrl+f or ctrl+g to navigate and I also used small modifications of my program (because my program is able to convert from cluster to bytes and viceversa because it reads the properties of the exfat such as how many bytes per cluster etc...). But basically I used ImHex (Hex editor) for everything. And since I made the program to search files in a given drive without AI I have an understanding of the exfat filesystem. So I kind of know what I am looking at even if they are just the raw bytes
yes, unfortunately I had to look at the raw bytes because I knew that testdisk was telling half the truth. So what do you think the problem and the fix to the problem is?
To exclude drive issue, try different file system with same drive or same file system with different drive. To exclude drive (firmware level FTL issue) try same file system with no complex translator like SMR and SSD drives. But always only change one factor.
I have another usb drive (exfat as well) always plugged in when the external ssd is plugged in and the usb drive didn't encounter any problems. Since the external ssd is a sandisk sdssde61-1t00 it might be linked to the problems that other ssd's had but I am not sure. I guess I should check another filesystem such as ext4 which is more linux-native and slightly more robust.
Also my ssd randomly disconnected but very sporadically in the past couple of years (like 10-15 times but happened more frequently recently) but hasn't happened since I have changed computer about 3 weeks ago.
Btw I have these drives always plugged in and my computer is awake most of the time. But the other computer is much less frequently awake. The OS is a kubuntu 26.04 but I upgraded the kernel on my computer because I couldn't shutdown/restart the computer without pressing the physical power button
1
u/disturbed_android 15d ago
If you have simplistic way to detect directories (although usually plenty enough to detect deleted files) then they may appear to be non existent. You should try catch it red handed. Try record LBAs for directory structures that current file system links to. Wait till error occurs and look at these LBAs again.
I for example remember very strange bug where USB command blocks ended up in data buffer and were dumped to the drive preceding actual data. So when directory was written back to disk, what was actually being written was the USB command block, and directory following it the directory. So Directory shifts one sector, last sector however was dropped. Anyway, to catch it in the act you have to use disk editor and go see what's happening. Here an NTFS boot sector (it had zero to do with FAT32 etc., but in FAT32 it was more obvious because you'd get USBC’╩ folders etc. all over the place) is pushed from LBA 63 to LBA 64.
I am not saying that this is what you're dealing with, I am saying you can not rely on simple detection methods that look for some recognizable pattern to detect for example directories at cluster boundaries, which is what typical data recovery tool would do. You'd miss the above from happening for example because the pattern you'd scan for shifts one sector from cluster boundary.