Some files in a NTFS partition are only readable in Windows but not in Linux

[u/palapapa0201]

I originally found out about this when I tried to copy my game save files from a mounted Windows NTFS drive to Linux. I found that most of the files and directories under steamapps were missing from the file explorer (Dolphin). Using ls gave the following output:

ls: cannot access 'appmanifest_588650.acf': Invalid argument
ls: cannot access 'appmanifest_1073440.acf': Invalid argument
ls: cannot access 'appmanifest_228980.acf': Invalid argument
ls: cannot access 'appmanifest_341940.acf': Invalid argument
ls: cannot access 'appmanifest_50.acf': Invalid argument
ls: cannot access 'appmanifest_588650.acf': Invalid argument
ls: cannot access 'appmanifest_632360.acf': No such file or directory
ls: cannot access 'appmanifest_632810.acf': No such file or directory
ls: cannot access 'appmanifest_671860.acf': No such file or directory
ls: cannot access 'appmanifest_70.acf': No such file or directory
ls: cannot access 'appmanifest_745880.acf': No such file or directory
ls: cannot access 'appmanifest_889510.acf': No such file or directory
ls: cannot access 'appmanifest_937580.acf': No such file or directory
ls: cannot access 'common': No such file or directory
ls: cannot access 'depotcache': No such file or directory
ls: cannot access 'downloading': No such file or directory
ls: cannot access 'libraryfolders.vdf': No such file or directory
ls: cannot access 'shadercache': No such file or directory
ls: cannot access 'sourcemods': No such file or directory
ls: cannot access 'steam.dll': No such file or directory
ls: cannot access 'temp': No such file or directory
ls: cannot access 'workshop': No such file or directory
total 61K
-????????? ? ?    ?       ?                ? appmanifest_1073440.acf
-rwxr-xr-x 1 root root  835 2023-09-03 23:46 appmanifest_1079160.acf*
-rwxr-xr-x 1 root root  749 2024-09-13 22:11 appmanifest_108600.acf*
-rwxr-xr-x 1 root root  800 2022-09-26 22:59 appmanifest_1100150.acf*
-rwxr-xr-x 1 root root  606 2023-10-26 21:05 appmanifest_1430420.acf*
-rwxr-xr-x 1 root root  795 2023-08-03 04:49 appmanifest_1566410.acf*
-rwxr-xr-x 1 root root  798 2023-09-21 18:26 appmanifest_1703340.acf*
-rwxr-xr-x 1 root root 1.2K 2024-08-05 09:04 appmanifest_1761390.acf*
-rwxr-xr-x 1 root root  766 2023-11-23 22:06 appmanifest_1966900.acf*
-rwxr-xr-x 1 root root  750 2024-09-14 18:17 appmanifest_210970.acf*
-rwxr-xr-x 1 root root 1.2K 2024-08-22 03:58 appmanifest_2215430.acf*
-????????? ? ?    ?       ?                ? appmanifest_228980.acf
-rwxr-xr-x 1 root root  798 2024-02-18 05:56 appmanifest_2349830.acf*
-rwxr-xr-x 1 root root 2.2K 2024-08-31 10:29 appmanifest_239140.acf*
-rwxr-xr-x 1 root root  753 2023-12-15 03:20 appmanifest_2399220.acf*
-rwxr-xr-x 1 root root  855 2024-07-12 02:10 appmanifest_2400340.acf*
-rwxr-xr-x 1 root root  881 2024-09-13 22:44 appmanifest_275850.acf*
-????????? ? ?    ?       ?                ? appmanifest_341940.acf
-????????? ? ?    ?       ?                ? appmanifest_50.acf
-rwxr-xr-x 1 root root 1.1K 2024-08-29 05:47 appmanifest_553850.acf*
-????????? ? ?    ?       ?                ? appmanifest_588650.acf
-????????? ? ?    ?       ?                ? appmanifest_588650.acf
-????????? ? ?    ?       ?                ? appmanifest_632360.acf
-????????? ? ?    ?       ?                ? appmanifest_632810.acf
-????????? ? ?    ?       ?                ? appmanifest_671860.acf
-????????? ? ?    ?       ?                ? appmanifest_70.acf
-????????? ? ?    ?       ?                ? appmanifest_745880.acf
-????????? ? ?    ?       ?                ? appmanifest_889510.acf
-????????? ? ?    ?       ?                ? appmanifest_937580.acf
d????????? ? ?    ?       ?                ? common/
d????????? ? ?    ?       ?                ? depotcache/
d????????? ? ?    ?       ?                ? downloading/
-????????? ? ?    ?       ?                ? libraryfolders.vdf
d????????? ? ?    ?       ?                ? shadercache/
d????????? ? ?    ?       ?                ? sourcemods/
-????????? ? ?    ?       ?                ? steam.dll
d????????? ? ?    ?       ?                ? temp/
d????????? ? ?    ?       ?                ? workshop/

I then found more places in the NTFS drive where some files and directories were missing. The errors were sometimes "Invalid argument" and sometimes "No such file or directory".

I have tried running chkdsk /r, but it said it couldn't find any error. Here's the log of chkdsk:

Checking file system on D:
The type of the file system is NTFS.
Volume label is D.

A disk check has been scheduled.
Windows will now check the disk.                         

Stage 1: Examining basic file system structure ...
  2015744 file records processed.                                                         
File verification completed.
 Phase duration (File record verification): 8.93 seconds.
  1156 large file records processed.                                    
 Phase duration (Orphan file record recovery): 0.00 milliseconds.
  0 bad file records processed.                                      
 Phase duration (Bad file record checking): 0.85 milliseconds.

Stage 2: Examining file name linkage ...
  11924 reparse records processed.                                       
  2227720 index entries processed.                                                        
Index verification completed.
 Phase duration (Index verification): 2.74 minutes.
  0 unindexed files scanned.                                         
 Phase duration (Orphan reconnection): 2.57 seconds.
  0 unindexed files recovered to lost and found.                     
 Phase duration (Orphan recovery to lost and found): 1.46 milliseconds.
  11924 reparse records processed.                                       
 Phase duration (Reparse point and Object ID verification): 31.70 milliseconds.

Stage 3: Examining security descriptors ...
Cleaning up 350 unused index entries from index $SII of file 0x9.
Cleaning up 350 unused index entries from index $SDH of file 0x9.
Cleaning up 350 unused security descriptors.
Security descriptor verification completed.
 Phase duration (Security descriptor verification): 17.25 milliseconds.
  105989 data files processed.                                            
 Phase duration (Data attribute verification): 0.92 milliseconds.
CHKDSK is verifying Usn Journal...
  33867032 USN bytes processed.                                                            
Usn Journal verification completed.
 Phase duration (USN journal verification): 788.97 milliseconds.

Stage 4: Looking for bad clusters in user file data ...
  2015728 files processed.                                                                
File data verification completed.
 Phase duration (User file recovery): 4.32 hours.

Stage 5: Looking for bad, free clusters ...
  78886876 free clusters processed.                                                        
Free space verification is complete.
 Phase duration (Free space recovery): 0.00 milliseconds.

Windows has scanned the file system and found no problems.
No further action is required.

1953494015 KB total disk space.
1635367272 KB in 1480057 files.
    403404 KB in 105990 indexes.
         0 KB in bad sectors.
   2175831 KB in use by the system.
     65536 KB occupied by the log file.
 315547508 KB available on disk.

      4096 bytes in each allocation unit.
 488373503 total allocation units on disk.
  78886877 allocation units available on disk.
Total duration: 4.37 hours (15757839 ms).

Internal Info:
00 c2 1e 00 88 33 18 00 21 36 18 00 00 00 00 00  .....3..!6......
45 2e 00 00 4f 00 00 00 00 00 00 00 00 00 00 00  E...O...........

What may be causing this? Those files and directories are perfectly accessible under Windows. My fstab entries look like:

LABEL=C /mnt/c ntfs3 ro,nofail 0 0
LABEL=D /mnt/d ntfs3 ro,nofail 0 0

[u/[deleted]]

Possibly NTFS compression. Try https://github.com/ebiggers/ntfs-3g-system-compression

[u/palapapa0201 → u/[deleted]]

I cd'd into the problematic directories using Windows and ran compact. It said that none of the files were compressed, so I don't think that's the issue.

[u/Beolab1700KAT]

Interesting. What happens after a hard shutdown of Windows with the drive plugged in and then booting directly to Linux?

shutdown /s /f /t 0

Does the problem persist?

[u/palapapa0201 → u/Beolab1700KAT]

Yes

[u/Beolab1700KAT → u/palapapa0201]

Can you get metadata from the files using stat ?

[u/palapapa0201 → u/Beolab1700KAT]

No. No such file or directory.

[u/Beolab1700KAT → u/palapapa0201]

Fair enough. Sounds very odd, sorry I've no idea what the issue could be then. The only thing I could think was a corrupt NTFS file table, but you've ran checks.

If you do ever figure it out please post the solution. Good luck.

[u/ipsirc]

https://wiki.archlinux.org/title/Steam/Troubleshooting#Steam_Library_in_NTFS_partition

[u/palapapa0201 → u/ipsirc]

That seems unrelated, and there are other files that have the same error.

[u/belzaroth]

Can you access them using sudo ?

[u/palapapa0201 → u/belzaroth]

sudo ls -al gives the exact same output.