This is mostly one of those “note to self” type posts so I can remember what I ran into. Maybe it’ll help somebody else as well, in which case: yay! ;]
TL;DR: check if you have case-sensitivity enabled on the directories that are giving you problems using fsutil file queryCaseSensitiveInfo <directory>. If it is enabled, you can disable it with fsutil file setCaseSensitiveInfo <directory> disable.
Intro
So I am mounting an NFS share from a Windows 10 machine on a Truenas server to try and copy files off the Windows machine onto the Truenas server. The mount works fine, and I can copy the majority of files, however some of the directories give me an input/output error when I try to cd into them or read their contents with ls.
Note: All IP addresses, user, server, and directory names have been redacted.
Details
Volume mounted: on Windows 10 Pro 21H2 19044.2006
Mounting from: TrueNAS-SCALE-22.02.3 (Debian 11), using mount.cifs version: 6.11 (or smbclient Version 4.15.9-TrueNAS)
Mount works fine using:
sudo mount.cifs //windows-server/d /mnt/test --verbose -o user=$WINDOWS_USER,pass=$PASSWD
Most directories on the mount are readable, but some fail with “ls: cannot access ‘/mnt/test/Problem Directory’: Input/output error”. Unfortunately that error is a tad generic, so at this point I’m really not sure what the actual problem is here.
Also tried with smbclient:
sudo smbclient //windows-server/d -U "$WINDOWS_USER"
cd "Problem Directory\"
cd \Problem Directory\: NT code 0xc00004ba
That NT code 0xc00004ba is a bit obscure, googling only turned up some reference to the MS-SMB docs from 2016 at https://winprotocoldoc.blob.core.windows.net/productionwindowsarchives/MS-SMB/%5BMS-SMB%5D-160714.pdf (see p. 33) which says it is “STATUS_FILE_IS_A_DIRECTORY The file that was specified as a target is a directory and the caller specified that it could be anything but a directory.” Which is… weird? I tried to cd into it, so clearly I was trying to access a directory… Again, I’m not really sure what this is trying to tell me here.
What makes all of this even odder is that when I make a copy of the problem directory using Windows Explorer, I can read the copy via the same CIFS mount just fine…???
I’ve tried the following so far:
- Make a copy of the directory – copy reads no problem (unfortunately not a general solution for me because there are hundreds of these with large files and I don’t have the space to make copies all of these)
- Try different options for
mount.cifs– I went all the way up to the command below, adding one option at a time, nothing made any difference (every version was able to successfully mount, but still couldn’t read the problem directories):
sudo mount.cifs //windows-server/d /mnt/test --verbose -o nounix,user=$WINDOWS_USER,pass=$PASSWD,vers=3.1.1,uid=1000,forceuid,gid=1000,forcegid,file_mode=0666,dir_mode=0777,rw,noperm,cache=strict,rwpidforward,mapchars,noacl,noresilienthandles,nopersistenthandles,nobrl,forcemandatorylock,noserverino,nouser_xattr,nodfs,noposixpaths,nocase
- Try
smbclient– you saw the error I got above - Looked in
dmesg, no relevant errors I could see - Looked in /var/log/samba4 to see if I could find anything there, nothing stood out to me there, either
- Ran
chkdsk /F D:on the Windows machine, no errors reported - Checked file/directory permissions on the Windows machine, the user I’m connecting with has “Full control” enabled on the path
- The Windows share is simply on the D: drive, has Everyone: Full Control enabled on the share
- Renamed the directory to see if there were weird characters in the name – made no difference
- Tried sharing only the problem directory – now I get an input/output error directly when trying to mount this new share, fails with “VFS: cifs_mount failed w/return code = -5” (from
dmesg). - Even tried running the mount with
strace, which got me this:
write(2, "mount.cifs kernel mount options:"..., 107mount.cifs kernel mount options: ip=10.x.x.x,unc=\\windows-server\problem,user=$WINDOWS_USER) = 107
write(2, ",pass=********", 14,pass=********) = 14
write(2, "\n", 1
) = 1
capset({version=_LINUX_CAPABILITY_VERSION_3, pid=2637977}, {effective=1<<CAP_DAC_READ_SEARCH|1<<CAP_SYS_ADMIN, permitted=1<<CAP_DAC_OVERRIDE|1<<CAP_DAC_READ_SEARCH|1<<CAP_SYS_ADMIN, inheritable=0}) = 0
mount("//windows-server/problem", ".", "cifs", 0, "ip=10.x.x.x,unc=\\\\windows-server."...) = -1 EIO (Input/output error)
capset({version=_LINUX_CAPABILITY_VERSION_3, pid=2637977}, {effective=1<<CAP_SYS_ADMIN, permitted=1<<CAP_DAC_OVERRIDE|1<<CAP_DAC_READ_SEARCH|1<<CAP_SYS_ADMIN, inheritable=0}) = 0
write(2, "mount error(5): Input/output err"..., 35mount error(5): Input/output error
) = 35
write(2, "Refer to the mount.cifs(8) manua"..., 93Refer to the mount.cifs(8) manual page (e.g. man mount.cifs) and kernel log messages (dmesg)
) = 93
which is just back to the same generic input/output error… :/
- Tried from my MacBook (MacOS Monterey 12.5.1) as well – similar problem: can mount the share just fine, most directories can be read OK, but the problem directories just come up empty (no errors I could find anywhere)
- Tried from another Windows 10 machine, here I got “Problem directory is not accessible. The file cannot be opened because the path has a case-sensitive directory.” – INTERESTING!
I googled that error message and got this helpful How-To Geek page. Tried disabling case-sensitivity on one of the problem directories, and presto! Now it worked just fine! Whew!
Solution
So I needed to disable the case-sensitive attribute in NTFS using fsutil across all these directories. Luckily there is a pretty easy way to get that done:
foreach ($dir in Get-ChildItem -Recurse -Directory -Name){fsutil file setCaseSensitiveInfo $dir disable}
Just run that command in all the parent directories where you’re having problems, and it’ll go though and disable case sensitivity in all subdirectories.
Prologue
I have no idea how I got into this scenario in the first place! I might have used rclone from wsl at some point to copy some files over from an older machine, but I really don’t know. I’m just glad I finally managed to figure this out – took me quite some time!