CIFS VFS No response for cmd n mid…
Aug 2nd, 2008 by bolt
On a few systems, I’ve noticed CIFS mounts have a tendency to lock up the system when transferring large files, or a shitload of small ones in quick succession. When this happens, the system may or may not completely lock up, and lines like these will appear in your syslog:
Aug 1 22:58:16 dreadnought kernel: CIFS VFS: No response to cmd 46 mid 25661 Aug 1 22:58:16 dreadnought kernel: CIFS VFS: Send error in read = -11 Aug 1 22:58:16 dreadnought kernel: CIFS VFS: No response for cmd 50 mid 25664 Aug 1 22:58:16 dreadnought kernel: CIFS VFS: No response to cmd 47 mid 25663 Aug 1 22:58:20 dreadnought kernel: CIFS VFS: No response to cmd 47 mid 25662 Aug 1 22:58:20 dreadnought kernel: CIFS VFS: Write2 ret -11, written = 0 Aug 1 22:58:20 dreadnought kernel: CIFS VFS: Send error in read = -9 Aug 1 22:58:20 dreadnought kernel: CIFS VFS: Write2 ret -9, written = 0 Aug 1 22:59:52 dreadnought kernel: CIFS VFS: No writable handles for inode Aug 1 23:00:29 dreadnought kernel: CIFS VFS: close with pending writes Aug 1 23:00:36 dreadnought kernel: CIFS VFS: close with pending writes Aug 1 23:01:16 dreadnought kernel: CIFS VFS: No writable handles for inode Aug 1 23:02:28 dreadnought kernel: CIFS VFS: close with pending writes Aug 1 23:03:23 dreadnought kernel: CIFS VFS: close with pending writes Aug 1 23:03:44 dreadnought kernel: CIFS VFS: close with pending writes Aug 1 23:05:35 dreadnought kernel: CIFS VFS: No writable handles for inode Aug 1 23:06:18 dreadnought kernel: CIFS VFS: close with pending writes Aug 1 23:06:35 dreadnought kernel: CIFS VFS: close with pending writes Aug 1 23:06:46 dreadnought kernel: CIFS VFS: No writable handles for inode Aug 1 23:06:46 dreadnought kernel: CIFS VFS: close with pending writes Aug 1 23:07:02 dreadnought kernel: CIFS VFS: close with pending writes Aug 1 23:07:46 dreadnought kernel: CIFS VFS: server not responding
I found a workaround which involves disabling Opportunistic Locking. This will have a small performance impact, but at least the thing wont hang up and use 100% CPU for no goddamn reason. Yes, this article has an angry tone, because this issue has eaten way too much of my time today.
To disable Opportunistic Locking, you have to set the contents of the file “/proc/fs/cifs/OplockEnabled” to 0. This file only exists after the cifs module has been loaded, and will be replaced if the module is unloaded and reloaded. Thus, I configured my /etc/rc.local script (use whatever equivalent your distro has to a startup script) to both load the cifs module and set the contents of said file:
# cifs client workaround modprobe cifs echo 0 > /proc/fs/cifs/OplockEnabled
There you go.
thanx a lot. this has been a nasty bug for me for a long time!!! should be default setting i guess….
A good solution, all I need now is a faster network drive.
Great idea…worked wonders, sometimes I forget about /proc
You know I have been looking for this for a long time, it took me forever to figure out what was going on. I have been running back up’s over the network every night, and about every other night the server would completely lock up, especially on large back ups. This sounds like it will fix the problem, I have made the change and we will see if this fixes the issue. It looks like this would be a more common problem, and have more visibility, but it has taken me a while to come across this site.
Thanks for the fix, and I understand your frustration, while my server is only for development purposes, it was still a pain to come in to work in the morning to a server not responding to anything and having to reboot.
I follow your instructions, but still have these errors
CIFS VFS: No response for cmd 50 mid 61463
CIFS VFS: No response for cmd 50 mid 61462
CIFS VFS: No response for cmd 50 mid 61461
CIFS VFS: No response to cmd 46 mid 61460
CIFS VFS: Send error in read = -11
CIFS VFS: No response for cmd 50 mid 61465
CIFS VFS: No response for cmd 50 mid 61466
CIFS VFS: No response for cmd 50 mid 61464
If I cat /proc/fs/cifs/OplockEnabled, it returns 0. Please help me
The same thing like Monkey Prince.
Help.
I’m very sorry, but the above solution worked for me. If you’ve made sure the file exists in the first place and hasn’t been renamed or anything in your distro, then I’m afraid I can’t help you much more, as the above solution works for me.
Touchdown! I use a Nagios server to check the “goodness” of a number of files on Windows servers. Since moving from hard metal to a virtual server the network has been occasionally locking up within minutes of a reboot. I was blaming ntp until I googled the CIFS error message.
I can now get my life back!
It is possible to change this setting automatically when loading the CIFS module. modprobe has a nice feature to execute a script when loading a module. Check man 5 modprobe.conf
Add a line to /etc/modprobe.conf or a file in /etc/modprobe.d containing the following line, this is one single line:
install cifs /sbin/modprobe –ignore-install cifs; echo 0 > /proc/fs/cifs/OplockEnabled ;
Thanks a million for posting this! This has been a real nightmare for me ever since upgrading to 9.10 from 9.04 (the problem didn’t seem to exist in 9.04).
We just put into production a RHEL5 Linux / Oracle server which uses the CIFS client to injest a number of files located on a Win2k8 Server. At 8PM I found similar errors and SAR showed %system cpu at to 99% and staying there until the server had was rebooted 2 hours later. I am guessing this post might have saved my ass..
I should add it was “Cold booted”
My function use cifs protocol to transfer file and it usually occur “CIFS VFS: Send error in read = -11
” error.
I searched the cifs reference and get a ways to resovle it.
http://pserver.samba.org/samba/ftp/cifs-cvs/linux-cifs-client-guide.pdf
I just lower the block of data what i want to write
My function use cifs protocol to transfer file and it usually occur “CIFS VFS: Send error in read = -11
” error.
I searched the cifs reference and get a ways to resovle it.
http://pserver.samba.org/samba/ftp/cifs-cvs/linux-cifs-client-guide.pdf
”
Performance Considerations
Server speed, server disk speed and network speed can constrain the overall performance of a cifs
mount, but in some cases, client client configuration settings can be changed to increase performance:
1) size of file write (wsize). The Linux CIFS client usually sends 56K writes (14 pages) and is
limited to 56K maximum unless mounted forcedirectio.
2) size of file read (rsize). The Linux CIFS client usually sends 16K reads (4 pages). Since CIFS
large network buffers are about 16K in size by default, increasing the rsize would have little
effect unless the setting of module load parameter CIFSMaxBufSize (via insmod) also is
increased.
”
I just lower the block of data what i want to write.
you can use command “modinfo cifs” to get the cifs read or write buffer size and buffersize scope.