Experimental Physics and
Industrial Control System

Mark Rivers <[email protected]> · Thu, 30 May 2013 20:37:58 +0000

Lewis,

Is your NFS server a Mac?

I used to boot the VME crates at NSLS from a Mac, and I saw exactly the problem you did with "ls" not working.

Mark

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of J. Lewis Muir
Sent: Thursday, May 30, 2013 3:07 PM
To: Andrew Johnson
Cc: [email protected]
Subject: Re: autosave failure S_rpcLib_RPC_CANTSEND

On 5/29/13 5:43 PM, Andrew Johnson wrote:
> Hi Lewis,
> 
> On 2013-05-29 J. Lewis Muir wrote:
>> The error looks like it's coming from the autosave module.  I'm guessing
>> maybe the RPC error suggests something wrong with communicating with the
>> NFS server where the auto_settings.sav file is being written, but I
>> don't know.  The NFS server log doesn't show any problem.  Once the
>> problem starts, it doesn't seem to go away until I reboot the IOC.
> 
> Once it has started, does an 'ls' (or any other file operation) at the IOC 
> console work against the NFS disk?

Hi, Andrew.

I don't know; I've never tried that.  I'll try it the next time it happens.

Well, hold that thought.  I just tried an 'ls' right now on the IOC
while it is not exhibiting the RPC problem, and it does not work.
Here's the output:

=== console ===
ioc> ls
imcais-sim.template
error reading dir , errno: 0x300016
value = -1 = 0xffffffff
===============

There are actually 45 entries (files or directories) in the current
working directory, and it only listed one of them (imcais-sim.template)
before reporting an error.

>> This problem has not always existed; it used to work fine.  I initially
>> thought it was a problem with autosave 5-0, since it seemed the problem
>> showed up for the first time after I had upgraded to 5-0.  But then I
>> rolled back to 4-8, and the problem is still appearing.  It's possible
>> the NFS server changed somehow, or maybe a network adapter, cable,
>> switch, etc. is failing, or none of these.
> 
> We have seen other problems ('ls' failing) against a Linux NFS server when 
> using NFSv3, but this problem goes away if we revert back to NFSv2.  You can 
> select the NFS version if you're mounting the drive manually, just use the 
> nfs2Mount (or nfs3Mount) command instead of nfsMount.  However autosave calls 
> nfsMount automatically in some cases which Tim Mooney can explain (probably 
> related to I/O failures) and that could undo the manual mount.

Interesting.

> I would prefer to fix the problem inside the NFSv3 code, but that could take 
> some time so in the meanwhile I've produced some OS images that don't contain 
> the NFSv3 client.  I can send you a copy of the mv5100 image if you want to 
> see if that helps.

Using nfsstat on the NFS server, I can confirm that the IOC NFS client
is using NFSv3.  Yes, if you'd be willing, I would love it if you could
send that image!  I'd like to try that to see if it helps.

Thanks!

Lewis

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System