Experimental Physics and
Industrial Control System

[email protected] · Thu, 30 Oct 2014 16:50:32 -0700 (PDT)

We observe frequent StreamDevice ReplyTimeouts when communicating
through a Moxa serial device server.  Details follow.  We ask
whether this is expected (or can be mitigated) given our conditions.
We wonder whether we should use some other communication method
(USB to serial adaptors, for example).

Jon Jacky
--------------

We have a soft IOC that controls three RS232 serial devices, via
StreamDevice on Asyn communicating over Ethernet to TCP/IP ports on a
Moxa 5610 serial device server.  The devices use a variety of commands but
usually they are just polled about once a second: the IOC sends
a command and the device sends some data back.  Usually each device
responds to a command in a fraction of a second, but frequently
(several times an hour, or more) we observe incidents where no
response arrives before our 5 sec StreamDevice ReplyTimeout is
exceeded.

We monitored RS232 traffic between the Moxa and each serial device and
found each device never fails to respond promptly to a command.  But
each time StreamDevice reports a ReplyTimeout, the command fails to
appear on the Moxa serial port after the IOC sends the command to the
corresponding Moxa TCP/IP port.  We used Wireshark to monitor TCP/IP
traffic between the IOC and the Moxa.  We found that during these
timeout incidents, the IOC sends several TCP Retransmission packets
and TCP Dup ACK packets to the Moxa port, the Moxa port also sends
some TCP Retransmission packets to the IOC.  So it appears there is a
problem in the network or in the Moxa.  We have not noticed any other
unusual network activity or problems during the timeouts, so we
suspect the Moxa.

We see timeouts on all three serial devices, but one device accounts
for most of the traffic (due to a higher data rate and more data).  We
removed the high-traffic device from the Moxa and instead connected it
to a USB port on the same IOC, through a USB to R232 converter.  In this
configuration, we no longer observe timeouts on any of the three
devices, so it appears the traffic from that one device was
interfering with communications for other two also.

The high-volume device communicates at 9600 baud.  In each 1-sec polling
cycle, the IOC sends about 80 characters and the device responds with
about 300 characters.  The two other devices use 1200 baud, commands are
about 10 characters and responses are about 75 characters.  

Is it expected that this amount of traffic should
overwhelm the Moxa?  Is there some other likely explanation for the
timeouts?

We wonder whether we should use some other communication method
(USB to serial adaptors, for example).

PS - This is not the same problem as I reported in "Asyn disconnects,
reconnects to serial device server", in
http://www.aps.anl.gov/epics/tech-talk/2014/msg01460.php, but it does
involve the same devices and the same Moxa.  We only saw that a few
times and we haven't seen any repeats, the present problem is frequent and
ongoing.

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System