Experimental Physics and
Industrial Control System

"Jeff Hill" <[email protected]> · Tue, 13 Mar 2007 11:21:08 -0600

> we still have one large switched network with several thousand nodes.

One option is to install an inexpensive firewall/router supporting NAT to
broker access between your control system and the "large switched network
with several thousand nodes". The benefits of NAT are many including freedom
to allocate your own IP addresses while still retaining access to the
interconnected internet (with NAT the firewall can substitute an inside IP
address for a dynamically allocated outside IP address whenever the frame
moves through the firewall). A CA GW can also provide transparent read only
access to the control system from outside the FW.

When presented with a description of the many benefits of a firewall for
your control system your management/network/it staff will probably be
amenable to the idea.

Jeff

> -----Original Message-----
> From: Burkhard Kolb [mailto:[email protected]]
> Sent: Tuesday, March 13, 2007 7:34 AM
> To: Jeff Hill
> Cc: [email protected]
> Subject: Re: Darwin and EPICS_CA_AUTO_ADDR_LIST issue
> 
> Jeff Hill wrote:
> >> I use the same list of IOCs on other linux boxes and have no problem
> >> there.
> >
> > This might be because what UDP responses are discarded off the end of
> the
> > finite length UDP queue might be highly dependent on what hosts manage
> to
> > jamb their responses in first - and therefore on the topology of the
> > network.
> >
> >> Anyhow this IOC is in the list but is actually really down - on
purpose.
> >
> > So, I still think it would be good idea to peek under the hood at the
> ICMP
> > traffic in case something unusual is going on (its not hard to do), but
> this
> > may only reveal that there is one ICMP response reflected off of each
> host
> > that isn't currently available.
> >
> >> We run a flat switched network.
> >
> > This probably means of course that the control system is all contained
> > within one subnet? If so then perhaps you can get by using ordinary
> > broadcasted search requests? Listing each host in the EPICS_CA_ADDR_LIST
> > certainly does have the disadvantage of competition from ICMP responses
> when
> > the routing system knows that the host that is being sent a search
> message
> > is unreachable. One wouldn't generally have that specific problem with
> > broadcast based search messages.
> >
> > Do you have an important reason to list the hosts individually in the
> > EPICS_CA_ADDR_LIST?
> >
> Well...
> As our network people still refuse to have routed networks here we have
> one large switched network with several thousand nodes. Many of them
> constantly produce broadcast messages contributing to quite some load.
> My concern was really the difference between Macs and the standard linux
> boxes here, where I don't see this behaviour.
> Maybe some of the Mac enthusiasts out there have an idea?
> 
> > Are you listing specific hosts in the EPICS_CA_ADDR_LIST in order to
> avoid
> > contacting certain hosts on the network? If so, one solution might be to
> use
> > a different UDP port for a subsystem of the overall control system. You
> > could possibly also use a ca gateway to selectively couple between
> control
> > subsystems operating on different ports. This might transparently
> provide
> > the appearance of one control system operating on a single port to
> certain
> > clients even though there are actually multiple underlying control
> systems
> > operating on independent ports.
> >
> > Multicast groups might provide a better way to manage grouping of hosts
> > independent of subnets - in contrast to using different UDP ports. This
> > isn't implemented yet but it's probably not a large amount of
> programming to
> > add that feature. The primary benefit would be avoidance of
> configuration
> > complexity - especially if the control system crosses over multiple
> subnets.
> >
> > Jeff
> >
> >> -----Original Message-----
> >> From: Burkhard Kolb [mailto:[email protected]]
> >> Sent: Friday, March 09, 2007 2:02 AM
> >> To: Jeff Hill
> >> Cc: [email protected]
> >> Subject: Re: Darwin and EPICS_CA_AUTO_ADDR_LIST issue
> >>
> >> Jeff Hill wrote:
> >>>> On my MAC (OSX 10.4.8), EPICS base-3.14.9, medm 3.11:
> >>>>
> >>>> When I set EPICS_CA_AUTO_ADDR_LIST to NO and have the
> >> EPICS_CA_ADDR_LIST
> >>>> pointing to the list of IOCs medm does not find all PVs. Sometimes
> the
> >>>> connections work for a short time then several IOCs are not seen
> >> anymore.
> >>>> If I set it to YES, medm finds them always.
> >>>>
> >>>> In the xterm window I get some error messages from really not
> existing
> >>>> IOCs:
> >>>> CAC: error = "Host is down" sending UDP msg to 140.181.98.50:5064
> >>>> But the lost ones are not reported!
> >>>>
> >>>> I use the same list of IOCs on other linux boxes and have no problem
> >>> there.
> >>>
> >>> What is the response to "ping -s 140.181.98.50"?
> >> ping: invalid packet size: `140.181.98.50'
> >> Did you mean -R or -r?
> >> Anyhow this IOC is in the list but is actually really down - on
purpose.
> >> But most of the other ones are VxWorks 68040 cpus all running the same
> >> kernel, same setup...
> >> What bothers me most, is that sometimes when I start a medm with a
> >> screen where vxStats infos from the IOCs is displayed, the IOCs show up
> >> and then drop out - in the case when EPICS_CA_AUTO_ADDR_LIST=NO.
> >>> PS: Behavior is known to vary between IP kernels for such unicast
> >> addresses
> >>> if there are two or more daemons (in this case two or more CA UDP
> >> servers)
> >>> listening to the same {IP address, port} tuple on the same host. The
> >> typical
> >>> behavior difference is that on certain IP kernels only one daemeon
> >> listening
> >>> on UDP port 5064 will receive a UDP message with a unicast destination
> >>> address, but in contrast on other IP kernels all daemeons listening on
> >> UDP
> >>> port 5064 will receive such messages. In my experience UDP frames with
> >>> broadcast address destinations always go to all registered listeners
> >>> listening on UDP port 5064 - a uniform behavior across IP kernel
> >>> implementations.
> >>>
> >>> PPS: One possible workaround to the above dilemma is to use directed
> >>> broadcast addresses (router forwarded broadcast addresses) in the
> >>> EPICS_CA_ADDR_LIST. This sometimes requires enabling of "broadcast
> >>> forwarding" features in your routers. We might also add support for
> >>> multicasting to future versions of CA.
> >>>
> >> We run a flat switched network.
> >>
> >>> Jeff
> >>>
> >>>> -----Original Message-----
> >>>> From: Burkhard Kolb [mailto:[email protected]]
> >>>> Sent: Thursday, March 08, 2007 9:28 AM
> >>>> To: '[email protected]'
> >>>> Subject: Darwin and EPICS_CA_AUTO_ADDR_LIST issue
> >>>>
> >>>> On my MAC (OSX 10.4.8), EPICS base-3.14.9, medm 3.11:
> >>>>
> >>>> When I set EPICS_CA_AUTO_ADDR_LIST to NO and have the
> >> EPICS_CA_ADDR_LIST
> >>>> pointing to the list of IOCs medm does not find all PVs. Sometimes
> the
> >>>> connections work for a short time then several IOCs are not seen
> >> anymore.
> >>>> If I set it to YES, medm finds them always.
> >>>>
> >>>> In the xterm window I get some error messages from really not
> existing
> >>>> IOCs:
> >>>> CAC: error = "Host is down" sending UDP msg to 140.181.98.50:5064
> >>>> But the lost ones are not reported!
> >>>>
> >>>> I use the same list of IOCs on other linux boxes and have no problem
> >>> there.
> >>>> Any idea?
> >>>> --
> >>>> -------------------------------------------------------------------
> >>>> Dr. Burkhard Kolb
> >>>> GSI mbH  |   KP1   |  Planckstr. 1  |  D-64291 Darmstadt
> >>>> Email: [email protected]                |  Tel.: +49 (0)6159 / 71 2667
> >>>> -------------------------------------------------------------------
> >>
> >> --
> >> -------------------------------------------------------------------
> >> Dr. Burkhard Kolb
> >> GSI mbH  |   KP1   |  Planckstr. 1  |  D-64291 Darmstadt
> >> Email: [email protected]                |  Tel.: +49 (0)6159 / 71 2667
> >> -------------------------------------------------------------------
> >
> 
> 
> --
> -------------------------------------------------------------------
> Dr. Burkhard Kolb
> GSI mbH  |   KP1   |  Planckstr. 1  |  D-64291 Darmstadt
> Email: [email protected]                |  Tel.: +49 (0)6159 / 71 2667
> -------------------------------------------------------------------

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System