Experimental Physics and
Industrial Control System

[email protected] · Fri, 16 Sep 2011 13:09:55 -0700

> pvlist = []
> for pvName in pvNames:
>     pvlist.append( epics.PV(pvName) )
> 
> for pvs in pvlist:
>     val = pv.get()
>     <store value>
> 
> With this sort of approach, I typically see on the order of 10ms per
> PV connection on startup.   That is, if I create and connect to 5K
> PVs, it takes ~50 seconds (meaning 40 to 80 seconds) to get initial
> values.  I believe that is mostly the CA library, not the python part,
> and I believe it would scale, suggesting that any save/restore process
> that runs once and then quit would take 10 minutes for 50K PVs.

Are you saying that 10 min for 50K PVs is also what is observed with C utilities?
10 min for 50K is rather long ...
If that is the case, then one whould be careful to backup only PV which are of interest
and not the 'entire' machine. 
How is this being managed at large installation?

> My understanding from a little playing with Cython is that it is very
> good at turning the slow parts of Python to C-like performance.  I
> think the "slow Python parts" for this code are
>  a) creating a list of 50k  names / PVs -- not that big a cost
> 
>  b) creating the 50k PV object -- shouldn't be that big, as it creates
> a CHID and asks for a connection callback -- which will not prevent
> you from going to the next python statement.
> 
>  c) the python parts of the actual PV.get() , which shouldn't be that
> big, especially for scalar values, and especially assuming that the
> connection has been established.

That's still 10 min for 50K PVs...

> 
> I think these are not very slow, though there might be some python GIL
> issues with having all those different bits of code run from the CA
> connection callbacks -- I haven't noticed a problem, but I haven't
> pushed this hard on it either.    I think most of the real time would
> be spent in the initial network connection part of the CA library --
> which is to say that I'd guess the speed of this loop would not be
> that much slower than the C equivalent.

I understand your points, but somehow I am having a hard time believing this.
Did anyone benchmark their backup platform?
(I remember seeing a few powerpoints on this issue a while back...)

Thank you for your input.

PS: 
Is there a difference between a connection to 50K Pvs versus, 50K connections to the same PVs?
That would make the generation of my test request file quite simple...
Just wondering

Thanks again.

-- 
Emmanuel

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System