EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  <20102011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  <20102011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
<== Date ==> <== Thread ==>

Subject: Re: archive server discussion
From: Matt Newville <[email protected]>
To: Patrick Thomas <[email protected]>
Cc: "[email protected] Techtalk" <[email protected]>
Date: Wed, 4 Aug 2010 10:15:30 -0500
Hi Patrick,

> Would it be possible to extend this, or something similar, to monitor and
> record changes for tens or hundreds of thousands of PVs in a mySQL database?

I think it is possible.  I'm currently archiving ~5000 PVs, but that's
not limited by what the system can handle, but just how many PVs we
actually want to monitor.  But to be honest, the majority of the PVs
we monitor change very slowly.

My concern for scaling up would be that you might really want to use
multiple threads or processes.  Currently, the EpicsArchiver code uses
the older EpicsCA python interface, which is single-threaded and uses
non-preemptive callbacks.  I have not yet converted this to pyepics3
(which lifts both restrictions), though that's on my ToDo list.  I
believe that allowing multiple collection threads could be be
important for handling a "huge" number of PVs, though I'm not sure
what "huge" is.

I think that whether mysql can handle the load is mostly a question of
hardware and mysql configuration, but I doubt that mysql would be the
limiting factor.

Cheers,

--Matt Newville <newville at cars.uchicago.edu> 630-252-0431

References:
archive server discussion james.rowland
Re: archive server discussion Matt Newville
Re: archive server discussion Patrick Thomas

Navigate by Date:
Prev: medm Mezger Anton-Chr.
Next: caPutLog Performance Gary Carr
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  <20102011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
Navigate by Thread:
Prev: Re: archive server discussion Patrick Thomas
Next: archive server discussion Loris Giovannini
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  <20102011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
ANJ, 02 Sep 2010 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·