Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017 Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
<== Date ==> <== Thread ==>

Subject: Re: How to use Archiver Appliance to archive PVs of only the latest period of time
From: "Shankar, Murali" <mshankar@slac.stanford.edu>
To: "Wang, Lin" <wanglin@ihep.ac.cn>
Cc: "tech-talk@aps.anl.gov" <tech-talk@aps.anl.gov>
Date: Mon, 30 Oct 2017 13:49:50 +0000

>> Should I use a Python script to directly operate the .pb files in the file system, or to interact with the Archiver Appliance BPL Servlet/HTTP interface


You can directly operate on the .pb files in the file system. As long as the appliance is not writing to these files; which it would not if the files are for older partitions; you can safely delete the files from the file system using os.remove.



Regards,

Murali




From: Wang, Lin <wanglin@ihep.ac.cn>
Sent: Monday, October 30, 2017 3:30 AM
To: Shankar, Murali
Cc: tech-talk@aps.anl.gov
Subject: Re: How to use Archiver Appliance to archive PVs of only the latest period of time
 
Hello Murali,

Sorry for following up this thread so late.

> You could then delete the older data using a Python script.
But how should I delete the older data using a Python script? Should I use a Python script to directly operate the .pb files in the file system, or to interact with the Archiver Appliance BPL Servlet/HTTP interface?


Thanks,
Lin


-----Original Messages-----
From:"Shankar, Murali" <mshankar@slac.stanford.edu>
Sent Time:2017-09-14 12:04:32 (Thursday)
To: "tech-talk@aps.anl.gov" <tech-talk@aps.anl.gov>, "wanglin@ihep.ac.cn" <wanglin@ihep.ac.cn>
Cc:
Subject: Re: How to use Archiver Appliance to archive PVs of only the latest period of time

Hi Lin,


>> In order to save storage space, we just want to archive PVs of the latest one month.


You should be able to create a policy for these 5000 PV's with a partition granularity of a month. That is, in your policies.py file, your LTS (or MTS) can be something like

pb://localhost?name=LTS&rootFolder=${ARCHAPPL_LONG_TERM_FOLDER}&partitionGranularity=PARTITION_MONTH.


If you add PV's to the archiver using this policy, you should be able to store data in chunks of one month each. 



>> the requirement may be adjusted to three or six months in future)

You could then delete the older data using a Python script. Or, if you decided to keep, let's say four months you could add a blackhole plugin after this plugin. For example, 

LTS -> partitionGranularity=PARTITION_MONTH​&hold=5&gather=1 followed by a 

Final -> 'blackhole://localhost'

would keep at least (5-1)=4 months data in your LTS.


For more details, please see http://slacmshankar.github.io/epicsarchiver_docs/api/edu/stanford/slac/archiverappliance/PlainPB/PlainPBStoragePlugin.html


Hope that helps.


Regards,

Murali





References:
Re: How to use Archiver Appliance to archive PVs of only the latest period of time Shankar, Murali
Re: How to use Archiver Appliance to archive PVs of only the latest period of time Wang, Lin

Navigate by Date:
Prev: Re: streamdevice and AD driver in same IOC Mark Rivers
Next: Re: streamdevice and AD driver in same IOC Hinko Kocevar
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
Navigate by Thread:
Prev: Re: How to use Archiver Appliance to archive PVs of only the latest period of time Wang, Lin
Next: MEDM display of string records not updating from PCASpy/PyEpics keith.thorne
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
ANJ, 30 Oct 2017 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·