EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <20172018  2019  2020  2021  2022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <20172018  2019  2020  2021  2022  2023  2024 
<== Date ==> <== Thread ==>

Subject: Re: How to use Archiver Appliance to archive PVs of only the latest period of time
From: "Shankar, Murali" <[email protected]>
To: "Wang, Lin" <[email protected]>
Cc: "[email protected]" <[email protected]>
Date: Mon, 30 Oct 2017 13:49:50 +0000

>> Should I use a Python script to directly operate the .pb files in the file system, or to interact with the Archiver Appliance BPL Servlet/HTTP interface


You can directly operate on the .pb files in the file system. As long as the appliance is not writing to these files; which it would not if the files are for older partitions; you can safely delete the files from the file system using os.remove.



Regards,

Murali




From: Wang, Lin <[email protected]>
Sent: Monday, October 30, 2017 3:30 AM
To: Shankar, Murali
Cc: [email protected]
Subject: Re: How to use Archiver Appliance to archive PVs of only the latest period of time
 
Hello Murali,

Sorry for following up this thread so late.

> You could then delete the older data using a Python script.
But how should I delete the older data using a Python script? Should I use a Python script to directly operate the .pb files in the file system, or to interact with the Archiver Appliance BPL Servlet/HTTP interface?


Thanks,
Lin


-----Original Messages-----
From:"Shankar, Murali" <[email protected]>
Sent Time:2017-09-14 12:04:32 (Thursday)
To: "[email protected]" <[email protected]>, "[email protected]" <[email protected]>
Cc:
Subject: Re: How to use Archiver Appliance to archive PVs of only the latest period of time

Hi Lin,


>> In order to save storage space, we just want to archive PVs of the latest one month.


You should be able to create a policy for these 5000 PV's with a partition granularity of a month. That is, in your policies.py file, your LTS (or MTS) can be something like

pb://localhost?name=LTS&rootFolder=${ARCHAPPL_LONG_TERM_FOLDER}&partitionGranularity=PARTITION_MONTH.


If you add PV's to the archiver using this policy, you should be able to store data in chunks of one month each. 



>> the requirement may be adjusted to three or six months in future)

You could then delete the older data using a Python script. Or, if you decided to keep, let's say four months you could add a blackhole plugin after this plugin. For example, 

LTS -> partitionGranularity=PARTITION_MONTH​&hold=5&gather=1 followed by a 

Final -> 'blackhole://localhost'

would keep at least (5-1)=4 months data in your LTS.


For more details, please see http://slacmshankar.github.io/epicsarchiver_docs/api/edu/stanford/slac/archiverappliance/PlainPB/PlainPBStoragePlugin.html


Hope that helps.


Regards,

Murali





References:
Re: How to use Archiver Appliance to archive PVs of only the latest period of time Shankar, Murali
Re: How to use Archiver Appliance to archive PVs of only the latest period of time Wang, Lin

Navigate by Date:
Prev: Re: streamdevice and AD driver in same IOC Mark Rivers
Next: Re: streamdevice and AD driver in same IOC Hinko Kocevar
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <20172018  2019  2020  2021  2022  2023  2024 
Navigate by Thread:
Prev: Re: How to use Archiver Appliance to archive PVs of only the latest period of time Wang, Lin
Next: MEDM display of string records not updating from PCASpy/PyEpics keith.thorne
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <20172018  2019  2020  2021  2022  2023  2024 
ANJ, 21 Dec 2017 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·