Experimental Physics and
Industrial Control System

Greg Lawson <[email protected]> · Thu, 29 Mar 2007 15:47:08 -0400

Hi Maren,

Just curious to know how fast you can get data from the disk to the LTO
2 drives while saving more data to the disk, and serving yesterdays data
to users from the disk...  

Sounds like trouble to me.

Greg Lawson
SNS Controls 

On Thu, 2007-03-29 at 09:03 -1000, Maren Purves wrote:
> Kay,
> 
> (only responding to the data volume problem)
> 
> On Thu, 29 Mar 2007, Kay-Uwe Kasemir wrote:
> > On Mar 29, 2007, at 02:04 , Terry Cornall wrote:
> >> I'm trying to dimension the archive requirements for all the 14 beamlines 
> >> for the Australian Synchrotron and am wondering if I can do it
> >> with one PC or if I need many. Of course, sampling or monitoring rates will 
> >> play a big part, but also I need to know what is possible per archive 
> >> engine.
> >
> > Problem 2:
> > The amount of data. One 'double' sample uses a little over
> > 20 bytes for the timestamp, status, sev, value.
> > For many values, the data file structure and index add
> > relatively little to that, but you'll still get about 20GB
> > per day. How do you intend to back that up?
> 
> [...]
> 
> > If somebody tells you that disks are cheap, please ask
> > that person to take care of your archiving, then run
> > as fast as you can.
> 
> you got a point, but I'm not sure that's what Terry asked.
> 
> And besides, with our wide field camera (motor control is
> EPICS, array control and data acquisition isn't) that has
> 4 arrays we get over 50GB per array in a good night with
> short integrations. With all 4 arrays we can get well over
> a Terabyte in a week.
> 
> Yes, it's an archiving problem, but it's entirely doable.
> LTO 2 drives in our case generally hold about 300 GB of
> compressed data (depends on how compressible your data
> are, that's why 'in our case').
> 
> Aloha,
> Maren
> 

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System