EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  <20132014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  <20132014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
<== Date ==> <== Thread ==>

Subject: RE: BEAUTY Archiver and data from Area detector
From: <[email protected]>
To: <[email protected]>, <[email protected]>
Date: Mon, 11 Nov 2013 09:05:57 +0000

Hi Mike,

 

I would recommend simply using one of the many file writing plugins that areaDetector ships with: you can store images in jpeg, tiff, netcdf, nexus and my personal favourite: hdf5. The list of supported formats is actually even longer as the GraphicsMagick plugin allow saving to any format the GraphicsMagick library supports... http://cars9.uchicago.edu/software/epics/NDPluginFile.html

 

Relational databases like mysql does not perform particularly well with constant streaming of image data. Certain file formats will do much better – especially when it is time to scale up a little bit (higher res. camera or higher framerate).

 

Cheers,

Ulrik

 

From: [email protected] [mailto:[email protected]] On Behalf Of Vítek Michal
Sent: 08 November 2013 17:46
To: [email protected]
Subject: BEAUTY Archiver and data from Area detector

 

Hello,

 

We would like to archive data from cameras (grabbed by EPICS IOC with area detector) with BEAUTY archiver. We have prepared testing setup with 1 camera (512x512 img, 25 FPS, grayscale), BEAUTY archiver, and MySQL db. Each block is running on independent computer, interconnection is over 1GB lan. The network traffic measured on the output of the EPICS IOC is ~6MB/s. The image can be displayed by Control System Studio intensity graph without problems.

 

I have also added following param to the BEAUTY configuration file:

 

org.csstudio.platform.libs.epics/max_array_bytes=300000

 

because it was complaining about the size of the incoming image.

 

The memory available for the Java programs is set to 2GB by OS Environmental variable.

 

When we start the archiver we can see that BEAUTY consumes all available resources (2GB of RAM and 99% CPU) in ~5s and then it crashes. When we changed the PV to be scanned every 5s then the BAUTY maintain to run consuming ~1GB of memory, and ~20% CPU and write duration  ~240s.

 

Thus I would like to ask if BEUATY is suitable for the archiving of images from camera or if I am doing something wrong…? Can anyone help me with this?

 

Cheers,

 

Mike


 

-- 

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
 



References:
BEAUTY Archiver and data from Area detector Vítek Michal

Navigate by Date:
Prev: Error while opening OPI Swati Soni2
Next: Re: Subscription updates halting? Dirk Zimoch
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  <20132014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
Navigate by Thread:
Prev: RE: BEAUTY Archiver and data from Area detector Malitsky, Nikolay D
Next: Subscription updates halting? Gibbs, Matt
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  <20132014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
ANJ, 20 Apr 2015 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·