Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017 Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
<== Date ==> <== Thread ==>

Subject: RE: Area Detector and high performance NVME devices
From: Mark Rivers <rivers@cars.uchicago.edu>
To: "'Mark S. Engbretson'" <Engbretson@anl.gov>, "tech-talk@aps.anl.gov" <tech-talk@aps.anl.gov>
Date: Tue, 27 Jun 2017 18:06:20 +0000

Hi Mark,

 

I am surprised that a raw file plugin is significantly faster than netCDF or HDF5.  I would like to see the tests, and figure out what is actually slowing them down, i.e. is it CPU bound, waiting for a semaphore, etc.?  Can you post actual benchmark results for the different plugins, i.e. frames/s and MB/s?

 

You should not need to do anything special to create a FIFO to buffer images while the disk is busy.  Every areaDetector plugin comes with such a FIFO, i.e. its input queue.  Just increase the QueueSize to be large enough to buffer all the images you need to store in one "burst".  You can also use the CircularBuffer plugin to do this, but it should really not be necessary, that is intended more for "triggered" applications where the buffer is emptied when a trigger condition is satisfied.

 

Mark

 

 

From: Mark S. Engbretson [mailto:Engbretson@anl.gov]
Sent: Tuesday, June 27, 2017 12:24 PM
To: Mark Rivers; tech-talk@aps.anl.gov
Subject: Area Detector and high performance NVME devices

 

Mark –

 

I have the adimec camera which generates data at  ~2.5 GB/s. I recently got my hands on a newer HP 840 with a HP Turbo Z nvme drive which claims a sustained write speed of 6 GB/S. None of the existing file plugin see any performance increase when writing to this device – I do not think that any are actually write limited.  I have modified a raw binary file plugin that I obtained from Keenan Lang that easily sustains the cameras write rate until the device is full.

 

Problem is – Raw data really doesn’t do anyone much good. I was thinking that perhaps a quick solution to my problem might be to change this Raw File plugin to look/act like a disk based fifo or circular buffer. This could collect to the limit of the hardware at full speed, and if someone wanted HDF output, they would just drain this queue at the speed that HDF files are generated.  Or is there an easier/better solution? I.e. any way that file plugins can use the new multi-thread model of AD 3.0?

 

I know the HDF files can be generates at very high speeds on Lustre file systems, but this seems to be using parallel HDF5. Is this something that Area Detector supports?


Replies:
RE: Area Detector and high performance NVME devices Mark S. Engbretson
References:
Area Detector and high performance NVME devices Mark S. Engbretson

Navigate by Date:
Prev: Area Detector and high performance NVME devices Mark S. Engbretson
Next: RE: Area Detector and high performance NVME devices Mark S. Engbretson
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
Navigate by Thread:
Prev: Area Detector and high performance NVME devices Mark S. Engbretson
Next: RE: Area Detector and high performance NVME devices Mark S. Engbretson
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
ANJ, 27 Jun 2017 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·