1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 <2017> 2018 2019 2020 2021 2022 2023 2024 | Index | 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 <2017> 2018 2019 2020 2021 2022 2023 2024 |
<== Date ==> | <== Thread ==> |
---|
Subject: | Area Detector and high performance NVME devices |
From: | "Mark S. Engbretson" <[email protected]> |
To: | "Mark Rivers" <[email protected]>, <[email protected]> |
Date: | Tue, 27 Jun 2017 12:23:59 -0500 |
Mark – I have the adimec camera which generates data at ~2.5 GB/s. I recently got my hands on a newer HP 840 with a HP Turbo Z nvme drive which claims a sustained write speed of 6 GB/S. None of the existing file plugin see any performance increase when writing to this device – I do not think that any are actually write limited. I have modified a raw binary file plugin that I obtained from Keenan Lang that easily sustains the cameras write rate until the device is full. Problem is – Raw data really doesn’t do anyone much good. I was thinking that perhaps a quick solution to my problem might be to change this Raw File plugin to look/act like a disk based fifo or circular buffer. This could collect to the limit of the hardware at full speed, and if someone wanted HDF output, they would just drain this queue at the speed that HDF files are generated. Or is there an easier/better solution? I.e. any way that file plugins can use the new multi-thread model of AD 3.0? I know the HDF files can be generates at very high speeds on Lustre file systems, but this seems to be using parallel HDF5. Is this something that Area Detector supports? |