Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017 Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
<== Date ==> <== Thread ==>

Subject: RE: Area Detector and high performance NVME devices
From: <ulrik.pedersen@diamond.ac.uk>
To: <engbretson@aps.anl.gov>, <rivers@cars.uchicago.edu>
Cc: tech-talk@aps.anl.gov
Date: Fri, 30 Jun 2017 08:06:53 +0000
If you fancy diving into the rabbit hole of using HDF5 in a multi-threaded environment then this is a good place to start: https://support.hdfgroup.org/HDF5/faq/threadsafe.html

Cheers,
Ulrik

-----Original Message-----
From: Engbretson, Mark S. [mailto:engbretson@aps.anl.gov] 
Sent: 30 June 2017 01:08
To: Rivers, Mark L.
Cc: Pedersen, Ulrik (DLSLtd,RAL,TEC); tech-talk@aps.anl.gov
Subject: Re: Area Detector and high performance NVME devices

But wait . . . It can not work that way. Threads pop in and out of existence all the time. They may or may not use the HDF5 libraries, but if they do, they certainly would not want to have a dependency on code or constructs that could go "poof" at any instant.

What i want is an effect more like just opening 2 or more distinct files at once. If they are doing totally different things, with totally unrelated life cycles, what is the mutex protecting? 

On Jun 29, 2017, at 18:16, Mark Rivers <rivers@cars.uchicago.edu> wrote:

What you propose would not work, because the problem is not in NDFileHDF5.cpp, it is in the underlying HDF5 library.  There are global structures there that you would not be making multiple instances of, so they need to be protected by the mutex.

-----Original Message-----
From: Mark S. Engbretson [mailto:Engbretson@anl.gov]
Sent: Thursday, June 29, 2017 5:05 PM
To: ulrik.pedersen@diamond.ac.uk; Mark Rivers
Cc: tech-talk@aps.anl.gov
Subject: RE: Area Detector and high performance NVME devices

For the sake of argument, what would happen if instead of creating multiple instances of the HDF File plugin, which is actually the exact same entity, I instead cloned the NDFileHDF5.cpp sources and made it a couple of unique plugins named <NDFileHDF5>_A,B,C,and D. I can do this on windows system and force every thread to have its own unique copy everything, so there is no conflicts in non thread safe code just as you would not expect any mutex issues from another process running on the system. 

I know that I should technically be able to use constructs such as  

void *handle1 = dlmopen(LM_ID_NEWLM, "/path/to/library.so", RTLD_NOW); void *handle2 = dlmopen(LM_ID_NEWLM, "/path/to/library.so", RTLD_NOW);

Where the references is to the same shared library, but via difference handles don't conflict.  But there should be easier ways to achieve the same effect. I.e, with multiple file plugins, there are no common anythings that should need that mutex lock or could be trashed.



-----Original Message-----
From: ulrik.pedersen@diamond.ac.uk [mailto:ulrik.pedersen@diamond.ac.uk]
Sent: Thursday, June 29, 2017 9:34 AM
To: Engbretson@anl.gov; rivers@cars.uchicago.edu
Cc: tech-talk@aps.anl.gov
Subject: RE: Area Detector and high performance NVME devices

-- 
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. 
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom


References:
Area Detector and high performance NVME devices Mark S. Engbretson
RE: Area Detector and high performance NVME devices Mark Rivers
RE: Area Detector and high performance NVME devices Mark S. Engbretson
RE: Area Detector and high performance NVME devices Mark Rivers
RE: Area Detector and high performance NVME devices Mark S. Engbretson
Re: Area Detector and high performance NVME devices ulrik.pedersen
RE: Area Detector and high performance NVME devices Mark S. Engbretson
RE: Area Detector and high performance NVME devices ulrik.pedersen
RE: Area Detector and high performance NVME devices Mark Rivers
RE: Area Detector and high performance NVME devices Mark S. Engbretson
RE: Area Detector and high performance NVME devices ulrik.pedersen
RE: Area Detector and high performance NVME devices Mark S. Engbretson
RE: Area Detector and high performance NVME devices Mark Rivers
Re: Area Detector and high performance NVME devices Engbretson, Mark S.

Navigate by Date:
Prev: Re: Area Detector and high performance NVME devices Engbretson, Mark S.
Next: RE: Area Detector and high performance NVME devices Mark Rivers
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
Navigate by Thread:
Prev: Re: Area Detector and high performance NVME devices Engbretson, Mark S.
Next: RE: Area Detector and high performance NVME devices Mark Rivers
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
ANJ, 30 Jun 2017 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·