Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017 Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
<== Date ==> <== Thread ==>

Subject: areaDetector and ADCore R3-0 released
From: Mark Rivers <>
To: " >> EPICS Tech Talk" <>
Date: Fri, 5 May 2017 18:40:02 +0000

New releases of the following areaDetector modules are now available. 


areaDetector R3-0

ADCore R3-0

ADSupport R1-2

ADSimDetector R2-5



These are the R3-0 release notes for areaDetector:


* Requires ADCore R3-0

* Requires asyn R4-31 or higher

* Requires EPICS base or higher



These are the major items from the R3-0 release notes for ADCore:


### Requirements

* This release requires EPICS base or higher because it uses the CFG rules which were fixed

  in that release.


### Incompatible changes

* This release is R3-0 rather than R2-7 because a few changes break backwards compatibility.

  * The constructors for asynNDArray driver and NDPluginDriver no longer take a numParams argument.

    This takes advantage of the fact that asynPortDriver no longer requires parameter counting as of R4-31.

    The constructor for ADDriver has not been similarly changed because this would require changing all drivers,

    and it was decided to wait until future changes require changing drivers before doing this.

  * The constructor for NDPluginDriver now takes an additional maxThreads argument.

  * NDPluginDriver::processCallbacks() has been renamed to NDPluginDriver::beginProcessCallbacks().

  * These changes will require minor modifications to any user-written plugins and any drivers that are derived directly

    from asynNDArrayDriver.

  * All of the plugins in ADCore, and other plugins in the areaDetector project

    (ADPluginEdge, ffmpegServer, FastCCP, ADPCO, ADnED) have had these changes made.

  * The constructors and iocsh configuration commands for NDPluginStdArrays and NDPluginPva have been changed

    to add the standard maxBuffers argument.  EXAMPLE_commonPlugins.cmd has had these changes made.

    Local startup scripts may need modifications.


### Multiple threads in single plugins (NDPluginDriver, NDPluginBase.template, NDPluginBase.adl, many plugins)

* Added support for multiple threads running the processCallbacks() function in a single plugin.  This can improve

  the performance of the plugin by a large factor.  Linear scaling with up to 5 threads (the largest

  value tested) was observed for most of the plugins that now support multiple threads.

  The maximum number of threads that can be used for the plugin is set in the constructor and thus in the

  IOC startup script.  The actual number of threads to use can be controlled via an EPICS PV at run time,

  up to the maximum value passed to the constructor.

  Note that plugins need to be modified to be thread-safe for multiple threads running in a single plugin object.


### NDPluginScatter

* New plugin NDPluginScatter is used to distribute (scatter) the processing of NDArrays to multiple downstream plugins.

  It allows multiple intances of a plugin to process NDArrays in parallel, utilizing multiple cores

  to increase throughput. It is commonly used together with NDPluginGather, which gathers the outputs from

  multiple plugins back into a single stream.

  This plugin works differently from other plugins that do callbacks to downstream plugins.

  Other plugins pass each NDArray that they generate of all downstream plugins that have registered for callbacks.

  NDPluginScatter does not do this, rather it passes each NDArray to only one downstream plugin.

  The algorithm for chosing which plugin to pass the next NDArray to can be described as a modified round-robin.

  The first NDArray is passed to the first registered callback client, the second NDArray to the second client, etc.

  After the last client the next NDArray goes to the first client, and so on. The modification to strict round-robin

  is that if client N input queue is full then an attempt is made to send the NDArray to client N+1,

  and if this would fail to client N+2, etc. If no clients are able to accept the NDArray because their queues are

  full then the last client that is tried (N-1) will drop the NDArray. Because the "last client" rotates according

  to the round-robin schedule the load of dropped arrays will be uniform if all clients are executing at the same

  speed and if their queues are the same size.


### NDPluginGather

* New plugin NDPluginGather is used to gather NDArrays from multiple upstream plugins and merge them into a single stream.

  When used together with NDPluginScatter it allows multiple intances of a plugin to process NDArrays

  in parallel, utilizing multiple cores to increase throughput.

  This plugin works differently from other plugins that receive callbacks from upstream plugins.

  Other plugins subscribe to NDArray callbacks from a single upstream plugin or driver.

  NDPluginGather allows subscribing to callbacks from any number of upstream plugins.

  It combines the NDArrays it receives into a single stream which it passes to all downstream plugins.

  The EXAMPLE_commonPlugins.cmd and medm files in ADCore allow up to 8 upstream plugins, but this number can

  easily be changed by editing the startup script and operator display file.


### NDPluginAttrPlot

* New plugin that caches NDAttribute values for an acquisition and exposes values of the selected ones to the EPICS

   layer periodically.  Written by Blaz Kranjc from Cosylab.  No documentation yet, but it should be coming soon.


### asynNDArrayDriver, NDFileNexus

* Changed XML file parsing from using TinyXml to using libxml2.  TinyXml was originally used because libxml2 was not

  available for vxWorks and Windows.  libxml2 was already used for NDFileHDF5 and NDPosPlugin, originally using pre-built

  libraries for Windows in ADBinaries.  ADSupport now provides libxml2, so it is available for all platforms, and

  there is no need to continue building and using TinyXml.  This change means that libxml2 is now required, and so

  the build option WITH_XML2 is no longer used.  XML2_EXTERNAL is still used, depending on whether the version

  in ADSupport or an external version of the library should be used.  The TinyXml source code has been removed from


* Added support for macro substitution in the XML files used to define NDAttributes.  There is a new NDAttributesMacros

  waveform record that contains the macro substitution strings, for example "CAMERA=13SIM1:cam1:,ID=ID34:".

* Added a new NDAttributesStatus mbbi record that contains the status of reading the attributes XML file.

  It is used to indicate whether the file cannot be found, if there is an XML syntax error, or if there is a

  macro substitutions error.


### PVAttribute

* Fixed a race condition that could result in PVAttributes not being connected to the channel.  This was most likely

  to occur for local PVs in the areaDetector IOC where the connection callback would happen immediately, before the

  code had been initialized to handle the callback. The race condition was introduced in R2-6.


### NDFileHDF5

* Fixed a problem with PVAttributes that were not connected to a PV.  Previously this generated errors from the HDF5

  library because an invalid datatype of -1 was used.  Now the data type for such disconnected attributes is set to

  H5T_NATIVE_FLOAT and the fill value is set to NAN.  No data is written from such attributes to the file, so the

  fill value is used.


### Viewers

* The ADCore/Viewers directory containing the ImageJ and IDL viewers has been moved to its own

[ADViewers repository](

* It now contains a new ImageJ EPICS_NTNDA_Viewer plugin written by Tim Madden and Marty Kraimer.

  It is essentially identical to except that it displays NTNDArrays from the NDPluginPva plugin,

  i.e. using pvAccess to transport the images rather than NDPluginStdArrays which uses Channel Access.

* has been changed to work with the new ProcessPlugin feature in NDPluginDriver by monitoring

  ArrayCounter rather than UniqueId.




These are the R1-2 release notes for ADSupport:


* Updated HDF5 to 1.10.1 which was officially released on May 1, 2017.

  This release fixes the SWMR problems and adds some new performance features.

* Updated Nexus to release 4.4.3.  This fixes some problems on 64-bit systems.

* Modified XML2 so it will build on vxWorks 5.5.  ADCore previously contained and used TinyXml,

  but now requires libxml2, so it needs to build on all platforms.




These are the R2-4 release notes for ADSimDetector:


* Removed parameter counting from driver, not needed with asyn R4-31.

* Added room for new PVs for attribute file status and macros in medm screen.

* Moved NDPluginFFT loading from this IOC to EXAMPLE_commonPlugins.cmd.

* Added new required maxBuffers argument when creating NDPluginStdArrays in simDetectorNoIOC.cpp.




The home page is here:


The top-level documentation is here:


Installation instructions are here:


areaDetector release notes are here:


ADSupport release notes are here:


ADCore release notes are here:


ADSimDetector release notes are here:






Navigate by Date:
Prev: Re: MEDM Close Display Lang, Keenan C.
Next: Re: MEDM Close Display Elmer Pensack
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
Navigate by Thread:
Prev: Re: MEDM Close Display Kevin Peterson
Next: Troubleshoot "Appliance Down" Issue in EPICS Archiver Appliance Abdalla Ahmad
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  <2017
ANJ, 07 May 2017 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·