[Xrays@aps.anl.gov] 2006 XSD Scientific Software Workshop User Survey

[Stefan Vogt]

Dear all,

here are my comments regarding your email on scientific software. I 
would like to point out that they are from the perspective of what I am 
most interested in, x-ray fluorescence microscopy.

Before addressing specifically the points that are raised below, I would 
like to make a more general, but possibly somewhat to the side comment. 
It is appreciated that efforts are being put into development of 
'scientific' software. However, I would like to point out that there is 
'room for improvement' in the software that is currently being used to 
run the beamlines, in terms of the efficiency of setting up scans, as 
well as the ability to view acquired data. It seems ironic that one can 
set up a 5 dimensional scan, but that (to the best of my knowledge) no 
software exists to actually view the acquire data. Or that I can find a 
very irregular structure that I wish to scan, but a scan that I will 
have to set up, ends up scanning mostly empty space, and wasting beamtime.

Both have direct impacts on the efficiency of operation.


> In order to prepare for this workshop we would like your input on
> what you see as the needs and opportunities for scientific
> software development at the APS and in the X-ray community, as

Personally, I believe that as the x-ray communities matures, and the 
science applications are done in a fairly routine user mode, we have to 
not only have an optimized beamline, but must also support users in data 
analysis, so that user's go home with as much pre-analysed data as 
possible. Emphasis should also be placed on useability. I believe both 
are particularily true for users from communities that do NOT have a 
physics background, etc (e.g., users from the life sciences).

> well as information that would support making a funding proposal
> for such resources.  In particular:
> 
> 1. What are the limitations of current tools for
>   x-ray data reduction, analysis, modeling, and simulation?
we acquire 'raw' data as spectra with counts/s, what our user's want in 
a first approximation is quantified data, reduced to elemental 
concentration (e.g., mmol Zn). Then, one should be able to analyse large 
quantities of acquired data semi-automatically, and report content for 
specific regions or areas across different scanned samples.

This involves per pixel fitting of spectra, possibly multivariate 
analysis, cluster analysis, comparison to a claibration curve, etc. 
Currently, I am not aware of any tool that would allow one to do all of 
this, in particular not across the different beamlines.
For sector 2, I have written an IDL based software to do some of this, 
but it is limited in that it requires the use of certain standards, and 
is not routinely useable at other beamlines.

> 2. What additional tools are needed?

APS has a fairly large number of microfluorescence beamlines, either 
present, or in planning/contruction. It would seem a very good idea if 
there was a supported set of software tools available / developped, that 
would make it possible to perform the tasks outlined above, and convert 
the acquired, raw data into 'useable', pre-analysed data for users.

As a next step, for each type of experiment, one should have a more 
specialised set of tools that allows one to further process the 
pre-analysed data, e.g., segment each acquired scan into specific 
(similar) areas, and automatically compare different areas for different 
sample conditions/treatments. This could be a cluster analysis or neural 
network type method that would be able to work on scans of cells, and 
seperate them into different characteristic subunits (e.g., background, 
cytoplama, nucleus), and compare these across different treatments.
For example, a typical user might be comeing in and ask, O.K., I have 
the two cell lines, how is their elemental content influenced by, and 
how do they react differently to, a treatment with this therapeutic 
drug. The user will probably not care about that one needs to fit the 
data in the first place, compare it to a calibration curve, find 
corresponding regions of the cell, extract that data, and then compare 
the quantified data from one treatment to the next. All the user is 
really interested in is in the result.

Additionally, tools that allow simulation of data from specimens, to 
investigate feasiblity of certain studies without the need to spend 
actual beamtime on it, should be available.

> 3. How can the existing tools be improved?
methods and software pieces for most of the required functionality is 
available somewhere, but it is not unified in any way. I believe one 
should instead have one COMMON piece of software that is useable across 
the different beamlines that have similar needs. A user should be able 
to use the same software tool (though possibly with different options), 
for data that was acquired, no matter whether it was acquired in sector 
2, 18 or 20; or, to go one step further, at APS, ESRF, or elsewhere.

> 
> 4. What will most affect the scientific impact of your work?

accurate analysis of acquired data is a must if we want to remain 
competitive. As the throughput of our beamlines increases, and the 
quality of the data improves, the analysis part is more and more of a 
bottleneck. As a first step, I believe that (semi)-automated data 
(pre)-analysis will be crucial (raw data -> pre-analysed).
The next step is then haveing more sophisticated routines that allows 
further processing with as little as possible user interaction, to 
tackle large datasets (e.g., compare cytoplasmatic vs nuclear elemental 
content of 100 cells that were imaged during the last run).


Cheers,
Stefan

-- 
Dr. Stefan Vogt,
Experimental Facilities Division
Advanced Photon Source
Argonne National Laboratory
phone: (630) 252-3071; beamline: -3711
fax: (630) 252-0140
e-mail: vogt at aps.anl.gov
http://www.stefan.vogt.net/