The Advanced Photon Source
a U.S. Department of Energy Office of Science User Facility

ORNL Cluster Help

by Jon Tischler

Connect to the Server

The first thing that you will need to do is to connect to the server, otherwise you are just talking to yourself. So go to the 'Server' menu and select one of the listed servers. You will probably want hpcs34, but if you are running the server on your own computer, then you can choose localhost.

Note, you can quit this program at any time and restart & reconnect to the server without affecting any jobs executing on the server in any way.

What is the server doing now?


At the top of the main dialog, the status of the server will be shown. Red means no connection, green means the server is idle, and blue means that the server is busy.

If the server is running (blue), then you can see what is happening by selecting "Server → View Queue on Server". This will put up a window showing what is happening. If nothing is running, then this menu item is disabled.
Setting Up a Job

To actually do some computations you have to provide all sorts of information to let the server know what to do and how to do it. There are various ways to provide this information.

The simplest way to set all input parameters is to open a local xml file (on your computer) that has all of the input parameters already in it. To do this go to the menu "File → Open", and selec the desired xml file. Later on I will explain how to create such a file.

Another easy way to get the required input information is to ask the server. If you go to the menu item "Server → Get All Values from Server", all of the input parameters currently set in the server will be downloaded into this program.

The normal way to provide the required information is by fillng out all of the dialogs. Since this is more time consuming, I suggest first using one of the two previous methods (file or download) to preset everything to reasonable values. Then proceed through all of the dialogs until all parameters are set.

When all values are entered and set to your satisfaction and Validated, you can save these settings to an xml file by the menu item "File" → "Save As...". This saved file can then be read in at a later date to either re-do the same calculation or one only slightly different.
Main Dialog

2D/3D button
3D means there is a wire scan and so a reconstruction should be done. If no reconstruction (no wire scan), then choose 2D.
Wait for files
This causes the analysis to analyze the data as it is taken. This is useful when you start the analysis before the measurement is finished. If the data have already been taken, do not check this box. For example, do not check the box if you are re-analyzing old data.
Index Images
Causes the images (either reconstructed or not) to be indexed. This needs to be checked if you want to know the orientation of a Laue pattern. If you do not have Laue pictures that can be oriented, then do not check. Usually you will check this box for white images. For monochormatic (or energy scans) you should probably not check this box.
path to binaries
This is the path to the progam files used by the cluster. So the path is relative to the cluster nodes. The default is ~/bin/. It is likely that this path cannot be even seen by your computer (but the cluster can).
path to data files
This is a prefix path that will be prepended to all data files, both input files and output files. It is not required, but it will make things easier for you.
geometry file:
This is the path to the geometry file (including the name). The full path to the geometry file from the server's point of view is "path to data files"/"geometry file".
xml file for output (only visible in 2D)
This is the path to the output xml file (contains result of indexing). You only specify this for 2D data. For 3D data, the name is automaticaly generated. The full path this file from the server's point of view will be "path to data files"/"xml file for output". This item is invisible when the 3D radio button is checked.

buttons:

Go
Start analysis running on the server using the supplied parameters. If buttong is disables (grayed out), then you need to first validate the parameters (see Validate All below).
Stop
Not yet implemented.
Quit
Quit this program.
Get ← Server
Get values to fill in ONLY this panel from the server. It only sets the values on this panel. This is not a very useful button.
Validate All
Send ALL values from this program (not just the ones on this panel) to the server and has the server check them for validity. This includes checking that all required paths and (most) of the files exist at the server. Use this just before the 'Go' button. You need to do this before pushing the Go button or doing a "Save As..." from the menu.
Other Dialogs

There are three other dialogs that need to be accessed before starting a calculation. They are accessed from the "Set" menu.

Set → Reconstruct...

This is used to set the parameters needed for doing a reconstruction (3D) data. For 2D data this panel is not needed.
file root:
path to the raw images. This is relative to the "path to data files" from the main panel.
first file number:
An integer that designates the first image in the reconstruction.
last file number:
An integer that designates the last image in the reconstruction. If you want the computer to figure out where the reconstruction ends, use Inf, and the reconstruction will stop when the computer runs out of input images.
output folder:
Relative path to the folder that receives the reconstructed images.
Depth Range, first:
depth of the first reconstructed image.
last:
depth of the last reconstructed image.
resolution:
distance (in depth) between each reconstructed image.
% of pixels to process:
Most fo the pixels in each Laue picture contain no information, so reconstructing them provides no benefit. For lots of big images, reconstructing all of the pixels in each image will take a long time, this parameter will help speed the process by ignoring the unimportant pixels. The procedure used to decide which pixels are important is as follows: the first image in each wire scan is examined, and the most intense 10% (if this value is set to 10) of the pixels are the only ones that are reconstructed. This parameter is useful for big Laue images. For energy wire scans which usually use only a small region of the detector, use 100%.
memory to use:
Amount of memory (in MiB = Megabytes) to use for the reconstruction. Good values are 1024 (equivalend to 1 Gigabyte) for the cluster and 2048 on the Mac.
Io normaliztion:
Can be used to normalize the data to a PV that was stored in each image. Often left blank.
Wire Edge
Tells the reconstruction which edge(s) of the wire to use. We usually use Both Edges of Wire. This works best unless your sample has good crystalline grains larger than the wire diameter. Then you should use Leading Edge of Wire. The Trailing Edge is only for completeness, and is seldom used.
output type
Determines the number type that is used for saving the reconstructed images. Usually use default output type. However, it is possible to save disk space by choosing 2 byte signed int (short). This only works well when the reconstruction does not produce any really strong peaks. If you do not know what you are doing, use the default.
detector to use
You can specify the detector that is used. Generally this is a bad idea, the analysis programs should figure this out for themselves based on information in each image. So just choose -1 (auto). However if something is wrong, you can force a particular detector geometry to be used by specifying the detector here. Do not mess with this if you are not sure, just use auto.

buttons:

Done
Dismisses this dialog, all values entered in this dialog will be saved for use. If this button is disabled, try the Validate button.
Cancel
Cancel this dialog, do NOT use the values entered here, they are discarded. This is equivalent to just closing the window.
Validate
Does a simple check on the entered values. This is only a local check, the server is not involved. You have to do a Validate to activate the Done button.
Set → Peak Search...

This controls the peak searching which is a preliminary to the indexing. It is not used for energy scans (e.g. an energy wire scan).
box size
Gives a size used for finding and fitting a peak. Typical values are 5 for sharp peaks, and 18 for larger peaks.
max R-factor
Reject peaks whose fit has an R-factor greater than this value. Usually use 0.5.
threshold
The threshold used for peak searching. Usually just use nan, and let the program choose one for itself.
min spot size
Reject spots with size smaller than this number. For larger spots use 1.1, for sharp peaks or for Gaussian shapes use smaller values such as 0.3.
min spot distance
Do not accept peaks that are within this distance from each other. This keeps fitted peaks from being to close together.
max number of spots
Used to limit the number of spots found and fitted. Use -1 to accept all. Hoever, big nubmers (>50) can sometimes cause problems with the indexing.
Smooth before fitting, Check Box
This does a smooth operation on the image before fitting, not usually needed. This only works when using peaksearchBox.
Gaussian/Lorentzian
Function to be used for fitting the peaks. The Gaussian definitely works better for sharp peaks, and is usually a better choice.

buttons:

Done
Dismisses this dialog, all values entered in this dialog will be saved for use. If this button is disabled, try the Validate button.
Cancel
Cancel this dialog, do NOT use the values entered here, they are discarded. This is equivalent to just closing the window.
Validate
Does a simple check on the entered values. This is only a local check, the server is not involved. You have to do a Validate to activate the Done button.

programs:

XY program:
Name of program (relative to the "binary path") to be used for finding peaks on images. Usually use peaksearch, sometimes peaksearchBox works better, but rarely. If you are using a special peak search routine, here is where you specify it.
piexl → Q program:
Name of program (relative to the "binary path") that takes ouput from XY program and the geometry file and produces the list of Q hats used by indexing program. Usually use pixels2qs.
combine program:
Name of program (relative to the "binary path") that takes Qhat ouput from multiple detectors and combines them into one file that is used by the indexing routine. For single detector data this program is not neede. Usually use combinePeaks.
Set → Index...

This dialog provides the parameters used by the indexing routine. The indexing routine uses the measured Qhats from the peak searching step to determine the orientation of the crystallite(s). The indexing routine will give a list of orientations if multiple Laue patterns are present. The output list should be ordered so that the best patern is first.

angle tolerance
The indexing program compares all pairs of measured Qhats to all pairs of possible Qhats. But only considers those pairs which have the same angular separation to within this tolerance. It also uses this angle in the final allocation of measured Qhats to a derived orientation.
cone angle
central hkl
max calc (keV)
These three parameters determine the set of possible hkl that are used when indexing a pattern. The hkl that are considered when indexing are all hkl that are within cone angle of the central hkl, and whose energy will be below max calc. Note, max calc only limits the hkl that are used to find an orientation, the indexing process contains a second pass where spots with higher energies may be included in a pattern.
max test (keV)
Energy used when determining which measured spots belong to an orientation. Once an orientation has been determined (using cone angle, central hkl, and max calc) another list of hkl's is generated this time accepting energies out to max test. It is this new list that is used to identify which measured spots belong to this orientation. Note, this process is done for each orientation found in a Laue pattern.
max no. of spots
Limits the number of spots used to index a pattern. The indexing program will only use the first N spots from the peak search program.
.xtl file
File with extension .xtl containing the definition of the crystal that we are trying to index. The actual file is relative to the "path to data files" in the main dialog.

buttons:

Done
Dismisses this dialog, all values entered in this dialog will be saved for use. If this button is disabled, try the Validate button.
Cancel
Cancel this dialog, do NOT use the values entered here, they are discarded. This is equivalent to just closing the window.

programs:

indexing program:
Name of program (relative to "path to binaries") that does the indexing. Usually euler. Unless you are using some special program (which does not yet exist), use euler.
Starting (and Stopping) the Remote Server
These are the commands typically used to start (and stop) the remote server.

First open a terminal window and log into the machine that you want to run the server on (usually hpcs34.xray.aps.anl.gov), and navigate to the folder containing the server python scripts.
Then it is usualy best (but not required) to start the server in a detachable window, so type the command:

$screen

This starts a new screen session which can be detached. Then actually start the server by executing the Python script, type:

$./server.py

The server should now be running. The terminal should tell you that output from the server is being redirected to a file named serverLog.txt in the current folder. You can look at this log file at any time from any terminal. You can now detach from this screen (and log out) without stopping the server To detach from this screen, type in:

$[Ctrl+A] [D]

that is just two characters, first the "Ctrl+A", and then a "D". Now the server is running in a detached screen and you can log out or do whatever you want without stopping the server.

To re-attach to the server process (probably to kill it), log into the same machine that you started it on and type:

$screen -r

This will re-attach you to your previously detached screen and allow you to do things such as killing the server. To kill the server, just type one character:
Ctrl+C

If you want to run the server without a separate screen session, simply log in and type:

$./server.py

This will work fine. However, if you log out or kill the terminal window, the server will be killed, which is probably not what you want. By using the screen command you can detach and log off the computer while the server continues to run, which is what you generally want for normal operations. It should also be possible to run the server with a nohup command, but I have not yet tried that, and don't see any advantage to using it.