MHATT-CAT Useful Operational Data for Run 2 of FY03

Eric Dufresne, Dohn Arms, Don Walko
MHATT-CAT
Started Jan. 21, 2003
(www.mhatt.aps.anl.gov/Sectors/Sector7/Operations/FY03_run2/)

• Introduction
• Summary of the shutdown activities.
• First day of beam for run 2 of FY03.
• Saturday, Feb. 1, 2003 (Mirror filtor repair).
• Sunday, Feb. 2, 2003 (time serie analysis).
• Saturday, Feb. 8, 2003 (MF2 trip).
• Tuesday, Mar. 4, 2003 (time serie).
• Thursday, Mar. 13, 2003 (time serie).
• Monday, Mar. 17, 2003 (time serie).
• Sunday, Mar. 30, 2003 (very long time serie).
• Monday, Apr. 7, 2003 (warming up time serie).
• Thursday, Apr. 10, 2003 (Kr edge data).
• Tuesday, Apr. 15, 2003 (Cu edge data).
• Wednesday, Apr. 16, 2003 (time serie).
• Lessons learned, to do list.

Introduction.

The work shown below is an account of useful operational data taken during
run 2 of FY03 on 7ID. This page will contain useful stability information on the
7ID High Heat Load Monochromator and also on the various repairs and problems 
identified during the run.

Summary of the January 2003 shutdown activities.

1) Two new Servers from Howard University were configured in preparation for
   an EPICS upgrade this spring.

2) Two labyryths were added to the 7ID-C hutch for better access for cables.
   One is downstream and connects to ID-D and will add flexibility in sharing
   motor and actuators between the two hutches.
   See this link from Don Walko for a few pictures..

3) The cryocooler was serviced (done once a year). A low pressure vessel 
   modification was also done by Dale Ferguson which is supposed to help to 
   reduce pressure transients in the closed loop. Dohn Arms and Don Walko fixed
   the exhaust line of the cryocooler and the APS recalibrated and permanently
   installed an oxygen monitor for 7ID-A. This monitor is next to the 7ID-A 
   hutch door control.

First day of beam time, Jan 29, 2003.

Today, we started by lining up the beamline High Heat Load Mono in preparation
for a shielding validation of the 7ID-C hutch around its new labyrynth. The 
test was done with 2.5 mm (H) by 1.0 mm (V) white beam slits opening. The gap 
was closed to 11 mm, and the mono set to 9.42 keV. The test did not reveal any
leaks and we were back to commissioning after 11h30am. Don Walko took the beam
afterwards to align the Newport Kappa in the beam.

Fig 01-29.1 show the intensity during a time series 
started on 01/29/03 at 16h23 and lasting 34 hours or so. The beam is in non 
top-up mode so the intensity varies following the ring current. Don Walko
accessed the hutch several times so the time serie has several discontinuities.

Fig 01-29.2 shows the beam position measured with the
7ID-C X-ray BPM. Note that the heat load on the mono was very high during the
shielding validation test so there was a huge heat load change when we finished
the test and closed the white beam slits to 1.0 mm (H) by 0.5 mm (V), and 
opening the undulator gap to run at 10 keV on n=1. The vertical beam motion 
during the first 3.5 hours is caused by this change in heat load, likely the
second crystal cooling, so a very long drift. The vertical beam motion around
20 hours is likely due to a mono second crystal tweak by DW.

Fig 01-29.1. Time series of the beam intensity in 7ID-C, starting at 16h23 on 01/29 and
lasting 34 hours. The 7ID-C diode sum, the ring current, ion chamber in B and C are shown.
The 7ID-C ion chamber was off for the first 20 hours so we see only background noise.

Fig 01-29.2. The beam position, 49.2 m from the source, or 19 m from the High Heat Load
mono in 7ID-C during the same time series as Fig. 01-29.1.

Saturday Feb. 1, 2003. (Mirror filter problems)

Today, the Mirror Filter Ion Pump controller created several beam dumps. On one incidence, the gate valve for MF2 closed, while on another instance the 
controller turned off the MF1 pump, each incident tripping our EPS and closing
the APS FEV. After a third failure near 3h45pm, I swapped the vacuum controller
for a new one, which is supposedly more robust. In the next few weeks we should
see if the problem has gone away or not...

Below are time series data of the beam intensity and position with a white 
beam set at 10.055 keV, the mono set to 10.0 keV and the L5-20 slits set to 
1.0mm (H) by 0.5 mm (V). Fig. 02-01.1 shows the beam intensity measured on
7ID starting on 02/01 at 00h35 and lasting over 3 hours. The time serie was 
interrupted at 3h37am due to the controller failure discussed above. 

Fig. 02-01.2 shows the beam position measured in 7ID-C
during the same time serie. The beam moves suddenly near t=0.4h, likely due to a
LN2 fill. The motion is about 60 and 40 um in the hor. and vert. direction 
respectively.

Fig 02-01.1. Time series of the beam intensity in 7ID-C, starting at 00h35 on 02/1 and
lasting 3 hours. The 7ID-C diode sum, the ring current, ion chamber in B and C are shown.
The 7ID-C ion chamber was off so we see only background noise.

Fig 02-01.2. The beam position, 49.2 m from the source, or 19 m from the High Heat Load
mono in 7ID-C during the same time series as Fig. 02-01.1.

Time serie of Sunday, February 2, 2003

Fig. 02-01.3 shows the beam intensity measured on
7ID starting on 02/01 at 23h52 and lasting over 22 hours. The time serie was 
interrupted by several APS beam dumps (t=2.64h, t=6h), and the floor 
coordinators did not reopen our shutter last night so Dohn Arms turned the beam
on again at t=17hrs or so. There were no vacuum incidents last night. 

Fig. 02-01.4 shows the beam position measured in 7ID-C
during the same time serie.  Dohn also told me he tweaked the mono which would
explain the vertical beam motion. This would explain the vertical beam motion 
near t=16h.

Fig 02-01.3. Time series of the beam intensity in 7ID-C, starting at 23h52 on 02/1 and
lasting 22 hours into 02/02. The 7ID-C diode sum, the ring current, ion chamber in B and C are shown.

Fig 02-01.4. The beam position, 49.2 m from the source, or 19 m from the High Heat Load
mono in 7ID-C during the same time series as Fig. 02-01.1.

Saturday Feb. 8, 2003

This Saturday, Feb. 8, 2003, Paul Evans paged ED about a trip of MF2 around 
10h05 am.  The beamline tripped and GV4 closed as well as the FEV. So ED did not
fix the problem when he switched the controller. This is consistent with his
previous experience of trying to do this. Paul reopened GV4 and got the FEV
reopened. On Feb 12, 2003, ED raised the set point on MF2 from 2e-5 to 2e-4
Torr in the hope that it makes the incidence less likely to happen.

Tuesday Mar. 4, 2003

Today, the APS changed over from top-up operation to high emmittance non-top
up operation. This is the last time such a mode will likely be scheduled since 
in the future the multibunch mode will be in non top-up and in low emmittance
mode. For the week on 3-4 to 3-10, the APS ran with Ex=6.88 nm-rad, and a 
coupling of .97%. 

Fig03-04.1 shows the intensities measured with the 
usal detectors. Here the X-ray energy set by the mono is 12.0 keV, the undulator
fundamental is set to 12.055 keV, and the L5-20 opening is 0.5mm by 0.5 mm. 
The various ion chamber and diode sum signals track the three fill well.
Fig03-04.2 show the beam position 49 m from the source at the entrance of 7ID-C. 
The beam position is fluctuating by +/- 50 and +/- 30 um in the horizontal and 
vertical position respectively. This amplitude is mostly caused by the non top
up operation of the ring although one can clearly see the effect of filling the 
LN2 cryocooler every 3 hours on the beam position.

Fig 03-04.1. Time series of the beam intensity in 7ID-C, starting at 08h29 on 03/4 and
lasting 34 hours into 03/05. The 7ID-C diode sum, the ring current, ion chamber in B and C are shown.

Fig 03-04.2. The beam position, 49.2 m from the source, or 19 m from the High Heat Load
mono in 7ID-C during the same time series as Fig. 03-04.1.

Thursday Mar. 13, 2003

On Thursday 3/13, I started a time series at 17h51 with the beam back in top 
up mode. The beam was switched back in low emmittance mode on 3/12/03 and the 
emmittance is Ex=2.69 nm-rad and Ey= 0.0568 nm-rad (2.1% coupling). 

Fig03-13.1 shows the intensities measured with the 
usal detectors. Here the X-ray energy set by the mono is 10.2 keV, the undulator
fundamental is set to 10.255 keV, and the L5-20 opening is 0.5mm by 0.5 mm. 
Fig03-13.2 show the beam position 49 m from the source at the entrance of 7ID-C. 
The beam position is fluctuating by 50 and 25 um p-p in the horizontal and 
vertical position respectively. These oscillations are caused by the filling 
of the LN2 cryocooler every 3 hours. It is worth comparing these results from
those obtained in non-top up operation last week. Clearly top-up provides a 
more stable beam (see the 0304 data).

Fig 03-13.1. Time series of the beam intensity in 7ID-C, starting at 17h51 on 03/13 and
lasting 68 hours into 03/16. The 7ID-C diode sum, the ring current, ion chamber in B and C are shown.

Fig 03-13.2. The beam position, 49.2 m from the source, or 19 m from the High Heat Load
mono in 7ID-C during the same time series as Fig. 03-13.1.

Monday Mar. 17, 2003

On Monday 3/17, I started a time series at 13h49 with the beam in top up mode.
Fig03-17.1 shows the intensities measured with the 
usal detectors. Here the X-ray energy set by the mono is 10.2 keV, the undulator
fundamental is set to 10.255 keV, and the L5-20 opening is 0.5mm by 0.5 mm. 
Fig03-17.2 shows the beam position 49 m from the source at the entrance of 7ID-C. 
The beam position is fluctuating by approx. 50 and 25 um p-p in the horizontal
and vertical position respectively. 

Fig 03-17.1. Time series of the beam intensity in 7ID-C, starting at 13h49 on 03/17 and
lasting 18 hours into 03/18. The 7ID-C diode sum, the ring current, ion chamber in B and C are shown.

Fig 03-17.2. The beam position, 49.2 m from the source, or 19 m from the High Heat Load
mono in 7ID-C during the same time series as Fig. 03-17.1.

Sunday Mar. 30, 2003

On Sunday 3/30, I started a time series at 01h42 with the beam in top up mode.
Fig03-30.1 shows the intensities measured with the 
usal detectors. Here the X-ray energy set by the mono is 10.0 keV, the undulator
fundamental is set to 10.055 keV, and the L5-20 opening is 0.5mm by 0.5 mm. 
Fig03-30.2 shows the beam position 49 m from the source at the entrance of 7ID-C. 

The data interruption between t = 54 and t = 62 hrs was the Tuesday April 1, 
8 hours study period. During this period, Don Walko filled up the cryocooler
buffer level from 48 % to 86 %. He noticed afterwards that the pressure of the
closed loop varies from 15 % to 21.5 % during a fill. Notice the significant
difference in the amplitude of the beam motion before and after
the Tuesday period. After the Tuesday period, the peak to peak variations are 
up to 100 um in the horizontal, while before they were only around 40 um. We 
should try to keep the buffer level between 50  and 60 % in the future.

Fig 03-30.1. Time series of the beam intensity in 7ID-C, starting at 01h42 on 03/30 and
lasting 190 hours into 04/06. The 7ID-C diode sum, the ring current, ion chamber in B and C are shown.

Fig 03-30.2. The beam position, 49.2 m from the source, or 19 m from the High Heat Load
mono in 7ID-C during the same time series as Fig. 03-30.1.

Monday Apr. 7, 2003

On Monday 4/7, I started a time series at 00h52 with the beam in top up mode.
Fig04-07.1 shows the intensities measured with the 
usal detectors. Here the X-ray energy set by the mono is 14.3256 keV, the
undulator fundamental is set to 14.417 keV, and the L5-20 opening is 0.5mm by
0.5 mm. The mono was operated until then at 10 keV so the power increased 
significantly on the first crystal. The gap was closed after midnight, 50 min 
before the start of the time serie. This explains the 2/3 drifts shown in the
figure.  

Fig04-07.2 shows the beam position 49 m from the source at the entrance of 7ID-C.
The drifts is correlated with a significant vertical beam motion of 450 um,
which corresponds to a 450 um/19.2m = 23 urad angular shift. Some of this
motion can be attributed to the heat load change. Note that I tried to look for
the Au L1 egde but could not easily find it. Perhaps the energy calibration of
the mono is off following the beamline change following the white beam 
experiment in 7ID-B in March.

Fig 04-07.1. Time series of the beam intensity in 7ID-C, starting at 00h52 on 04/07 and
lasting 6 hours. The 7ID-C diode sum, the ring current, ion chamber in B and C are shown.

Fig 04-07.2. The beam position, 49.2 m from the source, or 19 m from the High Heat Load
mono in 7ID-C during the same time series as Fig. 04-07.1.

Thursday Apr. 10, 2003

On Thursday 4/10, Eric recalibrated the mono around the Kr edge. A gas cell 
was made by filling an ion chamber with Kr. The HV was turned off on the 
chamber. The partial pressure had to be adjusted so that a small absorption
would be measured.

Fig04-10 shows the derivative of the absorption factor
versus energy. The energy was off by 15.9 eV.  Note that no calibration was 
performed before David Reis's experiment on 3/26 so this calibration offset
was likely caused by the monochromator disaplacement to its passthrough mode
when we did the white beam experiment in 7ID-B.  The FWHM of the atomic
resonance is 2.4 eV. One would expect the bandwidth of Si (111) to be around
2.0 eV at 14.3 keV. The Kr edge is at 14.3235 keV.  So the mono does not
significantly broadens the resonance,

Fig 04-10 shows the Kr edge data before and after the mono calibration.

Tuesday Apr. 15, 2003

On Tuesday 4/15, Eric checked the mono calibration around the Cu edge.
Fig04-15 shows the derivative of the absorption factor
versus energy. The energy calibration is right on as the first peak in the
derivative should be at 8.979 keV. The calibration for Kr and Cu are consistent.

Today the APS started to operate in non-top up mode, with 324 filled buckets. 
The lattice did not change and we are still in low emmittance mode, Ex=2.62
nm-rad, Ey = 0.0473 nm-rad, and the coupling is 1.8 %.

Fig 04-15 shows the Cu edge data after the kr edge mono calibration.

Wednesday Apr. 16 2003

On Wednesday 4/16, I started a time series at 03h05 with the beam in non top up
mode. Fig04-16.1 shows the intensities measured with the usal detectors.
Here the X-ray energy set by the mono is 8.979 keV, the undulator fundamental is
set to 9.007 keV, and the L5-20 opening is 0.3mm by 0.3 mm. A fill-on-fill of
the APS happened at t= 4.75 hrs in the time serie and is observed as an increase
in the diode and ion chamber signal. The beam dumped around t= 5.5 hrs, but
our shutter was not reopened until the end of the time serie. Note the 7ID-C ion
chamber was not connected.

Fig04-16.2 shows the beam position 49 m from the source at the entrance of
7ID-C. The usual LN2 fill beam motion is obvious starting at t = 0.2, 3 hrs.
Some motion is also seen near the fill-on-fill at t = 4.75 hrs. The motion
caused by the fill is much smaller than the LN2 induced motion.

Fig 04-16.1. Time series of the beam intensity in 7ID-C, starting at 03h05 on 04/16 and
lasting 8 hours. The 7ID-C diode sum, the ring current, ion chamber in B and C
are shown but the C-ion chamber was shielded with Pb.

Fig 04-16.2. The beam position, 49.2 m from the source, or 19 m from the High Heat Load
mono in 7ID-C during the same time series as Fig. 04-16.1.

Lessons learned during run 2 of FY03

Vacuum trips MF1+MF2     on 2/01/03
Vacuum trip  MF2         on 2/08/03 10 am
Set point raised on MF2  on 2/12/03 to 2E-4 from 2E-5 Torr by ED.
Mono Huber tripped EPS   on 3/12/03 when Don changed the mono energy.
Yoruba was suspended     on 3/21/03 when someone logged out... 
                                    2nd time it happens
				    Dohn Arms fixed this for good.
Crocooler fill level:    keep buffer between 50 and 60 %.
Vacuum trip MF1+MF2:     on 4/10/03 1h50 pm. MF1 controller ultimately tripped.
                         Real gas load noticed on mono pumps when GV3 open.
			 Should we replace the pump?

To do list:

-check and fix Y-BPM motor/driver (see 1/29/03).
-fix the EPICS LN2 level sensors on the Oxford cryocooler.
-complete remote shutter interface
-Reduce the speed of the pressure response of the Huber chamber gauge.
-Remove the suspend option for the OS on yoruba and all SUNs... (Done)