MHATT-CAT Useful Operational Data for Run 3 of FY04

written by MHATT-CAT staff.
MHATT-CAT
Started May 25, 2004
(www.mhatt.aps.anl.gov/Sectors/Sector7/Operations/FY04/run3/)

• Introduction
• Summary of the shutdown activities.
• Tuesday, May 25, 2004: First day of beam for run 3 of FY04: Cryocooler failure.
• Saturday, June 5, 2004: time serie analysis during top-up mode.
• Thursday, June 10, 2004: time serie analysis during top-up HYBRID mode,
  window and air transmission of 7ID-B at 14.3 keV.
• Friday, June 11, 2004: time serie analysis during top-up HYBRID mode.
• Saturday, June 12, 2004: realignment of L5-20.
• Monday, June 14, 2004: News from X-ray BPM from Glenn Decker.
• Tuesday, June 15, 2004: analysis of BPM data for period of 6/14 to 6/15.
• Friday, June 18, 2004: analysis of BPM data for period of 6/15 to 6/16.
• Monday, June 21, 2004: analysis of BPM data for period of 6/18 to 6/21.
• Sunday, June 27, 2004: analysis of BPM data for period of 6/27 21h19 to 6/28 8h19am (Earth quake in Ottawa IL at 1h15 am). See also response by Glen Decker.
• Friday, August 6, 2004: P5 failure and repair.
• Lessons learned, to do list.

Introduction.

The work shown below is an account of useful operational data taken during
run 3 of FY04 on 7ID. This page will contain useful stability information on the
7ID High Heat Load Monochromator and also on the various repairs and problems 
identified during the run.

Summary of the May 2004 shutdown activities.

• Several of us attended a beautiful meeting in San Diego on Ultrafast X-ray Science.
• At the beginning of the shutdown, a major failure occured for the micromono. A water line broke and filled the ion pump completely! Don Walko and Dohn Arms replaced the dirty pump with a spare pump. The water line are being repaired and won't be installed during the shutdown. The micromonochromator cannot be used until the cooling line is replaced. Don and Dohn did a great job to restore the beamline to operation!
• The 7ID PSS system was revalidated at the beginning of the shutdown.
• The 7ID-C Kappa was reinstalled by technicians from MicroControle at the beginning of the shutdown. See Don Walko's notes here., and specifically the May installation pictures here
• Dohn Arms procured a new labyrinth for the 7ID-B hutch. A picture of the new labyrinth can be found here.
• During the APS User Meeting, several of the 7ID-D Partner Users met to provide input to the proposed "7ID-E" laser hutch to be built at the end of the sector. Eric Landahl is in charge of the project and he will collaborate with the Engineering group to design the hutch. Eric L. and Dohn Arms at the end of the shutdown procured a new heat exchanger for the Amplifier system. It is under commissioning at the moment. The heat echanger should improve the amplified laser stability in the 7ID-D hutch.
• See the May 19, 2004 sector Update by Eric Dufresne.
• Dohn Arms upgraded the 7ID-C crate to the latest stable release of EPICS. The 7ID-C crate now has a 12 bit A/D and D/A Xiacom 540 card onboard. Also Dohn installed a new EPICS database for the Oxford cyberstar detector. He synchronized all our EPICS PV names with the new APS PV convention. He also installed a new full size VME crate for the 7ID-D hutch. Work is ongoing on the 7ID-B hutch to move it to the latest EPICS.
• At the end of the shutdown, Don Walko with help from Harold Gibson and the APS vacuum group removed two Be window in the 7ID-A hutch and removed the large empty Mirror Filter tanks. A full account of the work can be found here.

First day of beam time, Tuesday May 25, 2004.

The beamline today is broken following a cryocooler failure. The burst disk 
broke on 5/24 between 7-10pm, 12 hours before the start of the run...
On Monday night, ED established that the cryocooler burst disk was broken, and
planned the repair for Tuesday morning. Dan Burke and Glen Moonier replaced the 
burst disk in the morning and several upgrades were performed to the cryocooler
set up to prevent this from happening again:

1) cable rerouting to prevent someone from stepping on the power cable. This 
was the root cause of the problem. 
2) Proper anchoring and repair of the power cable connector.
3)New 50 PSI relief valve that should vent the closed loop if the pressure goes
too high. This will prevent the burst disk to break.

On Tuesday night, the system was returned to operation and Wednesday morning the
crystal was cooled to LN2 temperature. Everything is ready for operation.

• See A full account with pictures here.
• See also additional upgrades to the cryocooler that were done during the first week.

Saturday June 5, 2004: time series during Bernhard Adams's run, top-up.

A time serie was started on 06/05/04 at 01h57 am. The beamline is running on 7ID
with n=3, E = 17.92 keV, and the mono energy was set to 17.738 keV. The L5-20
slits were opened to 0.5mmx0.5mm. The ring is in top-up mode. There was a top-up
failure between t=18.5 and 21.3 hrs. The figures below show the beam intensity,
positions, and cryocooler status (dewar level and closed loop pressure). 

Fig 06-05.1. Time series of the beam intensity in 7ID-C, starting at 01h57m31s on 06/05/04 and
lasting about 52 hours. The 7ID-C diode sum, the ring current, ion chamber in B and C are shown.
A top-up failure occured near t= 18.5 hours.

Fig 06-05.2. The beam position, 49.2 m from the source, or 19 m from the High Heat Load mono
in 7ID-C during the same time series as Fig. 06-05.1.

Fig 06-05.3. The cryocooler level sensor and the closed loop pressure during the same time series.
Note the pressure is stable to about 0.2-0.3 psi during a LN2 fill.

Thursday June 10, 2004: time series during Linda Youngs's run, top-up, hybrid.

A time serie was started on 06/10/04 at 02h25 am. The beamline is running on 7ID
with n=3, E = 14.44 keV, and the mono energy was set to 14.3235 keV. The L5-20
slits were opened to 0.5mmx0.5mm. The ring is in top-up mode. 

Fig. 06-10.1 shows the usual intensities. All the ion chamber and diodes are stable 
to about 6% whereas the ring current is stable to 1%.

Linda Young and I also carefully calculated the losses from air and window 
transmission at 14.3235 keV. The table below shows the transmission losses.

item                     transmission
--------------------------------------
20 cm air 7ID-B             0.9613
4  1 mil kapton             0.9854
1  10 mils Be window 7ID-B  0.9872
1  0.5 um Ti foil (BPM)     0.9909
1  10 mils Be window 7ID-C  0.9872
29 cm air 7ID-C             0.9445
--------------------------------------
total transmission          0.864

With two ion chambers, one in 7ID-B and one in 7ID-C in air, we measured a ratio
of 77%. We went inside the hutch and found an old burn paper on the downstream 
Be window and removed it. We gained a bit more flux. We then increased the 
flow of He in 7ID-B from 0.2 to 0.3. We got a transmission ratio of 0.789. So
we are a bit more than 7% off. Pretty close. 

During the Day of June 10, our intensity started to drop. At 1am, it was about
1.25x10^13 ph/s in 7ID-B, then in the afternoon it was 1.14 x 10^13 ph/s.
On 6/11 at 1 am, the flux was 8.75x10^12 ph/s, after the tornado warning.
So during the course of a day, the 7ID-B intensity dropped to  0.875/1.25 = 0.7
of its initial value. We weren't able to find it back. It is not clear what is 
going on.

To answer Dohn Arms's question, do we have more flux now than before removing 
the two Be windows in 7ID-A? On 6/10 at 1 am we had more flux than during the
last run by 6%. One expects the transmission of two .254 mm Be window to be 97%
so we should expect 3% more flux. Clarly though, a 3 % measurement is difficult
to make. If one lines up the opening of the L5-20 by 10 um, than an error of 
10um results in a flux different of 10/500 = 2%  thus a 4% difference for a 
two dimensional area of 500umx500um. Most of our measurements are not better
than 4%. So we'll have to wait to work at very low energy to see measure 
any difference.

Fig 06-10.1. Time series of the beam intensity in 7ID-C, starting at 02h25m20s on 06/10/04 and
lasting about 8 hours. The 7ID-C diode sum, the ring current, ion chamber in B and C are shown.

Fig 06-10.2. The beam position, 49.2 m from the source, or 19 m from the High Heat Load mono
in 7ID-C during the same time series as Fig. 06-10.1.

Fig 06-10.3. The cryocooler level sensor (was off so no data) and the closed loop pressure during
the same time series. Note the pressure is stable to about 0.2-0.3 psi during a LN2 fill.

Friday June 11, 2004: time series during Linda Youngs's run, top-up, hybrid.

A time serie was started on 06/11/04 at 01h57 am. The beamline is running on 7ID
with n=3, E = 14.44 keV, and the mono energy was set to 14.3235 keV. The L5-20
slits were opened to 0.5mmx0.5mm. The ring is in top-up mode. There was a top-up
failure during the tornado warning at 8h30pm on 6/10, about 6 hours before the
start of the time series. This resulted in the monochromator cooling down again.
Eliott reopened the shutter at 9h30pm. So one can expect beam motion following
the reopening of the shutter.  The figures below show the beam intensity,
positions, and cryocooler status (dewar level and closed loop pressure). 

I witnessed the sudden rise of intensity at the beginning of Fig. 06-11.1
with data in the table below. I witnessed a 12 % rise in 0.37h, which is obvious
in the figure near t = 0.

         time 7ID-B kcps
         --------------
         2h43 238
         2h47 245
         2h52 255
         2h55 259
         3h03 263
         3h05 266

Fig. 06-11.2 shows the beam position. The beam moves by over 120 um up
during the time series and seems to settle after 6.5-7 hours. This must be 
caused by the reheating of the monochromator, suggesting a 12 hrs settling time
from 9h30 pm on 6/10. The horizontal beam motion is slightly affected by the 
cryocooler fill. Small horizontal beam motion occur near the crycocooler fill 
as seen in Fig. 06-11.3. Also an intensity bump seems to
correlate to the first cryocooler fill near t = 2.25h, but none is noticed near
t = 5.6h.

Fig 06-11.1. Time series of the beam intensity in 7ID-C, starting at 02h41m09s on 06/11/04 and
lasting about 8 hours. The 7ID-C diode sum, the ring current, ion chamber in B and C are shown.

Fig 06-11.2. The beam position, 49.2 m from the source, or 19 m from the High Heat Load mono
in 7ID-C during the same time series as Fig. 06-11.1.

Fig 06-11.3. The cryocooler level sensor and the closed loop pressure during the same time series.
Note the pressure is stable to about 0.2-0.3 psi during a LN2 fill.

Saturday June 11, 2004: L5-20 realignment, total flux is back! Eric turned off the regulator as well.

See the Elog58 here for the story on the realignment of the L5-20 slit opening

Monday June 14, 2004: Reply from Glenn Decker about XBPM not used in the orbit and analysis of time series from the weekend.

The 7ID-B and C ion chamber signals and X-ray BPM signals are shown for 
the last 37 hours on the Elog system.

You can see the plots here (avail. outside MHATT domain).

Following the 3 emails below, Glenn Decker called ED. The P1 X-ray BPM signal
was not included in the orbit feedback program so they will start to use them
again tomorrow after the machine studies period. The X-ray BPM data from 7ID 
are accurate, Glenn trusts them.

ED

From dufresne@umich.edu Mon Jun 14 14:17:42 2004
Date: Mon, 14 Jun 2004 13:08:26 -0400 (EDT)
From: Eric Dufresne 
To: Glenn A Decker 
Cc: ANL AMO's group & MHATT-CAT Staff 
Subject: Re: 7ID X-ray BPM

Hi Glenn,

	our 7ID beam felt less stable these days since last Wednesday. The
beam seems to mostly have significant intensity variations at the 5%
level. I did not remember seeing the 7ID X-ray BPM used in the orbit. I
guess they have been for a while. Is this correct? At the moment only the
RF BPM are used in the orbit.

Cheers,

Eric

> Thanks for pointing this out.  There was a screwup in communication resulting
> in running without the photon bpm's since last wednesday at 11 AM.  I'm not
> too pleased to learn this.
>
> Any time an id beamline requests steering, we have to reconfigure the orbit
> correction to omit that sector's photon bpms since the gap-dependent lookup
> tables will not work.  There was steering at 34ID last wednesday so operations
> did the steering and informed the right people but I wasn't on that list.  The
> default for operations (say if there is a steering in the middle of the night)
> is to fall back to the configuration using no id xbpm's anywhere.
>
> Sorry about that.  It appears that your readbacks haven't drifted too far
> by some miracle.  I'll try to get the correct configuration going again.
>
> Gd
>
>>From dufresne@umich.edu Mon Jun 14 12:59:00 2004
>>Date: Sat, 12 Jun 2004 00:43:52 -0400 (EDT)
>>From: Eric Dufresne 
>>To: Glenn A. Decker 
>>Subject: 7ID X-ray BPM
>>
>>
>>Hi Glen,
>>
>>hope everything is going well. I had a quick question to ask you about the
>>status of the 7ID X-ray BPM P1 and P2. I noticed it is not used in the
>>orbit feedback program so I am wondering how reliable they work right now
>>i.e. can I trust the data from them?
>>Cheers,
>>
>>Eric
>>

Tuesday June 15, 2004: analysis of time series data from 6/14 to 6/15.

Friday June 18, 2004: analysis of time series data from 6/15 to 6/17.

Monday June 21, 2004: analysis of time series data from 6/18 17h02 to 8am on 6/21.

Sunday June 27, 2004: Earth quake in Ottawa IL seen by 7ID-C X-BPM

Friday August 6, 2004: Repair of P5 following several major faults.

Lessons learned during run 2 of FY04

By removing the cryocooler low pressure fill modification, we have found that 
the Cryocooler fill pressure disturbance are less severe, and more rapid to 
return. The beam position variations are also less severe.


To do list:

-Check the EPS programing for the new vacuum configuration. When MF2 is 
turned off, I've noted that GV4 does not close. This does not pose any equipment
protection issues but it is not consistent with the MF1 pump response.
-We should upgrade our EPS system and pass the responsibility to John
Cawardine's group. This was done at S8 successfully.
-Fix the HHL Mono y2 jack. It is out of commission.

-complete remote shutter interface
-Reduce the speed of the pressure response of the Huber chamber gauge.
-Prevent the turbo from venting when the pump is turned off or a power failure
 occurs.