DECam Log of Configuration Changes and Events

Here are recorded dates of changes to the telescope, camera, operational procedures, and software that may be important for analyzing DECam data.

Pre-DES

  • March 2011: Mirror Re-aluminized.
  • November 10, 2011: Mirror washed
  • Feb. 20, 2012: DECam Installation Starts

DES Science Verification 2012-2013

  • 31 October: Was pointing map from 8 Oct observations installed on this night?
  • 5 November: Pointing map from 31 Oct observations installed this night?
  • 5 November: Guider changes, including new code to avoid choosing CRs as guide stars, delay guider readout until after shutter is all the way open, and change timing so that guider CCDs are not cleared while previous exposure is still reading out on science array. Latter change seems to have eliminated the readout striping issue, and former 2 have greatly reduced incidence of guider failing to find stars.
  • 6 November: Commissioning pointing map (re-)installed. Much improved track. Somewhat worse pointing.
  • 7 November: CCD N30 woke up dead after sky flats
  • 7 November: TCS software modifications: updated pointing model (not clear if based on old mosaic or newer models). Telescope response to 1" "kicks" now decoupled in RA, dec. Problem with response in RA (too large - now in both E and W directions)
  • 8 November: TCS modification to allow pointing S of -79 dec.
  • 11 Nov: correct telescope LUT installed at image # 150014
  • 15 Nov: Rasicam mirrors cleaned. OBSTAC given list of Minisurvey exposures to ignore (MS1a izY).
  • 19 Nov: OBSTAC max airmass is 1.5 and no longer ignoring the MS1a izY's.
  • 19 Nov: Change to SISPI so that BCAM is not recalibrated on every Configure operation.
  • 20 Nov: Black felt applied to parts of the telescope.
  • 22 Nov: BCAM analysis matrix changed, to align BCAM outputs with hexapod axis & center of rotation.
  • 24 Nov: New hexapod LUT installed by Kevin.
  • 25-30 Nov: Several changes to tracking, new pointing model, "RA damper motor" plugged in.
  • *02 Dec: Extended SV starts with image 1574341
  • 3 Dec, 4 Dec: New guider algorithms tested early in night and then reverted.
  • 3 Dec (and 4 Dec?): bad dome encoder, lots of occulted exposures. Then repaired.
  • 5 Dec: New guider algorithm installed (pyguide)
  • 6 Dec: Primary mirror washed.
  • 6 Dec: New filter focus LUT installed. Change to guider timing & feedback parameters (guider latency set to 0, 10% feedback)
  • 11 Dec: Ion pump moved.
  • 17 Dec: Mamac replaced on telescope primary support system- #22
  • 17 Dec: Work on RA cable wraps
  • 19 Dec: TCS tweak to fix accuracy of small offsets (German S.). Telescope rebalanced. Units of guider logs changed and now gives error signal instead of correction signal.
  • 26 Dec: Low-temp on CCDs (-120C) due to Neslab failure. 163962 is first image after?
  • 30 Dec: Tololo power outage, full warm-up, last image is 164862
  • 03 Jan: Imager cold again, first image is 164871
  • 09 Jan: Tololo power outage, warm up to -87C, last is 167505, first is 167506
  • 29 Dec - 14 Jan: Environmental database never came back up after the power outages, so temperatures etc. over this period are invalid.
  • Jan: ongoing work on the RA cable wrap.
  • 23-28 Jan: TCS, damper motor, cable wrap work during engineering.
  • 23-28 Jan: Engineering visit, probable board changes, DES last is 171879, DES next is 173202
  • Jan 28 ???: begin routine 5-axis AOS control of hexapod during DES halves of nights.
  • 30 Jan: Fixed the oil bearings on horseshoe, tracking friction greatly reduced.
  • 1 Feb: New chiller online, begin using for mirror and dome floor cooling. Consistent daytime cooling starts 4 Feb (?)
  • 1 Feb: Rasicam now properly receiving position of Blanco pointing.
  • 17 Feb: Image Health PSF outputs changed (table imagehealth_fp). Note change of units and some other things in following days.
  • 23 Feb: End of Extended-SV last image is 181243
  • 7 Mar: Primary Mirror Washed
  • 25 Mar: Guider configuration changed (eLog 67424). Smaller isolation range, search range and higher tolerance to missing guide stars
  • 25 Mar: New pointing model installed and a fix to the TCS offset code (cos(dec) term?)
  • 27 Mar: TCS Delay reduced from 5 seconds to 1 second
  • 27 Mar: Default AOS mode is now aos5 (all 5 degrees of freedom in closed loop)
  • 22 Apr: g-band filter baffle installed and tested
  • * 7 May:* Primary Mirror Wash
  • 12-24 May: full warm-up
  • 22-31 July: full warm-up
  • 12 August: The rest of the filter baffles installed and tested

DES Year 1, 2013

  • 31 August: Y1 Officially begins
  • 6 September: 23:00 to 02:00 +2.5 deg C Focal Plane Temperature blip
  • 15-21 October: Warm up for LN2 Pump Change,
  • 18 October: Dry Air dessicators for optics re-enabled (off since August 1st-ish
  • 29 October: Rasicam flat field updated. It was cleaned a few weeks ago…
  • 6 November: Rescaled IH algorithm to calculate seeing to match QC values.
  • 6 November: Modified obstac query of IH data use only exposures with ih_done and ih_accept both marked true in the exposure database
  • 6 November: Change the filter seeing correction table used by obstac to correct to i band instead of DIMM.
  • 7-10 November Repair dome floor cooling system temperature sensor, and change daytime mirror-cooling set point algorithm.
  • 11 November: excess noise becomes visible in the bias frames, probably present until sync cable fix on 30 Nov
  • 12 November: Chiller failure caused ~3K focal plane temperature excursion around 07:00 UT
  • 23-25 November: Hexapod broken
  • 26 November: Hexapod differential encoder failure, switched to encoder on the motor.
  • 30 November: Bad sync cable in readouts, power supply replaced, S30 dies.
  • 5 December: Mirror washed
  • 6 December: Small spike in vacuum pressure during manipulation of LN2 lines.
  • 18 December: External Photodiode/Dome light protection goes online
  • 18 December: New initial ztrim equation: 1464 - 122*upper_truss_temperature (instead of 1400 - 110*T)
  • 29 December: 22:00ish +6 degree C Focal PLane Temperature spike. These spikes damp out after a couple of hours

2014

  • c. 10 January 2014: New dome encoders installed and in use.
  • 14 January: Switched back to original Hexapod encodes ("command 11" was removed)
  • 14 January: Oil cooler in operation. Accelerometer installed in PF cage.
  • 21 January: Focal plane temperature glitch + 2.5C, normal before observing starts
  • 30 January: Change voltage on the bluest LED set.
  • 10 February: Y1 Ends
  • 11 March: Changed default constants: new gains, noise values, new filter constants for VR filter (DECAM:54)
  • 11 March: Changed DEC guider feedback from 10% to 5%
  • 14 March: Shutter stepper motor replaced; VR filter installed; paint applied to inner surfaces of shutter and FCM; new UV LEDs for DECal.
  • 25 March: DB9 cables for flat-field projectors damaged, perhaps a day before.
  • 27 March: Removed BCAMs from default exposure sequence (now only called for zeros and dome flats)
  • April: Sometime in April the TCS delay was set back to 3 s.
  • 12-16 May: Full warm-up* for LN2 pump replacement
  • 16 May: TCS Delay (3s) coupled to telescope slew, decoupled from dome movement (should the dome be last there won't be an extra delay)
  • 17 May: Fast readout mode set as default
  • 3 June: Primary mirror washed
  • 12 June: 17:00 ish +2.5 deg C Focal plane temp blip
  • 22 July: A one meter piece of metal strip varying 0.5 inch to 1 inch in width that completes the annulus that masks the down-turned edge of the Blanco primary mirror (east side) was installed. This piece had been missing for a long time.
  • 31 July: Changed ImageHealth to exclude dead CCDs (S30, N30, S7) and raised glowing edge cut from 15 to 25
  • 31 July: Primary mirror washed. Mirror Reflectance report from CTIO.
  • 6 August: 8:00 +3.5 deg C focal plane temperature blip, Y2 engineering images
  • 13 Aug: Dome air conditioner turned on. All other cooling systems turned back on after being off for a week or so for final stage of dome A/C installation.

DES Year 2, 2014 to 2015

  • 15 August: Beginning of DES Y2 with SS image 348359
  • 20 August: aTmCam Installation begins
  • 27 August: DJJ and JM replaced long-ago damaged DECal DB9 cables. Flats will have more counts.
  • 24 August: Either this day or previous RASICAM surface was damaged by unauthorized cleaning process. Now not quite as sensitive to high cirrus clouds.
  • 01 September: On or about this date all 33 air pads controllers upgrade finished.
  • 04 September: Jennifer M, DJJ realigned the west flat field lamp, which was 30 deg out of whack. It had obviously taken a lick. There was mechanical damage to the eastern lamp as well, but that wasn't reported as misaligned. David reports a 7-10% increase in the flat fields now that the alignment is improved.
  • 08 September: Dewar Turbo pump turned off. Ion pump turned on.
  • 08 October: RASICAM toe-up until ...
  • 16 October: Quartz Lamp on DECALS Optical bench replaced this day by DJJ.
  • 21 October: TASCA CTIO all-sky CCD camera (red to Y-band, only, with low S/N) expected to be online.
  • 21 October: HexapodLUT and Donut updated to latest versions (which were tested during engineering). Donut version is now trunk (old was SVreference), and HexapodLUT version is v20141010 (old was v20130731).
  • 13 November: Mirror washed. This is generally done every 3 months Alistair says.
  • 21 November: Primary Mirror cleaned with CO2
  • 21 November: Changed Coude Room O2 Sensor
  • 27 November: Swapped Helium Compressor No. 3 (New) in for the air-cooled one, which will remain as a spare for the time-being.
  • 01-04 December: Warm-up for pump replacement. Cool down on Dec 4th.
  • 09 December: Primary Mirror cleaned with CO2. This is generally done every week.
  • 12 December: OBSTAC Tweak for late-season observing. Add future year fields in case no Y1 or Y2 fields available.
  • 23 December: 3 AM-ish. K. Reil gets RASICAM going.
  • 04 January 2015: New hexapod LUT and new pointing model installed.
  • xx February 2015: New 4MAP LUT and default (need date)
  • 24 March 2015: Primary Mirror Wash
  • 30 March 2015: Change VR dome flat exposure time from 15s to 10s
  • 07 Avril 2015: Power incident. Heater crate turns off. FP cools to -115C before LN2 system turned off. Recovery in about 4 hrs. FP warmed to -95C during recovery before it went to normal -100C.
  • 20 April 2015: LN2 turned off for a bit anticipating the possibility for a problem like 4/20. Didn't happen. FP temp wiggle to -95C and back.
  • 25 May 2015: CRio failure (?), focal plane cools to -114 C in AM. Recovers quickly but goes to -95 before returning to -100C.
  • 8 June 2015: Red LEDs replaced in DECal, power level changed.
  • 25 June 2015: Partial camera warmup for major generator service (-60 C?)
  • 8 July 2015: Primary Mirror Wash
  • 25-31 July 2015: Full warm-up for engineering: LN2 line replacement, glycol service, DTS upgrade.

DES Year 3, 2015 to 2016

  • 31 July 2015: Camera cooldown. 1S and 1R pipes replaced. 1S vacuum isn't as good as we'd like.
  • 9-13 Aug 2013: Full warm-up (missing telemetry during service?)
  • 12 August 2015: New RASICAM primary mirror installed.
  • 25-29 Aug. 2015: Full warm-up to repair 1S. New focal plane cooling algorithm implemented, which changes focal plane operating temperature (by -1 C?)
  • 23 September 2015: Primary Mirror Wash
  • 26-Nov. 2015: Thanksgiving Day Curse returns with an early morning focal plane temperature drop from -100C to -120C and back.
  • 15-Dec. 2015: Change to flat field lamp illuminations. See DJJ.
  • 16 December 2015: Primary Mirror Wash
  • 19-24 Feb 2016: Dusting of inside of C1. Also full warm-up for cooling system maintenance and upgrades.
  • 2 Mar 2016: Change exposure times of dome flats.
  • 9 Mar 2016: Primary Mirror washed
  • 25 May 2016: New Hexapod LUT, New 4MAP LUT, New pointing map based on DONUT analysis of DECam images. See des-docdb-9233.
  • 28 May 2016: Backed out hexapod and pointing LUT. Pointing errors too large.
  • 23 July 2016: Focal plane temperature excursion of a few degrees C.

DES Year 4, 2016 to 2017

  • 16-17 August 2016: New Hexapod and Pointing LUTs installed.
  • 16-17 August 2016: Installed and tested active 4MAP control "AOS8" based on donut analysis.
  • 17 August 2016: Primary Mirror washed
  • 23 August 2016: AOS8 made default.
  • 13-18 October 2016: Full warm-up for LN2 line replacements and LN2 pump swap.
  • 2 November 2016: HA axis damping behavior after a long slew worsens. Damping not quite settled until 2 seconds into next exposure.
  • 12 November 2016: LN2 Pump stopped 45 minutes. Focal plane temperature excursions of a couple degrees C.
  • 13 Dec 2016: Full warm-up after power failure. Still warm on the 14th.
  • 15 Dec 2016: Cool down prior to start of observations this night. Hexapod problem discovered.
  • 16 Dec 2016: hexapod problem fixed. This night the "OPENSHUT" keyword was sometime in 1970.
  • 17 Dec 2016: OPENSHUT keyword fixed this afternoon. 1/2 deg. C focal plane temperature wiggle.
  • 21 Dec 2016: Primary Mirror washed
  • 26 Dec 2016: Full warm-up after roughing pump failure
  • 29 Dec 2016: CCD S30 comes back alive. No cause established. See Nov. 2013.
  • 9 Feb 2017: Minor warm-up (5 K) on controller power loss.
  • 21 Feb 2017: Minor (5K) warm up.
  • 22 March 2017: C1 cleaned inside and out.

DES Year 5, 2017 to 2018

  • 10-14 July 2017: Full warm-up for LN2 servicing
  • 19 July 2017: 6 deg. C temperature down spike.
  • 3 Aug 2017: r filter moved to VR slot
  • 15 Aug 2017: Start of Y5 on night of Aug. 14 (eng. night w/ TOO)
  • 16 Aug 2017 to 20 Aug 2017: Problems with the dome. At the start of this period it overshoots. On the 18th a software change. Through the night of the 20th it is slow to start moving, start of exposures with >8 deg slew are occluded at the beginning.
  • 3-6 Sep 2017: Full warm up the camera to solve 1r line vacuum problem.
  • 21 Sept 2017: 2 deg focal plane temperature wiggle. We were not on the sky.
  • 3-5 Nov 2017: Full warm up the camera for O2 sensor failure.
  • 10 Nov 2017: N15 light bulb turns on.
  • 15 Dec 2017: LN2 Pump failure. Full warm-up
  • 6,7 Jan 2018: All flat-field sources realigned.
  • Feb 2018: N15 light bulb appears gone by now.
  • 14 Mar 2018: controls crapout, focal plane temp does a wiggle down to -115 C.

DES Year 6, 2018 to 2019

  • 18-24 Mar 2018: Full warm up LN2 pump replacement, Filter Changer and Shutter Maintenance
  • 17 Apr 2018: -115C to -95C Focal Plane Temp wiggle
  • 19 Jun 2018: Full warm up Mirror re-aluminization.
  • 11 Jul 2018: Camera cool-down after re-aluminzation.
  • 20-28 Aug 2018: Filter Changer removed, repaired, reinstalled.
  • 31 Aug 2018: Problem with 1/2 of CCD N10 (#41) upper starts with images after 767395, which is the final image taken on August 30/31, 2018.
  • 4 Sept 2018: Clock card 8 replaced, N10 upper fixed, but DECam is short a backplane (?) (9 detectors) for the evening.
  • 5 Sept 2018: DECam problems of past few days all fixed.
  • 8 Sept 2018: CCD N10 (1 amp) bad serial CTE problem recurs starting with exposure 771163
  • 12 Sept 2018: CCD N10 amp OK, now expected to be intermittent, correctable with a script.
  • 24 Sept 2018: Transition card and cables for N10 swapped out. New data block at image 777289. Last N10 problem was Sept. 18.
  • 26 Oct 2018: A.W. reported that the chiller in the dome that cools the cage is out of commission.
  • 02 Nov 2018: It's reported that CCD 27 has 2% higher response in Y6 Y-band dome flats than it did before. This is attributed to a possibly higher CCD temperature. griz responses are "rock solid".
  • 18-22 Nov 2018: Full warm up LN2 Pump Replacement
  • 18 Feb 2019: For Primary Mirror Reflectance and Scattering Measurements.

 ACTR document with the measurements of the primary mirror: CTIO Docdb 695.