CTIO
Published on CTIO (http://www.ctio.noao.edu/noao)

CTIO Home > MOSAIC II CCD Imager > Known Problems

Known Problems

  • In the 16 channel mode, one of the amplifiers is not working [1]. [2]
  • Arcon crashes [3]
     
  • Header Information [4] is sometimes not present
  • Absolute timing [5] of exposures.
  • Focusing [6]
  • Temperature control [7] - if you are using the z' filter, read this!
     

 

Amplifier failure

A note on the recent amplifier failure on the Mosaic II camera

 

THIS INSTRUMENT HAS BEEN RETIRED. INFORMATION PROVIDED HERE IS FOR REFERENCE ONLY

Here is a schematic layout of Mosaic 2:

The CCD4, amplifier A died in 2009. Hence, nothing is read out from this part of CCD4. You have two options:

  • (1) observe in 16 channel mode, and 1/16 of your image will not be usable.

An example image is shown here (click to enlarge):

[8]
* In 16 channel mode the read-out time is 100 s

 

  • (2) observe in 8 channel mode and read out through the b amplfiers.

The whole image will be usable as seen here:

[9]

In 8 channel mode the read-out time is 150 s

 

Back To Mosaic II [10]

Page created: April 2010 by Andrea Kunder

Arcon crashes

What to do if Arcon crashes

Every so often, something or other happens which causes the Arcon software to hang, or otherwise get confused. In the vast majority of cases this can be fixed by simply restarting the software running on ctioa0. This takes only a couple of minutes.

 

On ctioa0:

  • Move the mouse pointer to an empty section of the desktop and hold down the right hand button to bring up the desktop menu. This is shown on the image below:

[11]

  • Select the item "End ARCON session"
  • Close black ARCON Status boxes by right clicking the menu in the top left, and selecting "Quit"

[12]

 

  • Move the mouse pointer to an empty section of the desktop and hold down the right hand button to bring up the desktop menu.
  • Select the item "(Re)Start ARCON" and release the mouse button.

[13]

  • Wait. After a few seconds the ARCON console, Acquisition, status and countdown windows will disappear, then reappear, and the software will go through the startup procedure.
  • You should normally answer "yes" to "do you want to synchronize parameters". This ensures that the detector parameters and filter positions, etc. match those in use prior to the re-start.
  • Any other windows on the desk top will be untouched so any processes running in them will be unaffected, and you can continue to work in these windows.

 

Some notes on how to avoid and cope with Arcon crashes.

Arcon crases can be avoided or ameliorated through the following practises:

  • Do not use the Mosaic II observing computer (ctioa8) for anything except observations
    Any data reduction, ftp-ing, script editting, web browsing or emailing should be done using another machine in the control room, such as ctiozm, ctioa7, or ctio4m
  • Don't edit scripts while they are running!
  • It is possible that the Arcon software may develop problems communicating with the telescope control system (router), particularly in trying to set the focus at the beginning of an exposure. If this is the case, a reboot may not be required. Ask your observer support staff to verify TCS communications, set the telescope focus manually, then use the GUI to tell the Arcon software where the focus is.
  • Do not use the "abort" command as this does not always function correctly. Instead, if you wish to terminate an exposure early, use the tchange command with a negative time interval to shorten the exposure time appropriately and force a graceful end to the exposure.
  • mosdither is known to feature increasing delays between exposures in a dither sequence. Use the mosocs [14] facility instead.
  • Should you lose one or more of the Arcon interface windows in the VNC window containing the IRAF interface, but not that containing the IRAF command line itself, you can restart the interface without restarting Arcon by typing "gwcclose" then "gwcopen" at the IRAF prompt.

 

Back To Mosaic II. [10]

 

Page created: 7 November 2006 by Tim Abbott
Page updated: April 2010 by Andrea Kunder

Fix Headers

The Problem:

There are times when the TCS does not collect information about an image, and hence, your image headers may not have all required information. You can see if your headers have the required information by typing in the iraf terminal:

cl> hsel obj*fits[1] $I,RA,Dec,filter yes

In this example, "obj2251.fits" does not have the RA, Dec, or filter in the header.

 

The Solution:

Before you do anything, make sure to save a copy of the original image.

You may use the iraf task "hedit" to update the parameters. Make sure to edit every single extension on your image.

You may download the task 'repairhdr' that will fix your headers as well. The advantage with downloading this tasks is that you only have to type one command, and then all 8- or 16-extensions will be updated.

repairhdr

The task "repairhdr" is a general task that can be used to fix any additional header information.
Download repairhdr here [15]

run the task

Place the task in a directory and make note of where you have saved it. (e.g.: /Users/mydownloads/)

Go to the directory that your images are in:
> cd /Users/myname/DATA/

Define the task:
> task repairhdr = /Users/mydownloads/repairhdr.cl

Call the task to update any fields:
> repairhdr obj2251.fits RA 06:45:53.74 ver-
> repairhdr obj2251.fits DEC -45:10:06.70 ver-
> repairhdr obj2251.fits filter "g SDSS c 6017" ver-

Back To Mosaic II [16]

Page created: February 2011 by Andrea Kunder

Limitations in absolute timing

2003 10 02: Limitations in absolute timing of Mosaic exposures

The original problem(s), described here [17], have been partially resolved. The residual errors in FITS keyword timing are not likely to present problems for the vast majority of observers, but some should still be aware of the following.


Background

There are a number of FITS header keywords relevant to Mosaic exposure timing. Among them are:

  • DATE-OBS - this is the UT time and date of the beginning of the exposure as derived from the Arcon detector controller. Arcon timing is now derived from NTP and should be quite precise.
  • LSTHDR - this is the local sidereal time as acquired from the telescope control system at a time near to the beginning of the exposure. The telescope's LST is derived from a GPS system sited in the 1.5-m dome and is believed to be well-behaved.

The majority of other Mosaic header time values are derived from DATE-OBS by calculation.

 

Observed behaviour

In the absence of a known, precise time source with which to directly compare DATE-OBS and LSTHDR, we can only reliably report on the differences between the two. Previously [18], significant drifts and glitches between the two have been observed and have compromised some data. Since the Mosaic run of May 2003, the Arcons have been synchronising their clocks via NTP and time values are considerably better behaved.

Besides a fixed offset of a few seconds, local sidereal time as calculated from the DATE-OBS keyword matches that reported in the LSTHDR to within an envelope ~1.5 sec. wide. The variation of differences between the two clocks shows structure which suggests truncation error, or something like it, may be the cause.

LSTHDR is subject to occasional dropouts, possibly related to communications problems with the TCS, in which it is not available in the Mosaic FITS header.

Absolute timing values are not reliable in dark and focus frames.

 

More details

The graph below shows the difference between LST as calculated from the DATE-OBS keyword and that reported in the LSTHDR keyword for 2003 up to September. The large deviations are all associated with dropouts in LSTHDR, sequences of darks and focus frames.

The graph below is the same as above, but expanded on the y axis. Note the significant improvement in bahaviour of the two clocks starting in May 2003. After this date, points more distant than ~1sec from a difference between the two clocks of ~4sec are associated with focus frames.

The plot below shows data from 3 nights in June 2003 on an expanded scale to show the behaviour in the course of a night. There is clearly some quantizing effect, but no longer any large-amplitude steady drifts.

back to Mosaic II [10]

 

Problems with Absolute timing

2003 03 10: Problems with absolute timing of Mosaic exposures

There exist unresolved problems with the absolute timing of Mosaic exposures. For most programs, these are not likely to present serious problems but observers should be aware of the following.

Background

There are a number of FITS header keywords relevant to Mosaic exposure timing. Among them are:

DATE-OBS - this is the UT time and date of the beginning of the exposure as derived from the Arcon detector controller. Arcon timing is derived from the host computer at boot time.
LSTHDR - this is the local sidereal time as acquired from the telescope control system at a time near to the beginning of the exposure. The telescope's LST is derived from a GPS system sited in the 1.5-m dome and is believed to be well-behaved.
The majority of other Mosaic header time values are derived from DATE-OBS by calculation.

Symptoms

The free-running Arcon clock appears to suffer from glitches and drifts:

The glitches result in discontinuous changes in the Arcon timing of amplitudes of several days. These tend to be obvious to the observer and will impact automated data reduction procedures which are sensitive to DATE-OBS. Their cause has not yet been identified.
Unfortunately, the Arcon internal clock is inadequate for precision timing on its own. Drifts of several seconds over the course of a night may be observed.

The cure

Arcon timing is reset at boot time, so drifts and glitches can be corrected by rebooting the system. It is advised that this be done at the beginning of each night anyway.

The offset between LSTHDR and DATE-OBS can be calibrated by an exposure taken immediately after a reboot. Compromised data can be repaired by back-calculating UT from LSTHDR and applying this offset.

This procedure should result in a precision of better than 1 sec.

More details

The graph below shows the difference between DATE-OBS and the keyword WEATDATE. WEATDATE is derived from the weather station and provides an additional comparison, although it should not be used as any form of timing reference. The large excursions near October 2001 should be ignored, but the points around 3 days offset in 2002 illustrate the large timing glitches. (The offsets are all plotted as if positive, but many are negative.)

[19]

The graph below shows the timing drifts as sideral seconds in the 2001 and 2002 calendar years.
[20]


Finally, this chart shows a three week period with the offsets color-coded by type.
[21]

Focusing

A Note on focussing the Mosaic camera

Getting best focus with the Mosaic camera can be a tricky business, here's why.

(For basic instructions on measuring and setting focus go here [22].)

 

Observations:

  • When seeing is 0.9" or better, the focus parabola as measured by mscfocus becomes flat-bottomed (well, actually, it becomes noisy) if a focus step of <=100 units is used. Under these circumstances, mscfocus becomes useless, if not actually dangerous -- it is trying to get best focus by fitting a parabola to noise. Worse, mscexam uses only the focus value with the smallest FWHM and its two nearest neighbours; that is, only three points on the focus parabola are input into the calculation.
  • When seeing is worse than this, clean parabolae are generated and mscfocus works just fine (provided that seeing is stable, of course -- if the seeing is varying while the focus sequence is collected, then all bets are off.)
  • If seeing is ~0.8" and a focus step of <=100 units is used, it becomes difficult to do "focus by eye" - the focus sequences look pretty much the same over several steps.
  • FWHM is better on the left half of the mosaic than the right, and there is a smooth transition between the two. (Best FWHM is to be found near the ends of CCDs 2 and 6, worst in CCD 4). If the seeing is ~0.8", roughly the same distribution of FWHM variation may be seen across several 50 unit focus steps. This distribution is also present in images of worsening seeing, but becomes less pronounced (I see it in images of well over 1" seeing). Most observers don't see it because they don't take the time to collect enough focus stars and play with mscfocus.
  • The best seeing I've seen in the last two mosaic observing blocks has been 0.75" (in R).

Conclusions:

  • The optical system is limiting the best delivered seeing to 0.75-0.8" in R.
  • The variations in FWHM across the field reflect distortions introduced by the telescope & corrector.
  • The actual seeing can be better than 0.75-0.8". Under these circumstances, changing the focus across some range of values (of width dependent on actual seeing) will change delivered seeing in an unpredictable manner because you're sampling optical distortions, not seeing, and mscfocus is rendered useless.
  • The well-known "W" shape of seeing variation through focus is an artefact of the distortions present in our particular optical system.
  • Chris's technique of looking for the roundest images in the focus sequences to find best focus may work well in good seeing, but the correct method would be to fit the wings of the focus parabola for out-of-focus image sizes >~0.8". We don't currently have the resources to impliment this.

Therefore we recommend:

  • Focus sequences should cover at least 500 focus units, probably more, especially in good seeing.
  • When the seeing is better than 0.9", there's a good chance mscfocus is giving you a bad focus value.
    Under these circumstances, I can think of three ways to get decent (if not best) focus:

1. focus-by-eye with a large focus step value

2. Chris's roundest-image technique.

Do an "mscexam" and hit "m". If the resulting profile looks North-South elongated like this:

focus_sm.jpg [23]

then you must decrease the focus value.

3. Collecting lots of stars in a focus image with ~100 unit steps and aiming at the middle of the flattish/noisy portion of the through-focus FWHM curves.

Back To Mosaic II [10]

Page created: 23 March 2004 by Tim Abbott
Updated: April 2010 by Andrea Kunder

Temperature control

Mosaic temperature control: a warning. If you are using z' filter, read this!

In late 2003, it was discovered that the temperature control loop for the CCDs in the Mosaic camera is not closed. That is, the temperature of the focal plane is free to vary according to its instantaneous thermal load and coupling quality with the LN2 tank. A warning is sounded if this temperature moves more than ten degrees away from 165K, but otherwise, the temperature can vary by several degrees Celcius within a night and by as many as 10-12 degrees Celcius over the course of an observing block.

It is not clear how long this has been the case. Possibly (probably?) since the instrument was commissioned.

The consequence will be to produce QE variations at the red end of the spectrum. In particular data taken with the SDSS z' filter will be affected, as for any other filter where the red cutoff is set by the drop off of the CCD QE rather than the filter itself. The variation in QE of a silicon detector can show up to 1% variation per degree temperature change in the far red.

 

Created: 23 March 2004, by Tim Abbott

Source URL (modified on 02/20/2012 - 12:17): http://www.ctio.noao.edu/noao/content/Known-Problems

Links
[1] http://www.ctio.noao.edu/noao/content/Amplifier-failure
[2] http://www.ctio.noao.edu/noao/content/amplifier-failure
[3] http://www.ctio.noao.edu/noao/content/arcon-crushes
[4] http://www.ctio.noao.edu/noao/content/fix-headers
[5] http://www.ctio.noao.edu/noao/content/limitations-absolute-timing
[6] http://www.ctio.noao.edu/noao/content/focusing
[7] http://www.ctio.noao.edu/noao/content/temperature-control
[8] http://www.ctio.noao.edu/noao/sites/default/files/instruments/imagers/ngc2808_16.jpg
[9] http://www.ctio.noao.edu/noao/sites/default/files/instruments/imagers/ngc2808.jpg
[10] http://www.ctio.noao.edu/noao/content/mosaic-ii-ccd-imager
[11] http://www.ctio.noao.edu/noao/sites/default/files/instruments/imagers/crash.jpg
[12] http://www.ctio.noao.edu/noao/sites/default/files/instruments/imagers/crash7.jpg
[13] http://www.ctio.noao.edu/noao/sites/default/files/instruments/imagers/crash8.jpg
[14] http://www.ctio.noao.edu/noao/content/MOSOCS
[15] http://www.ctio.noao.edu/noao/sites/default/files/instruments/imagers/repairhdr.cl
[16] http://www.ctio.noao.edu/noao/content/MOSAIC-II-CCD-Imager
[17] http://www.ctio.noao.edu/noao/content/Limitations-absolute-timing
[18] http://www.ctio.noao.edu/noao/content/Problems-Absolute-timing
[19] http://www.ctio.noao.edu/noao/sites/default/files/instruments/imagers/timing3.gif
[20] http://www.ctio.noao.edu/noao/sites/default/files/instruments/imagers/timing1.gif
[21] http://www.ctio.noao.edu/noao/sites/default/files/instruments/imagers/timing2.gif
[22] http://www.ctio.noao.edu/noao/content/cookbook
[23] http://www.ctio.noao.edu/noao/sites/default/files/instruments/imagers/focus1.jpg