ENR CTIO Blanco 4-m 2012-12-13

David Walker: The reason the emails did not go out was because yesterdays night report did not fill in its weather fields properly, I fixed that, but to avoid sending tonnes of test emails to everyone I changed the send email address to my own, and forgot to reset it to the end of night Blanco email alias. Sorry. Here goes.

The observer (Daniel Kelson) filled out the night report and went to bed.  His report did not go through though, so I have copied the information he input here, with some additional details.  There was one major problem at the end of the night: unrecoverable interlock errors having to do with the FCM. This will need to be remedied before observations can continue for the next night.

Minor problems:

(1) Pointing offsets at new fields were of order 2-3 arcmin. Mostly correctable but not to great accuracy.

(2) Glitches in the guider (or in hexapod controls) can be seen as sharp jumps of order 1-2 arcsec in centroids of the guide stars. These result in elongated images, with \epsilon=0.1-0.25.  So is a jump in the hexapod causing the guide stars to suddenly move? Or is a glitch in the guiders causing such a large perceived centroid change? Which is the chicken and which is the egg? 

Additional details from C. Kaleida:

At the beginning of the night, while I was going the pre-observer setup, there was a communication error between SISPI and TCS (around UT 20:35 13 Dec. 2012). This error occurred after I restarted the instance and GUIs, and then Configured in the Observer Console. TCS was up and running with no problems when this error occurred. To resolve this, I restarted the TCSINTERFACE role, the TO (Hernan Tirado) restarted TCS, and then I Configured the Observer Console again. After this calibration frames proceeded as usual. 

Between ~UT 03:34 and UT 06:38 we experienced occasional high ellipticity (i.e. 0.25 or greater), in between images of otherwise stable seeing and ellipticity. This appears to be the same previously known problem, which we are trying to gather more information on (Tim Abbott's Dec. 10th email). There may be a correlation between these high-ellipticity images, "dody" on the Donut Summary 1, the "hexdy" on the Hexapod Summary, and a swing of order 1.0 arcsec on the Centroid Evolution in the Guider GUI. Images with high ellipticity included 161054, 161055, 161060, 161072, 161074, 161078, 161091, 161093, 161095, 161106, and 161108.

Everything else went very well until the end of the night when we had interlock errors around UT 07:35 13 Dec 2012, in the middle of a standard star script.  In the Observer Console dialog box, the message read "PML at 6:06: Can't talk to component FCM. Error. Cannot insert alarm: (ProgrammingError_ column "alarm_id" is of type integer but expression type boolean LINE 1:"  I've tried restarting various roles that had errors (FCM, OCS) and reconfiguring, and have restarted the SISPI instance twice.  The errors have persisted.