ENR CTIO Blanco 4-m 2013-10-02

Some major problems (all most likely due to network problems) that should be looked into, fixed, and verified as fixed before afternoon calibrations: * Network problems meant that the DECal flat field system and the BCAMS could not be use tonight. * Network problems also caused difficulties with TCS at the beginning of the night. * Network problems likely caused the Observer1 console Exposure Table form to stop updating (a reset fixed this). * Network problems likely caused the Observer1 console Telemetry Viewer GUI to stop updating. * Network problems prevented submitting the CTIO Night Reprort via the CTIO web submission form. * Network problems prevented checks of files needed for GPS monitor analysis. * Wifi in the control room does not seem to be working. Here are some notes derived from today's entries in the DECam Electronic Notebook: * At around 16.30 LT, Received a message in the vnc system1 window - 'Hard disc failure'. Then observer1 crashed so we rebooted. Hernan contacted CIS. * We shutdown and restarted SISPI. The first configure failed, and the second waited for a few minutes - much longer than usual. We restarted SISPI again but same problem - DECal was failing (17.08). We restarted the DECal role, and tried configuring again, but again it was taking ages. Hernan restarted the DECal computer. * 17:56 LT: SISPI again not configuring - now BCAM fails to configure. Lots of connection errors today... Ann suggests BCAM may have to be power-cycled after rebooting the machines...Hernan tries this. OCS is stuck on (INITIALIZING), doesn't seem to be timing out, so we restart SISPI again. * 19:52 LT: Think problems stem from a network issue. (Niall's laptop won't connect to wireless or ethernet). Umberto tried resetting the 'main switch'. Hernan has restarted DECAL computer and server. Also tried power-cycling BCAM. Still, we've only been able to configure without BCAM and DECAL. Therefore no calibrations this afternoon. * 20:50: TCS problem (apparently they can't connect to the telescope) * 21:17 LT The TCS was also afflicted by the network problems. After a re-boot and manual recovery of the config files from backup, we are up and running at 21:05 LT. * At 21:45 LT, obstac queue stalled for a few minutes due to TCS network problem.