Hydra Status Report, Notes from the meeting April 2000 - Updates 2002, 2003

Knut Olsen, 4/3/2002

Updated 5/24/2002

Updated 7/14/2003

People and jobs assigned, in order of priority. See list below for job descriptions. Updates in light blue.

• Andres: 10, 17, 3
• Daniel: 3, 15, 16
• Javier: 1, 4, 5
• Knut: 3, 18, 20
• Manuel M.: 17
• Nick: 14, 20
• Oscar: 2, 14 (started), 19, 20
• Ramon: 11, 12, 13
• Ricardo: 3, 15, 16
• Rodolfo C.: 17
• Rolando: 4, 9, 7
• Tim: 2, 14
• Everyone: 1, 2, 6

SAFETY ISSUES:

1. A UPS power supply mounted on Hydra has failed. A consequence of this is that the red "panic button" is no longer functional. Moreover, when the Hydra carriage is dropped, the gripper is still able to move. Javier will notify us of the part that we have to re-order. Until it is installed, nobody should work on the gripper without Javier present. Javier points out: dropping the carriage or hitting the red button will, in fact, halt the gripper. However, this is done by activating the interlocks and applying the brakes to the gripper, rather than by cutting power. One might thus imagine that the gripper could move if the command to apply the brakes somehow fails. A redesign of the circuit is needed to have power cut. Done (7/2003) 
    
2. With the room being sealed against light leaks, it is potentially a fatal hazard if nitrogen, or another toxic substance, were spilled in the room. The entrance door should always be left open while filling the nitrogen flask. Tim, Oscar, and Ramon will locate and install a nitrogen detector in the room, to provide warning should nitrogen levels increase (and oxygen decrease) inside. (4/2003) Daniel's  reports installed: See Daniel's email [1].

POSITIONER:

3. A summary of the maintenance work to be performed regularly between observing blocks is needed. Will be done jointly by Ricardo, Daniel, Andres (gripper mechanics), and Knut. See the latest Hydra maintenance list. [2]
    
4. We saw several errors of "x-y stage not at destination". This problem has been traced to the Galil box. The spare box has been returned from Galil, but is still not functioning properly. Javier and Rolando have a good idea of which part is faulty, and are continuing to work on isolating the problem. While this work continues, the gripper and its associated electronics should not be touched without the knowledge of Javier and Rolando. FIXED. Problem was traced to bad EPROMs in the Galil boxes, have been reprogrammed. See Javier's e-mail [3] summarizing the work.
    
5. Most of one night was lost when the gripper was unable to move. The reported error was a tripped X limit switch. Javier was able to trace the problem to a faulty optocouple inside the Galil. Javier has temporarily fixed the problem, but a new part needs to be ordered. FIXED, see 4.
    
6. Fiber 74 was broken after it was misidentified as fiber 7 through the manual command "ThisIs 7". To prevent this from happening again, we should always use the command "thisis" instead of "ThisIs fiber_number" when recovering a lost fiber.
    
7. When Hydra is put on the telescope, the stowed positions of the fibers change slightly. This appears to be the result of a slight flexure between Hydra's off-telescope and on-telescope mounts. Rolando will correct the problem in software by implementing a lookup table to adjust the stowed positions. Done. Remember to also measure rotation of Hydra field at night.
    
8. After recovering from an "x-y stage not at destination" or similar error, we saw several instances in which the gripper would remove a fiber from the field, then replace it again immediately. Rolando describes this as a software "feature", which would require rewriting significant amounts of the Hydra software to fix. Not a high priority at the moment. Done. Rolando's new software is a huge improvement.

BENCH SPECTROGRAPH:

9. At the start of the run, we had a problem with the sequencer code and a capability missing from SetFiles. A temporary fix was put in place by Ramon with long-distance help from Marco Bonati. The problem appears to have been caused by unaccounted dependencies in a recent Arcon software upgrade. Rolando and Francisco are working together to implement a correction to SetFiles. Done.
    
10. Ramon and Javier had trouble with CommNode errors at the start of the run. The communication fibers apparently get bent at their entry point(?), and need a redesigned cover to prevent this from happening. Andres will look at it. Done.
     
11. One of Arcon's testpoint voltages was out of the allowed range, and produced frequent error messages. Ramon did not have a sufficient block of time to isolate the problem. As the images appeared to be unaffected, the problem was left alone. To work on the problem, Ramon needs to be able to illuminate the fibers while Hydra is off the telescope. Uncapping the fiber plug and using the dome lights will probably do the trick.
     
12. We saw occasional problems with a misbehaving bit somewhere in the Arcon acquisition system, which produced images that looked like they had bands of bad pixels. Ramon discovered that redoing the video calibration in Arcon fixed the problem temporarily. He is working on tracking down the cause of the problem. Done.
     
13. Ramon is continuing to work on why Hydra's original video board produces bias jumps of several ADU. It would be nice to have the board working, both to have a reliable backup and because the read noise was slightly lower with it installed.
     
14. Ramon and Knut spent the morning of the engineering night hunting for light leaks in the bench room. They found several, which they plugged with black tape. Oscar, Nick, and Tim are in charge of overseeing the significant safety issue (both to people and to the optical surfaces) involved with plugging the leaks permanently with black silicone. See also 16. Leaks plugged and room cleaned, awaiting reimaging to verify success. Done (7/2003).
     
15. Vacuum pumping of the nitrogen flask (necessary so as to make the nitrogen solid) continues to be a headache. While it would be wonderful not to have to continually pump the dewar, improving the thermal contact would likely need a complete redesign of the system. The limiting factor is undoubtedly the small volume and large surface area of the tube joining the CCD and the dewar; increasing the volume of this tube to improve the cooling would increase the obscuration of the beam and lower the spectrograph throughput. We thus need to become accustomed to using the vaccum system. Ricardo has written instructions for using the new, larger pump. The smaller pump should be used only as a backup. Ramon and Brooke have designed and fabricated new plumbing for the nitrogen filling and pumping unit, will be installed in May.
     
16. The bench spectrograph room is getting dirty, and the optical surfaces are suffering. The aluminum coating on the reflective mirror looks like it is degrading, and will probably need to realuminized. The glass camera corrector is very dirty.
    Part of the problem is that every time the curtains are opened, small fibers are released. If we can plug the remaining light leaks and seal all but one of the doors, then we could remove the curtains. Plugging the leaks completely is preferable to replacing the curtains with a solid structure, since this would limit access to the spectrograph. Sealing the room unfortunately brings a significant safety risk, if nitrogen or another toxic substance is spilled in the room, or if a fire consumes the oxygen inside. Room has been cleaned, but removal of curtains will wait.
     
17. The fiber mount doesn't reach high enough inclinations to provide compensation for the azimuthal angle of the echelle grating, because the encoder runs out of range. Andres will design an extension for the for the inclination encoder (time: 2days - 1 week), Manuel Martinez will connect the new encoder to the Smart Motor Controller (time: 1 day?), and Rodolfo will modify the SMC software to accept the new set of encoder values (time: 2 weeks). This will be done in May 2002. Mechanical work finished, Rodolfo working hard on software. Done (7/2003).
     
18. /ua13 data disk is filling up, Knut will track down a second disk. Adding another disk to ctioa0, an ancient SunOS machine, is difficult and was decided is too risky. Ron and Eduardo proceeded to repartition one of a0's existing disk to create a larger block of space, but this interrupted Ramon's progress with the CCD and has been put aside.

CALIBRATION SYSTEM:

19. The ThAr lamp was found burned out in March. Daniel replaced it, and reports that we have one spare remaining. Oscar will look into buying more. Daniel turned up three spare lamps, although two need modified housings.
     
20. Nick and Knut decided that a BG38 filter placed over the Ar penray lamp and ND1 filters placed over the He and Ne penrays would give balance to the penray wavelength calibration spectra. Oscar thinks that sleeves exist, but we have to make sure that the lamps won't overheat. We should test it off the telescope before installing the sleeves permanently on the chimney penrays.