Right after midnight (after the half-night handover) the quick reduce pipeline stopped reporting seeing and ellipticities. When we brought up the Portal display (the screen with the Stop/Start button on the left), we noticed "Disk Usage" at 100%!
There's a "Clean Disk" button on the left, but it was greyed out. After some fiddling, we realized that if we hit "Stop", the button was available.
We hit "Clean Disk", and that brought the disk usage down to 92%, but still red. Started the system, but it still didn't pick up new images.
We logged into quick1, fished around, and tried cleaning up in quickreduce/local/scratch (lots and LOTS of tmp directories).
We got the disk usage down to 82% by deleting some of those tmp directories.
At that point, when we hit Start, the system would appear to pick up one new image, reporting
"Transferring...",
"Processing...",
Number of CCDs Selected: 14
but nothing else would happen. It wouldn't report the seeing, and it wouldn't pick up the next image.
Tried stopping and starting repeatedly, and tried the 'reset' button a couple of times, but with no luck.
Total Times | Time Observed | Time Engineering | Time lost technical | Time lost weather | Time lost other | Total Program Time |
---|---|---|---|---|---|---|
2014-05-08 | 11.0 | 0.0 | 0.0 | 0.0 | 0.0 | 11 |
Other than the above, a stunning night with excellent (sub-arcsecond, even at high airmass) seeing for both observers!