ANDICAM - Phase II Submission Instructions
UNDER CONSTRUCTION. Thank you for your patience. Please use this ANDICAM Phase II menu for now.
1. The ANDICAM Phase II Observing Program Submission Process
Once a project has been approved for observing with ANDICAM, the PI needs to submit a detailed "Phase II" observing plan to the queue manager. This plan consists of a set of observations to be carried out, each "unit" observation of which is described by one or more data-acquisition template files called "Observation Templates" or "obs files". A set of written instructions for how to use these obs files to execute your program observations rounds out the Phase II submission package. Other elements may be required for more complex programs or those with long target lists.
We have adopted a web-based electronic submission process built around a suite of simple web forms. The observing program submission procedure is analogous to "Phase II" observing preparation for Hubble Space Telescope, Gemini, WIYN, or the ESO NTT and VLT. Indeed, many ideas and examples from those systems were used in considering the development of these tools. The CTIO 1.3m has only one instrument with three well-defined observing modes, so it should be very straightforward to use efficiently, and the Phase II process is therefore proportionally simpler than these other observatories. Our highly successful experience with this observing mode at the Yale 1-meter telescope bears out this expectation.
2. What kind of observing program do you have?
ANDICAM observing programs generally come in three broad types: Synoptic, Survey, and Target-Of-Opportunity.
Synoptic Programs
These are programs in which you wish to observe a relatively small number of variable targets many times over the course of an observing season. For these programs, you will generally create a small number of single-target obs files that will get executed many times.
In cases where each target requires either a large number of separate obs files (e.g., 5 obs files, one for each of the UBVRI filters), or a small number of obs files to be executed in a specific sequence with repeats (e.g., I+H and V+H dual-mode observations, executed in the sequence "VIIV"), you have the option of creating an additional multi-observation script that will define the observing sequence made up of particular obs files. For "complex" synoptic programs, use of multi-observation scripts can improve the efficiency of your program (there is less for the on-site observer to type).
Survey Programs
These are programs in which you have a relatively large number of targets that you wish to observe only once, but in many filters. For example, a program to obtain UBVRI+JHK images of a sample of 100 cool subdwarf stars would require you to create nearly 500 separate obs files using only single-target files. This is clearly ridiculous.
To make execution of such programs easier, you would settle on some standarized set of exposure times (e.g., either one set for all in each filter, or divide your sample into long and short exposure time groups), and then create a small set of multi-target obs files for each generic set. These obs files have no target specifications (name and coordinates), only filter and exposure settings.
You would then create a multi-observation script to wrap around these generic obs files which would be executed by the on-site observer to take your observations. The script prompts the observer to enter the target name each time, ensuring that the correct object name makes it into the header. A complete target list with coordinates, specifying which are to be observed with which multi-observation script, rounds out your Phase II submission.
For our example of the 100 star UBVRI+JHK observing program, we might divide the sample into two exposure time sets (long and short), and create 5 multi-target obs files for each filter combination for each set. This would reduce the number of Phase II observing files needed to execute the program from 500 to 12: 6 files for each of the long and short sets, each of which consist of 5 multi-target obs files and 1 multi-observation script to execute them. The advantages of combining generic obs files with scripts is considerable.
Target-Of-Opportunity (TOO) Programs
These are one-time programs triggered by transient events like novae, supernovae, gamma-ray bursts, etc. TOO programs must be pre-approved (if NOAO time), or you must warn the queue management team in advance if you think you will be requesting TOO observations. Politics apply, so keep on top of the situation. In the event of super-rare events (e.g., a Galactic or LMC/SMC Supernova), we will coordinate all observations by the SMARTS consortium as a whole.
TOO programs will be treated like single-target synoptic programs. You will need to prepare single-target obs files (and any optional multi-observation scripts) in advance of calling in the TOO trigger, and submit a Phase II program using the regular forms.
Important! TOO programs requested after 3:30pm Eastern Time (1530 EST or EDT) will be executed that night at the discretion of the queue manager, otherwise they will be deferred to the next night. We remind all principal investigators that they are specifically forbidden to call up the on-site observers directly, and note that the on-site observers are empowered to hang up on you politely (in fact, they are instructed to ignore all such extraneous calls). Maintaining a clear, unambiguous "chain of command" is absolutely essential for smooth remote queue operations, so no matter how important it is, please work through the proper channels. Repeat offenders will have their observing programs terminated with extreme prejudice.
3. Submitting an ANDICAM Phase II Observing Program
The Phase II submission process has 3 steps, as follows:
Step 1: Login to the Observation Preparation Pages
Projects awarded time with ANDICAM on the CTIO 1.3m Telescope (either via the SMARTS consortium or the NOAO TAC) are assigned a Project ID code their programs. This ID is your entry point into the observation preparation tools on this web server. Using these tools, you can create obs files for your observing program, and edit or delete existing obs files. The obs files reside on the server in separate directories assigned to each approved project.
If you do not find your project ID in the list, contact the SMARTS queue manager or NOAO SMARTS Program Coordinator (as appropriate). Please do not appropriate someone else's project space.
Step 2: Create Observing Templates and Scripts
Observation Template Files ("obs files" for short) are used by the ANDICAM data-taking system to execute your observations. A set of web forms are provided to help you create obs files that have the correct format (eliminating syntax errors if you try to make them by hand). In addition, these obs file creation tools provide estimates of the amount of time required to execute an observation, exclusive of target acquisition and setup time. These estimates will help you optimize the data-acquisition for your program, especially for those programs that will be acquiring simultaneous IR and CCD images. Instructions for the creation of obs files is described in the ANDICAM Observation Template Files document. Please read this document carefully.
If you are submitting a survey-type program, you also need to create a multi-observation script to execute your multi-target obs files. Programs that make use of multi-target obs files are required to submit valid multi-observation scripts with their Phase II program.
Similarly, if you have a complex synoptic program, you may wish to also prepare one or more optional multi-observation script to execute your single-target obs files in a particular sequence. These scripts are optional for single-target programs, and should only be done if necessary.
Step 3: Submit the Phase II Observing Program
Once you have created a complete set of obs files for all targets in your program (or a set of multi-target obs files and associated multi-observation scripts), the Phase II Observing Program Submission Form is used to submit the obs files, multi-observation scripts, and your detailed narrative instructions to the ANDICAM queue manager for eventual scheduling and implementation. This form is accessed from the main observing preparation menu after you have logged in (see Step 1 above).
The Phase 2 submission form copies your observing files and the instructions (in an ASCII text file) to an ftp staging disk, and emails the instructions to the queue management team (via the andicam-submit mailing list). Once on the staging disk, the files will be downloaded by the queue manager or by the on-site observers (at the queue manager's instructions) to CTIO where they will be used to execute your program when it is scheduled. Click here for detailed instructions for using the Phase II program submission form.
In preparing your Phase II submission for execution, the queue management team, in consultation with the instrument scientists, may require changes in your program if the proposed observations present an unusually complicated execution problem, or present book-keeping or execution-time problems for the on-site observers. Programs that require excessive execution time may be sent back for reduction in scope.
In addition to submitting your Phase II observing program, these forms can be used to submit updates, corrections, and amendments to your program. The nature of the submission should be clearly spelled out at the top of the narrative instructions.
NOTE: Phase II files and instructions must be submitted by 3:30pm EST/EDT for the targets to be scheduled in that night's queue. Please contact the queue manager if this deadline presents a major problem to your program's feasibility.
4. Observation Execution Time Estimation
Time with the ANDICAM is generally awarded as a given number of hours of observing time to be devoted to a project. "Observing" in this context includes not only the amount of time spent collecting photons, but also the target acquisition, setup, and system overheads. These overheads are included in your time allocation (same as they would be if you were awarded, for example, a night of observing or some number of HST orbits). How efficiently you can observe determines what fraction of your time allocation is spent collecting photons.
Each obs file you create includes an estimate of the execution time. This execution time only includes the operations that occur between the observer typing "GO" for that obs file and when the last image in the obs file has finished being readout. This overhead calculation includes detector setup and readout times, as well as the requested integration times. These times are all highly predictable. Separate estimates are given for the CCD and IR channels (as appropriate), to help you evaluate how much deadtime there will be (the execution time for an obs file is the longer of the two estimates). Details on how this estimate is computed is included in the ANDICAM Observation Template Files document.
This is not the whole story, however. The individual obsfile execution time estimates do not include the following unpredictable overheads:
1. Instrument configuration times, which typically run a 1-4 seconds per operation, and usually at most 10 seconds per obs file. These times are so short to be treated as "noise" in the process. For example, if the filter and exposure times from the previous obs file are the same as yours, there is no overhead, but if they must be changed, it takes 1 second to change the exptime and 1-4 seconds to change a filter (depending on the amount of motion required). Hence why we don't compute them since they depend unpredictably on what observations came before.
2. Telescope Pointing & Target Acquisition (TP&TA). This typically requires between 3 and 5 minutes per pointing, and includes such unpredictables as telescope slew time, target acquisition offsets (though the 1.3m points admirably well, sometimes your target may be hard to identify the first time), and guide star acquisition (time to select a suitable guide star and lock the guider). If your program is simple (point to a target and take a series of images using a set of obs file), this TP&TA overhead will have a minimal impact on your time allocation. However, if your program requires a lot of pointings or offsets at a particular location, it will cut into the observing efficiency because of the greater TP&TA requirements, ultimately reducing the amount of your time allocation that is actually spent collecting photons.
In estimating the amount of time your Phase II submission will require, the ANDICAM queue managers at Yale will factor in a nominal TP&TA overhead, and combine the overheads from your individual obs files. If the amount of time you request exceeds your allocation, or looks to present an unusually low-efficiency observation, you will be contacted by them and asked to revise your observing project so as to both meet your scientific needs and have a program that does not present execution problems for the 1.3m on-site observers.
Note that requests for additional calibration data (e.g., special flats other than the ones acquired generally for all projects, or special flux calibration targets) do count against your time allocation. These are handled on a case-by-case basis by the queue management team at Yale.
5. Phase II Submission Checklists
Synoptic or TOO Programs
- A set of single-target observation template files, with correct object names and coordinates.
- [optional] one or more multi-observation script files used to execute your obs files in a particular sequence (only as required).
- A Phase II Instructions Narrative that includes:
-
A summary target list, including coordinates.
- Instructions for what observations to take in what sequence (name the specific obs files or multi-observation scripts to be used).
- Conditions under which your program should and should not be observed (e.g., photometric conditions, or OK in any conditions; good or bad seeing is tolerable/intolerable, etc.).
- Any special instructions (e.g., offsets or particular placement of targets in the IR or CCD detector fields-of-view, bright star avoidance, etc.). If your program requires multiple offset pointings for a given target, please specify the offsets and which obs files are to be executed at each offset position.
- Any special disposition of the data (subject to review and approval by the queue management team).
- Check the boxes next to the obs files and multi-observation scripts to be submitted with these instructions (sometimes people forget!).
- If a finding chart is required, please give a WWW or FTP address where this may be picked up. GIF, JPEG, and PostScript are required. Finders must be labeled with the scale (arcsec) and the orientation (directions of N and E at least). Black on white-sky negative images work best.
Survey Programs
- A set of multi-target observation template files to be used for all of your targets (or subsets by exposure time groups).
- One or more multi-observation script files used to execute your multi-target obs files, making sure to include the prompt for the target name. These are required for survey programs.
-
A Phase II Instructions Narrative that includes:
- A complete target list with coordinates (RA and DEC in J2000 equinox, in hh:mm:ss and dd:mm:ss format for RA and Dec, respectively). Names must be simple and not contain any 's "s or other special characters. Long target lists may be provided off-line using URLs pointing to web pages or ftp sites with ASCII text versions (no Word or PostScript or other formats, please!) of these tables if they are too large to conveniently cut-and-paste into the submission form.
- Instructions for which multi-observation script is to be used to observe each target.
- If the same script is used for all targets, be sure to specify what the script does, and roughly how long it will take (add up the execution times for each obs file executed by the script, and add 5s of setup overhead of each obs file).
- Conditions under which your program should and should not be observed (e.g., photometric conditions, or OK in any conditions; good or bad seeing is tolerable/intolerable, etc.).
- Any special instructions (e.g., offsets or particular placement of targets in the IR or CCD detector fields-of-view, bright star avoidance, etc.). If multiple offset pointings for a given target are required, please specify the offsets and which obs files are to be executed at each offset position.
- Any special disposition of the data (subject to review and approval by the queue management team).
- Check the obs files and multi-observation scripts to be submitted with these instructions (sometimes people forget!).
- If a finding chart is required, please give a WWW or FTP address where this may be picked up. GIF, JPEG, and PostScript are required. Finders must be labeled with the scale (arcsec) and the orientation (directions of N and E at least). Black on white-sky negative images work best.
6. Program Tracking
In general, you will be informed by email when observations for your program have been acquired and are ready for pickup from the FTP server at Yale. In between times, however, you can track the progress of your program electronically in a number of ways:
ANDICAM Email Lists
You can receive a copy of the nightly observing logs by subscribing to the andicam-logs mailing list. Instructions for subscribing and unsubscribing to this list may be found in the ANDICAM Mail Lists document.
Observing Log Search Tool
When you login to the Phase II Observing Tools page for your project, a simple search tool is provided that will extract all observing log entries for your Project ID. You can also extract observing log entries for a restricted range of dates, or for which a particular keyword (e.g., a target name) matches. All searches on keywords are case-insensitive. This allows you to get to the info you need on your program's observations without having to wade through all of the others.
7. Problems and Feedback
If you have problems using these web tools, or have comments that you think will improve their usefulness, please send your feedback to the Queue Management personnel at SMARTS2 at Yale (see the contact list for the current personnel).
Updated: 2006 May 11 [rwp/osu]
Printer-friendly version