0.9-m - June 2002

New Stetson format for photometric reduction

NEW STUFF

We will run Stetson's new photometry programs. They are in /uw50/nick/daophot/newccd. I have them aliased as:

alias ccdnew $daophot/ccdnew

The advantage of these new programs is that one does not have to constantly re-edit the *.lib, *.tfm, files etc for subsets of the standard filters. Instead we can use files good for VBIRU photometry (Peter likes that order) for just BV photometry, or even just B photometry. The program inputs and outputs single magnitudes rather than V,B-V etc. That means you can just observe, say, B on a given night, and measure a color term for B in B-V (or whatever color). In addition, the programs have a space to include the telescope azimuth so we can more routinely check that the airmass is the same on either side of the meridian.

Of course, while you can solve for the color terms if you only observed in a single color, you can't calibrate new stars if you only have a single color.

Here is my first cut at a data reduction cookbook.

GENERAL REDUCTION NOTES for 0.9m f/13.5 data, 2048 format, Tek2K_3

Data taken 13/14 Jun 2002. Reduced to [OTZF] in the usual fashion.

1. Load data from tape. I used a directory called:

20020613/opt/t36

Added this to loginuser.cl

#
set t20020613 = /uw52/nick/sn/sn02dj/20020613/opt/t36

and .daophot
#
setenv t20020613 /uw52/nick/sn/sn02dj/20020613/opt/t36
alias t20020613 "cd $t20020613"

2. Copy over setup files (including the *.opt,*.tfm files) from

copy /uw50/nick/daophot/optfiles/t36/* .
copy /uw50/nick/daophot/optfiles/t60/f75/* .
cl < setup

3. Create data directory to copy all unwanted data

mkdir data
cpimh *.fits del+

4. Run:

hedit *.imh OBSERVAT "CTIO" add+ up+ ver-
setjd *.imh
setairmass *.imh
azimuth *.imh calaz- flagit+ update+

The last commmand will add a KEYWORD called AZFLAG to be -1 W and +1 E for the hourangle. This will be used later for plotting the residuals. This is one of my IRAF commands in nickcl

We also make a file called mch.cl to stuff the MCHFILE information in the header and make inobj* files for the DAOMATCH stuff later. Here you make a file of the objects, blank separated, preferably with some filter other than U first.

task sed = $foreign
files obj*.imh | sed s/.imh// > in1
hsel @in1 $I,filters,title yes > in2
emacs in2 (blank separate the data, make sure first image is not U)

fields in2 1 > in3

To make the official version you must edit in the *.mch file. I will later make a program to do this automatically, but for right now, edit in a KEYWORD called MCHFILE with this information. If you have run prog59 above:

!$myprog/prog59 in3
cl < mch.cl

else:

hedit @inx MCHFILE obj069 add+ ver- update+ show+

5. Make the *.inf file

The new Stetson format has the *.mch file as the last field (which makes COLLECT easier to run) but it also means that you have to put the *.mch information into the *.inf file now. In addition, the *.inf file does not have the file title anymore, which is too bad.

We will make two versions of the *.inf file. The *.dat version is the old one which can be used for bookkeeping.

del junk.dat
hsel @in1 $I,filters,utmiddle,airmass,exptime,hjd,title,ha yes > junk.dat

!$myprog/prog3a junk.dat
0
t20020613
/uw50/nick/daophot/irafstuff/filters_t36new.dat

Now do:
del junk.dat
hsel @in1 $I,filters,utmiddle,airmass,azflag,exptime,hjd,mchfile yes >junk.dat

!$myprog/prog3b junk.dat
0
t20020613
/uw50/nick/daophot/irafstuff/filters_t36new.dat

or

del junk.dat
hsel @in1 $I,CCDFLTID,utmiddle,airmass,azflag,exptime,hjd,mchfile yes >junk.dat
!$myprog/prog3b junk.dat
0
!$myprog/prog3b junk.dat
0
t20020613
/uw50/nick/daophot/irafstuff/filters_t36new.dat
uyalo
/uw50/nick/daophot/irafstuff/filters_yalo.dat

You will now have a correct *.inf file.

/uw50/nick/daophot/irafstuff/filters_t36new.dat:
'dia v' 1
'dia b' 2
'dia i' 3
'dia r' 4
'dia u' 5
'dia z' 6

If you have to fix any headers, you must enter:

a. RA,DEC,epoch. You must enter the RA and DEC twice to get the right
notation.
b. hsel obj28*.imh $I,date-obs yes | translit - "-T" " " > junk1
c. filecalc junk1 "$2;$3;$4;$5-4" > junk2
d. astt files=junk2 obser=CTIO > junk3
and add the ST
e. Add an HA.

6. Subtract off the sky in I if needed.

I RECOMMEND COPYING ALL THE I SKIES FOR THE RUN INTO A DIRECTORY AND FORMING THE SKY FROM ALL THE IMAGES AT THE SAME TIME.

hsel @in1 $I,filters,exptime yes | sort col=3
emacs in4
ccdl @in4
irsky @in4 niter=10 sigma=2.5 irfltid="FILTER2" insuf="n" outsuf="s"
cl < sub.cl
starmask s????.imh up=75
addbpm @in4
imren s????.pl %s%n%*.pl
imdel s????.imh
imdel Skyi.imh
irsky @in4 usem+ rej- sig=2.5 niter=10 irfltid="FILTER2" insuf="n" outsuf="s"
subtract off DC of sky
fmed Skyi temp xwin=9 ywin=9
display Skyi 1 ; display temp 2
imdel Skyi ; imren temp Skyi
cl < sub.cl

imren @in4 data
imren *.pl data
imren s????.imh %s%n%*.imh
hedit @in4 bpm del+ ver-

DAOPHOT

Make sure you have the *.opt files. Make sure the daophot.opt file has the right gain and readnoise.

daophot.opt:

allstar.opt:

photo.opt:

1. Measure the FWHM as:

del junk.dat
yaloshift @in1

etc.

2. Then run

!$myprog/prog39 junk.dat

This outputs fwhm.dat and fwhm1.dat. Use fwhm1.dat.

3. If you have standards, run BFIND2, using thresh about 6 for the bright stars.

For SN data, run BYALO. This will do BPASS2 and FINAL2. Use threshold of 5. For most f/13.5 data you can use a var = 1

If you use BPASS2 alone, edit the psf using:

!$myprog/prog11a r042 0.1

Lower the factor of 0.1 to about 0.07 for most frames.

Use dals for editing the psf stars. Often the center of a galaxy gets included in the psf.

If the SN or an important star was missed, run addals to add the object by hand.

Run FINAL2 to make the final psf phot and aperture files.

A note: DAOPHOT and all the programs identify stars by the x,y positions except in the case of making the psf. The psf is made from the file *.lst, and the stars here are identfied by the star name, not xy. If you change or add stars to the lst file, you must be sure that these names are the same as in the *.als files.

If you need to do ALLFRAME because the object is very weak, do:

a. make a *.mag file using DAOMASTER. Use 1 0.5 2 for input
b. renumber the *.mag stars using DAOPHOT
c. run BALLFRAME
d. run the following program to copy over the *.alf to *.als files
!$myprog/prog45 r055
e. make sure the *.mch file is pointing to the *.als data
f. run DAOMASTER to update the *.tfr file
!/uw50/nick/daophot/perl/daomaster.pl r032.mch

4. Make the *.lis file as

ls -1 *ap > t20020613.lis.

The *.lis file should have the same number of lines as the *.inf
file. You can check this as

wc *.lis *.inf

A note on file names. The following files should have the same name:
*.inf, *.lis, *.obs, *.tfm, *.clb. It also helps to call the directory by that name also. For instance, if there are 5 nights, the third night would be in directory n3, and the following files would be created in directory n3: n3.inf, n3.lis, n3.obs, n3.tfm and n3.clb.

At this point you may want to examine all the master images to see if there are any galaxies/double stars that will affect the growth curves.

Use in3 as a guide and run:

dals obj3333 suf=ap

5. Then run NDAOGROW. I used 3 unknowns. Last 2 are 0.9 and 0. I used 0.025mag error limits.

This produces *.tot files and a summary file *.gro. You can run "sm" at this point to see the growth curves. The command "see n3" will plot up 5 curves that represent the full range of seeing. The command "gro obj100" etc will plot the growth curves.

It is important to look at the curves quickly to see if they have appeared to converge.

In the new version of DAOGROW, the *.tot files have the sky in them.

If you need to rerun DAOGROW, run deldaogrow first.

To see the data:

rm -f junk ; sed 's/obj/gro obj/' in1 > junk
sm
see o20021109

It is important to check the data to see if DAOGROW has barfed on a frame with lots of saturated stars or galaxies.

6. Normally, one runs NDAOMATCH and NDAOMASTER to make the tables for each field.

This produces *.mch files for each field.

Peter's philosophy here is to have a directory with template images and *.tot files for the standards. You run NDAOMATCH starting with that file, and then feed in the program frames from the night in question. The *.mch file then has as a master image the template image. This works well provided that DAOMATCH runs flawlessly with all the program data.

I don't know if NDAOMATCH works better now. What I have done is to use yalocenter to make a junk file with shifts and run the following program. Put "als" or "tot" as needed.

yalocen inobjxxx

!$myprog/prog52b junk.dat als

This asks if you want to run daomaster. Do it.

!/uw50/nick/daophot/perl/daomaster.pl r032.mch

!ls -1 obj*.mch | wc
!ls -1 inobj* | wc

7. Display each first image in the *.mch files. Run the iraf task "fetch" and then the fortran task "fetch" to make the *.fet files.

The IRAF fetch inputs either an "a" key or an "x" key. Use the "a" key if the object looks like it can be centered. If the object is near a bad pix, use the "x" key.

NOTE THAT THERE IS A NEW VERSION OF FETCH. I NEED TO MODIFY IT TO WORK IN THE IRAF TASK.

You can get the new Stetson fields at:

http://cadcwww.hia.nrc.ca/cadcbin/wdbi.cgi/astrocat/stetson/query

Get the *.fits.gz, *.pos, *.pho files.

Run prog2 in /uw52/nick/stetson to make a *.tot file with the brighter stars.

prog2 NGC2437
displ NGC2437.fits
tvm 1 NGC2437.xy label+ mark=point points=5 color=202 tx=2 ny=-6 nx=10

To add stars into the *.lib file, use

~/daophot/library/prog12
PG2213.pho

To display the new stars and to cull only the bright ones:

prog2 NGC2437
displ NGC2437.fits
tvm 1 NGC2437.xy label+ mark=point points=5 color=202 tx=2 ny=-6 nx=10

If the STetson field is too big, run:

/uw52/nick/stetson/prog4

to clip the field.

If the standard fields cover a larger area than a single image, use MONTAGE2 on the *.mch file. This makes an image obj120m. The offsets (needed for FETCH) are sent to offset.dat

~/daophot/perl/montage2.pl obj120

For some of the Stetson fields, there are way too many stars to id. I would prefer some sort of id based on WCS, but here is a quick solution.

a. Copy over the *.tot file, such as Ru149.tot

!cp /uw52/nick/stetson/L95_100.tot .
!cp /uw52/nick/stetson/Ru149.tot .
!cp /uw52/nick/stetson/PG1323s.tot .
!cp /uw52/nick/stetson/PG0918.tot .
!cp /uw52/nick/stetson/PG1047.tot .
!cp /uw52/nick/stetson/L104.tot .

b. Run NDAOMATCH on Ru149.tot and obj1114.tot where the latter is the master image in the *.mch file.

x(1) = 2316.8152 + -0.0015 x(2) + -0.8646 y(2)
y(1) = 1110.2642 + 0.8646 x(2) + -0.0015 y(2)

c. Run

!/uw50/nick/daophot/perl/daomaster.pl L95_100
!/uw50/nick/daophot/perl/daomaster.pl Ru149
!/uw50/nick/daophot/perl/daomaster.pl PG1323s
!/uw50/nick/daophot/perl/daomaster.pl PG0918
!/uw50/nick/daophot/perl/daomaster.pl PG1047

to output a *.tfr file.

d. Run

!/uw52/nick/stetson/prog3 L95_100
!/uw52/nick/stetson/prog3 Ru149
!/uw52/nick/stetson/prog3 PG1323s
!/uw52/nick/stetson/prog3 PG0918
!/uw52/nick/stetson/prog3 PG1047

This will output the correct *.fet file for the master image. Believe me.

8. Now, if you are doing standards, enter the data into NCOLLECT.

This runs much more easily than in the past.

nick% ncollect
Name of output photometry file: t20020613

Creating new file.

9. Now you run NCCDSTD to get the transformations.

This new program inputs a *.lib and *.obs file which have the same magnitude order.

head landolt.lib

head t20020613.obs

This produces *.rsd files which you can plot with sm. Use "resids highz99r" and "resids highz99i" which inputs the data. There is a macro called rsd.macro that you copied over. Run SM and input:

sm
: macro read rsd.macro
etc

This macro plots up the data. Look especially carefully at the UT and X plots to look for trends. Add a "T" term to the solution if needed.

To see all the resids, do:

grep obj o20021204.rsd > junk.rsd

and plot

A quick way to find bad stars is to do:

grep o20030131.rsd -e \? -e # | sort +19

10. Run NCCDAVE to output the *.net and *.ave file.

Note that this program will output the specified number of filters in the *.obs file. Again, you can input the VBIRU *.lib file, and the reduced number of filters *.obs file to get out the *.net file.

11. Run NTRIAL to get the final reduced psf photometry.

You have to have a fresh copy of the *.tfr file by running daomaster.pl. The pairs file can be used to search for variables. NTRIAL uses the *.mch file to determine what data to input.

nick% ntrial

See below for variable star searches

SMALLER NUMBER OF FILTERS THAN VBIRU

If you have a smaller number of filters than VBIRU, do the following.

1. Make sure the *.obs file has only the filters you want to reduce.

For instance, if you want VB but you also have I in the *.obs file, remove the I data. The NCCDSTD program will attempt to reduce all the filters in the *.obs file.

2. Edit the *.tfm file to include only the filters of interest.

For instance, I used the following *.tfm file for VI reductions, but
inputting the VBIRU *.lib file. Note I had to change the color of O1 (V) from I2 to I3 because I have removed all the B information from the *.obs file.

I1 = M1
#I2 = M2-M1
I3 = M1-M3
#I4 = M1-M4
#I5 = M5-M2
O1 = M1 + A0 + A1*I3 + A2*X
#O2 = M2 + B0 + B1*I2 + B2*X
O3 = M3 + C0 + C1*I3 + C2*X
#O4 = M4 + D0 + D1*I4 + D2*X
#O5 = M5 + E0 + E1*I5 + E2*X

By doing this, NCCDSTD will run on only the VI data. NCCDAVE will only output the VI data. Very simple to use!

For NTRIAL, you can use the full *.inf, *.net, and *.clb file (VBIRU) even for the subset of filters. It is the *.mch file that limits the input data. NTRIAL is quite intelligent. For instance, I input the VI data in *.mch file. The *.clb file had the I color term as V-I and the V color term as B-V. The output *.fnl file had the I data correct, but no V data because it lacked the B filter for the color term. But the program still ran.

VARIABLE STAR SEARCH

If you want, you can use trial to search for variable stars.

You can input a file called *.prs to search for variable stars. You must have a file with pairs of observations where you don't think that the star varied between observations (like CR SPLITS in HST). The program will output lots of cool statistics files and also attempt a period analysis if it finds a variable.

To make a variable star search, set up a .prs file for SN1987A:

grep -h SN1987A *.inf | sort +7n > SN1987A.prs

This sorts on JD and does not put the annoying "sort" header crap in the output.

Then just go through the file putting blank lines between the pairs you want.

For this to work most efficiently, it is best to have all your various .inf files stored in the same location. Me, I have a directory named `save' where I keep all the master .mch, .mag, .inf files for fields that span many observing runs, and I have a directory for each observing run.

TO USE: Run the program once with

Limits on Variability, weight, period, magnitude: 9999, 9999, 9999, 0

It doesn't actually find any variables with these parameters, but it will put values of all the relevant indices into the output file. Use
supermongo or IDL or whatever to plot variability index vs magnitude, variability index vs weight, weight vs magnitude, and decide what
limits you believe in. Then run the program again specifying the

MINIMUM VARIABILITY INDEX, MINIMUM WEIGHT, 9999, MAGNITUDE LIMIT.

For each star with VARIABILITY INDEX > MINIMUM, WEIGHT > MINIMUM, and MAGNITUDE < MAXIMUM it will produce a file giving magnitude, sigma, filter ID and HJD for every observation of that star, but since you have set PERIOD = 9999 (or whatever) it will not actually try to derive a period and light curve (since that part of the software isn't trustworthy yet). Instead, you can feed those output files to whatever algorithm you favor.

A typical output is:

example of *.fnl output. I have clipped out the BIRU, chi, and sharp cols

WSI is the Welch/Stetson variability index. See Welch & Stetson 1993, AJ, 105, 1813. A large positive number is a variable. Non-variable
stars should scatter around 0.

WEI is described above. In this case, I had 7 frame pairs.

VAR = 1.2533*KUR*WSI. I am not clear as to the meaning of VAR.
D

back to top