Getting Started with data reduction in AIPS
Tutorial testers - please see for link to data and link to MERLIN procedure
Contents
Anita Richards, Cormac Reynolds
Your AIPS Nos (in decimal and ehex) are on a slip of paper in your introduction pack; see an organiser if you need more.
What is ''AIPS''?
The ''Astronomical Image Processing System'' is maintained at NRAO and the AIPS Cookbook provides a comprehensive guide but the examples are mostly based on VLA and VLBA data.
AIPS at JBO describes local use for beginners step by step and you could refer to this if you get stuck. Megan Argo has produced a short AIPS helpsheet.
For a step-by-step guide to EVN data analysis in AIPS, see the EVN Data Analysis Guide in the EVN user guide http://www.evlbi.org/user_guide
Using the MERLIN Procedure
An AIPS procedure is a series of tasks and commands. The MERLIN procedure provides an interface to the tasks needed for simple data reduction, such as loading uv data in FITS format, editing and calibrating the data and producing images and plots. You can specify a minimum of input parameter values and the default values of the rest should be suitable for automatic processing of most MERLIN continuum data. As you get more experience, you can set more inputs and edit and image the data interactively. You will also run the component tasks separately.
In this session the MERLIN Procedure is used to test your set-up and provide a quick tour of basic operations.
When you log on to a workstation and open an xterm, you are in /home/user. Make a suitable subdirectory, copy the AIPS runfile for the MERLIN procedure1 and link to the data file as follows, e.g. - if your aips No is 10 in decimal and 00A in ehex -
mkdir BASIC cd BASIC cp /home/user/MERLIN.ehex MERLIN.00A cp /home/user/MERLIN.HLP . ln -s /home/user/datastore/DQSO.FITS .
Set the required environments:
export RUNFIL=/home/user/BASIC export WORK=/home/user/BASIC
NB this is for the BASH shells used here; if you are working elsewhere with a c-shell, see environments.
Start AIPS
and enter your user number (in decimal).
AIPS 1: run MERLIN
Some messages terminating
AIPS 1: return; finish AIPS 1: tget addi AIPS 1: vers inn AIPS 1: inn ' ';incl ' ';inty ' ';inse 0
will appear (thi sis just for information so you know it has finished setting up.)
You can get a little, some, a lot of information about any AIPS task or documented procedure e.g. MERLIN by typing e.g.
AIPS 1:inp MERLIN AIPS 1:help MERLIN AIPS 1:docrt 1; explain MERLIN
(docrt 1 means display information on screen)
The Double Quasar
The double quasar (DQSO), the first gravitational lens discovered, was observed by MERLIN in May 1993 using 15x1 MHz centred on 1658 MHz. A nearby source J095738+552257 (DQSOREF) was observed for 2 out of every 10 min as the phase reference source. 3C286 was observed as the primary flux and polarization angle calibration source and OQ208 was observed as a point source and bandpass calibrator. Prior to writing the FITS file, OQ208 was used to apply an approximate flux scale and bandpass calibration. All 15 channels looked good and the DQSO is not extended enough to need wide field imaging so we squashed the data to a single 15 MHz channel. We have also done quite a lot of editing to save you time2
Testing the MERLIN procedure
These settings of the MERLIN procedure will load the data and index it and print out a scan listing. The procedure reads the FITS header and performs various bits of housekeeping including time-dependent calibration of 3C286 and OQ208 and using this to improve the flux scale. Next, the procedure derives time-dependent phase and amplitude corrections for DQSOREF. We assume that the position is accurate but Reid et al. (A&AS 1995, 110, 213) show that this source is resolved 3 so we ask the procedure to build up a model. This source is used for polarization lekage corrections and 3C286 is used to calibrate the polarization angle. The corrections are applied to the DQSO and this is imaged. The known size of the DQSO (Reid et al. A&AS 1995, 110, 213) means that we want a 1024x1024 pixel size for the image.
DQSO.FITS contains all 4 sources. Enter the appropriate inputs into the MERLIN procedure. If you have more than one disc also set INDISK.
These settings show you comments and plots in real time but you don't have to do anything other than press return.
AIPS 1:DIR 'WORK' AIPS 1:DOLOAD 2 AIPS 1:TARGET 'DQSO' AIPS 1:PHCAL 'DQSOREF' AIPS 1:ACAL 'OQ208' AIPS 1:DOPLOT 2 AIPS 1:DOPHSC 2 AIPS 1:DOPLZN 1 AIPS 1:DOTGSC 0 AIPS 1:DOFLUX 1 AIPS 1:IMSIZE 1024,1024 AIPS 1:DOTV 0
inp MERLIN
to check inputs
tput MERLIN
to save inputs
go MERLIN
to run the procedure.
NB It will ask if you want to enter the ACAL flux, but you don't need to - OQ208 is such a common calibration source the flux densities are in a look-up table. You can just press return.
Inspecting the results and printout
When it has finished, pcat shows you all the files which have been written. To inspect a file, e.g. the first one: getn 1; IMH shows you the image header. If you type docrt 1; explain MERLIN this tells you what the various extension tables are. Some of them formed the basis for the plots produced during the procedure.
MERLINProcedurePlots explains the printouts from this and later stages. We suggest that you don't need to regenerate copies froom your similar use of the procedure, but in general to get postscript print-out, make sure that you have the print environment set before starting AIPS and, inside the MERLIN procedure in AIPS, set DOPLOT 3.
To inspect other extension tables you can either use the generic task PRTAB4 or specific ones e.g. PRTAN to look at the antenna table. Set DOCRT 1 to inspect on-screen. Image files (extensions like ICL001, POLI and POLA for total intensity, polarized intensity and polarization angle respectively) can be viewed in the TV. Investigate some tasks (look at the INPuts and HELP (task) and the Message Server for more information) e.g.
AIPS 1:pcat AIPS 1:getn 1 AIPS 1:docrt 1 AIPS 1:task 'PRTAN' AIPS 1:inver 0 AIPS 1:go PRTAN AIPS 1:task 'PRTAB' AIPS 1:INP PRTAB AIPS 1:XINC 1;YINC 1 AIPS 1:inext 'SU' AIPS 1:go PRTAB AIPS 1:getn (an image file) AIPS 1:TVINI AIPS 1:TVLO AIPS 1:TVFI AIPS 1:TVWIN
Use TVWIN to select an off-source part of an image and use IMSTAT to measure the rms noise. You can then use the task KNTR to make a contour plot. This illustrates the way you can enter expressions for AIPS to evaluate (HELP POPSY lists the allowed symbols) - PIXSTD is the noise from IMSTAT.
AIPS 1:task 'KNTR' AIPS 1:getn (image file) AIPS 1:DOCONT 1 AIPS 1:DOGRE 1 AIPS 1:DOVECT -1 AIPS 1:CLEV 3*PIXSTD AIPS 1:LEV(1) = -1 AIPS 1:for i=2 to 15;LEV(i)=2**(i-2);end AIPS 1:DOTV 1
Use TVWIN again to select the image area and GO KNTR to send a plot to the TV.
Interactive use of the procedure
To get some practice in interactive use of the TV, you can re-run the procedure as follows. First, delete all except the first file. This example shows how to delete one file individually and delete the next 13 in a loop:
AIPS 1:getn 2; ZAP AIPS 1:for i=3:15;getn i;zap;end
If you didn't want to save e.g. files 1 and 13, you can close up the resulting gap(s) in the catalogue using RECAT. When you have just got the first file DQSO.MULTTB left, the following will make a copy of the original extension tables (attatched to DQSO.TASAV) and then delete the flag table from the main data set. You can delete any other extension file similarly. Alternatively, you can leave it as it is, but just check for any remaining bad points.
AIPS 1:task 'TASAV' AIPS 1:getn 1;outn inn AIPS 1:inp TASAV AIPS 1:go TASAV AIPS 1:inext 'FG';inver 1 AIPS 1:EXTDEST
Re-set the procedure:
AIPS 1:tget MERLIN AIPS 1:DOLOAD 0 AIPS 1:DOEDIT 1 AIPS 1:DOTGSC 1 AIPS 1:DOTV 2
This will allow you to edit each source in turn using IBLED. Remember to start for each source by setting STOKES FLAG to 1111. Be careful - or avoid - flagging by TIMERANGE, as sources which are not shown can be flagged and the time axis is non-linear (tick-marks close together conceal jumps in time when undisplayed sources were observed). It will also invite you to set boxes manually and decide when to stop cleaning in IMAGR. Note that this 'times out' and goes on automatically after a few minutes if you don't do anything.
EVN introduction
EVN continuum data at ERIS
The EVN continuum data in FITS-IDI format, copied from the JIVE archive, are accessible in /home/user/datastore/EVN_N04C3_CONT and the pipeline products are in a subdirectory /home/user/datastore/EVN_N04C3_CONT/PIPE
An EVN Tutorial
This tutorial will lead you through the steps of loading your EVN data and applying the results of the EVN pipeline to accomplish the first stages of the calibration procedure. For more details, see the EVN Data Analysis guide on the User Guide section of the EVN web pages (http://www.evlbi.org/user_guide).
Start in the directory where you will run the tutorial and, if you haven't already, set the TUTFITS and RUNFIL environments using commands like those described for the MERLIN exercise above, for the locations of the data and the runfile. Then download the tutorial EVNTU2.RUN and save it to $RUNFIL with the name EVNTU.ehex where ehex is your AIPS user id in ehex (as previously for the MERLIN run file). Note that this tutorial requires the data to be in $TUTFITS and the pipeline output to be in $TUTFITS/PIPE. e.g.
mkdir TUTFITS cd TUTFITS ln -s /home/user/datastore/EVN_N04C3_CONT/* . export TUTFITS=/home/user/TUTFITS
etc.
To start the tutorial, start AIPS, then
>RUN EVNTU >EVN
Loading your data
Use task fitld. Your data may be stored in multiple FITS files which are concatenated using the DOCONCAT parameter in fitld.
To load multiple FITS files from a disk, make sure they have the same base name followed by a sequence number (e.g. n04c3_1_1.IDI1, n04c3_1_1.IDI2, etc.), and use parameter ncount to specify how many you files you wish to load.
Set CLINT to the time interval between entries in the first CL table. This should be less than the coherence time.
>TASK 'FITLD' >INFIL 'TUTFITS:N04C3_1_1.IDI' >OUTDISK 1 >DOUVCOMP -1 >NCOUNT 6 >DOCONCAT 1 >CLINT 1/6 >WTTHRESH 0.3
Loading the user pipeline tables
Run fitld with INFIL set to the tasav FITS file.
>TASK 'FITLD' >INFIL 'TUTFITS:PIPE/n04c3.tasav.FITS >NCOUNT=1 >GO
This will write a second FITS file, containing no data, but which has the aips calibration extension files from the pipeline attached. These can later be copied to the raw data set using tacop.
Sorting your data
Run the task MSORT with SORT='TB' to sort the data in time-baseline (TB) order.
>TASK 'MSORT' >SORT 'TB' >GO
Indexing your data
Run INDXR on the sorted dataset to create an NX extension table.
>TASK 'INDXR' >INFIL '' >CPARM(3)=-1 >GO
Data inspection
Check the results of the EVN pipeline (available from http://www.jive.nl/archive, or for this tutorial http://www.merlin.ac.uk/ERIS/EVN_N04C3_CONT/PIPE).
Have a look at the FITS header by typing imhead. This will report the number of IFs (sub-bands), the number of frequency channels per IF, the observed frequencies and the number of Stokes parameters. There are a number of other tasks for providing useful information:
imhead: will give information on the observation - date, frequencies, etc.
listr: Run with OPTYPE='SCAN' this task will produce
- a listing of every scan in the dataset (also available from pipeline)
prtan: You will often need to know the number that aips assigns
- to each antenna (see pipeline).
vplot: Looking at amplitudes and phases with time
(BPARM=0,-1) and weights with time (BPARM=0,16) is recommended.
uvplt: The main use of this task is to plot amplitude
and phase against (u,v) distance although it is also useful for making plots of the (u,v) coverage (BPARM=6,7,2,0).
possm: Running this on short segments (approx 1 min) of
data with APARM(9)=1 will plot the amplitudes and phases as a function of frequency for each IF. It is also a good idea to look at the auto-correlations in possm (with APARM(8)=1) as these can reveal channels that are affected by RFI.
First check the image header.
>imhead;
Get a scan listing using listr.
>task 'listr' >optyp 'scan' >go
List of stations, number of visibilities and correlator integration time.
>task 'dtsum' >aparm 1,0 >go
View the autocorrelation spectrum using POSSM.
>task 'possm' >stokes 'half' >codetyp 'A&P' >aparm(2) 1 >aparm(7) 1 >aparm(8) 1 >aparm(9) 1 >solint -1 (scan average) >go
View the amplitude and phase as a function of time.
>task 'vplot' >stokes 'll' (or 'rr') >bparm(2) = -1 >go
And the cross-correlation spectra.
>task 'possm' >stokes = 'HALF' >aparm=0 >aparm(9)=1 >codetype 'A&P' >solint=-1 >bparm 0
Data flagging
Run tacop with the tasav file as the input and the sorted multi-source data file as the output. The output file is selected using GETONAME.
>TASK 'TACOP' >INEXT 'FG' >INVER 1 >OUTVER 1 >NCOUNT 1 >getn n (the tasav file from the pipeline) >geton m (the sorted dataset) >GO
Additional flagging can be done in a number of ways. uvflg is a non-interactive task and good for flagging whole telescopes, sources, etc. Interactive tasks include ibled, edita and editr.
Amplitude calibration
Copy the CL2 table from the pipeline tasav file to your multi-source file using tacop as with the FG table previously. The amplitude calibration can be checked in possm, vplot and uvplt. Note that the unit of the 'GAIN' in AIPS is actually the square root of the SEFD (System Equivalent Flux Density) of the telescope. You can compare these values with the nominal SEFDs of the EVN antennas available from the EVN status table (linked from http://www.evlbi.org/user_guide)
CL2 also incorporates the parallactic angle correction.
To also calibrate the weights according to the antenna sensitivity, whenever data is calibrated in aips the DOCALIB parameter should be set to 2.
Check the CL table with SNPLT.
>task 'snplt' >inext 'cl' >inver 2 >stokes '' >optyp 'ampl' >go
Phase calibration
The main part of phase calibration is "fringe-fitting" where the phase derivatives with respect to frequency and time are solved for and removed from the data.
Delay calibration
The slope of phase against frequency is removed using the task fring. This can solve for a single value of delay for each antenna, polarization and IF using a small amount of data on a bright, preferably unresolved source - 3 minutes in this case. Calibrate the data prior to running the task with the appropriate CL table (DOCALIB 2). As we are only calculating a delay correction we set DPARM(8)=1. DPARM(4) is the correlator integration time.
>TASK 'FRING' >CALSOUR '3C273B' >timerang 0,7,1,0, 0,7,4,0 >DOCALIB 2 >REFANT 1 >SOLINT 0 >APARM(1) 3 >APARM(6) 1 >DPARM(1) 1 >DPARM(4) 2 >DPARM(8) 1 >SNVER 0 >GO
Write the solutions to a new CL table using clcal, for all sources, not just the calibration source, with INTERPOL='2PT'.
Check the resulting CL table with SNPLT, similar to before, but this time looking at delay instead of amplitude.
>task 'snplt' >inext 'cl' >inver 3 >optype 'dela' >go
Fringe fitting
Full delay, rate and phase calibration is now done by running fring again. Apply the previous calibration and set DPARM(8)=0.
>TASK 'FRING' >CALSOUR '' >TIMERANG 0 >DOCALIB 2 >SOLINT 2 >DPARM(8) 0 >GO
Check the resulting SN table carefully in snplt. Spurious solutions may be edited using snedt or snsmo. Apply solutions with clcal, setting INTERPOL='AMBG' to use the rates to solve the 2*pi phase ambiguities.
You can also view the resulting CL table with SNPLT similar to before, checking phase, delay and rate.
Final calibration
Bandpass calibration
The bandpass is calibrated using the task bpass.
>TASK 'BPASS' >CALSOUR 'OQ208' >SOLINT -1 >DOCALIB 2 >BPVER 1 >BPASSPRM(1) 0 >BPASSPRM(2) 1 >BPASSPRM(10) 1 >GO
The bandpass solutions and their effect on the data can be viewed using possm.
>task 'possm' >stokes = 'HALF' >aparm=0 >aparm(8)=2 >aparm(9)=1 >codetype 'A&P' >solint 0 >bparm 0 >go
Checking the calibration
Check the data again, using possm and vplot as before, but this time with the calibration applied by specifying docal = 1
Applying the calibration
At this point the pre-calibration can be considered complete and the source can be separated from the multi-source file and the calibration applied at the same time using the task split or splat.
The data may now be averaged in both frequency and time using splat or a combination of split and uvavg. For limitations on how much time and frequency averaging is appropriate refer to Chapter 18 of "Synthesis Imaging in Radio Astronomy II"5 and the imaging limitations section of the EVN user guide (http://www.evlbi.org/user_guide/). The inputs here average the data into 20 second intervals and average all the channels in each IF together.
>TASK 'SPLAT' >SOURCE '' >DOCALIB 2 >DOBAND 1 >BPVER 1 >APARM(1) 3 >APARM(2) 2 >APARM(7) 1 >CHANNEL 16 >SOLINT 20/60 >GO
Finally, before proceeding with mapping and self-calibration of your data, it is probably necessary to perform some final editing. This can be done using tasks such as ibled and clip.
1 in general you can also get the most recent release from http://www.merlin.ac.uk/user_guide/AIPS_scripts.html
2 The procedure automatically flags the Mk-Lo baseline, if present, as it is usually confused.
3 It is always worth checking the relevant observatory archive or ADS or SIMBAD for the properties of calibration sources
4 Here, this is used to show that the flux density of the source 3C286 has been found from a standard look-up table and used to set the flux of OQ208.
5 Taylor, Carilli and Perley, eds., 1999, "Synthesis Imaging in Radio Astronomy II"