This guide is intended as the definitive reference for the ParselTongue scripting environment. It builds upon a number of previous sources, including the ParselTongue Tutorial and the ParselTongue Cookbook.

ParselTongue should run on just about any modern Linux or UNIX installation. There are only three direct dependencies for a successful ParselTongue install:

Obit has its own extensive dependency list, and is generally the biggest obstacle to installing ParselTongue. In addition to a C compiler, like gcc, the two other principal dependencies are on GLib 2.x, and the GSL (GNU Scientific Library). If you'd like to read FITS files directly from ParselTongue, you'll also need CFITSIO. Finally, visibility data access from ParselTongue requires either numarray, from STScI, or the newer NumPy. NumPy is to be preferred for new projects, as numarray is now deprecated, with all development halted, including bug fixes. Optionally, FFTW, PGPLOT, XMLRPC and SWIG may also be added, although these are not necessary for building ParselTongue. In particular, we encourage folks to be careful if installing SWIG, as it is only needed if you intend to change or extend Obit itself. Obit is rather picky about the version of SWIG it requires, however, and will fail to compile with some recent versions.

Now, some more particular advice for setting up ParselTongue under the two principal types of Linux distributions, along with Apple OS X.

$ rpm -q -a 

$ dpkg -l 

$ export PYTHON=/sw/bin/python 

% setenv PYTHON /sw/bin/python 

After the package gymnastics above, building Obit should be relatively anti-climactic. Here you can find a tarball for a modified version of Obit (20070702) that is known to work with ParselTongue. After downloading, simply extract the tarball, configure, and make:

$ tar zxvf Obit-20070702.tar.gz
$ cd Obit
$ ./configure
$ make 

If all dependencies have been satisfied, this should build with no trouble.

$ rm -rf ~/.ParselTongue
				

You can find the latest version of ParselTongue here. The only bit of information you will require during the install is the location of the Obit build you created in the previous step, which is passed to configure:

$ tar zxvf parseltongue-1.1.1.tar.gz
$ cd parseltongue-1.1.1
$ ./configure --with-obit=/path/to/Obit
$ make
$ sudo make install 

The last step must be performed either via sudo, or logged in directly as root. If you don't have superuser privileges on your machine, you can specify an installation directory different from the default (/usr/local) with the "--prefix" option to configure:

$ ./configure --prefix=$prefix --with-obit=/path/to/Obit 

If you installed ParselTongue in the default location, you should be all set at this point (/usr/local/bin should already be in your path). If you changed the default install location, you should remember to add this directory to your path as well:

$ export PATH=$PATH:$prefix/bin 

or:

% setenv PATH $PATH:$prefix/bin 

Whilst primarily intended for scripting, ParselTongue is perfectly suitable for interactive use. After invoking the ParselTongue, you will be prompted for your AIPS user number, and you will get back a standard Python prompt:

$ ParselTongue
Python 2.5.1c1 (release25-maint, Apr 12 2007, 21:00:25) 
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

Welcome to ParselTongue 1.0.5
Please enter your AIPS user ID number: 667
>>> 
			

The first time you do this, ParselTongue will take a few moments to respond, as it goes over your AIPS installation to see which tasks and adverbs are available.

User 667 is, of course, the Neighbour of the Beast.

ParselTongue can be used to run your scripts directly:

$ ParselTongue script.py 

This sets up the required AIPS environment, but to use ParselTongue from your scripts, you'll still need to import the appropriate module set. Here's a set that should get you started:

from AIPS import AIPS
from AIPSTask import AIPSTask, AIPSList
from AIPSData import AIPSUVData, AIPSImage
			

You can add more modules as you need them, and can of course import any other Python module you might require. One difference between running ParselTongue interactively and running a ParselTongue script is that your AIPS user ID will not be set unless your script explicitly does so with the following:

AIPS.userno=667

ParselTongue exposes a large subset of the AIPS interface to your scripts, including every AIPS Task. Whilst AIPS Verbs are not directly supported, their functionality can usually be mimicked with snippets of Python code. Under ParselTongue, an AIPS Task is represented by an object, and the attributes of that object correspond to the "adverbs" AIPS uses for parameter passing. For example, to load a FITS file into the AIPS catalogue, we can make use of the FITLD task as follows:

$ ParselTongue
Python 2.5.1c1 (release25-maint, Apr 12 2007, 21:00:25) 
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

Welcome to ParselTongue 1.0.5
Please enter your AIPS user ID number: 667
>>> fitld = AIPSTask('FITLD') # make a new AIPSTask object
>>> fitld.infile = "/home/sipior/data/N1066.fits"
>>> fitld.outname = "N1066"   # set output name in the AIPS catalogue
>>> fitld.go()                # despatch the job
			

Note that the choice of "fitld" as the object name is entirely arbitrary, and is intended as a mnemonic. More importantly (especially to you old AIPS hands), note that each instance of an AIPSTask has its own private set of adverbs! This allows for much greater flexibility than POPS, but be aware that these must be explicitly set. N.B. All adverb names are in the lower-case!

Data sets that have been loaded into the AIPS catalogue can be accessed as AIPSData objects, which we will discuss shortly in the section AIPSData, below. Once an AIPSData object is associated with a data set, that object can be used directly as an adverb in other AIPS Tasks.

As mentioned above, all of the Tasks found in AIPS are available to ParselTongue. To make use of them, you must first create an AIPSTask object with the name of the AIPS task as the sole argument. For example, you can create an instance of the MANDL task with the following:

>>> mandl = AIPSTask('MANDL')
			

The variable mandl is the handle by which you manipulate this instance of the task. So, to set the AIPS adverbs which govern this task, you only need to set the appropriate attributes on mandl:

>>> mandl.imsize = AIPSList([256]) # set the image size in cells
>>> mandl.outname = "MBROT"        # set the name of the output image
>>> mandl.outseq = 3               # set output image sequence number
			

To start a task running, simply invoke the go method for the task object:

>>> mandl.go()
MANDL1: Task MANDL  (release of 31DEC07) begins
MANDL1: Create MBROT       .MANDL .   3 (MA)  on disk  1  cno    1
MANDL1: Appears to have ended successfully
MANDL1: jop56        31DEC07 TST: Cpu=       0.0  Real=       1
>>>
			

To get the full AIPS help file for a task, simply supply an AIPSTask object as an argument to the help function:

>>> fitld = AIPSTask('FITLD')
>>> help(fitld)
FITLD
Type: Task
Use:  FITLD loads both maps and UV data from tape (or disc) to disc. It
      will only load FITS files, if any other type of file exist on the
      tape the task will fail, so users must skip over non-FITS files.
      The aim of this task is to read, in one pass, a tape written by
                         .  .  .  .  .
			

It is also important to reemphasise that attributes are private to the object to which they belong, so a different instance of the same AIPS task can have completely different attributes:

>>> fitld            = AIPSTask('FITLD')
>>> fitldtwo         = AIPSTask('FITLD')
>>> fitld.infile     = "one.fits"
>>> fitld.outname    = "FIRSTFITS"
>>> fitldtwo.infile  = "two.fits"
>>> fitldtwo.outname = "SECONDFITS"
>>> fitld.go()
FITLD1: Task FITLD  (release of 31DEC07) begins
FITLD1:     FITLD Version 10.5.1
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .
>>> fitldtwo.go()
FITLD1: Task FITLD  (release of 31DEC07) begins
FITLD1:     FITLD Version 10.5.1
.  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .
>>>
			

>>> mandl.imsize = [None,256]
>>> mandl.imsize = AIPSList([256]) # identical to the first line
>>> mandl.imsize[1:] = [256] # still more of the same
				

AIPS knows about two rather different types of data, those being images and UV data sets. Once you know the name, class, disk number and sequence number of your data set from the AIPS catalogue, you can create an AIPSData object from this information, and use the object as a handle to refer to your data set. For UV data, you'd use the following:

>>> uvdata = AIPSUVData('TARGET','UVDATA',1,2)
			

You can probably guess at the analogue for image data:

>>> image = AIPSImage('MANDELBROT','MANDL',3,2)
			

In addition to using these objects to initialise AIPSTask attributes as seen above, you can access and manipulate your data set directly using AIPSData methods. To verify that your data object corresponds to an actual data set in the AIPS catalogue, use the exists method:

>>> image.exists()
False
>>> uvdata.exists()
True
			

Manipulation is pretty much limited to deleting extension tables from a data set and deleting the data itself. The latter is a matter of calling the method zap() while the former has a somewhat more complicated interface:

>>> uvdata = AIPSUVData('N04C2', 'UVDATA', 1, 1)
>>> uvdata.zap_table('CL', 0)
			

deletes the last calibration tables from the UV data set N04C2.UVDATA.1 on disk 1. Note that plot files ('PL') and slice files ('SL') can also be deleted. So the following lines

>>> imgdata = AIPSImage('MANDELBROT', 'MANDL', 1, 1)
>>> imgdata.zap_table('PL', -1)
			

can be used to throw away all previously made plots.

Deleting a complete data set or image is much simpler. After

>>> uvdata.zap()
>>> imgdata.zap()
			

the data and image are forever lost.

Sometimes an AIPS task will fall over and leave your data in a "busy" state. To recover from this you can use

>>> uvdata.clrstat()
>>> imgdata.clrstat()
			

but do not do this on data that is being actively used by AIPS or ParselTongue. That would be bad.

>>> uvdata.header
...
>>> uvdata.header.telescop
'EVN     '
>>> image.header
...
>>> image.header.date_obs
'2005-03-01'
				

>>> uvdata.tables
[[1, 'AIPS HI'], [1, 'AIPS AT'], [1, 'AIPS NX'], [1, 'AIPS CL'], [1, 'AIPS FQ'], [1, 'AIPS AN'], [1, 'AIPS SU']]
				

>>> sutable = uvdata.table('SU', 1)
>>> for row in sutable:
...     print row.source, row.raapp, row.decapp
...
3C84             49.9489485008 41.513581044
DA193            88.8813073705 39.8151637728
				

>>> cltable = uvdata.table('CL', 1)
>>> print cltable[0]
...
				

>>> uvdata.antennas
['MC', 'WB', 'NT', 'JB']
>>> uvdata.sources
['3C84', 'DA193']
>>> uvdata.polarizations
['R', 'L']
>>> uvdata.stokes
['RR', 'LL', 'RL', 'LR']
>>> image.stokes
['I']
				

ParselTongue includes a seperate module for manipulating data beyond simple inspection of metadata. It allows full access to AIPS tables, and access to visibilities and image pixels.

The Wizardry functionality has certain limitations. The most important limitation is that access to remote data is not possible; only data on the locally visible AIPS disks can be accessed. This restriction comes about because the XML-RPC protocol used by ParselTongue for remote execution and data access is not suitable for transferring large amounts of data.

Table access in the Wizardry module is similar to normal table access, but with extra functionality for changing existing tables and creating new ones.

New tables can be appended to an AIPSData object via the attach_table method. The following bit of code creates a new CL table with a different amplitude calibration:

from AIPS import AIPS
from Wizardry.AIPSData import AIPSUVData

AIPS.userno = 667

data = AIPSUVData('N03L1', 'UVDATA', 1, 1)
oldcl = data.table('CL', 1)

newcl = data.attach_table('CL', 2, no_term=oldcl.keywords['NO_TERM'])
newcl.keywords['NO_ANT'] = oldcl.keywords['NO_ANT']

for row in oldcl:
row.real1 = [2 * x for x in row.real1]
row.real2 = [2 * x for x in row.real2]
newcl.append(row)

newcl.close()
oldcl.close()
			

As you can see, the table attribute of AIPSData objects are exposed as a set of rows, which can be modified as you like:

from Wizardry.AIPSData import AIPSUVData as WAIPSUVData
from AIPS import AIPS

AIPS.userno = 667

data = WAIPSUVData('N03L1', 'UVDATA', 1, 1)
table = data.table('CL', 1)

for row in table:
    row.real1 = [2 * x for x in row.real1]
    row.real2 = [2 * x for x in row.real2]
    row.update()

table.close()	
			

Note the import AIPSUVData as WAIPSUVData above. While not required, you may find it advantageous to import Wizardry objects with a slightly different name to more easily use both the Wizardry and mundane versions of AIPSData objects.

It may be the case that you have a data set within AIPS that needs to be moved to another AIPS installation. Beginning with version 1.1, the ParselTongue distribution includes the ParselFileServer program, which runs on the system you wish to transfer files to. Once it is running, you can have your scripts load the Utilities module, and copy data over with the rcopy method. Note also the use of the rdiskappend method for easily adding proxy disks for use by ParselTongue (the method returns the index in the AIPS.disks array corresponding to the newly-added disk).

import Utilities
import AIPS
import AIPSData  
import AIPSTask

AIPS.userno = 666
proxy = "http://my.pt.proxy.nl:8000"
rdisk = 3  # The number of the AIPS disk on the remote host

remoteno = Utilities.rdiskappend(proxy,rdisk)

# Set up the local data object, which exists, and the remote
# object, which doesn't exist (until the copy is performed)
ldata = AIPSData.AIPSImage('MANDL','MANDL',1,1)
rdata = AIPSData.AIPSImage('MANDL','MANDL',remoteno,1)

# Perform the copy
Utilities.rcopy(ldata,rdata)
                    

A number of useful tasks have been written for AIPS, but are not included in the installation proper. These external tasks can be seen and run by ParselTongue with very little extra trouble.

To get started, let's imagine that we have a task BLAH.EXE, which we would like to invoke in a ParselTongue script. We should also have the help file BLAH.HLP, used to generate the task parameter list. Now, to create a directory structure which will contain the external tasks you want to run. You'll need a HELP subdirectory, along with an $ARCH/LOAD directory, where $ARCH is the name AIPS uses for the architecture of your machine. This is almost certainly one of LINUX (Intel-based Linux), MACINT (Intel-based Mac OSX), or SOL (Solaris). So, on a PC running Linux, we'd do the following:

$ mkdir -p external/HELP
$ mkdir -p external/LINUX/LOAD
$ mv BLAH.HLP external/HELP
$ mv BLAH.EXE external/LINUX/LOAD
                    

Next, set an environmental variable for the path of your directory:

$ export MYTASKS=/home/sipior/external
			

Fire up ParselTongue as usual, and use the name of the environment variable when creating your AIPSTask instance:

>>> blah = AIPSTask('BLAH', version='MYTASKS')
>>> blah.adverbs = "whatever"
>>> blah.go()
			

ParselTongue can also execute jobs on remote computers, allowing you to make use of powerful, well-equipped machines from the comfort of your laptop, cell phone or whatever.

Executing jobs remotely with ParselTongue is pretty straightforward. ParselTongue uses the XML-RPC protocol to communicate with a ParselTongueServer instance running on the remote machine. So, the first step is to make sure that ParselTongue is installed on both the local and remote machines. Once this is done, start the XML-RPC server on the remote computer:

localhost$ ssh remotehost
remotehost% nohup ParselTongueServer &
			

The nohup command ensures that ParselTongueServer will not be terminated when you log off the system. Don't forget to ensure that the appropriate LOGIN.SH or LOGIN.CSH file has been sourced before starting the Server.

Once this is done, the following code snippet will set up an AIPSTask that will execute on the remote host when its go() method is invoked:

# Append remote server to global proxies variable. Specify the hostname of
# the remote server, along with the port number on which ParselTongueServer
# is listening (8000, by default)

AIPS.proxies.append("http://jop01.nfra.nl:8000")

# Instantiate a disk instance for the remote machine, in this case rdisk
# will refer to AIPS disk 1 on the proxy referred to by AIPS.proxies[1]

rdisk = AIPSDisk(AIPS.proxies[1],1)

# Append disk instance to global disks variable

AIPS.disks.append(rdisk)

# Use the appropriate AIPS.disks entry in constructing the data object Here,
# we could replace "2" with "len(AIPS.disks) - 1", since we know that this
# would refer to the entry we just appended to the AIPS.disks list.

data = AIPSImage('MANDL','MANDL',2,1,666)

# provide appropriate data objects for task parameters

job = AIPSTask('mandl')
job.outdata = data
# And we're off, on the proxy listening on port 8000 of host jop01.nfra.nl
job.go() 
			

As of version 1.1, ParselTongue is capable of running multiple AIPS tasks simultaneously. To be clear, this is not true parallelisation, with active communication between the computational nodes. Fundamentally, ParselTongue despatches AIPS tasks, and so a truly parallel ParselTongue would require a parallel AIPS. However, many data reduction problems benefit greatly from so-called "trivial parallelisation", which is a fancy way of saying that if you have four independent data sets, and four computers, you can handily save yourself a great deal of time by running four data reduction processes simultaneously.

With that proviso, let's talk about how one gets ParselTongue to do several things at once. The ParallelTask module contains all you need to arrange this. It contains the ParallelQueue and ParallelTask classes, but you will only need to make use of the former (which is simply a container for ParallelTask objects) directly. You can create your AIPSTask objects just as before, and add them to your work queue with the ParallelQueue.queue method. Once you have added a set of tasks, the ParallelQueue.go method will despatch them all to the various hosts pointed to by the relevant AIPSData objects.

In addition, the logging facility typically employed in a ParselTongue run (where messages are dumped to the terminal, and/or a single file), has some problems when dealing with multiple tasks, all sending back messages to the user in an essentially random order. To establish some useful order over your output, consider setting the log attribute for each AIPSTask in your task queue. As discussed in the Logging section, setting this attribute to a file object causes all AIPS output from that task to be redirected there.

An example might be useful here. The following script executes two simple processes (the MANDL task from AIPS), on two separate machines (jop01 and jop02). Both of the remote hosts are, of course, running an instance of the ParselTongueServer daemon to handle the remote task execution.

import ParallelTask
import time, sys
import Utilities, AIPS, AIPSData, AIPSTask

AIPS.userno = 666

queue = ParallelTask.ParallelQueue()

# Add the desired ParselTongue proxies, and entries for the remote disks
# that will hold the task output.
AIPS.proxies.append("http://jop01.nfra.nl:8000")
AIPS.proxies.append("http://jop02.nfra.nl:8000")
rdisk1 = AIPS.AIPSDisk(AIPS.proxies[len(AIPS.proxies)-1],1)
rdisk2 = AIPS.AIPSDisk(AIPS.proxies[len(AIPS.proxies)-1],2)
AIPS.disks.append(rdisk1)
AIPS.disks.append(rdisk2)

data1 = AIPSData.AIPSImage('MANDL','MANDL',len(AIPS.disks)-2,1,AIPS.userno)
data2 = AIPSData.AIPSImage('MANDL','MANDL',len(AIPS.disks)-1,2,AIPS.userno)

mandl1 = AIPSTask.AIPSTask('MANDL')
mandl2 = AIPSTask.AIPSTask('MANDL')
mandl1.outdata = data1
mandl2.outdata = data2
mandl1.imsize = [None,4096,4096]
mandl2.imsize = [None,4096,4096]

# Set the task output logs
mandl1.log = open("mandl1.log","a")
mandl2.log = open("mandl2.log","a")

# Add these tasks to the work queue
queue.queue(mandl1)
queue.queue(mandl2)

# And we're off!
queue.go()
                        

ParselTongue provides a simple, yet powerful way to capture a log of your session. If you set AIPS.log to a file object all output from AIPS tasks will be sent to that object:

>>> AIPS.log = open('ParselTongue.log', 'a')
			

In fact, the above is not the full truth. AIPS.log does not necessarily have to be a true Python file object; any object with a write method will do.

The output sent to AIPS.log includes the messages that are not being displayed on your terminal because the msgkill attribute was set to a negative value in the task object (this does not apply to messages killed by a positive value of this attribute; those are lost forever).

To stop logging, set AIPS.log to None.

>>> mandl = AIPSTask('MANDL')
# This sets the log for the mandl AIPSTask to the file mandl1.log
>>> mandl.log = open('mandl1.log','a')