A CARACal run involves running a sequence of CARACal workers following the order in which they
are listed in the user configuration file. For reference see List and Brief description of CARACal workers.
Users can design their own CARACal workflow with some level of flexibility. The exact workflow
(i.e., the exact sequence of CARACal workers) will depend on science goals but also on I/O
constraints. Below we provide two examples.
Recommended workflow
The currently recommended CARACal workflow minimises the data volume increase
during the course of a full CARACal run, and allows to treat the input .MS file(s)
as read-only.
This workflow is implemented in all sample configuration files located at
https://github.com/caracal-pipeline/caracal/tree/master/caracal/sample_configurations .
Compulsory worker to set up data/input/output directories.
Compulsory worker to specify the input .MS files.
Compulsory worker to set up target/calibrators names.
Split calibrators-only .MS files, one per input .MS file.
Prepare the calibrators-only .MS files for processing.
Flag the calibrators-only .MS files.
Derive the cross-calibration tables and apply them to the calibrators.
Inspect the calibrated calibrator’s visibilities to check the quality of the
cross-calibration.
Split target-only .MS files, one per input .MS file and target, applying the
cross-calibration on the fly.
Prepare the target-only .MS files for processing.
Flag the target-only .MS files.
Average the target-only .MS files in frequency for continuum imaging.
Flag line channels in the averaged target-only .MS files.
Make a continuum image of each target, self-calibrate, and transfer both gains
and continuum model to the full-frequency-resolution target-only .MS files.
Subtract the continuum, Doppler correct and make the line cube and moment
images from the target-only .MS files.
Mosaic the continuum images of the targets.
Mosaic the line cubes.
Note that this workflow includes multiple runs of several workers. These require a
“__<suffix>” in the configuration file, as described at the page Configuration file.
The data volume budget of this workflow is:
- input .MS:
- target-only .MS:
- DATA
- CORRECTED_DATA
- MODEL
- calibrators-only .MS (negligible data volume):
- DATA
- CORRECTED_DATA
- MODEL
- target-only frequency-averaged .MS (negligible data volume):
- DATA
- CORRECTED_DATA
- MODEL
Simple workflow
Simpler workflows than the recommended one are possible. For example, the workflow
below employs less worker runs. However, it modifies the input .MS file(s), results into
a larger data volume increase, and runs the self-calibration loop on larger .MS files.
Compulsory worker to set up data/input/output directories.
Compulsory worker to specify the input .MS files.
Compulsory worker to set up target/calibrators names.
Prepare the input .MS files for processing.
Flag the calibrators in the input .MS files.
Derive the cross-calibration tables and apply them to target and calibrators.
Inspect the calibrated calibrator’s visibilities to check the quality of the
cross-calibration.
Split cross-calibrated target-only .MS files, one per input .MS file and target.
Flag the target-only .MS files.
Make a continuum image of each target, self-calibrate, and transfer both gains
and continuum model to the full-frequency-resolution target-only .MS files.
Subtract the continuum, Doppler correct and make the line cube and moment
images from the target-only .MS files.
Mosaic the continuum images of the targets.
Mosaic the line cubes.
The data volume budget of this workflow is:
- input .MS
- DATA
- CORRECTED_DATA
- MODEL
- target-only .MS
- DATA
- CORRECTED_DATA
- MODEL
Typically, this is 50% larger than for the Recommended workflow above.