next up previous contents index
Next: Polarization Handling Up: imager Previous: UV table consistency:   Contents   Index


Really Huge Problems

[rgb]1,0,0IMAGER basically assumes everything fits into memory. This is in general quite fine for NOEMA data. However, for ALMA data, if you are working with a too small computer (such as my laptop, which is otherwise fine), you may be lacking physical memory (RAM, Random Access Memory), and [rgb]1,0,0IMAGER may become really inefficient by using Virtual Memory instead.

To avoid time losses, [rgb]1,0,0IMAGER prevents reading UV data whose size exceeds the available RAM, and warns the user if it exceeds half of the RAM. To treat these cases, [rgb]1,0,0IMAGER provides instead a number of tools that can work sequentially on the data set, instead of loading it in memory all at once.

  1. Working by subsets:
    the magentaREAD /RANGE command allows to select an ensemble of channels from a UV data. If this ensemble is small enough, [rgb]1,0,0IMAGER can work. At the end the magentaWRITE /APPEND and magentaWRITE /REPLACE command will allow to put these channels at their proper places in a deconvolved data cube.
  2. Working on UV data files:
    Some operations on UV data, like time averaging, separation of line from continuum emission, or self-calibration, and of course, spectral resampling, are best done using all valid channels to avoid loosing sensitivity. To allow [rgb]1,0,0IMAGER to do them even for large data files, most UV-related commands have a magenta/FILE option which instructs the command to work from the corresponding data file, instead of the UV buffers. This includes magentaUV_PREVIEW /FILE, magentaUV_BASELINE /FILE, magentaUV_FILTER /FILE and especially the magentaUV_SPLIT /FILE commands. Time averaging can be performed by magentaUV_TIME /FILE, and prior sorting by time order can be done by magentaUV_SORT /FILE. Spectral range extraction is possible by magentaUV_EXTRACT /FILE, spectral resampling by magentaUV_COMPRESS /FILE, magentaUV_RESAMPLE /FILE, and magentaUV_HANNING /FILE.
By using the above commands, all operations can be done in a quasi sequential way, avoiding to load in memory whole data sets.

Two commands have no equivalent using the [rgb]1,0,0IMAGER buffers, and work only on files. Their magenta/FILE option is used to provide an homogeneous syntax, but must be present for the command line to be valid.

  1. the magentaUV_MERGE /FILE
    that allows to merge together an arbitrary number of UV tables, in spectral line (with spectral resampling) or continuum (with flux scaling according to a spectral index) modes.
  2. the magentaUV_SPLIT /FILE
    that combines the capabilities of magentaUV_BASELINE and magentaUV_FILTER in a single command, since both operations require the same parameters and provide complementary informations.

The magentaUV_SORT /FILE command also has a different behaviour than its memory only version magentaUV_SORT. While the latter creates a transposed version of the UV table for internal use (it cannot be saved), the former keeps the normal organisation with the visibility axis first.

An example of use of such facilities is the magenta@violet image-mosaic script that splits the magentauv_map step into chunks that fit in the computer memory available to the user.


next up previous contents index
Next: Polarization Handling Up: imager Previous: UV table consistency:   Contents   Index
Gildas manager 2023-06-01