next up previous contents
Next: Positions Up: Reading the DATA column Previous: Dump cycle identification   Contents

Reading the DATA column by pieces

MRTCAL is designed to be able to process the subscans within an arbitrary limited buffer size (MSET BOOKKEEPING SPACE Value, default 512 MBytes). If the buffer size is large enough, the whole DATA column can be loaded at once in memory and its dumps can be treated in a row. If the buffer is not large enough, the DATA column has to be load by pieces, and each piece has to be processed separately from the other.

In practice, the calibration routine asks the reading routine to load the piece of DATA which contains at least a desired range of dumps (usually a cycle of a few dumps). At least means that the reading routine ensures that the range will be present entirely in the buffer, but more dumps will also be loaded in order to fill at best the buffer. On subsequent requests, if the desired range is already present in the buffer, nothing will be reloaded, in order to save calls to CFITSIO and I/O accesses.

At this stage, one should consider that CFITSIO is asked to read a block of rows from the DATA column. CFITSIO is not able to skip some bad rows here and there, and it would be unefficient to ask for reading the desired rows one by one. So CFITSIO reads a piece of the DATA column, including good and bad dumps. But right after, MRTCAL sets a 3D chunkset ( $N_{set} \times N_{pix}
\times N_{time}$) pointing to the good dumps only. Then the calling routine has just to work with the 3D chunkset, with no need to care for bad dumps.


next up previous contents
Next: Positions Up: Reading the DATA column Previous: Dump cycle identification   Contents
Gildas manager 2023-06-01