aussrc / possum_workflow Goto Github PK

Output mosaic files need SBIDs of input image cubes to the HISTORY

3D cubes require this filename convention

prefix.bandwidth.resolution.RADEC.TileID.Stokes.fits

(e.g. POSSUM.800-1088MHz.18asec.2148-5115.10978.i.fits) while MFS images require

POSSUM.944MHz.18asec.2148-5115.10978.t0.i.fits

where instead of bandwith use the central frequency of the MFS cube, and add t0 or t1 for Taylor 0 or 1. These filename updates will require reading the fits header of the output file, so it may be required that these are added after the mosaicking step of the pipeline.

Beamlog for recent observations (Pilot 2) is stored in a fits extension (hdul[1] or something).

These are not propagated. It is likely due to the splitting step.

We currently have to split the cube in frequency (number of subcubes specified by the NAN_TO_ZERO_NSPLIT parameter) to set NaN values to zero, which is required for robust method of beamcon_3D. Without setting NaN to zero, we get all NaN values in the entire frequency slice if there are any NaN values at all.

Try using the astropy method instead. This may take longer but does not require splitting and re-joining of the split cubes.

Tile components are generated for each observation and are intermediate products produced by the POSSUM pipeline. These are mosaicked later in the pipeline when the user (POSSUM team member) decided there are a sufficient number of components to complete a tile.

There is no guarantee that the observations will be adjacent, so these temporary tile components may need to be stored for longer than the scratch 30 day limit. We will need to store these in Acacia and move them back when necessary. Consider updating the mosaicking pipeline code (mosaic.nf) to pull images from Acacia rather than the shared filesystem.

Regex search for observation ID in filename currently does not include the A/B extension of some IDs. This needs to be fixed in order to work on all observations for Full Survey.

link: https://github.com/AusSRC/POSSUM_workflow/blob/main/modules/hpx_tile_map.nf#L44