LevelMatchStep

This step matches relative levels between tiles, using an algorithm very similar to Montage. We reproject tiles to a common astrometric grid, and then for each overlapping pair find the median per-pixel difference, and minimize for that. We do this in a two (or three)-pass process, firstly for dithers within a mosaic tile, then optionally for main science chip and lyot coronagraph, optionally between the four NIRCam short chips, and then for all mosaic tiles (creating stacked images). This maximises overlaps between mosaic tiles and produces better results from our testing.

For MIRI imaging, there is an optional recombine_lyot parameter. If you’ve used the lyot_separate step then this will match the coronagraph and main science chip and recombine them before any potential matching between different tiles in the mosaic.

For NIRCam short imaging, there is an optional combine_nircam_short parameter. This will match levels between the four NIRCam short chips and then will treat as one single image going into the final mosaic level matching stage.

For each of these three stages, you can match either with a simple constant offset (“level”) or with a plane (“level+match”). This can be controlled separately for each stage (fit_type_dithers, fit_type_recombine_lyot, fit_type_combine_nircam_short, fit_type_mosaic_tiles). By default, everything will just fit a constant offset.

When matching between mosaic tiles, we select a reference image that we deem to be the most flat, and will adjust any corrections relative to that. If you are using observations with dedicated backgrounds (as defined by Lv2Step, then it will select the image closest in time to the backgrounds. If not, it will use the image with the most overlapping image pairs.

N.B. This should be run once you have good local astrometry, and before you do MultiTileDestripeStep.

API

class pjpipe.LevelMatchStep(in_dir, out_dir, step_ext, procs, band, fit_type_dithers='level', fit_type_recombine_lyot='level', fit_type_combine_nircam_short='level', fit_type_mosaic_tiles='level', recombine_lyot=False, combine_nircam_short=False, do_local_subtraction=True, sigma=3, npixels=3, dilate_size=7, max_iters=20, max_points=10000, do_sigma_clip=False, weight_method='equal', min_area_percent=0.002, min_linear_frac=0.2, rms_sig_limit=2, reproject_func='interp', overwrite=False)

Perform background matching between tiles

This step performs background matching by minimizing the per-pixel differences between overlapping tiles. It does this first for dither groups, before creating a stacked image of these (to maximize areal coverage) and minimizing between all stacked images within a mosaic. This is necessary for observations that don’t really have a background, and performs significantly better than the JWST implementation.

N.B. If you use this, skymatch in the level 3 pipeline stage should be global or off, to avoid undoing this work

Parameters:

• in_dir – Input directory

• out_dir – Output directory

• step_ext – .fits extension for the files going into the step

• procs – Number of parallel processes to run

• band – JWST band

• fit_type_dithers – What kind of fit to do to match levels between dithers in a single mosaic tile. Options are ‘level’ (the default, just fits a single offset), and ‘level+slope’ (which will fit a plane)

• fit_type_recombine_lyot – What kind of fit to do to match levels between science and lyot in a single mosaic tile. Options are ‘level’ (the default, just fits a single offset), and ‘level+slope’ (which will fit a plane)

• fit_type_combine_nircam_short – What kind of fit to do to match levels between the four NIRCam short chips. Options are ‘level’ (the default, just fits a single offset), and ‘level+slope’ (which will fit a plane)

• fit_type_mosaic_tiles – What kind of fit to do to match levels between mosaic tiles. Options are ‘level’ (the default, just fits a single offset), and ‘level+slope’ (which will fit a plane)

• recombine_lyot – If True, will recombine the lyot coronagraph into the main chip after the initial round of level matching. This will force the main science chip to have a 0 correction, as the lyot seems to be more wobbly. Defaults to False

• combine_nircam_short – Whether to combine the four NIRCam short chips before matching in a mosaic. Defaults to False

• do_local_subtraction – Whether to do a sigma-clipped local median subtraction. Defaults to True

• sigma – Sigma for sigma-clipping. Defaults to 3

• npixels – Pixels to grow for masking. Defaults to 5

• dilate_size – make_source_mask dilation size. Defaults to 7

• max_iters – Maximum sigma-clipping iterations. Defaults to 20

• max_points – Maximum points to include in histogram plots. This step can be slow so this speeds it up. Defaults to 10000

• do_sigma_clip – Whether to do sigma-clipping on data when reprojecting. Defaults to False

• weight_method – How to weight in least-squares minimization. Options are ‘equal’ (equal weighting), ‘npix’ (weight by number of pixels), and ‘rms’ (weight by rms of the delta values). Defaults to ‘equal’

• min_area_percent – Minimum percentage of average areal overlap to remove tiles. Defaults to 0.002 (0.2%)

• min_linear_frac – Minimum linear overlap in any direction to keep tiles. Defaults to 0.2

• rms_sig_limit – Sigma limit for cutting off noisy fits. Defaults to 2

• reproject_func – Which reproject function to use. Defaults to ‘interp’, but can also be ‘exact’ or ‘adaptive’

• overwrite – Whether to overwrite or not. Defaults to False

calculate_delta(files, fit_type='level', stacked_image=False, procs=None)

Match relative offsets between tiles

Parameters:

• files (list) – List of files to match

• fit_type – Which type of fit to do. See ALLOWED_FIT_TYPES. Defaults to “level”

• stacked_image – Whether this is a stacked image or not. Default to False

• procs (int, optional) – Number of processes to run in parallel. Defaults to None, which is series

do_step()

Run level matching

find_optimum_deltas(delta_mat, npix_mat, rms_mat, lin_size_mat, valid_mat, fit_type='level', n_draws=25, n_iter=10000, convergence_abs_tol=1e-08, convergence_rel_tol=1e-05, ref_idx=None)

Get optimum deltas from a delta/weight matrix.

Taken from the JWST skymatch step, with some edits to remove potentially bad fits due to small areal overlaps, or noisy diffs, and various weighting schemes.

If we’re fitting a plane, delta_mat will be an NxNx3 matrix, and we’ll minimize over each of the last axes separately

Parameters:

• delta_mat (np.ndarray) – Matrix of delta values. These may be [a, b, c] coefficients if we’re fitting a plane

• npix_mat (np.ndarray) – Matrix of number of pixel values for calculating delta

• rms_mat (np.ndarray) – Matrix of RMS values

• lin_size_mat (np.ndarray) – 1/0 array for whether overlaps pass minimum linear extent

• valid_mat (np.ndarray) – 1/0 array for whether overlaps are valid

• fit_type – Which type of fit to do. See ALLOWED_FIT_TYPES. Defaults to “level”

• n_draws – When using the iterative method, we need to sample from the fitted plane. This controls how many draws we do. Defaults to 25

• n_iter – Maximum number of iterations before breaking out of the fitting routine. Defaults to 10,000

• convergence_abs_tol – Absolute tolerance to define convergence. Defaults to 1e-8

• convergence_rel_tol – Relative tolerance to define convergence. Defaults to 1e-5

• ref_idx – Index to define the zero level for all the level matching. Defaults to None, which will use the average correction

get_level_match(files1, files2, fit_type, ref_x=0, ref_y=0, plot_name=None, maxiters=10, plane_fit_maxiters=20, plane_fit_abs_tol=1e-08, plane_fit_rel_tol=1e-05)

Calculate relative difference between groups of files on the same pixel grid

Parameters:

• files1 – List of files to get difference from

• files2 – List of files to get relative difference to

• fit_type – Which type of fit to do. See ALLOWED_FIT_TYPES. Defaults to “level”

• ref_x – Reference x-coord to perform fits relative to. Defaults to 0

• ref_y – Reference y-coord to perform fits relative to. Defaults to 0

• plot_name – Output plot name. Defaults to None

• maxiters – Maximum iterations for the sigma-clipping. Defaults to 10

• plane_fit_maxiters – Maximum number of iterations for the plane fitting. Defaults to 20

• plane_fit_abs_tol – Absolute tolerance to define convergence in the plane fit. Defaults to 1e-8

• plane_fit_rel_tol – Relative tolerance to define convergence in the plane fitting. Defaults to 1e-5

get_per_dither_delta(dithers, fit_type='level', procs=1)

Function to parallelise getting the delta for each observation in a dither sequence

Parameters:

• dithers – List of dithers to get deltas for

• fit_type – Which type of fit to do. See ALLOWED_FIT_TYPES. Defaults to “level”

• procs – Number of processes to run simultaneously. Defaults to 1

get_plot_name(files1, files2)

Make a plot name from list of files for level matching

Parameters:

• files1 – First list of files

• files2 – Second list of files

get_reproject(file, optimal_wcs, optimal_shape, stacked_image=False)

Reproject files, maintaining list structure

Parameters:

• file – List or single file to reproject

• optimal_wcs – WCS to reproject to

• optimal_shape – output array shape for the WCS

• stacked_image (bool) – Whether this is a stacked image or not. Defaults to False

make_stacked_images(dithers, stacked_dir, procs=1)

Function to parallellise up making stacked dither images

Parameters:

• dithers – List of dithers to go

• stacked_dir – Where to save stacked images to

• procs – Number of simultaneous processes to run. Defaults to 1

match_dithers(dithers, fit_type='level')

Match levels between the dithers in each mosaic tile

Parameters:

• dithers – List of dither groups

• fit_type – Which type of fit to do. See ALLOWED_FIT_TYPES. Defaults to “level”

match_lyot_science(dithers, fit_type='level')

Match levels between each individual lyot/main science chip, and recombine

Parameters:

• dithers – List of dithers

• fit_type – Which type of fit to do. See ALLOWED_FIT_TYPES. Defaults to “level”

match_mosaic_tiles(dithers, fit_type='level')

Match levels between each mosaic tile

Parameters:

• dithers – List of dither groups

• fit_type – Which type of fit to do. See ALLOWED_FIT_TYPES. Defaults to “level”

match_nircam_short(dithers, fit_type='level')

Match levels between the NIRCam short chips

Parameters:

• dithers – List of dithers

• fit_type – Which type of fit to do. See ALLOWED_FIT_TYPES. Defaults to “level”

parallel_delta_matrix(ij, file_reproj, files, fit_type='level', ref_x=0, ref_y=0)

Function to parallelise up getting delta matrix values

Parameters:

• ij – List of matrix (i, j) values

• file_reproj – Reprojected file

• files – Full list of files

• fit_type – Which type of fit to do. See ALLOWED_FIT_TYPES. Defaults to “level”

• ref_x – Reference x value to subtract to centre the fits. Defaults to 0

• ref_y – Reference y value to subtract to centre the fits. Defaults to 0

parallel_get_reproject(idx, files, optimal_wcs, optimal_shape, stacked_image=False)

Light function to parallelise get_dither_reproject

Parameters:

• idx – File idx to reproject

• files – Full file list

• optimal_wcs – Optimal WCS for input stack of images

• optimal_shape – Optimal shape for input stack of images

• stacked_image – Stacked image or not? Defaults to False

parallel_make_stacked_image(dither, out_dir)

Light wrapper around parallelising the stacked image

Parameters:

• dither – Dither to stack

• out_dir – Directory to save to

parallel_match_lyot_science(dither, stacked_dir, fit_type)

Function to parallelise up combining the lyot back into the main science chip

Because the lyot seems to behave a little weirdly in its backgrounds from time-to-time, we force the main science correction to be 0 and put that all into the lyot

Parameters:

• dither – Dither to level match and combine

• stacked_dir – Directory contained stacked images

• fit_type – Which type of fit to do. See ALLOWED_FIT_TYPES. Defaults to “level”

parallel_match_nircam_short(dither, stacked_dir, fit_type)

Function to parallelise up matching levels between the short NIRCam chips

Parameters:

• dither – Dither to level match and combine

• stacked_dir – Directory contained stacked images

• fit_type – Which type of fit to do. See ALLOWED_FIT_TYPES. Defaults to “level”

parallel_per_dither_delta(dither, fit_type='level')

Function to parallelise up matching dithers

Parameters:

• dither – Input dither

• fit_type – Which type of fit to do. See ALLOWED_FIT_TYPES. Defaults to “level”