DownloadStep

This step automates downloading from MAST, sorting things into folders with the layout the pipeline will expect going forwards. You should pass at least a target or a proposal ID, and for proprietary data you should set login to True, at which point it will ask for a login API key. For large targets or those with backgrounds that are far away, you may need to bump up the radius parameter.

API

class pjpipe.DownloadStep(target=None, prop_id=None, download_dir=None, procs=1, radius=None, telescope='JWST', instrument_name=None, dataproduct_type=None, calib_level=None, extension=None, do_filter=True, breakout_targets=False, product_type=None, filter_gs=True, login=False, api_key=None, overwrite=False)[source]

Query and download data from MAST

Parameters:

  • target – Target name. Defaults to None, which will use the proposal ID to figure out names

  • prop_id – Proposal identifier. Defaults to None, but at least one of target, prop_id needs to be set

  • download_dir – Where to download to. Defaults to current directory if not set

  • procs – Number of processes to run in parallel. Defaults to 1

  • radius – Radius to search from centre of target in arcmin. Defaults to None, which uses astroquery default

  • telescope – Telescope to use. Defaults to ‘JWST’

  • instrument_name – Something like ‘NIRCAM/IMAGE’, to filter down what products you want. Defaults to None (no filtering)

  • dataproduct_type – These are things like ‘cube’, ‘image’ etc. Defaults to None (no filtering)

  • calib_level – Calibration levels to filter. Defaults to None (no filtering)

  • extension – File extensions to filter by. Defaults to None (no filtering)

  • do_filter – Whether to filter dataproduct_type/calib_level/extension. Defaults to True

  • breakout_targets – If targets is None, this will either download the whole proposal to a single directory (False), or breakout each target name to its own directory (True). Defaults to False

  • product_type – Product types to download. Defaults to None (no filtering)

  • filter_gs – Whether to filter guide stars from the observations. Defaults to True

  • login – For proprietary data, set to True. Defaults to False

  • api_key – If login is True, supply API key here

  • overwrite – If True, overwrites existing data. Defaults to False

do_step()[source]

Run download step

get_products(obs_list)[source]

Get products from an observations list

Parameters:

obs_list – List of observations to get products from

run_archive_query()[source]

Query archive, trimming down as requested on instrument etc.

run_download(max_retries=5)[source]

Download a list of observations

Parameters:

max_retries (int) – Maximum number of retries before failing. Defaults to 5

verify_integrity()[source]

Check files have downloaded successfully