pygrife.grizli_functions package#

Module containing modified grizli functions.

These functions were originally written by Gabe Brammer et al., and the original repository can be found at gbrammer/grizli. Their usage here follows the MIT licence, a copy of which is included in this module, at pygrife/grizli_functions/LICENCE.txt.

pygrife.grizli_functions.FLT_fns module#

Grizli functions used to load grism files and extract beam cutouts.

pygrife.grizli_functions.FLT_fns.init_from_input_multispec(self, flt, beam, conf=None, get_slice_header=True)#

Initialize a grizli.model.BeamCutout object from data objects.

The original function was taken from init_from_input, and has been modified to allow for a cutout overlapping multiple existing models in the contamination map (including multiple different spectra).

Parameters:

fltgrizli.model.GrismFLT: Parent FLT frame.
beamgrizli.model.GrismDisperser: Object and spectral order to consider.
confgrizli.grismconf.aXeConf: Pre-computed configuration file. If not specified will regenerate based on header parameters, which might be necessary for multiprocessing parallelization and pickling.
get_slice_headerbool: Get full header of the sliced data. Costs some overhead so can be skipped if full header information isn’t required.

pygrife.grizli_functions.FLT_fns.get_beams_with_spectrum(grp, id, size=10, center_rd=None, beam_id='A', min_overlap=0.1, min_valid_pix=10, min_mask=0.01, min_sens=0.08, mask_resid=True, get_slice_header=True, show_exception=False, spectrum_1d=None, is_cgs=False)#

Extract 2D spectra “beams” from the GroupFLT exposures.

The original function was taken from get_beams, and has been modified by:

Allowing a 1D spectrum to be passed through to beam.model.

Identifying the overlapping object IDs in the previous contamination model and segmentation map (if it exists), and passing them through to BeamCutout.

Parameters:

grpgrizli.multifit.GroupFLT: The container for multiple grism exposures.
idint: Catalog ID of the object to extract.
sizeint: Half-size of the 2D spectrum to extract, along cross-dispersion axis.
center_rd(float, float), optional: Extract based on RA/Dec rather than catalog ID.
beam_idstr: Name of the order to extract.
min_overlapfloat: Fraction of the spectrum along wavelength axis that has one or more valid pixels.
min_valid_pixint: Minimum number of valid pixels (beam.fit_mask == True) in 2D spectrum.
min_maskfloat: Minimum factor relative to the maximum pixel value of the flat f-lambda model where the 2D cutout data are considered good. Passed through to BeamCutout.
min_sensfloat: See BeamCutout.
mask_residbool: See BeamCutout.
get_slice_headerbool: Passed to BeamCutout.
show_exceptionbool: Show the exception if the beam cannot be loaded from an FLT file. False by default.
spectrum_1d[wavelengths, flux], optional: The flux spectrum and corresponding wavelengths of the object in the model. By default, this is calculated automatically from the stored object_dispersers.
is_cgsbool, optional: The flux units of spectrum_1d[1] are cgs f_lambda flux densities, rather than normalised in the detection band, by default False.

Returns:

list: List of BeamCutout objects.

pygrife.grizli_functions.FLT_fns.load_and_mod_FLT(grism_file, sci_extn, direct_file, pad, ref_file, ref_ext, seg_file, verbose, catalog, ix, use_jwst_crds)#

Helper function for loading GrismFLT objects.

Originally taken from grizli.multifit._loadFLT, modified to allow for storing an original segmentation map, used for the contamination model, and an updated segmentation map, used for current object extraction. Docstring compiled by PJW for consistency.

Parameters:

grism_filestr: The processed grism file, typically in the form "*GrismFLT.fits".
sci_extnint: Science extension to extract from grism_file. For WFC3/IR this can only be 1, though for the two-chip instruments WFC3/UVIS and ACS/WFC3 this can be 1 or 2.
direct_filestr: A direct exposure, corresponding to the grism exposure. If this is empty, ref_file will be used instead.
padint or tuple[int, int]: The padding in pixels, allowing modelling of sources outside the detector field of view. If a tuple is supplied, this is taken as (pady, padx).
ref_filestr or None: Undistorted reference image filename, e.g., a drizzled mosaic covering the area around a given grism exposure.
ref_extint: The FITS extension of the reference file containing the image.
seg_filestr or None: Segmentation image filename. If this is supplied, and it does not match the saved segmentation image, the returned GrismFLT object will have two additional attributes, orig_seg_file, and orig_seg.
verbosebool: Print verbose information.
catalogstr: Catalog filename assocated with seg_file. These are typically generated with “SExtractor”, but the source of the files themselves isn’t critical.
ixint: The index of the returned GrismFLT object in the parent GroupFLT object.
use_jwst_crdsbool: Use CRDS specwcs reference files for JWST instruments.

Returns:

grizli.model.GrismFLT: The instantiated grism object, with all attributes loaded.

pygrife.grizli_functions.FLT_fns.mod_transform_JWST_WFSS(self, verbose=True)#

Rotate data & wcs so that spectra are increasing to +x.

This function was modified from transform_JWST_WFSS, so that any stored original segmentation map (self.orig_seg) would also be rotated. Docstring compiled by PJW for consistency.

Parameters:

verbosebool: Print verbose information.

Returns:

bool: Returns True if the transformation completed.

pygrife.grizli_functions.FLT_fns.mod_compute_model_orders(self, id=0, x=None, y=None, size=10, mag=-1, spectrum_1d=None, is_cgs=False, compute_size=False, max_size=None, min_size=26, store=True, in_place=True, get_beams=None, psf_params=None, verbose=True)#

Compute the dispersed spectrum for a given object id.

The original function was taken from compute_model_orders, and has been modified to allow for a model computation overlapping multiple existing models in the contamination map (including multiple different spectra).

Parameters:

idint

Object ID number to match in the segmentation image.

x, yfloat

Center of the cutout to extract.

sizeint

Radius of the cutout to extract. The cutout is equivalent to:

xc, yc = int(x), int(y)
thumb = self.direct.data['SCI'][yc-size:yc+size, xc-size:xc+size]

magfloat

Specified object magnitude, which will be compared to the “MMAG_EXTRACT_[BEAM]” parameters in self.conf to decide if the object is bright enough to compute the higher spectral orders. The default value of -1 means that all orders listed in self.conf.beams will be computed.

spectrum_1dNone or [array, array]

Template 1D spectrum to convolve with the grism disperser. If None, assumes trivial spectrum flat in f_lambda flux densities. Otherwise, the template is taken to be:

wavelength, flux = spectrum_1d

is_cgsbool

Flux units of spectrum_1d[1] are cgs f_lambda flux densities, rather than normalized in the detection band.

compute_sizebool

Ignore x, y, and size and compute the extent of the segmentation polygon directly using compute_segmentation_limits.

max_sizeint or None

Enforce a maximum size of the cutout when using compute_size.

min_sizeint or None

Enforce a minimum size of the cutout when using compute_size.

storebool

If True, then store the computed beams in the OrderedDict self.object_dispersers[id].

If many objects are computed, this can be memory intensive. To save memory, set to False and then the function just stores the input template spectrum (spectrum_1d) and the beams will have to be recomputed if necessary.

in_placebool

If True, add the computed spectral orders into self.model. Otherwise, make a clean array with only the orders of the given object.

get_beamslist or None

Spectral orders to retrieve with names as defined in the configuration files, e.g., ['A'] generally for the +1st order of HST grisms. If None, then get all orders listed in the beams attribute of the aXeConf configuration object.

psf_paramslist

Optional parameters for generating an EffectivePSF object for the spatial morphology.

verbosebool

Print verbose information.

Returns:

bool or numpy.array

If in_place is True, return status of True if everything goes OK. The computed spectral orders are stored in place in self.model.

Returns False if the specified id is not found in the segmentation array independent of in_place.

If in_place is False, return a full array including the model for the single object.

pygrife.grizli_functions.catalogue_fns module#

Grizli functions used to regenerate the detection catalogue.

pygrife.grizli_functions.catalogue_fns.make_SEP_catalog(root='', sci=None, wht=None, threshold=2.0, get_background=True, bkg_only=False, bkg_params={'bh': 32, 'bw': 32, 'fh': 3, 'fw': 3}, verbose=True, phot_apertures='6, 8.33335, 11.66667, 16.66667, 20, 25, 50', aper_segmask=False, prefer_var_image=True, rescale_weight=True, err_scale=-inf, use_bkg_err=False, column_case=<method 'upper' of 'str' objects>, save_to_fits=True, include_wcs_extension=True, source_xy=None, compute_auto_quantities=True, autoparams=[2.5, <Quantity 0.35 arcsec>, 2.4, 3.8], flux_radii=[0.2, 0.5, 0.9], subpix=0, mask_kron=False, max_total_corr=2, detection_params={'clean': True, 'clean_param': 1, 'deblend_cont': 0.001, 'deblend_nthresh': 32, 'filter_kernel': array([[0.0049, 0.0213, 0.0513, 0.0687, 0.0513, 0.0213, 0.0049], [0.0213, 0.0921, 0.2211, 0.296 , 0.2211, 0.0921, 0.0213], [0.0513, 0.2211, 0.5307, 0.7105, 0.5307, 0.2211, 0.0513], [0.0687, 0.296 , 0.7105, 0.9511, 0.7105, 0.296 , 0.0687], [0.0513, 0.2211, 0.5307, 0.7105, 0.5307, 0.2211, 0.0513], [0.0213, 0.0921, 0.2211, 0.296 , 0.2211, 0.0921, 0.0213], [0.0049, 0.0213, 0.0513, 0.0687, 0.0513, 0.0213, 0.0049]]), 'filter_type': 'conv', 'minarea': 9}, bkg_mask=None, pixel_scale=0.06, log=False, gain=2000.0, extract_pixstack=30000000, sub_object_limit=4096, exposure_footprints=None, seg_image=None, in_dir=None, out_dir=None, seg_out_path=None, detect_cat=None, use_photutils=False, **kwargs)#

Make a source catalogue from drizzled images, using SExtractor.

This function was originally taken from make_SEP_catalog, and has been modified in the following ways:

Add in_dir and out_dir parameters, so that the operations can take place using files from one directory, but writing the output to another.
Add seg_out_path parameter, so that the name of the segmentation map output can be specified.
Add seg_image parameter. Using sep>=1.4.0, passing an array here allows one to bypass the object detection, and instead derive all the catalogue quantities for the specified objects.

Parameters:

rootstr

Rootname of the FITS images to use for source extraction. This function is designed to work with the single-image products from drizzlepac, so the default data/science image is searched by:

drz_file = glob.glob(f'{root}_dr[zc]_sci.fits*')[0]

Note that this will find and use gzipped versions of the images, if necessary.

The associated weight image filename is then assumed to be:

weight_file = drz_file.replace('_sci.fits', '_wht.fits')
weight_file = weight_file.replace('_drz.fits', '_wht.fits')

sci, whtstr

Filenames to override drz_file and weight_file derived from the root parameter.

thresholdfloat

Detection threshold for sep.extract.

get_backgroundbool

Compute the background with sep.Background.

bkg_onlybool

If True, then just return the background data array and don’t run the source detection.

bkg_paramsdict

Keyword arguments for sep.Background. Note that this can include a separate optional keyword pixel_scale that indicates that the background sizes bw, bh are set for a paraticular pixel size. They will be scaled to the pixel dimensions of the target images using the pixel scale derived from the image WCS.

verbosebool

Print status messages.

phot_aperturesstr or numpy:array_like

Photometric aperture diameters. If given as a string then assume units of pixels. If an array or list, can have units, e.g., astropy.units.arcsec.

aper_segmaskbool

If true, then run SEP photometry with segmentation masking. This requires the sep fork at gbrammer/sep.git, or sep >= 1.10.0.

prefer_var_imagebool

Use a variance image _wht.fits > _var.fits if found.

rescale_weightbool

If true, then a scale factor is calculated from the ratio of the weight image to the variance estimated by sep.Background.

err_scalefloat

Explicit value to use for the weight scaling, rather than calculating with 1`rescale_weight1`. Only used if err_scale > 0.

use_bkg_errbool

If true, then use the full error array derived by sep.Background.This is turned off by default in order to preserve the pixel-to-pixel variation in the drizzled weight maps.

column_casefunc

Function to apply to the catalog column names. E.g., the default str.upper results in uppercase column names.

save_to_fitsbool

Save catalog FITS file to {root}.cat.fits.

include_wcs_extensionbool

An extension will be added to the FITS catalog with the detection image WCS.

source_xy(x, y) or (ra, dec) arrays

Force extraction positions. If the arrays have units, then pass them through the header WCS. If no units, positions are zero-indexed array coordinates.

To run with segmentation masking (sep >= 1.10), also provide aseg and aseg_id arrays with source_xy, like:

source_xy = ra, dec, aseg, aseg_id

compute_auto_quantitiesbool

Compute Kron/auto-like quantities with compute_SEP_auto_params.

autoparamslist

Parameters of Kron/AUTO calculations with compute_SEP_auto_params.

flux_radiilist

Light fraction radii to compute with compute_SEP_auto_params, e.g., [0.5] will calculate the half-light radius (FLUX_RADIUS).

subpixint

Pixel oversampling.

mask_kronbool

Not used.

max_total_corrfloat

Not used.

detection_paramsdict

Parameters passed to sep.extract.

bkg_maskarray

Additional mask to apply to sep.Background calculation.

pixel_scalefloat

Not used.

logbool

Send log message to grizli.utils.LOGFILE.

gainfloat

Gain value passed to sep.sum_circle.

extract_pixstackint

See sep.set_extract_pixstack.

sub_object_limitint

See sep.set_sub_object_limit.

exposure_footprintslist, None

An optional list of objects that can be parsed with sregion.SRegion. If specified, add a column nexp to the catalog corresponding to the number of entries in the list that overlap with a particular source position.

seg_imagendarray, optional

A 2D array of the segmentation map. Each unique value in the array should correspond to the pixels associated with a specific object. Requires sep >= 1.3.0, or photutils. If not supplied, this will be generated instead using the SEP implementation of SourceExtractor.

in_dirstr or os.PathLike, optional

The directory containing the necessary input files (e.g. drizzled images). If not specified, files will be searched for in the current working directory.

out_dirstr or os.PathLike, optional

The directory to which all output will be written. If not specified, output files will be written to the current working directory.

seg_out_pathstr or os.PathLike, optional

The name or path to which the segmentation map will be saved.

detect_catTable, optional

If not None, this will be used to calculate Kron/auto-like quantities if compute_auto_quantities==True. This allows a different morphological catalogue to be used for detection and photometry.

use_photutilsbool, optional

If True, and seg_image!=None, measurements will be made using photutils.segmentation.SourceCatalog instead of SEP. By default False.

**kwargsdict, optional

Included in the original function, but seemingly not used.

Returns:

astropy.table.Table: Source catalog.

pygrife.grizli_functions.catalogue_fns.regen_multiband_catalogue(field_root='nis-wfss', threshold=1.8, detection_background=True, photometry_background=True, get_all_filters=False, filters=None, det_err_scale=-inf, phot_err_scale=-inf, rescale_weight=True, run_detection=True, detection_filter='ir', detection_root=None, output_root=None, use_psf_filter=True, detection_params={'clean': True, 'clean_param': 1, 'deblend_cont': 0.001, 'deblend_nthresh': 32, 'filter_kernel': array([[0.0049, 0.0213, 0.0513, 0.0687, 0.0513, 0.0213, 0.0049], [0.0213, 0.0921, 0.2211, 0.296 , 0.2211, 0.0921, 0.0213], [0.0513, 0.2211, 0.5307, 0.7105, 0.5307, 0.2211, 0.0513], [0.0687, 0.296 , 0.7105, 0.9511, 0.7105, 0.296 , 0.0687], [0.0513, 0.2211, 0.5307, 0.7105, 0.5307, 0.2211, 0.0513], [0.0213, 0.0921, 0.2211, 0.296 , 0.2211, 0.0921, 0.0213], [0.0049, 0.0213, 0.0513, 0.0687, 0.0513, 0.0213, 0.0049]]), 'filter_type': 'conv', 'minarea': 9}, phot_apertures=[<Quantity 0.36 arcsec>, <Quantity 0.500001 arcsec>, <Quantity 0.7000002 arcsec>, <Quantity 1.0000002 arcsec>, <Quantity 1.2 arcsec>, <Quantity 1.5 arcsec>, <Quantity 3. arcsec>], master_catalog=None, bkg_mask=None, bkg_params={'bh': 64, 'bw': 64, 'fh': 3, 'fw': 3, 'pixel_scale': 0.06}, use_bkg_err=False, aper_segmask=True, sci_image=None, clean_bkg=True, prefer_var_image=True, seg_image=None, in_dir=None, out_dir=None, seg_out_path=None, filt_auto_quantities=False, use_photutils=False)#

Generate a catalogue and run aperture photometry on all objects.

This function was originally taken from multiband_catalog, and has been modified in the following ways:

Add in_dir and out_dir parameters, so that the operations can take place using files from one directory, but writing the output to another.
Add seg_out_path parameter, so that the name of the segmentation map output can be specified.
Add seg_image parameter. Using a fork of SEP (sep >= 1.3.0, PJ-Watson/sep), passing an array here allows one to skip the object detection, and instead derive all the catalogue quantities for the specified objects.

Make a detection catalog and run aperture photometry on all available filter images with the SourceExtractor Python implementation sep.

Parameters:

field_rootstr: Rootname of detection images and individual filter images (and weights).
thresholdfloat: Detection threshold, see make_SEP_catalog.
detection_backgroundbool: Background subtraction on detection image, see get_background in make_SEP_catalog.
photometry_backgroundbool: Background subtraction when doing photometry on filter images, see get_background on make_SEP_catalog.
get_all_filtersbool: Find all filter images available for field_root.
filterslist, None: Explicit list of filters to include, rather than all available.
det_err_scalefloat: Uncertainty scaling for detection image, see err_scale in make_SEP_catalog.
phot_err_scalefloat: Uncertainty scaling for filter images, see err_scale on make_SEP_catalog.
rescale_weightbool: Rescale the weight images based on sep.Background.rms for both detection and filter images, see make_SEP_catalog.
run_detectionbool: Run the source detection. Can be False if the detection catalog file (master_catalog) and segmentation image ({field_root}-{detection_filter}_seg.fits) already exist, i.e., from a separate call to make_SEP_catalog.
detection_filterstr: Filter image to use for the source detection. The default "ir" is the product of make_filter_combinations. The detection image filename will be {field_root}-{detection_filter}_drz_sci.fits and with associated weight image {field_root}-{detection_filter}_drz_wht.fits.
detection_rootstr, None: Alternative rootname to use for the detection (and weight) image, i.e., {detection_root}_drz_sci.fits. Note that the _drz_sci.fits suffixes are currently required by make_SEP_catalog.
output_rootstr, None: Rootname of the output catalog file to use, if desired other than field_root.
use_psf_filterbool: For HST, try to use the PSF as the convolution filter for source detection.
detection_paramsdict: Source detection parameters, see make_SEP_catalog. Many of these are analogous to SourceExtractor parameters.
phot_apertureslist: Aperture diameters. If provided as a string, then apertures assumed to be in pixel units. Can also provide a list of elements with unit attributes, which are converted to pixels given the image WCS/pixel size. See make_SEP_catalog.
master_catalogstr, None: Filename of the detection catalog, if None then build as {field_root}-{detection_filter}.cat.fits.
bkg_masknumpy:array_like, None: Mask to use for the detection and photometry background determination, see make_SEP_catalog. This has to be the same dimensions as the images themselves.
bkg_paramsdict: Background parameters, analogous to SourceExtractor, see make_SEP_catalog.
use_bkg_errbool: Use the background rms array determined by sep for the uncertainties (see sep.Background.rms).
aper_segmaskbool: Use segmentation masking for the aperture photometry, see make_SEP_catalog.
sci_imagenumpy:array_like, None: Array itself to use for source detection, see make_SEP_catalog.
clean_bkgbool: If True, then the "*bkg.fits" files will be removed after the catalogue is created.
prefer_var_imagebool: If found, use _var.fits image for the full variance that includes the Poisson component.
seg_imagendarray, optional: A 2D array of the segmentation map. Each unique value in the array should correspond to the pixels associated with a specific object. If not supplied, this will be generated instead using the SEP implementation of SourceExtractor. Requires sep >= 1.3.0.
in_dirstr or os.PathLike, optional: The directory containing the necessary input files (e.g. drizzled images). If not specified, files will be searched for in the current working directory.
out_dirstr or os.PathLike, optional: The directory to which all output will be written. If not specified, output files will be written to the current working directory.
seg_out_pathstr or os.PathLike, optional: The name or path to which the segmentation map will be saved.
filt_auto_quantitiesbool, optional: Calculate Kron/auto-like quantities for each filter, in addition to those measured from the detection image. By default False.

Returns:

astropy.table.Table: Catalog with detection parameters and aperture photometry. This is essentially the same as the output for make_SEP_catalog but with separate photometry columns for each multi-wavelength filter image found.

pygrife.grizli_functions package#

pygrife.grizli_functions.FLT_fns module#

pygrife.grizli_functions.catalogue_fns module#

This Page