corgidrp.calibrate_nonlin#

Attributes#

nonlin_params_default

Exceptions#

CalNonlinException

Exception class for calibrate_nonlin.

Functions#

check_nonlin_params(nonlin_params)

Checks integrity of kgain parameters in the dictionary nonlin_params.

calibrate_nonlin(dataset_nl[, n_cal, n_mean, ...])

Function that derives the non-linearity calibration table for a set of DN

nonlin_kgain_dataset_2_stack(dataset[, apply_dq, ...])

Casts the CORGIDRP Dataset object for non-linearity calibration into a stack

Module Contents#

corgidrp.calibrate_nonlin.nonlin_params_default[source]#
corgidrp.calibrate_nonlin.check_nonlin_params(nonlin_params)[source]#

Checks integrity of kgain parameters in the dictionary nonlin_params.

Parameters:

nonlin_params (dict) – Dictionary of parameters used for calibrating nonlinearity.

exception corgidrp.calibrate_nonlin.CalNonlinException[source]#

Bases: Exception

Exception class for calibrate_nonlin.

corgidrp.calibrate_nonlin.calibrate_nonlin(dataset_nl, n_cal=20, n_mean=30, norm_val=2500, min_write=800.0, max_write=10000.0, lowess_frac=0.1, rms_low_limit=0.004, rms_upp_limit=0.2, pfit_upp_cutoff1=-2, pfit_upp_cutoff2=-3, pfit_low_cutoff1=2, pfit_low_cutoff2=1, make_plot=False, plot_outdir='figures', show_plot=False, verbose=False, nonlin_params=None, apply_dq=True)[source]#

Function that derives the non-linearity calibration table for a set of DN and EM values.

Parameters:

  • dataset_nl (corgidrp.Dataset) – The frames in the dataset are bias-subtracted. The dataset contains frames belonging to two different sets – Mean frame, a large array of unity gain frames, and set with non-unity gain frames. Mean frame – Unity gain frames with constant exposure time. These frames are used to create a mean pupil image. The mean frame is used to select pixels in each frame of the large array of unity gain frames (see next) to calculate its mean signal. In general, it is expected that at least 30 frames or more will be taken for this set. In TVAC, 30 frames, each with an exposure time of 5.0 sec were taken. Large array of unity gain frames: Set of unity gain frames with subsets of equal exposure times. Data for each subset should be taken sequentially: Each subset must have at least 5 frames. All frames for a subset are taken before moving to the next subset. Two of the subsets have the same (repeated) exposure time. These two subsets are not contiguous: The first subset is taken near the start of the data collection and the second one is taken at the end of the data collection (see TVAC example below). The mean signal of these two subsets is used to correct for illumination brightness/sensor sensitivity drifts for all the frames in the whole set, depending on when the frames were taken. There should be no other repeated exposure time among the subsets. In TVAC, a total of 110 frames were taken within this category. The 110 frames consisted of 22 subsets, each with 5 frames. All 5 frames had the same exposure time. The exposure times in TVAC in seconds were, each repeated 5 times to collect 5 frames in each subset – 0.077, 0.770, 1.538, 2.308, 3.077, 3.846, 4.615, 5.385, 6.154, 6.923, 7.692, 8.462, 9.231, 10.000, 11.538, 10.769, 12.308, 13.077, 13.846, 14.615, 15.385, and 1.538 (again). Set with non-unity gain frames: a set of subsets of frames. All frames in each subset have a unique, non-unity EM gain. For instance, in TVAC, 11 subsets were considered with EM values (EMGAIN_C): 1.65, 5.24, 8.60, 16.70, 27.50, 45.26, 87.50, 144.10, 237.26, 458.70 and 584.40. These correspond to a range of actual EM gains from about 2 to 7000. Each subset collects the same number of frames, which is at least 20 frames. In TVAC, each non-unity EM value had 22 frames. In each subset, there are two repeated exposure times: one near the start of the data collection and one at the very end. The exposure times of the frames in each EM subset do not need to be the same. For EM=1.65, the values of the exposure times in seconds were: 0.076, 0.758, 1.515, 2.273, 3.031, 3.789, 4.546, 5.304, 6.062, 6.820, 7.577, 8.335, 9.093, 9.851, 10.608, 11.366, 12.124, 12.881, 13.639, 14.397, 15.155, and 1.515 (repeated). And for EM=5.24, the 22 values of the exposure times in seconds were: 0.070, 0.704, 1.408, 2.112, 2.816, 3.520, 4.225, 4.929, 5.633, 6.337, 7.041, 7.745, 8.449, 9.153, 9.857, 10.561, 11.265, 11.969, 12.674, 13.378, 14.082, and 1.408 (repeated).

  • n_cal (int) – Minimum number of frames per sub-stack used to calibrate Non-Linearity. The default value is 20.

  • n_mean (int) – Minimum number of frames used to generate the mean frame. The default value is 30.

  • norm_val (int) – (Optional) Value in DN to normalize the nonlinearity values to. Must be greater than 0 and must be divisible by 20 without remainder. (1500 to 3000 recommended).

  • min_write (float) – (Optional) Minimum mean value in DN to output in nonlin. (800.0 recommended)

  • max_write (float) – (Optional) Maximum mean value in DN to output in nonlin. (10000.0 recommended)

  • lowess_frac (float) – (Optional) factor to use in lowess smoothing function, larger is smoother

  • rms_low_limit (float) – (Optional) rms relative error selection limits for linear fit. Lower limit.

  • rms_upp_limit (float) – (Optional) rms relative error selection limits for linear fit. Upper limit. rms_upp_limit must be greater than rms_low_limit.

  • pfit_upp_cutoff1 (int) – (Optional) polyfit upper cutoff. The following limits were determined with simulated frames. If rms_low_limit < rms_y_rel_err < rms_upp_limit, this is the upper value applied to select the data to be fitted.

  • pfit_upp_cutoff2 (int) – (Optional) polyfit upper cutoff. The following limits were determined with simulated frames. If rms_y_rel_err >= rms_upp_limit, this is the upper value applied to select the data to be fitted.

  • pfit_low_cutoff1 (int) – (Optional) polyfit upper cutoff. The following limits were determined with simulated frames. If rms_low_limit < rms_y_rel_err < rms_upp_limit, this is the lower value applied to select the data to be fitted.

  • pfit_low_cutoff2 (int) – (Optional) polyfit upper cutoff. The following limits were determined with simulated frames. If rms_y_rel_err >= rms_upp_limit, this is the lower value applied to select the data to be fitted.

  • make_plot (bool) – (Optional) generate and store plots. Default is True.

  • plot_outdir (str) – (Optional) Output directory to store figues. Default is ‘figures’. The default directory is not tracked by git.

  • show_plot (bool) – (Optional) display the plots. Default is False.

  • verbose (bool) – (Optional) display various diagnostic print messages. Default is False.

  • nonlin_params (dict) – (Optional) Dictionary of row and col specifications for the region of interest (indicated by ‘roi’) where the frame is illuminated and for two background regions (indicated by ‘back1’ and ‘back2’) where the frame is not illuminated. Must contain ‘rowroi1’,’rowroi2’,’colroi1’,’colroi2’,’rowback11’,’rowback12’, ‘rowback21’,’rowback22’,’colback11’,’colback12’,’colback21’,and ‘colback22’. The ‘roi’ needs one square region specified, and ‘back’ needs two square regions, where a ‘1’ ending indicates the smaller of two values, and a ‘2’ ending indicates the larger of two values. The coordinates of each square are specified by matching up as follows: (rowroi1, colroi1), (rowroi1, colroi2), (rowback11, colback11), (rowback11, colback12), etc. Defaults to nonlin_params_default specified in this file.

  • apply_dq (bool) – consider the dq mask (from cosmic ray detection) or not

Returns:

2-D array with nonlinearity values

for input signal level (DN) in rows and EM gain values in columns. The input signal in DN is the first column. Signal values start with min_write and run through max_write in steps of 20 DN.

Return type:

nonlin_arr (NonLinearityCalibration)

corgidrp.calibrate_nonlin.nonlin_kgain_dataset_2_stack(dataset, apply_dq=True, cal_type='nonlin', dataset_copy=True)[source]#

Casts the CORGIDRP Dataset object for non-linearity calibration into a stack of numpy arrays sharing the same commanded gain value. It also returns the list of unique EM values and set of exposure times used with each EM. Note: it also performs a set of tests about the integrity of the data type and values in the dataset.

Parameters:

  • dataset (corgidrp.Dataset) – Dataset with a set of of EXCAM illuminated

  • frames (pupil L1 SCI)

  • apply_dq (bool) – consider the dq mask (from cosmic ray detection) or not

  • cal_type (str) – If ‘kgain’, then sets of frames with the same exposure time for a given EM gain are truncated so that each has the same number of frames. Otherwise (for the ‘nonlin’ case), there is no truncation.

  • dataset_copy (bool) – flag indicating whether the input dataset will be preserved after this function is executed or not. If False, the output dataset will be the input dataset modified, and the input and output datasets will be identical. This is useful when handling a large dataset and when the input dataset is not needed afterwards. Defaults to True.

Returns:

list of data arrays associated with each frame (just the filepaths if data is None for input dataset) list of mean frames (just the filepaths if data is None for input dataset) array of exposure times associated with each frame array of datetimes associated with each frame list with the number of frames with same EM gain List of actual EM gains array of indices for timestamp ordering number of frames each set of frames with same exposure time truncated to for EM gain = 1 frames