Plot median of dark-corrected flats by wave region

[mgalloy]

Need a plot, by wave region, of the median of the dark-corrected flats. No masking, except the corners (use radius of 700 pixels).

[bberkeyU]

Pulling data from the ucomp_eng table in the database and looking at 1074 "flat_rcam_median_linecenter", flat_tcam_median_linecenter","flat_rcam_median_continuum","flat_tcam_median_continuum". These median values are computed from the interpolated dark corrected files applied to the science files. One data point for all 4 median flavors is produced from each L1 science file.

Plotting all the data, we get the following where each of the 4 flavors has an intensity offset but otherwise follows the same shape.

Drilling down to just the flat_rcam_median_linecenter data we find that there are at least 7 epochs in the ucomp flats. Here, labeled A-G. In the plot legend, I have included the DN value used to create the lower cut-off line.

Timestamps for the epochs are:
A: 2021-7-14 -> 2021-8-12 Start of mission
B: 2021-8-12 -> 2021-10-8 Lyot stop adjusted to 10.6mm
C: 2021-10-8 -> 2021-11-13 ??? No idea what is happening here
D: 2021-11-13 -> 2021-1-13 Lyot stop opened to 11.7mm
E: 2022-1-13 -> 2022-2-2 Temperature compensation turned off in the lyot filter. Field lens position check may have bumped lyot stop
F: 2022-2-2 -> 2022-6-17 Lyot stop opened up, a new field lens was installed, and cameras refocused. Most of this period had bad RCAM-TCAM radii #288
G: 2022-6-17-> 2022-12-01 Tcam Camera baffle/cable fix

[bberkeyU]

The continuum has a lower intensity than the onband because the continuum locations are probing the wings of the filter profile.

I used https://automeris.io/wpd/ to digitize the 1074 transmission profile from the Andover acceptable data.
1074-prefilter-transmision.csv

I overlaid the 1074 wave tunings on the filter transmission profile in this plot. The onband tunings are located at 1074.59, 1074.7, and 1074.81. The continuum tunings are centered +/- 1.256nm from the on-band tuning.

[bberkeyU]

Zooming in on the transition to the F region around Feb 02, 2022. We see the vertical line that drops from about 320 counts to 180 counts. All these points come from an interpolation between the flats: 20220202.205217.41.ucomp.1074.l0.fts and 20220202.222459.78.ucomp.1074.l0.fts. The first file looks normal and flat, but the second appears to be dark.

This changes the enhanced intensity images from the normal-looking file on 20220202.205352 and an overly bright one on 20220202.215930.

[detoma]

We need to use the profile in the code. We should ask the IG if they have the profile in a digitized form from Andover. We like it to be as close as possible to the real transmission. This must be a deconvolution of some kind that also takes into account the Lyot filter profile. Ideally we need the point spread function of the instrument so we can remove the instrumental effect. This usually is a slow step in the data calibration.

The pre-filter effect is not symmetric and seems larger than the continuum correction for solar and telluric lines that Steve applied to CoMP. I am surprised this never came up before. Steve must have assumed some instrumental profile to degrade the solar spectrum and. compute the continuum correction but I do not remember ever seen the Andover profiles. Mike and I should go over Steve's code for the continuum correction to see what he used there. He may have used an idealized flat response for the pre-filter.

[bberkeyU]

@detoma, I did a better capture; the update 1074-prefilter-transmision.csv file is probably as good as you can expect from a digitized scan. Which is why I removed references to the squiggles in the previous comment. I did make a couple of editorial decisions about the step sizes and the precision of the X and Y values; If these aren't the correct choices I would be happy to work with you to make other ones, but I think we are at a sub-pixel resolution so we probably won't see any more precise values.

The correction is pretty simple:

pro ucomp_l1_continuum_correction, file, $
                                   primary_header, $
                                   data, headers, $
                                   backgrounds, background_headers, $
                                   run=run, status=status
  compile_opt strictarr

  status = 0L

  ; TODO: implement
end```

[bberkeyU]

Assuming we are at not drift in the lyot filter and the commanded tuning matches the measured filter transmission values we should see:

On band (1074.7) 47.139% transmission
Blue wing (1073.44) : 29.57 % transmission
Red wing (1075.95): 38.49 % transmission
Combined continuum (29.57+38.49)/2 = 34.03 %

Would having the continuum transmission 3/4 the transmission of the onband explain the discrepancy in intensity between UCoMP and CoMP/DKIST/etc?

[detoma]

Ben, let's discuss today. The value of the transmission at one wavelength position is not enough to properly correct intensities. This is a convolution.

We do not know what the pre-filters looked like in COMP. They possibly were not flat either.

[bberkeyU]

From the IG filter inventory this is the plot of the CoMP 1074-70 filter which is what we used in the Mauna Loa UCoMP mission.

Note CoMP's lyot filter has a narrower free spectra range so the red/blue tunings are much closer to the on band when compared to UCoMP.

1074_CoMP-70.pdf