Linearity Correction

Using the notes from Bill Vaca, I re-examined correcting for the non-linearity in the detector.  The results show that this method corrects for the non-linearity quite well.  Below is an explanation of the processes used for the correction.

As discussed in the email from Bill Vaca, we want the counts to follow a linear path with time:
Dn = c*t, where c is a constant
But what we see for NIR detectors is something more like:
DN = a0 + a1*t + a2*t^2 + ...

We would like to have a correction for non-linearity that is independent of time, so instead of looking at counts, we will look at the rate, r (Dn/t) versus DN.
So again, we want:
r_ideal = Dn/t = c
But we observe:
r_obs = b0 + b1*Dn + b2*DN^2 + ...

To obtain a correction factor for EACH pixel we can use:
DN_corr = r_ideal/r_obs = 1/(b0/c + (b1*DN)/c + (b2*DN^2)/c + ...)

Next you must figure out what to use for the constant, c.  One choice would be b0, but as Bill pointed out, at small DN, the observed rate can actually drop.  I tried to use b0 and found that sometimes it caused the correction to fail (i.e., results would not be linear).  So I tried Bill's other suggestion, where you find the maximum value for the rate, r_max, as determined by the polynomial, r_obs.  This approach appears to be much more robust and is what I have adopted in my code.

Now the correction value is computed at each pixel in terms of DN as:
DN_corr  =  1/(b0/r_max + (b1*DN)/r_max + (b2*DN^2)/r_max)


I have adopted using a second order polynomial for several reasons.  1) Using a 4th order would sometimes cause pixels to be over-corrected and 2) the fits for images with only 4 reads (such as 100 1 egos) would often fail when a 4th order polynomial was used.  I believe that for images with more that 4 reads, a 4th order typical gives a better fit than a 2nd order (but not by much, this is something I should quantity), but for now I am using a 2nd order polynomial for all images.  The polynomial is only fit to pixels below saturation.  Right now that value is hard coded in, but could easily become a variable that changes with bias, etc...

Below is a plot that shows the correction for a 5X5 pixel area in a 25 1 egos image (N2 S0 N3 S0 N3 S0 N1 D0).  I have also examined 100 1 egos images and the correction still works very well.

There are still some issues dealing with "bad" pixels or pixels that have negligible flux levels.  I either try to ignore these pixels, since I will not obtain a meaningful fit, or they are fit, but typically are over corrected.  Since these are the pixels where we will not be using to make measurements, I do not see it as too much of a problem, but will try to come up with a more robust way of dealing with these "bad" pixels at a later time.  Any suggestions are welcomed.