dN/dmag test about descqa HOT 28 CLOSED

I think I see the problem in the notebook. YOu have a variable representing the full area, but you seem to have assigned its value based on the area of just 1 region:

full_area_sq_arcmin = float(len(hsc_randoms[ran_field_mask_1]))/100.0
full_area_sq_deg = full_area_sq_arcmin/3600.0

So your "full area" is too small by a factor of ~5. You later use this area to normalize densities for the whole catalog.

from descqa.

replace "spring week 2018" with "collaboration meeting 2018"!

better, but not exactly good...

from descqa.

I am going to start working on this.

from descqa.

Here's a preliminary result from this test, made by @duncandc: (and the corresponding entry on DESCQA web interface)

Lines are protoDC2 and buzzard_test, points are HSC. Difference in normalization may be due to a bug (still be explored).

See also LSSTDESC/DC2-production#21 for related comments.

from descqa.

@yymao @duncandc - There has been further discussion of this test in #50 (see specifically from #50 (comment) onwards). A few outstanding questions:

• as discussed there, we'd like to have some way of disabling this test for catalogs that have redshift cuts such that the N(<mag) calculation is significantly affected. Can this be implemented based on the maximum redshift of the light cone?
• the last status of this issue was that there were bugs to work out. Is this still in progress?
• validation criterion: we had discussed this on LSSTDESC/DC2-production#21 (see more specifically LSSTDESC/DC2-production#21 (comment) ) and the proposal I would like to make is that at limiting magnitudes of 23, 24, and 25 in i-band, we should require 20% agreement in N(<mag). At limiting magnitudes of 26 and 27, we should require 40% agreement. Pinging a few people who may have comments on this: @janewman-pitt-edu , @msimet .

from descqa.

@rmandelb To answer your questions ---

• Yes, that can be easily implemented.
• We have not made much progress since the Sprint Week (but @duncandc can correct me if that's not the case). I am not sure if there was really a bug in getting dN/dmag from the mocks, or just that the way the validation data is constructed is not consistent with how we do it with the mock.
• Those criteria can be implemented. As mentioned above, we need to make sure the validation data is correctly/consistently constructed.

from descqa.

The criteria Rachel suggests sounds reasonable to me.

from descqa.

I am testing my calculation of dn/dmag. Here is a comparison between my calculation and the result in the HSC data release paper. This suggests that there is a problem with the test. But for the life of me I can not find the problem. Particularly confusing is the offset in the peak where incompleteness sets in. the magnitudes are offset by ~1.

[compare lime green line and points]

from descqa.

@duncandc is there any chance that there's a missing 5*log(h) ~= -0.77?

from descqa.

apparent magnitudes don't scale with h, right?

from descqa.

no, i don't think they do

from descqa.

everyone should feel free to take a look at the notebook in the desqagen branch of my fork. https://github.com/duncandc/descqa/tree/descqagen/descqagen/app_mag_func_test

from descqa.

Apparent magnitudes have a contribution from the luminosity distance 5*np.log10(dl)+25 which assumes dl is in Mpc. But the catalog providers take care of this. What about magnitude definitions? Catalog fluxes are converted to magnitudes in the AB system. Does this match the definition in Hiroki et al?

from descqa.

@evevkovacs the most recent plot @duncandc showed is comparing the HSC data with HSC paper plot. So there seems to be something going on in Duncan's data reduction but we haven't figured out what it is.

It does seem that the dN/dmag in the HSC paper is closer to the catalogs'

from descqa.

@rmandelb, that mostly does the trick.

from descqa.

We are now about 20% lower than the data release paper. It would make me nervous if this is in the realm of details of magnitude types and flags used to mask the data...

from descqa.

Is this protoDC2 or Buzzard that is below the data release paper?

from descqa.

To be clear, the reason i ask is that we know about a few effects:

• in both sims, at low redshift, there are missing galaxies beyond some luminosity limit (and I don't remember what magnitude that would affect, but I believe it was @aphearin who showed some relevant plots on this)
• in protoDC2 (but not Buzzard) we are missing everything about z=1

Both of these effects have the right sign at least, but the severity and magnitude-dependence differs for the two effects.

from descqa.

Although, it just occurred to me that I got these points from plot digitizer... so I don't know how much trust we put in my steady hand...

from descqa.

This is a comparison between my measurement of dn/dmag and the data presented in figure 16 in Aihara+ 2017 (https://arxiv.org/abs/1702.08449).

from descqa.

Well, I would be comfortable at this point going back to your dn/dmag estimate from HSC vs. the sims...

from descqa.

Is the legend really correct? I believe the number counts in HSC should really flatten out beyond i=26, but the green points labeled HSC keep rising all the way to i=30.

from descqa.

@rmandelb I believe that is @duncandc's fit.

from descqa.

That is correct @yymao and @rmandelb. I have fit a power law to to extrapolate HSC. That was part fo the original test proposed at the sprint week.

from descqa.

OK. Well, I think the suppression at protoDC2 beyond i=23 has to do with the z=1 limit, which is particularly important there. I had made a suggestion somewhere for how to account for this (basically taking the DEEP2 dN/dz for mag-limited samples, and using that to "correct" for missing galaxies), but perhaps for now we should do comparisons with Buzzard?

Also, can you by default have a separate panel for the ratio? Log plots covering many orders of magnitude can be hard to interpret.

from descqa.

We could certainly look at dN(<m)/dmag and correct using the DEEP2 dN/dz estimates... then differentiate to get dN/dmag... it's better than nothing, at least.

from descqa.

@duncandc now that you have fixed the normalization issue, can you update PR #46 and #47 (together with Rachel's suggestion of adding a ratio panel in #47)? Once you update the PRs, I'll review and merge (if there's no further issues) them.

Also, do you have access to the DEEP2 validation data?

from descqa.

Done in #47.

from descqa.