gammapy / gamma-cat Goto Github PK

Source: https://github.com/gammapy/gamma-cat/blob/master/input/sources/tev-000137.yaml
Paper: http://adsabs.harvard.edu/abs/2016arXiv161005799S

Emailed corresponding author asking for SED points in Figure 2 just now.

We should implement Unified Content Descriptors (UCDs)
for at least some of our fields and add this to catalog formats that support it (e.g. VOTABLE, but also can be done in FITS).

E.g. this catalog
https://heasarc.gsfc.nasa.gov/W3Browse/all/hesscat.html
https://gist.github.com/cdeil/dd97ee29dc364012cf529cfb85012921
has such UCD values

      <FIELD name="name" datatype="char" arraysize="*"  ucd="meta.id;meta.main">
        <DESCRIPTION> Source Designation </DESCRIPTION>
      </FIELD>

      <FIELD name="ra" datatype="double" unit="degree" ucd="pos.eq.ra;meta.main">
        <DESCRIPTION> Right Ascension </DESCRIPTION>
      </FIELD>

and tools like Aladin or TOPCAT understand them.

VERITAS has published a lot of results (images, lightcurves, spectra) in FITS format:

http://veritas.sao.arizona.edu/veritas-science/veritas-results-mainmenu-72

We should add it here.

Ideally this should be done via Python scripts that download and re-format the data to our formats, so that it's reproducible. Or course contributions that manually add the data in our format would be welcome as well.

I noticed that after re-running ./make.py all the order of items in objects kept changing in docs/data/gammacat-sources.json, see e.g. this commit: 0858ad4

This is bad, because it makes it hard to find real changes in values in the git version history and to review those before committing updates.

I thought this was solved already by making sure OrderedDict is always used, see the gammacat.utils.load_json helper function (and I think json.dump in the gammacat.utils.write_json should preserve order).

My guess is that the issue was that we were creating dict and lose order on YAML read.
I hope this fixes the issue:
70ed20a

I'll keep this issue open as a reminder for a while.
Order in output files should be stable already.
If no problems appear any more this issue can be closed.

@GernotMaier suggested to look into the possibility to use SIMBAD or VIZIER from CDS instead of making a separate project.

E.g. some of the VERITAS data is found / available there:
http://simbad.u-strasbg.fr/simbad/sim-id?Ident=VER&NbIdent=cat&Radius=2&Radius.unit=arcmin&submit=submit+id

Most TeV data isn't available there though, and I'd be surprised if they want it all there.
It needs someone to collect, reformat and maintain it for every existing and future publication.

However, we should talk to them and at least interoperate as much as possible (e.g. on source identifiers) and document on our side what is available at CDS and how to find it.

I just saw this in

pos:
  ra: {val: 19h43m55s, err: 1s}
  dec: {val: +21d18m8s, err: 17s}

Note:

In [1]: from astropy.coordinates import Angle

In [2]: Angle('17s')
Out[2]: <Angle 17.0 arcsec>

In [3]: Angle('1s')
Out[3]: <Angle 1.0 arcsec>

So s is parsed as arcsec, i.e. correctly for errors given on dms.
But usually erorrs on hms are in "seconds of hourangle", which is 15 arcsec:

In [5]: Angle('0h0m1s')
Out[5]: <Angle 0.0002777777777777778 hourangle>

In [6]: Angle('0h0m1s').arcsec
Out[6]: 14.999999999999998

This can be reviewed and fixed in a systematic way by computing ellipticity of position errors.
Usually position errors are pretty circular.

In #2 @vorugantia lists a few sources that should be checked / added to gamma-cat.

This is a reminder issue for myself to do that.

• Any tevcat id beyond 257 did not have a tevcat2 id (the URL doesn't redirect). For these sources, I left the tevcat2_id field as null.
• tevcat id 129 is unpaired with any tgevcat source, but is SIMILAR to tevcat id 74 (which is paired).
• tevcat id 163 and 164 are unpaired, because tgevcat "combines" these two with tgevcat id 111.
• tevcat id 261 is unpaired. One of its alternative names does appear on tgevcat id 145, but this is paired with ANOTHER tevcat source (id 149).
• Same issue as above for both tevcat id 278 and 279 - they match up with tgevcat id 130, but that's already paired with tevcat id 143.
• Same issue again with tevcat id 253 - it matches with tgevcat id 37, but that's already paired with tevcat id 86.
• These are the other tevcat sources that simply do not have a tgevcat counterpart at all:
  • 88 - MilagroDiffuse
  • 174 - SNR G106.3+02.7
  • 263 - PKS 1441+25
  • 264 - S3 1227+25
  • 265 - S2 0109+22
  • 266 - PKS 1440-389
  • 267 - PKS 0625-35
  • 268 - HESS J1534-571
  • 269 - VER J1746-289
  • 270 - HESS J1813-126
  • 271 - HESS J1826-130
  • 272 - HESS J1828-099
  • 273 - HESS J1832-085
  • 274 - HESS J1844-030
  • 277 - HESS J1746-285
  • 280 - HESS J1852-000

Currently this is our SED spec:

We have ~ 20 spectra of type "PL2":
https://github.com/gammapy/gamma-cat/search?utf8=%E2%9C%93&q=pl2&type=Code

We're using the same parameter "norm" for differential and integral fluxes in the input files.
The documentation says this:

      norm:
        type: object
        additionalProperties: false
        unit: 1e-12 cm-2 s-1 TeV-1
        description: Flux norm parameter

So that's OK for "PL" differential spectra.

But for "PL2" integral spectral parameters, it's not documented what it should be.
@adonath told me it should be in 1e-12 cm-2 s-1.
But that's error prone on data entry if it's not even documented in the schema.

This is a reminder issue for me to fix this.
It'll require some thought / discussion and a week or two of work.
Assigning to me for now, but help welcome of course!

It will probably involve a large change of model input format for gamma-cat and also in Gammapy, e.g. adoption of the Fermi-LAT model XML serialisation format (or rather a YAML equivalent), or storing units in all the input files.

Since Astropy 1.3 one can pass strings to Quantity, so we could also change the pipeline to require units in the input files and parse those.

In #71 @pdeiml did data entry for an ECPL spectrum where only the integral flux is quoted (2006A%26A...448L..43A/tev-000037.yaml).

In these cases we have to compute a norm = amplitude parameter.
Here's how to do it with Gammapy:

# Code snippet to compute an ECPL spectrum norm from a given integral flux
# For https://github.com/gammapy/gamma-cat/pull/71/files
from copy import deepcopy
import astropy.units as u
from gammapy.spectrum.models import ExponentialCutoffPowerLaw
spec = ExponentialCutoffPowerLaw(
    index=1.45,
    amplitude=1 * u.Unit('cm-2 s-1 TeV-1'),
    reference=1 * u.TeV,
    lambda_=1 / (13.8 * u.TeV),
)

# This computes the `amplitude` parameter that corresponds to result in the integral flux given in the paper
paper_flux = u.Quantity(1.28e-11, 'cm-2 s-1')
int_flux = spec.integral(emin=1 * u.TeV, emax=10000 * u.TeV, ndecade=100)
amplitude = paper_flux / int_flux * spec.parameters.amplitude

# Let's check if, when using this amplitude, the integral flux of the source comes out right
spec2 = deepcopy(spec)
spec2.parameters.amplitude = amplitude
int_flux2 = spec2.integral(emin=1 * u.TeV, emax=10000 * u.TeV, ndecade=100)

print('amplitude = {:g}'.format(amplitude))
print('paper_flux = {:g}'.format(paper_flux))
print('int_flux = {:g}'.format(int_flux))
print('int_flux2 = {:g}'.format(int_flux2))
print('---')
print('spec: ', spec)
print('spec2: ', spec2)

Output:

amplitude = 1.02404e-11 1 / (cm2 s TeV)
paper_flux = 1.28e-11 1 / (cm2 s)
int_flux = 1.24995 1 / (cm2 s)
int_flux2 = 1.28e-11 1 / (cm2 s)
---
spec:  ExponentialCutoffPowerLaw
amplitude : 1 1 / (cm2 s TeV)
index : 1.45
lambda_ : 0.0725 1 / TeV
reference : 1 TeV
spec2:  ExponentialCutoffPowerLaw
amplitude : 1.02e-11 1 / (cm2 s TeV)
index : 1.45
lambda_ : 0.0725 1 / TeV
reference : 1 TeV

@pdeiml - For now, my suggestion would be that you do this computation by hand (or just use the value from here), and put a comment, something like

# Spectrum norm not given in the paper
# Computed from the given integral flux above 1 TeV of `1.28e-11 cm-2 s-1` with Gammapy

I've filed this new issue as a reminder that we need to add this as a utility function and document it, or do what we did for PL spectral shapes, to add more spectral models parameterised also by integral fluxes.

PS: for the stat error on the norm, I would suggest to use the approximation:

norm_err / norm = int_flux_err / int_flux

i.e. assume that the relative error is the same, and compute the norm error
from the given integral flux error.

This can be discussed / improved in the future, but for now, this info should let you finish up that PR.

We have these spectral points in input/papers/2006/2006ApJ...636..777A/tev-000095.ecsv

# - source_id: 95
# - source_name: CTB 37B / HESS J1713-381
# - paper_id: 2006ApJ...636..777A
# - url: https://www.mpi-hd.mpg.de/hfm/HESS/pages/publications/auxiliary/ApJ_636.html

There's also this paper where HESS put a new spectrum that goes to a bit higher energies:
http://adsabs.harvard.edu/abs/2008A%26A...486..829A

I couldn't find the SED info, probably we have to contact the corresponding authors.

This is a reminder issue, low priority.

I just saw that we have these two SEDs duplicated in the output:

$ find . -name '*%2526*'
./docs/data/sources/000094/gammacat_000094_2011A%2526A...528A.143H_sed.ecsv
./docs/data/sources/000118/gammacat_000118_2006A%2526A...460..365A_sed.ecsv

The correct ones are also present:

./docs/data/sources/000094/gammacat_000094_2011A%26A...528A.143H_sed.ecsv
./docs/data/sources/000118/gammacat_000118_2006A%26A...460..365A_sed.ecsv

This is a reminder issue for myself to

• remove the two incorrect SED files
• check the current scripts to make sure the incorrect files aren't re-generated
• maybe: implement a consistency check for files in the output folder that would find such incorrect old files.

I just found this:

• http://json-ld.org/
• https://en.wikipedia.org/wiki/JSON-LD

@GernotMaier @adonath @wegenmat @vorugantia - Should we try to use that to expose all the data from gamma-cat to users? Or do you have a better suggestion?

(there would still be a simple flat catalog, but that doesn't expose all the data we have)

Haven't looked at it much or tried it out ... starting now.

I just noticed that we had SED input/papers/2007/2007A%26A...472..489A/tev-000115.ecsv in the input folder, but it wasn't present in the output folder:

$ find . -name '*000115*'
./input/papers/2007/2007A%26A...472..489A/tev-000115.ecsv
./input/papers/2007/2007A%26A...472..489A/tev-000115.yaml
./input/sources/tev-000115.yaml

The reason was that source_id in the input file was incorrect.
Fixed in 201c198 .

This is a reminder issue for myself that I should implement a consistency check for the input header keys reference_id and source_id versus the folder and file names.

This is a reminder issue that we should add the data from the DESY light curve archive:
https://astro.desy.de/gamma_astronomy/magic/projects/light_curve_archive/index_eng.html

Paper: http://adsabs.harvard.edu/abs/2010A%26A...524A..48T
Contact: Elisa Bernardini

They define a "simple lightcurve format".
I think we should use something similar, but more consistent with the other formats from the gamma-astro-data-format specs.
Format discussions should go here: open-gamma-ray-astro/gamma-astro-data-formats#61 (comment)

Add data for HESS J1731-347, 2011A&A...531A..81H

Paper: http://adsabs.harvard.edu/abs/2011A%26A...531A..81H
Spectral points: https://www.mpi-hd.mpg.de/hfm/HESS/pages/publications/auxiliary/AA531_A81.html

Info for these HESS / HGPS sources is missing in gamma-cat:

• HESS J1018-589A
• HESS J1018-589B
• HESS J1554-550
• HESS J1641-462
• HESS J1800-240
• HESS J1930+188

@adonath - I won't get to this today. This is a reminder issue for myself to do it tomorrow.

This is a reminder issue to ingest the data from the HESS papers.

See https://github.com/gammapy/gamma-cat/blob/master/todo/todo_hess_aux.md

Whenever you add some data, please also update the status in that file and leave some notes,
so that we have a record.

I plan to move the README markdown documentation over to Sphinx soon.

This has many advantages, e.g.

• RST is more powerful markup language than Markdown (e.g. links)
• Can generate tables, plots of other things as part of the Sphinx build

Currently for the HGPS region, we're missing spatial model info for the following sources in gamma-cat:

WARNING:ctadc.sky_model:Missing spatial model: source_id=45, common_name=HESS J1018-589 A
WARNING:ctadc.sky_model:Missing spatial model: source_id=50, common_name=HESS J1119-614
WARNING:ctadc.sky_model:Missing spatial model: source_id=60, common_name=PSR B1259-63
WARNING:ctadc.sky_model:Missing spatial model: source_id=72, common_name=HESS J1457-593
WARNING:ctadc.sky_model:Missing spatial model: source_id=73, common_name=HESS J1458-608
WARNING:ctadc.sky_model:Missing spatial model: source_id=79, common_name=MSH 15-5-02
WARNING:ctadc.sky_model:Missing spatial model: source_id=158, common_name=SNR G323.7-01.0
WARNING:ctadc.sky_model:Missing spatial model: source_id=83, common_name=HESS J1614-518
WARNING:ctadc.sky_model:Missing spatial model: source_id=88, common_name=HESS J1640-465
WARNING:ctadc.sky_model:Missing spatial model: source_id=90, common_name=Westerlund 1
WARNING:ctadc.sky_model:Missing spatial model: source_id=97, common_name=CTB 37A
WARNING:ctadc.sky_model:Missing spatial model: source_id=104, common_name=HESS J1741-302
WARNING:ctadc.sky_model:Missing spatial model: source_id=107, common_name=Galactic Centre ridge
WARNING:ctadc.sky_model:Missing spatial model: source_id=159, common_name=Arc source
WARNING:ctadc.sky_model:Missing spatial model: source_id=109, common_name=Terzan 5
WARNING:ctadc.sky_model:Missing spatial model: source_id=111, common_name=HESS J1800-240A
WARNING:ctadc.sky_model:Missing spatial model: source_id=114, common_name=HESS J1808-204
WARNING:ctadc.sky_model:Missing spatial model: source_id=160, common_name=HESS J1813-126
WARNING:ctadc.sky_model:Missing spatial model: source_id=117, common_name=SNR G15.4+0.1
WARNING:ctadc.sky_model:Missing spatial model: source_id=161, common_name=HESS J1826-130
WARNING:ctadc.sky_model:Missing spatial model: source_id=119, common_name=LS 5039
WARNING:ctadc.sky_model:Missing spatial model: source_id=162, common_name=HESS J1828-099
WARNING:ctadc.sky_model:Missing spatial model: source_id=120, common_name=HESS J1831-098
WARNING:ctadc.sky_model:Missing spatial model: source_id=163, common_name=HESS J1832-085
WARNING:ctadc.sky_model:Missing spatial model: source_id=121, common_name=HESS J1832-093
WARNING:ctadc.sky_model:Missing spatial model: source_id=122, common_name=HESS J1833-105
WARNING:ctadc.sky_model:Missing spatial model: source_id=126, common_name=HESS J1843-033
WARNING:ctadc.sky_model:Missing spatial model: source_id=164, common_name=HESS J1844-030
WARNING:ctadc.sky_model:Missing spatial model: source_id=127, common_name=HESS J1846-029
WARNING:ctadc.sky_model:Missing spatial model: source_id=129, common_name=HESS J1849-000
WARNING:ctadc.sky_model:Missing spatial model: source_id=157, common_name=HESS J1852-000
WARNING:ctadc.sky_model:Missing spatial model: source_id=132, common_name=MGRO J1908+06
WARNING:ctadc.sky_model:Missing spatial model: source_id=134, common_name=HESS J1912+101
WARNING:ctadc.sky_model:Missing spatial model: source_id=135, common_name=W 51C
WARNING:ctadc.sky_model:Missing spatial model: source_id=141, common_name=VER J2016+371
WARNING:ctadc.sky_model:Missing spatial model: source_id=143, common_name=VER J2019+368

This list is printed by running ./make.py skymodels images for the ctadc script.
No need to keep track of data entry here, we can just re-run the script and add missing stuff until done.

cc @adonath

In #36 (comment) and #9 (comment) @wegenmat @Konstancja and I started to discuss how to input lightcurves that don't have an ADS bibcode, i.e. nopaper_id into gamma-cat.

My proposal would be that we put data_id: some_string for those, and then add a file or folder that looks something like this:

# Internal gamma-cat data references

- data_id: some_string
   url: if available
   comments: received via email from XYZ on ABC or whatever
- data_id: some_other_string
   comments: found it on my backup disk. origin unknown. Fits my theory, so I'm keeping it!

The data_id should be short strings that can be used in filenames, just like the paper_ids
I could implement this scheme and document it in the coming days.

You could wait for that, or start putting data_id keys in the LC files now, and then add the available info / comments to the data_references.yaml file later.

@wegenmat @Konstancja @cboisson - Thoughts?

We need data entry for this paper on PKS 2155-304 and PG 1553+113
http://adsabs.harvard.edu/abs/2016arXiv161201843H

Spectral point data is available here:
https://www.mpi-hd.mpg.de/hfm/HESS/pages/publications/auxiliary/PKS2155_HESSII_auxinfo.html

I'm starting to work on the gamma-cat webpage, splitting it up into a landing page and one that lists all sources and all papers we have.
Possibly there should be other pages, I'm not sure yet.

• https://gammapy.github.io/gamma-cat/
• https://github.com/gammapy/gamma-cat/tree/master/docs

The point is, we have multiple pages, and it would be nice to have a navbar.

I see the following options to set this up:

1. Duplicate HTML code across the pages (for the navbar and footer)
2. Use HTML imports (see here and here to make it work cross-browser
3. Use Python scripts and Jinja as a static website generator that fills in page-specific info into HTML page templates.

For now I'll start with 1, but I think it's worth considering option 2 and 3.

@vorugantia - When you get a chance, have a look and let me know what you think.

The main goal IMO is that those pages make it easy for us to search and navigate to any input file, and discover and download all files in output/data.
The nice HTML view of the data for end-users can then be on gamma-sky.net.

Reminder issue:
Lightcurve data from this new HESS paper on PKS 2155 should be added
http://adsabs.harvard.edu/abs/2016arXiv161003311H

I've contacted Jill Chevalier just now asking if the LC data is already available publicly in a machine-readable format. Probably not and this will have to wait a few weeks or months.

Data entry (YAML and ECSV for SED) needed for source_id=161 and this paper:
http://adsabs.harvard.edu/abs/2017arXiv170107002A

Spectral points, obtained from the corresponding author Ekrem Oguzhan Anguner on Feb 14, 2017 are here: https://gist.github.com/cdeil/ae625a52c9a87135a58821e3cb2747c3

I realised that quite a few FITS images have been released by IACTs for publications.

I'm not sure if we should

• add download links to them here
• add copies of them in this repo
• collect them in a separate repo

I'm also thinking maybe we should rename this repo / project to gamma-data or something more general if we do add images, lightcurves, flux points.

Currently there's a bug in the SED selection. When the output table is created the selection of the SED, is done with the method SEDList.get_sed_by_source_id(), which doesn't consider, from which paper the SED comes. Right now it returns the last SED, that is added to the SED list. This behaviour should be fixed.

Most astronomy papers first appear on ArXiv.
Example: http://arxiv.org/abs/1503.02711

They are then picked up by ADS the next day and the link from ArXiv to ADS looks like this and works forever.
Example: http://adsabs.harvard.edu/cgi-bin/bib_query?arXiv:1503.02711

But eventually (often a year later) the paper gets published and gets a proper bibcode that's independent of ArXiv.
Example: https://ui.adsabs.harvard.edu/#abs/2015A%26A...577A.131H

Currently we are using ADS bibcodes heavily, especially as directory names to collect the paper data in input/papers. This is documented a bit here: https://github.com/gammapy/gamma-cat/blob/master/README_DETAILS.md#paper-identifiers

This is an open question how we should handle this?

1. Start with the ADS ArXiv bibcode and stick with that in input?
2. Start with the ADS ArXiv bibcode and change that once a publication bibcode becomes available?
3. Introduce our own paper IDs (e.g. integers like we do for sources)?
4. Any other options?

In 5d7d107 I added the "arc source" as one source in this TeV catalog.
It's listed as two sources in TeVCat, which isn't supported by our current fields.

tevcat_id: 277 # , 269
tevcat2_id: LsZtAg # , 8YV5U0
tevcat_name: TeV J1746-289 #, TeV J1746-285

How should we handle such cases?
Should we make those fields lists to allow one to many relationships?

Even if we do, I think we might want to add a "finder chart" and "cone search" functionality to list all "nearby" sources from other catalogs of interest.

We should write a script to check gamma-cat for completeness and agreement of values against the HESS source catalog (see https://github.com/gammapy/gamma-cat/blob/master/README_ABOUT.md#related-resources for links).

This is a low priority reminder issue, I'm putting it under the 1.0 milestone.

Measurements from this paper should be added (spectral points already there):

• HESS J1641-463
• http://adsabs.harvard.edu/abs/2014ApJ...794L...1A
• https://github.com/gammapy/gamma-cat/blob/master/input/sources/tev-000089.yaml
• https://github.com/gammapy/gamma-cat/tree/master/input/papers/2014/2014ApJ...794L...1A

Add data:

• Source: HESS J1554-550 = SNR G327.1-1.1

• https://github.com/gammapy/gamma-cat/blob/master/input/sources/tev-000081.yaml

• http://adsabs.harvard.edu/abs/2011ICRC....7..185A

• no other separate HESS publication, will be in HGPS

• https://github.com/gammapy/gamma-cat/tree/master/input/papers/2011/2011ICRC....7..185A

• YAML file added

• Emailed Fabio Acero asking for spectral points just now.

One comment by @GernotMaier in a f2f discussion with @Konstancja @wegenmat and me was that

• gamma-cat should have the stated goal to be a complete archive of VHE data (even if we don't have the manpower to do this short-term).
• We should compute and show completeness of data availability in several ways:
  • for each paper
  • for each source
  • overall and for source classes

I plan to implement the suggestions here in the coming weeks, and then ping you all for review / feedback.

Paper: http://adsabs.harvard.edu/abs/2005A%26A...432L..25A
Spectral points: https://www.mpi-hd.mpg.de/hfm/HESS/pages/publications/auxiliary/AA432_L25.html

At the CTA consortium meeting last week I talked with Andrea Giuliani and he now mentioned
http://www.iasf-milano.inaf.it/~giuliani/astrisim/gsed/
to me via email.

From a brief look, it seems very similar to what we want to do with gamma-cat, only smaller in scope.
I didn't look yet what data they have that we haven't collected yet.
We should collaborate.
Discussing via email for now ...

SED is missing for

• Paper: https://ui.adsabs.harvard.edu/#abs/2011A%26A...529A..49H
• Data: https://github.com/gammapy/gamma-cat/tree/master/input/data/2011/2011A%2526A...529A..49H
• Source: HESS J1943+213
• source_id: 137
• Figure: http://www.aanda.org/articles/aa/full_html/2011/05/aa16545-11/F3.html

Written email to corresponding author Anna Szostek just now.
She left science a few years ago.
Might have to use a datapicker to get those.

Add data: VERITAS 2010 G54.1+0.3 2010ApJ...719L..69A
aka HESS J1930+188

Source: https://github.com/gammapy/gamma-cat/blob/master/input/sources/tev-000136.yaml
Paper on ADS: http://adsabs.harvard.edu/abs/2010ApJ...719L..69A
Paper data: https://github.com/gammapy/gamma-cat/tree/master/input/papers/2010/2010ApJ...719L..69A

YAML file is in gamma-cat.

Spectral points aren't available here: http://veritas.sao.arizona.edu/veritas-science/veritas-results-mainmenu-72/235-could-we-put-in-a-journal-reference-here-
Emailed Ester Aliu just now asking for spectral points.

MAGIC has published data in machine-readable format here:
http://vobs.magic.pic.es/fits/

We should ingest it in gamma-cat to have a complete catalog of TeV data in uniform format.

Help welcome!

I just entered an ECPL spectrum:
749958e#diff-501fd4323310af93fb916ffb0b91d117R22

The spectral model parametrisation and parameters are here:
http://www.aanda.org/articles/aa/full/2006/47/aa5546-06/aa5546-06.html

I think the norm might be processed incorrectly, since currently the catalog has:

spec_norm_1TeV: 2.0170070474945234e-11

but the differential flux at 1 TeV should be lower, the difference being the exp factor.

What we need is

• Start by checking this one source
• Check all ECPL spectra in gamma-cat
• Define a better spec result input format, or at least clearly document how the norm is processed for ECPL.

Yesterday @cboisson pointed http://hess.obspm.fr/ out to me.
It's a collection of published spectra and lightcurves for blazars observed by HESS.
For now I've added it to our list of related resources

It seems that some data is in text format and some in VOTable format:

• http://hess.obspm.fr/data/0032774001473402569_hess.txt
• http://hess.obspm.fr/data/0743797001473402971__1_hess.vot

It could be interesting for this project both at

• a resource of data we can ingest
• an example of data formats, especially the VOTable file. (I think the text file is a custom format for which we'd need to write a parser, i.e. we shouldn't use it).

@cboisson - Is that database at http://hess.obspm.fr/ still actively maintained?
Is is possible to get a dump of the available data there without having to click 100 times?

Suggestion by @pdeiml - At the moment we sometimes put source_name in input files, and sometimes not. The idea is that only source_id is used, and than source_name or common_name is only added to help with not getting confused about which file is for which source.

But putting names is problematic, it leads to inconsistencies between names used.
Options are:

a. Remove all source_name or common_name entries in input files.
b. Keep common_name, but document that it's optional and for convenience.

I'd like to sleep on it, but at the moment I'm leaning towards option a.

I think currently the SED info in gamma-cat is filled from the input SED folder.

It would be better to fill it from the docs/data output folder, to make sure it's clean and consistent with the separate SEDs we give to users.

@adonath - Can you do the change?
(if no, assign back to me and I'll do it next week)

This source should be added as a new source in input/sources:
http://tevcat.uchicago.edu/?mode=1&showsrc=182

It's not a clear detection, but still worth adding I think.

It goes by several names:

• Boomerang" source
• TeV J2228+611
• MGRO C4
• MGRO J2228+61

Currently we only have an integer source ID for the Open TeV catalog.

E.g. https://github.com/gammapy/gamma-cat/blob/master/input/sources/tev-000001.yaml has

source_id: 1

Other TeV catalogs have an ID, and a position-based identifier:

tevcat_id: 227
tevcat_name: TeV J0006+729

tgevcat_id: 1
tgevcat_name: TeV J0006+7259

We could also introduce such a position-based identifier and prefix it with TeV or VHE ("very high energy") or OTC ("Open TeV catalog").

• The pro is that it's useful to have such an identifier when working with sources and referring to them.
• The con is that it adds confusion for users (besides being sometimes helpful), e.g. the tevcat_name and tgevcat_name are often different and ours would be a third variation.
• In either case there's the problem that eventually sources break up in to multiple sources upon deeper observation (e.g. with CTA) and this is difficult to handle in such a "live" catalog.

@vorugantia @registerrier @cboisson @adonath - Thoughts?

This is a reminder issue, we don't have to decide this now. For now we can use the (unique in our catalog) field common_name when displaying info for a source.

This is a reminder issue that we should add data from:

"VERITAS Observations of The Galactic Center Ridge"
http://adsabs.harvard.edu/abs/2015arXiv150806311S

Following #63 (comment), I think we should support Python 3.4 with the scripts.

This is a reminder issue for myself to:

• Change the scripts to make them work on Python 3.4
• Add a Python 3.4 travis-ci build to make sure it stays that way

Make an IPython notebook that shows how to get the following data for every TeV source

• Spectral points
• How to dump spectral point data to text files
• Spectral global model (power-law or exp cutoff PL)
• Integral flux above 20 TeV computed from the spectral model

Proceeding: http://adsabs.harvard.edu/abs/2011ICRC....7..244S
No paper exists on this source.
The proceeding contains spectral points in Figure 2.
Emailed Arache and Henning Nov 10, 2016 asking for the points.

Folder already exists: https://github.com/gammapy/gamma-cat/tree/master/input/papers/2015/2015A%2526A...574A.100H
Paper: http://adsabs.harvard.edu/abs/2015A%26A...574A.100H
Spectral points: https://www.mpi-hd.mpg.de/hfm/HESS/pages/publications/auxiliary/auxinfo_SNRG349

This is probably more a list of question from a new user than an issue.

Goal is to add all the data for HESS J0632+057 (for now mainly from the paper The Astrophysical Journal, 780:168 (14pp), 2014 January 10; http://adsabs.harvard.edu/abs/2014ApJ...780..168A)

Lightcurve (table 1)

following schemas/lightcurve-format.md , the fluxes and errors should be given in Crab units? Is this really a good choice? Which Crab spectrum should be assumed here? Wouldn't it be better to use usual units (photons/cm^2/s)?

Do I understand it correctly that upper flux limits are indicated by a '-1' for the flux error? Is there a possibility to give both flux+fluxerror and upper flux limit?

Spectral results

I have several spectral results for different periods, how can I add this? These are spectra for different orbits and different orbital phases. I am not sure how to indicate this in the data.

Same thing for the spectral points (SEDs), which I have for different orbital periods.

Time periods vs orbital periods

For binaries, we often combine data from several orbits and e.g. calculate average flux points for a given orbital period (e.g. the average flux at apastron, taken over multiple orbits). I guess the light curve format would have to be expanded to take this into account.

Paper: https://arxiv.org/abs/1609.08671
Flux points (listed at the end of the paper): https://gist.github.com/cdeil/412bf8c648f364e47e71d2cc2651bf5c