kids-wl / kcap Goto Github PK

KiDS cosmology analysis pipeline

CMake 0.18% Makefile 0.41% Shell 0.08% Fortran 28.76% Python 9.52% MATLAB 0.12% C++ 2.47% TeX 0.04% C 56.96% Jupyter Notebook 0.58% Cython 0.90%

kcap's People

Contributors

Stargazers

Watchers

Forkers

lshuns

kcap's Issues

Priors in As or Ln(As)

@tilmantroester has looked at sampling with different priors on A_s

Blue: fiducial KV450 chain
Green: sample Ln As on the range 1e-10 to 20e-9 (note the different range than what was used in KV450)
Red: sample As with 1/x prior
Purple: sample Ln As

Multinest sampler choice

I just read Handley and Lemos: https://arxiv.org/pdf/1902.04029.pdf

They argue that Multinest in general doesn't converge in a high-dimensional space? Should I be concerned as we were planning to use Multinest for K1000?

h-priors and the tendancy for a high H-0 without a prior

From George Efstathiou:
It's interesting that the lensing surveys are all consistent.
I am, however, puzzled by the tendencies for these surveys to give low Omega_m and high H_0. In Pablo's thesis, I asked him to run on old KiDS with very wide priors and he found a peak value of H_0 ~180 km/s/Mpc, with plausible values lying in the tails of the posterior. You can see a
similar thing in your Fig. B1, where the favoured value of h is at the upper boundary of your prior. This worries me.

From John P in response:
I find myself wondering what the effect would be if we set a prior using the measured value of om_m h^3, with no restrictions on om_m or h separately. I think it's agreed that this is one of the most robust numbers to emerge from the CMB, so it would be interesting to see what happens to the tension with Planck if this minimal CMB information is included. You might think that adding any CMB data would pull us closer together - but it may also sharpen our errors, so not clear what will happen to the statistical significance.

Might be worth looking at, at some point....

No clustering options enabled in scale_cuts.py

I have noticed that there is no way currently to use the clustering statistics in scale_cuts.py, even though the twopoint_wrapper.py and twopoint.py that are used allow for the w, Pnn and cosebis for it. I can go ahead and implement it, but I would like to know to which version/branch I can later submit my pull request with the new version (@tilmantroester)?

Add basic unit tests

We should have a testing suite that checks the output of the pipeline against other codes (e.g., CCL, Monte Python). I.e., have a code comparison built in.

error on local build

Hello,

I'm installing kcap on a local computer since i want to run some montepython MCMCs with the Cosebis likelihood. Im using ubuntu 20.04 LTS, within a conda environment with:

python 3
future
pyyaml
numpy
scipy
astropy
pip
emcee
mpi4py

i clone the repository and enter:

git clone https://github.com/KiDS-WL/kcap.git
cd kcap

then i try to build with:
python build.py

but the following error occurs:

Collecting cosmosis-standalone
  Cloning https://bitbucket.org/tilmantroester/cosmosis.git (to revision kcap) to /tmp/pip-install-td9yh5ot/cosmosis-standalone_65ac9e7a5e1b4a59bb5113adfb881c59
  Running command git clone --filter=blob:none --quiet https://bitbucket.org/tilmantroester/cosmosis.git /tmp/pip-install-td9yh5ot/cosmosis-standalone_65ac9e7a5e1b4a59bb5113adfb881c59
  warning: filtering not recognized by server, ignoring
  Running command git checkout -b kcap --track origin/kcap
  Switched to a new branch 'kcap'
  Branch 'kcap' set up to track remote branch 'kcap' from 'origin'.
  Resolved https://bitbucket.org/tilmantroester/cosmosis.git to commit bab22e07db58905a311397f1e69c347d8821c9c5
  Preparing metadata (setup.py) ... done
Requirement already satisfied: pyyaml in /home/hikuri/software/anaconda3/envs/class3-2/lib/python3.9/site-packages (from cosmosis-standalone) (6.0)
Requirement already satisfied: future in /home/hikuri/software/anaconda3/envs/class3-2/lib/python3.9/site-packages (from cosmosis-standalone) (0.18.3)
Requirement already satisfied: emcee in /home/hikuri/software/anaconda3/envs/class3-2/lib/python3.9/site-packages (from cosmosis-standalone) (3.1.4)
Requirement already satisfied: numpy in /home/hikuri/software/anaconda3/envs/class3-2/lib/python3.9/site-packages (from cosmosis-standalone) (1.21.5)
Requirement already satisfied: scipy in /home/hikuri/software/anaconda3/envs/class3-2/lib/python3.9/site-packages (from cosmosis-standalone) (1.10.1)
Building wheels for collected packages: cosmosis-standalone
  Building wheel for cosmosis-standalone (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [45 lines of output]
      running bdist_wheel
      running build
      /tmp/pip-install-td9yh5ot/cosmosis-standalone_65ac9e7a5e1b4a59bb5113adfb881c59/cosmosis
      make: Entering directory '/tmp/pip-install-td9yh5ot/cosmosis-standalone_65ac9e7a5e1b4a59bb5113adfb881c59/cosmosis'
      make[1]: Entering directory '/tmp/pip-install-td9yh5ot/cosmosis-standalone_65ac9e7a5e1b4a59bb5113adfb881c59/cosmosis/datablock'
      python generate_sections.py section_names.txt fortran cosmosis_f90/cosmosis_section_names.F90
      python generate_sections.py section_names.txt c section_names.h
      g++ -O3 -g -fPIC -I/tmp/pip-install-td9yh5ot/cosmosis-standalone_65ac9e7a5e1b4a59bb5113adfb881c59/cosmosis/.. -Wall -Wextra -pedantic -std=c++14   -c -o datablock.o datablock.cc
      make[1]: g++: Command not found
      make[1]: *** [<builtin>: datablock.o] Error 127
      make[1]: Leaving directory '/tmp/pip-install-td9yh5ot/cosmosis-standalone_65ac9e7a5e1b4a59bb5113adfb881c59/cosmosis/datablock'
      make: *** [/tmp/pip-install-td9yh5ot/cosmosis-standalone_65ac9e7a5e1b4a59bb5113adfb881c59/cosmosis/config/subdirs.mk:11: all] Error 2
      make: Leaving directory '/tmp/pip-install-td9yh5ot/cosmosis-standalone_65ac9e7a5e1b4a59bb5113adfb881c59/cosmosis'
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-td9yh5ot/cosmosis-standalone_65ac9e7a5e1b4a59bb5113adfb881c59/setup.py", line 142, in <module>
          setup(name = 'cosmosis-standalone',
        File "/home/hikuri/software/anaconda3/envs/class3-2/lib/python3.9/site-packages/setuptools/__init__.py", line 87, in setup
          return distutils.core.setup(**attrs)
        File "/home/hikuri/software/anaconda3/envs/class3-2/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
        File "/home/hikuri/software/anaconda3/envs/class3-2/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/home/hikuri/software/anaconda3/envs/class3-2/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 973, in run_commands
          self.run_command(cmd)
        File "/home/hikuri/software/anaconda3/envs/class3-2/lib/python3.9/site-packages/setuptools/dist.py", line 1217, in run_command
          super().run_command(command)
        File "/home/hikuri/software/anaconda3/envs/class3-2/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
          cmd_obj.run()
        File "/home/hikuri/software/anaconda3/envs/class3-2/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 299, in run
          self.run_command('build')
        File "/home/hikuri/software/anaconda3/envs/class3-2/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
          self.distribution.run_command(command)
        File "/home/hikuri/software/anaconda3/envs/class3-2/lib/python3.9/site-packages/setuptools/dist.py", line 1217, in run_command
          super().run_command(command)
        File "/home/hikuri/software/anaconda3/envs/class3-2/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
          cmd_obj.run()
        File "/tmp/pip-install-td9yh5ot/cosmosis-standalone_65ac9e7a5e1b4a59bb5113adfb881c59/setup.py", line 122, in run
          compile_library(env)
        File "/tmp/pip-install-td9yh5ot/cosmosis-standalone_65ac9e7a5e1b4a59bb5113adfb881c59/setup.py", line 82, in compile_library
          subprocess.check_call(["make"], env=env, cwd="cosmosis")
        File "/home/hikuri/software/anaconda3/envs/class3-2/lib/python3.9/subprocess.py", line 373, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['make']' returned non-zero exit status 2.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for cosmosis-standalone
  Running setup.py clean for cosmosis-standalone
Failed to build cosmosis-standalone
ERROR: Could not build wheels for cosmosis-standalone, which is required to install pyproject.toml-based projects
Traceback (most recent call last):
  File "/home/hikuri/physics/COSMO/kcap/build.py", line 15, in check_cosmosis
    import cosmosis
ModuleNotFoundError: No module named 'cosmosis'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/hikuri/physics/COSMO/kcap/build.py", line 102, in <module>
    build(mpi=not args.no_mpi)
  File "/home/hikuri/physics/COSMO/kcap/build.py", line 67, in build
    cosmosis_env = check_cosmosis(mpi)
  File "/home/hikuri/physics/COSMO/kcap/build.py", line 26, in check_cosmosis
    install_cosmosis(env)
  File "/home/hikuri/physics/COSMO/kcap/build.py", line 34, in install_cosmosis
    subprocess.check_call([sys.executable, "-m", "pip", "install", cosmosis_source], env=env)
  File "/home/hikuri/software/anaconda3/envs/class3-2/lib/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/home/hikuri/software/anaconda3/envs/class3-2/bin/python', '-m', 'pip', 'install', 'git+https://bitbucket.org/tilmantroester/cosmosis.git@kcap#egg=cosmosis-standalone']' returned non-zero exit status 1.

what could be happening? i'm new to this KCAP, thanks in advance for your comments.

Greetings

Write number of z-bins of NZ_SOURCE into Header of FITS package

Hi all,

it would be quite useful to also write the total number of z-bins of NZ_SOURCE into the header of that FITS BinTable extension.
That way the Monte Python likelihoods could read this number out automatically (feature exists already already in loading function of each likelihood) without requiring the user to set this explicitly in the accompanying '*.data'-file of each likelihood (as done now).

emcee sampler error

I get this error when running the emcee sampler: File "/home/ma/miniconda3/envs/kcap_env/lib/python3.8/site-packages/cosmosis/samplers/emcee/emcee_sampler.py", line 134, in execute
for (pos, prob, rstate, extra_info) in self.ensemble.sample(
TypeError: sample() got an unexpected keyword argument 'lnprob0'

Updating emcee didn't help. I saw that other had this problem with cosmosis in this thread:
https://bitbucket.org/joezuntz/cosmosis/issues/312/install-issues-camb_interface

Apparently it has to do with the version of emcee and the compilers. Works fine on my laptop.

ValueError in camb module from line 'M, m, v = camb.version.split(".")'

I get the following error when running the camb module in my pipeline:

File "/home/bengib/kcap_NewInst/kcap/./cosmosis-standard-library/boltzmann/pycamb/camb_interface.py", line 43, in setup
M, m, v = camb.version.split(".")
ValueError: too many values to unpack (expected 3)

This is because the CAMB version I have downloaded when installing kcap has version number 1.1.2.1 (4 numbers instead of 3).

I have got round this by commenting out that line in my camb_interface.py file.

Cross-repo issue reference test

cheymans/Phase1_KiDS1000#1

The installation instructions result in python3.12 and things don't work

Hi,

I tried following the install instructions helpfully included in your top-level README.md.
Currently, anaconda3 will install python 3.12.3, and it seems like some of the code is not happy with some of the changes in python3.12 ...

(kcap_env) dlang@mn003:~/kcap$ cosmosis runs/config/KV450_no_sys.ini
Traceback (most recent call last):
  File "/home/dlang/.conda/envs/kcap_env/bin/cosmosis", line 2, in <module>
    import cosmosis_main
  File "/home/dlang/.conda/envs/kcap_env/bin/cosmosis_main.py", line 5, in <module>
    from future import standard_library
  File "/home/dlang/.conda/envs/kcap_env/lib/python3.12/site-packages/future/standard_library/__init__.py", line 65, in <module>
    import imp
ModuleNotFoundError: No module named 'imp'

I understand that's not a problem in the kcap code itself, but it seems like maybe the future package is not as future proof as one might hope...

Maybe you want to pin the python version in the conda_env.yaml file?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.