Giter VIP home page Giter VIP logo

pspipe's Introduction

https://travis-ci.com/simonsobs/PSpipe.svg?branch=master

PSpipe is a pipeline creator for the analysis of the high resolution maps of the large aperture telescope of the Simons Observatory. It contains tools for estimating power spectra and a multi-frequency likelihood interfaced with the cobaya MCMC sampler.

The pipelines are mainly written in python and make use of three different codes,

The package is licensed under the BSD license.

  • Python >= 3.8
  • FFTW: version 3 required

If the previous requirements are fulfilled, you can install the PSpipe package with its dependencies by doing

$ pip install --user git+https://github.com/simonsobs/PSpipe.git

If you plan to develop or want to use the different projects, it is better to checkout the latest version by doing

$ git clone https://github.com/simonsobs/PSpipe.git /where/to/clone

Then you can install the PSpipe library and its dependencies via

$ pip install --user /where/to/clone

Given the number of requirements, you can use a docker image already made with the needed libraries and everything compiled. You should first install docker for your operating system.

We have written a simple bash script to install the PSpipe docker and to clone the main PSpipe libraries. Just copy the script in a directory where you want to work with pspipe and run

$ ./run_docker.sh

This will open a new bash terminal with a full installation of PSpipe, pixell, NaMaster, pspy... For instance, you can start the ipython interpreter and run the following import command

$ ipython
Python 3.6.9 (default, Nov  7 2019, 10:44:02)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.11.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import pixell, pymaster, pspy

You can run the python scripts from the tutorials directory of PSpipe.

When you are done with the image, just type exit and you will go back to your local machine prompt.

It is also possible to start a jupyter server from the PSpipe image and run it into your web browser. Inside the image terminal, you have to start the jupyter server by typing

$ jupyter notebook --ip 0.0.0.0

Finally open the http link (something like http://127.0.0.1:8888/?token...) within your web browser and you should be able to run one of the python notebook.

Everything perfomed within the /home/pspipe/workspace directory will be reflected into the /where/to/work_with_pspipe on your host machine. You can then share configuration files, source codes, data files... between the running docker container and your local machine. Nothing will be lost after you exit from the docker container.

Docker for Mac limits the resource available to 2Gb of RAM by default, This might cause the code to crash unexpectedly with a cryptic Killed message. It can easily be modified, click on the docker logo (top right of your screen), go in Preferences/Resources and increase the RAM allocated to Docker.

You are not ready for it: youtube

pspipe's People

Contributors

adrien-laposta avatar damonge avatar dependabot[bot] avatar dwhan89 avatar earosenberg avatar sgiardie avatar stevekchoi avatar thibautlouis avatar xgarrido avatar xzackli avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pspipe's Issues

Lensed simulations with foreground for the LAT

For testing parameter recovery and the analytical covariance matrix computation, we need some semi-realistic lensed CMB + foreground maps.
We would need both galactic and extra galactic foreground in healpix and CAR pixellisation (we can project from one to another using the so map class).
We can probably start with simple gaussian extra galactic foregrounds and then iterate.

Covariance matrix calculation

Write a bunch of function that create the spectra covariance matrix. Both analytically and using monte carlo.
A place holder can be found in pspy/so_cov.py

python-3 issues

In order to install pspy and make it work with python3 I had to:

  1. change ConfigParser to configparser in so_config.py
  2. put parentheses around a bunch of print statements in so_dict.py
  3. change except ImportError, exc: into except ImportError, exc: in so_mpi.py

I can open a PR with these if useful

Spectra estimation

Write a bunch of function that do spherical transform, compute power spectra, bin them, and debiased the spectra using the mode coupling matrices.
A place holder can be found in pspy/so_spectra.py

Code logic

This is a general issue for discussing the logic of the power spectrum pipeline.
We will have tons of map to cross correlate, from different detector arrays, different frequency channel, possibly add the Planck data.
The code structure should be adaptable enough to deal with this.

Noise simulations for the LAT

The LAT scanning strategy will probably lead to an inhomogeneous scanning pattern leading to inhomogeneous noise properties. Simulations of this effect are needed to test the optimality of the estimator.

replace travis with github actions

I'd like to add some testing at some point, and it seems like the first step is to replace the travis config with some github configuration.

mpi4py with docker

FYI, pip installing mpi4py in the docker environment doesn't seem to be possible.

Map preprocessing

Ground pick up hurts us badly in AdvACT and might be there in the LAT data.
We want to construct a bunch of filters that can be applied to the data as the preprocessing stage.
A place holder can be found in pspy/so_map_preprocessing.py

Mode coupling calculation

Write a bunch of python functions allowing to generate mode coupling matrices from window function.
The mode coupling matrices should be able to handle Cls and Dls and pure/standard B mode estimation. They should be general enough to also produce the input for the covariance matrix calculation (see appendix of https://arxiv.org/pdf/1609.09730.pdf).
A placeholder can be found in pspy/so_mcm.py

General window function generation

Write a bunch of python functions allowing to create window functions for power spectrum estimation.
These windows should ideally be formed from:
-Masks: point source, galactic, extended sources, cluster, missing pixels
-Hit count/Noise properties
-Amount of cross linking in the data
-Apodisation routine
Ideally you will use the so map class provided in pspy/so_map.py allowing to handle both CAR and Healpix pixellisation. A placeholder can be found in pspy/so_window.py.
One important thing to be done is to code up a general distance function, that is, a function that return the distance to the closest masked pixel for HEALPIX and CAR pixellisation.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.