casacore / casarest Goto Github PK

View Code? Open in Web Editor NEW

4.0 4.0 12.0 18.27 MB

The remainder of the AIPS++ libraries that did not end up on casacore

License: GNU General Public License v3.0

CMake 0.64% Python 0.27% C++ 97.29% Shell 0.10% Perl 0.16% Makefile 0.01% C 0.17% Fortran 1.36%

casarest's Introduction

Casacore

A suite of C++ libraries for radio astronomy data processing.

Installation

Debian / Ubuntu

Casacore is now part of Debian and Ubuntu, use apt to search for the package you need:

$ apt-get update
$ apt-get search casacore

MacOS

Use Homebrew with the ska-sa tap to install casacore:

$ brew tap ska-sa/tap
$ brew install casacore

Docker

We now also publish docker images for each github branch on quay.io:

$ docker pull quay.io/casacore/casacore:master

Building from source

Getting the source code

The casacore source code is maintained on github.

You can obtain it using:

$ git clone https://github.com/casacore/casacore

Requirements

To compile casacore you need to meet the following requirements:

cmake
gfortran
g++
flex
bison
blas
lapack
cfitsio (3.181 or later)
wcslib (4.20 or later)
sofa (optional, only for testing casacore measures)
fftw3
hdf5 (optional)
numpy (optional)
boost-python (optional)
ncurses (optional)

On Debian / Ubuntu you can install these with:

$ sudo apt-get install build-essential cmake gfortran g++ libncurses5-dev \
   libreadline-dev flex bison libblas-dev liblapacke-dev libcfitsio-dev \
   wcslib-dev libfftw3-dev

and the optional libraries:

$ sudo apt-get install libhdf5-serial-dev python-numpy \
    libboost-python-dev libpython3.4-dev libpython2.7-dev

On CentOS7 you can install these with:

$ sudo yum install cmake cmake-gui gcc-gfortran gcc-c++ flex bison \
       blas blas-devel  lapack lapack-devel cfitsio cfitsio-devel \
       wcslib wcslib-devel ncurses ncurses-devel readline readline-devel\
       python-devel boost boost-devel fftw fftw-devel hdf5 hdf5-devel\
       numpy boost-python

Obtaining measures data

Various parts of casacore require measures data, which requires regular updating. You can obtain the WSRT measures archive from the ASTRON FTP server:

ftp://ftp.astron.nl/outgoing/Measures/

Extract this somewhere on a permanent location on your filesystem.

Compilation

In the casacore source folder run:

mkdir build
cd build
cmake ..
make 
make install

there are various flags available to cmake to enable and disable options:

$ cmake -DDATA_DIR=/usr/share/casacore/data -DUSE_OPENMP=ON \
    -DUSE_HDF5=ON -DBUILD_PYTHON=ON -DUSE_THREADS=ON

The DATA_DIR should point to the location where you extracted the measures data. Special variables %CASAROOT% and %CASAHOME% can be used here, which can be set at run time through the .casarc file.

You can build python3 support using -DBUILD_PYTHON3=ON. Note that CMake may have problems detecting the correct python3 libraries and headers, so probably you need to set them manually. For example:

-DPYTHON3_LIBRARY=/usr/lib/x86_64-linux-gnu/libpython3.4m.so
-DPYTHON3_INCLUDE_DIR=/usr/include/python3.4

To configure Python2 specific settings use:

PYTHON2_EXECUTABLE
PYTHON2_LIBRARY
PYTHON2_INCLUDE_DIR

To configure Python3 specific settings use:

PYTHON3_EXECUTABLE
PYTHON3_LIBRARY
PYTHON3_INCLUDE_DIR

If you run into problems with boost libraries, try setting -DBoost_NO_BOOST_CMAKE=True. This will be necessary if you have the libraries from NRAO casa in your PATH or LD_LIBRARY_PATH.

Documentation

http://casacore.github.io/casacore

Problems & bugs

If you have any issues compiling or using casacore, please open an issue on the issue tracker on github.

If you have patches please open a pull request. Your contributions are more than welcome! But to maintain a high code quality we have written a contribution manual, please read that first.

casarest's People

Contributors

Stargazers

Watchers

Forkers

gitter-badger o-smirnov jjdmol lao19881213 steve-ord bennahugo athanaseus cquike jasonruonanwang adrianjhpc voronkovma alxndrwllmsn

casarest's Issues

LOFAR & meqtrees-timba not compiling with current release

/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib/libcasa_synthesis.so: undefined reference to `casa::ArrayBase::baseReform(casa::ArrayBase&, casa::IPosition const&) const'
collect2: error: ld returned 1 exit status
make[3]: *** [CEP/Calibration/BBSTools/src/addUV2MS] Error 1
make[3]: Leaving directory `/«PKGBUILDDIR»/build/gnu_opt'
make[2]: *** [CEP/Calibration/BBSTools/src/CMakeFiles/addUV2MS.dir/all] Error 2
make[2]: *** Waiting for unfinished jobs....
/usr/bin/cmake -E cmake_progress_report /«PKGBUILDDIR»/build/gnu_opt/CMakeFiles

problems compiling

originally reported here:

ratt-ru/meqtrees-timba#22

Tony writes: I downloaded latest casacore - seems to compile and install OK. I then downloaded casarest 1.3.1 source code gz file, did mkdir build, cd build, cmake .. , but right away I get one of those horrendous bazillion line c++ error messages which seems to come down to

[NRC-005592LX 9:29am] [build]> make
[ 1%] Building CXX object components/CMakeFiles/casa_components.dir/SpectralComponents/CompiledSpectralElement.cc.o

blah blah blah

/home/twillis/Downloads/casarest-1.3.1/components/SpectralComponents/CompiledSpectralElement.cc: In member function ‘const casa::String& casa::CompiledSpectralElement::getFunction() const’:
/home/twillis/Downloads/casarest-1.3.1/components/SpectralComponents/CompiledSpectralElement.cc:107:58: error: ‘_getFunction’ was not declared in this scope
CountedPtr > f = _getFunction();
^
/home/twillis/Downloads/casarest-1.3.1/components/SpectralComponents/CompiledSpectralElement.cc:108:50: error: ‘_getFunction’ is not a member of ‘casa::SpectralElement’
return dynamic_cast >(SpectralElement::_getFunction().get())->getText();
^
/home/twillis/Downloads/casarest-1.3.1/components/SpectralComponents/CompiledSpectralElement.cc: In member function ‘virtual void casa::CompiledSpectralElement::_setFunction(const casa::String&)’:
/home/twillis/Downloads/casarest-1.3.1/components/SpectralComponents/CompiledSpectralElement.cc:114:57: error: ‘_getFunction’ was not declared in this scope
CountedPtr > f = _getFunction();
^
/home/twillis/Downloads/casarest-1.3.1/components/SpectralComponents/CompiledSpectralElement.cc:134:2: error: ‘_setFunction’ is not a member of ‘casa::SpectralElement’
SpectralElement::_setFunction(f);
^
make[2]: ** [components/CMakeFiles/casa_components.dir/SpectralComponents/CompiledSpectralElement.cc.o] Error 1
make[1]: *** [components/CMakeFiles/casa_components.dir/all] Error 2

casarest fails on lwimager

Upon compiling (for centos7) casarest (v1.3.1) directly after casacore (v2.3.0) in a standard LOFAR compilation I get an error regarding lwimager and hdf5 in the make. Is there a solution? I have now also tried casarest v1.4.2 and adding the HDF5 root dir, include dir and lib to the cmake with -D, but this did not help either...

�[0m[ 95%] Building CXX object synthesis/CMakeFiles/casa_synthesis.dir/Parallel/Applicator.cc.o
[ 95%] Building CXX object synthesis/CMakeFiles/casa_synthesis.dir/Parallel/MPIError.cc.o
[ 96%] Building CXX object synthesis/CMakeFiles/casa_synthesis.dir/Parallel/MPITransport.cc.o
[ 96%] Building CXX object synthesis/CMakeFiles/casa_synthesis.dir/Parallel/PabloIO.cc.o
[ 96%] Building CXX object synthesis/CMakeFiles/casa_synthesis.dir/Parallel/SerialTransport.cc.o
[ 97%] Building CXX object synthesis/CMakeFiles/casa_synthesis.dir/Utilities/FixVis.cc.o
[ 97%] Building CXX object synthesis/CMakeFiles/casa_synthesis.dir/Utilities/ThreadCoordinator.cc.o
[ 97%] Building Fortran object synthesis/CMakeFiles/casa_synthesis.dir/fortran/fgridft.f.o
[ 98%] Building Fortran object synthesis/CMakeFiles/casa_synthesis.dir/fortran/fmosaic.f.o
[ 98%] Building Fortran object synthesis/CMakeFiles/casa_synthesis.dir/fortran/fpbmos.f.o
[ 98%] Building Fortran object synthesis/CMakeFiles/casa_synthesis.dir/fortran/fpbwproj.f.o
[ 99%] Building Fortran object synthesis/CMakeFiles/casa_synthesis.dir/fortran/leakantso.f.o
[ 99%] Building Fortran object synthesis/CMakeFiles/casa_synthesis.dir/fortran/wprojgrid.f.o
Linking CXX shared library libcasa_synthesis.so
[ 99%] Built target casa_synthesis
Scanning dependencies of target lwimager
[100%] Building CXX object synthesis/CMakeFiles/lwimager.dir/apps/lwimager.cc.o
Linking CXX executable lwimager
�[91mCMakeFiles/lwimager.dir/apps/lwimager.cc.o: In function `casa::HDF5Group::HDF5Group(casa::HDF5Object const&, casa::String const&, bool, bool)':
lwimager.cc:(.text._ZN4casa9HDF5GroupC2ERKNS_10HDF5ObjectERKNS_6StringEbb[_ZN4casa9HDF5GroupC5ERKNS_10HDF5ObjectERKNS_6StringEbb]+0x7a): undefined reference to `casa::HDF5Group::init(int, casa::String const&, casa::String const&, bool, bool)'
�[0m�[91mcollect2: error: ld returned 1 exit status
�[0m�[91mmake[2]: *** [synthesis/lwimager] Error 1
�[0m�[91mmake[1]: *** [synthesis/CMakeFiles/lwimager.dir/all] Error 2
�[0m�[91mmake: *** [all] Error 2
�[0mThe command '/bin/sh -c cd ${INSTALLDIR}/casarest/build && make -j ${J}' returned a non-zero code: 2

lwimager major loop predicts incorrect model visibilities when using large model cubes

This bug has been driving me mental for a month, and caused me to tear the last of my hair out, but now I've localized it. Posting a report here for posterity. The bug comes from the CASA synthesis libs. I don't know if it's been fixed in CASA (and thus in the new https://github.com/radio-astro/casasynthesis package), we need to follow this up. (If it's not fixed in CASA then CHILES is fucked basically, since this affects the major loop of CLEAN).

The bug manifested itself as follows. When using lwimager to predict visibilities into MODEL_DATA from a large image cube, for example:

lwimager fillmodel=1 operation=csclean wprojplanes=0 field=0 spwid=0 stokes=I padding=1 cachesize=4096 niter=0 nchan=116 chanstart=16 chanstep=1 npix=6144 cellsize=0.06arcsec img_chanstart=16 mode=channel ms=CYG-C-1scan-testpaste.MS model=model3 -0-14.img fixed=1

I was getting different results on different machines (MODEL_DATA visibilities with a 10-20% difference!) Funnily enough, visibilities in the higher channels were always identical, it was only the lower-numbered channels that diverged.

I haven't pinpointed the bug but through experiments I've localized its behaviour. Somewhere in https://github.com/casacore/casarest/blob/master/synthesis/MeasurementEquations/CubeSkyEquation.cc is some funky code for deciding if an image cube is "too big" w.r.t. available RAM. If it is "too big", the predict() loop iterates over chunks of channels, picking a chunk size based on RAM size. If it's not too big, predict() processes the whole cube at once.

It looks like only the last chunk is predicted properly. Preceding chunks manifest the abovementioned error. The magnitude of the error depends on chunk size. So on a big-RAM machine where the whole cube is processed in one go, the visibilities are correct. On smaller-RAM machines, as soon as chunking kicks in, the lower channels get predicted incorrectly. Exact numbers depend on RAM size :))) so I was getting crazy inconsistencies between different machines.

Since it doesn't actually report on chunking explicitly, you need to look at the following console output:

2015-10-02 17:40:28     INFO    CSCleanImageSkyModel::solve     niter=0, assuming predict-only mode, converting model image to MODEL_DATA visibilities
0%....10....20....30....40....50....60....70....80....90....100%

It prints a 0 to 100% meter for each chunk. If you only see one 0...100% meter you're good, it did the whole cube in one go. If you see two or more meters, the bug has kicked in.

Obvious workaround for this use case is to split the cube in frequency to ensure single-chunk processing, and invoke lwimager multiple times. Will post more on this elsewhere.

ImageFFT.h missing from 1.4.0 package

Getting this error building MeqTrees against the new casarest package:

/home/oms/meqtrees-timba/MeqNodes/src/UVBrick.cc:33:48: fatal error: casarest/synthesis/Images/ImageFFT.h: No such file or directory
 #include <casarest/synthesis/Images/ImageFFT.h>

File needs to be packaged, from the looks of it.

License/Copying file

I am trying to package casarest for openSUSE (as a separate package from casacore); however, one issue is the lack of a License/Copying file specific to casarest, which would prevent my packages from getting into the official distro. Could you please clarify the licensing situation and perhaps add a LICENSE/COPYING file similar to the one casacore has?

Thanks.

The OrderedMap and SimpleOrderedMap are now deprecated in casacore

I'm having trouble building against casacore3.1 as the msvis component is using deprecated casacore classes. I don't want to force my users to use these as they do not need msvis. Is it possible to make some optional build targets. I could put together some edits to the cmake and submit a pull request?

Failing build on ubuntu

When trying to build casarest using the standard LOFAR Docker building, I got a huge bunch of errors

make
[  0%] Building CXX object components/CMakeFiles/casa_components.dir/SpectralComponents/CompiledSpectralElement.cc.o
In file included from /opt/casarest/src/components/SpectralComponents/CompiledSpectralElement.h:32:0,
                 from /opt/casarest/src/components/SpectralComponents/CompiledSpectralElement.cc:27:
/opt/casarest/src/components/SpectralComponents/SpectralElement.h:108:10: error: 'Double' does not name a type
  virtual Double operator()(const Double x) const;
          ^
/opt/casarest/src/components/SpectralComponents/SpectralElement.h:110:2: error: 'Bool' does not name a type
  Bool operator==(const SpectralElement& other) const;
  ^
/opt/casarest/src/components/SpectralComponents/SpectralElement.h:116:10: error: 'Double' does not name a type
  virtual Double operator[](const uInt n) const;
          ^
/opt/casarest/src/components/SpectralComponents/SpectralElement.h:119:15: error: 'String' does not name a type
  static const String* allTypes(Int &nall,
               ^

Those are the cmake flags used:

ake -DCMAKE_INSTALL_PREFIX=${INSTALLDIR}/casarest -DCASACORE_ROOT_DIR=${INSTALLDIR}/casacore -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-std=c++11 -O2 -march=native -DNDEBUG" ../src/

Can anybody explain to me what I am doing wrong here...? I use casacore 3.0.0 and for the rest all dependencies come from the ubuntu 16.04 repo.

1.7.0 marked as pre-release?

Is it intentional that 1.7.0 is marked as "pre-release" in the Github release list?

Make 1.8.0 release

various issues have been solved, so its time for a release.

cmake, building casarest using hdf5 support, fails

I try to build casarest with HDF5 support (I built the latest version of hdf5). To this end, I built casacore with the DHDF5_ROOT_DIR and DUSE_HDF5 flags.

For the cmake of casarest, I use the DHDF5_ROOT_DIR flag of cmake. The make crashes at almost 100% with

CMakeFiles/lwimager.dir/apps/lwimager.cc.o: In function 
`casa::HDF5Group::HDF5Group(casa::HDF5Object const&, casa::String const&, bool, bool)':
lwimager.cc:
(.text._ZN4casa9HDF5GroupC2ERKNS_10HDF5ObjectERKNS_6StringEbb[_ZN4casa9HDF5GroupC5ERKNS_10HDF5ObjectERKNS_6StringEbb]+0x7a):
 undefined reference to `casa::HDF5Group::init(int, casa::String const&, casa::String const&, bool, bool)'

I don't really know how to fix this. Any ideas?

compile fails with "no match for call to ‘(casa::Quantum<double>)"

Hi
compiling casarest master against casacore 2.4.1 with gcc 7.3.1 on CentOS7 fails with the folowing error:

no match for call to ‘(casa::Quantum)

entire cmake & make output in attachment

casarest_error.txt

casacore compilation in the same environment with flags "-DBUILD_PYTHON=True -DUSE_OPENMP=True -DUSE_FFTW3=True -DUSE_HDF5=True" went well

many thanks
Francesco

A non-issue :)

Just a pat on the back for release 1,7,0 I'm migrating to Ubuntu 20.04, which provides casacore 3,1 libraries as part of the release. I could then build the lwimager program directly from this release of casarest. :)
Thanks and cheers
Tony

Problem finding cfitsio in fedora core 26

The CMake script struggles to find the already installed cfitsio library.

I fixed the problem changing line 38 of cmake/FindCfitsIO.cmake
from
HINTS ${CFITSIO_ROOT_DIR} PATH_SUFFIXES include)
to
HINTS ${CFITSIO_ROOT_DIR} PATH_SUFFIXES include PATH_SUFFIXES cfitsio)

for a more general solution you might want to use
HINTS ${CFITSIO_ROOT_DIR} PATH_SUFFIXES include include/cfitsio include/libcfitsio0)

FITS format specifications.

I'm not sure how deep this goes, but when I used lwimager to make a 512x512 image, and I look in the FITS headers, I see the center pixels are CRPIX1/2 = 257. Noting also that CRVAL1/2 = 0.

The FITS spec notes that a pixel value references the center of a pixel so that an image starts at pixel value of 0.5 (since we use FORTRAN indexing) and ends at pixel value npix+0.5
So, for a 2x2 image, CRPIX=1.5 and for a 512x512 image it should be 256.5

As an aside, I am currently working on a pipeline to take a model image, a tdl file (fov gives cell size), parset file, etc and produce an lsm file with which I can make an image - the resulting model image should equal the input image. Has something like this been done? Certainly I think it could be useful for debugging this issue, so let me know if the code is useful and I'll share.

1.5.0 doesn't compile with casacore 3.1.1

I have tried under Ubuntu 18.04.2 in order to use it as one of the LOFARs software 4.0.2 requirements. It does not compile.

Best regards,
Alex

SO version and name information missing for casa_msvis

dpkg-shlibdeps: warning: can't extract name and version from library name 'libcasa_msvis.so'

probably this would help

set_target_properties(casa_msvis PROPERTIES OUTPUT_NAME casa_msvis)
SET_TARGET_PROPERTIES(casa_msvis PROPERTIES SOVERSION 0)

error: 'const class casacore::GaussianMultipletSpectralElement' has no member named 'getGaussians

Hi Admin, I'm trying to build a casarest debian package for ubuntu focal with casacore-3.4.0

/build/casarest-1.7.0/components/SpectralComponents/GaussianMultipletSpectralElement.cc: At global scope:
/build/casarest-1.7.0/components/SpectralComponents/GaussianMultipletSpectralElement.cc:191:1: error: no declaration matches 'const std::vector<casacore::GaussianSpectralElement>& casacore::GaussianMultipletSpec
tralElement::getGaussians() const'
  191 | GaussianMultipletSpectralElement::getGaussians() const {
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/build/casarest-1.7.0/components/SpectralComponents/GaussianMultipletSpectralElement.cc:191:1: note: no functions named 'const std::vector<casacore::GaussianSpectralElement>& casacore::GaussianMultipletSpectralE
lement::getGaussians() const'
In file included from /build/casarest-1.7.0/components/SpectralComponents/GaussianMultipletSpectralElement.cc:27:
/build/casarest-1.7.0/components/SpectralComponents/GaussianMultipletSpectralElement.h:120:7: note: 'class casacore::GaussianMultipletSpectralElement' defined here
  120 | class GaussianMultipletSpectralElement: public CompiledSpectralElement {
      |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/build/casarest-1.7.0/components/SpectralComponents/GaussianMultipletSpectralElement.cc: In member function 'virtual casacore::Bool casacore::GaussianMultipletSpectralElement::toRecord(casacore::RecordInterface&
) const':
/build/casarest-1.7.0/components/SpectralComponents/GaussianMultipletSpectralElement.cc:202:19: error: '_gaussians' was not declared in this scope; did you mean 'gaussians'?
  202 |  for (uInt i=0; i<_gaussians.size(); i++) {
      |                   ^~~~~~~~~~
      |                   gaussians
/build/casarest-1.7.0/components/SpectralComponents/GaussianMultipletSpectralElement.cc: In member function 'virtual void casacore::GaussianMultipletSpectralElement::set(const casacore::Vector<double>&)':
/build/casarest-1.7.0/components/SpectralComponents/GaussianMultipletSpectralElement.cc:225:2: error: '_gaussians' was not declared in this scope
  225 |  _gaussians[0].setAmpl(amp0);
      |  ^~~~~~~~~~
/build/casarest-1.7.0/components/SpectralComponents/GaussianMultipletSpectralElement.cc: In member function 'virtual void casacore::GaussianMultipletSpectralElement::setError(const casacore::Vector<double>&)':
/build/casarest-1.7.0/components/SpectralComponents/GaussianMultipletSpectralElement.cc:261:2: error: '_gaussians' was not declared in this scope
  261 |  _gaussians[0].setError(errors);
      |  ^~~~~~~~~~
/build/casarest-1.7.0/components/SpectralComponents/GaussianMultipletSpectralElement.cc: In member function 'virtual void casacore::GaussianMultipletSpectralElement::fix(const casacore::Vector<bool, std::allocat
or<bool> >&)':
/build/casarest-1.7.0/components/SpectralComponents/GaussianMultipletSpectralElement.cc:294:2: error: '_gaussians' was not declared in this scope
  294 |  _gaussians[0].fix(fixed);
      |  ^~~~~~~~~~
/build/casarest-1.7.0/components/SpectralComponents/GaussianMultipletSpectralElement.cc: In function 'std::ostream& casacore::operator<<(std::ostream&, const casacore::GaussianMultipletSpectralElement&)':
/build/casarest-1.7.0/components/SpectralComponents/GaussianMultipletSpectralElement.cc:320:51: error: 'const class casacore::GaussianMultipletSpectralElement' has no member named 'getGaussians'
  320 |  Vector<GaussianSpectralElement> gaussians = elem.getGaussians();
      |                                                   ^~~~~~~~~~~~
make[3]: *** [components/CMakeFiles/casa_components.dir/build.make:79: components/CMakeFiles/casa_components.dir/SpectralComponents/GaussianMultipletSpectralElement.cc.o] Error 1

master branch and 1.5 do not compile with casacore 3.2.1

When I build casarest, I get the following error:

/casarest/msvis/MSVis/VisibilityIterator.h:35:10: fatal error: casacore/casa/Containers/Stack.h: No such file or directory
#include <casacore/casa/Containers/Stack.h>
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

I get the same error when I build with casarest 1.5.0.

Could you please have a look?