Comments (6)
In case it helps, I can confirm that I still get the error when running within a 'clean' conda environment created as follows:
conda create --name shtest -c conda-forge pyshtools
which installs as follows:
Channels:
- conda-forge
- defaults
Platform: osx-arm64
Collecting package metadata (repodata.json): done
Solving environment: done
Package Plan
environment location: /Users/simon/anaconda3/envs/shtest
added / updated specs:
- pyshtools
The following packages will be downloaded:
package | build
---------------------------|-----------------
astropy-6.0.0 | py311h9ea6feb_0 8.7 MB conda-forge
astropy-iers-data-0.2024.2.5.0.30.52| pyhd8ed1ab_0 1.2 MB conda-forge
azure-core-cpp-1.10.3 | he231e37_1 282 KB conda-forge
azure-storage-blobs-cpp-12.10.0| h6aa02a4_0 401 KB conda-forge
azure-storage-common-cpp-12.5.0| h607ffeb_2 104 KB conda-forge
brotli-python-1.1.0 | py311ha891d26_1 335 KB conda-forge
bzip2-1.0.8 | h93a5062_5 119 KB conda-forge
c-ares-1.26.0 | h93a5062_0 142 KB conda-forge
cartopy-0.22.0 | py311h6e08293_1 1.8 MB conda-forge
certifi-2024.2.2 | pyhd8ed1ab_0 157 KB conda-forge
cfitsio-4.3.1 | h808cd33_0 743 KB conda-forge
cftime-1.6.3 | py311h9ea6feb_0 201 KB conda-forge
contourpy-1.2.0 | py311hd03642b_0 235 KB conda-forge
curl-8.5.0 | h2d989ff_0 148 KB conda-forge
ducc0-0.33.0 | py311h92babd0_0 1.4 MB conda-forge
font-ttf-ubuntu-0.83 | h77eed37_1 1.5 MB conda-forge
fonttools-4.48.1 | py311h05b510d_0 2.6 MB conda-forge
gdal-3.8.3 | py311h43f0207_2 1.6 MB conda-forge
geos-3.12.1 | h965bd2d_0 1.3 MB conda-forge
geotiff-1.7.1 | h7bcba05_15 114 KB conda-forge
ghostscript-10.02.1 | h965bd2d_0 56.4 MB conda-forge
gmt-6.5.0 | haf49670_0 35.2 MB conda-forge
hdf5-1.14.3 |nompi_h5bb55e9_100 3.3 MB conda-forge
idna-3.6 | pyhd8ed1ab_0 49 KB conda-forge
importlib-metadata-7.0.1 | pyha770c72_0 26 KB conda-forge
kealib-1.5.3 | h210d843_0 136 KB conda-forge
kiwisolver-1.4.5 | py311he4fd1f5_1 60 KB conda-forge
lcms2-2.16 | ha0e7c42_0 207 KB conda-forge
libarchive-3.7.2 | hcacb583_1 765 KB conda-forge
libblas-3.9.0 |21_osxarm64_openblas 15 KB conda-forge
libboost-headers-1.84.0 | hce30654_0 13.2 MB conda-forge
libcblas-3.9.0 |21_osxarm64_openblas 14 KB conda-forge
libcurl-8.5.0 | h2d989ff_0 342 KB conda-forge
libev-4.33 | h93a5062_2 105 KB conda-forge
libgdal-3.8.3 | hb4bbca3_2 8.1 MB conda-forge
libgfortran-5.0.0 |13_2_0_hd922786_3 108 KB conda-forge
libgfortran5-13.2.0 | hf226fd6_3 974 KB conda-forge
libglib-2.78.3 | hb438215_0 2.3 MB conda-forge
libgoogle-cloud-2.12.0 | h49bbb43_5 29.9 MB conda-forge
libgrpc-1.60.0 | hfc68871_1 4.0 MB conda-forge
libiconv-1.17 | h0d3ecfb_2 661 KB conda-forge
liblapack-3.9.0 |21_osxarm64_openblas 14 KB conda-forge
libnetcdf-4.9.2 |nompi_h291a7c2_113 666 KB conda-forge
libnghttp2-1.58.0 | ha4dd798_1 552 KB conda-forge
libopenblas-0.3.26 |openmp_h6c19121_0 2.8 MB conda-forge
libpng-1.6.42 | h091b4b1_0 258 KB conda-forge
libpq-16.1 | h0f8b458_7 2.3 MB conda-forge
libprotobuf-4.25.1 | h810fc01_1 2.1 MB conda-forge
librttopo-1.1.0 | hc8f776e_15 188 KB conda-forge
libspatialite-5.1.0 | h69abc6b_4 3.9 MB conda-forge
libsqlite-3.44.2 | h091b4b1_0 796 KB conda-forge
libxml2-2.12.5 | h0d0cfa8_0 574 KB conda-forge
llvm-openmp-17.0.6 | hcd81f8e_0 268 KB conda-forge
matplotlib-base-3.8.2 | py311hfdba5f6_0 7.5 MB conda-forge
minizip-4.0.4 | hc35e051_0 77 KB conda-forge
netcdf4-1.6.5 |nompi_py311ha6bebe6_100 449 KB conda-forge
nss-3.97 | h5ce2875_0 1.7 MB conda-forge
numpy-1.26.4 | py311h7125741_0 6.3 MB conda-forge
pandas-2.2.0 | py311hfbe21a1_0 14.1 MB conda-forge
pcre2-10.42 | h26f9a81_0 605 KB conda-forge
pillow-10.2.0 | py311hb9c5795_0 39.7 MB conda-forge
pip-24.0 | pyhd8ed1ab_0 1.3 MB conda-forge
pixman-0.43.2 | hebf3989_0 194 KB conda-forge
platformdirs-4.2.0 | pyhd8ed1ab_0 20 KB conda-forge
poppler-24.02.0 | h896e6cb_0 1.4 MB conda-forge
postgresql-16.1 | hc6ab77f_7 4.2 MB conda-forge
proj-9.3.1 | h93d94ba_0 2.5 MB conda-forge
pyerfa-2.0.1.1 | py311h9ea6feb_0 344 KB conda-forge
pygmt-0.11.0 | pyhd8ed1ab_0 148 KB conda-forge
pyproj-3.6.1 | py311h9a031f7_5 482 KB conda-forge
pyshtools-4.10.4 | py311h667c9c3_0 1.0 MB conda-forge
python-3.11.7 |hdf0ec26_1_cpython 13.9 MB conda-forge
python-tzdata-2023.4 | pyhd8ed1ab_0 143 KB conda-forge
python_abi-3.11 | 4_cp311 6 KB conda-forge
pytz-2024.1 | pyhd8ed1ab_0 184 KB conda-forge
pyyaml-6.0.1 | py311heffc1b2_1 183 KB conda-forge
scipy-1.12.0 | py311h4f9446f_2 14.8 MB conda-forge
setuptools-69.0.3 | pyhd8ed1ab_0 460 KB conda-forge
shapely-2.0.2 | py311h0815064_1 528 KB conda-forge
sqlite-3.44.2 | hf2abe2d_0 784 KB conda-forge
tiledb-2.19.1 | h49d9ff7_0 4.5 MB conda-forge
tk-8.6.13 | h5083fa2_1 3.0 MB conda-forge
tzcode-2024a | h93a5062_0 62 KB conda-forge
tzdata-2024a | h0c530f3_0 117 KB conda-forge
urllib3-2.2.0 | pyhd8ed1ab_0 92 KB conda-forge
wheel-0.42.0 | pyhd8ed1ab_0 56 KB conda-forge
xarray-2024.1.1 | pyhd8ed1ab_0 714 KB conda-forge
xerces-c-3.2.5 | hf393695_0 1.2 MB conda-forge
------------------------------------------------------------
Total: 315.9 MB
from shtools.
Could you let me know if this error occurs every time for the same value of lmax on a given machine? Or if this only occurs sometimes?
from shtools.
It turns out that I get the same error using lmax=110
on an Apple M1 max. The error occurred three successive times, so it is reproducible for me. I will try to find the lowest lmax that this crashes on my Mac, and then try to reproduce this on linux. If you could do the same, that would be great.
from shtools.
I ran 5 iterations on Apple M2 Max.
The function is successful consistently for lmax=69, and returns the error consistently for lmax=70
from shtools.
Here are the results of some tests, for lmax=5-130
- Fedora Asahi M1 (Python 3.11 and 3.12, pyshtools installed from source) : fails at LMAX=114 with DSTEGR info = 22
- macOS (M1 Max, python 3.11, pyshtools installed from source) : fails at LMAX = 103 with DSTEGR info = 22
- macOS (M1 Max, python 3.12, pyshtools installed from source) : fails at LMAX = 121 with DSTEGR info = 22
- macOS (intel, python 3.11, pyshtools installed with conda) : fails for LMAX>=70 with DSTEGR info = 15
- macOS (M1 max, python 3.11, pyshtools installed with conda) : fails for LMAX>=71
So, the error occurs on all machines, but at slightly different values of lmax
. Even on the same machine, the failure can occur at different values when using different versions of python. The value at which it fails is the worst when installing pyshtools by conda (python 3.11).
I am not really sure what this means. The good thing is that it is repeatable on a given machine, so it probably isn't a memory related problem. I suspect that the problem might be related to slightly different ways of compiling LAPACK/BLAS that cause the computations to be performed slightly differently (such as a different optimization, or some option like fast-math). To be honest, I am a little surprised that determining the eigenvectors of a 1000x1000 matrix works at all! For such large matrices, it is possible that there is a better way of doing this than the way I chose. If this is important to you, then perhaps we should look into it.
from shtools.
Thanks for this, I can add the following:
- macOS (M2 Max, python 3.12, pyshtools installed from source): fails at LMAX = 115 with DSTEGR info = 22
One other thing I noticed - the execution time appears to be significantly different between cases. The above config takes close to double the time of the config with python3.11 / pyshtools from conda.
For now, it is helpful to know that the issue is reproducible and that I can get better results with the different python versions and installation from source. (it was a little frustrating to upgrade from an intel Mac only to find the problem appeared worse based on the setup I was using initially on M2)
As for importance - my specific use case involves Earth's lithospheric magnetic field. Since LCS-1 is defined to degree 185, it would be nice to be able to do localisations to this degree. However, for the masks that I want to use (different from the test case) I am now able to get results for LMAX > 140, a lot higher than with the conda install. (In one case LMAX = 170 works, the failure point is somewhere between 170 and 185 - impressive indeed). Therefore, I am happy to work around this for now, and don't have a sense of how important this would be for other users. If you do look into this at some point I'd be happy to help run further tests.
from shtools.
Related Issues (20)
- Install issue can't build wheel. (Using Pycharm) HOT 2
- Exporting grid data to 3D model
- Sampling is incorrectly set when using DHRealGrid() class. HOT 1
- Release SHTOOLS 4.11 HOT 1
- Meson can't determine version of pyshtools when installing from a source tarball HOT 5
- jupyter is an (undeclared) build-time dependency HOT 6
- Compilation error on MacOS Sonoma HOT 8
- installing pyshtools lead to error: metadata-generation-failed (PyCharm) HOT 4
- Error message when attempting to install pyshtools on Mac OS X 14.2.0 HOT 15
- Release SHTOOLS 4.12.0 HOT 1
- Implement ducc backend to pyshtools gravity and magnetic classes HOT 2
- Help! I can't install pyshtools!!! HOT 8
- Apple silicon M1 macOS 14.4.1 library 'lapack' not found HOT 5
- Matrix support for pysh.legendre.pl... HOT 3
- Plotting methods in `shclasses` use deprecated matplotlib function `matplotlib.cm.get_cmap` HOT 1
- Release SHTOOLS 4.13.0 HOT 2
- Difference in Reconstruction HOT 14
- plotgmt does not work for grids with extend=False HOT 3
- The method of truncating spherical harmonic coefficients? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from shtools.