ImageD11 needs a new release on PyPI : current does not install with pip about imaged11 HOT 16 CLOSED

So I have something that gets past the CI for pip install ImageD11 today. It only seems to work if both pip and setuptools are up-to-date enough.

There is an option for `pip install ImageD11[full]', but I am not really motivated to debug it.

Assuming nothing bad was broken in this process, then it should be ready for release. The biggest change is renaming the ImageD11src folder to ImageD11. I expect this can cause problems but I didn't see how to get pip install -e . to work otherwise.

Anyway, I think it should install with pip now? Just needs the pypi release if nothing else is looking broken.

from imaged11.

don't forget to tag + release (tomorrow or next week...)

from imaged11.

No - sadly the sdist was broken. Thanks for the heads up. I have removed it from pypi and put a source distribution here on github which should include src/bldlib.py etc. For the moment I don't know how to get it to compile properly within pip. If you do a git clone, or grab the source here, then you just need to run the setup.py build step.

python setup.py build

... and it will execute code inside src/bldlib.py to compile the C methods with various compiler flags. Then you can do pip install . and it should work. Depending on your computer, the compiler flags might not be allowed, in which case please let me know and we can try to fix that. There should be some working examples for the build/test in the continuous integration codes (appveyor/circleci).

Let me know if that doesn't work or if you can see a way to make it work properly with pip.

I guess some actions should be:

• add the rest of the files to sdist (via setup.cfg?)
• hack some more in setup.py until pip can compile it

Feel free to send a pull request if you have any ideas.

from imaged11.

So pip wants a setup.py that understands "egg_info", "install", "develop", "bdist_wheel" and "clean".
https://pip.pypa.io/en/stable/reference/pip_install/

Logically it only needs to prepare the pyf files before numpy.distutils.Extension.build_src is run and build them once.

As I remember, the real problem is that distutils wanted to compile the c sources only once and cache the *.o files somewhere in the build tree. I didn't see a nicer way to get different flavors of compiled code ready for running on different machines.

from imaged11.

Thank you for picking this up so quickly!

Unfortunately I don't have too much experience with python development but If I spot anything I will let you know.

Running python setup.py build followed by pip install . is working well, and I'm happy to report that it runs as expected inside a pyenv virtual environment.

from imaged11.

Glad to hear it worked. I will leave the ticket open for now as a reminder to try to get "pip install" to work properly.

from imaged11.

Need a new release for the sparse stuff ...

from imaged11.

Any progress on this issue? We are now at 1.9.7 and the pypi is left at 1.9.5. This is mostly an issue when developing other packages with dependency on ImageD11. From my experience this is as simple as:

python3 -m build
twine upload dist/*

Although I don't know about the complied stuff. I would have expected that the user would not have to compile anything on their machine, but that all binaries where already pre-compiled within in the build. Is there anything we can do to help resolve this?

from imaged11.

There are a few bugfixes on my fork that ought to get merged back. On windows you were getting binaries but on linux it comes as source code. Are you running from git master now?

from imaged11.

Ok. I see. I am running of master, yes. Just a simple clone and install via the setup.py, which works excellent on ubuntu 20. But it would be nice if the package was pip installable so that any packages developed that depends on ImageD11 can also be pip installable.

from imaged11.

I think I merged everything I could find and currently the CI is green. There were a few problems with dependencies, so I would still like to clean up the pip installation saga. Might need to learn the requirements.txt method (https://stackoverflow.com/questions/29222269/is-there-a-way-to-have-a-conditional-requirements-txt-file-for-my-python-applica ... for h5py==2.10.0 on windows 32 bit and latest everywhere else).

from imaged11.

Nice :) 1.9.8 then now.

Could we not simply make the install_requires list be dynamically assembled? I think it makes the most sense to use the setup.py for this kind of things. Something like this in the setup.py file

needed =[
    "six",
    "numpy",
    "scipy",
    "pillow",
    "matplotlib",
    "xfab>=0.0.4",
    "fabio",
    "PyCifRW",
    # breaks travis for macos ?? "silx",
    "pyopengl",
    "pyopengltk",
    ]+ ["h5py==2.10.0"] if "win32" in sys.platform else ["h5py"]

which is also described in this post:

from imaged11.

Does that always work? Someone wanting to install a package without executing any code is going to run into problems. It could be the reason they invented a requirements mini-language.

I will try to vet the code to see which dependencies are really needed. h5py should be optional, something like an extras_require maybe.

It is not clear how to set things up so people "only pay for what they need"

from imaged11.

Not sure. I thought that the wheels that are put on pypi was dynamic in some sense. I.e that they ran the setup.py files in some way when calling
pip install ImageD11
But perhaps that is not the way it works. You probably know more about how the wheels operate than I do. For "Only pay for what they need" I would also guess that the extras_requires keyword is the way it is intended to be done. See this post for instance

from imaged11.

Hopefully there is something there now. Seems the sdist is very large, so there might be some cleaning to be done somewhere.

from imaged11.

Sweet! pip installs 1.9.8 like a charm for me. Thanks Jon!

from imaged11.