archiconda / build-tools Goto Github PK

View Code? Open in Web Editor NEW

282.0 7.0 50.0 51 KB

necessary build tools for the archiconda distribution

License: BSD 2-Clause "Simplified" License

Dockerfile 12.37% Makefile 5.69% Shell 7.40% Python 74.54%

build-tools's Introduction

Notice

This project has been superseded by miniforge, a project by conda-forge.

See:

Archiconda3

This repository holds many configuration scripts for the Archiconda3 distribution. Focused on porting conda-forge's work to 64 bit Arm processors.

Shippable: https://app.shippable.com/subs/github/Archiconda/dashboard
Anaconda: https://anaconda.org/archiarm/
Github: https://github.com/archiconda/

The goal of this repository is develop the groundwork needed to compile conda-forge on aarch64. Once that is layed out, we will be working toward backporting much of this work to conda-forge

Tasks to do:

How far along the stack do we need to go:

For users to install Archiconda3, they need to be able to install the conda package. conda-build, and anaconda-client do not need to be in that package, they can be obtained from conda-forge directly.

What this means, is that every package that conda depends on, to run, needs to be an arch specific package. The packages used to build those package do not need to be arch specific packages.

For now, these are the necessary packages:

python-3.7.0-hab5db58_3
ca-certificates-2018.11.29-ha4d7672_0
conda-env-2.6.0-1
libgcc-ng-7.3.0-h5c90dd9_0
libstdcxx-ng-7.3.0-h5c90dd9_0
libffi-3.2.1-h38784ca_1005
ncurses-6.1-hf484d3e_1002
openssl-1.0.2p-h14c3975_1002
xz-5.2.4-h14c3975_1001
yaml-0.1.7-h14c3975_3
zlib-1.2.11-h14c3975_1003
libedit-3.1.20170329-hf8c457e_1001
~~readline-7.0-h7ce4240_5~~ no, nobody wants readline.
tk-8.6.8-h14c3975_0
sqlite-3.26.0-hf8c457e_1000
asn1crypto-0.24.0-py37_1003
certifi-2018.11.29-py37_1000
chardet-3.0.4-py37_1
idna-2.7-py37_0
pycosat-0.6.3-py37h14c3975_0
pycparser-2.19-py37_0
pysocks-1.6.8-py37_0
ruamel_yaml-0.15.64-py37h14c3975_0
six-1.12.0-py37_1000
cffi-1.11.5-py37hb7f436b_1
setuptools-40.6.3-py37_0
cryptography-2.3.1-py37hb7f436b_1
wheel-0.32.3-py37_0
pip-18.1-py37_1000
pyopenssl-18.0.0-py37_0
urllib3-1.23-py37_0
requests-2.19.1-py37_0
conda-4.5.12-py37_1000

Limitations of the approach

We are basically compute bound at this point. shippable gives us 1 CI to use at a time (for one organization).

1 build, even for a nearly empty pure python package, takes about 3 minutes.

See for example the package setuptools: https://app.shippable.com/github/Archiconda/setuptools-feedstock/dashboard

The 3 minutes (180 seconds) are broken up as follows:

15s Shippable things we have no control over
5 seconds updating ubuntu's cache of apt
4 seconds installing bzip2 and curl
2 seconds downloading Archiconda3
40 seconds installing Archiconda3 (including conda-build and anaconda-client)
100 seconds building the package
5 seconds getting the package name
5 seconds to upload the package
1 second leaning up.

We can maybe cut 50 seconds of this by uploading our own container.

But ultimately, we only have 1 CI, with 1 parallel job at a time, so we cannot run too many feedstocks at once.

It does have 96 cores, so maybe we can find a different way to parallelize things? I really feel like that might be over complicating things.

How to speed things up

I haven't had many problems compiling standard software.

For example, python compiled on the first shot, and the issues were primarely due to the fact that some software hardcodes binutil dependencies.

Therefore, we can potentially not have shippable automatically get triggered.

Users would have to wait until a regular linux 64 bit build passes, before triggering the shippable build.

How to start:

system requirements

python
conda
conda constructor
selenium
chromedriver-binary
python-chromedriver-binary

Be friends with jjhelmus

He will build the following critial packages:

The compilers and libstdc

These include the following:

binutils_impl_linux-aarch64-2.29.1-hc862510_0.tar.bz2
binutils_linux-aarch64-2.29.1-h1dbaa89_0.tar.bz2
crosstool-ng-1.23.0.451_g5888cf1-5.tar.bz2
gcc_impl_linux-aarch64-7.3.0-h68995b2_0.tar.bz2
gcc_linux-aarch64-7.3.0-h98564e2_0.tar.bz2
gdb_linux-aarch64-7.12.1-h6bc79d0_0.tar.bz2
gfortran_impl_linux-aarch64-7.3.0-h5c90dd9_0.tar.bz2
gfortran_linux-aarch64-7.3.0-h98564e2_0.tar.bz2
gxx_impl_linux-aarch64-7.3.0-h5c90dd9_0.tar.bz2
gxx_linux-aarch64-7.3.0-h98564e2_0.tar.bz2
libgcc-ng-7.3.0-h5c90dd9_0.tar.bz2
libgfortran-ng-7.3.0-h6bc79d0_0.tar.bz2
libstdcxx-ng-7.3.0-h5c90dd9_0.tar.bz2

Technically you don't need make (listed below), but i've had so much trouble building it, I'm just going to take jjhelmus'

make-4.2.1-h7b6447c_1.tar.bz2

These packages are available from his c4aarch64 anaconda channel.

He also created a tag on anaconda.org, I assume through cross compilation, where you can bootstrap a conda for aarch64.

https://anaconda.org/jjhelmus/repo?label=aarch64_bootstrap

Now the fun stuff

Create a docker image that uses Centos7, and has Archiconda installed.
There is a bug in the version of conda in the bootstrap, that doesn't allow you to have multiple channels. The maximum number of channels is 2. This is why this next step is important.
Use that anaconda channel to create an anaconda installer. See the installer directory in this repositiory.
Start building pacakges.
You can try and build make, but after you do that, try and build m4 with your newly cut version of make. I couldn't get it to work. I think jjhelmus did something special. Or I did something wrong :/
The build order is documented #4

Building recipes

Most of the heavy lifting is done by fork_conda_forge.py, a python script that does:

Uses the github API to check if Archiconda already has the desired feedstock.
If not, it will fork it from conda-forge
Sets up an aarch64 branch.
Enables building the repository on shippable.
- To enable the repository on shippable, you must first request access to aarch64 machines.
- Once you get access, you need ~~to pay for an API key~~ ~~trust that I'm not stealing your creditials~~, inspect the code in fork_conda_forge.py to believe that I'm not stealing your credentials.
- You will see a chrome window popup, and some explination on the command line as to what is happening.
- Basically, I screen scrape and click buttons for you to enable the repository.
- You need to log in manually the first time so that the script can save your cookies.
Rerenders the recipe with conda smithy rerender
- You should install conda-smithy from Archiconda to rerender things correctly.
Pushes the changes to the aarch64 branch.

Because the repository has been enabled, pushing to that branch will trigger a build on shippable.

Docker image

The Docker image should:

Set the locale

# Set the locale
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8

Install a few basic pacakges. bzip2 and curl or wget are necessary to help bootstrap the process.
Install Archiconda3.
Add the archiconda3 to the path.

aarch64 docker on x86 using qemu

If you are creating the docker image while running on your personal laptop, it is useful to use qemu-static to run an aarch64 inside x86.

Put the line

# needed to build this on x86
COPY qemu-aarch64-static /usr/bin/

and you should be gravy.

Bulding using qemu and docker

I've found that it can be helpful to build and test using qemu and docker. It might be as simple as iterating quickly to find what dependencies were missing from the original packages.

Often, binutils are hardcoded, so you have to pass compilation flags to tell the build system what ar command to use.

It might be useful to mount a local directory to use with the docker build system.

For example, the following command mounts the registration directory which contains a bunch of feedstocks and runs the archiconda/centos7 image. The first command it runs is bash allows you to interact with the system.

docker run -v /home/mark2/git/aarch64/build-tools/registration:/feedstocks -i -t archiconda/centos7 bash

Be warned, CPU emulation is SLOW and will make your computer crawl.

I started compiling cmake at the same time it did on shippable, and didn't even get through the bootstrap when it has finished compiling.

build-tools's People

Contributors

Stargazers

Watchers

build-tools's Issues

How can I change the python version

I create a new conda env and try to install python 3.6.5, why it reports error.I want to use python 3.6 on my arm pc

Package build order

Now we get to build python

python 3.7
- removed xorg cdt and tk
  - skip tk tests in run tests.
- need to disable optimizations because compilation takes too long and I don't feel like waiting.
  - Compilation now takes about 20 mins
- remember to remove readline from tests too
  - Potentially, we can include the readline module during compilation, ignore the run exports, and include it as a test dependency. People should really use ipython if they need readline-like functionality. Prompt-toolkit is much better....
- rebuild with optimizations (build number: 1001 https://app.shippable.com/github/Archiconda/python-feedstock/runs/27/1/console)
python 3.6 (remove xorg cdt and tk)
- I had to disable a test because the locale isn't set up correctly. Right now, I don't really care Archiconda/python-feedstock@fdd0664
- rebuild with optimizations (build number: 1002 https://app.shippable.com/github/Archiconda/python-feedstock/runs/26/1/console
python 2.7 (remove xorg cdt and tk)
- make sure to disable test for the command idle which requires tk
- doesn't include runtime optimizations

Now we get to start building the python ecosystem.

Because python takes so long to build, I'm building a few packages on docker, but I only have python 3.7, so these will need to be rebuild.
These next 4 pacakges need:

conda config --set add_pip_as_python_dependency false

certifi
setuptools
wheel (requires setup tools)
pip

I think at this point the circular dependency is over.

These can be built together

Package in bold had to be made arch dependent..... yea it didn't last long until i wrote a script that just fixed the recipes I needed. Of course, the script doesn't keep track of what it patched, so who knows what packages are necessary for a minimal distribution.

pycparser <- had to upload 3.7 using qemu
cffi <- Was getting built before pycparser, so I had to built it on qemu before other packages start to fail.

Cryptography is annoying since it runs pytest....

pytest dependencies are too much ...

Now for the pytest

atomicwrites <- circular dependency with pytest
attrs <---- I didn't go through the dependencies of this one....
funcsigs # py27
more-itertools >=4.0
scandir # [py27] <- for pathlib2
pathlib2 >=2.2.0 # [py<36]
pluggy >=0.7
-[ ] py >=1.5.0
pytest

cryptography
- disabled tests due to pytest

These need cryptography

urllib3 >=1.21.1,<1.25
requests >=2.12.4,<3
yaml (c package, requires libtool)
ruamel_yaml
- requires pytest
conda
- Skip the tests because jjhelmus says they require defaults + x86
- conda-env >=2.6 <- just remove this dependency.

Other notes:
I mostly killed noarch packages because I just wanted the user to be able to install conda-build and anaconda-client, rather than to ship them in a single executable.

Other challenges I had

zstd: Needs CMAKE_AR Archiconda/zstd-feedstock@3b6d1bd
libarchive: is tricky, make sure to build its dependencies first. All of them. using the same version of conda-build. SPecifically, if it finds any of the .la files it thinks it needs, it will try too statically link everything.

shippable doesn't pull arm64v8/centos

shippable_retry sudo docker pull arm64v8/centos:
invalid reference format
retrying 1 of 3 times...
invalid reference format
retrying 2 of 3 times...
invalid reference format
retrying 3 of 3 times...

Then fails

See: https://app.shippable.com/github/Archiconda/cmake-feedstock/runs/6/1/console

matplotlib requries tcl/tk on linux

https://app.shippable.com/github/Archiconda/matplotlib-feedstock/runs/10/summary/console

I think there are now some CDTs available on this channel:

https://anaconda.org/c4aarch64/

but I don't really understand what CDTs are, so I'm apprehensive about using them.

pythran and numba

numba needs llvm and tbb

ModuleNotFoundError: No module named 'conda'

When I try to check 'conda' command after installing 'Archiconda', it throws the following error:

Traceback (most recent call last): File "/home/username/archiconda3/bin/conda", line 7, in <module> from conda.cli import main ModuleNotFoundError: No module named 'conda'

Python version: 3.7
OS: Ubuntu 18.04 LTS
Platform: NVIDIA Jetson Nano

Archiconda automatically activate base environment

Hi~
I use archiconda in ubuntu arm version and along with an aarch64 CPU. I found something interesting:

After installed, python3.7 in archiconda is activated by default(while system version is 3.5)
Though the python is activated, but there is not an '(base)' hint.
No "auto_activate_base" option in conda config --show.

Because of this, the terminal startup appears to be slower than before. How can I fix that? I am not willing to edit the conda init script in .bashrc because I am not familiar with it.

no support for constructor

conda/conda-pack#45

should we just try to use conda-pack???

scientific stack

Numpy and Scipy

mpi
openmp
blas
numpy 1.11 <- I chose this because many pinnings I've seen use numpy 1.11
- Remember to change the default pinnings.
numpy 1.15
scipy

Pillow

jpeg
libtiff
libpng -- won't pass tests on aarch64 pnggroup/libpng#266
freetype
pillow
- remove tk dependency from it.

Matplotlib

~~Have to build version 2.3.3 first because that is what python 2 can install..~~ Python 2.7 dropped

kiwisolver
tornado
matplotlib

Depend only on numpy and/or scipy

cytoolz
pywavelets

Now the fun ones

scikit-image -- patch out matplotlib
opencv -- don't link to qt

Machine Learning and stats

scikit-learn
pandas
pytorch ???? need to unvendor more things.

Even more

sympy
stan
pymc3
pytables
astropy
xarray

Archiconda3-0.2.3-Linux-aarch64.sh: line 342: /home/pi/archiconda3/bin/python: No such file or directory

How to deal with this situation? My hardware is Raspberry PI 4B, and my OS is Raspbian(AARCH64).

There exists this file:
~/archiconda3/bin $ ls -lah
total 44K
drwxr-xr-x 2 pi pi 4.0K Dec 25 15:45 .
drwxr-xr-x 10 pi pi 4.0K Dec 25 15:27 ..
lrwxrwxrwx 1 pi pi 8 Feb 26 2019 2to3 -> 2to3-3.7
-rwxr-xr-x 1 pi pi 123 Feb 26 2019 2to3-3.7
lrwxrwxrwx 1 pi pi 7 Feb 26 2019 idle3 -> idle3.7
-rwxr-xr-x 1 pi pi 121 Feb 26 2019 idle3.7
lrwxrwxrwx 1 pi pi 8 Feb 26 2019 pydoc -> pydoc3.7
lrwxrwxrwx 1 pi pi 8 Feb 26 2019 pydoc3 -> pydoc3.7
-rwxr-xr-x 1 pi pi 106 Feb 26 2019 pydoc3.7
lrwxrwxrwx 1 pi pi 9 Feb 26 2019 python -> python3.7
lrwxrwxrwx 1 pi pi 9 Feb 26 2019 python3 -> python3.7
-rwxr-xr-x 1 pi pi 14K Dec 25 15:45 python3.7
lrwxrwxrwx 1 pi pi 17 Feb 26 2019 python3.7-config -> python3.7m-config
lrwxrwxrwx 1 pi pi 9 Feb 26 2019 python3.7m -> python3.7
-rwxr-xr-x 1 pi pi 3.3K Feb 26 2019 python3.7m-config
lrwxrwxrwx 1 pi pi 17 Feb 26 2019 python3-config -> python3.7m-config
lrwxrwxrwx 1 pi pi 10 Feb 26 2019 pyvenv -> pyvenv-3.7
-rwxr-xr-x 1 pi pi 463 Feb 26 2019 pyvenv-3.7

and python is installed

Installing Opencv

How do I install opencv within my conda environment?

I tried to use sudo apt-get installl opencv-python which works but it installs everything globally I guess. It wont let me import the packages within my conda environment.

how to install gcc-7.3 or gcc-7.2?

@jjhelmus I've managed to make the first package, toolchain,

but it seems I need a compatible version of libgcc-ng. I can't find the feedstock for that one.

It looks from the notes that you cross compiled that one.

Could you help me get it to the version I have installed in the docker container??
The docker container uses gcc 7.4.
I don't really know how to install gcc 7.3 on aarch64

You can see the results of this build process:
https://app.shippable.com/github/hmaarrfk/make-feedstock/runs/4/1/console

I just got the rights for Archiconda to use aarch instances on shippable.

openblas?

numpy would be happy!

I can't install Archiconda

I downloaded the "Archiconda3-0.2.3-Linux-aarch64.sh" file.
chmod + x Archiconda3-0.2.3-Linux-aarch64.sh
command i used.
./Archiconda3-0.2.3-linux-aarch64.sh
I did but I get an error like this:

PREFIX=/home/tunahan/archiconda3
installing: python-3.7.1-h39be038_1002 ...
./Archiconda3-0.2.3-Linux-aarch64.sh: 342: ./Archiconda3-0.2.3-Linux-aarch64.sh: /home/tunahan/archiconda3/bin/python: not found

Lessons learned

Something .ci_support/linux_* has some things that are linux-64 specific, and not always translatable to linux-aarch64

cdt?

I could always adapt them from https://github.com/AnacondaRecipes/aggregate
but I don't feel like writing custom scripts for them.

Is there repos for China? I can not visit the default repo if it's without vnp.

Or can I create one giving service to ?

locale isn't setup correctly

It seems that the locale isn't setup correctly in debian
This is likely what was causing the tests to fail in python3.6

I keep getting

 export LC_ALL=en_US.UTF-8
/root/a3d0086a-ce4e-4c6a-a39a-a9fdb7eeefbf.sh: line 86: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

Seems like debian needs some special fanangaling

https://www.thomas-krenn.com/en/wiki/Perl_warning_Setting_locale_failed_in_Debian

qt?

bioconda tools

Hi,

I am running Archiconda on my Jetson Xavier NX. I am trying to install tools from bioconda and conda-forge, but I always get an error that the tools aren't available from my current channels - even thoguh I have both bioconda and conda-forge channels in my list.

Did anyone experience the same issue? Any fixes?

Cheers and thanks,
Stefan

building tk

Do I need to install dev libraries to compile tk???

I keep failing with

make: *** [Makefile:1134: tkTextBTree.o] Error 1
make: *** [Makefile:1137: tkTextDisp.o] Error 1
make: *** [Makefile:1140: tkTextImage.o] Error 1
make: *** [Makefile:1143: tkTextIndex.o] Error 1
make: *** [Makefile:1146: tkTextMark.o] Error 1
In file included from /root/archiconda3/conda-bld/tk_1547479430536/work/tk8.6.9/unix/../generic/tkPort.h:21:0,
                 from /root/archiconda3/conda-bld/tk_1547479430536/work/tk8.6.9/unix/../generic/tkInt.h:19,
                 from /root/archiconda3/conda-bld/tk_1547479430536/work/tk8.6.9/unix/../generic/tkMain.c:32:
/root/archiconda3/conda-bld/tk_1547479430536/work/tk8.6.9/unix/../generic/tk.h:96:13: fatal error: X11/Xlib.h: No such file or directory
 #   include <X11/Xlib.h>
             ^~~~~~~~~~~~
compilation terminated.
In file included from /root/archiconda3/conda-bld/tk_1547479430536/work/tk8.6.9/unix/../generic/tkPort.h:21:0,
                 from /root/archiconda3/conda-bld/tk_1547479430536/work/tk8.6.9/unix/../generic/tkInt.h:19,
                 from /root/archiconda3/conda-bld/tk_1547479430536/work/tk8.6.9/unix/../generic/tkBind.c:15:
/root/archiconda3/conda-bld/tk_1547479430536/work/tk8.6.9/unix/../generic/tk.h:96:13: fatal error: X11/Xlib.h: No such file or directory
 #   include <X11/Xlib.h>
             ^~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:999: tkMain.o] Error 1
In file included from /root/archiconda3/conda-bld/tk_1547479430536/work/tk8.6.9/unix/../generic/tkPort.h:21:0,
                 from /root/archiconda3/conda-bld/tk_1547479430536/work/tk8.6.9/unix/../generic/tkInt.h:19,
                 from /root/archiconda3/conda-bld/tk_1547479430536/work/tk8.6.9/unix/../generic/tkArgv.c:14:
/root/archiconda3/conda-bld/tk_1547479430536/work/tk8.6.9/unix/../generic/tk.h:96:13: fatal error: X11/Xlib.h: No such file or directory
 #   include <X11/Xlib.h>

https://app.shippable.com/github/Archiconda/tk-feedstock/runs/9/1/console

libarchive is failing

help requested from upstream:

conda-forge/libarchive-feedstock#25

make is still broken

whenever I try to compile make, it then fails at parsing the m4 makefiles.

I don't know how jjhelmus did it.

I have a feeling it is related to locales, but I can't be too sure right now.

I just took his package and uploaded it to my channel..... it is funny how the most core package is the hardest to build right....

conda-build dependencies

zstd: failed on https://app.shippable.com/github/Archiconda/zstd-feedstock/runs/1/1/console
- Needs 1.3.3 https://app.shippable.com/github/Archiconda/zstd-feedstock/dashboard

Error running link command: No such file or directory
make[2]: *** [lib/CMakeFiles/libzstd_static.dir/build.make:460: lib/libzstd.a] Error 2
make[1]: *** [CMakeFiles/Makefile2:128: lib/CMakeFiles/libzstd_static.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:118: all] Error 2

probably just missing setting ${AR} correctly.

libarchive
- waiting for 1.3.3 https://app.shippable.com/github/Archiconda/libarchive-feedstock/dashboard
pyyaml needed cython, should get built. https://app.shippable.com/github/Archiconda/pyyaml-feedstock/dashboard

Android termux install faild

Archiconda3 will now be installed into this location:
/data/data/com.termux/files/home/archiconda3

  - Press ENTER to confirm the location
  - Press CTRL-C to abort the installation
  - Or specify a different location below

[/data/data/com.termux/files/home/archiconda3] >>> 
PREFIX=/data/data/com.termux/files/home/archiconda3
installing: python-3.7.1-h39be038_1002 ...
./Archiconda3-0.2.3-Linux-aarch64.sh: 342: /data/data/com.termux/files/home/archiconda3/bin/python: not found
~ $ 
~ $ ls -alh /data/data/com.termux/files/home/archiconda3/bin/C
ls: cannot access '/data/data/com.termux/files/home/archiconda3/bin/C': No such file or directory
~ $ ls -alh /data/data/com.termux/files/home/archiconda3/bin/
total 44K
drwx------  2 u0_a131 u0_a131 4.0K Jun  5 22:14 .
drwx------ 10 u0_a131 u0_a131 4.0K Jun  5 22:14 ..
lrwxrwxrwx  1 u0_a131 u0_a131    8 Feb 26  2019 2to3 -> 2to3-3.7
-rwx------  1 u0_a131 u0_a131  123 Feb 26  2019 2to3-3.7
lrwxrwxrwx  1 u0_a131 u0_a131    7 Feb 26  2019 idle3 -> idle3.7
-rwx------  1 u0_a131 u0_a131  121 Feb 26  2019 idle3.7
lrwxrwxrwx  1 u0_a131 u0_a131    8 Feb 26  2019 pydoc -> pydoc3.7
lrwxrwxrwx  1 u0_a131 u0_a131    8 Feb 26  2019 pydoc3 -> pydoc3.7
-rwx------  1 u0_a131 u0_a131  106 Feb 26  2019 pydoc3.7
lrwxrwxrwx  1 u0_a131 u0_a131    9 Feb 26  2019 python -> python3.7
lrwxrwxrwx  1 u0_a131 u0_a131    9 Feb 26  2019 python3 -> python3.7
lrwxrwxrwx  1 u0_a131 u0_a131   17 Feb 26  2019 python3-config -> python3.7m-config
-rwx------  1 u0_a131 u0_a131  14K Feb 26  2019 python3.7
lrwxrwxrwx  1 u0_a131 u0_a131   17 Feb 26  2019 python3.7-config -> python3.7m-config
lrwxrwxrwx  1 u0_a131 u0_a131    9 Feb 26  2019 python3.7m -> python3.7
-rwx------  1 u0_a131 u0_a131 3.3K Feb 26  2019 python3.7m-config
lrwxrwxrwx  1 u0_a131 u0_a131   10 Feb 26  2019 pyvenv -> pyvenv-3.7
-rwx------  1 u0_a131 u0_a131  463 Feb 26  2019 pyvenv-3.7
~ $ /data/data/com.termux/files/home/archiconda3/bin/py
pydoc              pydoc3.7           python3            python3.7          python3.7m         pyvenv             
pydoc3             python             python3-config     python3.7-config   python3.7m-config  pyvenv-3.7         
~ $ /data/data/com.termux/files/home/archiconda3/bin/py
pydoc              pydoc3.7           python3            python3.7          python3.7m         pyvenv             
pydoc3             python             python3-config     python3.7-config   python3.7m-config  pyvenv-3.7         
~ $ /data/data/com.termux/files/home/archiconda3/bin/python
-bash: /data/data/com.termux/files/home/archiconda3/bin/python: No such file or directory

Why can't the compiler create executables

@jjhelmus Thanks for all your help bootstrapping.

Do you know why I'm unable to create executables with the compilers you provided:

checking whether the C compiler works... no
configure: error: in `/root/archiconda3/conda-bld/make_1546046933373/work':
configure: error: C compiler cannot create executables
See `config.log' for more details

https://app.shippable.com/github/Archiconda/make-make-feedstock/runs/1/1/console

I had to rename the repository make-make-feedstock because I deleted it by accident and shippable freaked out that a new repo was created with the same "old name" as before.

drop 2.7?

Archiconda automatically activate base environment

Hi~
I use archiconda in ubuntu arm version and along with an aarch64 CPU. I found something interesting:

After installed, python3.7 in archiconda is activated by default(while system version is 3.5)
Though the python is activated, but there is not an '(base)' hint.
No "auto_activate_base" option in conda config --show.

Because of this, the terminal startup appears to be slower than before. How can I fix that? I am not willing to edit the conda init script in .bashrc because I am not familiar with it.

Error building jupyterhub Docker image

I'm using the Archiconda3-0.2.3-Linux-aarch64.sh in the Dockerfile to install anaconda in the Jupyterhub build but getting some errors.

PackagesNotFoundError: The following packages are not available from current channels:

- configurable-http-proxy
- pycurl
- nodejs

Current channels:

- https://conda.anaconda.org/conda-forge/linux-aarch64
- https://conda.anaconda.org/conda-forge/noarch
- https://conda.anaconda.org/c4aarch64/linux-aarch64
- https://conda.anaconda.org/c4aarch64/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

https://anaconda.org

and use the search bar at the top of the page.

The command '/bin/sh -c wget -q https://github.com/Archiconda/build-tools/releases/download/0.2.3/Archiconda3-0.2.3-Linux-aarch64.sh -O /tmp/miniconda.sh && bash /tmp/miniconda.sh -f -b -p /opt/conda && /opt/conda/bin/conda install --yes -c conda-forge python=3.6 sqlalchemy tornado jinja2 traitlets requests pip pycurl nodejs configurable-http-proxy && /opt/conda/bin/pip install --upgrade pip && rm /tmp/miniconda.sh' returned a non-zero code: 1

Any idea if there is a way to get around this?

Optimized python builds

@jjhelmus my optimzied python builds timed out.

https://app.shippable.com/github/Archiconda/python-feedstock/dashboard

I'm also unsure about doing the Optimization on a CI. Seems a little spooky especially since this CI has a 96 core machine!!!!

I assume they have dual cravium processors
https://www.marvell.com/server-processors/thunderx-arm-processors/

Do you want to run the builds on your hardware?

python 2.7 failures

self = <numpy.core.tests.test_umath.TestComplexFunctions object at 0xffffad4c2690>

    @pytest.mark.skipif(is_longdouble_finfo_bogus(),
                        reason="Bogus long double finfo")
    def test_loss_of_precision_longcomplex(self):
>       self.check_loss_of_precision(np.longcomplex)

self       = <numpy.core.tests.test_umath.TestComplexFunctions object at 0xffffad4c2690>

../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python2.7/site-packages/numpy/core/tests/test_umath.py:2597: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python2.7/site-packages/numpy/core/tests/test_umath.py:2547: in check_loss_of_precision
Tests failed for numpy-1.15.4-py27_blas_openblashfae12ec_1000.tar.bz2 - moving package to /root/archiconda3/conda-bld/broken
WARNING:conda_build.build:Tests failed for numpy-1.15.4-py27_blas_openblashfae12ec_1000.tar.bz2 - moving package to /root/archiconda3/conda-bld/broken
    check(x_series, 50*eps)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

x = array([1.00000000e-20, 1.21736864e-20, 1.48198641e-20, 1.80412378e-20,
       ...871e-04, 6.73218092e-04, 8.19554595e-04, 9.97700064e-04],
      dtype=float128)
rtol = 9.629649721936179265279889712924637e-33

    def check(x, rtol):
        x = x.astype(real_dtype)
    
        z = x.astype(dtype)
        d = np.absolute(np.arcsinh(x)/np.arcsinh(z).real - 1)
        assert_(np.all(d < rtol), (np.argmax(d), x[np.argmax(d)], d.max(),
>                                 'arcsinh'))
E       AssertionError: (120, 1.7809563130653488765574048895753284e-10, 5.286342315078822882670006871837315e-21, 'arcsinh')

d          = array([0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
       ...000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00],
      dtype=float128)
dtype      = <type 'numpy.complex256'>
real_dtype = dtype('float128')
rtol       = 9.629649721936179265279889712924637e-33
x          = array([1.00000000e-20, 1.21736864e-20, 1.48198641e-20, 1.80412378e-20,
       ...871e-04, 6.73218092e-04, 8.19554595e-04, 9.97700064e-04],
      dtype=float128)
z          = array([1.00000000e-20+0.j, 1.21736864e-20+0.j, 1.48198641e-20+0.j,
       1.80...8092e-04+0.j,
       8.19554595e-04+0.j, 9.97700064e-04+0.j], dtype=complex256)

../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python2.7/site-packages/numpy/core/tests/test_umath.py:2520: AssertionError