anaconda-graveyard / anaconda-build Goto Github PK

View Code? Open in Web Editor NEW

2.0 2.0 4.0 1.03 MB

Conda server build --moved from Anaconda-Platform/anaconda-build

Python 93.34% Shell 3.47% Batchfile 3.19%

anaconda-build's People

Contributors

Stargazers

Watchers

Forkers

jakirkham digideskio kkabhi yutiansut

anaconda-build's Issues

Growing up size of log files (stdout)

From @mutirri on June 10, 2015 14:17

For example current size of logs equals to: 455MB, 645MB, 374MB, 404MB.
It cause a little delay when using chalmers log functionality (and it is going to proceed).
Checking old output of submissions also takes a little bit more time.
Besides, very long output from some building processes just hangs my browser (web interface).

Copied from original issue: Anaconda-Server/binstar-build-worker#11

NameError attempting to run a build with a worker on my local machine

I am following the directions at: http://docs.binstar.org/build-config.html#BuildWorkers

I set up a queue for dynd with

$ binstar-build queue --create dynd/q1

Then I started the build worker:

$ binstar-build worker dynd/q1
Using binstar api site https://api.anaconda.org
Starting worker:
User: dynd
Queue: q1
Platform: linux-64
Working Forever
Worker is waiting for the next job

Then I submitted a build from another terminal:

$ binstar-build submit . --queue dynd/q1

The following output came out of the build worker:

Starting build, 55a7e96094b2c15deecd93c1, dynd/libdynd#15.0
Setting alarm to terminate build after 216000 seconds
Building on worker pumpkin (platform linux-64)
Starting build dynd/libdynd#15.0
Fetching build data
Wrote build data to build_data/55a7e96094b2c15deecd93c1.tar.bz2
Running command: (iotimeout=300)
/home/gmarkall/work/bsb-worker/build_scripts/55a7e96094b2c15deecd93c1.sh --api-token dy-024442a6-6354-44a4-959c-59ab342ca789 --build-tarball /home/gmarkall/work/bsb-worker/build_data/55a7e96094b2c15deecd93c1.tar.bz2
Build Error: An unhandled exception occurred in the build worker---
Traceback (most recent call last):
  File "/home/gmarkall/miniconda3/lib/python3.4/site-packages/binstar_build_client/worker/worker.py", line 211, in build
    instructions=instructions)
  File "/home/gmarkall/miniconda3/lib/python3.4/site-packages/binstar_build_client/worker/worker.py", line 259, in run
    p0 = BufferedPopen(args, stdout=build_log, iotimeout=iotimeout, cwd=working_dir)
  File "/home/gmarkall/miniconda3/lib/python3.4/site-packages/binstar_build_client/worker/utils/buffered_io.py", line 42, in __init__
    self._iostream = IOStream(self.stdout, self._output, iotimeout, timeout, self.timeout_callback)
  File "/home/gmarkall/miniconda3/lib/python3.4/site-packages/binstar_build_client/worker/utils/streamio.py", line 35, in __init__
    if isinstance(line_iterator, file):
NameError: name 'file' is not defined

---[NameError] name 'file' is not defined 
Traceback (most recent call last):
  File "/home/gmarkall/miniconda3/lib/python3.4/site-packages/binstar_build_client/worker/worker.py", line 138, in _handle_job
    failed, status = self.build(job_data)
  File "/home/gmarkall/miniconda3/lib/python3.4/site-packages/binstar_build_client/worker/worker.py", line 211, in build
    instructions=instructions)
  File "/home/gmarkall/miniconda3/lib/python3.4/site-packages/binstar_build_client/worker/worker.py", line 259, in run
    p0 = BufferedPopen(args, stdout=build_log, iotimeout=iotimeout, cwd=working_dir)
  File "/home/gmarkall/miniconda3/lib/python3.4/site-packages/binstar_build_client/worker/utils/buffered_io.py", line 42, in __init__
    self._iostream = IOStream(self.stdout, self._output, iotimeout, timeout, self.timeout_callback)
  File "/home/gmarkall/miniconda3/lib/python3.4/site-packages/binstar_build_client/worker/utils/streamio.py", line 35, in __init__
    if isinstance(line_iterator, file):
NameError: name 'file' is not defined

Build Duration 5 seconds
Worker is waiting for the next job

and the output of the build is at https://anaconda.org/dynd/libdynd/builds/15/0

What went wrong? Is there some way around this issue?

More info about available hardware resources available for each of build-worker

Some users ask if they could get more information, like:
a) available total free disk space dedicated for build process,
b) available memory dedicated for build process,

The good place to present those information would be probably the below page:
https://anaconda.org/settings/build-queue

Visual Studio 2013 needed on Windows builders

See:
https://anaconda.org/antoine/llvmlite/builds/20/3

(note the Express/Community or SDK version is probably enough, no need for the full thing)

python setup.py install fails

python setup.py develop/install fails with the following error message

Searching for conda-server
Reading https://pypi.python.org/simple/conda-server/
Couldn't find index page for 'conda-server' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
No local packages or download links found for conda-server
error: Could not find suitable distribution for Requirement.parse('conda-server')

Given that there is no conda-server package on pypi, I'm confused why this was merged? Any advice on what to use instead? Revert this back to binstar?

More output on submit

Overlapping css at http://docs.binstar.org/draft/build/build_cli.html

Build Worker - pyOpenSSL unicode error on python2

[Build Targets]
binstar -q -t %TOKEN% upload --force --user binstar --package chalmers  --channe
l dev C:\Users\binstar\Miniconda\conda-bld\win-64\*.tar.bz2 --build-id 167
Using binstar api site https://api.binstar.org
[TypeError] data must be a byte string
Traceback (most recent call last):
  File "C:\Users\binstar\Miniconda\Scripts\binstar-script.py", line 9, in <modul
e>
    load_entry_point('binstar==0.10.1', 'console_scripts', 'binstar')()
  File "C:\Users\binstar\Miniconda\lib\site-packages\binstar_client\scripts\cli.
py", line 94, in main
    description=__doc__, version=version)
  File "C:\Users\binstar\Miniconda\lib\site-packages\binstar_client\scripts\cli.
py", line 76, in binstar_main
    return args.main(args)
  File "C:\Users\binstar\Miniconda\lib\site-packages\binstar_client\commands\upl
oad.py", line 202, in main
    callback=upload_print_callback(args))
  File "C:\Users\binstar\Miniconda\lib\site-packages\binstar_client\__init__.py"
, line 423, in upload
    s3res = requests.post(s3url, data=data_stream, verify=self.session.verify, t
imeout=10 * 60 * 60, headers=headers)
  File "C:\Users\binstar\Miniconda\lib\site-packages\requests\api.py", line 108,
 in post
    return request('post', url, data=data, json=json, **kwargs)
  File "C:\Users\binstar\Miniconda\lib\site-packages\requests\api.py", line 50,
in request
    response = session.request(method=method, url=url, **kwargs)
  File "C:\Users\binstar\Miniconda\lib\site-packages\requests\sessions.py", line
 464, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\binstar\Miniconda\lib\site-packages\requests\sessions.py", line
 576, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\binstar\Miniconda\lib\site-packages\requests\adapters.py", line
 370, in send
    timeout=timeout
  File "C:\Users\binstar\Miniconda\lib\site-packages\requests\packages\urllib3\c
onnectionpool.py", line 544, in urlopen
    body=body, headers=headers)
  File "C:\Users\binstar\Miniconda\lib\site-packages\requests\packages\urllib3\c
onnectionpool.py", line 349, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "C:\Users\binstar\Miniconda\lib\httplib.py", line 995, in request
    self._send_request(method, url, body, headers)
  File "C:\Users\binstar\Miniconda\lib\httplib.py", line 1029, in _send_request
    self.endheaders(body)
  File "C:\Users\binstar\Miniconda\lib\httplib.py", line 991, in endheaders
    self._send_output(message_body)
  File "C:\Users\binstar\Miniconda\lib\httplib.py", line 848, in _send_output
    self.send(message_body)
  File "C:\Users\binstar\Miniconda\lib\httplib.py", line 817, in send
    self.sock.sendall(datablock)
  File "C:\Users\binstar\Miniconda\lib\site-packages\requests\packages\urllib3\c
ontrib\pyopenssl.py", line 225, in sendall
    sent = self._send_until_done(data)
  File "C:\Users\binstar\Miniconda\lib\site-packages\requests\packages\urllib3\c
ontrib\pyopenssl.py", line 215, in _send_until_done
    return self.connection.send(data)
  File "C:\Users\binstar\Miniconda\lib\site-packages\OpenSSL\SSL.py", line 947,
in send
    raise TypeError("data must be a byte string")
TypeError: data must be a byte string

Binstar Cron tasks

Anaconda server cannot find git

The documentation (http://docs.binstar.org/examples.html) tells me that "binstar build MY_GIT_REPOSITORY" should work. So, when I try "binstar-build submit https://github.com/libdynd/dynd-python" (which is a repository that has a working conda recipe), I get "git: command not found".

https://binstar.org/izaid/dynd-python/builds/6/0

Build: selector expression ignored on .binstar.yml

The following seems to suggest that the .binstar.yml file does not respond to platform selector.
I'm building for linux64 hence do not expect that the conda install -c https://conda.binstar.org/jf freeimage command is executed

## Run before the script
before_script:
   - echo "before_script!"
   - conda install -c https://conda.binstar.org/mutirri freeimage # [linux64]
# mehhhh... does not respond to selector?
   - conda install -c https://conda.binstar.org/jf freeimage # [osx]

resulting binstar build

269Extracting packages ...
270 |                                                                       |   0%
271[freeimage           ] |################                                 |  33%
272[jpeg                ] |################################                 |  66%
273[libtiff             ] |#################################################| 100%
274[      COMPLETE      ] |#################################################| 100%
275Linking packages ...
276[      COMPLETE      ] |                                                 |   0%
277[freeimage           ] |################                                 |  33%
278[jpeg                ] |################################                 |  66%
279[libtiff             ] |#################################################| 100%
280[      COMPLETE      ] |#################################################| 100%
281conda install -c https://conda.binstar.org/jf freeimage
282Fetching package metadata: ...
283Error: No packages found in current linux-64 channels matching: freeimage

rename git repo from conda-server-build -> anaconda-build

Renaming the github repo from conda-server-build to anaconda-build would make sense.
The command line tool binstar-build should also be called anaconda-build. I'm not sure what to call conda-clean-build-dir.

patch is missing in docker linux-64 image

some of my recipes use the patch mechanism in the conda build. For this patch needs to be installed:

yum install -y patch

Maybe you would like to add it to the dockerfile ...

Probably somehow broken 'unregister' feature - invalid status on web page interface

From @mutirri on June 10, 2015 14:9

We have one win32 machine with build-worker (as far as I know).
On this machine was just one running binstar process: binstar-build.exe.

However on the page: https://binstar.org/organization/binstar/settings/build-queue
I saw two available workers.
Maybe chalmers restart -a command didn't properly perform unregister action.
Or maybe refresh time is too small.

Copied from original issue: Anaconda-Server/binstar-build-worker#9

Build logs are truncated either on anaconda.org or through the command line

This has been an issue for me since August 7. Basically, I can no longer see the entire build log of any package. I don't of any way to get at the build log.

See https://anaconda.org/dynd/libdynd/builds/130/0, for instance.

Specify dependency channels

My project depends on other packages in other channels. How do I reference these in my build/test script?

E.g. development Blaze depends on development versions of DyND and datashape. These are also hosted on binstar, how can I point to these versions in my conda recipe or binstar.yml script?

Some builders cannot decompress .xz files

See:
https://anaconda.org/antoine/llvmlite/builds/20/2

One possibility is to install "unxz" on non-Linux builders. Another possibility is to switch to a Python 3 version of conda, since Python 3 has xz / lzma compression bundled in the stdlib.

anaconda-build cannot download long messages

Using my dynd-python package as an example, "anaconda-build tail dynd/dynd-python 25". This gives me a number of errors like:

Traceback (most recent call last):
File "/home/irwin/miniconda/lib/python2.7/logging/handlers.py", line 862, in emit
self.socket.send(msg)
error: [Errno 90] Message too long
Logged from file info.py, line 37

AttributeError: 'NoneType' object has no attribute 'user' with binstar-build tail

$binstar-build tail 26.0
Using binstar api site http://api.alpha.binstar.org
[AttributeError] 'NoneType' object has no attribute 'user'
Traceback (most recent call last):
  File "/Users/aaronmeurer/anaconda/envs/binstar-build/bin/binstar-build", line 9, in <module>
    load_entry_point('binstar-build==0.10.0', 'console_scripts', 'binstar-build')()
  File "/Users/aaronmeurer/anaconda/envs/binstar-build/lib/python2.7/site-packages/binstar_build_client/scripts/build.py", line 31, in main
    version=version)
  File "/Users/aaronmeurer/anaconda/envs/binstar-build/lib/python2.7/site-packages/binstar_client/scripts/cli.py", line 76, in binstar_main
    return args.main(args)
  File "/Users/aaronmeurer/anaconda/envs/binstar-build/lib/python2.7/site-packages/binstar_build_client/commands/info.py", line 35, in tail
    log_items = binstar.tail_build(args.package.user, args.package.name, args.build_no, limit=args.n)
AttributeError: 'NoneType' object has no attribute 'user'

Logging error at the end of binstar-build tail -f

[Test]
cd tests\test-recipes\metadata
for /D %f in (*) do (C:\Users\binstar\conda-build-miniconda\conda-build --no-binstar-upload %~nf)
'C:\Users\binstar\conda-build-miniconda\conda-build' is not recognized as an internal or external command,
operable program or batch file.
'C:\Users\binstar\conda-build-miniconda\conda-build' is not recognized as an internal or external command,
operable program or batch file.
'C:\Users\binstar\conda-build-miniconda\conda-build' is not recognized as an internal or external command,
operable program or batch file.
'C:\Users\binstar\conda-build-miniconda\conda-build' is not recognized as an internal or external command,
operable program or batch file.
'C:\Users\binstar\conda-build-miniconda\conda-build' is not recognized as an internal or external command,
operable program or batch file.
'C:\Users\binstar\conda-build-miniconda\conda-build' is not recognized as an internal or external command,
operable program or batch file.
'C:\Users\binstar\conda-build-miniconda\conda-build' is not recognized as an internal or external command,
operable program or batch file.
'C:\Users\binstar\conda-build-miniconda\conda-build' is not recognized as an internal or external command,
operable program or batch file.
'C:\Users\binstar\conda-build-miniconda\conda-build' is not recognized as an internal or external command,
operable program or batch file.
'C:\Users\binstar\conda-build-miniconda\conda-build' is not recognized as an internal or external command,
operable program or batch file.
'C:\Users\binstar\conda-build-miniconda\conda-build' is not recognized as an internal or external command,
operable program or batch file.
'C:\Users\binstar\conda-build-miniconda\conda-build' is not recognized as an internal or external command,
operable program or batch file.
Exit BINSTAR_BUILD_RESULT=failure

Traceback (most recent call last):
  File "/Users/aaronmeurer/anaconda/envs/binstar-build/lib/python2.7/logging/handlers.py", line 861, in emit
    self.socket.send(msg)
error: [Errno 40] Message too long
Logged from file info.py, line 46
[error] Build Failed

This is from binstar-build tail conda/conda-build 29.0 -f (https://binstar.org/conda/conda-build/builds/29/0)

Can't start binstar-build worker on win-32 machine

Hi,

After upgrade binstar-build to version 0.10.7 I can't start the service on win-32 machine:

[error] There server 'https://api.anaconda.org' returned an error response
[error] Could not retrieve work items
[ServerError] Service Unavailable: The server cannot process the request due to
a high load ([POST] https://api.anaconda.org/build-worker/binstar/continuum/5592
a43f22f2e9357d8bb23c/jobs -> 503)
Traceback (most recent call last):
  File "c:\users\binstar\miniconda\lib\site-packages\binstar_build_client\worker
\worker.py", line 99, in job_loop
    job_data = bs.pop_build_job(args.username, args.queue, self.worker_id)
  File "c:\users\binstar\miniconda\lib\site-packages\binstar_build_client\mixins
\build_queue.py", line 36, in pop_build_job
    self._check_response(res, [200])
  File "c:\users\binstar\miniconda\lib\site-packages\binstar_client\__init__.py"
, line 160, in _check_response
    raise ErrCls(msg, res.status_code)
ServerError: (u'Service Unavailable: The server cannot process the request due t
o a high load ([POST] https://api.anaconda.org/build-worker/binstar/continuum/55
92a43f22f2e9357d8bb23c/jobs -> 503)', 503)

[error] There server 'https://api.anaconda.org' returned an error response
[error] Could not retrieve work items

Machine has connection to APIs:

C:\Users\binstar>curl -I api.binstar.org
HTTP/1.1 301 Moved Permanently
Content-Length: 184
Content-Type: text/html
Date: Wed, 08 Jul 2015 11:15:57 GMT
Location: https://api.binstar.org/
Server: nginx/1.4.3
Connection: keep-alive


C:\Users\binstar>curl -I api.anaconda.org
HTTP/1.1 301 Moved Permanently
Content-Length: 184
Content-Type: text/html
Date: Wed, 08 Jul 2015 11:16:05 GMT
Location: https://api.anaconda.org/
Server: nginx/1.6.2
Connection: keep-alive

bug in binstar-build tail logging output

https://gist.github.com/tswicegood/cfbc0bbb2b20ffe9bd9d

Not sure what this is, but appears the messages need to be chunked up before getting sent over the wire.

Init service of build-worker for windows machines

From @mutirri on June 10, 2015 14:21

This is just my theory.
Some time ago there was no build-worker for one of windows machines.
I logged into machine however my session was started from scratch.
Of course build-worker wasn't running there.
I started it manually and then it showed up in the web interface.

I think that if anyone is closing the session from windows machines then this action should be done by terminating the connection only (don't perform logout or similar actions).
Other approaches than just mentioned one may simply cause that login session will be killed (with all its subprocesses).

Maybe instead of using binstar.bat file which is launched through some user account we should write real init service for windows machines (this should be independent of user session) - however it's a just lazy thought.

Copied from original issue: Anaconda-Server/binstar-build-worker#12

Code does not handle mixed-slash paths on Windows

In https://github.com/Binstar/binstar-build/blob/master/binstar_build_client/utils/__init__.py#L7, the code should be changed to call os.path.split().

I looked elsewhere in the codebase and found no other uses of pathsep so I think this is the only instance of this particular problem.

This problem led to a build failure here: https://binstar.org/javascript/mocha/builds/1/4. In case that build is removed, the log traceback is pasted below:

1Build 1.4 Sumitted id:54d946405e7683344fda70a9
2This build is in the build queue "build-binstar-continuum"...
3Building on worker WIN-6KBD8SQ4HHD (platform win-64)
4Starting build javascript/mocha#1.4
5Build Error: An unhandled exception occurred in the build worker---
6Traceback (most recent call last):
7  File "C:\Miniconda\lib\site-packages\binstar_build_client\worker\worker.py", line 185, in build
8    conda_build_dir=self.args.conda_build_dir)
9  File "C:\Miniconda\lib\site-packages\binstar_build_client\worker\utils\script_generator.py", line 178, in gen_build_script
10    exports = create_exports(build_data)
11  File "C:\Miniconda\lib\site-packages\binstar_build_client\worker\utils\script_generator.py", line 113, in create_exports
12    conda_root_prefix = get_conda_root_prefix()
13  File "C:\Miniconda\lib\site-packages\binstar_build_client\utils\__init__.py", line 8, in get_conda_root_prefix
14    if conda_ext in os.listdir(entry):
15WindowsError: [Error 3] The system cannot find the path specified: 'C:\\Users\\binstar\\conda-build-miniconda\\Scripts/*.*'

Python 3 support

Python 3 support would be great.

Errors while performing 'rmtree' on non empty directories

From @mutirri on June 10, 2015 14:25

It seems that some cleaning functions - in particular conditions - can't be finished with success:

...
Build Error: An unhandled exception occurred in the build worker---
Traceback (most recent call last):
  File "C:\Users\binstar\Miniconda\lib\site-packages\binstar_build_client\worker\worker.py", line 211, in build
    instructions=instructions)
  File "C:\Users\binstar\Miniconda\lib\site-packages\binstar_build_client\worker\worker.py", line 241, in run
    rm_rf(working_dir)
  File "C:\Users\binstar\Miniconda\lib\site-packages\binstar_build_client\utils\rm.py", line 47, in rm_rf
    shutil.rmtree(path)
  File "C:\Users\binstar\Miniconda\lib\shutil.py", line 247, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "C:\Users\binstar\Miniconda\lib\shutil.py", line 256, in rmtree
    onerror(os.rmdir, path, sys.exc_info())
  File "C:\Users\binstar\Miniconda\lib\shutil.py", line 254, in rmtree
    os.rmdir(path)
WindowsError: [Error 145] The directory is not empty: u'C:\\Users\\binstar\\worker\\builds\\conda\\conda-env\\env'
...

Copied from original issue: Anaconda-Server/binstar-build-worker#13

Inside of a docker container, anaconda-build hangs on `conda clean -pt`

I'm trying to spin up a docker container to serve as a binstar-builder for our internal anaconda/binstar server. The builder is working (sort of), in that it is successfully receiving build commands from the anaconda/binstar server when I submit a binstar build request, but it is hanging when conda clean -pt is executed. What's the best way to deal with this?

I would imagine that this can be fixed by changing this to conda clean -pty? I'm happy to put in a PR to fix this, but I wanted to get your feedback on this first.

Using binstar api site https://conda.nsls2.bnl.gov/api
Starting worker:
User: latest
Queue: mac-builder
Platform: linux-64
Working Forever
Worker is waiting for the next job
Starting build, 55ccfa54cbcbd44eec1d73ac, latest/nslsii_dev_configuration#3.0
Setting alarm to terminate build after 216000 seconds
Building on worker 73d1d0120938 (platform linux-64)
Starting build latest/nslsii_dev_configuration#3.0
Fetching build data
Wrote build data to build_data/55ccfa54cbcbd44eec1d73ac.tar.bz2
Running command: (iotimeout=60)
/build_scripts/55ccfa54cbcbd44eec1d73ac.sh --api-token la-4ef5ec2b-c49d-4d3e-ad98-77ddf57bfd24 --build-tarball /build_data/55ccfa54cbcbd44eec1d73ac.tar.bz2

[Setup Build]
Host: 73d1d0120938
Setting engine
conda clean -pt
Cache location: /opt/conda/pkgs
Will remove the following tarballs:

pip-7.1.0-py27_1.tar.bz2                     1.4 MB
requests-2.7.0-py27_0.tar.bz2                594 KB
anaconda-client-1.0.1-py27_0.tar.bz2          90 KB
anaconda-build-0.10.7-py27_0.tar.bz2          65 KB
pytz-2015.4-py27_0.tar.bz2                   174 KB
conda-env-2.4.1-py27_0.tar.bz2                24 KB
psutil-2.2.1-py27_0.tar.bz2                   98 KB
pillow-2.9.0-py27_0.tar.bz2                  781 KB
six-1.9.0-py27_0.tar.bz2                      17 KB
wheel-0.24.0-py27_0.tar.bz2                   75 KB
jinja2-2.8-py27_0.tar.bz2                    263 KB
markupsafe-0.23-py27_0.tar.bz2                30 KB
pycosat-0.6.1-py27_0.tar.bz2                 151 KB
setuptools-18.0.1-py27_0.tar.bz2             341 KB
clyent-0.4.0-py27_0.tar.bz2                   14 KB
python-2.7.10-0.tar.bz2                     13.5 MB
conda-3.16.0-py27_0.tar.bz2                  172 KB
pyyaml-3.11-py27_1.tar.bz2                   295 KB
python-dateutil-2.4.2-py27_0.tar.bz2         219 KB
conda-build-1.16.0-py27_0.tar.bz2            198 KB
conda-3.16.0-py34_0.tar.bz2                  176 KB
yaml-0.1.6-0.tar.bz2                         246 KB
pytz-2015.4-py34_0.tar.bz2                   173 KB
setuptools-18.0.1-py34_0.tar.bz2             345 KB
libpng-1.6.17-0.tar.bz2                      214 KB
psutil-2.2.1-py34_0.tar.bz2                  102 KB
freetype-2.5.2-2.tar.bz2                     2.5 MB
jpeg-8d-0.tar.bz2                            699 KB
pillow-2.9.0-py34_0.tar.bz2                  805 KB
conda-build-1.16.0-py34_0.tar.bz2            200 KB
wheel-0.24.0-py34_0.tar.bz2                   76 KB
markupsafe-0.23-py34_0.tar.bz2                31 KB
python-dateutil-2.4.2-py34_0.tar.bz2         221 KB
pip-7.1.0-py34_1.tar.bz2                     1.4 MB
six-1.9.0-py34_0.tar.bz2                      17 KB
jinja2-2.8-py34_0.tar.bz2                    301 KB
clyent-0.4.0-py34_0.tar.bz2                   15 KB
requests-2.7.0-py34_0.tar.bz2                623 KB
python-3.4.3-1.tar.bz2                      23.1 MB
conda-env-2.4.1-py34_0.tar.bz2                24 KB
libtiff-4.0.2-1.tar.bz2                      930 KB
pyyaml-3.11-py34_1.tar.bz2                   330 KB
anaconda-build-0.10.7-py34_0.tar.bz2          66 KB
patchelf-0.6-0.tar.bz2                       179 KB
anaconda-client-1.0.1-py34_0.tar.bz2          92 KB
------------------------------------------------
Total:                                      51.1 MB


Timeout: No output from program for 60 seconds

Timeout: If you require a longer timeout you may set the 'iotimeout' variable in your .binstar.yml file
[Terminating]
Proceed ([y]/n)? Closing subprocess stdout PIPE
Build script exited with code -9
[error] Unknown build exit status -9 for build latest/nslsii_dev_configuration#3.0

"binstar-build init" from docs doesn't work

http://docs.binstar.org/continuous_integration.html says:

In order for Binstar to build your project, you need add a .binstar.yml file to the root of your repository. To do this run binstar-build init in the root repository commit the file and push to the github repository. See all of the .binstar.yml configuration options.

but if I try this:

[py27] D:\Develop\dynd-python\libraries\libdynd>binstar-build init
'binstar-build' is not recognized as an internal or external command,
operable program or batch file.

[py27] D:\Develop\dynd-python\libraries\libdynd>binstar build init
usage: binstar-script.py build [-h]
                               (-l LIST | -a | -t TAIL | -s | --resubmit RESUBMIT | --dry-run | --halt build_id | --halt-all)
                               [-n #] [-f]
                               [path] [OWNER/PACKAGE]
binstar-script.py build: error: one of the arguments -l/--list -a/--list-all -t/--tail -s/--submit --resubmit --dry-run --halt/--stop --halt-all/--stop-all is required

Build worker: Too many open files

Build 15.10 Sumitted id:54fdad345e76834c30c1422c
This build is in the build queue "build-binstar-continuum"...
Building on worker 0021-osxserver.local (platform osx-64)
Starting build blaze/odo#15.10
Build Error: An unhandled exception occurred in the build worker---
Traceback (most recent call last):
  File "/opt/anaconda/lib/python2.7/site-packages/binstar_build_client/worker/worker.py", line 205, in build
    instructions=instructions)
  File "/opt/anaconda/lib/python2.7/site-packages/binstar_build_client/worker/worker.py", line 253, in run
    p0 = BufferedPopen(args, stdout=build_log, iotimeout=iotimeout, cwd=working_dir)
  File "/opt/anaconda/lib/python2.7/site-packages/binstar_build_client/worker/utils/buffered_io.py", line 40, in __init__
    **kwargs)
  File "/opt/anaconda/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/opt/anaconda/lib/python2.7/subprocess.py", line 1223, in _execute_child
    errpipe_read, errpipe_write = self.pipe_cloexec()
  File "/opt/anaconda/lib/python2.7/subprocess.py", line 1175, in pipe_cloexec
    r, w = os.pipe()
OSError: [Errno 24] Too many open files

binstar-build does not accept `about: readme:` in `meta.yaml`

The binstar-build clients seems to suggest that the following is invalid inside the conda meta.yaml

about:
   - readme: Readme.md

Although this option is not documented (possibly a reason why there's so little detail on packages on binstar.org), it works perfectly fine in a local conda build.

I'm sure there's valid reasons why binstar-build checks meta.yaml but should it not be primarily up to conda build to decide what's valid and what's not. Not sure if the readme option is relatively new to conda build?

Need a way to specify conda channels

If some of the engines in your matrix need version-specific dependencies, and those dependencies are not in the main conda channels, you are stuck because you cannot set channels at this point.

An example is the Numba build matrix, where "singledispatch" is available in the numba channel, but not in the main channel:

engine:
 - python=2.6 argparse funcsigs unittest2 singledispatch
 - python=2.7 funcsigs singledispatch
 - python=3.3 singledispatch
 - python=3.4

There should be a binstar.yml option to enable additional channels. That would also be more user-friendly than using a "conda config" command line.

Add conda-server-build to anaconda channel

Now that binstar is being replaced by conda-server, it makes sense that binstar-build be replaced by conda-server-build. However, this does not yet exist in the anaconda channel. Can this please be added?

Also, this line needs to be changed ( https://github.com/Anaconda-Server/conda-server-build/blob/develop/conda.recipe/conda.yaml#L19 ).

Error during automatic start up of new submission

All machines reported the same message when crashed:

Using binstar api site https://api.anaconda.org
Starting worker:
User: binstar
Queue: continuum
Platform: linux-64
Working Forever
Removing worker 55b80a4b3664bf6a74552aa4
[NotFound] Not Found: Nothing matches the given URI ([POST] https://api.anaconda.org/build-worker/binstar/continuum/55b80a4b3664bf6a74552aa4/jobs -> 404) 
Traceback (most recent call last):
  File "/home/binstar/miniconda/bin/binstar-build", line 6, in <module>
    sys.exit(main())
  File "/home/binstar/miniconda/lib/python2.7/site-packages/binstar_build_client/scripts/build.py", line 31, in main
    version=version)
  File "/home/binstar/miniconda/lib/python2.7/site-packages/binstar_client/scripts/cli.py", line 76, in binstar_main
    return args.main(args)
  File "/home/binstar/miniconda/lib/python2.7/site-packages/binstar_build_client/commands/worker.py", line 47, in main
    worker.work_forever()
  File "/home/binstar/miniconda/lib/python2.7/site-packages/binstar_build_client/worker/worker.py", line 79, in work_forever
    self._build_loop()
  File "/home/binstar/miniconda/lib/python2.7/site-packages/binstar_build_client/worker/worker.py", line 180, in _build_loop
    for job_data in self.job_loop():
  File "/home/binstar/miniconda/lib/python2.7/site-packages/binstar_build_client/worker/worker.py", line 99, in job_loop
    job_data = bs.pop_build_job(args.username, args.queue, self.worker_id)
  File "/home/binstar/miniconda/lib/python2.7/site-packages/binstar_build_client/mixins/build_queue.py", line 36, in pop_build_job
    self._check_response(res, [200])
  File "/home/binstar/miniconda/lib/python2.7/site-packages/binstar_client/__init__.py", line 160, in _check_response
    raise ErrCls(msg, res.status_code)
NotFound: (u'Not Found: Nothing matches the given URI ([POST] https://api.anaconda.org/build-worker/binstar/continuum/55b80a4b3664bf6a74552aa4/jobs -> 404)', 404)

The recipes that might cause some problems (they were submitted as last and are not properly cleaned up from worker internal build_scripts dir) are:
A)
user: @jreback
pkg recipe: pandas
B)
user: @pitrou
pkg recipe: numba

Maybe these recipes influence somehow on how binstar worker is behaving.

binstar-build workers need to be able to target more than one A-Cloud label

Using a single default channel for automatically built packages breaks down when you’re working on multiple versions of software. There needs to be a mechanism for having multiple channels, or automatically assigning a channel name.

Use Case

I’m starting to work on a backwards incompatible change to conda-env which means its version is going to get bumped to v3.0.0. Since builds are automatically uploaded to binstar, as soon as I push a change that contains v3.0.0alpha in it, that build wins all future updates for anyone tracking the conda/c/dev channel.

At one point, binstar-build used to match build channels to the branch name. I’m not sure why that disappeared, but it's keeping me from updating the version number in a branch appropriately.

Add -f option to submit

Allow users to wait until the build is finished before exiting submit command

References to binstar

Still several references to binstar below:

$ ~/miniconda3/bin/anaconda-build submit .
Using binstar api site https://api.anaconda.org
Getting build product: /home/antoine/numba
Submitting 10 sub builds
 0) linux-32    python=2.6 argparse funcsigs unittest2 singledispatch  None           
 1) linux-32    python=3.4       None           
 2) linux-64    python=2.6 argparse funcsigs unittest2 singledispatch  None           
 3) linux-64    python=3.4       None           
 4) osx-64      python=2.6 argparse funcsigs unittest2 singledispatch  None           
 5) osx-64      python=3.4       None           
 6) win-32      python=2.6 argparse funcsigs unittest2 singledispatch  None           
 7) win-32      python=3.4       None           
 8) win-64      python=2.6 argparse funcsigs unittest2 singledispatch  None           
 9) win-64      python=3.4       None           
Archiving build directory for upload ...
Created archive (538 files); Uploading to binstar
 uploaded 824 of 824Kb: 100.00% ETA: 0.0 minutes

To view this build go to http://alpha.binstar.org/antoine/numba/builds/matrix/117

You may also run

    binstar-build tail -f antoine/numba 117

The python interpreter errors on windows machines

From @mutirri on June 10, 2015 14:15

From time to time I saw errors directly related to python interpreter - during user submissions.

Those dialog boxes showed up for couple of seconds and then just disappeared.
There was at least one error related to some internal windows DLL file also.
However this window disappeared too fast and I couldn't take a screenshot.
I wasn't able to find any information related to this situation in chalmers logs.

Copied from original issue: Anaconda-Server/binstar-build-worker#10

Linux 64, Linux 32 and Windows boxes/queues are down

I dont know if this is the right place, let me know if its not and i will move it.

Something is down in anaconda.org the boxes, the queues, i dont know. They have been down for a while. I am getting this error in all my builds:

Build 233.1 Sumitted id:55942e063273af558716dd01
There are no available build workers in the queue build-binstar-continuum for the platform linux-64 (available win-32, osx-64, win-64)
Your build was not submitted

Git Checkout from the command line

Qt dependency missing(?) on linux worker

Build 9.0 Sumitted id:55dc94ac42b65d55c5c3269c
This build is in the build queue "build/binstar/public"...
Building on worker docker-1 (platform linux-64)
Starting build richardotis/pycalphad#9.0
Docker Image: continuumio/binstar-build-linux64

[snip]

===== testing package: pycalphad-0.2+14.g2bb370c-py27_0 =====
import: u'pycalphad'
Traceback (most recent call last):
  File "/opt/miniconda/conda-bld/test-tmp_dir/run_test.py", line 25, in <module>
    import pycalphad
  File "build/bdist.linux-x86_64/egg/pycalphad/__init__.py", line 7, in <module>
  File "build/bdist.linux-x86_64/egg/pycalphad/plot/isotherm.py", line 8, in <module>
  File "/opt/miniconda/envs/_test/lib/python2.7/site-packages/matplotlib/pyplot.py", line 109, in <module>
    _backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup()
  File "/opt/miniconda/envs/_test/lib/python2.7/site-packages/matplotlib/backends/__init__.py", line 32, in pylab_setup
    globals(),locals(),[backend_name],0)
  File "/opt/miniconda/envs/_test/lib/python2.7/site-packages/matplotlib/backends/backend_qt4agg.py", line 17, in <module>
    from .backend_qt5agg import NavigationToolbar2QTAgg
  File "/opt/miniconda/envs/_test/lib/python2.7/site-packages/matplotlib/backends/backend_qt5agg.py", line 18, in <module>
    from .backend_qt5 import QtCore
  File "/opt/miniconda/envs/_test/lib/python2.7/site-packages/matplotlib/backends/backend_qt5.py", line 31, in <module>
    from .qt_compat import QtCore, QtGui, QtWidgets, _getSaveFileName, __version__
  File "/opt/miniconda/envs/_test/lib/python2.7/site-packages/matplotlib/backends/qt_compat.py", line 91, in <module>
    from PyQt4 import QtCore, QtGui
ImportError: libXext.so.6: cannot open shared object file: No such file or directory
TESTS FAILED: pycalphad-0.2+14.g2bb370c-py27_0
command exited with status 1
discarding /home/binstar/env/bin from PATH
Exit BINSTAR_BUILD_RESULT=failure

Leaving garbage in the system when submission fails

From @mutirri on June 10, 2015 14:31

If recipe crashes for any reason (it doesn't matter what cause that) then not everything is properly cleaned. The great example is running phantomjs for testing purposes.
I had to free some resources by killing a lot of hanging processes (63 some time ago, 48 of them one week ago and 11 today).

One possible solution for Linux - to be sure if we clean up all stuff properly - might be cgroups framework. However this approach can't be used for Windows and probably it is not available for MacOSX also.

Copied from original issue: Anaconda-Server/binstar-build-worker#14

jinja2 missing on some builders

jinja2 is a requirement when using templating conda-build configuration (meta.yaml)

See:
https://anaconda.org/antoine/llvmlite/builds/19/3
https://anaconda.org/antoine/llvmlite/builds/19/4

" --channel main" instead of " --channel dev" ?

How do I make this function return --channel main?

Right now my .binstar.yml file has

build_targets:
  channels: main
  files: conda

But that does not appear to be picked up in the build process, as seen here

libstdc++ missing on some builders?

See:
https://anaconda.org/antoine/llvmlite/builds/19/1

"""/usr/bin/ld: cannot find -lstdc++"""

Missing requirement

$ pip install --user binstar-build
Downloading/unpacking binstar-build
[...]
Successfully installed binstar-build binstar clyent python-dateutil
Cleaning up...
$ binstar-build --help
[...]
  File "/home/antoine/.local/lib/python2.7/site-packages/binstar_build_client/worker/worker.py", line 13, in <module>
    import psutil
ImportError: No module named psutil

.binstar.yml not found

mrocklin@workstation:~/workspace/blaze$ binstar-build submit https://github.com/binstar/blaze --test-only --queue build-binstar-continuum Using binstar api site http://api.alpha.binstar.org
Using repo name 'blaze' as the pkg name.
Using binstar api site http://api.alpha.binstar.org
Submitting the following repo for package creation: https://github.com/binstar/blaze
[BinstarError] Configuration file ".binstar.yml" not found in this repository Please add one this file to submit builds 
mrocklin@workstation:~/workspace/blaze$ ls .binstar.yml 
.binstar.yml

A few notes

Error says that binstar.yml doesn't exist. I disagree
Small language nit "Please add one this file to submit builds" should maybe be "Please add this file to submit builds. See http://.... for help/examples.
Text Using binstar api site http://api.alpha.binstar.org is repeated

UI Feedback about binstar build

Hello,

Some comments:

build pages (http://alpha.binstar.org/antoine/numba/builds/32) have no option to cancel a build (if running) or retry it (if done / cancelled)
build pages don't show the submit date & time
build pages don't show the submit metadata (e.g. git changeset, or tarball (if kept on the server), or tarball hashsum)
build pages don't use the same icons and color codes as the global builds summary (http://alpha.binstar.org/antoine/numba/builds); I find the colors and icons on build pages confusing (red means finished?)
build pages don't give detailed info about the specific builders (OS version, CPU, architecture, etc.)

Regards

tail results in internal server errors.

binstar-build tail -f wakari/wakari-cloud-server-conf 2
[BinstarError] Internal server error

gcc too old on Linux builders

See:
https://anaconda.org/antoine/llvmlite/builds/20/0
https://anaconda.org/antoine/llvmlite/builds/19/0 (different error message, but probably same underlying reason)

"""
215The selected GCC C++ compiler is not new enough to build LLVM. Please upgrade

216to GCC 4.7. You may pass --disable-compiler-version-checks to configure to

217bypass these sanity checks.
"""

(would be nice to fix that log window for better copy/paste experience, btw :-))

Test step fails to install built package

See:
https://anaconda.org/antoine/llvmlite/builds/19/2

The package has just been built but weirdly fails installing:

"""
554BUILD END: llvmlite--py27_0

555TEST START: llvmlite--py27_0

556Fetching package metadata: ........

557Error: No packages found in current osx-64 channels matching: llvmlite py27_0

558command exited with status 1
"""