zestsoftware / zest.releaser Goto Github PK

In a buildout-proejct (non python-package), I moved my files to
docs/changes.rst
docs.version.rst

but this breaks my release (use latest version 3.53)

INFO: Starting prerelease.
Enter version [0.5]:
Traceback (most recent call last):
File "/home/marvin/bin/fullrelease", line 46, in
sys.exit(zest.releaser.fullrelease.main())
File "/apps/buildouts/tools/buildout/eggs/zest.releaser-3.53-py2.7.egg/zest/releaser/fullrelease.py", line 21, in main
prerelease.main()
File "/apps/buildouts/tools/buildout/eggs/zest.releaser-3.53-py2.7.egg/zest/releaser/prerelease.py", line 168, in main
prereleaser.run()
File "/apps/buildouts/tools/buildout/eggs/zest.releaser-3.53-py2.7.egg/zest/releaser/baserelease.py", line 26, in run
self.execute()
File "/apps/buildouts/tools/buildout/eggs/zest.releaser-3.53-py2.7.egg/zest/releaser/prerelease.py", line 63, in execute
self._write_version()
File "/apps/buildouts/tools/buildout/eggs/zest.releaser-3.53-py2.7.egg/zest/releaser/prerelease.py", line 84, in _write_version
self.vcs.version = self.data['new_version']
File "/apps/buildouts/tools/buildout/eggs/zest.releaser-3.53-py2.7.egg/zest/releaser/vcs.py", line 210, in _update_version
setup_lines = open('setup.py').read().split('\n')
IOError: [Errno 2] No such file or directory: 'setup.py'

problem is https://github.com/zestsoftware/zest.releaser/blob/master/zest/releaser/vcs.py#L194

the dev is strip only in VERSION.txt we should provide a list of valide VERSION files (VERSION.txt and VERSION) also the next version is set only in VERSION.txt and not VERSION

Using latest zest.releaser 3.52

In some projects we want to move CHANGES.txt and VERSION.txt to
docs/CHANGES.rst , docs/VERSION.rst as part of a sphinx-documentation,

while changes.rst is currently allowed, version.rst is not.

The PypiConfig.want_release method is designed to read a .pypirc or setup.cfg file and return False if it contains:

[zest.releaser]
release = no

which is the sole content of my setup.cfg file.

However PypiConfig._read_configfile does a check of validity:

    if (not self.is_old_pypi_config() and
            not self.is_new_pypi_config()):
        # Safety valve
        self.config = None

the is_old_pypi_config and is_new_pypi_config methods check for server-login:username and distutils:index-servers respectively in your config file. If you have neither of these section:keys you activate the "Safety valve" and get no pypi config.

This means I cannot prevent release (unless I add dummy sections, about servers... that I don't want to release to).

Not quite sure what the solution is. One backwards incompatible idea would be to move the want_release method to SetupConfig, and no longer support release = no in your .pypirc file?

So am I doing something wrong, or happy to write a patch, if there's a solution agreed.

It would be nice if zest.releaser could do some more checks before releasing a package:

• Check if package/version already exists on repository (pypi, plone.org, etc.)
• Check if tag already exists in code repo (git/svn)

I now it's easy to check this by your self, but it would be a nice feature for the lazy people out there ;-)

regards mat

Sample setup.py

import os
...

setup(name='foo',
...
      message_extractors={'.': [
          ('**.py', 'lingua_python', None),
          ('**.pt', 'lingua_xml', None),
          ('**.jst', 'lingua_python', None),
      ]},
...
      )

The resulting error:

You may want to quit and fix this.
Do you want to continue with the release?
Enter version [1.0.0
/usr/lib/python2.7/distutils/dist.py:267:UserWarning:Unknowndistributionoption:'message_extractors'
warnings.warn]: 

It would be useful to have the possibility to specify where the history file is.

I've already written some code on my machine for that but I'm waiting for the issue #17 with the code that make SetupConfig available for all the release steps.

This issue offers one another solution to solve the issue #22

I used it a lot in the past but I recently formatted my PC and started on ubuntu 14.04 from scratch. I just installed it and ran into this:

# longtest 
sh: 1: rst2html.py: not found
sh: 1: rst2html: not found
sh: 1: rst2: not found
ERROR: Error generating html. Please install docutils (or zc.rst2).

Shouldn't docutils be installed as dependency?

It would be nice if zest releaser would issue a warning if one is preparing a release with source code managed via X(e.g. git) and setuptools-x (eg. setuptools-git) is not installed.

This helps avoiding building eggs that miss most of the source code, because the setuptools magic does not know how to handle checked in files information for X(e.g. git)

We have quite some hacks to intercept sys.exit, to set environment variables, to work around input-grabbing functions and so.

I think mock could make our lives easier. On the other hand, the current setup works fine...

But perhaps we don't have to do so many full-scale doctests if we can more easily mock selected methods within our own classes and then use mock to only check if they've been called?

Nose or unittest2 mean a more familiar test setup for many people. And it'll cut down on the number of dependencies we pull in for testing :-)

There could be some kind of logic to check if sdist did not produce a egg which look legit.

One common reason is that MANIFEST.in has some errors and it simply does not include the main directory of your package.

Could it be simple to add logic that sdist contains at least one file which is not init.py?

Because the egg didn't go to dist folder (plone/src/product.name/dist) I spent a while looking around for why my egg wasn't built, until I realised that I didn't have my plugins installed.

The egg was built of course - it had just landed in somewhere like:

 /tmp/product.name-0.2.3-Prt03G/gitclone/dist

Since setup.py upload does not support HTTPS, but twine does, it would be nice if zest.releaser supported that (at least additionally, but probably even instead of the old method).

Some legacy projects, often converted from CVS, have multiple subprojects
under a single trunk. This creates URLs that contain a regular svn layout
within an top level trunk. For example:

trunk/
    projectA/
        trunk/
        tags/
        branches/

    projectB
        trunk/
        tags/
        branches/

Currently, when trying to release projectB, zest.releaser tries to create trunk/tags rather than trunk/projectB/tags.

When running fullrelease into a buildout, I got a traceback as version.

After investigating, setup.py --version is called with sys.executable in a popen thus lost the sys.path context.

Most method are in zest/releaser/utils.py file.

The problem append when the computer have no version of setuptools installed at system level.

In README there is example how to use zest.releaser in buildout.

Should this recipe have

setuptools-git
setuptools_subversion
zest.poreleaser?

zest.releases goes great length in detail telling what the package can or cannot do. However, it actually fails to tell you how do you actually run the software to do a release. README is very confusing from the point of a new user and thus might hinder the adoption of a good tool.

The README should at least highlight the process for

1. Tag a release

2. Make a source distribution

3. Upload it to pypi

What could help is to have "Examples" section like with jarn.mkrelease in README:

http://pypi.python.org/pypi/jarn.mkrelease

Trying to release a piece of software, I get the following error:

stefanesmacbook% release
Tag needed to proceed, you can use the following command:
git tag 0.2.1.dev0.dev190.g98425aa
/usr/local/lib/python2.7/site-packages/setuptools/dist.py:298:UserWarning:Theversionspecified('0.2.1.dev0.dev190.g98425aa')isaninvalidversion,thismaynotworkasexpectedwithnewerversionsofsetuptools,pip,andPyPI.PleaseseePEP440formoredetails.
"details."%self.metadata.version -m "Tagging 0.2.1.dev0.dev190.g98425aa
/usr/local/lib/python2.7/site-packages/setuptools/dist.py:298:UserWarning:Theversionspecified('0.2.1.dev0.dev190.g98425aa')isaninvalidversion,thismaynotworkasexpectedwithnewerversionsofsetuptools,pip,andPyPI.PleaseseePEP440formoredetails.
"details."%self.metadata.version"
Run this command (Y/n)?
/bin/sh: -c: line 1: syntax error near unexpected token `'0.2.1.dev0.dev190.g98425aa''
/bin/sh: -c: line 1: `/usr/local/lib/python2.7/site-packages/setuptools/dist.py:298:UserWarning:Theversionspecified('0.2.1.dev0.dev190.g98425aa')isaninvalidversion,thismaynotworkasexpectedwithnewerversionsofsetuptools,pip,andPyPI.PleaseseePEP440formoredetails.'


Failed to create tag 0.2.1.dev0.dev190.g98425aa
/usr/local/lib/python2.7/site-packages/setuptools/dist.py:298:UserWarning:Theversionspecified('0.2.1.dev0.dev190.g98425aa')isaninvalidversion,thismaynotworkasexpectedwithnewerversionsofsetuptools,pip,andPyPI.PleaseseePEP440formoredetails.
"details."%self.metadata.version!

(prerelease also prints similar messages, but only as a warning.)

I suspect this is related to the fact that I have installed recently setuptools 8.2.1.

If I revert setuptools to, say, version 3.6, it works as before.

Hello,

Using the latest zest.releaser 3.39, I got surprised by following message when running "fullrelease" in my svn-trunk-folder:

CRITICAL: No version control system detected.

However nothing is wrong with my svn-checkout. I'm using a recent svn:

svn --version
svn, version 1.7.6 (r1370777)

The problem is that zest.releaser searches for a ".svn"-directory in the current location (choose.py)
That assumption no longer holds in subversion1.7.
From the SVN-book, http://svnbook.red-bean.com/en/1.7/svn-book.pdf (page 10):

Prior to version 1.7, Subversion maintained .svn administrative subdirectories in every versioned directory of your working copy. Subversion 1.7 offers a completely new approach to how working copy metadata is stored and maintained, and chief among the visible changes to this approach is that each working copy now has only one .svn subdirectory which is an immediate child of the root of that working copy."

Maybe it's easier to checkout the output of "svn info" executed in the current directory.

Could you clarify in the docs when I should use entry points in setup.py or in setup.cfg?

Like with README. HISTORY / CHANGELOG may nowadays have .rst extension for Github convenience purposes.

Where one should poke to add in detection for this?

I have a scenario where I'd like:

• to have a single canonical source for the version (release.ini, version.txt or whatever)

• get zest.releaser manage the cannonical location (above mentioned) but also:

  • update setup.py with version changes (simple string replace, or some "version = '(.*)'" regex)
  • update mypackage/init.py with version changes (simple string replace, or some "__version__ = '(.*)'" regex)

  Now this would be configurable (you'd give zest.releaser the list of files).

Essentially that is similar to what https://pypi.python.org/pypi/bumpversion does.

The advantage is that you don't need to have fragile version loading code anymore in your init.py or setup.py.

What do you think?

We could add a list of recommended extras in setup.py, to be installed with zest.releaser[recommended]. This would indicate that we recommend these extra packages if you want to properly release to PyPI. We will want to keep an eye on these, possibly run their tests to see if they break when we change something, and add minimum version requirements if a new version is really needed. I would add these:

    zest.pocompile
    check-manifest
    pyroma

Possibly gocept.zestreleaser.customupload, because I happily use it, but that has nothing to do with releasing to PyPI.

@reinout, what do you think of this?

Cc @iElectric as he wants something like this.

I tried to release this package:

https://github.com/miohtama/visualtitle

... with zest.pocompile installed (easy_install zest.pocompile before fullrelease)

However, zest.pocompile does not activate itself and only .po files end up in the sdist packaging.

How one could debug the issue?

I just released z3c.coverage 2.0.1 with one missing fix because I forgot to git pull before running fullrelease.

It would be nice if zest.releaser did a git fetch and compared HEAD with the origin (e.g. git log --oneline ..origin), and then asked me if I wanted to continue and miss those changes (defaulting to N).

I got this when trying to release Products.TinyMCE.

Traceback (most recent call last):
File "/opt/local/bin/release", line 9, in
load_entry_point('zest.releaser==3.30', 'console_scripts', 'release')()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/zest.releaser-3.30-py2.6.egg/zest/releaser/release.py", line 268, in main
releaser.run()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/zest.releaser-3.30-py2.6.egg/zest/releaser/baserelease.py", line 22, in run
self.execute()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/zest.releaser-3.30-py2.6.egg/zest/releaser/release.py", line 57, in execute
self._release()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/zest.releaser-3.30-py2.6.egg/zest/releaser/release.py", line 209, in _release
if setup_config.has_bad_commands():
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/zest.releaser-3.30-py2.6.egg/zest/releaser/pypi.py", line 81, in has_bad_commands
if self.config.getboolean('egg_info', 'tag_svn_revision'):
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/ConfigParser.py", line 349, in getboolean
v = self.get(section, option)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/ConfigParser.py", line 541, in get
raise NoOptionError(option, section)
ConfigParser.NoOptionError: No option 'tag_svn_revision' in section: 'egg_info'

While executing bin/fullrelease (buildout installation of zest.releaser). I got the following output:

$ bin/fullrelease 
INFO: Starting prerelease.
Enter version [1.1]: 
INFO: Set setup.py's version to '1.1'
INFO: Changed version from '1.1dev' to '1.1'
INFO: History file ./eggs/Zope2-2.12.23-py2.7-linux-x86_64.egg/ZTUtils/CHANGES.txt updated.
INFO: The 'svn diff':



OK to commit this (Y/n)? n

So, clearly, zest.releaser located the wrong file, which in my case should have been:

docs/HISTORY.txt

One quick and generic fix is to check, in BaseVersionControl.filefind() if the file found is actually under version control. With SVN this is simple by checking the exit status of the svn info <filename> command for a file, which is non-zero (error) for a file not under version control.

With git this would be slightly more complicated, requiring parsing the output of git status <filename> or checking if git log -1 <filename> produces any output at all.

I am writing a prerelease hook that checks in Travis if all tests pass for the current commit.

It would be really useful if the data dict items include the current commit information (e.g. the sha-1 in git).

I've installed zest.releaser 3.37 in a virtualenv in Windows7 (don't ask :).
The repository is an SVN.

When doing a fullrelease, it fails with a message "Failed to create 0.2!"
After inspection the following lines are not crossplatform

https://github.com/zestsoftware/zest.releaser/blob/master/zest/releaser/svn.py#L102

    def available_tags(self):
        ....
        tags = [line.replace('/', '') for line in tag_info.split('\n')]      # DOES NOT TAKE INTO ACCOUNT \r\n
        tags = [tag for tag in tags if tag]  # filter empty ones

My variable taginfo here is "0.1/\r\n0.2/\r\n", so I end up with a variable tags = [ "0.1\r", "0.2\r"]

A simple fix is to add an extra strip:

    def available_tags(self):
        ....
        tags = [line.replace('/', '') for line in tag_info.split('\n')]
        tags = [tag.strip() for tag in tags if tag]     # strip windows line-separators + filter empty ones

the nicest solution is to use the splitlines-method on a string:

    >>> tag_info =  "0.1/\r\n0.2/\r\n"
    >>> tags = [tag.strip('/') for tag in tag_info.splitlines()]
    >>> tag
             [ "0.1", "0.2"]

but I haven't checked if that works for all supported python-versions.
Cfr. http://docs.python.org/glossary.html#term-universal-newlines

Due to Github, people have started to use README.rst instead of README.txt.

However, zest.releaser does not check for its existenece:

     warning: sdist: standard file not found: should have one of README, README.txt 

Please add python3 support :-).

I want to use zest.releaser together wit git-flow. However, git-flow is writing tags and zest.releaser is also writing tags. So we get sets of unrelated tags. Would there be a way to avoid this situation. Maybe a flag to skip the writing of tags?

During prerelease, there's a possibility that zest cannot find a way to update your package version number. For example setting your version number in setup.py from a variable set in mypackage.__init__.py and forgetting to set python_file_with_version in a setup.cfg.

In this case, your package version number is readable, but not writeable.

This bug hit us today, and raised the question: why would zest continue if it couldn't update your version number?

When it does continue it get's really confused:

1. prerelease updates just the history file, no other file
2. postrelease updates just the history file, to a wrong value! Because it assumes prerelease incremented the package version somewhere.

Here's vcs.py:BaseVersionControl._update_version where zest looks for the right place to update a version string.

I propose that zest raise an exception at the end of that method, if it cannot find a place to update your package version number. Or maybe I'm missing something, is there a deliberate reason zest continues in this case?

I'm happy to write a patch if there's agreement on this.

Using python2.7 64-bit on Windows 7, with a virtualenv active.

While running prerelease, an odd error is raised while probing for the version number with setup.py --version (getting the whole traceback required a minor change to get_setup_py_version):

CRITICAL: The setup.py of this package has an error:
Traceback (most recent call last):
  File "setup.py", line 16, in <module>
    setup(
  File "c:\Users\Rio\Documents\mcedit2-distrib\mcedit2\src\mceditlib\setup.py", line 1, in <module>
    from setuptools import setup
  File "c:\Users\Rio\Documents\mcedit2-distrib\mcedit2\ENV64\lib\site-packages\setuptools\__init__.py", line 12, in <mod
ule>
    from setuptools.extension import Extension
  File "c:\Users\Rio\Documents\mcedit2-distrib\mcedit2\ENV64\lib\site-packages\setuptools\extension.py", line 8, in <mod
ule>
    from .dist import _get_unpatched
  File "c:\Users\Rio\Documents\mcedit2-distrib\mcedit2\ENV64\lib\site-packages\setuptools\dist.py", line 23, in <module>

    from setuptools.depends import Require
  File "c:\Users\Rio\Documents\mcedit2-distrib\mcedit2\ENV64\lib\site-packages\setuptools\depends.py", line 6, in <modul
e>
    from setuptools import compat
  File "c:\Users\Rio\Documents\mcedit2-distrib\mcedit2\ENV64\lib\site-packages\setuptools\compat.py", line 17, in <modul
e>
    import httplib
  File "C:\Python27-64\Lib\httplib.py", line 79, in <module>
    import mimetools
  File "C:\Python27-64\Lib\mimetools.py", line 6, in <module>
    import tempfile
  File "C:\Python27-64\Lib\tempfile.py", line 35, in <module>
    from random import Random as _Random
  File "C:\Python27-64\Lib\random.py", line 879, in <module>
    _inst = Random()
  File "C:\Python27-64\Lib\random.py", line 97, in __init__
    self.seed(x)
  File "C:\Python27-64\Lib\random.py", line 111, in seed
    a = long(_hexlify(_urandom(16)), 16)
WindowsError: [Error -2146893818] Invalid Signature

CRITICAL: No version found.

This error is not raised when running setup.py by itself.

After some investigating, I learned this error is the Windows DLL loader (Windows Side-By-Side) unable to locate the crypto libraries - because the SYSTEMROOT environment variable is not present! The root cause is in your utils.system function, which substitutes a nearly-empty env dict whenever the exe passed to utils.system is the Python exe used to run prerelease.

Unless I'm missing something, the fix is to copy os.environ and update its PYTHONPATH instead of create a new dict for env.

I thought zero at end presents svn revision and somehow git support is broken, but looking at source, it seems .dev0 is appended for development version.

For example having 0.1a2 as development version resolves to 0.1a2.dev0, which is rather confusing :)

https://github.com/zestsoftware/zest.releaser/blob/master/zest/releaser/postrelease.py#L16

If PyPi upload fails abort fullrelease

(otherwise reupload attemps become painful when you have done postrelease step)

Register and upload to pypi (y/N)? yes
INFO: Running: /Users/mikko/code/plone-venv/bin/python setup.py register -r pypi sdist --formats=zip upload -r pypi
Showing first few lines...
running register
running egg_info
writing paster_plugins to mfabrik.webandmobile.egg-info/paster_plugins.txt
writing requirements to mfabrik.webandmobile.egg-info/requires.txt
writing mfabrik.webandmobile.egg-info/PKG-INFO
...
Showing last few lines...
unrecognized .svn/entries format in
Upload failed (503): Server Maintenance

... oops

For various reasons I often have a statement like this

__version__ = '2.4.frog-knows'

in one of my *.py files, and a setup.py that extracts it from there. If I try to run fullrelease in such a package, it fails to update the version number and tries to create incorrect tags etc.

I'd like to be able to tell zest.releaser where the version is defined, perhaps in setup.cfg? Something like

[zest.releaser]
version-file = src/gtimelog/__init__.py

README tells that one should file issues on Launchpad, which in fact does not have a bug tracker activated

My changelog is in docs/changelog.rst but I have debian packaging code (debian/changelog) so zest releaser picks debian/changelog (because filename is shorter) while it definitely does not know how to handle the file.

Hi,

You might consider linking this tutorial from README:

http://opensourcehacker.com/2012/08/14/high-quality-automated-package-releases-for-python-with-zest-releaser/

Also if it's ok you can salvage bits directly to README from there.

I see in this line that maybe python 2.4 has some issues with creating tar.gz files

However since that's a really old python by now, can we add an option to upload tar files to pypi?

I'm using a dedicated python 2.7.5 and buildout to install Plone, but because setuptools is not installed in that python, I get the following error for longtest & prerelease:

/dev/Plone/Development/project/src/visualtitle$ ../../bin/longtest
Traceback (most recent call last):
  File "setup.py", line 16, in <module>
    from setuptools import setup
ImportError: No module named setuptools
ERROR: Error generating long description.

Zest.releaser works on a local directory that is checked out from svn/git/hg.

When trying to make people warm to adopt zest.releaser, I received feedback that it would be nice to be able to make releases just by pointing to a trunk-url. This feedback comes from a team that is doing some script-development on Windows behind corporate proxy.

The workflow would be then: they keep working like they do now; but when code is ready, they can SSH to some linux-box where there is a virtualenv installed and an appropriate .pypirc, and they just do:

fullrelease svn+ssh://user@host/packages/<my_package>/trunk

( the checkout is made in a tmp-folder.)

Hi,

I am looking a way to automatically set Sphinx documentation version number every tmie a do a release (docs being on readthedocs.org)

What would be the suggested approach?

zest.releases doesn't honour SVN added repository files when creating a package. I think this is the default setuptools behavior. Only .py files are packed. This caused malformed packages which have files left out.

Getting distribution for 'mfabrik.webandmobile'.
error: docs/HISTORY.txt: No such file or directory
An error occured when trying to install mfabrik.webandmobile 1.0.11. Look above this message for any errors that were output by easy_install.
While:
  Installing instance. 

Subversion 1.7.

My use-case: make a release, create an sdist-egg and scp this tarball to some location that serves as index-server.

For this I'm trying to use zest.releaser + gocept.zestreleaser.customupload (v1.4)

I configure my .pypirc as specified in the docs of gocept.zestreleaser.customupload,
but there is no sdist created that can be scp'ed.

It seems the generation of an sdist depends on the presence of a[distutils]-section in .pypirc, but you can't leave this empty without breaking things

So the question boils down to: how to create a tarball of the package without uploading with via zest.releaser and defer it to gocept.zestreleaser.customupload

Maybe I'm missing something obvious.

Looks like if you are developing Plone addon + Travis CI testing configuration, after release you need to update the version number to base.cfg file.

collective/collective.z3cform.datagridfield#12

Would there be a way to automate this with zest.releaser+

Sometimes I forget to add the changes to the CHANGES.rst file and I find out during a release.

A fix would be to detect whether or not the CHANGES.rst has some changes for this release.

It seems that the wheel format is not handled by zest.releaser. It would be nice that if [bdist_wheel] is present in setup.cfg then

python setup.py sdist bdist_wheel upload

is called instead of:

python setup.py sdist upload