pediapress / mwlib Goto Github PK

View Code? Open in Web Editor NEW

102.0 102.0 35.0 15.68 MB

mediawiki parser library

Python 94.08% Makefile 0.26% C++ 4.14% Dockerfile 0.03% Cython 1.50%

mwlib's People

Contributors

Stargazers

Watchers

mwlib's Issues

Image not fetched

The imageinfo for the image does not contain the title attribute, therefore it is not fetched. (check line 535 in fetch.py)

See snippet: http://en.wikivoyage.org/w/index.php?title=User:Volker.haas&oldid=2097385

In the article http://en.wikivoyage.org/wiki/Cape_Town (where the example was pulled from) another image fails to be fetched

test_nserve failure

tests/test_nserve.py:4: in

import pytest, gevent, urllib, urllib2, bottle
/usr/bin/bottle.py:48: in
sys.exit(127)
E SystemExit: 127
-------------------------------------------------------------------- Captured stderr ---------------------------------------------------------------------
This Python implementation (python2.7) is not supported by the script.
=============================================== 702 passed, 1 skipped, 9 xfailed, 1 error in 5.88 seconds ================================================

ERROR: dev-python/mwlib-0.15.10 failed (test phase):

I can't imagine this passing for you also. The failure occurs for ipython even more so.

dev-python/mwlib $ ipython -c "import bottle"
This Python implementation (python2.7) is not supported by the script.
An exception has occurred, use %tb to see the full traceback.

SystemExit: 127

To exit: use 'exit', 'quit', or Ctrl-D.

There are 2 things to consider. This trips the test post bottle 0.11.3 which is now up to 0.11.6. I have renamed the installed bottle script from bottle.py to bottle which was supposed to help. The second is that it trips over by py.test attempting to execute the test. Whether this is a bottle error or not I can't decipher.

$ python2.7 -c "import bottle"
happily imports.
$ ipython -c "import bottle" doesn't.

Either way, the test_nserve needs to absorb the code for bottle post 0.11.3.

Race condition in net/fetch.py

def run(self):
    self.report()
    dispatch_gr = gevent.spawn(callwhen, self.dispatch_event, self.dispatch)
    try:
        self.pool.join()
    finally:
        dispatch_gr.kill()

If the pool completes before the dispatch_gr is given processing time, doesn't this just end up killing dispatch_gr? If so, _refcall to fetch_imageinfo never occurs and assert not self.imageinfo_todo fails.

Installation fails: command 'x86_64-linux-gnu-gcc' failed with exit status 1

Not sure what to do with this error. All I did is install python-pip (just sudo apt-get install python-pip) and execute the installer (just pip install -i http://pypi.pediapress.com/simple/ mwlib as per the documentation).

Complete log:

Downloading/unpacking mwlib

  Getting page http://pypi.pediapress.com/simple/mwlib/
  URLs to search for versions for mwlib:
  * http://pypi.pediapress.com/simple/mwlib/
  Analyzing links from page http://pypi.pediapress.com/simple/mwlib/
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.12.15.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.12.15
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.12.16.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.12.16
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.12.17.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.12.17
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.13.1.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.13.1
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.13.2.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.13.2
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.13.3.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.13.3
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.13.4.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.13.4
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.13.5.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.13.5
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.13.6.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.13.6
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.13.7.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.13.7
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.13.8.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.13.8
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.13.9.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.13.9
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.13.10.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.13.10
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.13.11.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.13.11
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.14.0.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.14.0
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.14.1.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.14.1
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.14.2.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.14.2
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.14.3.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.14.3
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.15.0.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.15.0
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.15.1.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.15.1
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.15.2.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.15.2
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.15.3.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.15.3
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.15.4.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.15.4
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.15.5.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.15.5
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.15.6.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.15.6
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.15.7.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.15.7
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.15.8.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.15.8
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.15.9.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.15.9
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.15.10.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.15.10
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.15.11.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.15.11
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.15.12.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.15.12
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.15.13.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.15.13
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.15.14.zip (from http://pypi.pediapress.com/simple/mwlib/), version: 0.15.14
    Found link http://pypi.pediapress.com/packages/mirror/mwlib-0.15.15.tar.gz (from http://pypi.pediapress.com/simple/mwlib/), version: 0.15.15
  Using version 0.15.15 (newest of versions: 0.15.15, 0.15.14, 0.15.13, 0.15.12, 0.15.11, 0.15.10, 0.15.9, 0.15.8, 0.15.7, 0.15.6, 0.15.5, 0.15.4, 0.15.3, 0.15.2, 0.15.1, 0.15.0, 0.14.3, 0.14.2, 0.14.1, 0.14.0, 0.13.11, 0.13.10, 0.13.9, 0.13.8, 0.13.7, 0.13.6, 0.13.5, 0.13.4, 0.13.3, 0.13.2, 0.13.1, 0.12.17, 0.12.16, 0.12.15)
  Downloading from URL http://pypi.pediapress.com/packages/mirror/mwlib-0.15.15.tar.gz (from http://pypi.pediapress.com/simple/mwlib/)
  Running setup.py egg_info for package mwlib

    running egg_info
    creating pip-egg-info/mwlib.egg-info
    writing requirements to pip-egg-info/mwlib.egg-info/requires.txt
    writing pip-egg-info/mwlib.egg-info/PKG-INFO
    writing namespace_packages to pip-egg-info/mwlib.egg-info/namespace_packages.txt
    writing top-level names to pip-egg-info/mwlib.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/mwlib.egg-info/dependency_links.txt
    writing entry points to pip-egg-info/mwlib.egg-info/entry_points.txt
    writing manifest file 'pip-egg-info/mwlib.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found


    reading manifest file 'pip-egg-info/mwlib.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'pip-egg-info/mwlib.egg-info/SOURCES.txt'
  Source in /tmp/pip_build_martijn/mwlib has version 0.15.15, which satisfies requirement mwlib
Downloading/unpacking pyparsing>=1.4.11,<1.6 (from mwlib)

  Getting page http://pypi.pediapress.com/simple/pyparsing/
  URLs to search for versions for pyparsing>=1.4.11,<1.6 (from mwlib):
  * http://pypi.pediapress.com/simple/pyparsing/
  Analyzing links from page http://pypi.pediapress.com/simple/pyparsing/
    Found link http://pypi.pediapress.com/packages/hold/pyparsing-1.5.1.tar.gz (from http://pypi.pediapress.com/simple/pyparsing/), version: 1.5.1
    Found link http://pypi.pediapress.com/packages/hold/pyparsing-1.5.6.tar.gz (from http://pypi.pediapress.com/simple/pyparsing/), version: 1.5.6
  Using version 1.5.6 (newest of versions: 1.5.6, 1.5.1)
  Downloading from URL http://pypi.pediapress.com/packages/hold/pyparsing-1.5.6.tar.gz (from http://pypi.pediapress.com/simple/pyparsing/)
  Running setup.py egg_info for package pyparsing

    running egg_info
    creating pip-egg-info/pyparsing.egg-info
    writing pip-egg-info/pyparsing.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/pyparsing.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/pyparsing.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/pyparsing.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found


    reading manifest file 'pip-egg-info/pyparsing.egg-info/SOURCES.txt'
    writing manifest file 'pip-egg-info/pyparsing.egg-info/SOURCES.txt'
  Source in /tmp/pip_build_martijn/pyparsing has version 1.5.6, which satisfies requirement pyparsing>=1.4.11,<1.6 (from mwlib)
Downloading/unpacking timelib>=0.2 (from mwlib)

  Getting page http://pypi.pediapress.com/simple/timelib/
  URLs to search for versions for timelib>=0.2 (from mwlib):
  * http://pypi.pediapress.com/simple/timelib/
  Analyzing links from page http://pypi.pediapress.com/simple/timelib/
    Found link http://pypi.pediapress.com/packages/mirror/timelib-0.2.1.tar.gz (from http://pypi.pediapress.com/simple/timelib/), version: 0.2.1
    Found link http://pypi.pediapress.com/packages/mirror/timelib-0.2.2.tar.gz (from http://pypi.pediapress.com/simple/timelib/), version: 0.2.2
    Found link http://pypi.pediapress.com/packages/mirror/timelib-0.2.3.zip (from http://pypi.pediapress.com/simple/timelib/), version: 0.2.3
    Found link http://pypi.pediapress.com/packages/mirror/timelib-0.2.4.zip (from http://pypi.pediapress.com/simple/timelib/), version: 0.2.4
  Using version 0.2.4 (newest of versions: 0.2.4, 0.2.3, 0.2.2, 0.2.1)
  Downloading from URL http://pypi.pediapress.com/packages/mirror/timelib-0.2.4.zip (from http://pypi.pediapress.com/simple/timelib/)
  Running setup.py egg_info for package timelib

    running egg_info
    creating pip-egg-info/timelib.egg-info
    writing pip-egg-info/timelib.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/timelib.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/timelib.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/timelib.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found


    reading manifest file 'pip-egg-info/timelib.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'pip-egg-info/timelib.egg-info/SOURCES.txt'
  Source in /tmp/pip_build_martijn/timelib has version 0.2.4, which satisfies requirement timelib>=0.2 (from mwlib)
Downloading/unpacking bottle>=0.10 (from mwlib)

  Getting page http://pypi.pediapress.com/simple/bottle/
  URLs to search for versions for bottle>=0.10 (from mwlib):
  * http://pypi.pediapress.com/simple/bottle/
  Analyzing links from page http://pypi.pediapress.com/simple/bottle/
    Found link http://pypi.pediapress.com/packages/mirror/bottle-0.9.6.tar.gz (from http://pypi.pediapress.com/simple/bottle/), version: 0.9.6
    Found link http://pypi.pediapress.com/packages/mirror/bottle-0.9.7.tar.gz (from http://pypi.pediapress.com/simple/bottle/), version: 0.9.7
    Found link http://pypi.pediapress.com/packages/mirror/bottle-0.10.1.tar.gz (from http://pypi.pediapress.com/simple/bottle/), version: 0.10.1
    Found link http://pypi.pediapress.com/packages/mirror/bottle-0.10.2.tar.gz (from http://pypi.pediapress.com/simple/bottle/), version: 0.10.2
    Found link http://pypi.pediapress.com/packages/mirror/bottle-0.10.3.tar.gz (from http://pypi.pediapress.com/simple/bottle/), version: 0.10.3
    Found link http://pypi.pediapress.com/packages/mirror/bottle-0.10.4.tar.gz (from http://pypi.pediapress.com/simple/bottle/), version: 0.10.4
    Found link http://pypi.pediapress.com/packages/mirror/bottle-0.10.7.tar.gz (from http://pypi.pediapress.com/simple/bottle/), version: 0.10.7
    Found link http://pypi.pediapress.com/packages/mirror/bottle-0.10.9.tar.gz (from http://pypi.pediapress.com/simple/bottle/), version: 0.10.9
    Found link http://pypi.pediapress.com/packages/mirror/bottle-0.10.11.tar.gz (from http://pypi.pediapress.com/simple/bottle/), version: 0.10.11
    Found link http://pypi.pediapress.com/packages/mirror/bottle-0.11.1.tar.gz (from http://pypi.pediapress.com/simple/bottle/), version: 0.11.1
    Found link http://pypi.pediapress.com/packages/mirror/bottle-0.11.2.tar.gz (from http://pypi.pediapress.com/simple/bottle/), version: 0.11.2
    Found link http://pypi.pediapress.com/packages/mirror/bottle-0.11.3.tar.gz (from http://pypi.pediapress.com/simple/bottle/), version: 0.11.3
    Found link http://pypi.pediapress.com/packages/mirror/bottle-0.11.4.tar.gz (from http://pypi.pediapress.com/simple/bottle/), version: 0.11.4
    Found link http://pypi.pediapress.com/packages/mirror/bottle-0.11.5.tar.gz (from http://pypi.pediapress.com/simple/bottle/), version: 0.11.5
    Found link http://pypi.pediapress.com/packages/mirror/bottle-0.11.6.tar.gz (from http://pypi.pediapress.com/simple/bottle/), version: 0.11.6
    Found link http://pypi.pediapress.com/packages/mirror/bottle-0.12.1.tar.gz (from http://pypi.pediapress.com/simple/bottle/), version: 0.12.1
    Found link http://pypi.pediapress.com/packages/mirror/bottle-0.12.2.tar.gz (from http://pypi.pediapress.com/simple/bottle/), version: 0.12.2
    Found link http://pypi.pediapress.com/packages/mirror/bottle-0.12.3.tar.gz (from http://pypi.pediapress.com/simple/bottle/), version: 0.12.3
    Found link http://pypi.pediapress.com/packages/mirror/bottle-0.12.4.tar.gz (from http://pypi.pediapress.com/simple/bottle/), version: 0.12.4
    Found link http://pypi.pediapress.com/packages/mirror/bottle-0.12.5.tar.gz (from http://pypi.pediapress.com/simple/bottle/), version: 0.12.5
  Ignoring link http://pypi.pediapress.com/packages/mirror/bottle-0.9.6.tar.gz (from http://pypi.pediapress.com/simple/bottle/), version 0.9.6 doesn't match >=0.10
  Ignoring link http://pypi.pediapress.com/packages/mirror/bottle-0.9.7.tar.gz (from http://pypi.pediapress.com/simple/bottle/), version 0.9.7 doesn't match >=0.10
  Using version 0.12.5 (newest of versions: 0.12.5, 0.12.4, 0.12.3, 0.12.2, 0.12.1, 0.11.6, 0.11.5, 0.11.4, 0.11.3, 0.11.2, 0.11.1, 0.10.11, 0.10.9, 0.10.7, 0.10.4, 0.10.3, 0.10.2, 0.10.1)
  Downloading from URL http://pypi.pediapress.com/packages/mirror/bottle-0.12.5.tar.gz (from http://pypi.pediapress.com/simple/bottle/)
  Running setup.py egg_info for package bottle

    running egg_info
    creating pip-egg-info/bottle.egg-info
    writing pip-egg-info/bottle.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/bottle.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/bottle.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/bottle.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found


    reading manifest file 'pip-egg-info/bottle.egg-info/SOURCES.txt'
    writing manifest file 'pip-egg-info/bottle.egg-info/SOURCES.txt'
  Source in /tmp/pip_build_martijn/bottle has version 0.12.5, which satisfies requirement bottle>=0.10 (from mwlib)
Downloading/unpacking pyPdf>=1.12 (from mwlib)

  Getting page http://pypi.pediapress.com/simple/pyPdf/
  URLs to search for versions for pyPdf>=1.12 (from mwlib):
  * http://pypi.pediapress.com/simple/pyPdf/
  Analyzing links from page http://pypi.pediapress.com/simple/pyPdf/
    Found link http://pypi.pediapress.com/packages/mirror/pyPdf-1.12.tar.gz (from http://pypi.pediapress.com/simple/pyPdf/), version: 1.12
    Found link http://pypi.pediapress.com/packages/mirror/pyPdf-1.13.zip (from http://pypi.pediapress.com/simple/pyPdf/), version: 1.13
  Using version 1.13 (newest of versions: 1.13, 1.12)
  Downloading from URL http://pypi.pediapress.com/packages/mirror/pyPdf-1.13.zip (from http://pypi.pediapress.com/simple/pyPdf/)
  Running setup.py egg_info for package pyPdf

    running egg_info
    creating pip-egg-info/pyPdf.egg-info
    writing pip-egg-info/pyPdf.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/pyPdf.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/pyPdf.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/pyPdf.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found


    reading manifest file 'pip-egg-info/pyPdf.egg-info/SOURCES.txt'
    writing manifest file 'pip-egg-info/pyPdf.egg-info/SOURCES.txt'
  Source in /tmp/pip_build_martijn/pyPdf has version 1.13, which satisfies requirement pyPdf>=1.12 (from mwlib)
Downloading/unpacking apipkg>=1.2 (from mwlib)

  Getting page http://pypi.pediapress.com/simple/apipkg/
  URLs to search for versions for apipkg>=1.2 (from mwlib):
  * http://pypi.pediapress.com/simple/apipkg/
  Analyzing links from page http://pypi.pediapress.com/simple/apipkg/
    Found link http://pypi.pediapress.com/packages/mirror/apipkg-1.0.tar.gz (from http://pypi.pediapress.com/simple/apipkg/), version: 1.0
    Found link http://pypi.pediapress.com/packages/mirror/apipkg-1.2.zip (from http://pypi.pediapress.com/simple/apipkg/), version: 1.2
  Ignoring link http://pypi.pediapress.com/packages/mirror/apipkg-1.0.tar.gz (from http://pypi.pediapress.com/simple/apipkg/), version 1.0 doesn't match >=1.2
  Downloading apipkg-1.2.zip

  Downloading from URL http://pypi.pediapress.com/packages/mirror/apipkg-1.2.zip (from http://pypi.pediapress.com/simple/apipkg/)
  Running setup.py egg_info for package apipkg

    running egg_info
    creating pip-egg-info/apipkg.egg-info
    writing pip-egg-info/apipkg.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/apipkg.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/apipkg.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/apipkg.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found


    reading manifest file 'pip-egg-info/apipkg.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    no previously-included directories found matching '.svn'

    no previously-included directories found matching '.hg'

    writing manifest file 'pip-egg-info/apipkg.egg-info/SOURCES.txt'
  Source in /tmp/pip_build_martijn/apipkg has version 1.2, which satisfies requirement apipkg>=1.2 (from mwlib)
Downloading/unpacking qserve>=0.2.7 (from mwlib)

  Getting page http://pypi.pediapress.com/simple/qserve/
  URLs to search for versions for qserve>=0.2.7 (from mwlib):
  * http://pypi.pediapress.com/simple/qserve/
  Analyzing links from page http://pypi.pediapress.com/simple/qserve/
    Found link http://pypi.pediapress.com/packages/mirror/qserve-0.0.1.tar.gz (from http://pypi.pediapress.com/simple/qserve/), version: 0.0.1
    Found link http://pypi.pediapress.com/packages/mirror/qserve-0.1.0.zip (from http://pypi.pediapress.com/simple/qserve/), version: 0.1.0
    Found link http://pypi.pediapress.com/packages/mirror/qserve-0.1.1.zip (from http://pypi.pediapress.com/simple/qserve/), version: 0.1.1
    Found link http://pypi.pediapress.com/packages/mirror/qserve-0.2.1.zip (from http://pypi.pediapress.com/simple/qserve/), version: 0.2.1
    Found link http://pypi.pediapress.com/packages/mirror/qserve-0.2.3.zip (from http://pypi.pediapress.com/simple/qserve/), version: 0.2.3
    Found link http://pypi.pediapress.com/packages/mirror/qserve-0.2.4.zip (from http://pypi.pediapress.com/simple/qserve/), version: 0.2.4
    Found link http://pypi.pediapress.com/packages/mirror/qserve-0.2.5.zip (from http://pypi.pediapress.com/simple/qserve/), version: 0.2.5
    Found link http://pypi.pediapress.com/packages/mirror/qserve-0.2.6.zip (from http://pypi.pediapress.com/simple/qserve/), version: 0.2.6
    Found link http://pypi.pediapress.com/packages/mirror/qserve-0.2.7.zip (from http://pypi.pediapress.com/simple/qserve/), version: 0.2.7
    Found link http://pypi.pediapress.com/packages/mirror/qserve-0.2.8.zip (from http://pypi.pediapress.com/simple/qserve/), version: 0.2.8
  Ignoring link http://pypi.pediapress.com/packages/mirror/qserve-0.0.1.tar.gz (from http://pypi.pediapress.com/simple/qserve/), version 0.0.1 doesn't match >=0.2.7
  Ignoring link http://pypi.pediapress.com/packages/mirror/qserve-0.1.0.zip (from http://pypi.pediapress.com/simple/qserve/), version 0.1.0 doesn't match >=0.2.7
  Ignoring link http://pypi.pediapress.com/packages/mirror/qserve-0.1.1.zip (from http://pypi.pediapress.com/simple/qserve/), version 0.1.1 doesn't match >=0.2.7
  Ignoring link http://pypi.pediapress.com/packages/mirror/qserve-0.2.1.zip (from http://pypi.pediapress.com/simple/qserve/), version 0.2.1 doesn't match >=0.2.7
  Ignoring link http://pypi.pediapress.com/packages/mirror/qserve-0.2.3.zip (from http://pypi.pediapress.com/simple/qserve/), version 0.2.3 doesn't match >=0.2.7
  Ignoring link http://pypi.pediapress.com/packages/mirror/qserve-0.2.4.zip (from http://pypi.pediapress.com/simple/qserve/), version 0.2.4 doesn't match >=0.2.7
  Ignoring link http://pypi.pediapress.com/packages/mirror/qserve-0.2.5.zip (from http://pypi.pediapress.com/simple/qserve/), version 0.2.5 doesn't match >=0.2.7
  Ignoring link http://pypi.pediapress.com/packages/mirror/qserve-0.2.6.zip (from http://pypi.pediapress.com/simple/qserve/), version 0.2.6 doesn't match >=0.2.7
  Using version 0.2.8 (newest of versions: 0.2.8, 0.2.7)
  Downloading qserve-0.2.8.zip

  Downloading from URL http://pypi.pediapress.com/packages/mirror/qserve-0.2.8.zip (from http://pypi.pediapress.com/simple/qserve/)
  Running setup.py egg_info for package qserve

    running egg_info
    creating pip-egg-info/qserve.egg-info
    writing pip-egg-info/qserve.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/qserve.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/qserve.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/qserve.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found


    reading manifest file 'pip-egg-info/qserve.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'pip-egg-info/qserve.egg-info/SOURCES.txt'
  Source in /tmp/pip_build_martijn/qserve has version 0.2.8, which satisfies requirement qserve>=0.2.7 (from mwlib)
Requirement already satisfied (use --upgrade to upgrade): lxml in /usr/lib/python2.7/dist-packages (from mwlib)

Downloading/unpacking py>=1.4 (from mwlib)

  Getting page http://pypi.pediapress.com/simple/py/
  URLs to search for versions for py>=1.4 (from mwlib):
  * http://pypi.pediapress.com/simple/py/
  Analyzing links from page http://pypi.pediapress.com/simple/py/
    Found link http://pypi.pediapress.com/packages/mirror/py-1.3.4.tar.gz (from http://pypi.pediapress.com/simple/py/), version: 1.3.4
    Found link http://pypi.pediapress.com/packages/mirror/py-1.4.4.zip (from http://pypi.pediapress.com/simple/py/), version: 1.4.4
    Found link http://pypi.pediapress.com/packages/mirror/py-1.4.5.zip (from http://pypi.pediapress.com/simple/py/), version: 1.4.5
    Found link http://pypi.pediapress.com/packages/mirror/py-1.4.6.zip (from http://pypi.pediapress.com/simple/py/), version: 1.4.6
    Found link http://pypi.pediapress.com/packages/mirror/py-1.4.7.zip (from http://pypi.pediapress.com/simple/py/), version: 1.4.7
    Found link http://pypi.pediapress.com/packages/mirror/py-1.4.8.zip (from http://pypi.pediapress.com/simple/py/), version: 1.4.8
    Found link http://pypi.pediapress.com/packages/mirror/py-1.4.9.zip (from http://pypi.pediapress.com/simple/py/), version: 1.4.9
    Found link http://pypi.pediapress.com/packages/mirror/py-1.4.10.zip (from http://pypi.pediapress.com/simple/py/), version: 1.4.10
    Found link http://pypi.pediapress.com/packages/mirror/py-1.4.11.zip (from http://pypi.pediapress.com/simple/py/), version: 1.4.11
    Found link http://pypi.pediapress.com/packages/mirror/py-1.4.12.zip (from http://pypi.pediapress.com/simple/py/), version: 1.4.12
    Found link http://pypi.pediapress.com/packages/mirror/py-1.4.13.tar.gz (from http://pypi.pediapress.com/simple/py/), version: 1.4.13
    Found link http://pypi.pediapress.com/packages/mirror/py-1.4.14.tar.gz (from http://pypi.pediapress.com/simple/py/), version: 1.4.14
    Found link http://pypi.pediapress.com/packages/mirror/py-1.4.15.tar.gz (from http://pypi.pediapress.com/simple/py/), version: 1.4.15
    Found link http://pypi.pediapress.com/packages/mirror/py-1.4.16.tar.gz (from http://pypi.pediapress.com/simple/py/), version: 1.4.16
    Found link http://pypi.pediapress.com/packages/mirror/py-1.4.17.tar.gz (from http://pypi.pediapress.com/simple/py/), version: 1.4.17
    Found link http://pypi.pediapress.com/packages/mirror/py-1.4.18.tar.gz (from http://pypi.pediapress.com/simple/py/), version: 1.4.18
    Found link http://pypi.pediapress.com/packages/mirror/py-1.4.19.tar.gz (from http://pypi.pediapress.com/simple/py/), version: 1.4.19
    Found link http://pypi.pediapress.com/packages/mirror/py-1.4.20.tar.gz (from http://pypi.pediapress.com/simple/py/), version: 1.4.20
  Ignoring link http://pypi.pediapress.com/packages/mirror/py-1.3.4.tar.gz (from http://pypi.pediapress.com/simple/py/), version 1.3.4 doesn't match >=1.4
  Using version 1.4.20 (newest of versions: 1.4.20, 1.4.19, 1.4.18, 1.4.17, 1.4.16, 1.4.15, 1.4.14, 1.4.13, 1.4.12, 1.4.11, 1.4.10, 1.4.9, 1.4.8, 1.4.7, 1.4.6, 1.4.5, 1.4.4)
  Downloading from URL http://pypi.pediapress.com/packages/mirror/py-1.4.20.tar.gz (from http://pypi.pediapress.com/simple/py/)
  Running setup.py egg_info for package py

    running egg_info
    creating pip-egg-info/py.egg-info
    writing pip-egg-info/py.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/py.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/py.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/py.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found


    reading manifest file 'pip-egg-info/py.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'pip-egg-info/py.egg-info/SOURCES.txt'
  Source in /tmp/pip_build_martijn/py has version 1.4.20, which satisfies requirement py>=1.4 (from mwlib)
Downloading/unpacking sqlite3dbm (from mwlib)

  Getting page http://pypi.pediapress.com/simple/sqlite3dbm/
  URLs to search for versions for sqlite3dbm (from mwlib):
  * http://pypi.pediapress.com/simple/sqlite3dbm/
  Analyzing links from page http://pypi.pediapress.com/simple/sqlite3dbm/
    Found link http://pypi.pediapress.com/packages/mirror/sqlite3dbm-0.1.4.tar.gz (from http://pypi.pediapress.com/simple/sqlite3dbm/), version: 0.1.4
  Downloading sqlite3dbm-0.1.4.tar.gz

  Downloading from URL http://pypi.pediapress.com/packages/mirror/sqlite3dbm-0.1.4.tar.gz (from http://pypi.pediapress.com/simple/sqlite3dbm/)
  Running setup.py egg_info for package sqlite3dbm

    running egg_info
    creating pip-egg-info/sqlite3dbm.egg-info
    writing pip-egg-info/sqlite3dbm.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/sqlite3dbm.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/sqlite3dbm.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/sqlite3dbm.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found


    reading manifest file 'pip-egg-info/sqlite3dbm.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    no previously-included directories found matching 'docs/_build'

    writing manifest file 'pip-egg-info/sqlite3dbm.egg-info/SOURCES.txt'
  Source in /tmp/pip_build_martijn/sqlite3dbm has version 0.1.4, which satisfies requirement sqlite3dbm (from mwlib)
Downloading/unpacking simplejson>=2.3 (from mwlib)

  Getting page http://pypi.pediapress.com/simple/simplejson/
  URLs to search for versions for simplejson>=2.3 (from mwlib):
  * http://pypi.pediapress.com/simple/simplejson/
  Analyzing links from page http://pypi.pediapress.com/simple/simplejson/
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-1.9.2.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 1.9.2
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-2.0.9.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 2.0.9
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-2.1.1.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 2.1.1
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-2.1.6.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 2.1.6
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-2.2.1.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 2.2.1
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-2.3.0.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 2.3.0
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-2.3.2.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 2.3.2
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-2.3.3.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 2.3.3
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-2.4.0.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 2.4.0
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-2.5.0.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 2.5.0
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-2.5.2.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 2.5.2
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-2.6.0.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 2.6.0
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-2.6.1.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 2.6.1
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-2.6.2.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 2.6.2
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-3.0.3.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 3.0.3
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-3.0.5.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 3.0.5
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-3.0.7.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 3.0.7
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-3.0.8.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 3.0.8
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-3.1.0.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 3.1.0
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-3.1.2.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 3.1.2
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-3.1.3.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 3.1.3
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-3.2.0.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 3.2.0
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-3.3.0.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 3.3.0
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-3.3.1.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 3.3.1
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-3.3.2.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 3.3.2
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-3.3.3.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 3.3.3
    Found link http://pypi.pediapress.com/packages/mirror/simplejson-3.4.0.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version: 3.4.0
  Ignoring link http://pypi.pediapress.com/packages/mirror/simplejson-1.9.2.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version 1.9.2 doesn't match >=2.3
  Ignoring link http://pypi.pediapress.com/packages/mirror/simplejson-2.0.9.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version 2.0.9 doesn't match >=2.3
  Ignoring link http://pypi.pediapress.com/packages/mirror/simplejson-2.1.1.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version 2.1.1 doesn't match >=2.3
  Ignoring link http://pypi.pediapress.com/packages/mirror/simplejson-2.1.6.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version 2.1.6 doesn't match >=2.3
  Ignoring link http://pypi.pediapress.com/packages/mirror/simplejson-2.2.1.tar.gz (from http://pypi.pediapress.com/simple/simplejson/), version 2.2.1 doesn't match >=2.3
  Using version 3.4.0 (newest of versions: 3.4.0, 3.3.3, 3.3.2, 3.3.1, 3.3.0, 3.2.0, 3.1.3, 3.1.2, 3.1.0, 3.0.8, 3.0.7, 3.0.5, 3.0.3, 2.6.2, 2.6.1, 2.6.0, 2.5.2, 2.5.0, 2.4.0, 2.3.3, 2.3.2, 2.3.0)
  Downloading from URL http://pypi.pediapress.com/packages/mirror/simplejson-3.4.0.tar.gz (from http://pypi.pediapress.com/simple/simplejson/)
  Running setup.py egg_info for package simplejson

    running egg_info
    creating pip-egg-info/simplejson.egg-info
    writing pip-egg-info/simplejson.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/simplejson.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/simplejson.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/simplejson.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found


    reading manifest file 'pip-egg-info/simplejson.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'pip-egg-info/simplejson.egg-info/SOURCES.txt'
  Source in /tmp/pip_build_martijn/simplejson has version 3.4.0, which satisfies requirement simplejson>=2.3 (from mwlib)
Downloading/unpacking roman (from mwlib)

  Getting page http://pypi.pediapress.com/simple/roman/
  URLs to search for versions for roman (from mwlib):
  * http://pypi.pediapress.com/simple/roman/
  Analyzing links from page http://pypi.pediapress.com/simple/roman/
    Found link http://pypi.pediapress.com/packages/mirror/roman-1.4.0.tar.gz (from http://pypi.pediapress.com/simple/roman/), version: 1.4.0
    Found link http://pypi.pediapress.com/packages/mirror/roman-2.0.0.zip (from http://pypi.pediapress.com/simple/roman/), version: 2.0.0
  Using version 2.0.0 (newest of versions: 2.0.0, 1.4.0)
  Downloading roman-2.0.0.zip

  Downloading from URL http://pypi.pediapress.com/packages/mirror/roman-2.0.0.zip (from http://pypi.pediapress.com/simple/roman/)
  Running setup.py egg_info for package roman

    running egg_info
    creating pip-egg-info/roman.egg-info
    writing pip-egg-info/roman.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/roman.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/roman.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/roman.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found


    reading manifest file 'pip-egg-info/roman.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no files found matching '*.rst'

    warning: no previously-included files matching '*.pyc' found anywhere in distribution

    writing manifest file 'pip-egg-info/roman.egg-info/SOURCES.txt'
  Source in /tmp/pip_build_martijn/roman has version 2.0.0, which satisfies requirement roman (from mwlib)
Downloading/unpacking gevent (from mwlib)

  Getting page http://pypi.pediapress.com/simple/gevent/
  URLs to search for versions for gevent (from mwlib):
  * http://pypi.pediapress.com/simple/gevent/
  Analyzing links from page http://pypi.pediapress.com/simple/gevent/
    Found link http://pypi.pediapress.com/packages/mirror/gevent-0.12.0.tar.gz (from http://pypi.pediapress.com/simple/gevent/), version: 0.12.0
    Found link http://pypi.pediapress.com/packages/mirror/gevent-0.13.6.tar.gz (from http://pypi.pediapress.com/simple/gevent/), version: 0.13.6
    Found link http://pypi.pediapress.com/packages/mirror/gevent-0.13.7.tar.gz (from http://pypi.pediapress.com/simple/gevent/), version: 0.13.7
    Found link http://pypi.pediapress.com/packages/mirror/gevent-0.13.8.tar.gz (from http://pypi.pediapress.com/simple/gevent/), version: 0.13.8
    Found link http://pypi.pediapress.com/packages/mirror/gevent-1.0a3.tar.gz (from http://pypi.pediapress.com/simple/gevent/), version: 1.0a3
    Found link http://pypi.pediapress.com/packages/mirror/gevent-1.0b1.tar.gz (from http://pypi.pediapress.com/simple/gevent/), version: 1.0b1
    Found link http://pypi.pediapress.com/packages/gevent/gevent-1.0b1-85-gce9936a.zip (from http://pypi.pediapress.com/simple/gevent/), version: 1.0b1-85-gce9936a
    Found link http://pypi.pediapress.com/packages/gevent/gevent-1.0b1-87-g3cfec78.zip (from http://pypi.pediapress.com/simple/gevent/), version: 1.0b1-87-g3cfec78
    Found link http://pypi.pediapress.com/packages/mirror/gevent-1.0b2.tar.gz (from http://pypi.pediapress.com/simple/gevent/), version: 1.0b2
    Found link http://pypi.pediapress.com/packages/gevent/gevent-1.0b2-134-gb3e3099.zip (from http://pypi.pediapress.com/simple/gevent/), version: 1.0b2-134-gb3e3099
    Found link http://pypi.pediapress.com/packages/mirror/gevent-1.0b3.tar.gz (from http://pypi.pediapress.com/simple/gevent/), version: 1.0b3
    Found link http://pypi.pediapress.com/packages/mirror/gevent-1.0rc1.tar.gz (from http://pypi.pediapress.com/simple/gevent/), version: 1.0rc1
    Found link http://pypi.pediapress.com/packages/mirror/gevent-1.0rc2.tar.gz (from http://pypi.pediapress.com/simple/gevent/), version: 1.0rc2
    Found link http://pypi.pediapress.com/packages/mirror/gevent-1.0.tar.gz (from http://pypi.pediapress.com/simple/gevent/), version: 1.0
  Ignoring link http://pypi.pediapress.com/packages/mirror/gevent-1.0a3.tar.gz (from http://pypi.pediapress.com/simple/gevent/), version 1.0a3 is a pre-release (use --pre to allow).
  Ignoring link http://pypi.pediapress.com/packages/mirror/gevent-1.0b1.tar.gz (from http://pypi.pediapress.com/simple/gevent/), version 1.0b1 is a pre-release (use --pre to allow).
  Ignoring link http://pypi.pediapress.com/packages/gevent/gevent-1.0b1-85-gce9936a.zip (from http://pypi.pediapress.com/simple/gevent/), version 1.0b1-85-gce9936a is a pre-release (use --pre to allow).
  Ignoring link http://pypi.pediapress.com/packages/gevent/gevent-1.0b1-87-g3cfec78.zip (from http://pypi.pediapress.com/simple/gevent/), version 1.0b1-87-g3cfec78 is a pre-release (use --pre to allow).
  Ignoring link http://pypi.pediapress.com/packages/mirror/gevent-1.0b2.tar.gz (from http://pypi.pediapress.com/simple/gevent/), version 1.0b2 is a pre-release (use --pre to allow).
  Ignoring link http://pypi.pediapress.com/packages/gevent/gevent-1.0b2-134-gb3e3099.zip (from http://pypi.pediapress.com/simple/gevent/), version 1.0b2-134-gb3e3099 is a pre-release (use --pre to allow).
  Ignoring link http://pypi.pediapress.com/packages/mirror/gevent-1.0b3.tar.gz (from http://pypi.pediapress.com/simple/gevent/), version 1.0b3 is a pre-release (use --pre to allow).
  Ignoring link http://pypi.pediapress.com/packages/mirror/gevent-1.0rc1.tar.gz (from http://pypi.pediapress.com/simple/gevent/), version 1.0rc1 is a pre-release (use --pre to allow).
  Ignoring link http://pypi.pediapress.com/packages/mirror/gevent-1.0rc2.tar.gz (from http://pypi.pediapress.com/simple/gevent/), version 1.0rc2 is a pre-release (use --pre to allow).
  Using version 1.0 (newest of versions: 1.0, 0.13.8, 0.13.7, 0.13.6, 0.12.0)
  Downloading from URL http://pypi.pediapress.com/packages/mirror/gevent-1.0.tar.gz (from http://pypi.pediapress.com/simple/gevent/)
  Running setup.py egg_info for package gevent

    running egg_info
    creating pip-egg-info/gevent.egg-info
    writing requirements to pip-egg-info/gevent.egg-info/requires.txt
    writing pip-egg-info/gevent.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/gevent.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/gevent.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/gevent.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found


    reading manifest file 'pip-egg-info/gevent.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'pip-egg-info/gevent.egg-info/SOURCES.txt'
  Source in /tmp/pip_build_martijn/gevent has version 1.0, which satisfies requirement gevent (from mwlib)
Downloading/unpacking odfpy>=0.9,<0.10 (from mwlib)

  Getting page http://pypi.pediapress.com/simple/odfpy/
  URLs to search for versions for odfpy>=0.9,<0.10 (from mwlib):
  * http://pypi.pediapress.com/simple/odfpy/
  Analyzing links from page http://pypi.pediapress.com/simple/odfpy/
    Found link http://pypi.pediapress.com/packages/mirror/odfpy-0.9.tar.gz (from http://pypi.pediapress.com/simple/odfpy/), version: 0.9
    Found link http://pypi.pediapress.com/packages/mirror/odfpy-0.9.3.tar.gz (from http://pypi.pediapress.com/simple/odfpy/), version: 0.9.3
    Found link http://pypi.pediapress.com/packages/mirror/odfpy-0.9.4.tar.gz (from http://pypi.pediapress.com/simple/odfpy/), version: 0.9.4
    Found link http://pypi.pediapress.com/packages/mirror/odfpy-0.9.5.tar.gz (from http://pypi.pediapress.com/simple/odfpy/), version: 0.9.5
    Found link http://pypi.pediapress.com/packages/mirror/odfpy-0.9.6.tar.gz (from http://pypi.pediapress.com/simple/odfpy/), version: 0.9.6
  Using version 0.9.6 (newest of versions: 0.9.6, 0.9.5, 0.9.4, 0.9.3, 0.9)
  Downloading from URL http://pypi.pediapress.com/packages/mirror/odfpy-0.9.6.tar.gz (from http://pypi.pediapress.com/simple/odfpy/)
  Running setup.py egg_info for package odfpy

    running egg_info
    creating pip-egg-info/odfpy.egg-info
    writing pip-egg-info/odfpy.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/odfpy.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/odfpy.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/odfpy.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found


    reading manifest file 'pip-egg-info/odfpy.egg-info/SOURCES.txt'
    writing manifest file 'pip-egg-info/odfpy.egg-info/SOURCES.txt'
  Source in /tmp/pip_build_martijn/odfpy has version 0.9.6, which satisfies requirement odfpy>=0.9,<0.10 (from mwlib)
Requirement already satisfied (use --upgrade to upgrade): Pillow in /usr/lib/python2.7/dist-packages (from mwlib)

Requirement already satisfied (use --upgrade to upgrade): distribute in /usr/lib/python2.7/dist-packages (from mwlib)

Downloading/unpacking greenlet (from gevent->mwlib)

  Getting page http://pypi.pediapress.com/simple/greenlet/
  URLs to search for versions for greenlet (from gevent->mwlib):
  * http://pypi.pediapress.com/simple/greenlet/
  Analyzing links from page http://pypi.pediapress.com/simple/greenlet/
    Found link http://pypi.pediapress.com/packages/mirror/greenlet-0.2.tar.gz (from http://pypi.pediapress.com/simple/greenlet/), version: 0.2
    Skipping link http://pypi.pediapress.com/packages/mirror/greenlet-0.3.1.win-amd64-py2.7.exe (from http://pypi.pediapress.com/simple/greenlet/); unknown archive format: .exe
    Found link http://pypi.pediapress.com/packages/mirror/greenlet-0.3.1.tar.gz (from http://pypi.pediapress.com/simple/greenlet/), version: 0.3.1
    Found link http://pypi.pediapress.com/packages/mirror/greenlet-0.3.1.zip (from http://pypi.pediapress.com/simple/greenlet/), version: 0.3.1
    Found link http://pypi.pediapress.com/packages/mirror/greenlet-0.3.2.zip (from http://pypi.pediapress.com/simple/greenlet/), version: 0.3.2
    Found link http://pypi.pediapress.com/packages/mirror/greenlet-0.3.3.zip (from http://pypi.pediapress.com/simple/greenlet/), version: 0.3.3
    Found link http://pypi.pediapress.com/packages/mirror/greenlet-0.3.4.zip (from http://pypi.pediapress.com/simple/greenlet/), version: 0.3.4
    Found link http://pypi.pediapress.com/packages/mirror/greenlet-0.4.0.zip (from http://pypi.pediapress.com/simple/greenlet/), version: 0.4.0
    Found link http://pypi.pediapress.com/packages/mirror/greenlet-0.4.1.zip (from http://pypi.pediapress.com/simple/greenlet/), version: 0.4.1
    Found link http://pypi.pediapress.com/packages/mirror/greenlet-0.4.2.zip (from http://pypi.pediapress.com/simple/greenlet/), version: 0.4.2
  Using version 0.4.2 (newest of versions: 0.4.2, 0.4.1, 0.4.0, 0.3.4, 0.3.3, 0.3.2, 0.3.1, 0.3.1, 0.2)
  Downloading from URL http://pypi.pediapress.com/packages/mirror/greenlet-0.4.2.zip (from http://pypi.pediapress.com/simple/greenlet/)
  Running setup.py egg_info for package greenlet

    running egg_info
    creating pip-egg-info/greenlet.egg-info
    writing pip-egg-info/greenlet.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/greenlet.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/greenlet.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/greenlet.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found


    reading manifest file 'pip-egg-info/greenlet.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'pip-egg-info/greenlet.egg-info/SOURCES.txt'
  Source in /tmp/pip_build_martijn/greenlet has version 0.4.2, which satisfies requirement greenlet (from gevent->mwlib)
Installing collected packages: mwlib, pyparsing, timelib, bottle, pyPdf, apipkg, qserve, py, sqlite3dbm, simplejson, roman, gevent, odfpy, greenlet

  Running setup.py install for mwlib

    Running command /usr/bin/python -c "import setuptools;__file__='/tmp/pip_build_martijn/mwlib/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-dtC2qD-record/install-record.txt --single-version-externally-managed
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-2.7
    creating build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/nshandling.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/odfconf.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/trustedrevs.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/serve.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/caller.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/_locale.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/_version.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/__init__.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/old_uparser.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/advtree.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/nslave.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/authors.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/postman.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/utils.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/dummydb.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/bookshelf.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/linuxmem.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/client.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/expander.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/_conf.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/lrucache.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/podclient.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/imgmap.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/mathutils.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/odfstyles.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/wiki.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/uparser.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/metabook.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/odfwriter.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/treecleaner.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/argv.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/main_trampoline.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/nserve.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/tagext.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/options.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/status.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/conf.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/filequeue.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/htmlcolornames.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/writerbase.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/l10n.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/snippets.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/utoken.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/mathml.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/allnodes.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/timeline.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/expr.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/log.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/nuwiki.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/sanitychecker.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/parse_collection_page.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/strftime.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/uniq.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/treecleanerhelper.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/infobox_magic.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/myjson.py -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/dumpparser.py -> build/lib.linux-x86_64-2.7/mwlib
    creating build/lib.linux-x86_64-2.7/mwlib/templ
    copying mwlib/templ/__init__.py -> build/lib.linux-x86_64-2.7/mwlib/templ
    copying mwlib/templ/pp.py -> build/lib.linux-x86_64-2.7/mwlib/templ
    copying mwlib/templ/evaluate.py -> build/lib.linux-x86_64-2.7/mwlib/templ
    copying mwlib/templ/marks.py -> build/lib.linux-x86_64-2.7/mwlib/templ
    copying mwlib/templ/parser.py -> build/lib.linux-x86_64-2.7/mwlib/templ
    copying mwlib/templ/mwlocals.py -> build/lib.linux-x86_64-2.7/mwlib/templ
    copying mwlib/templ/scanner.py -> build/lib.linux-x86_64-2.7/mwlib/templ
    copying mwlib/templ/nodes.py -> build/lib.linux-x86_64-2.7/mwlib/templ
    copying mwlib/templ/magic_nodes.py -> build/lib.linux-x86_64-2.7/mwlib/templ
    copying mwlib/templ/magic_time.py -> build/lib.linux-x86_64-2.7/mwlib/templ
    copying mwlib/templ/misc.py -> build/lib.linux-x86_64-2.7/mwlib/templ
    copying mwlib/templ/magics.py -> build/lib.linux-x86_64-2.7/mwlib/templ
    running egg_info
    writing requirements to mwlib.egg-info/requires.txt
    writing mwlib.egg-info/PKG-INFO
    writing namespace_packages to mwlib.egg-info/namespace_packages.txt
    writing top-level names to mwlib.egg-info/top_level.txt
    writing dependency_links to mwlib.egg-info/dependency_links.txt
    writing entry points to mwlib.egg-info/entry_points.txt
    warning: manifest_maker: standard file '-c' not found


    reading manifest file 'mwlib.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'mwlib.egg-info/SOURCES.txt'
    copying mwlib/EasyTimeline.pl -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/_expander.re -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/_uscan.cc -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/_uscan.re -> build/lib.linux-x86_64-2.7/mwlib
    copying mwlib/snippets.txt -> build/lib.linux-x86_64-2.7/mwlib
    creating build/lib.linux-x86_64-2.7/mwlib/apps
    copying mwlib/apps/__init__.py -> build/lib.linux-x86_64-2.7/mwlib/apps
    copying mwlib/apps/buildzip.py -> build/lib.linux-x86_64-2.7/mwlib/apps
    copying mwlib/apps/client.py -> build/lib.linux-x86_64-2.7/mwlib/apps
    copying mwlib/apps/make_nuwiki.py -> build/lib.linux-x86_64-2.7/mwlib/apps
    copying mwlib/apps/render.py -> build/lib.linux-x86_64-2.7/mwlib/apps
    copying mwlib/apps/serve.py -> build/lib.linux-x86_64-2.7/mwlib/apps
    creating build/lib.linux-x86_64-2.7/mwlib/async
    copying mwlib/async/__init__.py -> build/lib.linux-x86_64-2.7/mwlib/async
    creating build/lib.linux-x86_64-2.7/mwlib/net
    copying mwlib/net/__init__.py -> build/lib.linux-x86_64-2.7/mwlib/net
    copying mwlib/net/fetch.py -> build/lib.linux-x86_64-2.7/mwlib/net
    copying mwlib/net/sapi.py -> build/lib.linux-x86_64-2.7/mwlib/net
    creating build/lib.linux-x86_64-2.7/mwlib/parser
    copying mwlib/parser/__init__.py -> build/lib.linux-x86_64-2.7/mwlib/parser
    copying mwlib/parser/nodes.py -> build/lib.linux-x86_64-2.7/mwlib/parser
    copying mwlib/parser/styleanalyzer.py -> build/lib.linux-x86_64-2.7/mwlib/parser
    creating build/lib.linux-x86_64-2.7/mwlib/refine
    copying mwlib/refine/__init__.py -> build/lib.linux-x86_64-2.7/mwlib/refine
    copying mwlib/refine/_core.pyx -> build/lib.linux-x86_64-2.7/mwlib/refine
    copying mwlib/refine/compat.py -> build/lib.linux-x86_64-2.7/mwlib/refine
    copying mwlib/refine/core.py -> build/lib.linux-x86_64-2.7/mwlib/refine
    copying mwlib/refine/parse_table.py -> build/lib.linux-x86_64-2.7/mwlib/refine
    copying mwlib/refine/tagparser.py -> build/lib.linux-x86_64-2.7/mwlib/refine
    copying mwlib/refine/uparser.py -> build/lib.linux-x86_64-2.7/mwlib/refine
    copying mwlib/refine/util.py -> build/lib.linux-x86_64-2.7/mwlib/refine
    creating build/lib.linux-x86_64-2.7/mwlib/siteinfo
    copying mwlib/siteinfo/__init__.py -> build/lib.linux-x86_64-2.7/mwlib/siteinfo
    copying mwlib/siteinfo/fetch_siteinfo.py -> build/lib.linux-x86_64-2.7/mwlib/siteinfo
    copying mwlib/siteinfo/siteinfo-de.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo
    copying mwlib/siteinfo/siteinfo-en.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo
    copying mwlib/siteinfo/siteinfo-es.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo
    copying mwlib/siteinfo/siteinfo-fr.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo
    copying mwlib/siteinfo/siteinfo-it.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo
    copying mwlib/siteinfo/siteinfo-ja.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo
    copying mwlib/siteinfo/siteinfo-nl.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo
    copying mwlib/siteinfo/siteinfo-no.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo
    copying mwlib/siteinfo/siteinfo-pl.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo
    copying mwlib/siteinfo/siteinfo-pt.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo
    copying mwlib/siteinfo/siteinfo-simple.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo
    copying mwlib/siteinfo/siteinfo-sv.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo
    creating build/lib.linux-x86_64-2.7/mwlib/writer
    copying mwlib/writer/__init__.py -> build/lib.linux-x86_64-2.7/mwlib/writer
    copying mwlib/writer/fontswitcher.py -> build/lib.linux-x86_64-2.7/mwlib/writer
    copying mwlib/writer/formatter.py -> build/lib.linux-x86_64-2.7/mwlib/writer
    copying mwlib/writer/imageutils.py -> build/lib.linux-x86_64-2.7/mwlib/writer
    copying mwlib/writer/licensechecker.py -> build/lib.linux-x86_64-2.7/mwlib/writer
    copying mwlib/writer/miscutils.py -> build/lib.linux-x86_64-2.7/mwlib/writer
    copying mwlib/writer/scripts.txt -> build/lib.linux-x86_64-2.7/mwlib/writer
    copying mwlib/writer/styleutils.py -> build/lib.linux-x86_64-2.7/mwlib/writer
    copying mwlib/writer/wplicenses.csv -> build/lib.linux-x86_64-2.7/mwlib/writer
    copying mwlib/templ/evaluate.c -> build/lib.linux-x86_64-2.7/mwlib/templ
    copying mwlib/templ/nodes.c -> build/lib.linux-x86_64-2.7/mwlib/templ
    running build_ext
    building 'mwlib._uscan' extension

    creating build/temp.linux-x86_64-2.7
    creating build/temp.linux-x86_64-2.7/mwlib
    x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c mwlib/_uscan.cc -o build/temp.linux-x86_64-2.7/mwlib/_uscan.o

    x86_64-linux-gnu-gcc: error trying to exec 'cc1plus': execvp: No such file or directory

    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

    Complete output from command /usr/bin/python -c "import setuptools;__file__='/tmp/pip_build_martijn/mwlib/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-dtC2qD-record/install-record.txt --single-version-externally-managed:

    running install

running build

running build_py

creating build

creating build/lib.linux-x86_64-2.7

creating build/lib.linux-x86_64-2.7/mwlib

copying mwlib/nshandling.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/odfconf.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/trustedrevs.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/serve.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/caller.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/_locale.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/_version.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/__init__.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/old_uparser.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/advtree.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/nslave.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/authors.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/postman.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/utils.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/dummydb.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/bookshelf.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/linuxmem.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/client.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/expander.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/_conf.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/lrucache.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/podclient.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/imgmap.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/mathutils.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/odfstyles.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/wiki.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/uparser.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/metabook.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/odfwriter.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/treecleaner.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/argv.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/main_trampoline.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/nserve.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/tagext.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/options.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/status.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/conf.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/filequeue.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/htmlcolornames.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/writerbase.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/l10n.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/snippets.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/utoken.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/mathml.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/allnodes.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/timeline.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/expr.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/log.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/nuwiki.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/sanitychecker.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/parse_collection_page.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/strftime.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/uniq.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/treecleanerhelper.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/infobox_magic.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/myjson.py -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/dumpparser.py -> build/lib.linux-x86_64-2.7/mwlib

creating build/lib.linux-x86_64-2.7/mwlib/templ

copying mwlib/templ/__init__.py -> build/lib.linux-x86_64-2.7/mwlib/templ

copying mwlib/templ/pp.py -> build/lib.linux-x86_64-2.7/mwlib/templ

copying mwlib/templ/evaluate.py -> build/lib.linux-x86_64-2.7/mwlib/templ

copying mwlib/templ/marks.py -> build/lib.linux-x86_64-2.7/mwlib/templ

copying mwlib/templ/parser.py -> build/lib.linux-x86_64-2.7/mwlib/templ

copying mwlib/templ/mwlocals.py -> build/lib.linux-x86_64-2.7/mwlib/templ

copying mwlib/templ/scanner.py -> build/lib.linux-x86_64-2.7/mwlib/templ

copying mwlib/templ/nodes.py -> build/lib.linux-x86_64-2.7/mwlib/templ

copying mwlib/templ/magic_nodes.py -> build/lib.linux-x86_64-2.7/mwlib/templ

copying mwlib/templ/magic_time.py -> build/lib.linux-x86_64-2.7/mwlib/templ

copying mwlib/templ/misc.py -> build/lib.linux-x86_64-2.7/mwlib/templ

copying mwlib/templ/magics.py -> build/lib.linux-x86_64-2.7/mwlib/templ

running egg_info

writing requirements to mwlib.egg-info/requires.txt

writing mwlib.egg-info/PKG-INFO

writing namespace_packages to mwlib.egg-info/namespace_packages.txt

writing top-level names to mwlib.egg-info/top_level.txt

writing dependency_links to mwlib.egg-info/dependency_links.txt

writing entry points to mwlib.egg-info/entry_points.txt

warning: manifest_maker: standard file '-c' not found



reading manifest file 'mwlib.egg-info/SOURCES.txt'

reading manifest template 'MANIFEST.in'

writing manifest file 'mwlib.egg-info/SOURCES.txt'

copying mwlib/EasyTimeline.pl -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/_expander.re -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/_uscan.cc -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/_uscan.re -> build/lib.linux-x86_64-2.7/mwlib

copying mwlib/snippets.txt -> build/lib.linux-x86_64-2.7/mwlib

creating build/lib.linux-x86_64-2.7/mwlib/apps

copying mwlib/apps/__init__.py -> build/lib.linux-x86_64-2.7/mwlib/apps

copying mwlib/apps/buildzip.py -> build/lib.linux-x86_64-2.7/mwlib/apps

copying mwlib/apps/client.py -> build/lib.linux-x86_64-2.7/mwlib/apps

copying mwlib/apps/make_nuwiki.py -> build/lib.linux-x86_64-2.7/mwlib/apps

copying mwlib/apps/render.py -> build/lib.linux-x86_64-2.7/mwlib/apps

copying mwlib/apps/serve.py -> build/lib.linux-x86_64-2.7/mwlib/apps

creating build/lib.linux-x86_64-2.7/mwlib/async

copying mwlib/async/__init__.py -> build/lib.linux-x86_64-2.7/mwlib/async

creating build/lib.linux-x86_64-2.7/mwlib/net

copying mwlib/net/__init__.py -> build/lib.linux-x86_64-2.7/mwlib/net

copying mwlib/net/fetch.py -> build/lib.linux-x86_64-2.7/mwlib/net

copying mwlib/net/sapi.py -> build/lib.linux-x86_64-2.7/mwlib/net

creating build/lib.linux-x86_64-2.7/mwlib/parser

copying mwlib/parser/__init__.py -> build/lib.linux-x86_64-2.7/mwlib/parser

copying mwlib/parser/nodes.py -> build/lib.linux-x86_64-2.7/mwlib/parser

copying mwlib/parser/styleanalyzer.py -> build/lib.linux-x86_64-2.7/mwlib/parser

creating build/lib.linux-x86_64-2.7/mwlib/refine

copying mwlib/refine/__init__.py -> build/lib.linux-x86_64-2.7/mwlib/refine

copying mwlib/refine/_core.pyx -> build/lib.linux-x86_64-2.7/mwlib/refine

copying mwlib/refine/compat.py -> build/lib.linux-x86_64-2.7/mwlib/refine

copying mwlib/refine/core.py -> build/lib.linux-x86_64-2.7/mwlib/refine

copying mwlib/refine/parse_table.py -> build/lib.linux-x86_64-2.7/mwlib/refine

copying mwlib/refine/tagparser.py -> build/lib.linux-x86_64-2.7/mwlib/refine

copying mwlib/refine/uparser.py -> build/lib.linux-x86_64-2.7/mwlib/refine

copying mwlib/refine/util.py -> build/lib.linux-x86_64-2.7/mwlib/refine

creating build/lib.linux-x86_64-2.7/mwlib/siteinfo

copying mwlib/siteinfo/__init__.py -> build/lib.linux-x86_64-2.7/mwlib/siteinfo

copying mwlib/siteinfo/fetch_siteinfo.py -> build/lib.linux-x86_64-2.7/mwlib/siteinfo

copying mwlib/siteinfo/siteinfo-de.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo

copying mwlib/siteinfo/siteinfo-en.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo

copying mwlib/siteinfo/siteinfo-es.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo

copying mwlib/siteinfo/siteinfo-fr.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo

copying mwlib/siteinfo/siteinfo-it.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo

copying mwlib/siteinfo/siteinfo-ja.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo

copying mwlib/siteinfo/siteinfo-nl.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo

copying mwlib/siteinfo/siteinfo-no.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo

copying mwlib/siteinfo/siteinfo-pl.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo

copying mwlib/siteinfo/siteinfo-pt.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo

copying mwlib/siteinfo/siteinfo-simple.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo

copying mwlib/siteinfo/siteinfo-sv.json -> build/lib.linux-x86_64-2.7/mwlib/siteinfo

creating build/lib.linux-x86_64-2.7/mwlib/writer

copying mwlib/writer/__init__.py -> build/lib.linux-x86_64-2.7/mwlib/writer

copying mwlib/writer/fontswitcher.py -> build/lib.linux-x86_64-2.7/mwlib/writer

copying mwlib/writer/formatter.py -> build/lib.linux-x86_64-2.7/mwlib/writer

copying mwlib/writer/imageutils.py -> build/lib.linux-x86_64-2.7/mwlib/writer

copying mwlib/writer/licensechecker.py -> build/lib.linux-x86_64-2.7/mwlib/writer

copying mwlib/writer/miscutils.py -> build/lib.linux-x86_64-2.7/mwlib/writer

copying mwlib/writer/scripts.txt -> build/lib.linux-x86_64-2.7/mwlib/writer

copying mwlib/writer/styleutils.py -> build/lib.linux-x86_64-2.7/mwlib/writer

copying mwlib/writer/wplicenses.csv -> build/lib.linux-x86_64-2.7/mwlib/writer

copying mwlib/templ/evaluate.c -> build/lib.linux-x86_64-2.7/mwlib/templ

copying mwlib/templ/nodes.c -> build/lib.linux-x86_64-2.7/mwlib/templ

running build_ext

building 'mwlib._uscan' extension

creating build/temp.linux-x86_64-2.7

creating build/temp.linux-x86_64-2.7/mwlib

x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c mwlib/_uscan.cc -o build/temp.linux-x86_64-2.7/mwlib/_uscan.o

x86_64-linux-gnu-gcc: error trying to exec 'cc1plus': execvp: No such file or directory

error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

----------------------------------------

Cleaning up...

  Removing temporary dir /tmp/pip_build_martijn...
Command /usr/bin/python -c "import setuptools;__file__='/tmp/pip_build_martijn/mwlib/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-dtC2qD-record/install-record.txt --single-version-externally-managed failed with error code 1 in /tmp/pip_build_martijn/mwlib

Exception information:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 134, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 241, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1298, in install
    requirement.install(install_options, global_options, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 625, in install
    cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False)
  File "/usr/lib/python2.7/dist-packages/pip/util.py", line 670, in call_subprocess
    % (command_desc, proc.returncode, cwd))
InstallationError: Command /usr/bin/python -c "import setuptools;__file__='/tmp/pip_build_martijn/mwlib/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-dtC2qD-record/install-record.txt --single-version-externally-managed failed with error code 1 in /tmp/pip_build_martijn/mwlib

This is on Linux Mint 16. Fully up-to-date (apart from python, as instructed by the docs). Any ideas?

There is no default image alignment

I had to change align = '' to align = 'None' in class ImageLink (nodes.py), to be able to parse mediawiki articles which have images defined without a set alignment.

No warning of missing templates

I was getting some weird behavior (while parsing List of Governors of Alabama, for what it's worth) where table cells that should have contained ArticleLinks contained just newlines. I quickly realized that the cause must be that they were wrapped in the {{sortname}} template, but it wasn't until after a lot of digging inside core.py and expander.py that I determined that it was because that template didn't exist.

As it happens, I was working with a small subset of pages in my CDB, which among other things had filtered out templates. I realize this is a noob mistake to leave out the templates, but it would have been helpful if getParsedArticle had printed a warning or thrown an exception instead of just silently deleting apparently simple page content. I'm not sure what the best way to do this is, or if it's even widely desirable behavior, so I don't have a specific patch proposal.

Add support for python3

With python2 being discontinued in 2020, python3 support should be added.

Also some linux dist have switched to python3.

Link text

According to http://en.wikipedia.org/wiki/Help:Link in case of [[a|b]]cd text of this link is bcd. Mwlib separates cd from link and creates text node at the same level as link node. Is it possible to obtain bcd as link text?

Books not regenerated when cache is deleted

When running 'mw-serve-ctl --cache-dir=/tmp/mwlib --purge-cache=0' books are not regenerated.
Restarting mw-qserve fixes this issue, but it doesn't seem like a good solution.

mwlib fails to fetch images if img_auth is enabled

The problem is in the download_to_file in mwlib/net/fetch.py: the CookieJar from the mwapi class is not forwarded to the new urllib2 opener.

Missing 'docs/configfiles.txt' but it remains referenced in documentation

In docs/commands.rst, it twice mentions what sounds like a very helpful file, docs/configfiles.txt:

The value for this option describes the source of MediaWiki articles and images for the command and can be of one of the following types:
...
This value for --config corresponds to type=mwapi in a configuration file (see docs/configfiles.txt),
...

A filename of a configuration file (see docs/configfiles.txt).

However, there appears to be no configfiles.txt in the docs folder. A quick gander at the revision history through the tags shows that it was removed before version 0.13. Where now, then, is the documentation on writing a config file and the options available? Surely the existing text referring to docs/configfiles.txt should be updated to wherever that information is now, right? I see that 554d0de removed configfiles.txt, but there would still seem to be a need for such documentation of configuration files since for instance mw-zip reports

mw-zip --help
Usage: mw-zip [OPTIONS] [ARTICLETITLE...]

Options:
  -h, --help            show this help message and exit
  -c CONFIG, --config=CONFIG
                        configuration file, ZIP file or base URL
...

so configuration files are presumably still supported...

Inline "thumb" images lost in ODF export

When exporting to odf, inline "thumb" images are missing in the resulting file. The situation is a little tricky, because with the pediapress renderer, if immediately ODF is requested, images may not be fetched at all, so:

Save Collection as Book
From the Books page download the PDF first (which has thumb images)
From the Books page download the ODF (images get in from the cache)

The resulting ODF has a Pictures sub-folder with all the images inside, yet they are not displayed in Writer.

unzip the downloaded ODF, prettify content.xml (eg. xmllint -format)
in content.xml move any "draw:frame…" nodes below (hierarchically spoken) the next "text:p…" node
zip the directory to "newfile.odf"

The new file shows the inline images perfectly!

http://odf-validator.rhcloud.com/ does not find any errors in content.xml. Here's to the specification http://docs.oasis-open.org/office/v1.2/cd05/OpenDocument-v1.2-cd05-part1.html#element-draw_frame Should be OK as is, but lo!

Wrong color in pdfs

Note: I'm not sure whether this issue come from mwlib or mwlib.rl.

If I put tags "font color=red" the text in the wiki is red, BUT in the pdf it appears green !
I also tried other colours :

green appears black
yellow → cyan
blue → blue

Quite odd isn't it ?

Thing funny I noticed: if you put font color=red in the pageheader variable (for instance), it works !
It seems the parser is in cause.

document wgCollectionPODPartners

wgCollectionPODPartners needs documentation. see https://bugzilla.wikimedia.org/show_bug.cgi?id=33536

Use apipkg >=1.2 instead of mwlib.apipkg

apipkg 1.2 has been released on PyPI yesterday. Its CHANGELOG contains:

1.2
----------------------------------------

- Allow to import from Aliasmodules  (thanks Ralf Schmitt)
- avoid loading __doc__ early, so ApiModule is now fully lazy

It should be now possible to revert commit 3a20582, which added mwlib.apipkg. setup.py currently specifies dependency on apipkg even thought mwlib.apipkg is actually used. If commit 3a20582 is reverted, then dependency on apipkg in setup.py should be raised to >=1.2.

Parser test test_pull_in_alpha

This test should pass.

mwlib provides top-level argv.py file, which can pollute the Python package namespace

As part of the review process for getting mwlib into Fedora, the package reviewer stated the following:

Potential future issues:

* The package installs a toplevel argv.py file/module.  This seems to just be
  used for the command line scripts for parsing the command line.  Since this
  has the potential to pollute the python package namespace with a somewhat
  common name, suggest to upstream that they may want to restructure like
  this::
    mv argv.py mwlib/._argv.py
    sed s/import argv/from mwlib import _argv as argv/

Reference: https://bugzilla.redhat.com/show_bug.cgi?id=790154#c1

bail out early in setup.py when using a unsupported python version

We should bail out with a nice error message if people try to use an unsupported python version. people don't read the docs.

test suite in mwlib-0.13.7 errors out for one apparent reason

Having all the new deps for this version installed, the test suite should have all that is needed.
it appears to run them all with success but from the ebuild it bails out claiming the test phase failed.

creating build-2.7/scripts-2.7
copying and adjusting sandbox/nslave.py -> build-2.7/scripts-2.7
copying and adjusting sandbox/nserve.py -> build-2.7/scripts-2.7
copying and adjusting sandbox/postman.py -> build-2.7/scripts-2.7
changing mode of build-2.7/scripts-2.7/nslave.py from 644 to 755
changing mode of build-2.7/scripts-2.7/nserve.py from 644 to 755
changing mode of build-2.7/scripts-2.7/postman.py from 644 to 755

Source compiled.

Testing of dev-python/mwlib-0.13.7 with CPython 2.7...

PYTHONPATH=build-2.7/lib.linux-x86_64-2.7 py.test
============================= test session starts ==============================
platform linux2 -- Python 2.7.2 -- pytest-2.2.3
collecting 0 items * ERROR: dev-python/mwlib-0.13.7 failed (test phase):

Testing failed with CPython 2.7 in python_test_function() function
Call stack:
```
ebuild.sh, line   85:  Called src_test
```
environment, line 3751: Called distutils_src_test
environment, line 1205: Called python_execute_py.test '-P' '$(_distutils_get_PYTHONPATH)' '--'
environment, line 2232: Called python_execute_function 'python_test_function'
environment, line 2116: Called die
The specific snippet of code:

                  die "${failure_message}";

If you need support, post the output of 'emerge --info =dev-python/mwlib-0.13.7',
the complete build log and the output of 'emerge -pqv =dev-python/mwlib-0.13.7'.
The complete build log is located at '/mnt/gen2/TmpDir/portage/dev-python/mwlib-0.13.7/temp/build.log'.

However; run from source directory it appears to achieve success.

archtester mwlib # cd /mnt/gen2/TmpDir/portage/dev-python/mwlib-0.13.7/work/mwlib-0.13.7/tests/
archtester tests # cd ../
archtester mwlib-0.13.7 # py.test -v tests
=============================================== test session starts ===============================================
platform linux2 -- Python 2.7.2 -- pytest-2.2.3 -- /usr/bin/python2.7
collecting 389 itemsarchtester mwlib-0.13.7 #

I have no idea where to look into this seeing it offers no debug output since it seems to have not tripped over any error. It may be from tests, which require installed mw-zip script.. This is just the type of thing that plays havoc with the sandbox structure setup by portage. The test suites would be much more distro friendly by not doinf such things.

gevent.coros missing

Semaphore from gevent.coros is used in fetch.py and sapi.py, but Semaphore is in gevent.lock.

This causes the first line of the test to fail:
mw-zip -c :en -o test.zip Acdc Number
(from http://mwlib.readthedocs.io/en/latest/installation.html#test-install)

Replacing 'coros' with 'lock' fixes it.

This may be an issue with gevent not deprecating coros nicely but thought you should know :)

No dep on gevent listed

While using FPM to build this, the dependencies listed did not include gevent. It took me a while (not being an intimate of python) to track down the dependency. Here is the list of dependencies that fpm discovered:

Requires: python-pyparsing >= 1.4.11
Requires: python-pyparsing < 1.6
Requires: python-timelib >= 0.2
Requires: python-bottle >= 0.10
Requires: python-pypdf >= 1.12
Requires: python-apipkg >= 1.2
Requires: python-qserve >= 0.2.7
Requires: python-lxml
Requires: python-py >= 1.4
Requires: python-sqlite3dbm
Requires: python-simplejson >= 2.3
Requires: python-roman
Requires: python-gevent
Requires: python-odfpy >= 0.9
Requires: python-odfpy < 0.10
Requires: python-pillow
Requires: python-setuptools

MD5 with FIPS mode enabled

The following error occurs when FIPS mode on RHEL 7 is enabled:

Traceback (most recent call last):
File "/usr/bin/bottle.py", line 862, in _handle
return route.call(**args)
File "/usr/bin/bottle.py", line 1740, in wrapper
rv = callback(*a, **ka)
File "/usr/lib64/python2.7/site-packages/mwlib/nserve.py", line 171, in dispatch_command
return Application().dispatch(request)
File "/usr/lib64/python2.7/site-packages/mwlib/nserve.py", line 220, in dispatch
collection_id = self.new_collection(request.params)
File "/usr/lib64/python2.7/site-packages/mwlib/nserve.py", line 263, in new_collection
collection_id = make_collection_id(post_data)
File "/usr/lib64/python2.7/site-packages/mwlib/nserve.py", line 79, in make_collection_id
sio.write(calc_checksum(mbobj))
File "/usr/lib64/python2.7/site-packages/mwlib/metabook.py", line 211, in calc_checksum
return md5(metabook.dumps()).hexdigest()
ValueError: error:060800A3:digital envelope routines:EVP_DigestInit_ex:disabled for fips

This is happening because MD5 is not allowed if the server is to be FIPS compliant. Line 211 of metabook.py is:

def calc_checksum(metabook):
return md5(metabook.dumps()).hexdigest()

pyparsing-2* support

Is the code still bound to the old versions of pyparsing?

Items using the convert template give error unknown operator: u','

Articles using the convert template are rendered with errors. See the Atlantic Ocean article, for example, which uses the code {{convert|106,400,000|km2|sqmi}} in the opening paragraph to display the size of the ocean in miles.

The text should be rendered as 41,100,000 sq mi but instead appears as unknown operator: u','

mw-zip -c :en -o test.zip Atlantic_ocean
mw-render -c test.zip -o test.pdf -w rl

mwlib doesn't recognize noinclude tags if they contain spaces.

expanding noinclude tags containing spaces results in:

In [9]: expander.expandstr("<noinclude >foo</noinclude>")
EXPAND: '<noinclude >foo</noinclude>' -> u'<noinclude >foo'
Out[9]: u'<noinclude >foo'

The same might be true for includeonly, onlyinclude (and others?).

Error messages should be simplified in some cases

One of MediaWiki users reported recently this error (it's here reformatted, since the HTML output is in plain):

An error occured on the render server: 
RuntimeError: 
RuntimeError: command failed with returncode 256: ['mw-zip', 
   '-o', '/home/pp/cache/27/276bad5cc679d168/collection.zip', 
   '-m', '/home/pp/cache/27/276bad5cc679d168/metabook.json', 
   '--status', 'qserve://localhost:14311/276bad5cc679d168:makezip', 
   '--template-blacklist', 'MediaWiki:PDF Template Blacklist', 
   '--template-exclusion-category', 'Exclude in print', 
   '--print-template-prefix', 'Print', 
   '--print-template-pattern', '$1/Print', 
   '--username', 'wikicollect', '--password', '{OMITTED}']
Last Output:
    2012-06-19T21:24:01 mwlib.options.warn &gt;&gt; Both --print-template-pattern and 
          --print-template-prefix (deprecated) specified. Using --print-template-pattern only.
    1%  creating nuwiki in u'/home/pp/cache/27/276bad5cc679d168/tmpbw0czq/nuwiki'
    removing tmpdir u'/home/pp/cache/27/276bad5cc679d168/tmpbw0czq'
    memory used: res=16.4 virt=123.1
    1% error Traceback (most recent call last):
      File "/home/pp/local/bin/mw-zip", line 9, in &lt;module&gt;
        load_entry_point('mwlib==0.13.6', 'console_scripts', 'mw-zip')()
      File "/home/pp/local/lib/python2.7/site-packages/mwlib/apps/buildzip.py", line 155, in main
        make_zip(output, options, env.metabook, podclient=podclient, status=status)
      File "/home/pp/local/lib/python2.7/site-packages/mwlib/apps/buildzip.py", line 50, in make_zip
        make_nuwiki(fsdir, metabook=metabook, options=options, podclient=podclient, status=status)
      File "/home/pp/local/lib/python2.7/site-packages/mwlib/apps/make_nuwiki.py", line 152, in make_nuwiki
        assert x.wikiident in id2wiki, "no wikiconf for %r (%s)" % (x.wikiident,  x)
    AssertionError: no wikiconf for None (&lt;article {'_env': &lt;mwlib.wiki.Environment object at 0x1d8d6d0&gt;, 
           'content_type': u'text/x-wiki', 'title': u'Main Page', 'timestamp': u'1325800496', 'type': 'article', 'revision': u'1'}&gt;)
     in function system, file ./bin/nslave.py, line 63 in function qaddw, file /home/pp/local/lib/python2.7/site-packages/qs/slave.py, line 66

This is basically saying "Could not figure out your MediaWiki URL configuration", as his Short URL setup prevented PediaPress service to access this wiki properly.

This verbose output was helpful to figure out what's wrong, but maybe in some cases a dedicated exception should be delivered.

links containing square brackets are treated differently than by mediawiki

It seems like mwlib treats links containing triple brackets a bit
different than Mediawiki.

For example: "[[WP:IPA for German|[ˈalbɐt ˈaɪnʃtaɪn]]]" is rendered
like this
start_link[ˈalbɐt ˈaɪnʃtaɪn]end_link
by the Wikipedia sandbox, while parser.ParseString() creates this tree:

NamespaceLink target=u'WP:IPA for German' ns=4
u'[\u02c8alb\u0250t \u02c8a\u026an\u0283ta\u026an'
u']\n'

(with the right square bracket is outside the link)

If the link text is removed, like this "[[[ˈalbɐt ˈaɪnʃtaɪn]]]",
Wikipedia displays it as regular text while ParseString creates a
tree that resembles the one above.

I've created a tiny patch that
edit: dosn't work as advertised, sorry.

update changelog and documentation about "template blacklisting"

template blacklisting has been removed as of mwlib 0.15 and it's not going to come back. the documentation lacks behind and the changelog doesn't mention it explicitly (though it does mention that specific change that caused it).

Error while dispatching download

I start my servers like so, and can render articles no problem:

mw-qserve -p 14311 -i 127.0.0.1 &
nserve --port 8899 --qserve 127.0.0.1:14311 &
nslave --serve-files-port 8898 --cachedir /var/cache/mwlib/ &

But I get the following output when trying to download the rendered PDF:

ERROR while dispatching 'download': {'qserve': ('localhost', 14311), 'collection_id': '4a3642d5dcbcbc22', 'is_new': False}
Traceback (most recent call last):
  File "/usr/local/python27/lib/python2.7/site-packages/mwlib/nserve.py", line 238, in dispatch
    return method(collection_id, request.params, is_new)
  File "/usr/local/python27/lib/python2.7/site-packages/mwlib/nserve.py", line 384, in do_download
    download_url = res["result"]["url"]
KeyError: 'result'
Traceback (most recent call last):
  File "/usr/local/python27/bin/bottle.py", line 862, in _handle
    return route.call(**args)
  File "/usr/local/python27/bin/bottle.py", line 1732, in wrapper
    rv = callback(*a, **ka)
  File "/usr/local/python27/lib/python2.7/site-packages/mwlib/nserve.py", line 171, in dispatch_command
    return Application().dispatch(request)
  File "/usr/local/python27/lib/python2.7/site-packages/mwlib/nserve.py", line 243, in dispatch
    raise exc
KeyError: 'result'

Is this a known problem or a bug? I checked the cache directory for that collection ID and the PDF does exist at /var/cache/mwlib/4a/4a3642d5dcbcbc22/output.rl.

mwlib fails when MediaWiki is behind authenticating reverse proxy

We are running a MediaWiki behind a Apache Reverse Proxy that authenticates users and passes this information to MediaWiki. When trying to setup Collection extension I was unable to pdf to render as it was unable to connect. These are some steps I took:

Supply user/password, still failed. No log entries found on apache, error 401
Provide details of self signed cert, as there were no log entries in apache I thought it might be because of this. still error 401
Buy a real ssl certs! As there were no log entries in apache it must be because something goes wrong before the connection is established. Still error 401
Install my own rendering service.... still no luck! Still error 401
Connect to MediaWiki direct, by passing the reverse proxy. MediaWiki throws an exception as no credentials are supplied.

In the end I checked the code and found that the ssl connection is build up without supplying the username and password. The username and password are used only to connect in the API login call.

This was easily fixed by adding the password manager to the build_opener, now the connection can opened and the API can be reached.
See my HARD CODED (sorry) solution below, I think this can be generally adopted as the password manager only is used when required.

class mwapi(object):
    def __init__(self, apiurl):
        self.apiurl = apiurl
        self.baseurl = apiurl  
        passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
        passman.add_password(None, apiurl, 'username', 'password')
        self.auth_handler = urllib2.HTTPBasicAuthHandler(passman)
        self.opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookielib.CookieJar()), self.auth_handler)

mwlib should pass cookies for authentication for login-required wikis

As per subject. See https://phabricator.wikimedia.org/T142924 and the related code.

That hack wouldn't be necessary if there were a way that mwlib could pass cookies the way that parsoid does.

test_zipwiki.Test_xnet_zipwiki.test_getTemplate() fails

test_zipwiki.Test_xnet_zipwiki.test_getTemplate() fails in mwlib 0.15.3 (with both Python 2.6 and 2.7). It passes in mwlib 0.14.3.

=================================== FAILURES ===================================
______________________ Test_xnet_zipwiki.test_getTemplate ______________________

self = <test_zipwiki.Test_xnet_zipwiki object at 0x3d5c290>

    def test_getTemplate(self):
        page = self.wikidb.normalize_and_get_page(u'Infobox Film', 10)
>       assert isinstance(page.rawtext, unicode)
E       AttributeError: 'NoneType' object has no attribute 'rawtext'

tests/test_zipwiki.py:47: AttributeError

TypeError when expanding NAMESPACE

[py27]  ~/ % mw-render -c :zh -o test.pdf -w rl '维基百科'
MISSING FONTS: 'AR PL UMing HK','Nazli','UnBatang','Arundina Serif','Lohit Telugu','Sarai','Lohit Punjabi','Lohit Oriya','AnjaliOldLipi','Kedage','LikhanNormal','Lohit Tamil','Khmer'
creating nuwiki in '/tmp/tmps_8Rgt/nuwiki'
removing tmpdir '/tmp/tmps_8Rgt'
memory used: res=33.7 virt=164.3
reading /tmp/tmpILSK0Q/revisions-1.txt
removing '/tmp/tmpILSK0Q'
Traceback (most recent call last):
  File "/home/ralf/local/bin/mw-render", line 9, in <module>
    load_entry_point('mwlib==0.13.1', 'console_scripts', 'mw-render')()
  File "/home/ralf/code/mwlib/apps/render.py", line 211, in main
    return Main()()
  File "/home/ralf/code/mwlib/apps/render.py", line 176, in __call__
    writer(env, output=tmpout, status_callback=self.status, **writer_options)
  File "/home/ralf/rl/mwlib/rl/rlwriter.py", line 2179, in writer
    r.writeBook(output=output, coverimage=coverimage, status_callback=status_callback)
  File "/home/ralf/rl/mwlib/rl/rlwriter.py", line 467, in writeBook
    art = self.buildArticle(item)
  File "/home/ralf/rl/mwlib/rl/rlwriter.py", line 356, in buildArticle
    revision=item.revision)
  File "/home/ralf/code/mwlib/nuwiki.py", line 422, in getParsedArticle
    return uparser.parseString(title=title, raw=raw, wikidb=self, lang=self.siteinfo["general"]["lang"])
  File "/home/ralf/code/mwlib/refine/uparser.py", line 34, in parseString
    input = te.expandTemplates(True)
  File "evaluate.py", line 295, in mwlib.templ.evaluate.Expander.expandTemplates (mwlib/templ/evaluate.c:5930)
  File "evaluate.py", line 282, in mwlib.templ.evaluate.Expander._expand (mwlib/templ/evaluate.c:5567)
  File "evaluate.py", line 28, in mwlib.templ.evaluate.flatten (mwlib/templ/evaluate.c:1103)
  File "evaluate.py", line 30, in mwlib.templ.evaluate.flatten (mwlib/templ/evaluate.c:1134)
  File "nodes.py", line 210, in mwlib.templ.nodes.Template.flatten (mwlib/templ/nodes.c:4619)
  File "nodes.py", line 291, in mwlib.templ.nodes.Template._flatten (mwlib/templ/nodes.c:5976)
  File "evaluate.py", line 28, in mwlib.templ.evaluate.flatten (mwlib/templ/evaluate.c:1103)
  File "evaluate.py", line 30, in mwlib.templ.evaluate.flatten (mwlib/templ/evaluate.c:1134)
  File "nodes.py", line 210, in mwlib.templ.nodes.Template.flatten (mwlib/templ/nodes.c:4619)
  File "nodes.py", line 277, in mwlib.templ.nodes.Template._flatten (mwlib/templ/nodes.c:5777)
  File "/home/ralf/code/mwlib/templ/magics.py", line 563, in __call__
    res = m(args) or ''  # FIXME: catch TypeErros
  File "/home/ralf/code/mwlib/templ/magics.py", line 209, in wrapper
    return f(self, pagename)
  File "/home/ralf/code/mwlib/templ/magics.py", line 254, in NAMESPACE
    ns, partial, full = self.nshandler.splitname(pagename)
  File "/home/ralf/code/mwlib/nshandling.py", line 128, in splitname
    title = unicode(title, 'utf-8')
TypeError: coercing to Unicode: need string or buffer, tuple found

Support multiwiki metabook

Currently, articles with an interwiki title are not parsed correctly by the render server. Wiki identity should be determined from the title, and the appropriate wikiconf loaded.

We can match the target article's base url against the siteinfo array, which saves us the trouble of parsing interwiki link syntax.

implement "Labeled Section Transclusion" functionality

wikisource uses it, see http://www.mediawiki.org/wiki/Extension:Labeled_Section_Transclusion for more information

Accept pyparsing 2.0.1

pyparsing 2.0.0 does not support Python 2, but pyparsing 2.0.1 restored support for Python 2.6 and 2.7. Dependency in setup.py of mwlib should be updated.

--- setup.py
+++ setup.py
@@ -42,7 +42,7 @@
         build_deps()   # this is a git clone

     install_requires = [
-        "pyparsing>=1.4.11,<1.6", "timelib>=0.2", "bottle>=0.10",
+        "pyparsing>=1.4.11,!=2.0.0", "timelib>=0.2", "bottle>=0.10",
         "pyPdf>=1.12", "apipkg>=1.2", "qserve>=0.2.7", "lxml",
         "py>=1.4", "sqlite3dbm", "simplejson>=2.3", "roman", "gevent",
         "odfpy>=0.9, <0.10", "Pillow", "setuptools"]

link not detected

In the example below (line 2) the link is not detected if the protocol is not specified. Line 3 shows the working example

In [1]: from mwlib.uparser import simpleparse as p

In [2]: p('[//toolserver.org/~geohack/geohack.php?pagename=Benutzer%3AVolker.haas/Test&language=de&params=51.4213888889_N_9.64805555556_E_dim:100_region:DE-NI_type:waterbody&title=Namentlicher+Beginn+Weser bla]')
Article
    u'['
    u'//toolserver.org/~geohack/geohack.php?pagename=Benutzer%3AVolker.haas/Test&language=de&params=51.4213888889_N_9.64805555556_E_dim'
    u':'
    u'100_region'
    u':'
    u'DE-NI_type'
    u':'
    u'waterbody&title=Namentlicher+Beginn+Weser bla'
    u']'
Out[2]: Article

In [3]: p('[http://toolserver.org/~geohack/geohack.php?pagename=Benutzer%3AVolker.haas/Test&language=de&params=51.4213888889_N_9.64805555556_E_dim:100_region:DE-NI_type:waterbody&title=Namentlicher+Beginn+Weser bla]')
Article
    NamedURL->u'http://toolserver.org/~geohack/geohack.php?pagename=Benutzer%3AVolker.haas/Test&language=de&params=51.4213888889_N_9.64805555556_E_dim:100_region:DE-NI_type:waterbody&title=Namentlicher+Beginn+Weser'
        u' bla'
Out[3]: Article

{{formatnum}} for numbers <10000

In Russian Wikipedia generating of PDF for page with {{formatnum:2000}} throws error.
Pages with {{formatnum:2 000}} (wrong input) or {{formatnum:20000}} generates without errors.

In Russian there is difference between format number lower and greater 10000: {{formatnum:2000}} → 2000, {{formatnum:20000}} → 20 000.

Error log:

RuntimeError: command failed with returncode 256: ['mw-render', '-w', 'rl', '-c', 'cache/1f/1fe54c6375ddf613/collection.zip', '-o', 'cache/1f/1fe54c6375ddf613/output.rl', '--status', 'qserve://localhost:14311/1fe54c6375ddf613:render-rl', '--template-blacklist', 'MediaWiki:PDF Template Blacklist', '--template-exclusion-category', u'\u0412\u0438\u043a\u0438\u043f\u0435\u0434\u0438\u044f:\u0418\u0441\u043a\u043b\u044e\u0447\u0435\u043d\u0438\u044f \u0438\u0437 \u043f\u0435\u0447\u0430\u0442\u0438', '--print-template-prefix', u'\u041f\u0435\u0447\u0430\u0442\u044c', '--print-template-pattern', u'$1/\u041f\u0435\u0447\u0430\u0442\u044c', '--language', 'ru']
Last Output:
    2012-12-03T02:44:49 mwlib.options.warn >> Both --print-template-pattern and --print-template-prefix (deprecated) specified. Using --print-template-pattern only.
    MISSING FONTS: 'Gujarati'
    1%  reading /tmp/tmp-mw-renderAp_clG/tmpsh4z6_/revisions-1.txt
    set locale to 'ru_RU' based on the language 'ru'
    1% laying out 0% error removing '/tmp/tmp-mw-renderAp_clG/tmpsh4z6_'
    Traceback (most recent call last):
      File "/home/pp/local/bin/mw-render", line 6, in <module>
        main()
      File "/home/pp/local/lib/python2.6/site-packages/mwlib/apps/render.py", line 241, in main
        return Main()()
      File "/home/pp/local/lib/python2.6/site-packages/mwlib/apps/render.py", line 206, in __call__
        writer(env, output=tmpout, status_callback=self.status, **writer_options)
      File "/home/pp/local/lib/python2.6/site-packages/mwlib/rl/rlwriter.py", line 2188, in writer
        r.writeBook(output=output, coverimage=coverimage, status_callback=status_callback)
      File "/home/pp/local/lib/python2.6/site-packages/mwlib/rl/rlwriter.py", line 469, in writeBook
        art = self.buildArticle(item)
      File "/home/pp/local/lib/python2.6/site-packages/mwlib/rl/rlwriter.py", line 358, in buildArticle
        revision=item.revision)
      File "/home/pp/local/lib/python2.6/site-packages/mwlib/nuwiki.py", line 421, in getParsedArticle
        return uparser.parseString(title=title, raw=raw, wikidb=self, lang=self.siteinfo["general"]["lang"])
      File "/home/pp/local/lib/python2.6/site-packages/mwlib/refine/uparser.py", line 34, in parseString
        input = te.expandTemplates(True)
      File "evaluate.py", line 295, in mwlib.templ.evaluate.Expander.expandTemplates (mwlib/templ/evaluate.c:6507)
      File "evaluate.py", line 285, in mwlib.templ.evaluate.Expander._expand (mwlib/templ/evaluate.c:6107)
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xa0 in position 1: ordinal not in range(128)
     in function system, file /home/pp/local/bin/nslave.py, line 64

test fails in tests/test_expander_time.py

$ py.test tests/
...
============================================ FAILURES =============================================
_________________________________________ test_codes[43] __________________________________________

s = 'xrU', expected = 'XVI', date = '1970-1-1 + 16 second'

    def e(s, expected, date="09 Feb 2008 10:55:17"):
>       expandstr(u'{{#time:%s|%s}}' % (s, date), expected)

tests/test_expander_time.py:8: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

s = '{{#time:xrU|1970-1-1 + 16 second}}', expected = 'XVI', wikidb = None, pagename = 'thispage'

    def expandstr(s, expected=None, wikidb=None, pagename='thispage'):
        """debug function. expand templates in string s"""
        if wikidb:
            db = wikidb
        else:
            db = DictDB(dict(a=s))

        te = evaluate.Expander(s, pagename=pagename, wikidb=db)
        res = te.expandTemplates()
        print "EXPAND: %r -> %r" % (s, res)
        if expected is not None:
>           assert res==expected, "expected %r, got %r" % (expected, res)
E           AssertionError: expected 'XVI', got u'<strong class="error">Error: invalid time</strong>'

/usr/lib64/python2.7/site-packages/mwlib/templ/misc.py:49: AssertionError
-------------------------------------- Captured stdout call ---------------------------------------
EXPAND: u'{{#time:xrU|1970-1-1 + 16 second}}' -> u'<strong class="error">Error: invalid time</strong>'
_____________________________________ test_time_minus_days[0] _____________________________________

s = '{{#time:Y-m-d| 20070827000000 -12 day}}', expected = '2007-08-15', wikidb = None
pagename = 'thispage'

    def expandstr(s, expected=None, wikidb=None, pagename='thispage'):
        """debug function. expand templates in string s"""
        if wikidb:
            db = wikidb
        else:
            db = DictDB(dict(a=s))

        te = evaluate.Expander(s, pagename=pagename, wikidb=db)
        res = te.expandTemplates()
        print "EXPAND: %r -> %r" % (s, res)
        if expected is not None:
>           assert res==expected, "expected %r, got %r" % (expected, res)
E           AssertionError: expected '2007-08-15', got u'<strong class="error">Error: invalid time</strong>'

/usr/lib64/python2.7/site-packages/mwlib/templ/misc.py:49: AssertionError
-------------------------------------- Captured stdout call ---------------------------------------
EXPAND: '{{#time:Y-m-d| 20070827000000 -12 day}}' -> u'<strong class="error">Error: invalid time</strong>'

This is with today's git (c41330f), but those two tests fail also with mwlib 0.15.11 and 0.15.14.

command failed with returncode 256

url:
http://it.wikisource.org/w/index.php?title=Speciale:Libro&bookcmd=rendering&return_to=Adelchi%2FNotizie+storiche&collection_id=ffd2e4924f536f25&writer=rl

output:

Si è verificato un errore sul server di conversione: RuntimeError: command failed with returncode 256: ['mw-render', '-w', 'rl', '-c', 'cache/ff/ffd2e4924f536f25/collection.zip', '-o', 'cache/ff/ffd2e4924f536f25/output.rl', '--status', 'qserve://localhost:14311/ffd2e4924f536f25:render-rl', '--template-blacklist', 'MediaWiki:PDF Template Blacklist', '--template-exclusion-category', 'Escludi dalla stampa', '--print-template-prefix', 'Stampa', '--print-template-pattern', '$1/Stampa', '--language', 'it'] Last Output: riteBook(output=output, coverimage=coverimage, status_callback=status_callback) File "/home/pp/local/lib/python2.6/site-packages/mwlib.rl-0.12.10-py2.6.egg/mwlib/rl/rlwriter.py", line 469, in writeBook art = self.buildArticle(item) File "/home/pp/local/lib/python2.6/site-packages/mwlib.rl-0.12.10-py2.6.egg/mwlib/rl/rlwriter.py", line 358, in buildArticle revision=item.revision) File "/home/pp/local/lib/python2.6/site-packages/mwlib-0.13.2-py2.6-linux-x86_64.egg/mwlib/nuwiki.py", line 421, in getParsedArticle return uparser.parseString(title=title, raw=raw, wikidb=self, lang=self.siteinfo["general"]["lang"]) File "/home/pp/local/lib/python2.6/site-packages/mwlib-0.13.2-py2.6-linux-x86_64.egg/mwlib/refine/uparser.py", line 63, in parseString a = compat.parse_txt(input, title=title, wikidb=wikidb, nshandler=nshandler, lang=lang, magicwords=magicwords, uniquifier=uniquifier, expander=te) File "/home/pp/local/lib/python2.6/site-packages/mwlib-0.13.2-py2.6-linux-x86_64.egg/mwlib/refine/compat.py", line 193, in parse_txt sub = core.parse_txt(raw, **kwargs) File "/home/pp/local/lib/python2.6/site-packages/mwlib-0.13.2-py2.6-linux-x86_64.egg/mwlib/refine/core.py", line 1037, in parse_txt combined_parser(parsers)(tokens, xopts) File "/home/pp/local/lib/python2.6/site-packages/mwlib-0.13.2-py2.6-linux-x86_64.egg/mwlib/refine/core.py", line 646, in call p(x, xopts) File "/home/pp/local/lib/python2.6/site-packages/mwlib-0.13.2-py2.6-linux-x86_64.egg/mwlib/refine/core.py", line 762, in init tokens[i] = m(name, vlist, inner or u"", xopts) File "/home/pp/local/lib/python2.6/site-packages/mwlib-0.13.2-py2.6-linux-x86_64.egg/mwlib/refine/core.py", line 887, in create_pages children = parse_txt(te.expandTemplates(True), File "evaluate.py", line 295, in mwlib.templ.evaluate.Expander.expandTemplates (mwlib/templ/evaluate.c:5932) File "evaluate.py", line 282, in mwlib.templ.evaluate.Expander._expand (mwlib/templ/evaluate.c:5569) File "evaluate.py", line 28, in mwlib.templ.evaluate.flatten (mwlib/templ/evaluate.c:1103) File "evaluate.py", line 30, in mwlib.templ.evaluate.flatten (mwlib/templ/evaluate.c:1134) File "nodes.py", line 210, in mwlib.templ.nodes.Template.flatten (mwlib/templ/nodes.c:4619) File "nodes.py", line 291, in mwlib.templ.nodes.Template._flatten (mwlib/templ/nodes.c:5976) File "evaluate.py", line 28, in mwlib.templ.evaluate.flatten (mwlib/templ/evaluate.c:1103) File "evaluate.py", line 30, in mwlib.templ.evaluate.flatten (mwlib/templ/evaluate.c:1134) File "nodes.py", line 210, in mwlib.templ.nodes.Template.flatten (mwlib/templ/nodes.c:4619) File "nodes.py", line 291, in mwlib.templ.nodes.Template._flatten (mwlib/templ/nodes.c:5976) File "evaluate.py", line 30, in mwlib.templ.evaluate.flatten (mwlib/templ/evaluate.c:1134) File "nodes.py", line 149, in mwlib.templ.nodes.SwitchNode.flatten (mwlib/templ/nodes.c:3416) File "evaluate.py", line 28, in mwlib.templ.evaluate.flatten (mwlib/templ/evaluate.c:1103) File "evaluate.py", line 30, in mwlib.templ.evaluate.flatten (mwlib/templ/evaluate.c:1134) File "nodes.py", line 210, in mwlib.templ.nodes.Template.flatten (mwlib/templ/nodes.c:4619) File "nodes.py", line 277, in mwlib.templ.nodes.Template._flatten (mwlib/templ/nodes.c:5777) File "/home/pp/local/lib/python2.6/site-packages/mwlib-0.13.2-py2.6-linux-x86_64.egg/mwlib/templ/magics.py", line 563, in call res = m(args) or '' # FIXME: catch TypeErros File "/home/pp/local/lib/python2.6/site-packages/mwlib-0.13.2-py2.6-linux-x86_64.egg/mwlib/templ/magics.py", line 209, in wrapper return f(self, pagename) File "/home/pp/local/lib/python2.6/site-packages/mwlib-0.13.2-py2.6-linux-x86_64.egg/mwlib/templ/magics.py", line 254, in NAMESPACE ns, partial, full = self.nshandler.splitname(pagename) File "/home/pp/local/lib/python2.6/site-packages/mwlib-0.13.2-py2.6-linux-x86_64.egg/mwlib/nshandling.py", line 128, in splitname title = unicode(title, 'utf-8') TypeError: coercing to Unicode: need string or buffer, NoneType found in function system, file /home/pp/local/lib/python2.6/site-packages/mwlib-0.13.1-py2.6-linux-x86_64.egg/EGG-INFO/scripts/nslave.py, line 63

interwiki links should not appear in the pdf

I tried with http://it.wikisource.org/wiki/Odissea/Libro_I, at the end of the text I see interwiki links, they shouldn't appear in the output. thanks

Inconsistent importation of json and simplejson

Some modules firstly try to import simplejson, while others json. This can result in importing both json and simplejson:

$ python2.7
Python 2.7.2+ (2.7:f1ee3bb6ba64+, Jan 29 2012, 20:33:23) 
[GCC 4.5.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> import mwlib.siteinfo.fetch_siteinfo
>>> sys.modules["json"]
<module 'json' from '/usr/lib64/python2.7/json/__init__.pyc'>
>>> sys.modules["simplejson"]
<module 'simplejson' from '/usr/lib64/python2.7/site-packages/simplejson/__init__.pyc'>
>>>

I suggest that all modules firstly try to import simplejson before json.

Undeterministic behavior when parsing styles

I have run into a problem with mwlib when parsing the Hungarian Wikipedia. There is a bug in the source of the page "Háromhúros brácsa": the number of opening and closing apostrophes around a text segment do not match: ''d'''. I understand how this can confuse a parser; however, I cannot understand why it has to confuse it in a different way each time!

Here are the logs of two runs:

handle_style Style"''"
handle_default
handle_text: d
handle_style Style"'''"
handle_default

handle_style Style"''"
handle_default
handle_text: helyett
handle_style Style"'''"
handle_default

vs.

handle_style Style"''"
handle_default
handle_text: d

handle_default
handle_text: ' helyett
handle_style Style"''"
handle_default

Also, the page handling remains inconsistent and undeterministic throughout the rest of the page. Our code relies on parsing the page two times, and with the same results, so this is a showstopper for us.

Please find the page source here: http://hu.wikipedia.org/w/index.php?title=H%C3%A1romh%C3%BAros_br%C3%A1csa&action=edit

Update installation doc

The installation doc mentions a very old ubuntu release, it should be updated so it mentions both debian and ubuntu.

PDF for complex scripts

The sample preview mode of Pediapress renders complex scripts for internal pages properly. The cover pages are garbled. Download of PDF also is garbled. As the problem is fixed in preview, it will help, if it can be fixed at other places.

Check the related bug in Wikimedia bugzilla https://bugzilla.wikimedia.org/show_bug.cgi?id=28206

File name too long

got the following traceback today:

  File "/home/pp/local/lib/python2.6/site-packages/gevent/greenlet.py", line 327, in run
    result = self._run(*self.args, **self.kwargs)
  File "/home/pp/local/lib/python2.6/site-packages/mwlib/net/fetch.py", line 204, in download_to_file
    out = open(temp_path, "wb")
IOError: [Errno 36] File name too long: '/home/pp/sandbox/cache/fc/fc05cf45aeb81b18/tmpV2C4gb/nuwiki/images/~1662~~1585~~1608~~1606~~1583~~1607~:~1585~~1575~~1740~~8204~~1607~~1575~~1740~ ~1605~~1608~~1587~~1608~~1740~ ~1585~~1740~~1575~~1587~~1578~~8204~~1580~~1605~~1607~~1608~~1585~~1740~ ~1583~~1607~~1605~ ~1575~~1740~~1585~~1575~~1606~ ~1576~~1575~ ~1581~~1583~~1608~~1583~ ~1583~~1585~~1589~~1583~ ~1583~~1585~ ~1575~~1587~~1578~~1575~~1606~~8204~~1607~~1575~.png\xc2\xb7'

TOC names with wiki section syntax not imported into odt format

For Telugu WIkisource, I am using {{C| section name}} with section heading wiki syntax like two or three is equal to signs. When importing to odt, these are appearing as {{{1}}}

Sample page: http://te.wikisource.org/wiki/%E0%B0%B5%E0%B0%BE%E0%B0%A1%E0%B1%81%E0%B0%95%E0%B0%B0%E0%B0%BF:Arjunaraoc/%E0%B0%AA%E0%B1%81%E0%B0%B8%E0%B1%8D%E0%B0%A4%E0%B0%95%E0%B0%BE%E0%B0%B2%E0%B1%81/%E0%B0%86%E0%B0%82%E0%B0%A7%E0%B1%8D%E0%B0%B0%E0%B1%81%E0%B0%B2%E0%B0%9A%E0%B0%B0%E0%B0%BF%E0%B0%A4%E0%B1%8D%E0%B0%B0%E0%B0%AE%E0%B1%81-%E0%B0%AA%E0%B1%8D%E0%B0%B0%E0%B0%A5%E0%B0%AE%E0%B0%AD%E0%B0%BE%E0%B0%97%E0%B0%AE%E0%B1%81

{{REVISIONID}} = 0 ?

when I have the magicword {{REVISIONID}} within a wikipage, the generated document has got 0 instead of the actual revId of the page.
I took a look at the code (magics.py and evaluate.py) and revisionid is never set, therefore the default is used which is 0. I don't know how to get the revision Id of a page, to set it.

Is that the wanted behaviour ?

Error with fonts on pl.wikisource

The output of some books fails on pl.wikisource with an error.

See https://pl.wikisource.org/w/index.php?title=Specjalna:Ksi%C4%85%C5%BCka&bookcmd=rendering&return_to=Wiki%C5%BAr%C3%B3d%C5%82a%3AKsi%C4%85%C5%BCki%2FKraszewski+J%C3%B3zef+Ignacy+-+Dwie+kr%C3%B3lowe&collection_id=3ed0a7b2dde692d4&writer=rl

Some books works fine like: https://pl.wikisource.org/wiki/Wiki%C5%BAr%C3%B3d%C5%82a:Ksi%C4%85%C5%BCki/Konopnicka_Maria_-_Pan_Balcer_w_Brazylii

New releases still need to be tagged

When installing "mwlib" one still gets 0.15.15 as opposed to 0.15.16 or 0.15.17 which is the latest version. Thanks and cheers.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.