textile / python-textile Goto Github PK

View Code? Open in Web Editor NEW

This project forked from sebix/python-textile

67.0 67.0 21.0 753 KB

A Python port of Textile, A humane web text generator

License: Other

Python 100.00%

markup-converter markup-language textile

python-textile's Introduction

PHP-Textile

Textile reference | Live editor

PHP-Textile is a modern Textile markup language parser for PHP. Textile is a humane web text generator that takes lightweight, readable, plaintext-like markup language and converts it into well formed HTML.

Install

Using Composer:

$ composer require netcarver/textile

Usage

The Textile parser can be accessed through the Netcarver\Textile\Parser class. The class is highly configurable, and actual parsing happens with the parse method:

require './vendor/autoload.php';
$parser = new \Netcarver\Textile\Parser();
echo $parser->parse('h1. Hello World!');

Parsing untrusted input

If you are using PHP-Textile to format user-supplied input, blog comments for instance, remember to enable restricted parser mode:

$parser = new \Netcarver\Textile\Parser();
echo $parser
    ->setRestricted(true)
    ->parse('!bad/image/not/allowed.svg!');

In restricted mode PHP-Textile doesn’t allow more powerful formatting options such as inline styles, and removes any raw HTML.

Parsing single-line fields

If you are using PHP-Textile as a field-level formatter to parse just inline spans and glyphs, use the setBlockTags method to disable block tags:

$parser = new \Netcarver\Textile\Parser();
echo $parser
    ->setBlockTags(false)
    ->parse('Hello *strong* world!');

The above outputs:

Hello <strong>strong</strong> world!

Doctypes

Currently, PHP-Textile can target either XHTML or HTML5 output with XHTML being the default for backward compatibility. The targeted doctype can be changed via the setDocumentType method:

$parser = new \Netcarver\Textile\Parser();
echo $parser
    ->setDocumentType('html5')
    ->parse('HTML(HyperText Markup Language)');

Setting alternate glyphs

Textile’s typographic substitutions can be overridden with the setSymbol method. If you need to setup Textile to do non-standard substitutions, call setSymbol before you parse the input with parse.

$parser = new \Netcarver\Textile\Parser();
$parser
    ->setSymbol('half', '1&#8260;2')
    ->parse('Hello [1/2] World!');

The symbol names you can pass to setSymbol can be found here.

Prefixing relative image and link paths

Setting prefix might be useful if you want to point relative paths to certain consistent location:

$parser = new \Netcarver\Textile\Parser();
$parser
    ->setImagePrefix('/user/uploads')
    ->setLinkPrefix('/')
    ->parse('!image.jpg! "link":page');

Getting in contact

The PHP-Textile project welcomes constructive input and bug reports from users. Please open an issue on the repository for a comment, feature request or bug.

Development

See CONTRIBUTING.textile.

python-textile's People

Contributors

Stargazers

Watchers

Forkers

mblayman murdej rczajka pws21 mhendricks philippbosch ox-it crw mst-textile orcasgit adam-iris boguscurry di2mot brondsem appli-intramuros mitya57 lyzo rayleyva alts xakon kpot

python-textile's Issues

Incorrect handling of quote entities in extended pre block

Hello, I try to wrap this text:

pre.. import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import ru.onyma.job.Context;
import ru.onyma.job.RescheduleTask;

import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;

/**
 * @author ustits
 */
public abstract class MainService<T> extends RescheduleTask implements Context<T> {

  private static final Logger log = LoggerFactory.getLogger(MainService.class);
  private final ScheduledExecutorService scheduler;

  private boolean isFirstRun = true;
  private T configs;

  public MainService(final ScheduledExecutorService scheduler) {
    super(scheduler);
    this.scheduler = scheduler;
  }

  @Override
  public void setConfig(final T configs) {
    this.configs = configs;
    if (isFirstRun) {
      scheduler.schedule(this, 0, TimeUnit.SECONDS);
      isFirstRun = false;
    }
  }

  @Override
  public void stop() {
    super.stop();
    scheduler.shutdown();
    try {
      scheduler.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS);
    } catch (InterruptedException ie) {
      log.warn("Unable to wait for syncs termination", ie);
      Thread.currentThread().interrupt();
    }
  }

  protected final T getConfigs() {
    return configs;
  }
}

and I getting this result:

<pre>

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import ru.onyma.job.Context;
import ru.onyma.job.RescheduleTask;

import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;

/**
 * @author ustits
 */
public abstract class MainService<T> extends RescheduleTask implements Context<T> {

  private static final Logger log = LoggerFactory.getLogger(MainService.class);
  private final ScheduledExecutorService scheduler;

  private boolean isFirstRun = true;
  private T configs;

  public MainService(final ScheduledExecutorService scheduler) {
    super(scheduler);
    this.scheduler = scheduler;
  }

  @Override
  public void setConfig(final T configs) {
    this.configs = configs;
    if (isFirstRun) {
      scheduler.schedule(this, 0, TimeUnit.SECONDS);
      isFirstRun = false;
    }
  }

  @Override
  public void stop() {
    super.stop();
    scheduler.shutdown();
    try {
      scheduler.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS);
    } catch (InterruptedException ie) {
      log.warn(&#8220;Unable to wait for syncs termination&#8221;, ie);
      Thread.currentThread().interrupt();
    }
  }

  protected final T getConfigs() {
    return configs;
  }
}</pre>

but on txstyle.org I getting this result:

<pre>import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import ru.onyma.job.Context;
import ru.onyma.job.RescheduleTask;

import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;

/**
 * @author ustits
 */
public abstract class MainService&lt;T&gt; extends RescheduleTask implements Context&lt;T&gt; {

  private static final Logger log = LoggerFactory.getLogger(MainService.class);
  private final ScheduledExecutorService scheduler;

  private boolean isFirstRun = true;
  private T configs;

  public MainService(final ScheduledExecutorService scheduler) {
    super(scheduler);
    this.scheduler = scheduler;
  }

  @Override
  public void setConfig(final T configs) {
    this.configs = configs;
    if (isFirstRun) {
      scheduler.schedule(this, 0, TimeUnit.SECONDS);
      isFirstRun = false;
    }
  }

  @Override
  public void stop() {
    super.stop();
    scheduler.shutdown();
    try {
      scheduler.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS);
    } catch (InterruptedException ie) {
      log.warn(&quot;Unable to wait for syncs termination&quot;, ie);
      Thread.currentThread().interrupt();
    }
  }

  protected final T getConfigs() {
    return configs;
  }
}</pre>

Difference in parse ". Python parser return ”, php parser return ". &#8221 in not transform in > within pre tag.

Missing files in the source tarball.

The 3.0.0 tar file misses a bunch of files, some of which I'd consider important.

Comparing the 2.3.16 and the 3.0.0 tar files, these files are missing:

.coveragerc
CHANGELOG.textile
CONTRIBUTORS.txt
LICENSE.txt
README.textile
pytest.ini
requirements.txt
tests/test_attributes.py
tests/test_block.py
tests/test_cli.py
tests/test_footnoteRef.py
tests/test_getRefs.py
tests/test_getimagesize.py
tests/test_github_issues.py
tests/test_glyphs.py
tests/test_image.py
tests/test_imagesize.py
tests/test_lists.py
tests/test_retrieve.py
tests/test_span.py
tests/test_subclassing.py
tests/test_table.py
tests/test_textile.py
tests/test_textilefactory.py
tests/test_urls.py
tests/test_utils.py
tests/test_values.py

AttributeError: 'NoneType' object has no attribute 'groups'

>>> import textile
>>> textile.__version__
'2.3.6'

works as expected:

>>> textile.textile('# xxx\n# yyy\n *blah*')
u'\t<ol>\n\t\t<li>xxx</li>\n\t\t<li>yyy\n <strong>blah</strong></li>\n\t</ol>'

removing space between \n and *blah*, raises an error

>>> textile.textile('# xxx\n# yyy\n*blah*')
---------------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\Python\virtualenv\egu\django19\lib\site-packages\textile\core.py", line 1360, in textile
    return Textile(html_type=html_type).parse(text)
  File "D:\Python\virtualenv\egu\django19\lib\site-packages\textile\core.py", line 250, in parse
    text = self.block(text)
  File "D:\Python\virtualenv\egu\django19\lib\site-packages\textile\core.py", line 466, in block
    block = Block(self, tag, atts, ext, cite, line)
  File "D:\Python\virtualenv\egu\django19\lib\site-packages\textile\objects\block.py", line 29, in __init__
    self.process()
  File "D:\Python\virtualenv\egu\django19\lib\site-packages\textile\objects\block.py", line 123, in process
    self.content = self.textile.graf(self.content)
  File "D:\Python\virtualenv\egu\django19\lib\site-packages\textile\core.py", line 602, in graf
    text = self.textileLists(text)
  File "D:\Python\virtualenv\egu\django19\lib\site-packages\textile\core.py", line 297, in textileLists
    return pattern.sub(self.fTextileList, text)
  File "D:\Python\virtualenv\egu\django19\lib\site-packages\textile\core.py", line 312, in fTextileList
    tl, start, atts, content = m.groups()
AttributeError: 'NoneType' object has no attribute 'groups'

Missing test files on pypi

The file on pypi/pythonhosted does not include the tests, just the tests/fixtures/README.txt file. However when I locally execute python setup.py sdist all the tests are included, so the MANIFEST.in file should be correct. Do you know what's wrong here?

Textile raises IndexError on empty string

When textile is run with empty string as argument it raises unexpected IndexError

textile.textile('')

.../textile/core.pyc in textile(text, html_type, encoding, output)
   1336 
   1337     """
-> 1338     return Textile(html_type=html_type).parse(text)
   1339 
   1340 

.../textile/core.pyc in parse(self, text, rel, sanitize)
    247                         'pre', 'h[1-6]',
    248                         'fn{0}+'.format(regex_snippets['digit']), '###']
--> 249                 text = self.block(text)
    250                 text = self.placeNoteLists(text)
    251         else:

.../textile/core.pyc in block(self, text)
    451                     line = '{0}\n{1}'.format(out.pop(), line)
    452                 whitespace = ' \t\n\r\f\v'
--> 453                 if ext or not line[0] in whitespace:
    454                     block = Block(self, tag, atts, ext, cite, line)
    455                     if block.tag == 'p' and not has_raw_text(block.content):

IndexError: string index out of range

I worked in textile 2.2 and 2.1, where it returned empty string.

Image style attributes no longer work in 2.1.7 release

In version 2.1.6 (and all previous versions) you had the ability for an image to be resized by style attributes (see lines 1243 - 1300, and especially 1271 - 1272):

def image(self, text):
        """
>>> t = Textile()
>>> t.image('!/imgs/myphoto.jpg!:http://jsamsa.com')
'<a href="http://jsamsa.com" class="img"><img src="/imgs/myphoto.jpg" alt="" /></a>'
>>> t.image('!</imgs/myphoto.jpg!')
'<img src="/imgs/myphoto.jpg" style="float: left;" alt="" />'
"""
        pattern = re.compile(r"""
(?:[\[{])? # pre
\! # opening !
(\<|\=|\>)? # optional alignment atts
(%s) # optional style,class atts
(?:\. )? # optional dot-space
([^\s(!]+) # presume this is the src
\s? # optional space
(?:\(([^\)]+)\))? # optional title
\! # closing
(?::(\S+))? # optional href
(?:[\]}]|(?=\s|$)) # lookahead: space or end of string
""" % self.c, re.U | re.X)
        return pattern.sub(self.fImage, text)

    def fImage(self, match):
        # (None, '', '/imgs/myphoto.jpg', None, None)
        align, atts, url, title, href = match.groups()
        atts = self.pba(atts)

        if align:
            atts = atts + ' style="%s"' % self.iAlign[align]

        if title:
            atts = atts + ' title="%s" alt="%s"' % (title, title)
        else:
            atts = atts + ' alt=""'

        if not self.isRelURL(url) and self.get_sizes:
            size = imagesize.getimagesize(url)
            if size:
                atts += " %s" % size

        if href:
            href = self.checkRefs(href)

        url = self.checkRefs(url)
        url = self.relURL(url)

        out = []
        if href:
            out.append('<a href="%s" class="img">' % href)
        if self.html_type == 'html':
            out.append('<img src="%s"%s>' % (url, atts))
        else:
            out.append('<img src="%s"%s />' % (url, atts))
        if href:
            out.append('</a>')

        return ''.join(out)

In version 2.1.7 you dropped some of this (lines 1327 - 1391):

def image(self, text):
        """
>>> t = Textile()
>>> t.image('!/imgs/myphoto.jpg!:http://jsamsa.com')
'<a href="http://jsamsa.com" class="img"><img alt="" src="/imgs/myphoto.jpg" /></a>'
>>> t.image('!</imgs/myphoto.jpg!')
'<img align="left" alt="" src="/imgs/myphoto.jpg" />'
"""
        pattern = re.compile(r"""
(?:[\[{])? # pre
\! # opening !
(\<|\=|\>)? # optional alignment atts
(%s) # optional style,class atts
(?:\. )? # optional dot-space
([^\s(!]+) # presume this is the src
\s? # optional space
(?:\(([^\)]+)\))? # optional title
\! # closing
(?::(\S+))? # optional href
(?:[\]}]|(?=\s|$)) # lookahead: space or end of string
""" % self.c, re.U | re.X)
        return pattern.sub(self.fImage, text)

    def fImage(self, match):
        # (None, '', '/imgs/myphoto.jpg', None, None)
        align, atts, url, title, href = match.groups()
        atts = self.pba(atts)
        size = None

        alignments = {'<': 'left', '=': 'center', '>': 'right'}

        if not title:
            title = ''

        if not self.isRelURL(url) and self.get_sizes:
            size = imagesize.getimagesize(url)

        if href:
            href = self.checkRefs(href)

        url = self.checkRefs(url)
        url = self.relURL(url)

        out = []
        if href:
            out.append('<a href="%s" class="img">' % href)
        out.append('<img')
        if align:
            out.append(' align="%s"' % alignments[align])
        out.append(' alt="%s"' % title)
        if size:
            out.append(' height="%s"' % size[1])
        out.append(' src="%s"' % url)
        if title:
            out.append(' title="%s"' % title)
        if size:
            out.append(' width="%s"' % size[0])
        if self.html_type == 'html':
            out.append('>')
        else:
            out.append(' />')
        if href:
            out.append('</a>')

        return ''.join(out)

Any reason why you did this? I recently updated to 2.1.7 and all my textile-based image styles (custom width and heights) no longer work.

url_parse chokes on Unicode characters

Trying to parse "Chögyam Trungpa":https://www.google.com/search?q=Chögyam+Trungpa.

Here is the exception I am seeing:

Traceback (most recent call last):
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/flask/app.py", line 1988, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/flask/app.py", line 1641, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/newrelic-2.72.0.52/newrelic/hooks/framework_flask.py", line 98, in _nr_wrapper_Flask_handle_exception_
    return wrapped(*args, **kwargs)
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/flask/app.py", line 1544, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/flask/app.py", line 1639, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/flask/app.py", line 1625, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/newrelic-2.72.0.52/newrelic/hooks/framework_flask.py", line 40, in _nr_wrapper_handler_
    return wrapped(*args, **kwargs)
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/hyper/web/lib/auth.py", line 15, in decorated_function
    return f(*args, **kwargs)
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/hyper/web/blueprint/chat/__init__.py", line 116, in list_posts
    updated_at=int(time.time())
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/flask/templating.py", line 134, in render_template
    context, ctx.app)
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/flask/templating.py", line 116, in _render
    rv = template.render(context)
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/newrelic-2.72.0.52/newrelic/api/function_trace.py", line 98, in dynamic_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/jinja2/environment.py", line 989, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/jinja2/environment.py", line 754, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/hyper/web/blueprint/chat/templates/list_posts.html", line 1, in top-level template code
    {% extends "chat_page.html" %}
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/hyper/web/blueprint/chat/templates/chat_page.html", line 1, in top-level template code
    {% extends "layout.html" %}
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/hyper/web/templates/layout.html", line 27, in top-level template code
    {% block content %}
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/hyper/web/blueprint/chat/templates/chat_page.html", line 13, in block "content"
    {% block main_column %}{% endblock %}
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/hyper/web/blueprint/chat/templates/list_posts.html", line 11, in block "main_column"
    {% include('_post_list.html') %}
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/hyper/web/blueprint/chat/templates/_post_list.html", line 9, in top-level template code
    {% include('_post_item.html') %}
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/hyper/web/blueprint/chat/templates/_post_item.html", line 56, in top-level template code
    <span class="value">{{ post.get_display_message()|safe }}</span>
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/hyper/model/base.py", line 411, in get_display_message
    html_type='html5'
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/textile/core.py", line 1367, in textile_restricted
    text)
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/textile/core.py", line 251, in parse
    text = self.block(text)
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/textile/core.py", line 456, in block
    block = Block(self, tag, atts, ext, cite, line)
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/textile/objects/block.py", line 29, in __init__
    self.process()
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/textile/objects/block.py", line 121, in process
    self.content = self.textile.graf(self.content)
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/textile/core.py", line 582, in graf
    text = self.links(text)
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/textile/core.py", line 605, in links
    return self.replaceLinks(text)
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/textile/core.py", line 718, in replaceLinks
    text = re.compile(pattern, flags=re.X | re.U).sub(self.fLink, text)
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/textile/core.py", line 874, in fLink
    url = self.shelveURL(self.encode_url(urlunsplit(uri_parts)))
  File "/usr/local/www/hyper/env/local/lib/python2.7/site-packages/textile/core.py", line 929, in encode_url
    query = quote(unquote(parsed.query), b'=&?/')
  File "/usr/lib/python2.7/urllib.py", line 1299, in quote
    return ''.join(map(quoter, s))
KeyError: u'\xf6'

Based on information from https://stackoverflow.com/questions/15115588/urllib-quote-throws-keyerror, I believe the following line needs to include .encode('utf-8')

textile/core.py line 929

        query = quote(unquote(parsed.query), b'=&?/')

        query = quote(unquote(parsed.query.encode('utf-8')), b'=&?/')

That is a quick-fix (for python2) but I am not entirely sure what is going on in this bit of code, so there may be a better fix to be performed.

edit: that change makes many unittests fail in python3, so it is a no-go. Unittests pass for python2.

Import of pytest in setup.py results in build failure (ImportError: No module named pytest)

When installing from tarball there is an ImportError exception due to import of pytest in setup.py. Here is one way to reproduce:

~$virtualenv pve
New python executable in /Users/rjollos/pve/bin/python2.7
Also creating executable in /Users/rjollos/pve/bin/python
Installing setuptools, pip, wheel...done.
(pve) ~$. pve/bin/activate
(pve) ~$pip install --no-cache-dir --no-binary :all: textile
Collecting textile
  Downloading textile-2.3.6.tar.gz (45kB)
    100% |████████████████████████████████| 51kB 412kB/s 
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/qf/y5yt86vn54j_sy7dv1f8hb3r0000gn/T/pip-build-V8XzTX/textile/setup.py", line 4, in <module>
        import pytest
    ImportError: No module named pytest
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/qf/y5yt86vn54j_sy7dv1f8hb3r0000gn/T/pip-build-V8XzTX/textile/

With import pytest in setup.py, pytest would at least need to be included in install_requires.

testSanitize fails in suite in release -2.1.8

.............................................................................................................................................E.....
===============================================================
ERROR: textile.tests.Tests.testSanitize
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib64/pypy/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/mnt/gen2/TmpDir/portage/app-text/pytextile-2.1.8/work/textile-2.1.8/textile/tests/__init__.py", line 474, in testSanitize
    expect = textile.Textile().textile(test, sanitize=True)
  File "/mnt/gen2/TmpDir/portage/app-text/pytextile-2.1.8/work/textile-2.1.8/textile/functions.py", line 290, in textile
    text = sanitizer.sanitize(text)
  File "/mnt/gen2/TmpDir/portage/app-text/pytextile-2.1.8/work/textile-2.1.8/textile/tools/sanitizer.py", line 16, in sanitize
    stream = walker(tree)
TypeError: 'NoneType' object is not callable

Name                      Stmts   Miss  Cover   Missing
-------------------------------------------------------
textile                       3      0   100%   
textile.functions           840     19    98%   32-33, 39-42, 216, 416-419, 485-486, 623-624, 895, 1073, 1237, 1465-1466
textile.tools               158    156     1%   9-264
textile.tools.imagesize      27     10    63%   15-16, 29, 33-34, 38-42
textile.tools.sanitizer      12      4    67%   9-10, 18-20
-------------------------------------------------------
TOTAL                      1040    189    82%   
----------------------------------------------------------------------
Ran 147 tests in 186.359s

FAILED (errors=1

This was a run under pypy. Consistent single fail of this test under py2.7 3.3 3.4 and pypy.
Do you require anything else?

Should be an option not to remove scheme on autolinked/self-linked URLs

When auto_link=True, or when self-linking a URL like "$":http://www.example.com/ the scheme (http://) is removed from the displayed link.

I realize this follows the PHP version, but looking at the history behind that change (textile/php-textile#48 (comment)) it looks like something tossed in without much discussion. I (and several people I showed this to) feel like this was a bad decision, as it makes it less clear that the given text is actually a URL. It seems like there should at least be an option to turn this off.

Ironically, the one case where I think it should be removed is for mailto: links, but it's not. (This does deviate from the PHP version, which removes them.)

release notes?

Are there somewhere release notes?
Specally between v2.1.5 and v2.1.8

Missing files in the tarball

Please include

README.textile
LICENSE.txt

in the tarball. For building the Fedora package, we have to separately download them.

It might also be good to have the tests dir as part of the tarball, so we can run the testsuite while creating the RPM.

Table header formatting

Hello there,

I'm having a spot of bother in that column spanning headers (\3_. header for a header that would span three columns) don't render correctly.

This works on an ancient version I have, but not in 2.1.8.

"caps" are not replaced correctly

The last version I used was PyTextile 2.1.4, which did not try to replace caps. Now in version 2.2.2, unexpected HTML conversions occur for text that contains capital letters.

For example, the text "CFString" in a textile file becomes "CFString" in HTML, and "TCP/IP" becomes "TCP/IP".

My preference is to have a way to disable this substitution entirely because it is not necessary. Still, the regular expressions could be improved by scanning text beyond an all-caps range; for instance, require all-caps ranges to be immediately followed by spaces or punctuation, and certainly not lowercase letters.

Integration with tabulate

Dear team,

I am trying to run textile on top of tabulate (https://pypi.org/project/tabulate/)? In my case, I have weird behaviours as the tabulate tables are not shaped correctly, especially with the tabulate style "grid". Potentially something to look at!

Thank you, Nicolas

Link build with $ sign without "http" prefix broken.

textile('"$":www.google.com.br') raise a IndexError.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/vagrant/.virtualenvs/jbl-web/local/lib/python2.7/site-packages/textile/core.py", line 1412, in textile
    return Textile(html_type=html_type).parse(text)
  File "/home/vagrant/.virtualenvs/jbl-web/local/lib/python2.7/site-packages/textile/core.py", line 248, in parse
    text = self.block(text)
  File "/home/vagrant/.virtualenvs/jbl-web/local/lib/python2.7/site-packages/textile/core.py", line 499, in block
    block = Block(self, tag, atts, ext, cite, line)
  File "/home/vagrant/.virtualenvs/jbl-web/local/lib/python2.7/site-packages/textile/objects/block.py", line 32, in __init__
    self.process()
  File "/home/vagrant/.virtualenvs/jbl-web/local/lib/python2.7/site-packages/textile/objects/block.py", line 125, in process
    self.content = self.textile.graf(self.content)
  File "/home/vagrant/.virtualenvs/jbl-web/local/lib/python2.7/site-packages/textile/core.py", line 635, in graf
    text = self.links(text)
  File "/home/vagrant/.virtualenvs/jbl-web/local/lib/python2.7/site-packages/textile/core.py", line 658, in links
    return self.replaceLinks(text)
  File "/home/vagrant/.virtualenvs/jbl-web/local/lib/python2.7/site-packages/textile/core.py", line 774, in replaceLinks
    text = re.compile(pattern, flags=re.X | re.U).sub(self.fLink, text)
  File "/home/vagrant/.virtualenvs/jbl-web/local/lib/python2.7/site-packages/textile/core.py", line 921, in fLink
    text = text.split(":")[1]
IndexError: list index out of range

I believe it's a issue since it works fine on textile sandbox
https://txstyle.org/doc/12/links

Problems with inline HTML (<pre>)

Since upgrading to 2.3.1 from 2.2.x, all the inline code blocs in my site are mis-processed:

buffer = """
So here I am porting my ancient "newspipe":newspipe "front-end":blog/2006/09/30/0950 to "Snakelets":Snakelets and "Python":Python, and I've just trimmed down over 20 lines of "PHP":PHP down to essentially one line of "BeautifulSoup":BeautifulSoup retrieval:

<pre>
def parseWapProfile(self, url):
  result = fetch.fetchURL(url)
  soup = BeautifulStoneSoup(result['data'], convertEntities=BeautifulStoneSoup.HTML_ENTITIES)
  try:
    width, height = soup('prf:screensize')[0].contents[0].split('x')
  except:
    width = height = None
  return {"width": width, "height": height}
</pre>

Of course there's a lot more error handling to do (and useful data to glean off the "XML":XML), but being able to cut through all the usual parsing crap is immensely gratifying.
"""

... turns into:

<p>So here I am porting my ancient <a href="newspipe">newspipe</a> <a href="blog/2006/09/30/0950">front-end</a> to <a href="Snakelets">Snakelets</a> and <a href="Python">Python</a>, and I&#8217;ve just trimmed down over 20 lines of <a href="PHP"><span class="caps">PHP</span></a> down to essentially one line of <a href="BeautifulSoup">BeautifulSoup</a> retrieval:</p>\n\n<pre syntax="python">\ndef parseWapProfile(self, url):\n  result = fetch.fetchURL(url)\n  soup = BeautifulStoneSoup(result[&#8216;data&#8217;], convertEntities=BeautifulStoneSoup.HTML_ENTITIES)\n  try:\n    width, height = soup(&#8216;prf:screensize&#8217;)<sup class="footnote" id="fnrev363c2af623094ac3a94774d87ffe2583-1"><a href="#fn363c2af623094ac3a94774d87ffe2583-1">0</a></sup>.contents<sup class="footnote"><a href="#fn363c2af623094ac3a94774d87ffe2583-1">0</a></sup>.split(&#8216;x&#8217;)\n  except:\n    width = height = None\n  return {<a href="">width</a> width, <a href="">height</a> height}\n</pre>\n\n\t<p>Of course there&#8217;s a lot more error handling to do (and useful data to glean off the <a href="XML"><span class="caps">XML</span></a>), but being able to cut through all the usual parsing crap is immensely gratifying.</p>

...turning the code inside into a mess of footnote references, whereas the previous version left the code tags alone. Testing the PRE tag in isolation works, but not if it follows a text paragraph.

testsuite: internal error with coverage 5.0.X

Running the testsuite with coverage 5.0.3 yields

platform linux -- Python 3.8.1, pytest-4.6.9, py-1.8.0, pluggy-0.13.0
rootdir: .../textile-4.0.0, inifile: pytest.ini, testpaths: tests
plugins: cov-2.8.1
collected 206 items
[...]
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/usr/lib64/python3.8/site-packages/coverage/sqldata.py", line 1046, in execute
INTERNALERROR>     return self.con.execute(sql, parameters)
INTERNALERROR> sqlite3.IntegrityError: UNIQUE constraint failed: meta.key
[...]

According to nedbat/coveragepy#883 this can be fixed adding this to the .coveragerc file

[run]
parallel = True

In my tests worked.

Alignment in restricted mode works not correctly.

In tests for checking aligment you use symbols '<' and '>'

result = t.image('!</imgs/myphoto.jpg!')

But in fact, these symbols was parsed in html special chars. Their looks like '<' and '>' . And therefore working only

Simple way in pattern of reg exp function


\\!                  # opening !
(\<|\=|\>)?         # optional alignment atts
...

replace by

...
\\!                  # opening !
(\&lt;|\=|\&gt;)?         # optional alignment atts
...

and further
alignments = {'<': 'left', '=': 'center', '>': 'right'}
replace by
alignments = {'\<': 'left', '=': 'center', '\>': 'right'}

EDIT 2017-08-29 17:04 by @sebix: markup

Missing author/maintainer in package

When building the wheel and uploading, this warnings pops up:

warning: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) must be supplied

On pypi @ikirudennis (dburke) is the package owner, thus I propose to put you into the maintainer fields.

Incorrect wrap pre-froematted value

I try to format this value:

pre.. word

another

word

yet anothe word

but get the incorrect output with excess pre tags:

<pre><pre>&lt;pre&gt;&lt;pre&gt;word\n\nanother&lt;/pre&gt;\n\nword&lt;/pre&gt;\n\nyet anothe word</pre></pre>

screen

How to fix it?

using sys.exectuable instead of hardcoded 'python' in test command

in tests/test_cli.py line 5

'python' is used inside a command, but for python3 installation, e.g. on openSUSE, this doesn't work, since python would be the 2.7 version. Perhaps sys.executable should be used here?

I ran into this issue while trying to package textile for openSUSE as a python3 package, not sure, if this is also used in other locations.

Arun

Incredibly abysmal performance.

import textile

src = u'This @example@ is "neither":/defn/neither complete *nor* trite, *though _simple_*.'
bigsrc = src * 100  # Amplify any issues.

%timeit textile.textile(src)
#1000 loops, best of 3: 1.4 ms per loop

This doesn't seem bad. Until you compare it to my experimental parser, which is almost 3x faster.

from marrow.texting.engine import Parser

%timeit Parser(src).render()
#1000 loops, best of 3: 510 µs per loop

The output of the engines on this sample is nearly identical: (only difference is an inexplicable leading tab in the python-textile version)

textile.textile(src).strip() == Parser(src).render()  # True

It gets worse when using the 100x larger bigsrc value, but it gets worse in a very unusual way:

%timeit Parser(bigsrc).render()
#10 loops, best of 3: 39.3 ms per loop

%timeit textile.textile(bigsrc)
#1 loop, best of 3: 6.18 s per loop

That's six seconds, or 4285x slower to process 100x as much text. (Vs. 28x slower to process 100x as much text, which is actually an increase in efficiency and performance vs. the smaller source.) This is baffling.

Important Note: I do not mean this ticket to be a spammy advertisement for my own package. My package was an experiment in parser design five years ago and not suitable for any real use.

Compatibility with Google App Engine

Hi,

Can't seem to run it in GAE.

I used this guide to add textile to my Python 2.7 GAE project and found that textile 2.2.0 depends on regex 2014.10.09.

I tried to install regex 2014.10.09 using the same manual and using pip (and copy it to my project's Lib directory), and none helped. I see that the regex package has _regex.c and _regex.h files in it, so I guess I cannot use regex 2014.10.09 in GAE as it's not pure Python...

Is there any way to run it in Google App Engine?

Also submitted a StackOverflow question here.

Key error u'##'

I try to transform this text:

# test
### test
## test

and get exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/dmitry/.local/lib/python2.7/site-packages/textile/core.py", line 1368, in textile
    return Textile(html_type=html_type).parse(text)
  File "/home/dmitry/.local/lib/python2.7/site-packages/textile/core.py", line 250, in parse
    text = self.block(text)
  File "/home/dmitry/.local/lib/python2.7/site-packages/textile/core.py", line 471, in block
    block = Block(self, tag, atts, ext, cite, line)
  File "/home/dmitry/.local/lib/python2.7/site-packages/textile/objects/block.py", line 32, in __init__
    self.process()
  File "/home/dmitry/.local/lib/python2.7/site-packages/textile/objects/block.py", line 124, in process
    self.content = self.textile.graf(self.content)
  File "/home/dmitry/.local/lib/python2.7/site-packages/textile/core.py", line 607, in graf
    text = self.textileLists(text)
  File "/home/dmitry/.local/lib/python2.7/site-packages/textile/core.py", line 297, in textileLists
    return pattern.sub(self.fTextileList, text)
  File "/home/dmitry/.local/lib/python2.7/site-packages/textile/core.py", line 349, in fTextileList
    self.olstarts[tl] = self.olstarts[tl] + 1
KeyError: u'##'

Can you fix this?

Incompatible with Python 2.6 due to use of flags argument to re.sub

The commit cf7faf77 uses the flags argument to re.sub, which is only available since Python 2.7.

This result in a test failure with Python 2.6 for the Trac project:

Traceback (most recent call last):
  File "C:\projects\trac\trac\mimeview\tests\txtl.py", line 84, in test_html
    """)
  File "C:\projects\trac\trac\mimeview\tests\txtl.py", line 35, in _render
    result = self.renderer.render(self.context, 'textile', text)
  File "C:\projects\trac\trac\mimeview\txtl.py", line 73, in render
    output = render_textile(content)
  File "C:\projects\trac\trac\mimeview\txtl.py", line 46, in render_textile
    return textile.textile(text)
  File "C:\Python26\lib\site-packages\textile\core.py", line 1417, in textile
    return Textile(html_type=html_type).parse(text)
  File "C:\Python26\lib\site-packages\textile\core.py", line 237, in parse
    text = normalize_newlines(text)
  File "C:\Python26\lib\site-packages\textile\utils.py", line 114, in normalize_newlines
    out = re.sub(r'^[ \t]*\n', '\n', out, flags=re.M)
TypeError: sub() got an unexpected keyword argument 'flags'

textile to links containing parentheses fails

See spec(20 May 2009).

>>> import textile
>>> textile.__version__
'2.2.2'
>>> from textile import textile
>>> r = textile('''\
... This is a link to a \
... ["Wikipedia article about Textile":http://en.wikipedia.org/wiki/Textile_(markup_language)].
... ''')
Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
  File "c:\Python27\lib\site-packages\textile\core.py", line 1661, in textile
    head_offset=head_offset)
  File "c:\Python27\lib\site-packages\textile\core.py", line 267, in parse
    text = self.block(text, int(head_offset))
  File "c:\Python27\lib\site-packages\textile\core.py", line 780, in block
    cite, line)
  File "c:\Python27\lib\site-packages\textile\core.py", line 913, in fBlock
    content = self.graf(content)
  File "c:\Python27\lib\site-packages\textile\core.py", line 1105, in graf
    text = self.links(text)
  File "c:\Python27\lib\site-packages\textile\core.py", line 1160, in links
    text = re.compile(pattern, re.X | re.U).sub(self.fLink, text)
  File "c:\Python27\lib\site-packages\textile\core.py", line 1203, in fLink
    out = ''.join([pre, out, post, tail])
TypeError: sequence item 3: expected string or Unicode, NoneType found
>>>

I don't know what is Textile 2.5 which is mentioned at TODO.textile, but this spec is at least 8 years old, so python-textile may have to render this correctly, or at least it should not stop render by Exception but should ignore this markup.

Table build without space after aligment raise a AttributeError.

textile('|=.First Header |=. Second Header |') raise a AttributeError.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/dmitry/.local/lib/python2.7/site-packages/textile/core.py", line 1412, in textile
    return Textile(html_type=html_type).parse(text)
  File "/home/dmitry/.local/lib/python2.7/site-packages/textile/core.py", line 248, in parse
    text = self.block(text)
  File "/home/dmitry/.local/lib/python2.7/site-packages/textile/core.py", line 499, in block
    block = Block(self, tag, atts, ext, cite, line)
  File "/home/dmitry/.local/lib/python2.7/site-packages/textile/objects/block.py", line 32, in __init__
    self.process()
  File "/home/dmitry/.local/lib/python2.7/site-packages/textile/objects/block.py", line 125, in process
    self.content = self.textile.graf(self.content)
  File "/home/dmitry/.local/lib/python2.7/site-packages/textile/core.py", line 641, in graf
    text = self.table(text)
  File "/home/dmitry/.local/lib/python2.7/site-packages/textile/core.py", line 291, in table
    return table.process()
  File "/home/dmitry/.local/lib/python2.7/site-packages/textile/objects/table.py", line 45, in process
    caption = Caption(**cmtch.groupdict())
AttributeError: 'NoneType' object has no attribute 'groupdict'

Expected output (from textile sandbox):

<table>
		<tr>
			<td>=.<strong>First Header</strong> </td>
			<td style="text-align:center;">Second Header </td>
		</tr>
</table>

Support for thead/tbody

Textile 2.2 added support for thead and tbody: http://textile.sitemonks.com/?eg=tables

|^.
|_. header |_. header |
|-.
| body | body |

Breaks on Python 3.2

Hello,

I suppose that is not big priority to support Python 3.2, but since this commit on this line: 1e5d98e#diff-bcabf8c90bd02845f81578f71c50c091R63 I could not install the library.

So I'm stuck with the version 2.1.5 of the library.

This issue only happen on Python 3.2, the version 3.1, 3.3, 3.4 are not affected.

Traceback on a Travis build:
https://travis-ci.org/Fantomas42/django-blog-zinnia/jobs/33796226#L249

Regards

Conditional dependencies are lost in wheel package.

As wheel docs specify,

In wheel, the only way to have conditional dependencies (that might only be needed on certain platforms) is to use environment markers as defined by PEP 426.

So, for example, dependency on ordereddict for Python 2.6 is lost and therefore Textile doesn't install properly from wheel on Python 2.6.

Problems with ":" inside quotes

I've recently upgraded from 2.2.2 to 2.3.1, and out of the 7000-odd documents on my site, a few hundred of them are breaking the new parser.

After fiddling about with it on Jupyter notebook, I isolated the following test case:

from textile import Textile
t = Textile(html_type="html5")
buffer="""
* Folders with ":" in their names are displayed with a forward slash "/" instead. (Filed as "#4581709":Radar:4581709, which was considered "normal behaviour" - quote: "Please note that Finder presents the 'Carbon filesystem' view, regardless of the underlying filesystem.")

"""
t.parse(buffer)

This blows up with the following traceback:

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-9-27ca3ebfabb8> in <module>()
----> 1 t.parse(buffer)

/Users/rcarmo/.pyenv/versions/2.7.10/lib/python2.7/site-packages/textile/core.pyc in parse(self, text, rel, sanitize)
    249                         'pre', 'h[1-6]',
    250                         'fn{0}+'.format(regex_snippets['digit']), '###']
--> 251                 text = self.block(text)
    252                 text = self.placeNoteLists(text)
    253         else:

/Users/rcarmo/.pyenv/versions/2.7.10/lib/python2.7/site-packages/textile/core.pyc in block(self, text)
    454                 whitespace = ' \t\n\r\f\v'
    455                 if ext or not line[0] in whitespace:
--> 456                     block = Block(self, tag, atts, ext, cite, line)
    457                     if block.tag == 'p' and not has_raw_text(block.content):
    458                         line = block.content

/Users/rcarmo/.pyenv/versions/2.7.10/lib/python2.7/site-packages/textile/objects/block.pyc in __init__(self, textile, tag, atts, ext, cite, content)
     27         self.inner_atts = OrderedDict()
     28         self.eat = False
---> 29         self.process()
     30 
     31     def process(self):

/Users/rcarmo/.pyenv/versions/2.7.10/lib/python2.7/site-packages/textile/objects/block.pyc in process(self)
    119 
    120         if not self.eat:
--> 121             self.content = self.textile.graf(self.content)
    122         else:
    123             self.content = ''

/Users/rcarmo/.pyenv/versions/2.7.10/lib/python2.7/site-packages/textile/core.pyc in graf(self, text)
    580 
    581         text = self.getRefs(text)
--> 582         text = self.links(text)
    583 
    584         if not self.noimage:

/Users/rcarmo/.pyenv/versions/2.7.10/lib/python2.7/site-packages/textile/core.pyc in links(self, text)
    601         does not match a trailing parenthesis.  It gets caught by tail, and
    602         we check later to see if it should be included as part of the url."""
--> 603         text = self.markStartOfLinks(text)
    604 
    605         return self.replaceLinks(text)

/Users/rcarmo/.pyenv/versions/2.7.10/lib/python2.7/site-packages/textile/core.pyc in markStartOfLinks(self, text)
    649                         if re.search(r'\S$', possibility, flags=re.U): # pragma: no branch
    650                             balanced = balanced + 1
--> 651                         possibility = possible_start_quotes.pop()
    652                     else:
    653                         # If quotes occur next to each other, we get zero

IndexError: pop from empty list

This is fairly common markup in my docs, too. I often quote non-alphanumeric stuff for emphasis. Also, what happened to the head_offset parameter to parse()? I see no mention of it anywhere. Is there a changelog someplace?

Incorrect transform unicode url

Hello, I try to create a link with Unicode characters:
https://myabstractwiki.ru/index.php/Заглавная_страница.
After the copy, this link from navigation string and paste to textile link transform to
https://myabstractwiki.ru/index.php/%D0%97%D0%B0%D0%B3%D0%BB%D0%B0%D0%B2%D0%BD%D0%B0%D1%8F_%D1%81%D1%82%D1%80%D0%B0%D0%BD%D0%B8%D1%86%D0%B0.
But the link in rendered HTML looks incorrect:
https://myabstractwiki.ru/index.php/%C3%90%C2%97%C3%90%C2%B0%C3%90%C2%B3%C3%90%C2%BB%C3%90%C2%B0%C3%90%C2%B2%C3%90%C2%BD%C3%90%C2%B0%C3%91%C2%8F_%C3%91%C2%81%C3%91%C2%82%C3%91%C2%80%C3%90%C2%B0%C3%90%C2%BD%C3%90%C2%B8%C3%91%C2%86%C3%90%C2%B0.
I suspect that this is due to double encoding in Unicode (see line 945 - 948):

path = '/'.join(  # could be encoded slashes!
            quote(unquote(pce).encode('utf8'), b'')
            for pce in parsed.path.split('/')
        )

How to fix this problem?
Thanks.

Encoding issue with img alt/title

I have a similar issue to #30, still on python 2.7: With an image tag, the tostringlist still fails.

Not sure how or why the A tag works for me, but the image doesn't.

Added test and a fix in https://github.com/jeroenp/python-textile/commit/a6a2b9a520db14c9b7bf73f219967226078d9d02

Empty description lists throw error

I played a bit with the american fuzzy lop (afl) :)

The examples used for afl are all copied from the examples on the website. It found this issue:

- :=
-

it throws:

Traceback (most recent call last):
  File "/home/ele/tmp/pyfuzz/code3.py", line 8, in <module>
    textile.textile(sys.stdin.read())
  File "/usr/lib/python3.6/site-packages/textile/core.py", line 1415, in textile
    return Textile(html_type=html_type).parse(text)
  File "/usr/lib/python3.6/site-packages/textile/core.py", line 248, in parse
    text = self.block(text)
  File "/usr/lib/python3.6/site-packages/textile/core.py", line 499, in block
    block = Block(self, tag, atts, ext, cite, line)
  File "/usr/lib/python3.6/site-packages/textile/objects/block.py", line 32, in __init__
    self.process()
  File "/usr/lib/python3.6/site-packages/textile/objects/block.py", line 125, in process
    self.content = self.textile.graf(self.content)
  File "/usr/lib/python3.6/site-packages/textile/core.py", line 645, in graf
    text = self.redcloth_list(text)
  File "/usr/lib/python3.6/site-packages/textile/core.py", line 1177, in redcloth_list
    return pattern.sub(self.fRCList, text)
  File "/usr/lib/python3.6/site-packages/textile/core.py", line 1188, in fRCList
    atts, content = m.groups()
AttributeError: 'NoneType' object has no attribute 'groups'

SyntaxWarnings with Python 3.8

When byte-compiling with Python 3.8 (on Fedora Rawhide) the following warnings are printed:

/usr/lib/python3.8/site-packages/textile/core.py:728: SyntaxWarning: "is not" with a literal. Did you mean "!="?
/usr/lib/python3.8/site-packages/textile/core.py:728: SyntaxWarning: "is" with a literal. Did you mean "=="?
/usr/lib/python3.8/site-packages/textile/core.py:751: SyntaxWarning: "is" with a literal. Did you mean "=="?
/usr/lib/python3.8/site-packages/textile/core.py:766: SyntaxWarning: "is" with a literal. Did you mean "=="?
/usr/lib/python3.8/site-packages/textile/core.py:1413: SyntaxWarning: "is" with a literal. Did you mean "=="?

Autolink should ignore urls within image references

e.g.
!http://www.ox.ac.uk/favicon.ico!
becomes the non-functioning
<img src="<a href="http: www.ox.ac.uk favicon.ico" rel="nofollow"">"www.ox.ac.uk/favicon.ico" />

Incorrect wrap code with Java generics in pre

Hello, I tried to wrap next code in pre..:

public class Tynopet<T extends Framework> {}

final List<List<String>> multipleList = new ArrayList<>();

and get the output:

public class Tynopet&lt;T extends Framework&gt; {}

final List<list<string>&gt; multipleList = new ArrayList&lt;&gt;();</list<string>

How to fix this? Thank you

":" in URLs should not be encoded

Following up from #27, custom URL schema processing requires the colon to not be URL-encoded.

IndexError string index out of range on whitespace only string

When I try to process a string that is whitespace only, I get an IndexError:

In [1]: from textile import textile

In [2]: textile(' ')
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)

Ref: #26

Unicode errors in a "pre." block

Textile throws an error with:

textile.textile(u'pre. smart ‘quotes’ are not smart!')

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/workspace/anaconda/envs/django/lib/python2.7/site-packages/textile/core.py", line 1366, in textile
    return Textile(html_type=html_type).parse(text)
  File "/workspace/anaconda/envs/django/lib/python2.7/site-packages/textile/core.py", line 250, in parse
    text = self.block(text)
  File "/workspace/anaconda/envs/django/lib/python2.7/site-packages/textile/core.py", line 450, in block
    block = Block(self, **match.groupdict())
  File "/workspace/anaconda/envs/django/lib/python2.7/site-packages/textile/objects/block.py", line 29, in __init__
    self.process()
  File "/workspace/anaconda/envs/django/lib/python2.7/site-packages/textile/objects/block.py", line 104, in process
    content = '{0}\n'.format(content)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2018' in position 6: ordinal not in range(128)

I found that adding

from __future__ import unicode_literals

to the top of block.py fixed this issue.

Restricted mode and CSS attribute parsing

Since version 2.3.0, style attributes like h2{position:absolute;top:200px}. This is a title started to come through into the output when using textile_restricted. In 2.2.2 and earlier, textile_restricted kept those out.

Is there a way in the latest versions to omit style attributes from being rendered?

Missing ordereddict dependency on Python 2.6

The packaging is missing ordereddict for Python 2.6. There is a Travis build for Python 2.6, but it hacks in the install of ordereddict in .travis.yml:

if [[ $TRAVIS_PYTHON_VERSION == 2.6 ]] ; then pip install ordereddict; fi

Here is an example of a tox run that doesn't include the addition above. https://travis-ci.org/handroll/handroll/jobs/37888634

I was able to fix this problem by adding ordereddict to my setup.py file, but I think python-textile should include it.

Encoding on title attribute for a link tag fails

The textile.utils generate_tags() function crashes with a UnicodeDecodeError when you use special characters on the title attribute, for example:

"Tëxtíle (Tëxtíle)":http://lala.com

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 9: ordinal not in range(128)

I tried solving the issue by encoding the inserted content, since that mixes the bytes with a unicode string that causes the decode error, but that makes textile crash somewhere in core.py:

element_tag.insert(len(element_tag) - 1, content.encode(enc))

What works for me is decoding the result from elementtree:

        element_tag = [v.decode(enc) for v in ElementTree.tostringlist(
                       element, encoding=enc, method='html')]

Platform: python 2.7.11

Key error on russian hash-route link

Hello I try to transform this link with hash-route and Russian literals, and get error KeyError: u'\u043b'

Example:

s = u'"link":https://ru.vuejs.org/v2/guide/components.html#Входные-параметры'
textile.textile(s)

>>> ...KeyError: u'\u043b'

How to fix this? Thank you.

Tables don't output thead

Table headers are not wrapped in a thead tag. Libraries such as DataTables.net require a thead section. Additionally, it is a more proper and semantic way of formatting a table.

>>> import textile
 >>> a = """
 ... |_. header|
 ... |row 1|
 ... """
 >>> print(textile.textile(a))
    <table>
        <tr>
            <th>header</th>
        </tr>
        <tr>
            <td>row 1</td>
        </tr>
    </table>

expected output would be:

    <table>
        <thead>
            <tr>
                <th>header</th>
            </tr>
        </thead>
        <tbody>
            <tr>
                <td>row 1</td>
            </tr>
        <tbody>
    </table>

version

 % pip3 show textile
 ---
 Metadata-Version: 2.0
 Name: textile
 Version: 2.3.3

Image markup incorrectly strips a dot character from URI

Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import textile
>>> textile.__version__
'2.3.10'
>>> textile.textile('!./path/to/image.png!')
u'\t<p><img alt="" src="/path/to/image.png" /></p>'
>>> textile.textile('!.image.png!')
u'\t<p><img alt="" src="image.png" /></p>'
>>> textile.textile('!./image.png!')
u'\t<p><img alt="" src="/image.png" /></p>'

Breaks with Python < 2.7

Pasted from the original repo merge pull request (sebix#9) as per @ikirudennis comment.

Your release breaks if you use any Python release < 2.7 w.r.t. table rendering. Here is a simple test case:

Python 2.6:

Python 2.6.6 (r266:84292, Aug 28 2012, 10:55:56) 
[GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import textile
>>> s="""
... _This_ is a *test*
... """
>>> html = textile.textile(s)
>>> print html
    <p><em>This</em> is a <strong>test</strong></p>
>>> t="""
... |^.
... |_. Foo|_. Bar|_. Baz|
... |-.
... |this|is|test|
... """
>>> html = textile.textile(t)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/path/to/python2.6/site-packages/textile/functions.py", line 1580, in textile
    html_type=html_type)
  File "/path/to/python2.6/site-packages/textile/functions.py", line 258, in textile
    text = self.block(text, int(head_offset))
  File "/path/to/python2.6/site-packages/textile/functions.py", line 754, in block
    cite, line)
  File "/path/to/python2.6/site-packages/textile/functions.py", line 872, in fBlock
    content = self.graf(content)
  File "/path/to/python2.6/site-packages/textile/functions.py", line 1058, in graf
    text = self.table(text)
  File "/path/to/python2.6/site-packages/textile/functions.py", line 447, in table
    return pattern.sub(self.fTable, text)
  File "/path/to/python2.6/site-packages/textile/functions.py", line 457, in fTable
    for row in [x for x in re.split(r'\|\s*?$', match.group(3), flags=re.M) if x]:
TypeError: split() got an unexpected keyword argument 'flags'

It works under 2.7:

Python 2.7.1 (r271:86832, Aug  5 2011, 03:30:24) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import textile
>>> s="""
... _This_ is a *test*
... """
>>> html = textile.textile(s)
>>> print html
    <p><em>This</em> is a <strong>test</strong></p>
>>> t="""
... |^.
... |_. Foo|_. Bar|_. Baz|
... |-.
... |this|is|test|
... """
>>> html = textile.textile(t)
>>> print html
    <table>
        <thead>
            <tr>
               <th>Foo</th>
               <th>Bar</th>
               <th>Baz</th>
            </tr>
        </thead>
        <tbody>
            <tr>
                <td>this</td>
                <td>is</td>
                <td>test</td>
            </tr>
        </tbody>
    </table>
>>>

I understand that this is due to how Python has implemented re.split() under 2.7 - "Changed in version 2.7: Added the optional flags argument."

You might like to update your fork to handle Python < 2.7. At the very least you should explicitly tell users in your forks README that your release breaks Textile table rendering (and maybe elsewhere where you use the re.split() method).

Double encoding of code blocks

When using the texttile_restricted function, data in code blocks is double encoded. With a document like the following:

h1. xml example

bc. 
<foo>
  bar
</foo>

I would expect it to output to look something like:

<h1>xml example</h1>

<pre><code>
&lt;foo&gt;
  bar
&lt;/foo&gt;
</pre></code>

Instead, it's generating something like:

<h1>xml example</h1>

<pre><code>
&amp;lt;foo&amp;gt;
  bar
&amp;lt;/foo&amp;gt;

Markup not parsed if followed by certain characters

I am using textile in a project using legacy data ported over form a Rails project that was using the Redcloth textile gem.

We have a number of text formatted like this that it seems textile is unable to parse:

_(artist-name)Ty Segall_’s

the expected output would be

Ty Segall’s

however textile.textile returns the raw string: