pyparsing / pyparsing Goto Github PK

Python library for creating PEG parsers

License: MIT License

Python 98.41% HTML 1.59%

python python2 python3 python-2 python-3 parser-combinators parsing-expression-grammar parsing parsing-library text-processing

pyparsing's Introduction

PyParsing -- A Python Parsing Module

Introduction

The pyparsing module is an alternative approach to creating and executing simple grammars, vs. the traditional lex/yacc approach, or the use of regular expressions. The pyparsing module provides a library of classes that client code uses to construct the grammar directly in Python code.

[Since first writing this description of pyparsing in late 2003, this technique for developing parsers has become more widespread, under the name Parsing Expression Grammars - PEGs. See more information on PEGs here .]

Here is a program to parse "Hello, World!" (or any greeting of the form "salutation, addressee!"):

from pyparsing import Word, alphas
greet = Word(alphas) + "," + Word(alphas) + "!"
hello = "Hello, World!"
print(hello, "->", greet.parseString(hello))

The program outputs the following:

Hello, World! -> ['Hello', ',', 'World', '!']

The Python representation of the grammar is quite readable, owing to the self-explanatory class names, and the use of '+', '|' and '^' operator definitions.

The parsed results returned from parseString() is a collection of type ParseResults, which can be accessed as a nested list, a dictionary, or an object with named attributes.

The pyparsing module handles some of the problems that are typically vexing when writing text parsers:

extra or missing whitespace (the above program will also handle "Hello,World!", "Hello , World !", etc.)
quoted strings
embedded comments

The examples directory includes a simple SQL parser, simple CORBA IDL parser, a config file parser, a chemical formula parser, and a four-function algebraic notation parser, among many others.

Documentation

There are many examples in the online docstrings of the classes and methods in pyparsing. You can find them compiled into online docs. Additional documentation resources and project info are listed in the online GitHub wiki. An entire directory of examples can be found here.

License

MIT License. See header of the pyparsing __init__.py file.

History

See CHANGES file.

pyparsing's People

Contributors

Stargazers

Watchers

Forkers

aleivag cngkaygusuz freakwill cgy1992 siweisun firefoxxy8 aldokkani kaymccormick hugovk xecgr klahnakoski brysj22952 s3rius chu604 james-emerton termim mitko86 akhileshbhople smallbaby honglinquan paulmcg-iio cclauss andrew-mikos matlab2000 tomasbubela-1 a402539 jk-livongo njarvis kkyon crowsonkb tinylambda hhy5277 silky-littleraisin likaiguo benbenbang rooierakkert ttreptow barbaragrape evhub sthagen ishank011 eswald m-agboola spectre1811 sonalisinghal multimeric rdavidson1994 aditya999123 anubhavshakya stdavids grahamrjuk nobodyinperson underscoreasterisk haoxuu bdragon28 nikiluc lallouslab sandlerben musabhusaini carelvd pcornier whisperingpixel xmonader sakarsalunke ansobolev toonarmycaptain wangjiabiao dominikpott seabreg epage appcoreopc zendesk hqcfly kms70847 lw0815 ian2009 extremeunix egberts liuxiaochen0625 akatuy bennylangston1 diegoguedes bjrnfrdnnd chrisgemignani rcoup yanchenwanggu ptmcg akainocat riyazy geektemo rfv wassekaran ishanbhatia98 tony-santos infothrill mattcarmody eric-seekas venkata16924b scottj97 lvhehehe

pyparsing's Issues

wikispaces is closing down

Wikispaces is closing down and PyParsing has a wealth of information there.

Use Markdown in readme

The current readme is not suitable for a GitHub repo, consider turning to markdown?

Is it a bug?

Following is the test code. Is it a bug? It confused me.

p = (Literal('=')| '\\' + Literal('eq'))('sign')
r = p.parseString('=')
print(r.sign)
# => ['=']

p = (Literal('=')| Literal('eq'))('sign')
r = p.parseString('=')
print(r.sign)
# => '='

scanString() is different with CaselessKeyword (x) and Keyword(x, caseless=True)

The CaselessKeyword(x) version does not seem to have the same preceding \b (or whatever) break as the Keyword(x, caseless=True).

I'm very new to pyparsing but I would expect the two versions to behave the same.

$ ipython 
Python 3.6.7 (default, Oct 22 2018, 11:32:17) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.2.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from pyparsing import Keyword, CaselessKeyword, __version__                                                                  

In [2]: frule = Keyword('t', caseless=True) + Keyword('yes', caseless=True)                                                          

In [3]: crule = CaselessKeyword('t') + CaselessKeyword('yes')                                                                        

In [4]: list(frule.scanString('not yes'))                                                                                            
Out[4]: []

In [5]: list(crule.scanString('not yes'))                                                                                            
Out[5]: [((['t', 'yes'], {}), 2, 7)]

In [6]: __version__                                                                                                                  
Out[6]: '2.3.0'

DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working

...Python37\lib\site-packages\pkg_resources\_vendor\pyparsing.py:943: DeprecationWarning: Using or
importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  collections.MutableMapping.register(ParseResults)
...Python37\lib\site-packages\pkg_resources\_vendor\pyparsing.py:3226: DeprecationWarning: Using or
 importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  elif isinstance( exprs, collections.Iterable ):

infixNotation() might be made better

I am going through the match attempts for my parser moz-sql-parser while parsing

select * from task where build.product is not null and build.product!='firefox'

The parser does a lot of trackback as it tries to match the WHERE clause; much more than it should. Please change infixNotation() to take advantage of the fact that clusters of infix binary operators always begin with an operand, alternate operator and operand, and end with an operand:

[build.product] is [not null] and [build.product] != ['firefox']

(you may notice my grammar did not include not as a unary operator to infixNotation())

By matching this alternating pattern the matching process can go much faster. Precedence is handled with some post processing to organize this into a tree. Of course, the difficultly will be splitting up a single infixNotation() call, infected with unary and trinary operators, into a hierarchy of infixNotation() calls containing only binary operators.

Thank you

How do you install pyparsing?

I am new to python. How do I install pyparsing?

Discussion: GSOC?

I am looking to mentor some work on pyparsing. I intend for the coding to proceed on a fork, but I am wondering how much (or how little) you (the project owner) would like to be involved:

You can be hands off (default) - we will work off a fork, and submit a series of PRs near the end of the summer.
You can provide reviews (1 or 2 hours per week) - we can work on a pyparsing branch, and you get to direct the work in progress.
You can help mentor (4hours per week) - and guide the student on a near-daily basis.

In all cases, I will give you updates on the project, I will be around to guide the student, and ensure the administrative stuff is done. IF this happens, then the student will be working, full time from about May to end of August.

The project has been added here: https://wiki.mozilla.org/Community:SummerOfCode19:Brainstorming#2019_Proposed_Project_List

Rename the Restructured files

Could you rename https://github.com/pyparsing/pyparsing/blob/master/HowToUsePyparsing.txt to .rst? That way it will render on github and can be read a bit easier.

dictOf erroneously matches empty string

Because dictOf uses ZeroOrMore for its internal repetition, it is possible to for it to match the empty string, which causes problems when it is also part of a larger repetitive expression (loops forever).

Needs to be changed to use OneOrMore.

I designed two new parser classes

I was excited for the new version. I am a fan of pyparsing.
I contribute a new parser class as following (would be renamed).

class Meanwhile(pp.ParseExpression):
    """Strings have to match all sub-expressions at the same time
        Grammar:
            Meanwhile([A, B, ...])
       
        Example:
            A = Meanwhile([IDEN, ~('_'+ DIGIT)]) # IDEN / ('_'+ DIGIT)
            to parse identifiers such as _a123 excluding _123
            Equivalently,
            A = Meanwhile()
            A.append(IDEN)
            A.exclude('_'+ DIGIT)  # <==> A / '_'+ DIGIT
    """

    def __init__(self, exprs=[]):
        super(Meanwhile, self).__init__(exprs)
        self.mayReturnEmpty = all(e.mayReturnEmpty for e in self.exprs)
        self.setWhitespaceChars(self.exprs[0].whiteChars)
        self.skipWhitespace = self.exprs[0].skipWhitespace
        self.callPreparse = True

    def parseImpl(self, instring, loc, doActions=True):
        postloc, result = self.exprs[0]._parse(instring, loc, doActions)
        for e in self.exprs[1:]:
            if not e.matches(instring, loc):
                raise _Exception(instring, len(instring), e.errmsg, self)
        return postloc, result

    def append(self, other):
        if isinstance(other, str):
            other = pp.ParserElement._literalStringClass(other)
        return self.append(other)

    def exclude(self, other):
        if isinstance(other, str):
            other = pp.ParserElement._literalStringClass(other)
        return self.append(~other)

    def __truediv__(self, other):
        return self.append(~other)

    def checkRecursion(self, parseElementList):
        subRecCheckList = parseElementList[:] + [self]
        for e in self.exprs:
            e.checkRecursion(subRecCheckList)
            if not e.mayReturnEmpty:
                break

    def __str__(self):
        if hasattr(self, "name"):
            return self.name

        if self.strRepr is None:
            self.strRepr = "%s{%s}"%(str(self.exprs[0]), " ".join(str(e) for e in self.exprs[1:]))

        return self.strRepr

In fact I define several parsers. LeadedBy works as FollowedBy, as its name.

class LeadedBy(_Enhance):
    """
    Works as FollowedBy

    Lookahead matching of the given parse expression.  C{LeadedBy}
    does not advance the parsing position within the input string, it only
    verifies that the specified parse expression matches at the current
    position.  C{LeadedBy} always returns a null token list."""
    def __init__(self, expr, start=0, retreat=None):
        '''
        Arguments:
            expr -- a parse expression
        
        Keyword Arguments:
            start {number} -- where it starts to search expr (default: {0})
            retreat {[type]} -- when it is given, start = loc - retreat (default: {None})
        '''
        super(LeadedBy, self).__init__(expr)
        self.mayReturnEmpty = True
        self.start = start
        self.retreat = retreat

    def parseImpl(self, instring, loc=0, doActions=True):
        if self.retreat is None:
            start = self.start
        else:
            start = loc - self.retreat
        if (self.expr + stringEnd).searchString(instring[start:loc], maxMatches=1):
            return loc, []
        else:
            raise _Exception(instring, loc, self.errmsg, self)

2.3.1 not backward compatible to 2.3.0

Please try out the attached test file. It runs in 2.3.0 but causes errors in 2.3.1.
pyparsing_error.py.gz

Use string boundary check instead of IndexError.

When IndexError has thrown in user's parse action function, some parse elements consume it and, raise ParseError. It makes user confused.

number = Word(nums)
def number_action():
    raise IndexError
number.setParseAction(number_action)
symbol = Word('abcd', max=1)
expr = number | symbol
expr.parseString('1 + 2')

Traceback (most recent call last):
  File "test.py", line 9, in <module>
    expr.parseString('1 + 2')
  File "/home/joon/.pyenv/versions/pypy2.7-5.8.0/site-packages/pyparsing.py", line 1632, in parseString
    raise exc
ParseException: Expected {W:(0123...) | W:(abcd)} (at char 5), (line:1, col:6)

This shows what I say. There's a problem in number_action function, but MatchFirst consumes IndexError and raise ParseError. It makes user to debug grammar rules. Because pyparsing throws ParseError, not IndexError at number_action.

PyPI information is outdated

For the latest version of PyParsing the PyPi entry:

The home page points to a wikispaces page but wikispaces is closing down
The download link points to sourceforge when a github link may be more appropriate.

Returning unchanged tokens from a parseAction generates doubly nested dictionaries

Attaching a parse action, which only returns the tokens, to a parser, which generates a single group in the ParseResults list, produces a doubly nested dictionary. If the ParseResults list contains two groups there is no double nesting. See code below, and dump() output further down (Rider is doubly nested).

Is this expected behaviour, and if so, why is it happening?

from __future__ import print_function
    
from pyparsing import *

name = Word(alphas)('name')
score = Word(nums + '.')('score')
nameScore = Group(name + score)
line1 = nameScore('Rider')
line2 = nameScore('Rider') + nameScore('Bull')

result1 = line1.parseString('Mauney 46.5')
result2 = line2.parseString('Mauney 46.5 Asteroid 46')

print("### before parse action is added ###")
print("result1.dump():\n" + result1.dump() + "\n")
print("result2.dump():\n" + result2.dump() + "\n")

line1.setParseAction(lambda t: t)
line2.setParseAction(lambda t: t)

result1 = line1.parseString('Mauney 46.5')
result2 = line2.parseString('Mauney 46.5 Asteroid 46')

print("### after parse action was added ###")
print("result1.dump():\n" + result1.dump() + "\n")
print("result2.dump():\n" + result2.dump() + "\n")

### before parse action is added ###
result1.dump():
[['Mauney', '46.5']]
- Rider: ['Mauney', '46.5']
  - name: 'Mauney'
  - score: '46.5'

result2.dump():
[['Mauney', '46.5'], ['Asteroid', '46']]
- Bull: ['Asteroid', '46']
  - name: 'Asteroid'
  - score: '46'
- Rider: ['Mauney', '46.5']
  - name: 'Mauney'
  - score: '46.5'

### after parse action was added ###
result1.dump():
[['Mauney', '46.5']]
- Rider: [['Mauney', '46.5']]
  - Rider: ['Mauney', '46.5']
    - name: 'Mauney'
    - score: '46.5'

result2.dump():
[['Mauney', '46.5'], ['Asteroid', '46']]
- Bull: ['Asteroid', '46']
  - name: 'Asteroid'
  - score: '46'
- Rider: ['Mauney', '46.5']
  - name: 'Mauney'
  - score: '46.5'

forgot pdb

pyparsing/pyparsing.py

Line 1241 in 2fe1bb0

pdb.set_trace()

Undefined name: 'err' in examples/SimpleCalc.py

Python 3 scoping rules are different than Python 2's. That means that 'err' is no longer defined in the three lines below...

flake8 testing of https://github.com/pyparsing/pyparsing on Python 3.7.1

$ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics

./examples/SimpleCalc.py:108:15: F821 undefined name 'err'
        print(err.line)
              ^
./examples/SimpleCalc.py:109:20: F821 undefined name 'err'
        print(" "*(err.column-1) + "^")
                   ^
./examples/SimpleCalc.py:110:15: F821 undefined name 'err'
        print(err)
              ^
3    F821 undefined name 'err'
3

To see the scoping rule differences, try the following in both Python 2 and Python 3.

#!/usr/bin/env python3

from __future__ import print_function

err = None

try:
    print("Before:", err)
    import a_module_that_does_not_exist
except ImportError as err:
    print("During:", err)

print("After:", err)

License file missing from the latest source tarball at PyPI

I guess that the directory move in 27e183a is the cause for that.

advantages of sexpdata vs pyparsing s-expressions

I just found that one can parse s-expressions using pyparsing using the example file here. But there is also sexpdata.

What are the advantages and disadvantages of using sexpdata vs pyparsing?

Export arbitrary ParseElement as EBNF

I'm not really sure what the tractability of this issue is, so first I would ask: are grammars represented with PyParsing within the class of context free grammars?

If that's the case, it would be incredibly useful to export a grammar defined through the PyParsing DSL as an EBNF document. This would provide a programmatic specification in a standard format that other tools could use as well.

lookahead with tokens returned

Hi,

I'm trying to move some code from using parsley (https://github.com/pyga/parsley) to pyparsing.

It was going so well but I think I've run into a wall.

The code I am moving uses this expression from parsley:

~~expr: Positive lookahead. Fails if the next item in the input does not match expr. Consumes no input.

Which I was hoping to replace with FollowedBy, however FollowedBy always returns a null token list, I need the results from the FollowedBy returned so I can utilise them further downstream.

Any ideas how I would go about this?

PyParsing parsing producing different results in 2.3.0

Hi,

Since PyParsing 2.3.0 I'm having issue with the parsing of my grammar. 2.2.2 (the previous release) doesn't have this behavior.

My grammar is as follows:

def build_parser():
    """
    Build a pyparsing parser for our custom topology description language.

    :return: A pyparsing parser.
    :rtype: pyparsing.MatchFirst
    """
    from pyparsing import (
        Word, Literal, QuotedString,
        StringStart, StringEnd,
        alphas, nums, alphanums,
        Group, OneOrMore, Optional
    )

    number = Word(nums)
    text = QuotedString('"')
    identifier = Word(alphas, alphanums + '_')

    attribute = (
        identifier('key') + Literal('=') +
        (text | number | identifier)('value')
    )
    attributes = (
        Literal('[') +
        OneOrMore(Group(attribute))('attributes') +
        Literal(']')
    )

    node = identifier('node')
    port = node + Literal(':') + (identifier | number)('port')
    link = port('endpoint_a') + Literal('--') + port('endpoint_b')

    environment_spec = (
        StringStart() + attributes('environment') + StringEnd()
    )
    nodes_spec = (
        StringStart() + Optional(attributes) +
        OneOrMore(Group(node))('nodes') + StringEnd()
    )
    ports_spec = (
        StringStart() + Optional(attributes) +
        OneOrMore(Group(port))('ports') + StringEnd()
    )
    link_spec = (
        StringStart() + Optional(attributes) +
        link('link') + StringEnd()
    )

    statement = environment_spec | link_spec | ports_spec | nodes_spec

    return statement

And when I try to parse the following string:

'hs1:1 -- sw1:1'

In PyParsing 2.3.0 I get :

(
    ['hs1', ':', '1', '--', 'sw1', ':', '1'],
    {
        'endpoint_a': [(['hs1', ':', '1'], {})],
        'endpoint_b': [(['sw1', ':', '1'], {})],
        'link': [(['hs1', ':', '1', '--', 'sw1', ':', '1'], {})],
        'node': ['hs1', 'sw1'],
        'port': ['1', '1'],
    },
)

Which is very different from what I was getting in version 2.2.2:

(
    ['hs1', ':', '1', '--', 'sw1', ':', '1'],
    {
        'endpoint_a': [
            ((['hs1', ':', '1'], {'node': [('hs1', 0)], 'port': [('1', 2)]}), 0),
        ],
        'endpoint_b': [
            ((['sw1', ':', '1'], {'node': [('sw1', 0)], 'port': [('1', 2)]}), 4),
        ],
        'link': [
            (
                (
                    ['hs1', ':', '1', '--', 'sw1', ':', '1'],
                    {
                        'endpoint_a': [
                            (
                                (
                                    ['hs1', ':', '1'],
                                    {'node': [('hs1', 0)], 'port': [('1', 2)]},
                                ),
                                0,
                            ),
                        ],
                        'endpoint_b': [
                            (
                                (
                                    ['sw1', ':', '1'],
                                    {'node': [('sw1', 0)], 'port': [('1', 2)]},
                                ),
                                4,
                            ),
                        ],
                        'node': [('hs1', 0), ('sw1', 4)],
                        'port': [('1', 2), ('1', 6)],
                    },
                ),
                0,
            ),
        ],
        'node': [('hs1', 0), ('sw1', 4)],
        'port': [('1', 2), ('1', 6)],
    },
)

I'm not sure what changed, since the release notes doesn't address change in behavior, or my grammar was badly described since the beginning, since this is code from 2015.

Any help on the matter or enlightenment on the direction to take is very welcome.

Thanks

indentedBlock + scanString allows indentation that doesn't use whitespace

I've written a test for pyparsing that demonstrates this issue:

    def runTest(self):
        stack = [1]
        parser = pp.indentedBlock(pp.Literal('A'), indentStack=stack, indent=True)

        # Passes
        a = parser.searchString("""
            A
            A
            A
        """)
        self.assertEqual(len(a[0][0]), 3)

        # Also works, even though it shouldn't
        b = parser.searchString("""
        ____A
            A
            A
        """)
        self.assertEqual(len(b[0][0]), 3)

Basically, because checkSubIndent doesn't actually care about the characters used in indentation, it will allow this ____A line, if it's the first line of a scanString, and it actually starts parsing at the A.

I'm thinking I should add some sort of check that the text used to indent each line is only whitespace?

Bug in OneOrMore/Or

I am seeing some potentially odd behaviour, or it's just me misunderstanding something.

I've narrowed it down to this trivial example (that on it's own doesn't make a lot of sense but it's part of something bigger)

import pyparsing as pp
a = pp.OneOrMore(
    pp.Or([
        pp.White(exact=1),
    ]))

expr1 = pp.White(exact=1) + pp.ZeroOrMore(pp.Word(pp.printables))
expr2 = a + pp.ZeroOrMore(pp.Word(pp.printables))

print(expr1.parseString(" foobar"))
print(expr2.parseString(" foobar"))

expr1 works and returns [' ', 'foobar'] (I'm not actually interested in the output)
expr2 fails with:

ParseException: Expected {<SPC><TAB><CR><LF>} (at char 1), (line:1, col:2)

Doesn't expression a evaluate to the same as pp.White(exact=1) in the case of " "?

The reason for expression a is that in my actual code the Or contains some other expressions for matching punctuation.

Parsing lists of objects instead of just strings of characters?

Today I made use of this awesome library to parse a list of divs while scraping a webpage. This webpage did not have sufficient structure in situ, but I was able to construct a string that pointed to the independent elements and parse that string based on a custom grammar. Mapping the parse tree back for that string gave me the parse tree for my divs.

This got me thinking about if there might be any value in being able to parse lists of arbitrary objects instead of just strings of characters? All of my hacking then to maintain a 1:1 mapping between my object and a string representation would be unnecessary. It might also be more efficient if we could do this directly. Do you think there would any value in supporting such a scenario (or even if there is a better way to do something like this)?

Distribute tests in pypi tarball

Hello in various distributions we try to make sure the packages we in the end ship to users are really working and for that we execute upstream unittests to ensure some validity. Esp. with python that is not compiled it is quite good idea to check that with distribution patching in python we didn't break something.

As such could you please include in Manifest.in the files executed by the tox/travis for us to validate against?

Slight documentation change to oneOf

The documentation for oneOf is correct, but it's easy to get tripped up by the following code:

from pyparsing import *

# Fails
expr = oneOf('count', 'sum')

# Works
expr = oneOf(['count', 'sum'])

def parse_expression(_input):
  results = expr.parseString(_input)
  print(results.dump())


parse_expression('sum')

That's just because oneOf takes a list object and not several parameters. It might be helpful to include a short note that you can't call it like I tried to above.

Great library, btw :)

statemachine example not working in Python3

Need to update statemachine example to work with Python3 importlib.
Also, add traffic light and library book examples.
(Probably needs to be in its own subdirectory of the examples directory, similar to verilog samples)

import error sklearn

Hello I am trying to import train_test_split module from sklearn and I am getting list index out of range error.

Below is my code:
import sklearn
from sklearn import model_selection
from sklearn.model_selection import train_test_split

Below is the error:
from sklearn import model_selection
Traceback (most recent call last):

File "", line 1, in
from sklearn import model_selection

File "C:\Users\gunngunner\Anaconda3\lib\site-packages\sklearn\model_selection_init_.py", line 19, in
from ._validation import cross_val_score

File "C:\Users\gunngunner\Anaconda3\lib\site-packages\sklearn\model_selection_validation.py", line 31, in
from ..metrics.scorer import check_scoring, _check_multimetric_scoring

File "C:\Users\gunngunner\Anaconda3\lib\site-packages\sklearn\metrics_init_.py", line 7, in
from .ranking import auc

File "C:\Users\gunngunner\Anaconda3\lib\site-packages\sklearn\metrics\ranking.py", line 36, in
from ..preprocessing import label_binarize

File "C:\Users\gunngunner\Anaconda3\lib\site-packages\sklearn\preprocessing_init_.py", line 6, in
from ._function_transformer import FunctionTransformer

File "C:\Users\gunngunner\Anaconda3\lib\site-packages\sklearn\preprocessing_function_transformer.py", line 5, in
from ..utils.testing import assert_allclose_dense_sparse

File "C:\Users\gunngunner\Anaconda3\lib\site-packages\sklearn\utils\testing.py", line 63, in
from nose.tools import raises as _nose_raises

File "C:\Users\gunngunner\Anaconda3\lib\site-packages\nose_init_.py", line 1, in
from nose.core import collector, main, run, run_exit, runmodule

File "C:\Users\gunngunner\Anaconda3\lib\site-packages\nose\core.py", line 11, in
from nose.config import Config, all_config_files

File "C:\Users\gunngunner\Anaconda3\lib\site-packages\nose\config.py", line 8, in
from nose.util import absdir, tolist

File "C:\Users\gunngunner\Anaconda3\lib\site-packages\nose\util.py", line 12, in
from nose.pyversion import ClassType, TypeType, isgenerator, ismethod

File "C:\Users\gunngunner\Anaconda3\lib\site-packages\nose\pyversion.py", line 36, in
import new

File "C:\Users\gunngunner\new.py", line 6, in
if a[item]%2 != 0:

IndexError: list index out of range

SyntaxWarning: invalid escape sequence \w

Someone reported a SyntaxWarning when using pyparsing with Python 3.8: pypa/pip#6362

It's caused by the following line starting with make_html = inside a docstring:

pyparsing/pyparsing.py

Lines 3073 to 3083 in fed0f3d

 def sub(self, repl): 

 """ 

  Return Regex with an attached parse action to transform the parsed 

  result as if called using `re.sub(expr, repl, string) <https://docs.python.org/3/library/re.html#re.sub>`_. 

  Example:: 

  make_html = Regex(r"(\w+):(.*?):").sub(r"<\1>\2</\1>") 

  print(make_html.transformString("h1:main title:")) 

  # prints "<h1>main title</h1>" 

  """

Add "project" label, and use them for larger projects/refactorings

Some of the issues, like recent #67, are too big for a single sitdown. There are also goals hiding in the wiki that are "project" like. Any one of them may be big enough for a student summer project.

Please add a "project" tag to mark these: Large items with low priority. As the Issue gets older, it will fall to the bottom so we need not see it, but at least we have a list of potential projects some others can tackle.

simple_unit_tests.TestResultsModifyingParseAction fails

When running the test suite with Python 2.7 (and with #47 merged in), I get this failed test:

[   58s] ======================================================================
[   58s] FAIL: runTest (simple_unit_tests.TestResultsModifyingParseAction)
[   58s] ----------------------------------------------------------------------
[   58s] Traceback (most recent call last):
[   58s]   File "/home/abuild/rpmbuild/BUILD/pyparsing-2.3.1/simple_unit_tests.py", line 59, in runTest
[   58s]     self.assertEqual(result.asDict(), test_spec.expected_dict)
[   58s] AssertionError: {'ave': 30, 'sum': 153, 'min': 1, 'max': 89} != {'ave': 30.6, 'sum': 153, 'min': 1, 'max': 89}
[   58s] - {'ave': 30, 'max': 89, 'min': 1, 'sum': 153}
[   58s] + {'ave': 30.6, 'max': 89, 'min': 1, 'sum': 153}
[   58s] ?           ++
[   58s] 
[   58s] -------------------- >> begin captured stdout << ---------------------
[   58s] 
[   58s] A parse action that adds new key-values - OneOrMore({integer}...)
[   58s] [27, 1, 14, 22, 89]
[   58s] - ave: 30
[   58s] - max: 89
[   58s] - min: 1
[   58s] - sum: 153
[   58s] 
[   58s] --------------------- >> end captured stdout << ----------------------
[   58s] 
[   58s] ----------------------------------------------------------------------

add tutorial

add links to tutorial articles. examples exist but different from tutorials

Generated Sphinx docs include large strings for defined unicode ranges

The sphinx documentation generates documentation for the new pyparsing_unicode classes' alphas, nums, printables, etc. strings. These inflate the generated pyparsing.html file to over 12MB in size.

Either these strings should be suppressed from the docs, or the generated code post-processed to clip them to a reasonable length.

PEP 8 and code division

Hello,

I was wondering if there is any interest to format the code to follow PEP 8 standards and modularize it to separated sub-packages to keep it divided by functionality?
I do believe that would be beneficial as code would become easier to maintain, contribute, test and document.

I'm working with PyParsing quite often and I was thinking to start that task. Just let me know if that's something you consider or this is out of the current development scope.

Define Meanwhile class

I defined Meanwhile class. It can be applied in some cases where others fail. Rename it if you have a good name. Following is the code.

class Meanwhile(pp.ParseExpression):
    """Parse expression whose all sub-expressions have to be matched at the same time

        Grammar:
            Meanwhile([A, B, ...]), where A, B, ... are parse expressions.
       
        Example:
            >>> A = Meanwhile([IDEN, ~('_'+ DIGIT)]) # IDEN % ('_'+ DIGIT) for short
            >>> to parse identifiers such as _a123 excluding _123
            >>> A.parseString('_a123')  # => ['_a123']
            >>> A.parseString('_123')   # => ParseException

            >>> A = Meanwhile([IDEN, pp.Word('?_123456xend'), pp.Word('_+-*/123x')+'end']) # (_|x) (_123x)* end
            >>> A.parseString('_xend')  # => ['_xend']
            >>> A.parseString('_123end') # => ['_123end']
            >>> A.parseString('_abc')   # => ParseException
    """

    def __init__(self, exprs=[]):
        '''
        Keyword Arguments:
            exprs {list} -- list of parse expressions (default: {[]})
        '''
        super(Meanwhile, self).__init__(exprs)
        self.mayReturnEmpty = all(e.mayReturnEmpty for e in self.exprs)
        self.setWhitespaceChars(self.exprs[0].whiteChars)
        self.skipWhitespace = self.exprs[0].skipWhitespace
        self.callPreparse = True

    def parseImpl(self, instring, loc, doActions=True):
        postloc, result = self.exprs[0]._parse(instring, loc, doActions)
        for e in self.exprs[1:]:
            if not e.matches(instring[loc:], parseAll=True):
                raise _Exception(instring, len(instring), e.errmsg, self)
        return postloc, result

    def __mod__(self, other):
        if isinstance(other, str):
            other = pp.ParserElement._literalStringClass(other)
        return self.exprs.append(~other)

    def checkRecursion(self, parseElementList):
        subRecCheckList = parseElementList[:] + [self]
        for e in self.exprs:
            e.checkRecursion(subRecCheckList)
            if not e.mayReturnEmpty:
                break

    def __str__(self):
        if hasattr(self, "name"):
            return self.name

        if self.strRepr is None:
            self.strRepr = "%s{%s}"%(str(self.exprs[0]), " ".join(str(e) for e in self.exprs[1:]))

        return self.strRepr

Fixed a bug, added a new example.

pyparsing for python code

Hi, I want to parse python code.
I want to make some script like python, and I wanted to find some code example to parse python.
Of course, this github contains simple examples, such as calc or sql.
But if possible, may I know whether there is parsing example for python code?

Thanks.

Passing whitespace characters to White() not in whiteStrs causes KeyError

Environment: macOS 10.14.1, Python 2.7.15 from MacPorts, pyparsing 2.3.0 from pypi

When constructing a White() token with characters in the ws= parameter that are not in the White.whiteStrs dict, such as \u00a0 (Unicode NO-BREAK SPACE), the constructor raises a KeyError when it tries to set the self.name property. Example:

Python 2.7.15 (default, Oct  1 2018, 15:59:56)
[GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.11.45.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from pyparsing import White
>>> w = White(ws=u' \t\u00a0')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyparsing.py", line 3132, in __init__
    self.name = ("".join(White.whiteStrs[c] for c in self.matchWhite))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pyparsing.py", line 3132, in <genexpr>
    self.name = ("".join(White.whiteStrs[c] for c in self.matchWhite))
KeyError: u'\xa0'

My workaround for now is to manually add the necessary key/value pairs into the whiteStrs dict:

>>> from pyparsing import White
>>> White.whiteStrs[u'\u00a0'] = "<NO-BREAK SPACE>"
>>> w = White(u' \t\u00a0')
>>> w.name
'<SPC><TAB><NO-BREAK SPACE>'

It may be a good idea to provide a method for setting whiteStrs entries, or have the name generation process gracefully handle non-existent keys.

Problem parsing strings that are single quoted and ends with double single escaped quote character

Hi,
I'm trying to parse the following string in Python with pyparser.
This is part of a much more complicated use case and I've extracted the relevant part here.

>>> from pyparsing import QuotedString
>>> QUOTED_STRING = QuotedString("'", escChar=("''"), unquoteResults=False)
>>> stringToParse = "'A string that ''ends'' with double single quotes ''1'''"
>>> val = QUOTED_STRING.parseString(stringToParse).asList()
>>> print val[0]
'A string that ''ends'' with double single quotes ''1'
>>> import pyparsing
>>> print pyparsing.__version__
2.3.1

The problem here is that I'm expecting to get the string
'A string that ''ends'' with double single quotes ''1'''
but val[0] equals the following in my case
'A string that ''ends'' with double single quotes ''1'

Is this a bug? or do I miss something here?
Kind regards,
Laurent

recursion depth issues

A customer reported a traceback to us with a maximum recursion depth exceeded error originating from pyparsing.
The customer had entered an SQL where clause not dissimilar to the one in the attached script (that reproduces the issue).
I am working around this by using sys.setrecursionlimit() which is unpleasant.

with high_recursion_limit():
    tokens, start, end = where_clause.scanString(s).next()

"Don't write stupid queries" is a tempting answer - but it does not make the software more reliable.

pyparsing_recursion.txt

about the return value of infixNotation

infixNotation always returns a list of list, as [[['-', 2], '-', ['-', 11]]]. I think the outer list is redundant, providing no information. Why not omit it?

SkipTo() generates a ParseResults object in the result dictionary under some circumstances

In the first parse result below I expected the element in the results dictionary matching parser prefix to be of type str, like for the first element in the ParseResults list, but the actual type is ParseResults.

If the string 'suffix' is omitted from SkipTo, results in both the list and dictionary match the expected str type.

Why is pyparsing generating a ParseResults object in the first case?

pyparsing 2.2.0

from pyparsing import *

data = Literal("DATA")

# SkipTo with suffix
prefix = SkipTo(data + 'suffix')("prefix")
p = prefix + data

text = "prefixDATAsuffix"

p.runTests(text, parseAll=False)
res1 = p.parseString(text)
print("res1[0]:", type(res1[0]))
print('res1["prefix"]:', type(res1["prefix"]))

# SkipTo without suffix
prefix2 = SkipTo(data)("prefix")
p2 = prefix2 + data

p2.runTests(text, parseAll=False)
res2 = p2.parseString(text)
print("res2[0]:", type(res2[0]))
print('res2["prefix"]:', type(res2["prefix"]))

prefixDATAsuffix
['prefix', 'DATA']
- prefix: ['prefix']

res1[0]: <class 'str'>
res1["prefix"]: <class 'pyparsing.ParseResults'>      <-----

prefixDATAsuffix
['prefix', 'DATA']
- prefix: 'prefix'

res2[0]: <class 'str'>
res2["prefix"]: <class 'str'>

Missing LICENSE file

Source archive on PYPI is missing LICENSE file

indentedBlock not clearing the indent stack when it partially matches, and then fails

First of all, thanks for this amazingly powerful and expressive library. I'm so glad to have found it.

I want to parse a language that uses indentation semantically, and thus I am using the indentedBlock function. However, I want to use it in combination with scanString(), because not all text in my input string belongs to this language.

However, in doing this I have noticed a bug in the indentedBlock. If you try to parse a string that does include an indented block, but doesn't completely parse, the indent stack will not be reverted to how it was, and thus it will fail to parse all correct statements.

To demonstrate this, I've made a simple test case: https://github.com/TMiguelT/PyparsingIndent/blob/master/indent.py. The comments explain that, if the indentedBlock expression matches, but then the rest of the parsing fails, then the parser will fail to match anything from then on.

API documentation not available

The API documentation on the pyparsing homepage seems to be empty. Is there any other way to access it?

Infinite recursion

There is currently an issue with infinite recursion that occurs with the latest version of python-dice (v2.4.0), using Python 3.7 and pyparsing 2.4.0, which does not occur when using pyparsing 2.3.1.

Simply executing dice.roll('6') will immediately cause the program to start using up all available memory, potentially causing the system to freeze.
If a keyboard interrupt is used to stop the execution, the traceback looks like this:

>>> dice.roll('6')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Program Files\Python37\lib\site-packages\dice\__init__.py", line 24, in roll
  File "C:\Program Files\Python37\lib\site-packages\dice\__init__.py", line 43, in _roll
    ast = parse_expression(string)
  File "C:\Program Files\Python37\lib\site-packages\dice\__init__.py", line 38, in parse_expression
    return dice.grammar.expression.parseString(string, parseAll=True)
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 1811, in parseString
    self.streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3728, in streamline
    super(And, self).streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3655, in streamline
    e.streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3728, in streamline
    super(And, self).streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3655, in streamline
    e.streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 4651, in streamline
    self.expr.streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 4651, in streamline
    self.expr.streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3892, in streamline
    super(MatchFirst, self).streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3655, in streamline
    e.streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3728, in streamline
    super(And, self).streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3655, in streamline
    e.streamline()
  
  ...
  
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 4651, in streamline
    self.expr.streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3892, in streamline
    super(MatchFirst, self).streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3655, in streamline
    e.streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3728, in streamline
    super(And, self).streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3655, in streamline
    e.streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 4651, in streamline
    self.expr.streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3892, in streamline
    super(MatchFirst, self).streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3655, in streamline
    e.streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3728, in streamline
    super(And, self).streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3681, in streamline
    self.errmsg = "Expected " + _ustr(self)
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3774, in __str__
    self.strRepr = "{" + " ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3774, in <genexpr>
    self.strRepr = "{" + " ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 4671, in __str__
    retString = _ustr(self.expr)
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3931, in __str__
    self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3931, in <genexpr>
    self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 4671, in __str__
    retString = _ustr(self.expr)
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3931, in __str__
    self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3931, in <genexpr>
    self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 4671, in __str__
    retString = _ustr(self.expr)
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3931, in __str__
    self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3931, in <genexpr>
    self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 4671, in __str__
    retString = _ustr(self.expr)
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3931, in __str__
    self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3931, in <genexpr>
    self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3774, in __str__
    self.strRepr = "{" + " ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3774, in <genexpr>
    self.strRepr = "{" + " ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 4671, in __str__
    retString = _ustr(self.expr)
  
  ...
    
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3931, in __str__
    self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3931, in <genexpr>
    self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3774, in __str__
    self.strRepr = "{" + " ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3774, in <genexpr>
    self.strRepr = "{" + " ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 4671, in __str__
    retString = _ustr(self.expr)
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3931, in __str__
    self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
KeyboardInterrupt

The relevant expression can be seen at https://github.com/borntyping/python-dice/blob/v2.4.0/dice/grammar.py.

I believe this is due to the change in ed2f5ec (#71).
Edit: I've now tested this and the issue does not occur when this change is reverted.

cc @calebj

Import error in IronPython 2.7.7

Hey @ptmcg
This error happens on module import in IronPython 2.7.7 (file paths are simplified to ...\ for brevity)

Traceback (most recent call last):
 File "...\script.py", line 4, in 
 File "...\pyparsing.py", line 4734, in 
 File "...\pyparsing.py", line 1261, in setParseAction
 File "...\pyparsing.py", line 1043, in _trim_arity
IndexError: index out of range: -1

I revised the line # 1043:1044 to below to avoid the issue:
Before:

    LINE_DIFF = 6
    # IF ANY CODE CHANGES, EVEN JUST COMMENTS OR BLANK LINES, BETWEEN THE NEXT LINE AND
    # THE CALL TO FUNC INSIDE WRAPPER, LINE_DIFF MUST BE MODIFIED!!!!
    this_line = extract_stack(limit=2)[-1]
    pa_call_line_synth = (this_line[0], this_line[1]+LINE_DIFF)

After:

    LINE_DIFF = 6
    # IF ANY CODE CHANGES, EVEN JUST COMMENTS OR BLANK LINES, BETWEEN THE NEXT LINE AND
    # THE CALL TO FUNC INSIDE WRAPPER, LINE_DIFF MUST BE MODIFIED!!!!
    exstack = extract_stack(limit=2)
    if exstack:
        this_line = extract_stack(limit=2)[-1]
        pa_call_line_synth = (this_line[0], this_line[1]+LINE_DIFF)
    else:
        pa_call_line_synth = ''

This temporarily resolves the issue but I hope you can provide a more appropriate fix.
Thank you for pyparsing.

unitTests.UnicodeTests fail with python2

With tarball pyparsing-2.3.1~test5.tar.gz I get this one test failing with Python 2:

$ python2 unitTests.py  -v
Beginning test of pyparsing, version 2.3.1
Python version 2.7.15 (default, May 21 2018, 17:53:03) [GCC]
.........................................................................................
>>>> Starting test UnicodeTests
[u'\u039a\u03b1\u03bb\u03b7\u03bc\u03ad\u03c1\u03b1', ',', u'\u03ba\u03cc\u03c3\u03bc\u03b5', '!']
<<<< End of test UnicodeTests


E..........................................................................................
>>>> Starting test UnicodeTests
[u'\u039a\u03b1\u03bb\u03b7\u03bc\u03ad\u03c1\u03b1', ',', u'\u03ba\u03cc\u03c3\u03bc\u03b5', '!']
<<<< End of test UnicodeTests


E..
======================================================================
ERROR: UnicodeTests
----------------------------------------------------------------------
Traceback (most recent call last):
  File "unitTests.py", line 94, in _runTest
    self.runTest()
  File "unitTests.py", line 3666, in runTest
    result = pp.Dict(pp.OneOrMore(pp.Group(key_value))).parseString(sample)
  File "/home/matej/build/python/python-pyparsing/pyparsing-2.3.1/pyparsing.py", line 1729, in parseString
    raise exc
ParseException: Expected "=" (at char 13), (line:1, col:14)

======================================================================
ERROR: UnicodeTests
----------------------------------------------------------------------
Traceback (most recent call last):
  File "unitTests.py", line 94, in _runTest
    self.runTest()
  File "unitTests.py", line 3666, in runTest
    result = pp.Dict(pp.OneOrMore(pp.Group(key_value))).parseString(sample)
  File "/home/matej/build/python/python-pyparsing/pyparsing-2.3.1/pyparsing.py", line 1729, in parseString
    raise exc
ParseException: Expected "=" (at char 13), (line:1, col:14)

----------------------------------------------------------------------
Ran 183 tests in 2.346s

FAILED (errors=2)

Complete build log with python2 run for unitTests.py omitted

More intelligent or user-friendly exceptions

import pyparsing as pp
a = pp.Literal('Hello') + pp.delimitedList(pp.pyparsing_common.identifier('subname'), '.')('Name')
a.parseString('Hello 123')

It will report
pyparsing.ParseException: Expected identifier (at char 6), (line:1, col:7)

I hope that it can point out the name of the expression Name and what I input, for example
I expected "Name"(or "subname"), but got the numbers "123".

More ambitious idea is that it can report more than one errors, it can jump the current error to parse the next word.

Just a suggestion.

Ignoring character patterns

Hi, I'm working on a better parser for a project that I run - the parser is for a templating language that allows a concept of "lookups" which will execute functions and replace the text with the results. A lookup is in the form of: ${lookup_name lookup_values} and they can be nested (you can provide a lookup as a lookup value).

I came up with the following use of pyparsing, but it fails on one of the test cases - largely because I feel like I'm missing something key to this kind of a parser. Here's my code:

#!/usr/bin/python

from pyparsing import (
    CharsNotIn,
    Forward,
    OneOrMore,
    Suppress,
    White,
    Word,
    alphanums,
    alphas,
)


def build_parser():
    _open = "${"
    _close = "}"

    lookup = Forward()

    plain_string = CharsNotIn(_open + _close).setResultsName("contents")

    lookup_name = Word(alphas, alphanums + "_").setResultsName("name")
    lookup_input = OneOrMore(plain_string | lookup).setResultsName("contents")

    lookup << (
        Suppress(_open) + lookup_name + Suppress(OneOrMore(White(" ")))
        + OneOrMore(lookup_input) + Suppress(_close)
    )

    exp = OneOrMore(lookup | plain_string)

    return exp


if __name__ == '__main__':
    tests = [
        ("apple",
         "apple"),
        ("banana$apple",
         "banana$apple",),
        ("apple, apple, banana",
         "apple, apple, banana"),
    ]

    exp = build_parser()
    for t, expected in tests:
        r = exp.parseString(t)
        if not r.contents == expected:
            print "Failure:"
            print "  expected: %s" % expected
            print "    result: %s" % r.contents

And the output:

$ python test_parser.py
Failure:
  expected: banana$apple
    result: banana

Is there a better way of doing the:

plain_string = CharsNotIn(_open + _close).setResultsName("contents")

That only ignores strings that start with ${ and end with }?

Thanks! And thanks for this awesome project!

	def sub(self, repl):
	"""
	Return Regex with an attached parse action to transform the parsed
	result as if called using `re.sub(expr, repl, string) <https://docs.python.org/3/library/re.html#re.sub>`_.

	Example::

	make_html = Regex(r"(\w+):(.*?):").sub(r"<\1>\2</\1>")
	print(make_html.transformString("h1:main title:"))
	# prints "<h1>main title</h1>"
	"""