aclements / biblib Goto Github PK

Simple, faithful BibTeX parser and algorithms for Python 3

License: MIT License

Python 100.00%

biblib's Introduction

Biblib provides a simple, standalone Python3 package for parsing BibTeX bibliographic databases, as well as algorithms for manipulating BibTeX entries in BibTeX-y ways.

There are a lot of BibTeX parsers out there. Most of them are complete nonsense based on some imaginary grammar made up by the module's author that is almost, but not quite, entirely unlike BibTeX's actual grammar. BibTeX has a grammar. It's even pretty simple, though it's probably not what you think it is. The hardest part of BibTeX's grammar is that it's only written down in one place: the BibTeX source code.

Biblib's parser is derived directly from the WEB source code for BibTeX and hence (barring bugs in translation) should be fully compatible with BibTeX's own parser.

Features

BibTeX-compatible .bib file parser
BibTeX-compatible name parser for fields like author
Crossref resolution
BibTeX-compatible title casing
Translator for common TeX markup (like accents) to Unicode (which can, in turn, be used in HTML and other formats).

Installation

Since biblib has no external dependencies or C modules, you can use biblib in your project by simply unpacking it under your source tree and adding

sys.path.append('biblib')

before importing it.

Biblib can also be installed system-wide with

python3 setup.py install

Examples

There are a few simple examples of biblib's use in examples/. To run these dircetly from the source tree, use, for example

PYTHONPATH=$PWD ./examples/bibparse test.bib

Recognized grammar

For reference, the .bib parser implements a grammar equivalent to the following PEG. All literals are matched case-insensitively.

bib_db = comment (command_or_entry comment)*

comment = [^@]*

ws = [ \t\n]*

ident = ![0-9] (![ \t"#%'(),={}] [\x20-\x7f])+

command_or_entry = '@' ws (comment / preamble / string / entry)

comment = 'comment'

preamble = 'preamble' ws ( '{' ws preamble_body ws '}'
                         / '(' ws preamble_body ws ')' )

preamble_body = value

string = 'string' ws ( '{' ws string_body ws '}'
                     / '(' ws string_body ws ')' )

string_body = ident ws '=' ws value

entry = ident ws ( '{' ws key ws entry_body? ws '}'
                 / '(' ws key_paren ws entry_body? ws ')' )

key = [^, \t}\n]*

key_paren = [^, \t\n]*

entry_body = (',' ws ident ws '=' ws value ws)* ','?

value = piece (ws '#' ws piece)*

piece
    = [0-9]+
    / '{' balanced* '}'
    / '"' (!'"' balanced)* '"'
    / ident

balanced
    = '{' balanced* '}'
    / [^{}]

biblib's People

Stargazers

Watchers

biblib's Issues

pypi discrepancies

Is this code the latest version? pypi has 0.1.2 available, but the version string in the code here is 0.1.0.

The pypi version also seems to require a non-existent module magic and a module unidecode which is not mentioned here in the source code...

recursion error upon reading .bib file

Hi I just installed biblib and run the following simple example from the tutorials

import biblib
db  = biblib.FileBibDB('path/to/file.bib', mode='r')
for entry in db.values():
      print(entry)

I get the following error

Traceback (most recent call last):
  File "/python3.6/site-packages/biblib/_entry.py", line 48, in __new__
    return cls._registeredTypes[inputdict['ENTRYTYPE']](inputdict)
  File "/python3.6/site-packages/biblib/_entry.py", line 48, in __new__
    return cls._registeredTypes[inputdict['ENTRYTYPE']](inputdict)
  File "/python3.6/site-packages/biblib/_entry.py", line 48, in __new__
    return cls._registeredTypes[inputdict['ENTRYTYPE']](inputdict)
  [Previous line repeated 997 more times]
RecursionError: maximum recursion depth exceeded while calling a Python object

Even tried increasing the recursion stack but nothing

import sys
sys.setrecursionlimit(10000)

Any ideas?

No longer works with python 3.10

This is essentially abandonware. It has a dependency upon collections.Iterable, which was removed in python 3.10.

Is there a way to recover the actual errors after they have been collected with InputErrorRecoverer?

When the parser tries to parse a BibTeX entry that it already has, using the Parser.parse() function, does so using a recoverer.
The exception thrown should include information about why it failed, and, as seen in line 274 of bib.py:

self._fail('repeated entry')

this is programmed. But the recoverer only throws the amount of input errors, not the message. Is there a way to catch this message? It is important to me to be able to know why the exception is being thrown.

About PyPI package?

Hi,

So there is a biblib package on PyPI which is unfortunately not the correct one, I was wondering though if there was any plan for releasing this one though. It would be super convenient for automation.

Cheers,

Thomas

@comment should be followed with delimiter

A valid @comment command should be followed with a white space character, a comma, or a left brace. This is because BibTeX first scans for a identifier following @ (bibtex.web#L5433) and then determines if it's a comment (bibtex.web#L5465-L5470). Thus the comment rule should be 'comment' &[ \t\n{(].

See the following LaTeX test case.

\documentclass{article}
\begin{filecontents}[noheader,overwrite]{\jobname.bib}
@comment @book{test-comment-1,
  note = {The entry type is "book".}
}

@comment@book{test-comment-2,
  note = {The entry type is "comment@book".}
}
\end{filecontents}

\begin{filecontents}[noheader,overwrite]{\jobname.bst}
ENTRY { note } { } { }
FUNCTION {default.type}
{ "type: " type$ * top$
  "key: " cite$ * top$
}
FUNCTION {book} {default.type}
FUNCTION {comment@book} {default.type}
READ
ITERATE {call.type$}
\end{filecontents}

\begin{document}
\nocite{*}
\bibliographystyle{\jobname}
\bibliography{\jobname}
\end{document}

BibTeX treats comment@book as the entry type.

This is BibTeX, Version 0.99d (TeX Live 2023)
The top-level auxiliary file: main.aux
The style file: main.bst
Database file #1: main.bib
type: book
key: test-comment-1
type: comment@book
key: test-comment-2

BTW, there are duplicated rules named comment and they can be distinguished as "implicit comment" and "comment command".

Name parser's incorrect handling of certain accents such as \\H

This is a wonderful package, but I have run into a problem: In TeX and LaTeX, once can specify accents such as \H either in the form \H{o} or \H o. In the latter case, the NameParser code splits the second form incorrectly.
biblib.algo.parse_names("Sz\\H{o}l\\H{o}, Abel and Sz\\H ol\\H o, Baker")
[Name(first='Abel', von='', last='Sz\\H{o}l\\H{o}', jr=''),
Name(first='Baker', von='Sz\\H ol\\H', last='o', jr='')]

For reasons I don't understand, surrounding the second \H o with braces for Baker will parse that name correctly, even though the first \H o still contains a space.
biblib.algo.parse_names("Sz\\H ol{\\H o}, Baker")
[Name(first='Baker', von='', last='Sz\\H ol{\\H o}', jr='')]

This particular problem can be solved by converting to Unicode first
biblib.algo.parse_names(biblib.algo.tex_to_unicode("Sz\\H{o}l\\H{o}, Abel and Sz\\H ol\\H o, Baker"))
[Name(first='Abel', von='', last='Szőlő', jr=''),
Name(first='Baker', von='', last='Szőlő', jr='')]
but that approach strips out braces needed, for example, to specify an institution name and have it parsed as all being a last name, not split into name components. E.g.,
biblib.algo.parse_names(biblib.algo.tex_to_unicode("Sz\\H ol\H o, Baker and {NRC Committee}"))
[Name(first='Baker', von='', last='Szőlő', jr=''),
Name(first='NRC', von='', last='Committee', jr='')]

I think that the NameParser's algorithm must special-case accent specifiers such as \H, \r, \u, \v, etc., to be sure not to split tokens on spaces following them, just as it now special-cases spaces at a brace depth > 0.

Derived from the WEB source?

How did you derive the parser from the WEB source? I'm trying to create a Javascript version of this lib, but the pascal produced from bibtex.web by tangle is very hard to read.