juliadocs / documentercitations.jl Goto Github PK

I just gave this a first try and noticed that my Zotero Export looks like

@book{Boumal:2023,
  title = {An {{Introduction}} to {{Optimization}} on {{Smooth Manifolds}}},
  author = {Boumal, Nicolas},
  year = {2023},
  month = mar,
  edition = {First},
  publisher = {{Cambridge University Press}},
  doi = {10.1017/9781009166164},
  isbn = {978-1-00-916616-4}
}

I am not sure why Zotero just gives curly braces to everyone, but probably it thinks it is necessary to preserve capitalisation.
Besides me also fighting how the keys will be rendered correctly the current literature then looks like

So all curly braces are kept. In LaTeX these are not kept.
Sure this is also partly Zotero being stupid, but it would be great it curly braces could be removed here in the output

After #32 is merged, we can extend eprint support to include bioRxiv and HAL (in addition to the default arXiv)

• For HAL see e.g. https://hal.science/hal-03909427/, where the eprint id is hal-XXXXXXXX.

• For bioRxiv, see https://www.biorxiv.org/collection/biophysics. The eprint id is the last part of of the DOI (after 10.1101/)

The behavior of non-arXiv preprint should be documented, including the fact that arbitrary prefixes will render correctly, but only explicitly supported preprint servers will be linked automatically

In this section

https://juliadocs.org/DocumenterCitations.jl/stable/syntax/#Citations-in-docstrings

there is a very nice description, that I already follow in my docs as well. In html this looks really nice. Would it be possible to render it that nice on REPL as well? Currently a link like

in [Bergmann et. al., SIAM J Imag Sci, 2014](@cite BergmannLausSteidlWeinmann:2014:1)

renders on REPL like

in Bergmann et. al., SIAM J Imag Sci, 2014 (@cite BergmannLausSteidlWeinmann:2014:1).

and has no collar markup like usually links do omitting the bracket and collaring it would be super nice.

Clicking on a citation takes the reader out the current context, as it generally links to the References page of the project. Even though the browser's back button will return the location of the citation, it can still be disorienting.

A nicer experience would be to show the full reference when hovering over the citation. A similar feature for footnotes was requested in JuliaDocs/Documenter.jl#2080.

As mentioned there, this feature may be possible to implement with pure CSS, or it might require some javascript. And example for javascript that implements something like this would be https://www.jqueryscript.net/text/Inline-Footnotes-On-Hover.html (demo).

Adding javascript functionality to DocumenterCitations would raise the question on how a user would set this up. We already have instructions for manually adding CSS. Adding .js files to that (especially if they're required for the plugin to function properly) might be a bit too much. It would be good have have capabilities in Documenter to enable plugins to inject custom css/js, see JuliaDocs/Documenter.jl#1612.

On the other hand, if JuliaDocs/Documenter.jl#2080 was implemented first, that might allow DocumenterCitations to just reuse that implementation.

(Sorry for posting yet another issue)

I noticed two interesting things about non canonical bibliographies, i.e. when I want to have a “Literature” section at the end of my documentation page.

assume the following: I have a source file called src/functions/proximal_maps.jl whose functions shall be documented on a page /functions/proximal_maps.html so I have a file docs/src/functions/proximal_maps.md to do that. This file could in a minimal sense look like in

# [Proximal Maps](@id proximalMapFunctions)

```@autodocs
Modules = [Manopt]
Pages   = ["proximal_maps.jl"]
```

# Literature

```@bibliography
Pages = ["functions/proximal\_maps.md"]
Canonical=false
```

There is a few things to note

1. for @autodocs a file name is enough, while @bibliography needs the complete relative path
2. for @autodocs I do not have to escape the _, for @bibliography I do.
3. For @bibliography it would be nice to have a way to say (maybe something similarly short as the nice * “I want all references from this markdown file”, because this relative path to the markdown file I am in is currently something I type at several pages.

Besides these three points, the workflow overall is quite nice.

Transferred from JuliaQuantumControl/QuantumCitations.jl#7

It might be better not to generate raw HTML for the bibliography. See the comments at JuliaDocs/Documenter.jl#1162 (comment)

@Seelengrab requested support for the urldate field in #52 (comment).

In principle, adding "accessed on" information can already be done with the note field, so I'd like to clarify the motivation for supporting an additional urldate field and its expected behavior.

I could think of the following two motivations:

1. Citation management software like Zotero may export urldate automatically, and it would be convenient not to have to re-edit the information into the note field.
2. A "smart" urldate field could ensure consistent rendering of the "accessed on" date.

I'd consider especially the second point as a good reason to have a urldate field. That is, urldate should be processed in some non-trivial way. For this to work,

• The urldate field must be in the format YYYY-MM-DD. I checked that this is the case for biblatex: it will ignore anything in a different format.
• The date will be parsed and validated, and rendered as, e.g., "Accessed on <human-readable-date>", where <human-readable-date> is reformatted from the original YYYY-MM-DD. It seems like biblatex uses the American MM/DD/YYYY (maybe that depends on my locale settings), but since this is generally ambiguous, I'd probably opt for "Oct 24, 2023"

See https://s3.amazonaws.com/julialang-reports/nanosoldier/pkgeval/by_date/2023-10/15/DocumenterCitations.primary.log

I think the problem is that Pkg is used for the documentation/doctests somewhere, but is not declared a dependency/test-dependency/docs-dependency.

Just saw on CI that the Pages= argument might have been changed from absolute to relative files? See the error message on CI at

https://github.com/JuliaManifolds/Manopt.jl/actions/runs/6717763433/job/18256227165?pr=294#step:9:1220

┌ Error: Invalid "functions/gradients.md" in Pages attribute of @bibliography block on page src/functions/gradients.md: No such file "src/functions/functions/gradients.md".
└ @ DocumenterCitations ~/.julia/packages/DocumenterCitations/HU6z2/src/expand_bibliography.jl:279

especially the message which file was not found indicated, that removing functions/ should fix it (will try) – which is of course not so much a problem, but a breaking change in a non-breaking update.

I would like to add notes to references to mention open access preprints (since we have BibTeX and not BibLaTeX where I could use eprint=), but if I provide a note= field, this is not shown in the :alpha style. Is there an easy trick to add this (i.e. a modification / an own style)?
Or is that something where Bibliography takes care of the rendering?

From https://discourse.julialang.org/t/ann-documentercitations-v1-0-0/101562/6

I'm not exactly sure what the use case is for this, but it would certainly be easy to implement.

The most straightforward syntax for showing all references in the .bib file except for three specific ones would probably be

```@bibliography
*
! BrifNJP2010
! GoerzDiploma2010
! GoerzPhd2015
```

That is, !, followed by a space (since technically citation keys are allowed to start with !)

In DocumenterCitations 1.2, a @book entry could have both a URL and a DOI. The former would be automatically linked from the title, and the latter from the publisher information.

The 1.3 release changed how the auto-links work. From NEWS.md:

There were some subtle changes in how DOI and URL fields are linked in the rendered documentation, which may require adjustments. In particular, in version 1.3.0, non-@article entries that do not have both a title and a booktitle can only have a DOI or a URL, but not both. This most likely occurs for @book entries. For @book entries that have both a DOI and a URL, the URL should be placed in the note field (using \url/\href, as appropriate).

Unfortunately, this means the .bib files that worked fine with version 1.2 had to be modified for version 1.3; arguably a violation of semantic versioning. Thus, the behavior in 1.3 should be considered a bug.

To restore non-breaking compatibility with version 1.2, any entry that defines all the required fields for its specific type should be able to automatically link both a URL and a DOI. For the @book type, both title and publisher/year are required, so it would be safe to link the URL (or DOI, if no URL) from the title and the DOI (if no URL) from the publisher and year information.

Personally, I have some minor aesthetic problems with linking from the publisher, and I would generally prefer to link the URL from the note field for a book that has both a URL and a DOI. However, purely for backwards compatibility, automatic linking should still work, as it worked in version 1.2.

Link checking errors when using citations from
https://github.com/ali-ramadhan/DocumenterCitations.jl

Following up on ali-ramadhan/DocumenterCitations.jl#22 (comment), it could be nice to support arbitrary CSL styles. Even though personally, I'm happy with the new built-in styles, I just want to give an outline of how this could be implemented:

In the newly released v1.0, the CitationBibliography plugin can now be instantiated with a style keyword parameter. For the built-in styles, this is just a name (symbol). However, in general it can be an arbitrary object, enabling the "style" to have options or hold internal state. Thus, it would be conceivable to have, e.g.,

bib = CitationBibliography(
    "refs.bib";
    style=CSLStyle("chicago-author-date.csl")
)

in docs/make.jl (randomly using one of the many CSL styles as an example). The various internal DocumenterCitations routines would then receive the CSLStyle object as a parameter and specialize to generate the appropriate output.

We would be open to any PR implementing the CSLStyle and the methods to render the bibliography according to the CSL specification. Or, support for CSL could also be implemented as an independent package extending DocumenterCitations.

Maybe something that @Azzaare would be interested in.

See https://github.com/JuliaQuantumControl/QuantumCitations.jl. Personally I think everything there should be included here. Perhaps it would be easier to rename that repository and move it here instead if replaying all commits? Thoughts @goerz?

I found no way to include a citation in a docstring of a method.

"""
     f(x)

A dummy function to show the problem [dummycitation](@cite).
"""
function f(x)
    x+1
end

Maybe there is one...

Citation is read from the references.bib, but not found later.
The following error show up:

Warning: Bibliography anchor for key 'dummycitation' not found.
└ @ DocumenterCitations .....\.julia\packages\DocumenterCitations\f0eDy\src\citations.jl:456

A workaround is to place the citation in a .md of your documentation pages.

[dummycitation](@cite)

Then the citation is rendered even inside doscstrings.

Using DocumenterCitations v1.2.0 with Julia 1,9.1.

In a project implementing multiple papers' algorithms, the original bibtex files are served as assets. But they can also be used to create references. The approach I tried was

allbibs = CitationBibliography[]
bibfiledir = joinpath(@__DIR__, "src", "assets", "bibtex")
for bibfile in readdir(bibfiledir, join=true)
    push!(allbibs, CitationBibliography(bibfile))
end

and passing plugins = allbibs to makedocs(), but Documenter.jl throws the error LoadError: ArgumentError: only one copy of CitationBibliography may be passed. Currently cannot find a way to add multiple bib files (other than adding a part in make.jl to cat all files in a main bib file, but that's rather hacky). Any advice/guidance would be helpful.

I narrowed my make file down to

Scenario 1

using Manopt
using Documenter: DocMeta, HTML, MathJax3, deploydocs, makedocs
using DocumenterCitations

# (e) ...finally! make docs
bib = CitationBibliography(joinpath(@__DIR__, "src", "references.bib"); style=:authoryear)
print(bib) # displays all my references in a complicated formal -> references.bib found
makedocs(bib;
    format=HTML(; mathengine=MathJax3(), prettyurls=get(ENV, "CI", nothing) == "true"),
    modules=[Manopt],
    sitename="Manopt.jl",
    pages=[
        "Home" => "index.md",
    ],
)
deploydocs(; repo="github.com/JuliaManifolds/Manopt.jl", push_preview=true)

and in my index.md

```@bibliography
Pages = ["index.md"]
Canonical = false
```

This does not find the cites and warns that the links do not exist like

┌ Warning: invalid local link: unresolved path in index.md
│   link.text =
│    1-element Vector{Any}:
│     "Boumal:2023"
│   link.url = "@cite"
└ @ Documenter.Writers.HTMLWriter ~/.julia/packages/Documenter/bYYzK/src/Writers/HTMLWriter.jl:2081

But generates the right bibliography at the end of the index page.

Scenario 2

If I instead set line 2 to

using Documenter

I get

ERROR: LoadError: MethodError: no method matching HTML(; mathengine::MathJax3, prettyurls::Bool)

Closest candidates are:
  HTML(::T) where T got unsupported keyword arguments "mathengine", "prettyurls"
   @ Base docs/utils.jl:30
  HTML(::Any...) got unsupported keyword arguments "mathengine", "prettyurls"
   @ Base docs/utils.jl:33

Stacktrace:
 [1] top-level scope
   @ ~/Repositories/Julia/Manopt.jl/docs/make2.jl:11

If I leave out the format= keyword in the makedocs I am back to references not being found.

Scenario 3

I leave out the Canonical=false in the index.md (maybe one needs at least one canonical one for the links to work?)

The I get

[ Info: Populate: populating indices.
ERROR: LoadError: MethodError: no method matching header_level(::BibInternal.Entry)

Closest candidates are:
  header_level(::Markdown.Header{N}) where N
   @ Documenter ~/.julia/packages/Documenter/bYYzK/src/Utilities/Utilities.jl:380

Stacktrace:
 [1] populate!(contents::Documenter.Documents.ContentsNode, document::Documenter.Documents.Document)
   @ Documenter.Documents ~/.julia/packages/Documenter/bYYzK/src/Documents.jl:451
 [2] populate!(document::Documenter.Documents.Document)
   @ Documenter.Documents ~/.julia/packages/Documenter/bYYzK/src/Documents.jl:413
 [3] runner(#unused#::Type{Documenter.Builder.Populate}, doc::Documenter.Documents.Document)
   @ Documenter.Builder ~/.julia/packages/Documenter/bYYzK/src/Builder.jl:247
 [4] dispatch(#unused#::Type{Documenter.Builder.DocumentPipeline}, x::Documenter.Documents.Document)
   @ Documenter.Utilities.Selectors ~/.julia/packages/Documenter/bYYzK/src/Utilities/Selectors.jl:170
 [5] #2
   @ ~/.julia/packages/Documenter/bYYzK/src/Documenter.jl:273 [inlined]
 [6] cd(f::Documenter.var"#2#3"{Documenter.Documents.Document}, dir::String)
   @ Base.Filesystem ./file.jl:112
 [7] makedocs(components::CitationBibliography; debug::Bool, format::Documenter.Writers.HTMLWriter.HTML, kwargs::Base.Pairs{Symbol, Any, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:modules, :sitename, :pages), Tuple{Vector{Module}, String, Vector{Pair{String, String}}}}})
   @ Documenter ~/.julia/packages/Documenter/bYYzK/src/Documenter.jl:272
 [8] top-level scope
   @ ~/Repositories/Julia/Manopt.jl/docs/make2.jl:11
in expression starting at /Users/ronnber/Repositories/Julia/Manopt.jl/docs/make2.jl:11

The reduced makefile is submitted currently also on the corresponding branch, cf. https://github.com/JuliaManifolds/Manopt.jl/blob/kellertuer/documentercitations/docs/make2.jl

and I am running with

  [e30172f5] Documenter v0.27.25
  [daee34ce] DocumenterCitations v1.0.0

And I am by now a bit lost how to get this to run, I can either not have working links (but a correct bibliography at the end) or error messages. Is there some magic I am missing?

At least some markdown syntax that might occur in a BibTeX entry is not properly escaped by the internal tex_to_markdown function:

julia> import DocumenterCitations: tex_to_markdown

julia> tex_to_markdown("arXiv: [1210.2145](https://arxiv.org/abs/1210.2145)")
"arXiv: [1210.2145](https://arxiv.org/abs/1210.2145)"

People may have .bib files with entries like the above because at some point I speculated on this being a workaround to deal with the missing support for \url/\href. I was actually wrong about that even at the time: you would have been able to put html in the .bib file in the previous version of DocumenterCitations, not markdown. The fact that markdown worked was exploiting a bug even then (the same bug as now).

The correct and only officially suppored way to define a link in a BibTeX entry (as of now) is to use LaTeX syntax:

julia> tex_to_markdown("arXiv: \\href{https://arxiv.org/abs/1210.2145}{1210.2145}")
"arXiv: [1210.2145](https://arxiv.org/abs/1210.2145)"

Specifically for preprint links, it would be recommended to use the eprint field, which is fully supported. See the documentation for details.

I'm not going to go out of my way to fix this bug. First of all, I'm not completely sure how to escape arbitrary markdown for the stdlib markdown parser (I'm not even sure it's always possible)

Presumably, for the above string it would be sufficient to escape the brackets:

julia> import Markdown

julia> Markdown.parse("arXiv: \\[1210.2145\\](https://arxiv.org/abs/1210.2145)")
  arXiv: [1210.2145](https://arxiv.org/abs/1210.2145)

Second, it seems unnecessary to break existing .bib files that use the "workaround".

But, for the record: Having markdown syntax in the .bib file is officially exploiting an bug in DocumenterCitations. I strongly discourage the workaround, and I make no guarantees that it will continue to work. In particular, as soon as someone files a bug report that is due to a failure to escape brackets, I'll have to do something about this.

In my bibliography, I have some authors that are not a person's name, but some corporation.
Example:

@Misc{OEIS,
  bibkey        = {OEIS},
  author        = {OEIS Foundation Inc.},
  title         = {The {O}n-{L}ine {E}ncyclopedia of {I}nteger {S}equences},
  howpublished  = {Published electronically at https://oeis.org},
  year          = {2023},
  url           = {https://oeis.org}
}

Unfortunately, this name gets printed as O. F. Inc. in the bibliography due to the first and middle names being abbreviated.
Once Humans-of-Julia/BibInternal.jl#21 is released as BibInternal.jl v0.3.5, one can provide such names with double-braces, e.g. author = {{OEIS Foundation Inc.}} to the parser. It remains for DocumenterCitations.jl to remove one layer of braces before printing (and to bump the compat of BibInternal.jl).

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Consider my (little lazy) example

@techreport{deCasteljau:1959,
    AUTHOR      = {de Casteljau, Paul},
    institution   = {Enveloppe Soleau 40.040, Institute National de la Propriété Industrielle, Paris.},
    TITLE       = {Outillage methodes calcul},
    YEAR        = {1959}
}
@techreport{deCasteljau:1959b,
    AUTHOR      = {de Casteljau, Paul},
    INSTITUTION = {Enveloppe Soleau 40.040, Institute National de la Propriété Industrielle, Paris.},
    TITLE       = {Outillage methodes calcul - my lazy b},
    YEAR        = {1959}
}

and a temp.md.

[deCasteljau:1959](@cite), [deCasteljau:1959b](@cite)
```@bibliography
Pages = ["temp.md"]
Canonical=false
```

Then the resulting page looks like

So the generated key clashes (considering them like hashes). They should at least be [Cas59a], and [Cas59b] I think?

I might be mistaken, but I think the Keys would also respect the prefix of the name so that it would actually be [dCa69] – but maybe I remember that wrongly as well. The main part of this issue is the clash in keys, since this might also happen for other “Key collisions” I think (same author last names first letters in the same year for example).

PS: I promise this is the last issue for now, I even managed to rework my references and have a PR at https://manoptjl.org/previews/PR281/ that is (besides checking that non canonical bibliographies exist on all pages with references) reworked and I think it is overall quite nice! I just stumbled about this issue here by accident, so it does not appear in my references.

I followed this part of the documentation to try to get the list of references sorted in a name-year-title manner, but it seems that when using :numeric all cited references are still sorted in order of appearence, and only the non-cited references are then sorted. Is there a way to sort all references in a name-year-title manner, even when using :numeric? I also only use the *-type bibliography as I want to have one extensive list of references.

Transferred from JuliaQuantumControl/QuantumCitations.jl#11

This plugin is incompatible with the linkcheck option to makedocs.

This may or may not be fixed by #3

Via @odow: It would be nice if the bibliography block could provide backlinks to where the reference was cited in the document.

This would likely require a fourth pass:

1. Collect Citations
2. Render Bibliography
3. Expand citations to link to the items in the bibliography
4. Add backlinks to the rendered bibliography ←

As of #46, the "rendered" bibliography is not longer a raw HTML block, but a special markdown subtree, which is only converted to HTML (or LaTeX) during the final Writer stage. That would seem to make this feasible.

Consider the minimum example

using Documenter
using Documenter: doctest
using DocumenterCitations

doctest(Documenter)

# makedocs(
#    CitationBibliography(joinpath(@__DIR__, "bibliography.bib")),
#  ...
#

The returns the following error (but continues processing)

 Error: You are `using CitationBibligraphy`, but did not pass the plugin to `makedocs`

As far I can tell, there's no way to pass CitationBibliography to doctest. I am not sure what should be done in this case, but demoting the error to a warning seems like a good first step.

I'd prefer to write Quantum optimal control [BrumerShapiro2003](@cite)[BrifNJP2010](@cite)[Shapiro2012](@cite)[KochJPCM2016](@cite)[SolaAAMOP2018](@cite)[MorzhinRMS2019](@cite)[Wilhelm2003.10132](@cite)[KochEPJQT2022](@cite), which currently renders as "Quantum optimal control [1][2][3][4][5][6][7][8]", as the shorter
Quantum optimal control [BrumerShapiro2003, BrifNJP2010, Shapiro2012, KochJPCM2016, SolaAAMOP2018, MorzhinRMS2019, Wilhelm2003.10132, KochEPJQT2022](@cite) and have it render as the collapsed "Quantum optimal control [1-8]". Note that the rendered text has two separate links.

Parsing the citation syntax with multiple keys is already supported in the current master of DocumenterCitations:

julia> using DocumenterCitations: CitationLink

julia> using Markdown

julia> function _link(text::String)
           md = Markdown.parse(text)
           return md.content[1].content[1]
       end

julia> cit = CitationLink(_link("[BrumerShapiro2003, BrifNJP2010, Shapiro2012, KochJPCM2016, SolaAAMOP2018, MorzhinRMS2019, Wilhelm2003.10132, KochEPJQT2022](@cite)"))
CitationLink(
    link=Markdown.Link(Any["BrumerShapiro2003, BrifNJP2010, Shapiro2012, KochJPCM2016, SolaAAMOP2018, MorzhinRMS2019, Wilhelm2003.10132, KochEPJQT2022"], "@cite"),
    cmd=cite,
    style=nothing,
    keys=["BrumerShapiro2003", "BrifNJP2010", "Shapiro2012", "KochJPCM2016", "SolaAAMOP2018", "MorzhinRMS2019", "Wilhelm2003.10132", "KochEPJQT2022"],
    note=nothing,
    capitalize=false,
    starred=false,
    link_text=nothing
)

However, the feature is currently undocumented, since there is no (or at least no straightforward way) to render it with Documenter 0.27: right now, the format_citation function only modifies the link text of the citation link, but it doesn't break out of the structure of the single markdown link.

With the master version of Documenter, it should be possible to extend this: The format_citation for the above CitationLink should return an entire markdown snippet with multiple links, and that markdown snippet should then be parsed internally and inserted into the output tree.

How collapsed rendering would look with author-year styles is still to be determined. I'd mimic whatever RevTeX does as closely as possible.

@goerz asked for testers in another pull request here.

Everything seems to work fine, and I actually prefer the new style, but I need to edit my documents for the new style to look good, so maybe a different default should be considered.

release version of DocumenterCitations with default options

master branch of this repository

As I mentioned, I actually prefer what I can do with the new style, so I will be editing my documentation to use the new style. However, the new style does seem to break things if left on as default.

Congrats on the brand new release!
I just tried it with a bib file from https://zbib.org/, and I got a strange error due to a citation key with underscores rabiner_tutorial_1989:

┌ Error: Invalid bibtex key (nested markdown)
└ @ DocumenterCitations ~/.julia/packages/DocumenterCitations/ZXtrs/src/citations.jl:107
ERROR: Invalid citation: [rabiner*tutorial*1989](@cite)

Is it a Markdown thing or a DocumenterCitations thing?

Following up from #39 (comment)

If there are missing references (citations with no corresponding entries in a bibliography, e.g. due to a missing key in the .bib file), makedocs will show errors like this:

[ Info: ExpandCitations
┌ Warning: Bibliography anchor for key 'Mills2014b' not found.
└ @ DocumenterCitations ~/.julia/packages/DocumenterCitations/f0eDy/src/citations.jl:456
[ Info: CrossReferences: building cross-references.
┌ Error: invalid local link/image: file does not exist in src/PALEOcopse.md
│   link =
│    @ast MarkdownAST.Link("@cite", "") do
│      MarkdownAST.Text("Mills2014b")
│    end
│    
└ @ Documenter ~/.julia/packages/Documenter/nGNwI/src/utilities/utilities.jl:44

The second error is probably more confusing than helpful (and definitely redundant). It would be better if during the ExpandCitations step, citations without a corresponding bibliography entry would be transformed to something that's no longer a markdown link. That way, it wouldn't get picked up by the later CrossReferences step.

Maybe just keeping the "link text" in square brackets and making it red or something would do.