gomarkdown / markdown Goto Github PK

See https://kramdown.gettalong.org/quickref.html#block-attributes for the "official definition".

mmark supports this, but only when they are used before the block element, and the syntax is {...}.

{title="Blockquote title"}
> A nice blockquote

is supported and adds an attribute title, if something is done with it depends on the final presentation, i.e. maybe some javascript acts on this.

For this to work, html/renderNode() need to know it exists:

	case *ast.Del:
		r.outOneOf(w, entering, "<del>", "</del>")
	case *ast.BlockQuote:
		r.outOneOfCr(w, entering, "<blockquote>", "</blockquote>")

In this case <blockquote> need to have ID, classes and attributes applied. This function works on ast.Node, so ast.Node need to have a attribute. This leads to some pretty invasive change to ast.Node as it needs to carry them.

Is it possible to adjust the renderer (or parser, not sure) to convert double underscores to underlined text?

so that

__underline__

becomes

<u>underline</u>

basically im looking at discord for a reference here.
I know this is basically "non standard" markdown, but i kinda want underline support for a usecase of mine.

I've go a (very valid) feature request to allow multiple languages in a document mmarkdown/mmark#85.

I'm still toying with the implementation, but I think a 'language' option to the RenderOptions make sense. I can implement this for the xml render to see how it looks.

I know this code was originally forked from BlackFriday but at this point it seems more historical than any intention to merge upstream based on #89.

The reason removing the fork status is nice is because some features of GitHub don't work when you are a fork project. For example code search:

I have contacted GitHub support in the past to do this.

import (
	"github.com/gomarkdown/markdown"
	"github.com/gomarkdown/markdown/html"
)

	htmlFlags := html.CommonFlags | html.HardLineBreak
	opts := html.RendererOptions{Flags: htmlFlags}
	renderer := html.NewRenderer(opts)

Gives undefined: html.HardLineBreak error.

How can I fix it?

Hello,

A long time ago I forked blackfriday and created https://github.com/miekg/mmark.

This adds a whole bunch of new syntax elements, see https://github.com/miekg/mmark/wiki/Syntax

I was also waiting for the blackfriday v2 branch, which never happened, until I saw this repo.

Are you receptive to receive some of the changes we plan to make to gomarkdown?

See https://mmark.nl/plan for the current plan

A
:   This is A.

    b
    :   b1
 
    c
    :   c1
 
    d
    :   d1

Makes b part of the paragraph belonging to 'This is A.'. If you print the AST you get:

Document ''
    List ''
        ListItem ''
            Paragraph ''
                Text 'A'
        ListItem ''
            Paragraph ''
                Text 'This is A.'
            Paragraph ''
                Text 'b'
            List ''
                ListItem ''
                    Paragraph ''
                        Text 'b1'
                ListItem ''
                    Paragraph ''
                        Text 'c'
                ListItem ''
                    Paragraph ''
                        Text 'c1'
                ListItem ''
                    Paragraph ''
                        Text 'd'
                ListItem ''
                    Paragraph ''
                        Text 'd1'

For the html renderer if you have multiple classes, they will be returned as:
class="x" class="y", this is wrong. It should be class="x y".

If I define a new ast.Node in my own code and then run the html parser, it panic because it doesn't recognize the node type (as expected):

% ~/g/src/github.com/mmarkdown/mmark/mmark -html learninggo.md                                                                                                          ~/html/learninggo mmark2 +
panic: Unknown node *mast.References

goroutine 1 [running]:
github.com/mmarkdown/markdown/html.(*Renderer).RenderNode(0xc420116000, 0x575060, 0xc420114310, 0x576c00, 0xc4201142a0, 0xc4200bb901, 0x0)
...

An unknown node renderer hook would help here or add mast.Reference to the ast package.

I think an unknown hook renderer might be easier here.

mmark supports including new files with {{filename}} and code include (will be put in the code blocks) with <{{code.go}}.

This should still be a generic (i.e. not geared towards specifically the IETF) feature. So I'm wondering if you would accept a PR for it?

Markdown:

m := `ČĆĐŠŽ čćšž`

Rendered HTML:
<p>ČĆĐŠŽ čćšž</p>

Expected HTML:
<p>&#268;&#262;&#272;&Scaron;&#381; &#269;&#263;&scaron;&#382;</p>

Code:

package main

import (
	"fmt"
	"strings"

	"github.com/gomarkdown/markdown"
)

func main() {
	m := `ČĆĐŠŽ čćšž`
	md := []byte(m)
	var sb strings.Builder
	sb.Write(markdown.ToHTML(md, nil, nil))
	fmt.Print(sb.String())
}

Hi, I was looking at using the project to parse markdown, modify the AST somewhat and then render markdown bytes (back to a file). I couldn't immediately see a way to render markdown—are there facilities for this (any flavour)?

Hello, i checked out the extensions but can't figure out a way to parse a smartfract string to its html equivalent.

The table code always calls addBlock:

 func (p *Parser) table(data []byte) int {
       table := &ast.Table{}
       p.addBlock(table)
        i, columns := p.tableHeader(data)
        if i == 0 {
             p.tip = table.Parent
               ast.RemoveFromTree(table)
                return 0
        }

Now I'm using addBlock to add the attribute to the node; but because table sits before any list parsing all the attribute end up the empty table block. Interestingly this should be identical to:

 func (p *Parser) table(data []byte) int {
        i, columns := p.tableHeader(data)
        if i == 0 {
                return 0
        }
     table := &ast.Table{}
     p.addBlock(table)

probably because of side effects in tableHeader.

Anyhow it seems strange to add block and then remove it again.

See pandoc and mmark
pandoc: https://pandoc.org/MANUAL.html#numbered-example-lists
mmark: https://github.com/miekg/mmark/wiki/Syntax#example-lists

also ponder if we really need this.

Hello,

Can you please merge this PR: russross/blackfriday#412

It's a shame that such good PR's were ignored in the original repo without any comments or feedback.

PS: I assume you know the trick to merge any PR.. just add https://github.com/Vonng/blackfriday as a remote, and merge those commits from the v2 branch.

Copying the original author: @Vonng

Hi,

are you open to adding a non-blocking-space extension? See mmarkdown/mmark#80

Pandoc uses '\ ' for this, and other markdown parsers allow for this as well.

It's now a []byte, but we use it as a boolean.

Table footers are fairly standard in markdown. Mmark supports them, we should add support here as well. See https://github.com/miekg/mmark/wiki/Syntax#tables

Name    | Age
--------|-----:
Bob     | 27
Alice   | 23
======= | ====
Charlie | 4

shows a footer is indicted with the = character.

I'm looking to duplicate to the functionality of func (p *Parser) parseRefsToAST(). In this case because I also want to add references to the document. This function is hard to implement without access to p.addBlock, p.block.

We could either make this a function callback or something? Alternatively I can add a "fake" node to my tree, then walk the tree until I see that node and then add some children to it, and call Parse on each of them.

Captions work, but not for table IIRC - double check and if not implement.

Now the the client needs to do the include administration it needs to know when the current directory is not valid anymore. I.e. in the case of:

{{testdata/includes-includes}}

and again

{{testdata/includes-includes}}

Where testdata/includes-includes includes another file called testdata/includes-includes-includes;

Second level

When everything works, so should output:

./mmark -fragment test.md 
<t>first level</t>
<t>Second level</t>
<t>and again</t>
<t>first level</t>
<t>Second level</t>

Currently this shows:

2018/08/13 09:43:13 Failure to read /home/miek/g/src/github.com/mmarkdown/mmark/testdata/testdata/includes-includes: open /home/miek/g/src/github.com/mmarkdown/mmark/testdata/testdata/includes-includes: no such file or directory
<t>first level</t>
<t>Second level</t>
<t>and again</t>

because for the second include I'm setting the full path wrong, because I'm assuming I'm still in the included file. This is because I have no way of knowing p.block(data) has exited and i should revert back to the original include path.

With the following diff:

diff --git a/parser/block.go b/parser/block.go
index 3e8a59f..0a67a52 100644
--- a/parser/block.go
+++ b/parser/block.go
@@ -117,6 +117,7 @@ func (p *Parser) block(data []byte) {
                                included := f(path, address)
                                p.block(included)
                                data = data[consumed:]
+                               p.Opts.ReadIncludeFn("up", nil)
                                continue
                        }
                }

And my code reacting to this fake include file "up" by jumping back one level, everything works. But this means we would need another include function callback to signal, current file has been processed. Maybe some along the lines p.Opts.ReadIncludeDoneFn

Mmark adds a lot of new syntax, would be good to know what you would accept for PRs.

• asides (text prefixed with A>, like blockquote but generates aside in HTML5)
• toml title block - https://github.com/miekg/mmark/wiki/Syntax#title-block
• documents matters {mainmatter}, {backmatter} and {frontmatter} - https://github.com/miekg/mmark/wiki/Syntax#document-divisions
• Document parts probably drop this from mmark for the time begin
• Captions under figures, tables and code blocks (figures should be block level elements, so i can't be just images. I've use F> as a syntax for that.
• callout in code blocks - https://github.com/miekg/mmark/wiki/Syntax#callouts (requires block level attributes to enable
• Example lists - https://github.com/miekg/mmark/wiki/Syntax#example-lists
• Indices - https://github.com/miekg/mmark/wiki/Syntax#indices
• Citations - https://github.com/miekg/mmark/wiki/Syntax#citations
• Special header, those that start with .# name
• Short references (#id) skip the [] part.
• Includes

I can see you won't take everything :-) I would be nice to extend gomarkdown without having to fork it.

How does this do with github's flavor of markdown? Are there any known parts this doesn't support? https://github.github.com/gfm/

Hi, it looks like this repository failed to vgo get. (I expect you aware of vgo.)

It failed because of the pre-existing version tags from the russross/blackfriday.

And it seems like your repository diverged entirely, so the tags are not valid anymore.

Removing it will make your repository vgo gettable, if you like.

When fenced code block is matched inside quote block that's followed later by some non-quoted fenced code block, it is incorrectly parsed, matching end of fenced code block greedily.

Try parsing and then rendering this text:

Bug demo:

> quoted block 1 start
>
> ```
> fenced pre block 1
> ```
>
> quoted block 2 end

Paragraph

```
fenced pre block 2
```

> quoted block 2 start
>
> ```
> fenced pre block 3
> ```
>
> quoted block 2 end

It's incorrectly parsed/rended to this html:

<p>Bug demo:</p>

<blockquote>
<p>quoted block 1 start</p>

<pre><code>&gt; fenced pre block 1
&gt; ```
&gt;
&gt; quoted block 2 end

Paragraph

</code></pre>

<p>fenced pre block 2</p>

<pre><code>
quoted block 2 start

</code></pre>

<p>fenced pre block 3
&ldquo;`</p>

<p>quoted block 2 end</p>
</blockquote>

Please see this gist with complete code reproducing this output (also notice how github parser/renderer handles this case as expected).

I raised this on the blackfriday github as russross/blackfriday#535 but looks like this is a more actively maintained version of that codebase.

Any interest in a PR that converts text emojis such as :smile: to URLs such as https://twemoji.maxcdn.com/2/svg/1f604.svg?

This supports all of the github emojis, and probably a few more. The base for the URL is a well-known CDN hosting the twitter/temoji images.

Not sure how useful this actually is, but we have support for it in mmark:

Abbreviations
See https://michelf.ca/projects/php-markdown/extra/#abbr, any text defined by:

*[HTML]: Hyper Text Markup Language
Allows you to use HTML in the document and it will be expanded to <abbr title="Hyper Text Markup Language">HTML</abbr>. If you need text that looks like an abbreviation, but isn't, escape the colon:

*[HTML]\: HyperTextMarkupLanguage

Fix in xml and xml2 renderers, but needs fix in html as well.

Depends on some attribute helper function that need to be finished and upstreamed here as well.
See https://github.com/mmarkdown/mmark/blob/master/mast/nodes.go#L20

Initially reported at russross/blackfriday#536

I am working on a Hugo project that uses blackfriday internally to render its content.
This is the nested list that I am using:

## Project Setup and Run
1. Importing the project into Eclipse
   - Open Eclipse *-> Click* File *-> Click* Import *-> Click* General *-> Click* Existing Projects into Workspace. Follow on screen instructions.
   - Make sure all JARs and class folders on the build path are present for the project.
2. Set API Keys
   - Sign up for 
      - Google Maps JavaScript API key [here](https://accounts.google.com/ServiceLogin?passive=true&continue=https%3A%2F%2Fdevelopers.google.com%2Fmaps%2Fdocumentation%2Fjavascript%2F&service=ahsid#identifier)
      - Weather API- OpenWeatherMap key [here](https://home.openweathermap.org/users/sign_up)
      - Dark Sky API key [here](https://darksky.net/dev/register)
   - Set the above keys in `wetter\WebContent\resources\includes\api.jsp` file; lines 4,5,6.
     ```Java
     String api_key_GoogleMapsJavaScript   = "INSERT_YOUR_KEY";
     String api_key_OpenWeatherMap         = "INSERT_YOUR_KEY";
     String api_key_DarkSky                = "INSERT_YOUR_KEY";
     ```
3. Optionally set Proxy ie. `http.proxyHost` && `http.proxyPort` in files:
   - `wetter\WebContent\resources\includes\api.jsp`; line 24
4. Run the project on Apache Tomcat server.

I can not get this to render consistently due to the below reasons.

Fenced blocks within nested lists appear as inline code (when not preceded by empty new line)
The above sample produces this:

The only way to get the code block to render properly is to add a new blank line before the fenced code (and further intent it) which gives this:

But this hack leads to another issue.
The addition of the new line before the fence code in a list, surrounds the corresponding and all subsequent bullet contents in paragraph tags ie. <li><p>...</p></li> (even those which were earlier simply rendered as <li>...</li> ). Similar behaviour was also reported at gohugoio/hugo#5291.

This becomes prominent by the extra paragraph padding at the top level lists (2. Set API Keys...., 3. Optionally set Proxy.... ) in the screenshot above.
In all likelihood, I can at least made things look consistent by adding some extra css like

li > p {
  padding:0;
  margin:0
}

However, this doesn't address the underlying issue directly:

1. the need for fenced code block to be preceded by a new line in a list
2. stray addition of paragraph tags where not required

Expected Output:

<h2>Project Setup and Run</h2>
<ol>
  <li>
    Importing the project into Eclipse
    <ul>
      <li>Open Eclipse <em>-&gt; Click</em> File <em>-&gt; Click</em> Import <em>-&gt; Click</em> General <em>-&gt; Click</em> Existing Projects into Workspace. Follow on screen instructions.</li>
      <li>Make sure all JARs and class folders on the build path are present for the project.</li>
    </ul>
  </li>
  <li>
    Set API Keys
    <ul>
      <li>
        Sign up for
        <ul>
          <li>Google Maps JavaScript API key <a href="https://accounts.google.com/ServiceLogin?passive=true&amp;continue=https%3A%2F%2Fdevelopers.google.com%2Fmaps%2Fdocumentation%2Fjavascript%2F&amp;service=ahsid#identifier">here</a></li>
          <li>Weather API- OpenWeatherMap key <a href="https://home.openweathermap.org/users/sign_up">here</a></li>
          <li>Dark Sky API key <a href="https://darksky.net/dev/register">here</a></li>
        </ul>
      </li>
      <li>
        Set the above keys in <code>wetter\WebContent\resources\includes\api.jsp</code> file; lines 4,5,6.
        <pre><code class="language-Java">String api_key_GoogleMapsJavaScript   = &quot;INSERT_YOUR_KEY&quot;;
String api_key_OpenWeatherMap         = &quot;INSERT_YOUR_KEY&quot;;
String api_key_DarkSky                = &quot;INSERT_YOUR_KEY&quot;;
</code></pre>
      </li>
    </ul>
  </li>
  <li>
    Optionally set Proxy ie. <code>http.proxyHost</code> &amp;&amp; <code>http.proxyPort</code> in files:
    <ul>
      <li><code>wetter\WebContent\resources\includes\api.jsp</code>; line 24</li>
    </ul>
  </li>
  <li>Run the project on Apache Tomcat server.</li>
</ol>

Allow inline math to be specified with just $, just like in LaTeX

Hi, apologies in advance for posting to the wrong place.

I am using Hugo, am new to Go, and want to update mmark in hugo from 1.3.x to 2.0.x. Hitting a buzzsaw of compiler errors, as expected.

I am writing to ask some guidance. Should I be using this package, or mmark. Using mmark straight (just changing the repo to gomarkdown one) doesn't work as is:

import (
...
	"github.com/mmarkdown/mmark"
...
)

The compiler complains that that isn't a module. And looking at the mmark docs, I don't see how to include it in a Go program. So I think I should be using this repo, not the mmark one. Is that correct?

Any other tips?

Again, sorry, this isn't an issue but I'm not sure how else to reach out and ask questions.

Hello,

By default we can wrap Markdown code with inline tags.. so <span class="foo">_italics_</span> will render the internal Markdown.

But it does not work when wrapped with <div>, and that's a major limitation! That disallows us from setting specific classes for Markdown tables, etc.

The good thing is that div wrapping is allowed in CommonMark spec: russross/blackfriday#404 (comment)

As per http://spec.commonmark.org/0.18/#example-108, this is a valid way of div-wrapping Markdown text that should be rendered:

<div>

*Emphasized text*

</div>

The newlines after <div> and before </div> are needed.

Is this something that could be added?

This sort of works with Blackfriday but because of an undocumented ugly hack.. example.. it would be nice to get rid of those empty divs in the hack.

Many thanks.

1. a

    ~~~go
    c
    ~~~

### A

1. b

Gets this AST:

Document ''
    List ''
        ListItem ''
            Paragraph ''
                Text 'a'
            CodeBlock 'c\n'
            Heading ''
                Text 'A'
            List ''
                ListItem ''
                    Paragraph ''
                        Text 'b'

The heading is part of the list, while it should break the list. If I dedent the codeblock two spaces it's ok:

Document ''
    List ''
        ListItem ''
            Paragraph ''
                Text 'a'
    CodeBlock '  c\n'
    Heading ''
        Text 'A'
    List ''
        ListItem ''
            Paragraph ''
                Text 'b'

A blockquote doesn't show this behavior.

Hello.
How to disable "html entities" at markdown.ToHTML?

What's your stance and sharing code in renderers?

Things like:

• func (r *Renderer) ensureUniqueHeadingID(id string) string
• func blockAttrs(node ast.Node) []string
• isLetter(), isSpace (esp for supporting full UTF-8 instead of the ASCII subset)
• .... more?

I need these in my XML output as well (esp, because it so similar to html). My initial idea is just to fork the html/renderer.go code and make it output XML.

The functions that are not methods on *Renderer can probably be moved to another package, named text or render or something?

reverting e55c6b3 makes things not crash.

This is odd, because apparently p.tip is not set and everything depending on this being set, because no matter what (i.e. not table found, this would be executed).

Hey,

I'm maintain a project called Hugo. We are using both MMark, which in its current version uses this library, and Blackfriday; so, we're thinkering about what to do. I have looked at the changeset between this fork and BF2, and notice a performance degradation I think should be looked into:

go test -run="NONE" -bench="BenchmarkReference$" -count=3 -test.benchmem=true
▶ benchcmp -best 0.bench 1.bench
benchmark                old ns/op     new ns/op     delta
BenchmarkReference-4     1150301       1823330       +58.51%

benchmark                old allocs     new allocs     delta
BenchmarkReference-4     4370           5127           +17.32%

benchmark                old bytes     new bytes     delta
BenchmarkReference-4     1151378       647938        -43.72%

It looks like 91e18ae is the "problematic" commit. One could argue that Hugo is too concerned about this, and that may be true, but the numbers above indicate that getting rid of that linked list in favour for a, possibly, easier to read "Children slice", isn't worth it. And I think that will be even more clear when you start to take advantage of having an AST: Parse once, render many.

Needs a small blurp in the README.

It probably makes sense to have some kind of automated mechanism to sync patches from upstream. The code layout if a bit different, but this should be doable.

For all the mmark stuff I also tried to put as much into new files to avoid merge conflicts.

Sad to hear about BlackFriday. I use the package in a Markdown to PDF tool (mdtopdf). I filed an issue with BF on this same topic but noticed that the project seems to have died.

Based this excellent write up:
https://stackoverflow.com/questions/29255746/how-encode-rune-into-byte-using-utf8-in-golang

and my little experiment in playground:
https://play.golang.org/p/38X88KLr5aE

I believe this package will need to support "string" as input, instead of or in addition to "[]byte", in order to allow non-ASCII in Markdown. Using utf8.DecodeRuneInString() to ensure proper handling. See also:
https://blog.golang.org/strings

The start of footnotes are just rendered as:
<div class="footnotes">
there is no way to add something here as there is no ast.Footnote that gets rendering.

In Learning Go https://miek.nl/go I would like to start a new chapter, i.e. # Footnotes before generating the list.

So, I just discovered this project, alas after enduring some pain with blackfriday. First of all, I wanted to give some praise, this package does looks more elegant, great work :-)

I've been writing a renderer for the terminal over at https://github.com/MichaelMure/go-term-markdown, to be used for https://github.com/MichaelMure/git-bug. It currently use blackfriday but I think I'll migrate over this package.

Would you be interested in merging it here ? Or even to give me a hand (one can dream) ?

I need to actively ignore footnotes; so they don't show up in the document, but then also suppress the output footnote list at the end of the document.

This can be easily done with a NoFootnoteList option in the parser options.

error-tag:
:   access-denied

error-path:
:   Identifies the requested protocol operation.  The following example represents the
\<edit-config> protocol operation in the NETCONF base namespace:

    ```
    hallo
    ```

is not parsed correctly, because of the "" it at least shows up as . When using ~~~` it even ignores that:

error-tag:
:   access-denied

error-path:
:   Identifies the requested protocol operation.  The following example represents the
\<edit-config> protocol operation in the NETCONF base namespace:

    ~~~
    hallo
    ~~~

Document ''
    List ''
        ListItem ''
            Paragraph ''
                Text 'error-tag:'
        ListItem ''
            Paragraph ''
                Text 'access-denied'
        ListItem ''
            Paragraph ''
                Text 'error-path:'
        ListItem ''
            Paragraph ''
                Text 'Identifies the requested protocol ope...'
                Text '<'
                Text 'edit-config> protocol operation in th...'
                Code '\nhallo\n'
                Text ''

--- FAIL: TestListWithMalformedFencedCodeBlock (0.00s)
    helpers_test.go:72: 
        Input   ["1. one\n\n    ```\n    code\n\n2. two\n"]
        Expected["<ol>\n<li>one\n```\ncode\n2. two</li>\n</ol>\n"]
        Actual  ["<ol>\n<li><p>one\n```\ncode\n2. two</p></li>\n</ol>\n"]
    helpers_test.go:72: 
        Input   ["1. one\n\n    ```\n    - code\n\n2. two\n"]
        Expected["<ol>\n<li>one\n```\n- code\n2. two</li>\n</ol>\n"]
        Actual  ["<ol>\n<li><p>one\n```\n- code\n2. two</p></li>\n</ol>\n"]

Support sub- and superscript, from mmarks syntax doc:

For superscript use ^ and for subscripts use ~. For example:

H~2~O is a liquid. 2^10^ is 1024.
Inside a super- or subscript you must escape spaces. Thus, if you want the letter P with 'a cat' in subscripts, use P~a\ cat~, not P~a cat~.

% ./mmark -html ~/html/learninggo/learninggo.md                                                                                                          ~/g/src/github.com/mmarkdown/mmark master
2018/08/10 22:33:57 Failure to read /home/miek/g/src/github.com/mmarkdown/mmark/preface.md: open /home/miek/g/src/github.com/mmarkdown/mmark/preface.md: no such file or directory
2018/08/10 22:33:57 Failure to read /home/miek/g/src/github.com/mmarkdown/mmark/introduction.md: open /home/miek/g/src/github.com/mmarkdown/mmark/introduction.md: no such file or directory
<p>% Title = &quot;Learning Go&quot;

cwd should have been set to ~/html/learninggo but it isn't. Need to double check what happens here.

The support of the sequences =====, ***** and -----
for titles are not covered, right?

See at:
https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet

It would be nice if there were releases. I tried to package a downstream client and it was broken since the latest release by an API change.