https://github.com/markdown-it/markdown-it/blob/master/lib/helpers/normalize_reference.js

Probably, should be moved to utils.

I am trying to build a bunch of image renders which would look at the image URL and based on that, render an iframe for YouTube, Vimeo, Twitter, etc or just fallback to a default image. Since I want to keep each of these as separate plugins, I am trying to do it with inline.ruler.before as:

md.inline.ruler.before('emphasis', 'foobar', function() { alert('before emphasis'; }
md.render('![foo](http://foo)'); // shows alert

Some CommonMark 0.17 tests in the section "Emphasis and strong emphasis", notably those containing intraword underscores, fail with the current master.

One example is пристаням_стремятся_ — it should render as <p>пристаням_стремятся_</p> and not as <p>пристаням<em>стремятся</em></p>.

for example, i want to parse @twitter mentions, asynchronously check to make sure that the name is a valid name, then rewrite the url. is there a way to do this?

glancing at the source code, it looks like i would manipulate the tokens between parsing/rendering.

It's very popular request to add header ancors. Prior to do it, we need to discuss possible security problems and solutions.

Read first

• gjtorikian/html-pipeline#111 (comment) - discussion about github implementation
• http://www.slideshare.net/x00mario/in-the-dom-no-one-will-hear-you-scream - awesome presentation about dom clobbering

Problems:

1. id - collisions
2. name - dom clobbering
3. cross-conflicts when multiple docs on the same page have the same headers

Possible solutions

1. Do nothing
   • unsafe, you need to control content, or site will be vulnerable
2. Add prefix
   • will require js to keep references work
   • without js will make manual references typing not convenient
     • does anyone type such way?
     • not a problem for autogenerated tocs (we can add prefix to both anchors and refs)
3. Add per-doc unique prefix
   • not convenient in use. required in very limited cases

Need to discuss better solutions, and what to do by default, because anchors are really needed

current status

1. Must have not empty default prefix
   • options.anchorPrefix - instance default. env.anchorPrefix - every-time-render override
2. Open questions:
   • default prefix name?
     • - or -- of no objections (short and easy to type)
   • should we autofix local relative anchor links? any bad side effects?

Parse blew markdown, 3.0.5 use 0m20.459s, and 3.0.4 use 0m2.981s

[@addyosmani](https://github.com/addyosmani),
[@borismus](https://github.com/borismus),
[@carsonmcdonald](https://github.com/carsonmcdonald),
[@chriseppstein](https://github.com/chriseppstein),
[@danwrong](https://github.com/danwrong),
[@davidmaxwaterman](https://github.com/davidmaxwaterman),
[@desandro](https://github.com/desandro),
[@hemanth](https://github.com/hemanth),
[@isaacs](https://github.com/isaacs),
[@josh](https://github.com/josh),
[@jrburke](https://github.com/jrburke),
[@marcelombc](https://github.com/marcelombc),
[@marcooliveira](https://github.com/marcooliveira),
[@mklabs](https://github.com/mklabs),
[@MrDHat](https://github.com/MrDHat),
[@necolas](https://github.com/necolas),
[@paulirish](https://github.com/paulirish),
[@richo](https://github.com/richo),
[@rvagg](https://github.com/rvagg),
[@sindresorhus](https://github.com/sindresorhus),
[@SlexAxton](https://github.com/SlexAxton),
[@sstephenson](https://github.com/sstephenson),
[@svnlto](https://github.com/svnlto),
[@tomdale](https://github.com/tomdale),
[@uzquiano](https://github.com/uzquiano),

repeat codes:

var MarkdownIt = require('markdown-it');

    var md = new MarkdownIt({
      html: true,
      linkify: true,
    });
    var readme = fs.readFileSync(path.join(fixtures, 'eat-cpu.md'), 'utf8');
    md.render(readme);

Some important branches in significant rules like table are not covered well.

$make coverage

=============================== Coverage summary ===============================
Statements   : 97.18% ( 1897/1952 ), 3 ignored
Branches     : 93.62% ( 1100/1175 ), 4 ignored
Functions    : 100% ( 166/166 )
Lines        : 99.01% ( 1707/1724 )
================================================================================

1. Add missed tests, where it's significant.
2. Add /* istanbul ignore if */ to safety checks, that will never be executed.

Hi,

First of all, thanks for the great library!

I have a little question on configuration. I am trying to use the library to do only a couple of things (**bold**, [link](...), and auto insertion of <p> and <br/>). Could you please give me some direction of how to do that? For now I even fail to find a way of disabling rules via config hash, and don't know where to find a list of all available rules.

autolinker's just an implementation detail, so perhaps an abstraction of the options might be nice. i'm interested in:

• newWindow
• stripPrefix

Expected: https://github.com/IonDen/ion.rangeSlider/blob/master/readme.md#settings
Actually: http://spmjs.io/package/ion-rangeslider

Happened in markdown 2.x & 3.x

Seems you have a cli, but its not added to my PATH automatically after doing npm install -g markdown-it because your package.json is missing a "bin", I think.

console.log(
  require('markdown-it')({html: true}).render(
    "<!-- one-liner -->\n\n<!-- two-liner\n\n-->"
  )
)

The closing tag on the two-line comment is getting encoded:

<!-- one-liner -->
<!-- two-liner
<p>--&gt;</p>

Is this expected behavior or a bug?

// cc @rwaldron who opened https://github.com/npm/newww/issues/631

Could you explain how to use markdown-it with Jade include instead of marked?

include:md pages/home.md

I run: npm-install markdown-it -g

I expect: markdown-it CLI utility in PATH

I get: CLI ulitily at: /usr/local/lib/node_modules/markdown-it/bin/markdown-it.js instead

Hi,

Could you please make images support in "links" rule optional?

A config then might look something like this:

var md = new MarkdownIt('commonmark', {images: false});

md.configure({
  components: {
    core: {
      rules: [
        'block',
        'inline',
      ]
    },
    block: {
      rules: [
        'paragraph'
      ]
    },
    inline: {
      rules: [
        'links', // will not handle `![image](...)` because of `{images: false}`
        'text'
      ]
    }
  }
});

Table rule should add/remove columns when the current row's column count doesn't match the column count established by the first row.

var md = MarkdownIt();
console.log(md.render('Heading 1|Heading 2|Heading 3\n---|---|---\n1|2\n1|2|3|4\n'));

<table>
<thead>
<tr><th>Heading 1</th><th>Heading 2</th><th>Heading 3</th></tr>
</thead>
<tbody>
<tr><td>1</td><td>2</td></tr>
<tr><td>1</td><td>2</td><td>3</td><td>4</td></tr>
</tbody>
</table>

Here the table rows are 2 and 4 columns wide, whereas the header row has 3 columns.

UPDATE: GitHub allows less columns, but removes the extra columns.

Like the title says what is the diference between this two options in markdown-it or i understand wrong something cuz in the library exist a reference function and a link function.

Thanks.

example

Inline-level tokens can to be containers for text data. Should be sub_open + text + sub_close.

Since markdown is widely used by developers, it's useful to support code block. While, for those who have math background, math support can be equally helpful.
I ever extended from chjj/marked to buddys/marktex, which add math block. It's troublesome to maintain a repo by myself, shall I join you ?

Here is one of my blog containing many math blocks (once a time it's powered by buddys/marktex):
http://harttle.github.io/reading/introduction-to-algorithms.html

Add block flag to container tokens. Use the same logic for \n as in referece parcer.

Current normalizer is not correct. I had no time to do it right, because that's related to CommonMark spec and tests. Now all major problems solved, and it's time to care about details.

Reading:

• https://en.wikipedia.org/wiki/URL_normalization

Implementation candidate:

• https://github.com/defunctzombie/node-url (+4.7k gzipped, sorry)
• https://github.com/petkaantonov/urlparser/blob/master/src/urlparser.js - fast parser, but without normalizer

Example
One has to escape the underscores in the url but I don't think that this is the expected behavior.

1. It does not supports IDLs and non english letters in URL paths.
2. It supports only http/https schemas.
3. It contains HTML parser, not needed in our case (size, security).
4. It has some unpleasant not fixed bugs in tracker.

Probably, it worth to make a new package - could not find good alternate implementations.

Things to read about good autolinking:

• http://daringfireball.net/2009/11/liberal_regex_for_matching_urls
• http://stackoverflow.com/a/21925491/1031804

Design draft:

• https://gist.github.com/puzrin/ce95a25581a4d069e173

Other things to check:

• https://github.com/kevva/url-regex
• https://github.com/ljosa/urlize.js
• https://github.com/webmodules/urlregexp

URI schemas:

• http://www.iana.org/assignments/uri-schemes/uri-schemes.xhtml
• http://en.wikipedia.org/wiki/URI_scheme

This is occurring with 3.0.0 .. to my understanding of Markdown, entering an HTML tag will cause it to pass through to the output. This only seems to be true for standard HTML tags, whereas non-standard get encoded.

The behavior is visible here: https://markdown-it.github.io/

Entering <b>Hi there</b> and a bolded "Hi there" is displayed. Hence the standard tag is passed through unencoded as desired.

Entering <hello-world></hello-world> and instead that string is encoded with HTML entities so that it appears in the output as <hello-world></hello-world>. On the other hand <helloworld></helloworld> (no dashes in tag name) is passed through unencoded as desired.

• standard tag: passes through unencoded (as desired)
• non-standard tag with no dashes in tag name: passes through unencoded (as desired)
• non-standard tag with dashes in tag name: encoded to HTML entities (BUG)

With AkashaCMS I'm using a large quantity of non-standard tags and processing them with jQuery code to produce the output.

I'm initializing markdown-it with:

var md        = require('markdown-it')({
  html:         true,        // Enable html tags in source
  xhtmlOut:     false,        // Use '/' to close single tags (<br />)
  breaks:       false,        // Convert '\n' in paragraphs into <br>
  // langPrefix:   'language-',  // CSS language prefix for fenced blocks
  linkify:      true,        // Autoconvert url-like texts to links
  typographer:  true,         // Enable smartypants and other sweet transforms

  highlight: function (/*str, , lang*/) { return ''; }
});

var normalizeLink = require('./common/utils').normalizeLink;
console.log(normalizeLink('http://example.com/?q=%3f'));

Output: http://example.com/?q=%253f

var md = MarkdownIt();
console.log(md.render('[link](http://example.com/?q=%3f)'));

Output: <p><a href="http://example.com/?q=%253f">link</a></p>

when i try to use quotes like this:

md = """
'single'

"double"

'"double in single"'

"'single in double'"

"""
settings =
  typographer:  true
  quotes:       """«»()"""
MarkdownIt  = require 'markdown-it'
parser      = new MarkdownIt settings
console.log parser.render md

the result looks like this:

  <p>(single)</p>
  <p>«double»</p>
  <p>(&quot;double in single&quot;)</p>
  <p>«’single in double’»</p>

(Note that i used these specific replacements only in order to get a clearly discernible output).

i think this is problematic on more than one count.

First, how quotes are translated seems to be depending on whether they're enclosed in another pair of quotes. If there is any argument for this kind of behavior, it should be configurable.

Next, replacing a " in the source by a &quot entity (i.e. U+0022) may make sense in the context of an HTML attribute; in the context of an HTML text node, it's clutter.

Also, single quotes within double quotes do get replaced—but not by what was configured; rather, ’ characters (Right Single Quotation Marks, U+2019) appear in the target text. This may be justified by saying that some ASCII single quotes cannot be unambiguously interpreted as either opening or closing, and, hence, should be read as an apostrophe. Then again, the typogr module appears to get these cases right.

(As an aside, the configuration format—a single string with four characters (codepoints?)—is quite limiting and the documentation not very clear: "when typographer enabled, and smartquotes on"—where do i have to turn smartquotes on?).

Bad:

https://markdown-it.github.io/#md64=eyJzb3VyY2UiOiJhbiAqKig6Kio8YnI+IiwiZGVmYXVsdHMiOnsiaHRtbCI6ZmFsc2UsInhodG1sT3V0IjpmYWxzZSwiYnJlYWtzIjpmYWxzZSwibGFuZ1ByZWZpeCI6Imxhbmd1YWdlLSIsImxpbmtpZnkiOnRydWUsInR5cG9ncmFwaGVyIjp0cnVlLCJfaGlnaGxpZ2h0Ijp0cnVlLCJfc3RyaWN0Ijp0cnVlLCJfdmlldyI6InNyYyJ9fQ==

Reference

http://spec.commonmark.org/dingus.html?text=an%20**%28:**%3Cbr%3E

Probably, spec error, because <> are not punctuation chars.

example

First  | Second 
------ | ------
cell 1 | cell 2
qweqrt | find | grep "test"
qweqrt | find \| grep "test"
qweqrt | `find | grep "test"` second column
qweqrt | `find \| grep "test"` second column
qweqrt | `find | grep "test"` second column

There are no easy way to define | in table cells.

We want to replace marked with markdown-it in our project, but we're using an asynchronous syntax highlighting function:

https://github.com/npm/marky-markdown/blob/7317bc933c4c999c7f52b9f638a174d10b2a076e/lib/render.js#L5-L26

Is there a way to do this with markdown-it?

// @fengmk2 @puzrin

I'm working on something where users can use markdown. I'd like for all of the links to open in a new window. The following code works, but is this the best way to achieve this?

var md = new markdownit("default", {
    breaks: true, linkify: true, typographer: true
});
//Override link render to open all link in a new window
md.renderer.rules.link_open = (tokens, idx) => {
    var title = tokens[idx].title ? (' title="' + md.utils.escapeHtml(md.utils.replaceEntities(tokens[idx].title)) + '"') : '';
    return '<a href="' + md.utils.escapeHtml(tokens[idx].href) + '"' + title + ' target="_blank">';
};

I also want to allow fenced code to have a simpler syntax where you don't need to have the triple backticks on their own lines. Input example would be:

```for (var i = 0; i <= items.length; i++) {
  console.log(i);
}```

How can I achieve this? Right now I've got a hacky partial solution where it treats the language parameter as the first line of code, but the last line isn't seen.

//allow simpler fenced code
md.renderer.rules.fence = (tokens, idx, options) => {
    var token = tokens[idx];
    console.log(token)
    return '<pre>'
        + md.utils.escapeHtml(token.params+"\n" +token.content)
        + '</pre>\n';
};

I know this isn't a good solution, and it seems like md.renderer.rules.fence isn't the right place to do this sort of thing. How can I achieve this?

Preliminary suggested syntax is the same, as for comonmark fenced code blocks, but with different marker:

:::name params
markdown content
:::

More to read:

• http://talk.commonmark.org/t/what-could-a-spoiler-tag-extension-look-like/767
• http://talk.commonmark.org/t/block-directives/802

Well, 2.2.0 looks really good, and it's time to plan breaking things again :) . Next direction is simplification of plugins writing.

• consider adding postblock chain for references, footnotes & abbrs, or moving those rules at block level.
• Move footnotes, abbr, sub/sup, mark, ins to plugins
  • internals are expected to be stable, and we should not have problems with dependencies management
  • that's a good way to check that plugins system is consistent
• Split links to link + image.
• Review rule names and check if any should be changed
• Check if we could move strings normalization from ParserBlock to better place
  • it should be done on standalone .renderInline() call,but should not be repeated for each inline on .render().
  • pay attention to upcoming "unicode" changes in 0.14
• consider use .attr everywhere - that will be extendable.
• drop fences extention - not needed

PS. Please, don't post here things, not related to architecture change. Use separate tickets.

strike

Goals:

• unify renderers as much as possible
• simplify AST build
• unify attributes rendering

token:
  type: link_open # ? may be "link", but that can make pairs scan more complex.
  tag: a

  # join as -1 | 0 | 1 ?
  single: false # true for pairs
  open: true # opening / closing tag

  map: [ line_begin, line_end ] # may be sometime [ line_begin, line_end, col_begin ]

  # only static things, with standard processing (html escape)
  # may be with boolean attrs
  # (?) complex things like multiple classes - now as single string, words separated by space
  attrs:
    - [ href, data ]
    - [ target, _blank ]
    - [ title, 'titke text' ]
    - [ alt, 'alt text' ]
    - [ id, '#...' ]

  markup: string # optional, describes used markup (bullet char, emphasis marker, hr markup)

  # self-closing tokens (fence, code)
  content: unparsed content
  fence_offset: 0
  info: ''

  # (?) link, image only (or via attributes?)
  destination: "http://..."

  # heading only
  heading_level: 3

  # paragraph only
  tight: false # true - member of tight list, should be hidden

  # inline container
  children: [...] # inline tags 

This ticket is for core devs only.

• nesting checks not covered with tests
• seems those can be significantly simplified by moving to upper level.

I wrote a plugin for markdown-it which adds support for hashtags: markdown-it-hashtag. Right now it doesn't work like it should but I found out that one would just have to add

case 0x23/* # */:

to lib/rules_inline/text.js and all specs would succeed.

I'm looking at making a plugin for the wintersmith static site generator to use markdown-it rather than marked.

One feature wintersmith has is the ability to link to pre-build content, and have the link be rewritten while building the static content to be a correct link to the final location. It adds this functionality by patching marked.

Is this something that markdown-it would consider adding as a proper feature?

Reproducible in the demo at https://markdown-it.github.io/:

It appears rendering for ordered lists is also ignoring line breaks (visible above).

List of ideas with unclear added value. Probably, that will never be implemented.

Unify top level tokens attributes to increase monomorphism level

• doesn't help on practice (see mono branch)

Consider flexible attributes support

.attr property in tokens

• good for maruku syntax support (that should happen sometime)
• reduces number of changes, required by plugins
• can cause noticeable slowdown
• needs conflicts resolution with existing attributes and attributes merge

Use AST

• in theory, that can optimize some scans, but on practice will cause code complexity increase.
  • will require traverser
  • convenient use will be still impossible without selectors.
  • it seems more effective to simplify conversion to existing AST, instead of use custom ones.

Consider renderer unifiation & simplification of AST conversion

This can be usefull, but will break existing tokens format. Token should have all info to build AST with minimal efforts. Now it's not convenient to rewrite all rendered methods, and difficult to support new tokens, added by plugins.

It worth to reconsider tokens format, to use "default" methods for as many tokens as possible:

• need token name.
• need to calculate \n between tokens (block / inline).
• need to understand if token is "paired" or self-closing.
• need to dump custom attributes.

(!) can cause significant speed loss :(

That's internal memo. Please, post your questions to separate tickets.

Here is a jsfiddle http://jsfiddle.net/v6xcua5c/ that demonstrates the problem.

require('markdown-it')('zero').render('foo\nbar');

Reported here #12 (comment)

Hi, I was wondering if there is already an function (or AST visitor) that would give me an array with all text values (from the AST generated from markdown)?

For example if I have this controller

    md = require('markdown-it')();
    md_html = md.render(md_text);
    tokens = md.parse(md_text)
    params = { md_text: md_text, md_html: md_html, tokens: tokens}
    html = @jade_Service.renderJadeFile jade_Page, params
    @res.send html

which uses this jade view

extends ../_layouts/page_home.jade
block content
  #application
    .row
        .col-6
            .lge-container
                h3 Render Markdown PoC
    .row
        .col-6
              .lge-container
                  label Raw markdown:
                  form(action='', method='POST')
                    .input-group
                        textarea#text_md.form-control(type='text', placeholder='write markdown here', name='md_text')= md_text
                        span.input-group-btn
                            button#btn_render.btn-search-bar(type='submit') Render

        .col-3
            .lge-container
                label Rendered markdown:
                .input-group
                    p!= md_html
                    pre= md_html
                    if tokens
                        pre= tokens.json_Pretty()
                        for token in tokens
                            li
                                ul= token.json_Str()

to produce this HTML page (as a PoC)

As you can see on this (rough) page, I already have access to the AST of the original markdown text, which is great (including the fact that AST already has nice artefacts like the line number of the original text).

Looking at the AST I can see that if I transverse it with a recursive function, I will be able to extract all text values. My question is: Does that method already exist? (i.e. no need to re-invent the wheel)

note: I need the text values of the markdown since I'm creating a search engine which needs to look for specific keywords

There are no any documentation or tutorial on how to write syntax plugins for Markdown-it, neither in this repository nor in internet.

You already support table headers (<thead>) but it'd be nice to be able to add table footers too (<tfoot>).

Kramdown supports them: http://kramdown.gettalong.org/syntax.html#tables

Maybe something along those lines?

First Header  | Second Header
------------- | -------------
Content Cell  | Content Cell
Content Cell  | Content Cell
============= | =============
First Footer  | Second Footer

Code that reproduces the problem:

var MarkdownIt = require('markdown-it');
var pygmentize = require('pygmentize-bundled');

var md = new MarkdownIt({
  html: true,
  linkify: true,
  highlight: function(str, lang) {
    pygmentize({ lang: lang, format: 'html' }, str, function(err, result) {
      return !err ? result.toString() : '';
    });
  }
});

console.log(md.render('```python\n' +
'print("Hello world!")\n' +
'```'));

Expected output: HTML with pygmentize spans and classes inside a code block.
Actual output: Plain text inside a code block.

Pygmentize itself returns escaped HTML. Am I missing something here?

e.g.:

# h1

## h2

### [h3](/already/linky)

#### h4

##### h5

###### h6

# h1 foo bar 中文 no-ascii should be ok_-/

=>

<h1 id="h1">h1</h1>
<h2 id="h2">h2</h2>
<h3 id="h3-%2FFalready%2Flinky"><a href="/already/linky">h3</a></h3>
<h4 id="h4">h4</h4>
<h5 id="h5">h5</h5>
<h6 id="h6">h6</h6>
<h1 id="h1-foo-bar-%E4%B8%AD%E6%96%87-no-ascii-should-be-ok_-%2F">h1 foo bar 中文 no-ascii should be ok_-/</h1>

Should I implementation this with syntax extensions?

Canditades:

1. Decode entries via DOM, instead of havind big hash (saves ~8k gzipped).
2. Replace autolinker
3. Split rare features to external packages

Entity decode for browser:

var ta;

function decodeEntities(input) {
  ta = ta || document.createElement('textarea');
  ta.innerHTML = input;
  return y.value;
}

UPD . Please, note, this ticket is for memo, not for discussion. In teory, it's possible to reduse size from 30kb gzipped to 20kb gzipped, but this question has very low priority. Everyone who need to increase priority, should write something like this:

I have site [link] with total assets XXXkb gzipped , and difference in 10kb is significant for me

Such info can make sense for priority. Everything else will not make sense.

i'd like to do per-render options. is this what env is? ex. only enable html if i'm rendering it from an administrator.

Is there a way to use pandoc-citeproc? Or what would be the fastest way to implement citations based on a bibtex file?