I need htmlparser2 because it's much less strict than parse5. It would allow us to parse any HTML content without any preprocessing based on the browser specification this e.g useful for tools like formatter etc...

Subject of the issue

The doctype node is currently more complex than what HTML supports.

Now that we have xast, there’s definitely no reason to be so complex.

Actual behaviour

There are name, public, and system fields which could have any string value, adding unnecessary complexity.

Expected behaviour

HTML allows the value of name to be case-insensitive html, no public, and an optional system of about:legacy-compat.

I propose removing name, public, and system, and thus making a doctype look like this:

{"type": "doctype"}

Other tools (specifically, hast-util-to-html) could have an option to add the doctype legacy string (SYSTEM "about:legacy-compat"), and maybe even have an option to use HTML instead of html.

The current hast uses VProperties (from virtual-dom) for attributes on elements.

As previously outlined in the readme:

VTree’s property format is built for speed, not for easy of use. Developers
need intimate knowledge of where things should go. Should className go on
node.properties or node.properties.attributes? (Answer: “To set
className on SVG elements, it's necessary to use [attributes]”).
That, and the hassle of using
a = node.properties.attributes; a.className = (a.className ? a.className + ' ' : '') + 'foo';
to add a class, simply sucks.

With hast, I’d like a single object (properties) instead of multiple (properties, properties.*, properties.attributes).

In addition, I’d like to support Array’s as values (in addition to string, number, or boolean) to reflect “space separated tokens” and “comma separated tokens”.

• space separated values: className, rel, httpEquiv, ping, sizes, sandbox, headers, sorted, accept-charset, for, autocomplete, autofill, itemtype, itemref, itemprop, accesskey, dropzone;
• comma separated values: srcset, accept, coords, cols, rows.

Most properties are space-separated, so that should be the default when compiling an Array property value, with a white-list for comma separated values.

Also, the style property should be an object mapping camelCased CSS properties, e.g., borderBottomWidth, to strings.

Type for Raw HTML

There's no type for raw node.

Problem

In TypeScript, when try to push raw type nodes (e.g. u('raw', 'some raw html')) on to a hast tree, you will get a complier error, because raw is not included in acceptable children types.

Expected behaviour

Should there be a Raw type declared publicly?

Unknown elements, such as SVG and MathML, but also ng-button will probably just be recognised as elements.

I’m not yet sure how self-closing elements are handled (e.g., in SVG)...

PostHTML looks very similar to hast and its plugins are to some extent reminiscent of mdast's attacher—transformer pattern.

The main difference seems to be that hast is going to use the same tree format as mdast and retext and will benefit from unist utils, but maybe there are some other notable differences.

Have you seen that project before and how can you compare it to hast?

Related to: wooorm/property-information#14

The sizes attribute is parsed differently depending on whether it is part of a link element or an img element. The pull request referenced above would change sizes to be commaOrSpaceSeparated which would be parsed here as splitting on commas first, and then on spaces after rejoining the array into a space separated string.

For the element:

<img
    sizes="(max-width: 600px) 100vw, 800px"
    src="example.png"
    srcset="example.png 400px, example.png 800px"
/>

the sizes attribute would be parsed and turned into the array

["(max-width:", "600px)", "100vw", "800px"]

instead of the expected

["(max-width: 600px) 100vw", "800px"]

I was thinking of changing the above referenced line to be:

result = result.indexOf(',') >= 0 ? commas(result) : spaces(result)

but that breaks existing tests since they seem to be handling the case of "comma AND space separated".

I'm also not sure if that change would cover all cases since it seems that the sizes grammar would allow for a solo <media-condition> <source-size-value> which shouldn't be split up

Initial checklist

• I read the support docs
• I read the contributing guide
• I agree to follow the code of conduct
• I searched issues and couldn’t find anything (or linked relevant results below)

Problem

It could be useful to have a special type for fragments.

A fragment node type could help some plugins which need to preserve some isolated data for itself or replace a node with an array of nodes. Imagine a component for this case, that results in multiple nodes and has no single 'root' element. The idea is similar to JSX's fragment.

The plugins I bear in mind:

• include — another file is imported but it has multiple nodes. It can be done with a splice() like below:

  parent.children.splice(index, 1, ...loadedRoot.children)

  but having a fragment node enables storing additional information (original properties, actual file location etc) in the node for possible future use.

• components

  There is no limitation for number of children for the <template> element, i.e. a web component could yield multiple children instead itself. If a component gets replaced (like in rehype-componets plugin), it might be useful to preserve old data for other plugins that work on data based on that plugin.

Solution

Having a following interface could be useful.

interface Fragment <: Parent {
  type: "fragment"
}

Alternatives

With rehype, nested root nodes effectively work as fragments. This was tested running the following example:

const tree = {
  type: 'root',
  children: [
    { type: 'text', value: 'outside nested root\n' },
    {
      type: 'element',
      tagName: 'div',
      children: [
        {
          type: 'root',
          children: [{
          type: 'element',
          tagName: 'span',
          children: [{ type: 'text', value: 'inside nested root' }],
        }]}
      ]
    }
  ],
}

const file = await rehype()
  .data('settings', { fragment: true })
  .stringify(tree)

console.log(String(file))

This code does not throw and yields the following HTML:

outside nested root
<div><span>inside nested root</span></div>

The proposal is actually made for a sanity in specific cases. I understand that this is not a part of HTML and is rather a common practice in frameworks like React, Vue etc. and could be rejected as well as #20. However, as it could be useful for some plugins and does not require a lot of effort in the implementation, I am bringing this to your consideration.

I'm new to hast so I might be misunderstanding this, but I think the "no" in <div hidden=no></div> in the following excerpt of readme.md should be in quotation marks:

The DOM is strict in reflecting HTML and hast is not.
Where the DOM treats <div hidden=no></div> as having a value of true and
<img width="yes"> as having a value of 0, these should be reflected as
'no' and 'yes', respectively, in hast.

Also, I'm confused about the meaning of the first sentence in this excerpt. If hast always reflects the written value of html and the DOM is flexible about it, shouldn't this sentence read "hast is strict in reflecting HTML and the DOM is not"? Thanks for any clarification

TL;DR

I’m thinking out loud. We need namespace information. I can think of three solutions. Not sure which is best.

Introduction

HTML has the concept of elements: things like <strong></strong> are normal elements. There’s a subcategory of “foreign elements”: those from MathML (mi) or from SVG (rect).

A practical example of why this information is needed is because of tag-name normalisation: in HTML, tag-names are case-insensitive. In SVG or MathML, they are not. And, unfortunately tag-names themselves cannot be used to detect whether an element is foreign or not, because there are elements which exist in multiple spaces. For example: var in HTML and MathML, and a in HTML and SVG.

Take the following code:

<!doctype html>
<title>Foreign elements in HTML</title>
<h1>HTML</h1>
<a href="#">HTML link</a>
<var>htmlVar</var>
<svg>
  <a href="#">SVG link</a>
  <span>SVG</span>
  <a href="#">SVG link</a>
</svg>
<math>
  <mi>mathMLVar</mi>
  <span>MathML</span>
  <mi>mathMLVar</mi>
</math>

When running the following script:

var length = document.all.length;
var index = -1;
var node;
while (++index < length) node = document.all[index], console.log([node.tagName, node.namespaceURI, node.textContent]);

Yields:

[Log] ["HTML", "http://www.w3.org/1999/xhtml", "Foreign elements in HTML↵HTML↵HTML link↵htmlVar↵↵ …G↵  SVG link↵↵↵  mathMLVar↵  MathML↵  mathMLVar↵↵"] (3)
[Log] ["HEAD", "http://www.w3.org/1999/xhtml", "Foreign elements in HTML↵"] (3)
[Log] ["TITLE", "http://www.w3.org/1999/xhtml", "Foreign elements in HTML"] (3)
[Log] ["BODY", "http://www.w3.org/1999/xhtml", "HTML↵HTML link↵htmlVar↵↵  SVG link↵  SVG↵  SVG link↵↵↵  mathMLVar↵  MathML↵  mathMLVar↵↵"] (3)
[Log] ["H1", "http://www.w3.org/1999/xhtml", "HTML"] (3)
[Log] ["A", "http://www.w3.org/1999/xhtml", "HTML link"] (3)
[Log] ["VAR", "http://www.w3.org/1999/xhtml", "htmlVar"] (3)
[Log] ["svg", "http://www.w3.org/2000/svg", "↵  SVG link↵  "] (3)
[Log] ["a", "http://www.w3.org/2000/svg", "SVG link"] (3)
[Log] ["SPAN", "http://www.w3.org/1999/xhtml", "SVG"] (3)
[Log] ["A", "http://www.w3.org/1999/xhtml", "SVG link"] (3)
[Log] ["math", "http://www.w3.org/1998/Math/MathML", "↵  mathMLVar↵  "] (3)
[Log] ["mi", "http://www.w3.org/1998/Math/MathML", "mathMLVar"] (3)
[Log] ["SPAN", "http://www.w3.org/1999/xhtml", "MathML"] (3)
[Log] ["MI", "http://www.w3.org/1999/xhtml", "mathMLVar"] (3)

Note 1: Non-foreign elements break out of their foreign context.
Note 2: HTML is case-insensitive (normalised to upper-case), foreign elements are case-sensitive.

Proposal

I propose either of the following:

• Add namespace on some nodes (notably, root, <mathml>, <svg>). To determine the namespace of a node, check its closest ancestor with a namespace.
• Add namespace on root nodes (and wrap <svg> and <mathml> in roots). To determine the namespace of a node, check its closest root for a namespace. This changes the semantics of roots somewhat.
• Add namespace on any element.

The downsides of the first two as that it’s hard to determine the namespace from an element in a syntax tree without ancestral getters. However, both make moving nodes around quite easy.
The latter is verbose, but does allow for easy access. However, it makes it easy for things to go wrong when shuffling nodes around.

Note: detecting namespaces upon creation (in rehype-parse), is very do-able. I’d like to make the usage of hastscript and transformers very easy too, though!

Do let me know about your thoughts on this!

Currently, there’s duplicate data in directives (processing instructions and declarations):

{
  "type": "directive",
  "name": "!doctype",
  "value": "!doctype html"
}

I’d like to propose the following instead:

{
  "type": "directive",
  "name": "!doctype",
  "value": "html"
}

But it’s breaking so it will have to wait a while.

Hi, why is <template> not handled as every other element? In the current state, we always have to handle it explicitly this results in code like

minify(tree)
visitor(ast, (node) => {
	if (node.tagName === 'template') {
		minify(node.content)
		...
	}
})

proposal

1. Handle it like any other elmement

I was just working on hast-include (similar to posthtml-include), to show the difference between PostHTML and hast, and ran into the problem of configuring void nodes (used to control whether or not an ending tag is expected) and open-close nodes (used to determine which opening tags implicitly imply the closing of other elements).

Now, plug-ins might want to configure these mappings, e.g., the include plug-in defines a custom include element which is a void element. It should be parsed and compiled (and maybe treated by other plug-ins?) as void.

To ensure this, I propose exposing the void and open-close dictionaries on either file or processor.
E.g., file.namespace('hast').void or processor.void.

The disadvantage of this is that, when patching file, users have to keep track of that (e.g., when running parse, run, and compile with different parsers). The disadvantage of patching processor is that currently, Parser and Compiler have no access to it (but that’s fixable of course)!

htmlparser2 supports a weird mix of XML and HTML, such as processing instructions and directives. HTML, the standard, does not. For example, processingInstructions, directives (other than doctypes), and cdata are not supported in HTML.

There’s a new branch up for rehype which uses parse5, a standard compliant HTML parser. It doesn’t support processing instructions or cdata. I’m going to remove support for those. Plus, I’m replacing the directive with the one allowed directive: a doctype node.

As doctypes have a name, public identifier, and system identifier, maybe those should be supported on the interface?

interface Doctype <: Node {
  type: "doctype";
  name: string?;
  public: string?;
  system: string?;
}

Where:

<!DOCTYPE html>

Yields:

{
  "type": "doctype",
  "name": "html",
  "public": null,
  "system": null
}