mvonballmo / ts-earthli-markup-language Goto Github PK

• Enhance tag library to store the following for each tag:
  • Block vs. Inline
  • Preserve newlines
  • Needs end tag
• Add a list of error tokens to a high-level tag
• Make the validator that collects errors a high-level tokenizer wrapper around the formatter
• This level processes tokens from the LowLevelTokenizer in the following way:
  • Define a "default" tag (e.g. paragraph) to indicate how to handle text/newline before the first block tag appears
  • Text: add to stream
  • Newline: check tag properties to determine how to handle:
    • If preserve newlines: add to stream
    • Else if immediately after an open or close tag, discard
    • Else add to stream
  • Tag: collect attributes
    • attribute: add to tag; go to value-expectation mode
    • value: to to attribute-expectation mode
    • error: add to tag or attribute;
    • text or start tag: add current tag to stream, process anew
    • end tag: if expected, add tag to stream, add end tag to stream; else add error
  • EndTag: if expected, add to stream; else add error (out-of-order or no-end-tag-expected)

The formatter can use the following algorithm to process this stream:

For example, for lists:

• Text: add to current item
• Newline: store current item; create new item
• Inline start tag: add to current item
• Inline end tag: add to current item
• Block start tag: store current item; create new item
• Block end tag: store current item

We might be able to make this easier to process by fine-tuning the token output. Not sure it's necessary though. E.g. for HTML paragraphs:

• Text: Emit <p> if necessary, emit text
• Newline: emit <p> if necessary; emit </p>
• Inline start tag: If supported, emit <p> if necessary, emit
• Inline end tag: If supported, emit (don't have to worry about out-of-order)
• Block start tag: Emit </p> if necessary
• Block end tag: We'll never get this, because the other formatter handles it

The current text-processor includes support for truncating a text to a certain number of displayable characters. Tag text is not included in the count.

If the caller requests a 10-character truncation of the text <h>This is longer than 10 characters</h>, the the formatter will return <h>This is lo...</h>.

• For a request to truncate text t to n characters, return at most n characters of non-tag (visible) text
• Any open tags in the truncated texts should be closed
• If the text was truncated, then an ellipsis "..." should be included after the text and before any final closing tags