When opening a weaved document with the console open, you can observe that the browser attempts to load a font for KaTeX. The weaver seems to be using relative URL's for these fonts, but they aren't anywhere to be found in this repository.

These references should be changed to use the fonts hosted by Cloudflare at https://cdnjs.com/libraries/KaTeX. With that being said, I'm wondering if that's actually the correct solution. The KaTeX CSS loaded from Cloudflare also includes some references to fonts, and there's some overlap between that CSS and the one embedded directly in weaved documents, so perhaps some of this CSS can simply be deleted? (And perhaps the same is true for the scripts?)

I'll likely look into this when I have more time, but figured I would point it out in case someone already knows more about why there's duplication between the embedded and external CSS/JS.

I've just tested out this tool.

When invoking lit -c test.lit with the following contents, no compilation occurs.

@code_type cpp .cpp
@Compiler lit test.lit && clang *.cpp
@error_format %s
@comment_type // %s

However, by removing the @error_format line, the above invocation DOES start the compilation process.

Hi,

I love your tool. I would like to integrate it into my personal blog. However, I don't know of a good way to do this. I can't really make a blog post because I am unable to include the proper front matter on the generated HTML/MD code. Additionally, Literate generates full HTML instead of just MD.

I suppose I would need two features in order to generate Markdown that could then be processed as a blog post:

1. Output Markdown file instead of HTML file
2. Allow the user to prepend a YAML front-matter to the resulting Markdown file

If you think this is an okay idea and have a suggested way to implement such a thing, I would be more than happy to take a crack at it.

Cheers,
Max

Hi,

Is the given Markdown compiler run section-by-section? I cannot get footnotes to work properly (using kramdown+GFM) unless I put the footnote body right under the paragraph, and then the footnote appears in the middle of the text. I would expect to be able to define the footnote wherever and have it appear at the bottom in the resulting HTML.

Cheers,
Max

I've been using literate together with python, and I've noticed that at the end of the files, literate outputs at least three newlines, when in the original script there isn't a single one specified, is there a reason for this? Or a way for literate to not output this extra newlines?

something like @{toc}

Line directives appear to always insert a space between the prefix given as a command line argument and the actual number. Mercury however requires line directives to be in format #<num> with no space in between. Perhaps support could be added for format strings like "#%d", so that it remains backwards compatible with directives lacking the %d.

I could not find a way to include the figure that is generated by a codeblock in the html file.

Here is a Windows build, in case one is interested:

lit_d09f0d1_win64.zip

As a counterpart to the noWeave block modifier it would be quite useful to have a noTangle modifier, for when you want code blocks to appear only in the documentation (e.g. examples, non-compiling code) but not in the actually generated files...

Hi, I'd like to customize the HTML output a bit, using jQuery-UI. I'm trying to add the following to a file:

<link rel="stylesheet" href="http://code.jquery.com/ui/1.12.0/themes/base/jquery-ui.css">
<script src="../scripts/jquery-3.1.0.min.js"></script>
<script src="../scripts/jquery-ui-1.12.0/jquery-ui.min.js"></script>

<script>
$( function() {
    $( document ).tooltip();
} );
alert("hola");
</script>

<style>
  label {
    display: inline-block;
    width: 5em;
  }
</style>

but this input is parsed by Literate, generating invalid HTML/JS code:

<p><link rel="stylesheet" href="http://code.jquery.com/ui/1.12.0/themes/base/jquery-ui.css">
<script src="../scripts/jquery-3.1.0.min.js"></script>
<script src="../scripts/jquery-ui-1.12.0/jquery-ui.min.js"></script>
</p>
<p><script>
<span class="math">( function() {
    </span>( document ).tooltip();
} );
alert("hola");
</script>
</p>
<p><style>
  label {
    display: inline-block;
    width: 5em;
  }
</style>
</p>

As it can be seen, it is adding the inline html into <p> paragraphs, and applying the pretty printer to the content of these paragraphs...

I don't know if this is a bug, regarding this claim in the documentation:

The markdown used by Literate is slightly different than normal markdown: underscores do not do anything, and you can directly inline html. 

But, if not, perhaps a good compromise is to implemenet a @raw-include directive that does not pretty print... or some kind of "verbatim" blocks in the file itself...

Thanks

What do you think about using this project: Tufte CSS as the base CSS setup? (Their Demo.)

I edited together a super rough view of what it would look like for Literate: on codepen. Just without syntax highlighting.

I can give a stab at adjusting the current HTML and CSS setup.

I think the main pro would be the clean margin notes that can hold some of the metadata that is currently below the code blocks. As well as a refined responsive design and print view.

I've also adapted their setup before for a basic static site generator, so I know my way around the markup (although it is minimal.)

Alternatively I can make an attempt at opening up the way markup is generated by Literate to allow for broader formatting changes?

I am having trouble including multiple files. Whenever I have multiple includes it gets into a loop of including the same file over and over, and complaining about redefinition. Each include works independently but not together. I have made the included files super simple to avoid error.

@title My title
  
@s Theory
@include theory.lit
 
@s Code
@include code.lit

It's barely detecting anything correctly, as shown below (not even comments). I'd much rather have no syntax highlighting at all than this garbage output (there doesn't appear to be an option for this?). The lang-scheme part is landing in the pre tag correctly, but the prettify code seems to be interpreting it incorrectly. The highlighting in vim is correct (I declared @code_type scheme .scm). I also tried lisp to no effect (except making the vim highlighting slightly worse).

Whoops, I misspelled 'these' in the screenshot, but I'm not going to go take it again just for that...

I am new to D but don't mind looking into helping out with this tool. Do you have any advice on getting started?

Not sure if this is implemented but one of the features from Org/Babel/noweb is indicating whether a block will be included in the weave or not (i.e. for Leaving repetitive code out of the HTML export)

If this isn't implemented I can take a look at getting it in.

Just a suggestion to mention noWeave in the manual. I was looking for such an option, but didn't find it until I looked through the source code.

I'm not sure how links are implemented. Thus, book.lit contains this part:

[Hello World](hello.lit)
[Word Count](wc.lit)
[Hangman](hangman.lit)
[Multiple Files](multiple_files.lit)
[Reference](reference.lit)

I would expect the text in the [] to be the text to be shown, and the text in the () to be the linked file. But instead, the text shown comes from the @title of the lit file. That is unexpected. Is this
on purpose?

Would it be possible to document links? Thanks!

The extractor should preserve diagnostic metadata so that stack traces point to the source markdown files instead of the extracted files.

When I run lit and look at the output I see

$ ./bin/lit --out-dir out examples/wc.lit
$ grep -n 'Header files to include' examples/wc.lit
37:@{Header files to include}
48:--- Header files to include
$ head -5 out/wc.c

/* wc.c */
/* Header files to include */
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>

but I would like head -6 out/wc.c to show something like

/* wc.c */
/* Header files to include */
#line 49 "examples/wc.lit"
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>

which uses the #line directive to specify the file and line for a section of generated code.

This is C preprocessor specific.
A solution that generalizes to other output languages might require a side-table of line numbers (maybe sourcemap format) that could feed into a language-specific post-processor that

• produces a JSR 000045 input for Java files
• fixes byte-code line mappings in .pyc files for python
• etc.

First, thanks for such a good work.
Would you consider adding a 'run' command to Literate, allowing to add the output of said command to the generated .html file?
Something like

@run echo "<br>some markdown or html string<br>"

I'm searching a tool allowing me to describe and test the syntax of my (humble and in devellopment) programming language in a simple manner.

Hope I'm clear.

Are you able to produce a Windows binary for this?

Suppose we have a program written in Literate, and some collaborators on that program who, for whatever reason, only want to work on the generated program source files. This could happen when working on a literate project in a large company, or when trying to port an existing project into Literate while it still receives patches.

We have ways of pointing compiler errors in the generated sources back to the literate sources. Is it feasible to similarly backport the output of diff on a generated source back to the literate sources?

For example, suppose we have a literate source L from which we generate a program source file S. We modify S and obtain S'. Running diff(S, S') gives us patches P_1, ..., P_n that we can apply to S with patch to obtain S'. Is it possible to "lift" the patches P_j to the literate source L to get patches P'_1, ..., P'_n that give us a literate source L' that generates S'?

In wonky math speak, generating the sources is a function f : L -> S and we have a function patch : S -> S'. Can we always lift it automatically to L -> L' such that the diagram

L --> L'
|     |
v     v
S --> S'

commutes?

If so, contributors or coworkers can remain ignorant of Literate and we can easily benefit from their contributions.

Might be nice if it also added table support:
pandoc.org/README.html#tables

I've been using the md-compiler branch, in order to be able to include latex equations. The issue I am having is that it looks like '' are removed from equations:

The following doesn't work

$$\frac{1}{2}$$

I can solve it by adding a '' as follows

$$\\frac{1}{2}$$

When lit is compiling as well as weaving, I get SIGSEGV.
Top of my file is:

@code_type c++ .cpp
@comment_type // %s
@Compiler lit -t main3.lit && g++ -c main3.cpp

It has compile errors. When I run lit -c, I get

'lit -c main3.lit' terminated by signal SIGSEGV (Address boundary error)

I think this is because the compiler outputs lines that lit can not process.
This is in main.lit:

        auto matches = matchFirst(line, r);

        string linenum = matches["linenum"];
        string fname = matches["filename"];
        string message = matches["message"];

I think one needs to first check if matches is empty. For example, if I condition the rest with

if( !matches.empty ) {

I don't get SIGSEGV, lit exits normally. I'm not sure if the best is if or assert.

I thought D won't have these annoying SIGSEGV!

2 $s between backticks in markdown on the same line results in an equation, which isn't really desired between backticks in markdown.

@s Unintentional equation

`$(FOO)` and `$(BAR)` are examples.

Results in:

...
<a name="1:1"><div class="section"><h4>1. Unintentional equation</h4></a>
<p><code>&lt;span class="math"&gt;(FOO)</code> and <code>&lt;/span&gt;(BAR)</code> are examples.
</p>
...

Where the <span/>s have been trapped inside <code/>s & are shown in the rendered HTML page like so:

All I get is an ugly "@{blockname}" string :(

I'd like to configure this as a .lit parser in Gitea but it needs the compiler to print the output on stdout instead of a file.

With the "Hello World in C" example in the Literate manual, if I add an @code_type declaration for many languages (ex., 'python', 'prolog', 'lisp'), I receive the following trace:

core.exception.RangeError@source/weaver.d(284): Range violation
----------------
??:? _d_arraybounds [0x56dfa3]
??:? weaver.__array [0x561873]
??:? immutable(char)[] weaver.weaveChapter(parser.Chapter, parser.Program, immutable(char)[][immutable(char)[]], immutable(char)[][][immutable(char)[]], immutable(char)[][][immutable(char)[]], immutable(char)[][][immutable(char)[]]) [0x55a8e9]
??:? void weaver.weave(parser.Program) [0x558f6f]
??:? void main.lit(immutable(char)[], immutable(char)[]) [0x50ae91]
??:? _Dmain [0x50c5b4]
??:? _D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ6runAllMFZ9__lambda1MFZv [0x5700b2]
??:? void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).tryExec(scope void delegate()) [0x570008]
??:? void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll() [0x57006e]
??:? void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).tryExec(scope void delegate()) [0x570008]
??:? _d_run_main [0x56ff65]
??:? main [0x50c6fb]
??:? __libc_start_main [0x782a8f44]

This does not happen with an @code_type of c, and indeed the program weaves fine that way.

When using the @compiler and @error_format directive, the error output of the used compiler is not parsed.

Build

Build from HEAD (6a9a3af) using ldc2 on MacOS/arm64

Observed Behavior

When running lit -c hello.lit on

@code_type lua .lua
@comment_type -- %s
@compiler lit hello.lit && luacheck --formatter plain hello.lua
@error_format %f:%l:%s: %m

@title Hello World

@s Hello world program

--- hello.lua
local msg = "Hello World"

@{print msg}
---

@s printing

--- print msg
print(msg)
end
---

the output remains

lit hello.lit && luacheck --formatter plain hello.lua
hello.lua:6:1: expected <eof> near 'end'

which is the unparsed output of hte luacheck command.

Expected Behavior

Parsing and translating the filename and line number in the output

lit hello.lit && luacheck --formatter plain hello.lua
hello.lit:20:1: expected <eof> near 'end'

Most opensource projects nowadays have a README in markdown format. It would be great to be able to have Literate generate this README for me.

To achieve this I need Literate to output markdown.

Although I agree that a README should not contain all source code, it would be nice to be able to reference certain code blocks in the README. Usage examples and performing project related tasks.

Also, if Literate outputs markdown, it would play nicer with other tools like pandoc and many other tools that allow markdown as input.

Maybe it's already possible to do this but I haven't been able to get it to work.

When using an external markdown program, some words in the resulting html are concatenated, meaning "some word" becomes "someword". I am using cmark as markdown program.

The bug appears if the markdown compiler does not produce a space at the end of a line.

The fix is easy: in weaver.d append a space on every line read. Patch appended.

diff --git a/lit/weaver.lit b/lit/weaver.lit
index 0c8630f..bb058c4 100644
--- a/lit/weaver.lit
+++ b/lit/weaver.lit
@@ -128,7 +128,7 @@ if (useMdCompiler) {
     pipes.stdin.close();
     auto status = wait(pipes.pid);
     string mdCompilerOutput;
-    foreach (line; pipes.stdout.byLine) mdCompilerOutput ~= line.idup;
+    foreach (line; pipes.stdout.byLine) mdCompilerOutput ~= line.idup ~ " ";
     if (status != 0) {
         warn(p.file, 1, "Custom markdown compilation failed: " ~ mdCompilerOutput ~ " -- Falling back to built-in markdown compiler");
         html = filterMarkdown(md, MarkdownFlags.disableUnderscoreEmphasis);
@@ -373,7 +373,7 @@ if (useMdCompiler) {
     pipes.stdin.close();
     auto status = wait(pipes.pid);
     string mdCompilerOutput;
-    foreach (line; pipes.stdout.byLine) mdCompilerOutput ~= line.idup;
+    foreach (line; pipes.stdout.byLine) mdCompilerOutput ~= line.idup ~ " ";
     if (status != 0) {
         warn(c.file, 1, "Custom markdown compilation failed: " ~ mdCompilerOutput ~ " -- Falling back to built-in markdown compiler");
         html = filterMarkdown(md, MarkdownFlags.disableUnderscoreEmphasis);

As seen in the screenshot below, the parser thinks that the bits in between the single quote marks is a string. In fact, this is a polymorphic data structure with type argument 'a.

I declared @code_type ocaml ml and also tried caml, sml, etc — but none helped the problem.

a lot of MD processors have syntax highlighting. I am unsure if Literate provides this. If not, can a means to pass a syntax highlighting hint please be added?

It would be nice if the output file specification for a code block could include a sub directory which would be created while tangling. It would be handy for languages like Java where the sources have to be in a directory tree.

Example

--- src/main/java/com/acme/Hello.java
CODE
---

It would be great if there was some way of "piping" code output to html. Currently I do this by writing to a file in an assets directory and then including that file.

One simple approach that would already help is to have a way to generate unique filenames, so that you can save to @{uuid 1} in the code and later include @{uuid 1} or a similar approach.

Is there a reason for @s generating headers exclusively with <h4>? As far as I get from the documentation, the only way to have proper (but unnumbered) subsections is using markdown's # style headers... but I need to use <h5> which may be too small...

As for numbered headers using markdown, one could use --md-compiler 'pandoc --number-section', but still, you require at least one @s in order to render code blocks, and means rendering an <h4>...

I guess I would be happy with either:

• configurable @s depth, either inline or via cli
• not requiring @s for rendering code

Thanks for this cool project, hope you don't get tired of my reports ;)

The main file:

@code_type clojure .clj
@comment_type ;; %s

@title Hello Clojure

@s Section

--- example.clj
(ns user)
@{a function}
@{main}

---

@s The meat and potatoes!

--- a function
(defn hello [hi]
(println hi))

---

@s Some other thing!

--- main
(defn -main [& args])

---

@include hello_java.lit

hello_java.lit

@code_type java .java
@comment_type // %s

@title Name

@s Introduction

--- name.java
// Hello World, World.

---

HTML output:

The included hello_java.lit file ends up being between sections Another and Some other thing!, rather than right after Some other thing!.

I tested this by moving the @include statement below each of the sections and it always imports above the section it is below.

Anothing thing I noticed: the 1. Introduction anchor (from the included file) is the same as 1. Section, so it conflicts with linking to section 1.

Hello,

I am working with the latest version of Literate. References to code blocks that are contained in other code blocks are translated to clickable links in the html output file. But references that are contained in normal text are written verbatim as plain text.

Example:

@title Test

@s

--- Code Block 1
AAA
---

--- Code Block 2
@{Code Block 1}
---

@{Code Block 1}

According to the manual both cases should work the same way.

Is this a bug?

Best regards,
Manfred

I was going to attempt to make a contribution to Literate, but fell at the first hurdle. The mistake could well be mine, of course, but any help appreciated...

$ git clone [email protected]:zyedidia/Literate.git
Cloning into 'Literate'...
Enter passphrase for key '[redacted]':
remote: Enumerating objects: 1979, done.
remote: Total 1979 (delta 0), reused 0 (delta 0), pack-reused 1979
Receiving objects: 100% (1979/1979), 2.09 MiB | 3.51 MiB/s, done.
Resolving deltas: 100% (1323/1323), done.
$ cd Literate/
$ make --debug
GNU Make 4.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2014 Free Software Foundation, Inc.
Licence GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Reading makefiles...
Updating goal targets....
 File 'release' does not exist.
   File 'lit/markdown/source' does not exist.
  Must remake target 'lit/markdown/source'.
Submodule 'lit/markdown' (https://github.com/zyedidia/dmarkdown) registered for path 'lit/markdown'
Cloning into '/mnt/c/Users/nik/dev/Literate/lit/markdown'...
Submodule path 'lit/markdown': checked out '02b9b68e7f78e83e9f1e2d391263ad57ba988c3f'
  Successfully remade target file 'lit/markdown/source'.
   File 'd-files' does not exist.
     File 'bin/tangle' does not exist.
    Must remake target 'bin/tangle'.
dub --root=lit/tangle build
Performing "debug" build using /usr/bin/dmd for x86_64.
tangle ~master: building configuration "application"...
lit/tangle/src/util.d(55,41): Error: no property canFind for type Modifier[]
lit/tangle/src/util.d(55,89): Error: no property canFind for type Modifier[]
lit/tangle/src/util.d(70,24): Error: no property canFind for type Modifier[]
lit/tangle/src/util.d(78,31): Error: no property canFind for type Modifier[]
/usr/bin/dmd failed with exit code 1.
Makefile:11: recipe for target 'bin/tangle' failed
make: *** [bin/tangle] Error 2
$

E.g., using -w, -t and -c together cause no output to be written. Likewise, -w and -t (without the -c compiler option) still causes no output to be written.

@zyedidia: Could you make it so that I can nest @books if I can't do so already? What I'd like to do is have a top-level @book-directive file containing the top-level table of contents, then have lit pull in any tables of contents it finds in @book-directive files linked to from this top-level one imported in to that latter one, allowing me to string together nested tables of contents. For example:

In the top-level @book-directive file:

⋮

@s Table of Contents

[Reference to another `@book` directive file relative to this one](link-target)

In the referenced @book-directive file:

⋮

@s Table of Contents

[Reference to another `@book` directive file relative to this one](link-target)

Desired result in top-level @book-directive file:

⋮

# Table of Contents

<!--Top-level ToC entry-->
  <!--Next-level ToC entry pulled in from secondary file-->

Basically, I want to be able to string tables of contents together like that. This might be related to #7, methinks? It might be useful to do something similar with ToCs outside of @book-directive files if those ever exist.