munificent / craftinginterpreters Goto Github PK

possible changes

• why System.exit with 65?
• "parenthsized" typo
• "then do…” That" - should there be a period after the end-quote mark?
• "determine that from" - "that" refers to something in the previous paragraph so I had to do a double take to figure out what "that" was as my mind had removed that mental stack frame.
• the word "token" is used a few paragraphs before being defined. Unfortunately, I totally understand why. There's a chicken and egg situation here. Still, it felt odd.
• "identifier" used but I didn't understand what it meant. The word is also defined in "A Map of the Territory" but I'd forgotten what it might altogether. In fact, I'd forgotten I'd ever read it and only found it through Ctrl+F.
• "It works its" => "The scanner works its"
• "close cousins the reserved words" - maybe add comma after "cousins"

questions and comments

• The grayed out code with only the important line in black works really well. Very easy to read.
• This is out there. Is it possible to have an entirely unicode language? Like imagine all keywords and variables name and everything in Chinese. Is that just as easy nowadays as ASCII based? Think of the line character count if everyone used Chinese variable names!

Add a sidenote that mirrors the note about smushing in Game Programming Patterns.

Making modifies sources, so your next commit will include these unwanted changes.

$ make
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   site/representing-code.html
	modified:   site/scanning.html
	modified:   site/the-lox-language.html

no changes added to commit (use "git add" and/or "git commit -a")

https://youtu.be/_t8hpEKb4gk

The old rules apply. Rather than edit and pull request, I'll just point out some stuff with bullet points and let you decide which make sense. Much of what I say may be too picky so feel free to ignore any number of the bullet points.

possible errors

• "languages designers" => "language designers"
• "Then target backends" reads like a sentence fragment, consider "Target backends then"
• "YARV is A Bytecode Virtual Machine." - unusual casing in this sentence.
• "artifical" - typo.
• "virtual machine (“VM”)" - VM being in quotes just struck me as odd.

clarifications

• what's the difference between a VM and interpreter? I'm still not sure after reading.

Hi, loving the book so far. Quick suggestion. In the intro to section 3 you suggest that readers can use your interpreter while they read through the book:

A tutorial isn’t very fun if you can’t try the code out yourself. Alas, you don’t have a Lox interpreter yet, since you haven’t built one!

Fear not. You can use mine. [ link to this repo ]

I think it would be helpful to add setup instructions to the README here, I definitely don't know where to get started.

Minimize cascaded errors. Once a single error is found, the parser no longer really knows what's going on. It tries to get itself back on track and keep going, but if it gets confused, it may report a slew of ghost errors that don't indicate other real problems in the code. When the first error is fixed, they disappear, because they merely represent the parser's own confusion. These are annoying because they can scare the user into thinking their code is in a worst state than it is.

There is a typo in The Standard Library - 3.10.
Where you say "you need to decide value it evaluates to", you probably meant to say "you need to decide which value it evaluates to".

In the second IR section sidenote it mentions that in GCC the target backends operate on GIMPLE. They operate on RTL. GIMPLE is really the middle end IR that is later converted to RTL.

Hello!

I'm very excited about your book. Can you also write a small extension for exporting interpreted code to binary?

Thanks!

Loving the book so far, thank you!

I think I found a minor syntax error:

In the code snippet from: http://www.craftinginterpreters.com/representing-code.html#metaprogramming-the-trees

    defineAst(outputDir, "Expr", Arrays.asList(
      "Binary   : Expr left, Token operator, Expr right",
      "Grouping : Expr expression",
      "Literal  : Object value",
      "Unary    : Token operator, Expr right",
    ));

The comma after Unary causes (at least my javac 1.8.0_121) to emit

Error(24, 9) java: illegal start of expression

According to 3.4.1:

If either operand of + is a string, then the other operand is converted to a string (if it isn’t already one) and the results are concatenated.

But both interpreter currently say:

Operands must be two numbers or two strings.

"inflation in the jargon economy lead today" -> "inflation in the jargon economy led today"

"overhwelmed" -> "overwhelmed"

Section 1.3 gives a great description of bootstrapping. After the description of Lox in Chapter 3 I think an interesting challenge would be:

Is it possible to bootstrap Lox with one of the given Lox implementations?
Why or why not?

I spent some time actually trying this and it was a very interesting exercise in terms of discovering the limits of the language. As I hit limits I would alter the jlox implementation (via a builtin or language feature) to keep going. We'll see how far I get :-)

It's annoying that it reappears on every page.

The Scanner Class 4.4

while (!isAtEnd()) { // We are the beginning of the next lexeme.

Source code has correct comment :)
https://github.com/munificent/craftinginterpreters/blob/master/java/com/craftinginterpreters/lox/Scanner.java#L49

In section 3.9.5:

class Breakfast {
  init(meat, bread) {
    this.meat = bread;
    this.bread = bread;
  }

I believe you meant
this.meat = meat

Chapter 4, section 4.4, second code example:
// We are the beginning of the next lexeme. should be:
// We are at the beginning of the next lexeme.

It just seems weird to me that Lox treats !0 as false.

The assignment operator wasn't mentioned in the Scanning chapter. Either you should explain it in the chapter, or make it a challenge.

I'm trying to do challenge 1 at the end of chapter 3 and write a sample Lox program and run it using you example interpreter. I got the interpreter compile very easily with make as indicated in #8.

If I do a basic example with a class (Breakfast - just like your samples) it works fine and I get the expected output of Enjoy your ham and toast, Dear Reader but when I try and do inheritance (Brunch) I get the error Only instances have fields. [line 3]. I'm overriding both serve and init and in the overriding init I call super.init.

Working code:

class Breakfast {
  init(meat, bread) {
    this.meat = meat;
    this.bread = bread;
  }

  serve(who) {
    print "Enjoy your " + this.meat + " and " + this.bread + ", " + who;
  }
}

var baconAndToast = Breakfast("ham", "toast", "tea");
baconAndToast.serve("Dear Reader");

Code that errors:

class Breakfast {
  init(meat, bread) {
    this.meat = meat;
    this.bread = bread;
  }

  serve(who) {
    print "Enjoy your " + this.meat + " and " + this.bread + ", " + who;
  }
}

class Brunch < Breakfast {
  init(meat, bread, drink) {
    super.init(meat, bread);
    this.drink = drink;
  }

  serve(who) {
    print "Enjoy your " + this.meat + " and " + this.bread + " and " + this.drink + ", " + who;
  }
}

var baconAndToastAndTea = Brunch("ham", "toast", "tea");
baconAndToastAndTea.serve("Dear Reader");

Any help here would be much appreciated.

possible errors (if being really picky)

• copyright year is 2015, I forgot to point this out earlier
• "Ritchie had this idea called ..." - This might be nothing but the reader might have forgotten who "Ritchie" is, as he was only briefly mentioned in an earlier chapter:
  • "You don’t have to be the reincarnation of Dennis Ritchie"
• the term "Ref counting" isn't formally introduced. "Reference counting is and then "Ref counting" is used. This is me being super picky.
  • "There are two main techniques for managing memory: reference counting and tracing garbage collection (usually just called “garbage collection” or “GC”). Ref counters are much ..."
  • could say something like "reference (ref) counting" or something a bit better. I don't have a great solution here.
• The term "literal" is used. This is first used just once in "A Map of the Territory":
  • "Others may be several characters long, like numbers (123), string literals ("hi!"), and identifiers (min)."
    I don't believe "literal" is defined anywhere. I have similar feelings with other terms like "expressions" etc.. It might be worth having a glossary or something to that effect.
• "Functions are first-class in Lox" - I don't think "first-class" is defined.
• sometimes "first-class", sometimes "first class".

comments

• I love how the "Unified Theory of Garbage Collection" PDF name is "bacon-garbage".

Since it's checking for truthiness, not true-ness.

Great book so far!

I ran into the following when compiling the code:

make[1]: Entering directory `/home/craig/Downloads/craftinginterpreters-master'
java/com/craftinginterpreters/lox/Interpreter.java:42: error: ')' expected
    globals.define("clock", new NativeFunction(0, this::clock));
                                                      ^
java/com/craftinginterpreters/lox/Interpreter.java:42: error: ';' expected
    globals.define("clock", new NativeFunction(0, this::clock));
                                                       ^
java/com/craftinginterpreters/lox/Interpreter.java:42: error: not a statement
    globals.define("clock", new NativeFunction(0, this::clock));
                                                        ^
java/com/craftinginterpreters/lox/Interpreter.java:42: error: ';' expected
    globals.define("clock", new NativeFunction(0, this::clock));
                                                             ^
java/com/craftinginterpreters/lox/Interpreter.java:42: error: constructor NativeFunction in class NativeFunction cannot be applied to given types;
    globals.define("clock", new NativeFunction(0, this::clock));
                            ^
  required: int,JavaFunction
  found: int,Interpreter
  reason: actual argument Interpreter cannot be converted to JavaFunction by method invocation conversion
java/com/craftinginterpreters/lox/Resolver.java:294: error: method push in class Stack<E> cannot be applied to given types;
    scopes.push(new HashMap<>());
          ^
  required: Map<String,Boolean>
  found: HashMap<Object,Object>
  reason: actual argument HashMap<Object,Object> cannot be converted to Map<String,Boolean> by method invocation conversion
  where E is a type-variable:
    E extends Object declared in class Stack
6 errors
make[1]: *** [build/java/com/craftinginterpreters/lox/Callable.class] Error 1
make[1]: Leaving directory `/home/craig/Downloads/craftinginterpreters-master'
make: *** [jlox] Error 2

Pretty sure the answer is "You have an old Java version" but may want to state that the version needed is newer than 1.7.

Thanks!

The document is copyright 2015 but only has the first 4 chapters. The rest say "coming soon!" but that's obviously not true. Please do your readers a favor and include information about the actual status and progress of this book and when they can realistically expect to see the rest of it. As it is, reading this is a waste of time when so much of the document is missing. Also it's a waste of time to post issues about the content, which I had intended to do.

How about a wiki page for implementations in other languages? That would minimize PRs. With time the list will be too long for the README anyway.

" It treats all of your semicolons as meaningless whitespace unless it encounters a parse error."

I think that's supposed to be newlines.

Hey there. I'm a programming language enthusiast and I'm trying to understand how garbage collection works by reading your code. I'm new to C as well. So please excuse me for my ignorance and feel free to close the issue if it's moot.

I'm referring to the collectGarbage() (line 273 in memory.c).

From my understanding, vm.objects is a linked list of objects that are dynamically allocated. In line 273, when you free an object, I assume the next pointer gets freed as well. So when you traverse the linked list again, the objects after the freed object will not be accessed due to the dangling next pointer of the previous object (which is pointing to the freed object).

Since the next pointer of the previous object is not updated to point to the object after the freed object, I'm having a hard time understanding how this could work.

As I am new to C, my understanding may not be correct and there may be subtleties that I've missed. Please feel free to correct me otherwise!

In the Instantiation and Initialization subsection of the Classes section. I believe the line

this.meat = bread;

should be

this.meat = meat;

Can you please explain, what are those object types:

typedef enum {
//> Methods and Initializers not-yet
  OBJ_BOUND_METHOD,
//< Methods and Initializers not-yet
//> Classes and Instances not-yet
  OBJ_CLASS,
//< Classes and Instances not-yet
//> Closures not-yet
  OBJ_CLOSURE,
//< Closures not-yet
//> Calls and Functions not-yet
  OBJ_FUNCTION,
//< Calls and Functions not-yet
//> Classes and Instances not-yet
  OBJ_INSTANCE,
//< Classes and Instances not-yet
//> Calls and Functions not-yet
  OBJ_NATIVE,
//< Calls and Functions not-yet
  OBJ_STRING,
//> Closures not-yet
  OBJ_UPVALUE
//< Closures not-yet
} ObjType;

Source code

OBJ_STRING is string, OBJ_INSTANCE, OBJ_FUNCTION and OBJ_CLASS are understandable, but what are all other for?

It's a little weird to parse "--" and "++" when we don't have identifiers yet. The shift operators aren't super interesting, but at least they make sense.

In Chapter 3, section 3.9.5, in the third code block there is:

this.meat = bread;
this.bread = bread;

I believe the first line there should be this.meat = meat;

The java compiler can be run from within a Java program. Kinda cool thing to point out in a side note. Definitely a bad idea to actually do in the book, but a nice thing to point out in a side note.

I've never coded in Java but I've been able to follow along the code in the book and translate it to my language of choice (Xojo). Where I'm struggling (and I think others might too) is visualising the output of your helper tool GenerateAst as I can't figure out how to compile the visitor interfaces for the various Expression subclasses. Trying to visualise this:

private static void defineVisitor(
      PrintWriter writer, String baseName, List<String> types) {
    writer.println("  interface Visitor<R> {");

    for (String type : types) {
      String typeName = type.split(":")[0].trim();
      writer.println("    R visit" + typeName + baseName + "(" +
          typeName + " " + baseName.toLowerCase() + ");");
    }

    writer.println("  }");
  }

in my head hurts. Would it be possible to either provide a compiled binary of the GenerateAst tool for those of us who have never used Java or at least provide a few more code snippets of completed classes for easier visualisation?

Hi,

"we create finite a set of rules" -> "we create a finite set of rules"

In the chapter 6 challenge to add comma expressions it says

...(You will have to define a new syntax tree node for a comma expression too.)...

and this is how it’s done in the answers too.

Is this really true—isn’t it just another binary operator? In my solution I have used Expr.Binary with , as the operator and everything (as far as I can tell) works fine.

Looking at the history, this challenge was recently changed from implementing increment and decrement operators, which would need a new tree node (or at least changing the existing unary node).

So is the current language a remnant from the original challenge, or is the comma operator so special that it gets it own node whilst all the other binary operators have to share one between them?

In the side note of 6.3.2 it says The parser safely parses it but then reports it as an error instead of producting a syntax tree.

I believe the word producting here should be producing.

It's 2017, not 2015.

Hey, so I'm really enjoying reading this, but as I have very little access to the interwebz, getting all the libs for the python build system is a pain, and printing it from a browser looks like shit (and removes all the fun side-comments) I was wondering if you could add a PDF version? I don't care if it's not super-pretty, but just having a version I can dl easily and read on my ereader would be nice!

Thanks and keep up the awesome book!! 😺

"Again, nil sneaks in the back door." feels off.

See the next code (I've minimized the original to showcase the bug):

fun caller(g) {
	print "in caller: g before calling it: ";
	print g;

	g();

	print "in caller: g after calling it (should not be nil!): ";
	print g;
}

fun callCaller() {
	var capturedVar = nil;

	fun g() {
		// Commenting the next line out prevents the bug!
		capturedVar = 123;

		// Returning anything also fixes it, even nil:
		// return nil;

		// Or moving callCaller's code to the top level.
	}

	caller(g);
}

callCaller();

Running this with clox prints:

in caller: g before calling it: 
<fn 0x7fbb39c03e20>
in caller: g after calling it (should not be nil!): 
nil

jlox works as expected.

In chapter 3, when you introduce classes, you explain that we can pass objects to functions.
Wouldn't be worth it to talk about value semantics and reference semantics ? What happens if I pass an object to a function and that function calls a method that mutates the object ?

Maybe I'm a bit obssessed by these semantics (might be because of my C++ background) but it seems important to clarify this. At least, deciding what semantics are used by default, at best, using a paragraph to describe it

Especially given the use of the word "latter" in http://www.craftinginterpreters.com/the-lox-language.html#design-note, it seems the title should list Expressions & Statements in the same order as the text.

DESIGN NOTE: STATEMENTS AND EXPRESSIONS
Lox has both expressions and statements. Some languages omit the latter. Instead, they treat declarations and control flow constructs as expressions too.

very simple fix on the last line
(in JLox.java)

private static void run(String source) {
Scanner scanner = new Scanner(source);
List tokens = scanner.scanTokens();
Parser parser = new Parser(tokens);
Expr expression = parser.parse();
if (expression != null) interpreter.interpret(expression);
}

We need a production for cooked and pick "poached". That’s a nonterminal, so we add that. Now we’re back to the protein, so we add "eggs". We bounce back to breakfast and add "with". Now all that’s left is to pick a production for bread. We’ll pick "English muffin". That’s again a nonterminal, so we add that and we’re done:

Every single nonterminal should be terminal there.

In "The Lox Language", you have a code snippet where you say:
1 == 2; // true.
I thought you were joking for a minute, but the rest of the examples provided immediately afterwards are correct and don't have jokey answers.

Section 6.1 has a side-note that says:

Ignoring issues around floating-point roundoff and overflow,
it doesn’t really matter whether you treat multiplication as
left- or right-associative — you’ll get the same result either
way. Division definitely does matter.

While this is a true statement, maybe the wording could be stronger to say that such things should not be ignored in practice? Properly programming floating-point arithmetic takes a certain amount of care. It is easy to come wth "surprising" examples. Like this one in jlox:

$ ./jlox 
> print(0.1 + (0.2 + 0.3));
0.6
> print((0.1 + 0.2) + 0.3);
0.6000000000000001
>

This side-note also might not be a bad place to refer folks to What Every Computer Scientist Should Know About Floating-Point Arithmetic.

design-notes 1.2.5 :

"I know a lot of language hackers who whose careers are based on this."

design-note: what's in a name? :

"One the hardest challenges in writing this book was coming up with a name for the language it implements."

This line in chapter 2.3 may need adjustment:

So go has a compiler, is an interpreter, and is also a compiler.

Also, I am by no means an expert on this field (that's why I am reading the book!), but is go really an interpreter, or is it the OS/CPU which actually interprets the go generated executable. Whereby the command, go run acts as a wrapper to compile, then start the interpretation which is done by the OS/CPU?

Anyway, thanks again for another great book!

A glossary of the technical terms used throughout the book would be useful. E.G. I had to go back to find what the acronym AST meant as it was only defined in the foot/sidenotes of the page.

The sentence "In a fully compiled language, the code implementing the runtime gets inserted directly into the resulting executable." isn't what actually happens. Dynamic linking is used and space is saved.