teal-language / tl Goto Github PK

In order to type-check real programs, we need to handle require().

Given a test.lua with this content:

return;

I get this:

1 syntax error:
test.lua:1:7: syntax error

This is legal Lua, so I expect it to be okay.

Will tl support something equivalent to TypeScript's declaration files?

In LÖVE, all of the API is exposed through a global love table. I would like to declare all the functions in this table so that I can call them in a type-safe manner.

I would like to do the same for existing Lua libraries as well (such as Penlight).

Support for using namespaced exported types was added in 42b7c1c (as opposed to requiring non-local types to be global). This needs to be documented in the tutorial!

The following tl code produces an error:

local love_graphics = record
	print: function(text: string, x: number, y: number): nil
end

Here's the error (I'm on the definition_files branch):

lua: lib/tl//tl.lua:1668: attempt to index a nil value (local 'val')
stack traceback:
        lib/tl//tl.lua:1668: in function <lib/tl//tl.lua:1645>
        (...tail calls...)
        lib/tl//tl.lua:1725: in function <lib/tl//tl.lua:1712>
        (...tail calls...)
        lib/tl//tl.lua:4730: in function 'tl.process'
        lib/tl/tl:71: in main chunk
        [C]: in ?

I think parameter names inside function declarations should be allowed. It would make writing declaration files a bit easier. Also, text editors would be able to parse declaration files and show parameter names in tooltips.

Not really sure I understand this issue, but I reduced it to a reasonable test case.

I was checking some real-world Lua code for which some require()d modules have type definitions, but I isolated this particular error to the Lua code only.

Code:

local methods = {};

function methods:method1()
        return { data = function () return 1, 2, 3 end }
end

function methods:method2()
        local one, two, three
        one, two, three = self:method1():data();
        return one, two, three
end

local a = setmetatable({}, { __index = methods });

print(a:method2())

Output of tl check:

========================================
2 errors:
t6.lua:9:7: variable is not being assigned a value
t6.lua:9:12: variable is not being assigned a value
========================================
4 unknown variables:
t6.lua:8:8: one
t6.lua:8:8: two
t6.lua:8:8: three
t6.lua:15:7: a.method2

The assignment is clearly happening (the script prints 1, 2, 3), so I'm not sure what triggers the errors.

Installed with luarocks --local install tl

cat bug.tl

local Env = record
globals: {string:Variable}
modules: {string:Type}
end

local TypeCheckOptions = record
lax: boolean
env: Env
result: Result
end

local something: TypeCheckOptions =

tl check bug.tl

/usr/bin/lua5.3: /usr/share/lua/5.3/tl.lua:4012: attempt to index a nil value (local 'last')
stack traceback:
/usr/share/lua/5.3/tl.lua:4012: in upvalue 'get_assignment_values'
/usr/share/lua/5.3/tl.lua:4143: in field 'after'
/usr/share/lua/5.3/tl.lua:1822: in function </usr/share/lua/5.3/tl.lua:1816>
(...tail calls...)
/usr/share/lua/5.3/tl.lua:1863: in upvalue 'recurse_node'
/usr/share/lua/5.3/tl.lua:4713: in function 'tl.type_check'
/usr/share/lua/5.3/tl.lua:4811: in function 'tl.process'
/usr/lib/luarocks/rocks-5.3/tl/0.1.0-1/bin/tl:115: in main chunk
[C]: in ?

Preserving line numbers is important so that stack traces produced by the Lua VM produce the same line numbers of the input .tl file when it runs the generated .lua file (which is the .tl file stripped of type annotations).

The implementation of tl.pretty_print_ast does not currently preserve the line numbers of the tokens in the input AST. The list of tokens parsed by tl.lex does preserve the line number of each token, and the the parser stores them as well.

pretty_print_ast needs to check the current line against the node being printed and insert extra linebreaks if it is "behind". It should also not insert linebreaks "of its own" if the input .tl file does not contain them (i.e. if the input that was lexed is a huge single line, the dumped AST output should be as well).

The function is not doing the right thing now: it adds linebreaks on its own and writes each whole construct in the after callback... it may be possible to fix this by starting to use the before callback, but some constructs may require tweaking the visitor function (for the type checker, I had to add before_statements as an "in-between" stage of the forin construct — this might be necessary for other constructs as well).

tl does not detect an error in this code:

local function abc(x: boolean)

end

abc("(✿◠‿◠)")
abc(123)
abc({})

Is this a bug?

The following tl code causes a syntax error:

local printfn: function(string, number, number): nil

If I remove the nil return value at the end, the error is gone.

Is this a bug?

Test cases for bug included here, change from pending to it to enable the test then run busted to test:

https://github.com/hishamhm/tl/blob/master/spec/operator/not_spec.lua

When converting a file from .lua to .tl, it is useful to tell apart things that were not typed yet from things that were typed and have errors.

I think it would be useful if the CLI supported some way to generate Lua scripts into a separate directory. For instance,

tl --outdir=build/ gen game/main.tl

...would generate

build/game/main.lua

That way, I would not have to pollute my source tree with .lua files. I could simply add the build directory to .gitignore.

One of the overloads of love.graphics.print expects a table that looks like this:

love.graphics.print({{1, 1, 1, 1}, "Hello", {1, 0, 0, 1}, " World"})

I tried to declare this function, but I get a syntax error:

global love_graphics = record
	print: function(coloredtext: {{number} or string})
end

global love = record
	graphics: love_graphics
end

Should this syntax for union types work?

local Color = record
   red: number
   green: number
   blue: number
end

local c: Color = {
  red = 2,
  green = 3,
  bleu = 4
}

The above is an error in strict tl (bleu is not blue), but is not an error in lax Lua mode because valid fields are nullable and extra fields can be ignored without breaking structural equivalence. However, when assigning from a table literal (and not a variable or expression), this is more likely a bug. Probably best to detect this and report a warning.

The scope of type variables is currently not correctly controlled. This can manifest as producing accidentally recursive types — the visible symptom is a stack overflow when attempting to print the type using show_type. (The problem is not with show_type, but with the construction of the Type object instead.)

Here is a minimal test case, triggered by type inference on {}:

local t = {}
for i, a in ipairs(t) do
   for j, b in ipairs(a) do
      print(i, j, "value: " .. b)
   end
end

My plan for fixing this is via a revamp of type variables, storing them organized by scope in the main symbol table st alongside variables and types, instead of their own typevars tables.

User has a half-baked X.d.tl for a module X.lua.
Issue: tl check X.d.tl only checks the definition file ignoring parts of X.lua not covered by X.d.tl.

As the result, tl check X.d.tl shows nothing and tl check X.lua shows everything as unknown.
But former should show those parts that aren't covered by definition file.

Maybe I'm asking something stupid but the project has no chat-room/community place so couldn't ask before writing here.

The definitions of standard_library in tl.tl are not complete.

I've been adding them on an "as-needed" basis, but all definitions for Lua 5.3 (which would be a good starting point for a tl standard library) are not there yet.

An easy way to help with development is to add more entries to that table with the missing functions. It shouldn't be hard to write down the types of most functions by following the examples of the ones that are already there.

I've got a machine where /usr/bin/lua is Lua 5.3.

tl$ readlink -f $( which lua )
/usr/bin/lua5.3
tl$ lua -v
Lua 5.3.3  Copyright (C) 1994-2016 Lua.org, PUC-Rio
tl$ ./tl
lua: .//tl.lua:1: module 'compat53.module' not found:
	no field package.preload['compat53.module']
	no file './/compat53/module.lua'
...etc

Running busted produces a lot of errors because there's no compat53 module, which makes sense since it's not needed on 5.3.

Some functions in LÖVE have multiple overloads. For instance, love.graphics.print.

Right now, it seems like tl only checks the last overload:

global love_graphics = record
	print: function(text: string, x: number, y: number, r: number, sx: number, sy: number, ox: number, oy: number, kx: number, ky:number)
	print: function(coloredtext: {any}, x: number, y: number, r: number, sx: number, sy: number, ox: number, oy: number, kx: number, ky:number)
end

global love = record
	graphics: love_graphics
end

require("love")

function love.draw()
	love.graphics.print("Hello lol", 100, 100)
end

main.tl:4:22: argument 1: got string "Hello lol", expected {any}

(In this specific example, perhaps a union type would've been better)

Adding this issue as a reminder. It would be good to run make and tl tl.tl in CI for pull requests (I'm most familiar with Travis, so that would be my go-to option).

A syntax error is thrown on this code, for the semicolon in the table:

local t = {
   foo = "bar";
}

Some environments like Love2D and OpenResty pre-declare globals, for which we would like to specify using declaration files. These, however, don't have a corresponding require() call, since they are preloaded.

From #28:

However, what if we want to declare the fields in a global created using Lua's C API? To achieve this, I think we'd have to use an empty .lua file:

application.d.tl:

global application = record
	quit: function()
end

application.lua:

-- Empty file; the `application` global is created in C code

main.tl:

require("application") -- Without the empty `application.lua` file, we'd get a 'module not found' error in the Lua code
application.quit()

This is a bit weird, but I don't know if there's a better way of handling this.

My reply:

this thought crossed my mind too. One solution would be something like:

tl -llove main.tl

This would pre-load love.d.tl when type checking, similar to -l in the lua interpreter.

This would not be too hard to implement, it would be very suitable as a first contribution to the project. A bonus feature would be to load a configuration file so that one wouldn't need to pass these via the CLI every time, using a tlconfig.lua similar to what TypeScript does with tsconfig.json.

(this may go into documentation later)
What are the key differences and similarities with Typed Lua developed by Andre which is now live in Titan and its forks?

Related: #24

This test currently fails:

   it("ok with not not", function()
      local tokens = tl.lex([[
         local z = true
         z = not not x
      ]])
      local _, ast = tl.parse_program(tokens)
      local errors = tl.type_check(ast)
      assert.same({}, errors)
   end)

There are several problems with the parsing of variadic functions (functions using ...):

• parse_argument_list is a bit too lenient currently, as it accepts ... in any position of the argument list instead of only at the end (in other words, accepts function foo(x, ..., y) end but shouldn't)
• parse_function_value does not set vararg = true if a vararg is present in the argument list (used to parse function foo(x, y, ...) end)
• parse_type does not set vararg = true if a vararg is present in the type list (used to parse local f: function(x:number, y:number, ...))

(The type checker also has unfinished work with variadic functions, but this issue is specific to the parsing stage.)

I suppose this syntax should work?

global function x()

end

Right now, I get the following error:

expected a local variable definition

I'm playing around with tl. Right now, I'm trying to create a simple Point class.

Here's what I came up with:

local Point = record
	x: number
	y: number
end

local PointMetatable: METATABLE = {
	__index = Point
}

local function Point_new(x: number, y: number): Point
	local self = setmetatable({}, PointMetatable) as Point

	self.x = x or 0
	self.y = y or 0

	return self
end

function Point.move(self: Point, dx: number, dy: number)
	self.x = self.x + dx 
	self.y = self.y + dy
end

local pt: Point = Point_new(1, 2)
pt:move(3, 4)

This appears to work. However, if I rename

local function Point_new

to

function Point.new

...then, the type checker throws an error when I try to call Point.new(1, 2), because Point is a type:

cannot index something that is not a record: type {x: number, y: number, new: function(number,number):Point, move: function(Point,number,number)}

Is there some way for me to define the constructor as Point.new ?

Also, is this a proper way of creating a class-like table in tl? Perhaps my code is not even supposed to work? :) (I understand that tl is an ongoing project, and that some features may not be fully implemented yet...)

You asked on twitter, but I don't use it nor mastodon, so I let myself answer here: my name idea is: "lute"! :)

Otherwise, please try to not go with TypedLua/TypeLua; this would be too similar to the previous project of this name in my eyes...

Good luck! :)

We need to support something like

-- foo.tl
local foo = {}

foo.IceCream = record
   flavor: string
end

function foo.get_ice_cream(): foo.IceCream
   return { flavor = "strawberry" }
end

return foo

and be able to use the type in other modules like

-- bar.tl
local bar = {}

local foo = require("foo")

function bar.print_ice_cream(ic: foo.IceCream)
   print("ice cream: " .. ic.flavor)
end

return bar

What's the license for code in this repo?

Currently TL just shows the plain list of "unknown variables".
It may be valuable to know if it is:

1. a toplevel variable
2. an argument of a toplevel function
3. inside a function.

Currently, the lexer does not support long strings ([[hello]], [===[hello]===]) and long comments (--[[hello]], --[===[hello]===]).

Since the implementation of those would be rather similar, I listed both here. The lexer needs to remember the number of = seen once it enters a long string or comment and only close it when it sees the correct number.

It would be nice if TL could prevent this kind of error somehow:

local function add(a: number, b: number): number
	return a + b
end

add() -- attempt to perform arithmetic on a nil value (local 'a')

Perhaps TL could implement TypeScript's syntax for optional parameters, and assume that non-optional parameters are required?

local function dumbadd(x: number, y?: number): number
    y = y or 0
    return x + y
end

dumbadd() -- error: missing argument 'x'

Allow global variables to be declared so they don't generate unknown variable errors.

An idea is to add

global name: type

as syntax, which creates a declaration added to the top-level st entry.

We should accept redeclaration of a name with an identical type, but report errors on attempts to redeclare a global with a different type.

The parser is too lenient on incorrect output and sometimes lets bad code go through which causes crashes/assertion errors in the type checker.

Adding parser tests with decent coverage to the test suite should deal with this.

For example, here:

f( ( string.gsub("hello, world", "world", "Earth") ) )

The extra parentheses around string.gsub ensure that f receives only one argument (instead of the the two arguments normally returned by string.gsub).

The tl parser is currently dismissing these parentheses as unnecessary, but that's not true in Lua code.

I think the method definition feature is a bit inconvenient here:

love.d.tl:

global love = record
	draw: function()
end

main.tl:

require("love")

-- There's a typo in the function name, but the type-checker does not throw an error
function love.draws()
	print("draw stuff...")
end

I suppose it would be better to throw an error in this case?

It would be nice to have Syntax highlighting for tl in various editors. I suspect turning on Lua syntax highlighting will cover most bases (works okay in Vim), but specialized keyword highlighting would be a welcomed quality-of-life improvement.

Not sure if this should be included as part of this repository or maintained in other repo(s).

Also not sure if syntax highlighting should be community-driven, although I assume @hishamhm probably would not want to maintain syntax files for editors they do not use.

Some common editors:

• Vim
• Emacs
• VSCode
• Atom
• Others?

Depends on #59.

Right now, it seems like I need to manually call tl gen <filename> for every tl file in my project.

While I can write a script to automate this, I suppose it would be nice if there was a built-in way of generating Lua files for multiple tl modules at once.

For instance, in TypeScript, we can specify the input files inside tsconfig.json and run tsc to compile the whole project.

Creating and testing edits to tl.tl feels a bit awkward because changing tl itself requires bootstrapping from a "working" version of tl It would be nice if some documentation could be added describing how one should edit tl.tl and regenerate tl.lua and/run the test suite in a single shell command.

As someone not particularly familiar with Lua or or the luarocks build system it was a bit difficult getting a development environment up and running for edits.

I think it would be nice if we could use a command-line argument to bundle multiple tl scripts as a single Lua file. TypeScript supports this by using the --outFile argument.

Right now, tl's source code is contained inside a single .tl file. If this feature gets implemented, then tl's source code could be split into multiple files. This would probably make the code a bit easier to browse.

Is this project related to typed lua, titan lang or pallene? Is it meant to be a more minimal version? Like Lua + typing vs titan/pallene opting to be "companion languages"?

I'm very interested in the project based on the talk, but I would like to see more about the intended features and design goals.

cont. #28
As def-files are supported, we need a definite place for external modules definitions. It must be expanded by community. The following options come to my mind:

1. separate git repo
2. upsteam (module's) repos

Hisham, maybe alternative ideas?

Can some, at least basic, docs be made. The notation, the new types introduced etc. Interest is growing, and making those available will allow people to help you test and find/squash bugs.

Currently the lexer does not support 'single quoted strings', only "double quoted strings"