clio-lang / clio Goto Github PK

An import pattern should be added to the lexer, rules should be written for parser and generator, and the necessary runtime (if needed) should be created.

• Add import pattern in core/patterns.js
• Add from pattern in core/patterns.js
• Add parser rules in core/clio.beef
• Add generator rules in core/generator.js
• Add required builtins and runtime in core/internals

Import statement should work with below patterns:

import * from url
import name1 name2 name3 from url
import url
import name1 as name2 from url
import url as name

url means (http|ws)s?://.*

Related: #66
Another approach would be to parse path|url when generating and not on parse time, this'll save us from writing duplicated parser rules.

For beef files, there's a vscode addon hosted here:
https://github.com/pouya-eghbali/bean/tree/master/vscode-beef

Main reason is bundle size
Some options:

• Self written parser
• Commander

Is your feature request related to a problem? Please describe.
clio build is sending the compiled files to .clio/target/node by default.

Describe the solution you'd like
Not considering the --destination parameter override, the output directory for the build command should be configured from clio.toml. An example toml config could look like this:

[build]
directory = "build"
target = "node"

[target.node]
directory = "src"
target = "node"

[target.browser]
directory = "src"
target = "browser"

Describe alternatives you've considered
We should create two fields on clio.toml:

• target: to give a hint to the compiler for what target is this build;
• build: as discussed on Telegram, to store in which directory to put the builds (default to build).

The resulting compiled JavaScript files should be placed in ${build}/${target}.

Additional context
The build command should have a --target override to build a specific directory to a specific target. This way, the programmer can generate a project for browser target, but can compile a host for node target using this feature.

ToDo

• Add fields to packageConfig
• Update template repositories
• Let command line flag override toml

I noticed that some CLI commands crash with an ugly error message when an incorrect filepath is provided. These crashes should be catched, handled and displayed as a human-friendly message:

• clio run
• clio new
• clio ast
• clio highlight
• clio compile

EDIT: @andreyluiz noted, that the CLI could need some color in general. This will be part of this issue

Describe the bug
When running clio build without a [target.<target>] section in toml, the build fails

To Reproduce
Steps to reproduce the behavior:

1. Generate a project with clio new foo
2. Change into the directory
3. Open up the toml manifest and remove the [target.node] section:
4. Run build
5. Admire error

Error: Cannot use 'in' operator to search for 'node' in undefined

Expected behavior
In case no target is specified, the default target field from the build section should be used as a target directory. The following should therefore be a valid toml config. Notice the missing [target.node] section.

title = "template-node"
description = ""
version = "0.1.0"
license = "MIT"
main = "src/main.clio"
authors = [ "Your Name <[email protected]>" ]
keywords = ""

[build]
directory = "build"
target = "node"

[scripts]
test = "No tests specified"

[dependencies]
stdlib = "latest"

Versions (please complete the following information):

• OS: macOS Catalina
• Clio: v0.2.0 dev (da7e238)
• Node: v12.13.1
• NPM: 6.13.4

Possible solutions

• No default value in package config

Additional Notes
Might be fixed with #98, wait for it to merge

I've been used to using doublequotes for strings, however they are not usable in clio yet. What do you think about implementing it? Are there any downsides?

Add installation instructions for the aur package of clio

Clio modules should be able to be hosted in a Docker container. A directory containing the project including the clio.toml file can be passed in. The project is hosted via clio host

EDIT after telegram discussion

We have agreed that hosting will either be possible through a built-in process manager (separate issue will be created), or via a docker container. The user should be able to write a Dockerfile that builds on top of the official Clio Dockerfile to host their files.

FROM clio-lang/clio:latest
COPY . .
EXPOSE 3000
CMD ["clio", "host", "index.clio"]

As discussed below, we need at least a language reference before we release the next version of Clio. We can use The Rust Book and The Swift Language Guide as a reference (pun intended).

Criteria

• The import system is documented (#66, #72, #67)
• Keywords are documented
• Data types are documented
• Expressions are documented (functions, conditionals, ranges...)
• Function decorators are documented
• Features of the new CLI are documented
• TOML fields are documented

Room for Discussion

• Should we switch from GitBook to mdBook?

src $ clio compile eg.clio out.js

/Users/johnleung/.nvm/versions/node/v10.14.1/lib/node_modules/clio-lang/index.js:30
  fs.writeFileSync(path, source);
  ^

ReferenceError: fs is not defined

eg.clio is just '2' -> print

Describe the bug

Pre-commit hooks do not run on free-bsd because #!/bin/bash is not available. https://github.com/observing/pre-commit could be forked to use a more generic #! like /usr/bin/env bash and then published as a new package. Another (probably simpler solution) would be to replace the pre-commit dep with a different one entirely

Things to check:

• Is /usr/bin/env always given on most operating systems?

To Reproduce
Steps to reproduce the behavior:

1. Run npm i
2. Make changes
3. Commit

Expected behavior
Pre-commits run smoothly

Versions (please complete the following information):

• OS: free-bsd
• Clio: 0.1.6
• Node: latest
• NPM: latest

Additional context
https://github.com/observing/pre-commit

Is your feature request related to a problem? Please describe.
Following up on #64, we should define a strategy to make the built JavaScript file from Clio source ready to run in the browser.

Describe the solution you'd like
clio build script should do some steps:
1 - Compile the Clio source into JS code (this is being done already);
2 - Install dependencies on the bundle directory (there will be a package.json there);
3 - Call an NPM script (declared in the package.json) to bundle the JavaScript files;

clio build should take a flag --no-bundle that should avoid step three from happening.

Just like clio build, clio run should cover all the 3 steps describe above, and execute the 4th step to make the compiled source available through a static HTTP server.

Describe alternatives you've considered
Inside the build directory (conventioned as build/browser), we should have a package.json with two scripts to bundle and host. We are considering using parcel to bundle because it is a zero-configuration bundler. For the HTTP I suggest http-server.

Additional context
Add any other context or screenshots about the feature request here.

Currently only arrays, numbers and strings are defined and are allowed in flows. In grammar it should be explicitly defined where they're allowed to appear (For example you can allow strings as function args, but not as data in a flow). These definitions are missing in some places, we need to find them and make necessary changes. This issue should be kept open until we have finished defining all of the types.

Describe the bug
An instance of IO is printed to stdout when running a compiled program. I can't exactly point out where it is printed yet.

To Reproduce
Steps to reproduce the behavior:

1. Create project using clio new
2. Compile the following program using the v0.2.0 dev version:

fn fib n: 
  if n < 2:
    "smaller than 2" -> print
  else:
    "larger or equal 2" -> print

2 -> fib -> print

3. Run node build/node/src/main.clio.js to execute the binary

Expected behavior
Just "larger or equal 2" should be printed.

Screenshots

Versions (please complete the following information):

• OS: macOS Catalina
• Clio: v0.2.0 (dev)
• Node: v12.13.1
• NPM: 6.13.4

Possible solutions

• Probably just a random console.log somewhere

clio_env and .clio-cache are in the working tree when a new project is created. That should not be the case.

We're currently using the shelljs library to generate a project via clio new. That breaks platform compatibility, as the shell syntax for UNIX and windows are quite different.

• Rewrite CLI commands to use FS instead of shelljs
• Rewrite tests to use functions provided by the cli instead of calling Clio commands via shell
• Review usage of file paths in CLI functions. Should accept absolute paths to simplify tests

We are currently able to install clio dependencies via clio deps add greeter. We should also be able to install npm dependencies this way. One way to do this is to add a flag to the command like so: clio deps add --npm express
If you come up with alternative ideas, please comment them on this issue.

How to do:
mapi, fold/reduce, map2, partition?

Hi!
I have read your Medium article about Clio Lang and I really loved it.
I'd love to contribute to the project, is there a contribution guide to follow?

Thank you!

To clean up the tests, we discussed to move them to a subdirectory of the subject to be tested.
Currently, the structure look like this:

- root/
-- module/
---- submodule/
------ file.js
-- tests/
---- submodule/
------ file.test.js

The new structure should look like the following:

- root/
-- module/
---- submodule/
------ file.js
------ tests/
-------- file.test.js

Room for discussion

• Should the testdirectories be called tests or __tests__? Both ways are conventional with Jest

The intro at https://blog.usejournal.com/introduction-to-clio-40dbbf9c250b requires a paid Medium subscription. I would rather pay for Clio, if it's any good.

Describe the bug

git deps get fails when fetching a Github URL

To Reproduce

Steps to reproduce the behavior:

1. Run clio deps add https://github.com/clio-lang/rethinkdb/archive/v2.3.3.zip
2. See error

{
  _: [ 'deps', 'add' ],
  '$0': '/usr/local/bin/clio',
  url: 'https://github.com/clio-lang/rethinkdb/archive/v2.3.3.zip'
}
Downloading https://github.com/clio-lang/rethinkdb/archive/v2.3.3.zip...
Added https://github.com/clio-lang/rethinkdb/archive/v2.3.3.zip to the dependencies list
FetchError: request to https://https//github.com/clio-lang/rethinkdb/archive/v2.3.3.zip/archive/master.zip failed, reason: getaddrinfo ENOTFOUND

3. No directory rethinkdb-2.3.3 in clio_env.

Expected behavior

We should find a directory rethinkdb-2.3.3 in clio_env and no error.

Versions (please complete the following information):

• OS: Mac OS X 10.15.2
• Clio: 0.1.6
• Node: v13.5.0
• NPM: 6.13.4

Possible solutions

• malformed URL

Is your feature request related to a problem? Please describe.
When a new project is created, it is not clear how to target it for the browser. There is no script for Clio language available on the generated project, also there are no instructions on how to do it.

Describe the solution you'd like
I would like to have a flag on the new command like --browser where I specify that this project is intended to run on the browser. If this file is declared, then the project should be generated with everything set up to start hacking with Clio on the browser.

Describe alternatives you've considered
I considered using https://github.com/Rich-Harris/degit to accomplish that. We would have two or three sample repositories with ready-to-start structures and using degit, we simply create the projects using the repositories as a base. Degit is pretty modular and language agnostic. It has an API and inits a new git repository, instead of a simple git clone.

Additional context
Clio new command should make the required directories (src, build, docs...).

I like Clio, but I realised that only language without minimal set of modules is road to nowhere. I believe that Clio could use NodeJS modules for the first time.

And my question: does Clio have any binding for JS? If it has, will it be possible to share example?

An import pattern should be added to the lexer, rules should be written for parser and generator, and the necessary runtime (if needed) should be created.

• Add import pattern in core/patterns.js
• Add from pattern in core/patterns.js
• Add parser rules in core/clio.beef
• Add generator rules in core/generator.js
• Add required builtins and runtime in core/internals

Import statement should work with below patterns:

import * from module
import name1 name2 name3 from module
import module
import name1 as name2 from module
import module as name

For beef files, there's a vscode addon hosted here:
https://github.com/pouya-eghbali/bean/tree/master/vscode-beef

Discussion:

We need to know how where the compile modules should be stored and how they should be compiled. There are several approaches to this:

1. Modules are compiled on publish: This will simplify requiring them in compiled code, but it'll make it impossible to import modules from git or other places (not published to Clio package repository).

2. Modules are fetched and compiled on install:

   • Modules are fetched, compiled, and stored in a clio_modules directory. The clio_modules directory will be published to npm (If the user wants to publish their project to npm). This makes the project size huge. require function should be patched to make this work.

   • Modules are fetched, compiled, and stored in node_modules/clio_modules. This method does not need patching require function, however, if user publishes the package to npm, since the node_modules directory isn't published it will break the module.

   • Fetch and compile Clio dependencies on npm install. We can put Clio dependencies in clioDependencies field of package.json and write a postinstall hook for the package. This way all compiled packages will depend on Clio cli and tools.

Clio new command should make the required directories (src, build, docs...).

We're currently using a hand built lexer, parser and code generator for Clio. These aren't feature rich and are causing us a few issues (lexing/parsing errors not being clear for example). Instead of extending the lexer / parser I decided to use a library for doing this.

I chose chevrotain, it has really good support, documentation and API. This will also make it easier for contributors who wish to add features to the language.

I suggest we add to the package.json file the field engines to instruct who will use or collaborate with the project which version of Node and/or NPM to use. For example:

"engines": {
  "node": ">=10.0",
  "npm": ">=3"
},

Optionally, we can also add a .nvmrc on the root of the project, so who uses NVM could just do an nvm use on the root of the project to know which version to install with the NVM. This, though is really optional since it relies on an external tool which maybe few people use (I use it).

First example:

fn fib n:
  if n < 2:  n
      else: (n - 1 -> fib) +
            (n - 2 -> fib)

[0:10] -> * fib -> print

$ clio run eg.clio 

 At file "eg.clio" when calling print:

   4 │             (n - 2 -> fib)
   5 │ 
   6 │ [0:10] -> * fib -> print
                          ^ At line 6 char 19

 Exception: BigInt is not defined

Installed as in instruction:
npm i -g clio-lang

Currently in generated code we require internals from .internals/*. This needs to be changed. We should probably move them into a separate npm package, and in our generated files import them from there, for example require("@clio-lang/internals"). This package should be a dependency of the generated node package and should be included in its package.json

Currently only if elif else is defined, other forms (for example if else) aren't defined and they do not work. Currently we have 1 parser rule for each combination (if_elif_conditional, if_else_conditional, ...), we should probably rename all of them to just conditional.

• Fix parsing rules (Rename to conditional and make the rule a valid entry in blocks)
• Replace if_elif_else_conditional in generator with conditional and make necessary modifications

Hi,

I am curious about the parallel programming in Clio. As I was going through the code (not in-depth ) as it is more kind of a wrapper over js. being js works on single thread. Have you included the thread concept through Native support of C++ lib of Nodejs . It will be pretty interesting to see.

Describe the bug
Dependencies for project can't be listed, crashes with an error instead

To Reproduce
Steps to reproduce the behavior:

1. Run clio new foo
2. cd into project
3. Run clio deps
4. Command crashes

Expected behavior
Dependencies are listed

Context

/Users/frankeg/dev/clio/node_modules/yargs/yargs.js:1183
      else throw err
           ^

ReferenceError: showDependencies is not defined
    at Object.exports.handler (/Users/frankeg/dev/clio/cli/commands/deps.js:12:3)
    at Object.runCommand (/Users/frankeg/dev/clio/node_modules/yargs/lib/command.js:240:40)
    at Object.parseArgs [as _parseArgs] (/Users/frankeg/dev/clio/node_modules/yargs/yargs.js:1095:41)
    at Object.get [as argv] (/Users/frankeg/dev/clio/node_modules/yargs/yargs.js:1029:21)
    at Object.<anonymous> (/Users/frankeg/dev/clio/cli/index.js:20:16)
    at Module._compile (internal/modules/cjs/loader.js:689:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
    at Module.load (internal/modules/cjs/loader.js:599:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
    at Function.Module._load (internal/modules/cjs/loader.js:530:3)

Versions (please complete the following information):

• OS: macOS Catalina
• Clio: v0.2.0 dev (3adc12f)
• Node: 10.15.2
• NPM: 6.11.3

Possible solutions

• Import showDependencies into module

after running clio init, bash wont display any input.
Testet on MacOS

Steps to reproduce

1. make a new directory
2. run clio init
3. wait for stdlib to download
4. type input

This issue follows #64 and #87.

Is your feature request related to a problem? Please describe.
Our current template projects https://github.com/clio-lang/template-node and https://github.com/clio-lang/template-browser have a static package.json inside build/<target>. In the case of browser target, it also has an index.html If the programmer resolves to change the build directory, the package.json file has to be moved. And this is not ideal.

Describe the solution you'd like
When running clio build, the package.json and index.html (in browser targets) should be automagically generated inside wherever the bundle directory is.

Describe alternatives you've considered
We could use an NPM init to do that, or just create a file from a string. For browser target, index.html should be created too.

Additional context
The NPM scripts on the package.json.

1. build script should be parcel build index.html --out-dir public;
2. Add a run script with parcel index.html --out-dir public;

build for clio build and run for clio run. 👌

Also, we need to create a .gitignore too. I think we can use https://raw.githubusercontent.com/github/gitignore/master/Node.gitignore.

An import pattern should be added to the lexer, rules should be written for parser and generator, and the necessary runtime (if needed) should be created.

• Add import pattern in core/patterns.js
• Add from pattern in core/patterns.js
• Add parser rules in core/clio.beef
• Add generator rules in core/generator.js
• Add required builtins and runtime in core/internals

Import statement should work with below patterns:

import * from path
import name1 name2 name3 from path
import path
import name1 as name2 from path
import path as name

For beef files, there's a vscode addon hosted here:
https://github.com/pouya-eghbali/bean/tree/master/vscode-beef

As discussed in the group, we'd like to have a html generator for code documentation.

An example documentation would look like his:

---
@doc 
# This function converts x to y
Lorem ipsum
[Markdown links!](https://foo.bar/)

@param a number to convert
@returns converted string
---
fn foo x:
  x -> toY

Important to note, is that the three dashes (---) represent a block comment. If a @doc symbol is present in this comment, it will be treated as documentation. README.md files either in project root or in modules will be included in the documentation

Running clio docs should compile the docs and put the generated html into the root/docs directory. It should be possible to write markdown inside of the documentation, that will be compiled to html. Possibly, a --open flag could be passed, which opens the generated docs in your browser.

A possible solution would be, to compile the Clio documentation to javascript and use jsdoc, however, further research is to be done about markdown parsing.

In order to begin working on this issue, #24 needs to be completed.

Room for discussion

• Somehow open the local documentation for clio itself, instead of showing the docs for the current package.
• In the future, we could maybe add manpage-like docs, like clio docs flow, which will point to the documentation for the flow operator.

Criteria

• Multiline comments that include @doc are treated as documentation
• Running clio docs generates the docs as html
• Docs support markdown
• Compiled documentation lives in root/docs
• README.md files are included
• clio docs --open opens the generated docs in a browser

Is your feature request related to a problem? Please describe.
With pure Clio it is somewhat too imperative to manipulate DOM in the browser.

Describe the solution you'd like
Clio is a pure functional language. So we need a pure functional way to manipulate DOM.

Describe alternatives you've considered
We need to create a library to provide to the developer functions to easily manipulate DOM.

Additional context
@garritfra posted this snippet on our Telegram channel:

Personally I like it. Really like it. We could follow this to get started.

Medium is not open.

"You’ve reached the end of your free member preview for this month. Become a member now for $5/month to read this story and get unlimited access to all of the best stories on Medium."

The wiki here may be a good alternative.

How to split a string by char (or empty char) to an array?

The new compiler should generate a package.json file in dest/.

Dependencies aren't handled correctly. No package.json is generated (See #69) and no dependencies are installed. Dependencies should be listed in the package.json and the compiler should do a npm install in dest/ after generating this file.

ToDo

• Generate package.json when calling clio build
• Install required clio dependencies to node_modules of build dir
• Install required npm dependencies to node_modules of build dir
• Add node dependencies field to toml
• Remove build directory from templates

This code:

import express from express/express

express -> call => app

fn handler req res:
    "hello world!" -> res.send

app -> get "/" handler
5000 -> app.listen

throws the following error:

ReferenceError: main is not defined
    at Object.clio_require (/home/garrit/dev/clio/internals/import.js:186:67)
    at module.exports (/home/garrit/dev/clio-express/.clio-cache/index.clio.js:3:20)
    at do_import (/home/garrit/dev/clio/internals/import.js:357:29)
    at clio_import (/home/garrit/dev/clio/internals/import.js:395:10)
    at process_file (/home/garrit/dev/clio/index.js:40:12)
    at Object.require.command.argv [as handler] (/home/garrit/dev/clio/index.js:110:7)
    at Object.runCommand (/home/garrit/dev/clio/node_modules/yargs/lib/command.js:242:26)
    at Object.parseArgs [as _parseArgs] (/home/garrit/dev/clio/node_modules/yargs/yargs.js:1078:30)
    at Object.get [as argv] (/home/garrit/dev/clio/node_modules/yargs/yargs.js:1012:21)
    at Object.<anonymous> (/home/garrit/dev/clio/index.js:222:16)

I believe it has something to do with the node modules, as i get the same issue when trying to import lodash

putting @eager next to fn i.e:
@eager fn greet name: 'hello' name -> print

causes the following error
TypeError: analyzers[node.name] is not a function at /usr/lib/node_modules/clio-lang/evaluator/analyzer.js:874:42 at Array.map (<anonymous>) at analyze (/usr/lib/node_modules/clio-lang/evaluator/analyzer.js:873:25) at analyzer (/usr/lib/node_modules/clio-lang/evaluator/analyzer.js:885:10) at do_import (/usr/lib/node_modules/clio-lang/internals/import.js:341:23) at clio_import (/usr/lib/node_modules/clio-lang/internals/import.js:395:10) at process_file (/usr/lib/node_modules/clio-lang/index.js:40:12) at Object.require.command.argv [as handler] (/usr/lib/node_modules/clio-lang/index.js:110:7) at Object.runCommand (/usr/lib/node_modules/clio-lang/node_modules/yargs/lib/command.js:242:26) at Object.parseArgs [as _parseArgs] (/usr/lib/node_modules/clio-lang/node_modules/yargs/yargs.js:1078:30)

Questions

1. Why
   '2' -> Number -> print
   raised exception?

   1 │ '2' -> Number -> print
              ^ At line 1 char 7
 Exception: Cannot read property 'call' of undefined

But online editor outputs 2?

2. In online editor,
   '2' -> Number -> typeof -> print

outputs function Number() { [native code] }
It should output "string"

Thinking it's returning typeof Number, I edit it to:
('2' -> Number) -> typeof -> print
and it outputs the same function Number() { [native code] }, why?

3. Say if I want to use parseInt(n, base) instead.
   Run of '2' -> parseInt -> typeof -> print gives:

   1 │ '2' -> parseInt -> typeof -> print
              ^ At line 1 char 7
 Exception: Cannot read property 'call' of undefined

4. Say if it doesn't error, is this how I pass the 2nd argument base to parseInt?
   Run of '2' -> parseInt 10 -> typeof -> print

To run the test, puppeteer is currently needed. We should get rid of that dependency, as it bulks up the build

• Remove puppeteer dependency

I'm working on a new Clio, the syntax and the features will be (almost) the same, but the way the language works will be different. There are several reasons for this:

• A better parser, with better error reporting
• Cleaner CST / AST
• Cleaner code generation
• No more custom types in core, we'll compile to vanilla JS
• No more custom scopes, custom functions, etc

Here are the required steps and the current state:

• Write a new lexer
• Write a new parser
• Make a new AST format
• Make a new code generator
• Make a core library to provide functional programming / lazy evaluation tools

You can check the code of the new parser here: https://github.com/pouya-eghbali/bean
I'll move this repository to Clio organization when all above steps are finished.

Is your feature request related to a problem? Please describe.
We don't have a way of importing Clio from a CDN. This is a very common practice on the front-end world.

Describe the solution you'd like
We need to host Clio on a CDN provider.

Describe alternatives you've considered
CDN.js is a good choice. It is free and widely used by the community.

Additional context
We need to open a PR on https://github.com/cdnjs/cdnjs to make it available. This issue will be used to track the progress.

The project should have a development pipeline, ensuring that the tests always pass on each commit.
Here are some CI providers:

• TravisCI
• CircleCI
• AppVeyor

Nessessary steps

• Add clio to TravisCI
• Add .travis.yml
• Add Travis build status badge to readme

Possible configurations

• run all tests on every commit
• test builds against multiple node versions (8, 9 and 10)

For compilation, I did what Python does when I first wrote Clio. Python compiles the files on import and caches them in a __pycache__ directory. I did the same and this caused a lot of issues with imports.

Node doesn't know about our cache directory or our compile on import strategy, so on every import I had to check:

1. is this a js import?
   1.1. import js from .. (we're in cache dir)
2. is it a clio import?
   2.1. resolve import path from ..
   2.2. compile if it's not compiled
   2.3. import from ../path/to/file/cache/file
3. is it an unknown import?
   3.1. assume clio import
   3.1. if import fails, assume js

If the above fails, we had to check places outside the project directory, for example globally installed clio modules (if such thing exists)

I also wrote a require function for the browser. It was possible to just do:

<script type="text/clio">
"Hello world" -> print
</script>

Or

<script type="text/clio" src="index.clio"></script>

And even this was possible:

<script type="text/clio">
import something from somewhere
data -> something 
</script>

We used to compile on the browser, calculate a hash for each file, and cache the compile results on the browser.

I suggest we don't do any of these anymore. For node, we'll compile all files in the project directory and put them in a .clio/cache/node, this solves the issues with imports, it simplifies dependency management, and a lot of other things.

For the browser, I suggest we do something similar and then generate a bundle from the compiled files.