clojure-rs / clojurers Goto Github PK

In main.rs we manually setup our environment as

    // Register our macros / functions ahead of time
    let add_fn = rust_core::AddFn{};
    let str_fn = rust_core::StrFn{};
    let do_fn = rust_core::DoFn{};
    let nth_fn = rust_core::NthFn{};
    let do_macro = rust_core::DoMacro{};
    let concat_fn = rust_core::ConcatFn{};
    let print_string_fn = rust_core::PrintStringFn{};
    // Hardcoded fns
    let lexical_eval_fn = Value::LexicalEvalFn{};
    // Hardcoded macros
    let let_macro = Value::LetMacro{};
    let quote_macro = Value::QuoteMacro{};
    let def_macro = Value::DefMacro{};
    let fn_macro = Value::FnMacro{};
    let defmacro_macro = Value::DefmacroMacro{};
    
    let environment = Rc::new(Environment::new_main_environment());
    
    let eval_fn = rust_core::EvalFn::new(Rc::clone(&environment));
    
    environment.insert(Symbol::intern("+"),add_fn.to_rc_value());
    environment.insert(Symbol::intern("let"),let_macro.to_rc_value());
    environment.insert(Symbol::intern("str"),str_fn.to_rc_value());
    environment.insert(Symbol::intern("quote"),quote_macro.to_rc_value());
    environment.insert(Symbol::intern("do-fn*"),do_fn.to_rc_value());
    environment.insert(Symbol::intern("do"),do_macro.to_rc_value());
    environment.insert(Symbol::intern("def"),def_macro.to_rc_value());
    environment.insert(Symbol::intern("fn"),fn_macro.to_rc_value());
    environment.insert(Symbol::intern("defmacro"),defmacro_macro.to_rc_value());
    environment.insert(Symbol::intern("eval"),eval_fn.to_rc_value());
    environment.insert(Symbol::intern("lexical-eval"),lexical_eval_fn.to_rc_value());
    environment.insert(Symbol::intern("nth"),nth_fn.to_rc_value());
    environment.insert(Symbol::intern("concat"),concat_fn.to_rc_value());
    environment.insert(Symbol::intern("print-string"),print_string_fn.to_rc_value());
    ...

instead, this should all be moved to

pub fn clojure_core_environment() -> Rc<Environment> {
   ...
}

and main should just contain the one liner

let environment = Environment::clojure_core_environment();

If I recall, due to Value containing

enum Value { 
  ...
  IFn(Rc<dyn IFn>)
  ..
}

As well as some similar cases containing trait objects, we can not just use [derive(..PartialEq,Hash)] , but we do need these as our persistent maps will use Values as their keys.

Since a rough, basic persistent map was added at the last minute, so was a last minute implementation of Hash and PartialEq on Value. However, I'm pretty sure only Hash needs to be defined, and PartialEq can be defined in terms of Hash. Whatever the case, this code is an example of something that was something of a last minute hack, and has not been proofread yet, so I'd like to make sure this implemented as it should be.

Hi there !

I don't know if it's the right place, if not I'm sorry. I also just came across your project which seems really cool and of special interest to me.

I got the same idea about implementing Clojure in Rust and I have been working on a Clojure implementation of my own in the last few weeks. It is not as advanced as yours (I mostly only have the parser part so far) and I have only used the standard library. You want to take a look you can find it at https://github.com/arbroween/claire

Would you be interested to collaborate or take contributors ? :)

Originally posted by @arbroween in #1 (comment)

File affected: reader.rs

Clojure 1.10.3:

clj -e '(let [k :foo/bar/qux] [k (namespace k) (name k)])'

[:foo/bar/qux "foo" "bar/qux"]

ClojureRS 75c2bc156b6e43635f57e6f4b0abf180eb477047:

cargo run -- -e ':foo/bar/qux'

:foo/bar

even if script succeeds, it returns an error condition

$ cargo run -- examples/hello_world.clj
Hello, world
#Condition["Tried to read empty stream; unexpected EOF"]

expected success case:

no more output after Hello, world

Ie, have something like this

user=> (+ 1 
.. 2 
.. 3)
6

Instead of what you have right now, which is

user=> (+ 1 

user=> 2 

user=> 3)
6

My (Bevy-based) app does not have a src/clojure/core.clj nor a src/clojure/string.clj so the resulting load errors (observed by printing the return value of the try_eval_file calls) are silently ignored/swallowed/eaten/etc (due to the let _ = ...s).

Ie, instead of

Value::Condition(format!(
     "Type mismatch; Expected instance of clojure.lang.String, Recieved type {}",
     args.get(0).unwrap().type_tag()
)),

write

Value::Condition(format!(
    "Type mismatch; Expected instance of {}, Recieved type {}",
    TypeTag::String,
    args.get(0).unwrap().type_tag()
))

Related to issue #21 (comment) , which should be done first

Early on I decided to try a few different signatures for invoke to gain an intuition on what would work best.

Overall, I see that the program currently basically lives in a Rc<Value>, spending its time being 'seen' by many, and sometimes being cloned and transformed into a new value. It seems better to fit with the rest of the program and also just expect Rc<Value>s in invoke
(ie, to go from

    fn invoke(&self,args: Vec<&Value>) -> Value {

to

     fn invoke(&self,args: Vec<Rc<Value>>) -> Value {

. Plus, I am currently in a situation where invoke needs a Rc<Value> of a &Value being passed in for one of the functions I'm implementing (assoc), and so to (currently) get this we basically have to needlessly clone &Value and Rc::new(..) that, causing the same Value to exist in multiple, independent spots in memory with their own family of Rc references. The nice thing is we are dealing with Values, not mutating entities with identity, so we should be able to efficiently store them once and reference them all over, rather than cloning them for each 'user' who wants to deal with one. I don't know if there are any unforseen trade offs of this

There are other places with indiscriminant cloning that likely can be changed a bit, but we will get to that

Off the top of my head, areas affected:

• ifn.rs, as the IFn trait will have its invoke signature changed
• value.rs, as eval will be changed to pass a vector of Rc<Value>, not &Value
• rust_core.rs, as all of our primitive rust-wrapping-functions, our structs that implement IFn, are defined here

I'd like to meet the community, but unfortunately the Discord invite has expired!

Hello!

Very interested in this project. Just wanted to share this repo in case it could be useful:

https://github.com/alex-dixon/serde_edn

Support for metadata is needed, i.e.

meta, with-meta functionality.

For a similar consistency issue in value.rs as our std::string::String issue, in value.rs, we used to always use the qualified name for a Value variant, and the unqualified name for the value it wraps (ie,
when referring to the PersistentList variant of Value

enum Value { 
  ...
  PersistentList(PersistentList),
  ...
}

we would later write it like this

let new_persistent_list = Value::PersistentList(PersistentList { .. });

where the inner value, PersistentList, is written outright, non-qualified, and the wrapper Value variant, PersistentList, is qualified as Value::PersistentList.

At some point, Value began to be used in an unqualified way, for reasons I have forgotten, and now sometimes its referred to as Value::Variant(innerValue) (either because of changes that got left behind, or cases of ambiguity, where I'm thinking its the latter) and sometimes just Variant(innerValue). I would like to make this consistent again by just qualifying all cases as it used to be

Ie for defining a Clojure primitive that thinly wraps a rust function

API for clojure.core/get - Clojure v1.10.2 (stable)

(get map key)
(get map key not-found)

for src in '(get,{},:k)#_nil' '(get,{:k,:v},:k)#_:v' '(get,{},:k,:not-found)#_:not-found' '(get,{:k,:v},:k,:not-found)#_:v'; do clj -e "(prn,$src)"; done

nil
:v
:not-found
:v

ClojureRS 1a1681f supports only

(get map key)

for src in '(get,{},:k)' '(get,{:k,:v},:k)' '(get,{},:k,:not-found)' '(get,{:k,:v},:k,:not-found)'; do cargo run -- -e "$src"; done

nil
:v
#Condition["Wrong number of arguments given to function (Given: 3, Expected: 2)"]
#Condition["Wrong number of arguments given to function (Given: 3, Expected: 2)"]

You can get the current namespace from the environment. An accessor will have to be added to access it (that field will remain private so we can inforce whatever invariants are involved with changing the current namespace)

Hi! This is a neat project!
I'd like to understand how it handles memory management.
Thanks!

Really would like to implement these basically now, before the system gets too much bigger, especially as we are starting to use Conditions and want to handle them (such as with reading a file, which will eventually get an eof error but will want to handle that by ignoring it)

Structures like PersistentList (and especially ones like PersistentVector, which will internally change) should not have public internals, and should be interacted with through their public interface; we have already had one potential bug where the cached count was used in PersistentList in an impossible scenario (ie, we were matching on the case of PersistentList(_,_,count = 0) , which would actually never happen, as an empty list would be expressed as PersistentList::Empty)

Off top of my head, need to:

1. Make counting always read with len
2. Use cons to attached a head to a tail
3. Convert PersistentVector into Vector explicitly, rather than doing so by cloning its internal vals Vector

https://github.com/Tko1/ClojureRS/issues/12#issue-606564082

#14 (comment)

Ensure formatting is kept consistent, automating that process. Not sure if there's a point to adding clippy into the mix too

There is an inconsistency in value.rs, in that when Value wraps something (ie, like as follows)

enum Value { 
   I32(i32),
   PersistentList(PersistentList),
   ...
}

We just write it, well, like that, using the unqualified name of the inner value being wrapped inside the variant, and importing, say in this case,

use crate::persistent_list::PersistentList;

But when we wrap strings, we don't import it and as such have to use the fully qualified name everywhere to differentiate it from the Value::String variant; ie we have

enum Value { 
   I32(i32),
   PersistentList(PersistentList),
   ...
   String(std::string::String) // <--- 
   ...
}

and everywhere else we have things like

std::string::String::from("#macro[let*]")

Instead, add use std::string::String; to top and replace std::string::String throughout the file with String, for consistency and, well, brevity. And if instead anyone has any suggestion instead for a change to this style altogether (such as instead of removing the qualified name on every wrapped time, adding the qualified name for every wrapped time) and a strong reasoning, mention that instead.

• make sure "" can be parsed by the reader
• add unit test

clojure.core=> ""
#Condition["Reader Error: could not read next form; Error(("\"\"", Tag))"]

example, if I use refer in clojure.core:

(ns clojure.core)
(refer 'clojure.interop)

any invalid expression causes stack overflow

clojure.core=> asdfaf
thread 'main' has overflowed its stack
fatal runtime error: stack overflow
[1]    77210 abort      cargo run

any valid expression does not cause stack overflow, if I remove the refer, no issues

As a Rust developer (and ex-Clojure developer) i am quite interested in your repo!

However it all depends on how you want to collaborate: is it "about you" or "about us"?

Symbol expansion currently happens in our Namespaces struct's get function -- basically, it tries to figure out where we are referring to, based off of what namespace we are currently in, and what symbol we are asking for. For instance, in: my-ns , asking for: + will eventually cause get to find + in clojure.core/+.

However, recently we had to add a function for getting the Var @ a symbol, rather than its value -- and because it was made separately from the branch that added namespace-refers to symbol expansion, we now have two functions with symbol expansion, and one of them is now out of sync. Blah blah typical DRY blah blah. And while we surely will not need any more get functions (and in fact, we will be back to having just one get function, as get will be rewritten in terms of get_var().deref()), I am pretty sure we will use symbol expansion yet again for, I don't know, the backquote possibly. Either way, it should be its own function

Title says basically everything.

Development will live on my fork. Feel free to make any comment about the implementation here.

Here is the minimum set of things that must work before a PR is created:

• ability to connect a lein client to the server,
• add proper CLI handling to decide whether if we start a nrepl server or drop in a regular repl,
• handle multiple connections at the same time.

Most of the work here is inspired of the implementation of borkdude's nrepl server.

Below is a more detalled todo list of what needs to be done:

• start a basic server, listening to a predefined port,
• add a way to deserialize requests to a rust HashMap,
• handle the clone operation,
• handle the eval operation,
• figure out if we need to send some ack or not,
• handle other necessary but still not discovered operations,
• discuss about whether if different connections should be able to share defined functions,
• handle proper cli (see Erkk comment on discord),
• get testing from various people, with various setups.

If you have any question or suggestion, feel free to comment on this issue or to contact me on Discord.

Generalize exceptions like type mismatch, wrong number of arguments, etc, as these are being written by hand each time

For now I'm just putting together a discord, as I've always preferred its feature-set. That being said, I'm open to whatever works best for the most people

@scileo do you have a discord?

Link here: https://discord.gg/aS5Sa8p

(This is actually being worked on by erkkikeranen, I have just created this issue at the last minute so there is a formal ticket for his contribution)

Hi, I'm exploring the use of transducers on rust, I have a small POC here:

https://gist.github.com/mamcx/b57f557fda1a3e99cfff6e2276054b0e

in case you consider it interesting.

BTW, I'm struggling in implement "zip/sql-like joins" in case you know where to look :)

File affected: reader.rs

Reader right now is

1. Often using nom's ws macro to consume whitespace around a read, and to my knowledge this is deprecated.

2. Only looking for traditional whitespace. In Clojure, however, , is considered whitespace as well, and this case needs to be added, as well as any edge case I've never heard of if one exists

So, the preferable solution is to create a parsing function for clojure whitespace, ie something like

// This is named clojure_whitespace rather than whitespace to emphasize this is consuming
// what Clojure considers to be whitespace, including the , 
// Am open to a better name 
pub fn clojure_whitespace_parser(input:&[u8]) -> IResult<&[u8], String> {
   ...
}

And then, instead of combining this with another parser with nom's macros like

    named!(someParserThatConsumesWhitespaceFirst,
	   ws!(tag!("}")));

We would use some function combinator, like preceded

let someParserThatConsumesWhitespaceFirst = preceded(clojure_whitespace_parser, ... )

This would involve updating all spots that currently consume whitespace, which should be

1. try_read
2. try_read_map
3. try_read_string
4. try_read_vector
5. try_read_list

Most likely this will just involve integrating https://docs.rs/im/14.3.0/im/ into

persistent_vector.rs

and adding a persistent_hashmap.rs also built on this (not to be confused with our already existing persistent_list_map, which is a non-efficient persistent hashmap implemented as an associative-list for small datasets and, currently, used for prototyping things related to maps)