microsoft / verona Goto Github PK
View Code? Open in Web Editor NEWResearch programming language for concurrent ownership
Home Page: https://microsoft.github.io/verona/
License: MIT License
Research programming language for concurrent ownership
Home Page: https://microsoft.github.io/verona/
License: MIT License
Related to #104, this task is about lowering the operations, constructs (like if, when, where), lexical scopes, lambda logic, future registration, dispatch and completion, etc.
This can be a mix of existing dialects (std, loop) and Verona-specific constructs, but anything that touches specific Verona types must be Verona-specific nodes as well, so likely to be mostly Verona anyway.
Acceptance criteria:
Not included:
Hello, please add FAQ entry why Verona is being implemented in C++.
It is stated that research is inspired by Rust/Pony, but current codebase is written in C++.
Are there any particular reason not using safe language?
Actually if not Rust, than perhaps C# as being safe language, and it's research project, so speed isn't such an issue.
Submodule at path src/rt/external/snmalloc
is at revisiondbfb645
, whereas upstream is at revision 4212ac8
.
You can update the submodule by running:
git submodule update --remote src/rt/external/snmalloc
Strict region ownership makes it impossible to do some efficient representations. Although, regions allow a doubly linked list to be expressed, that is actually insufficient for the common pattern of using a doubly linked list. Typically, you keep a reference to the Node to allow O(1) removal, but due to strict region ownership, that limits the applicability, and does not allow that reference to be across any concurrency boundary, e.g. in different cowns.
The runtime supports a concept of external reference that allows for a safe cross cown reference. The concurrency is safe as promotion back to the original reference requires a reference to the owning region, and thus cannot be promoted unless you already have access to the object's region.
Here is a proposed API surface we could use to expose external region references:
class External[T]
{
// Creates an external reference to an object in a region.
// We treat the external reference as immutable as it is effectively
// an immutable token that can be exchanged for access to the original
// reference.
get_handle(v: T & mut) : External[T] & imm {...}
// External references can be promoted to the original reference
// Returns None if
// * the provided region is not the one containing this object, or
// * the object has been collected by this regions allocation strategy
// external references may not keep objects live.
// Otherwise, returns the reference passed to `get_handle` to create
// the handle.
promote[U](self: mut, region: U & mut) : (T & mut) | (None & imm) {...}
}
The core motivating example for external references is doubly linked notification lists.
class NotificationNode
{
next: NotificationNode;
prev: NotificationNode;
to_notify: cown[INotify];
}
The client of the notification center does something like:
var n = new NotificationNode;
n.to_notify = ...
var e = ExternalReference.get_handle(n);
// Transfer ownership of the region containing n, to the notification center.
// Internally, this would merge the region containing the node into
// the region containing the doubly linked list
when (notification_center) { notification_center.add(n) }
...
//some time later the external reference can be used to remove the
//node from the doubly linked list. It promotes the external reference
//and if successful, can perform the standard DLL pointer surgery on the node.
when (notification_centre) { notification_center.remove(e) }
Is it better to surface this as a class with a static and an instance method. Or should this be more primitively exposed in the type system to allow for better flow typing? At the moment, this would require a match
on the result, would the following be better:
// e: External[T]
if (e is_in r)
{
// e: T & mut
...
}
This is a more complex change. Proposal to stick to exposing as API until we have experience with using?
The raw API can be exposed directly without any additional work.
The full example requires region merge to be exposed in the language as well.
The run time has a unit test for this functionality.
Tokenise the source in a simple structure. Examine the trade-offs of adding lexical blocks {} as lists of tokens instead of using a simple stream and keeping the context in the parser.
Acceptance criteria:
Not included:
With the type system designed in #104, we need to create the system that will convert from opaque MLIR types to explicit Verona dialect types. This will likely incur in inter-procedural analysis and may need to generate multiple copies of the same function with different types.
We'd want to keep those auto-generate functions in sync, or at least related to each other with some form of annotation (ex. from which base function they're derived, other sibling stubs) and we may need to more aggressively inline or elide those auto-generated functions.
We could implement this as a precursor of the partial conversion (opaque -> Verona), so that the lowering happens in a more natural way, or we could make that as part of the lowering, but we need to be careful with partially lowered types and stubs. In any case, we'll need a cleanup pass before validation.
The initial support is critical to get right, with the remaining support included as it's added, later in the process.
Acceptance criteria:
Not included:
@sylvanc has a parser in a local repo that can parse Verona. It has a PEG definition that is flexible and the code is reasonably straightforward, both which are great for early prototyping. We should merge this in the src
directory and use the generated AST to start lowering MLIR.
We may want to build a robust parser later, with @davidchisnall's recommendations in #102, but that'll be for another ticket.
Freshly cloned repo following build steps in docs
Versions
Os: Windows 10
cmake version 3.17.20032601-MSVC_2
Visual Studio 2019 v16.6.0
"C:\Users\Nate\source\repos\verona\build\INSTALL.vcxproj" (default target) (1) ->
"C:\Users\Nate\source\repos\verona\build\ALL_BUILD.vcxproj" (default target) (3) ->
"C:\Users\Nate\source\repos\verona\build\src\interpreter\interpreter-sys.vcxproj" (default target) (9) ->
(ClCompile target) ->
C:\Users\Nate\source\repos\verona\src\rt\region\externalreference.h(55): error C2280: 'verona::rt::Object::Obj
ect(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpreter\
interpreter-sys.vcxproj]
C:\Users\Nate\source\repos\verona\src\rt\region\region_arena.h(237): error C2280: 'verona::rt::RegionBase::Reg
ionBase(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpre
ter\interpreter-sys.vcxproj]
C:\Users\Nate\source\repos\verona\src\rt\region\region_trace.h(72): error C2280: 'verona::rt::RegionBase::Regi
onBase(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpret
er\interpreter-sys.vcxproj]
C:\Users\Nate\source\repos\verona\src\interpreter/object.h(66): error C2280: 'verona::rt::V<verona::interprete
r::VMCown,verona::rt::RegionType::Cown,verona::rt::Cown>::V(void)': attempting to reference a deleted function [
C:\Users\Nate\source\repos\verona\build\src\interpreter\interpreter-sys.vcxproj]
C:\Users\Nate\source\repos\verona\src\interpreter/object.h(75): error C2280: 'verona::rt::V<verona::interprete
r::VMCown,verona::rt::RegionType::Cown,verona::rt::Cown>::V(void)': attempting to reference a deleted function [
C:\Users\Nate\source\repos\verona\build\src\interpreter\interpreter-sys.vcxproj]
C:\Users\Nate\source\repos\verona\src\interpreter\interpreter.cc(28): error C2280: 'verona::rt::V<verona::inte
rpreter::EmptyCown,verona::rt::RegionType::Cown,verona::rt::Cown>::V(void)': attempting to reference a deleted f
unction [C:\Users\Nate\source\repos\verona\build\src\interpreter\interpreter-sys.vcxproj]
C:\Users\Nate\source\repos\verona\src\rt\region\externalreference.h(55): error C2280: 'verona::rt::Object::Obj
ect(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpreter\
interpreter-sys.vcxproj]
C:\Users\Nate\source\repos\verona\src\rt\region\region_arena.h(237): error C2280: 'verona::rt::RegionBase::Reg
ionBase(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpre
ter\interpreter-sys.vcxproj]
C:\Users\Nate\source\repos\verona\src\rt\region\region_trace.h(72): error C2280: 'verona::rt::RegionBase::Regi
onBase(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpret
er\interpreter-sys.vcxproj]
C:\Users\Nate\source\repos\verona\src\interpreter/object.h(66): error C2280: 'verona::rt::V<verona::interprete
r::VMCown,verona::rt::RegionType::Cown,verona::rt::Cown>::V(void)': attempting to reference a deleted function [
C:\Users\Nate\source\repos\verona\build\src\interpreter\interpreter-sys.vcxproj]
C:\Users\Nate\source\repos\verona\src\interpreter/object.h(75): error C2280: 'verona::rt::V<verona::interprete
r::VMCown,verona::rt::RegionType::Cown,verona::rt::Cown>::V(void)': attempting to reference a deleted function [
C:\Users\Nate\source\repos\verona\build\src\interpreter\interpreter-sys.vcxproj]
C:\Users\Nate\source\repos\verona\src\interpreter\object.cc(40): error C2280: 'verona::rt::Object::Object(void
)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpreter\interpre
ter-sys.vcxproj]
C:\Users\Nate\source\repos\verona\src\rt\region\externalreference.h(55): error C2280: 'verona::rt::Object::Obj
ect(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpreter\
interpreter-sys.vcxproj]
C:\Users\Nate\source\repos\verona\src\rt\region\region_arena.h(237): error C2280: 'verona::rt::RegionBase::Reg
ionBase(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpre
ter\interpreter-sys.vcxproj]
C:\Users\Nate\source\repos\verona\src\rt\region\region_trace.h(72): error C2280: 'verona::rt::RegionBase::Regi
onBase(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpret
er\interpreter-sys.vcxproj]
C:\Users\Nate\source\repos\verona\src\interpreter/object.h(66): error C2280: 'verona::rt::V<verona::interprete
r::VMCown,verona::rt::RegionType::Cown,verona::rt::Cown>::V(void)': attempting to reference a deleted function [
C:\Users\Nate\source\repos\verona\build\src\interpreter\interpreter-sys.vcxproj]
C:\Users\Nate\source\repos\verona\src\interpreter/object.h(75): error C2280: 'verona::rt::V<verona::interprete
r::VMCown,verona::rt::RegionType::Cown,verona::rt::Cown>::V(void)': attempting to reference a deleted function [
C:\Users\Nate\source\repos\verona\build\src\interpreter\interpreter-sys.vcxproj]
C:\Users\Nate\source\repos\verona\src\rt\region\externalreference.h(55): error C2280: 'verona::rt::Object::Obj
ect(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpreter\
interpreter-sys.vcxproj]
C:\Users\Nate\source\repos\verona\src\rt\region\region_arena.h(237): error C2280: 'verona::rt::RegionBase::Reg
ionBase(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpre
ter\interpreter-sys.vcxproj]
C:\Users\Nate\source\repos\verona\src\rt\region\region_trace.h(72): error C2280: 'verona::rt::RegionBase::Regi
onBase(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpret
er\interpreter-sys.vcxproj]
C:\Users\Nate\source\repos\verona\src\interpreter/object.h(66): error C2280: 'verona::rt::V<verona::interprete
r::VMCown,verona::rt::RegionType::Cown,verona::rt::Cown>::V(void)': attempting to reference a deleted function [
C:\Users\Nate\source\repos\verona\build\src\interpreter\interpreter-sys.vcxproj]
C:\Users\Nate\source\repos\verona\src\interpreter/object.h(75): error C2280: 'verona::rt::V<verona::interprete
r::VMCown,verona::rt::RegionType::Cown,verona::rt::Cown>::V(void)': attempting to reference a deleted function [
C:\Users\Nate\source\repos\verona\build\src\interpreter\interpreter-sys.vcxproj]
"C:\Users\Nate\source\repos\verona\build\INSTALL.vcxproj" (default target) (1) ->
"C:\Users\Nate\source\repos\verona\build\ALL_BUILD.vcxproj" (default target) (3) ->
"C:\Users\Nate\source\repos\verona\build\src\interpreter\interpreter.vcxproj" (default target) (5) ->
C:\Users\Nate\source\repos\verona\src\rt\region\externalreference.h(55): error C2280: 'verona::rt::Object::Obj
ect(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpreter\
interpreter.vcxproj]
C:\Users\Nate\source\repos\verona\src\rt\region\region_arena.h(237): error C2280: 'verona::rt::RegionBase::Reg
ionBase(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpre
ter\interpreter.vcxproj]
C:\Users\Nate\source\repos\verona\src\rt\region\region_trace.h(72): error C2280: 'verona::rt::RegionBase::Regi
onBase(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpret
er\interpreter.vcxproj]
C:\Users\Nate\source\repos\verona\src\interpreter/object.h(66): error C2280: 'verona::rt::V<verona::interprete
r::VMCown,verona::rt::RegionType::Cown,verona::rt::Cown>::V(void)': attempting to reference a deleted function [
C:\Users\Nate\source\repos\verona\build\src\interpreter\interpreter.vcxproj]
C:\Users\Nate\source\repos\verona\src\interpreter/object.h(75): error C2280: 'verona::rt::V<verona::interprete
r::VMCown,verona::rt::RegionType::Cown,verona::rt::Cown>::V(void)': attempting to reference a deleted function [
C:\Users\Nate\source\repos\verona\build\src\interpreter\interpreter.vcxproj]
C:\Users\Nate\source\repos\verona\src\interpreter\interpreter.cc(28): error C2280: 'verona::rt::V<verona::inte
rpreter::EmptyCown,verona::rt::RegionType::Cown,verona::rt::Cown>::V(void)': attempting to reference a deleted f
unction [C:\Users\Nate\source\repos\verona\build\src\interpreter\interpreter.vcxproj]
C:\Users\Nate\source\repos\verona\src\rt\region\externalreference.h(55): error C2280: 'verona::rt::Object::Obj
ect(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpreter\
interpreter.vcxproj]
C:\Users\Nate\source\repos\verona\src\rt\region\region_arena.h(237): error C2280: 'verona::rt::RegionBase::Reg
ionBase(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpre
ter\interpreter.vcxproj]
C:\Users\Nate\source\repos\verona\src\rt\region\region_trace.h(72): error C2280: 'verona::rt::RegionBase::Regi
onBase(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpret
er\interpreter.vcxproj]
C:\Users\Nate\source\repos\verona\src\interpreter/object.h(66): error C2280: 'verona::rt::V<verona::interprete
r::VMCown,verona::rt::RegionType::Cown,verona::rt::Cown>::V(void)': attempting to reference a deleted function [
C:\Users\Nate\source\repos\verona\build\src\interpreter\interpreter.vcxproj]
C:\Users\Nate\source\repos\verona\src\interpreter/object.h(75): error C2280: 'verona::rt::V<verona::interprete
r::VMCown,verona::rt::RegionType::Cown,verona::rt::Cown>::V(void)': attempting to reference a deleted function [
C:\Users\Nate\source\repos\verona\build\src\interpreter\interpreter.vcxproj]
C:\Users\Nate\source\repos\verona\src\interpreter\object.cc(40): error C2280: 'verona::rt::Object::Object(void
)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpreter\interpre
ter.vcxproj]
C:\Users\Nate\source\repos\verona\src\rt\region\externalreference.h(55): error C2280: 'verona::rt::Object::Obj
ect(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpreter\
interpreter.vcxproj]
C:\Users\Nate\source\repos\verona\src\rt\region\region_arena.h(237): error C2280: 'verona::rt::RegionBase::Reg
ionBase(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpre
ter\interpreter.vcxproj]
C:\Users\Nate\source\repos\verona\src\rt\region\region_trace.h(72): error C2280: 'verona::rt::RegionBase::Regi
onBase(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpret
er\interpreter.vcxproj]
C:\Users\Nate\source\repos\verona\src\interpreter/object.h(66): error C2280: 'verona::rt::V<verona::interprete
r::VMCown,verona::rt::RegionType::Cown,verona::rt::Cown>::V(void)': attempting to reference a deleted function [
C:\Users\Nate\source\repos\verona\build\src\interpreter\interpreter.vcxproj]
C:\Users\Nate\source\repos\verona\src\interpreter/object.h(75): error C2280: 'verona::rt::V<verona::interprete
r::VMCown,verona::rt::RegionType::Cown,verona::rt::Cown>::V(void)': attempting to reference a deleted function [
C:\Users\Nate\source\repos\verona\build\src\interpreter\interpreter.vcxproj]
C:\Users\Nate\source\repos\verona\src\rt\region\externalreference.h(55): error C2280: 'verona::rt::Object::Obj
ect(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpreter\
interpreter.vcxproj]
C:\Users\Nate\source\repos\verona\src\rt\region\region_arena.h(237): error C2280: 'verona::rt::RegionBase::Reg
ionBase(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpre
ter\interpreter.vcxproj]
C:\Users\Nate\source\repos\verona\src\rt\region\region_trace.h(72): error C2280: 'verona::rt::RegionBase::Regi
onBase(void)': attempting to reference a deleted function [C:\Users\Nate\source\repos\verona\build\src\interpret
er\interpreter.vcxproj]
C:\Users\Nate\source\repos\verona\src\interpreter/object.h(66): error C2280: 'verona::rt::V<verona::interprete
r::VMCown,verona::rt::RegionType::Cown,verona::rt::Cown>::V(void)': attempting to reference a deleted function [
C:\Users\Nate\source\repos\verona\build\src\interpreter\interpreter.vcxproj]
C:\Users\Nate\source\repos\verona\src\interpreter/object.h(75): error C2280: 'verona::rt::V<verona::interprete
r::VMCown,verona::rt::RegionType::Cown,verona::rt::Cown>::V(void)': attempting to reference a deleted function [
C:\Users\Nate\source\repos\verona\build\src\interpreter\interpreter.vcxproj]
0 Warning(s)
44 Error(s)
Time Elapsed 00:01:33.82```
I know Verona is designed for concurrency, but I'm trying to understand what Verona's ambitions are regarding parallelism. The only reference I could find to this in the repository was a link to this example: https://github.com/microsoft/verona/blob/master/testsuite/demo/run-pass/fib.verona
This is an interesting example, but I'm not sure if it answers my main question: does Verona intend to tackle high-perfomance data parallelism? For example, supporting operations like parallel iteration to accelerate a simple loop over a flat array of numerical data.
Verona's strategy is apparently to provide one extremely powerful, safe model of concurrency, instead of Rust's approach of hiding unsafe code behind safe interfaces so that things like concurrency can be implemented as a library. Although Verona seems to have many of the same abstractions used by Rust library authors to implement libraries like Rayon, I believe these libraries generally require some unsafe code at their core.
Are Verona's concurrency primitives expected to be sufficient to implement a high-performance data parallelism library like Rayon?
Thanks!
Verona currently does not have a system to prevent a cown's message queue from growing out of memory. I nearly have a working example of backpressure for many cowns sending to a single cown on my fork, which is based on the backpressure system in the Pony runtime. Before I continue to extend this to multimessages in general, I would like to know if this is a feature that the team working on Verona would like to see implemented.
With the new front-end (#111, #116) merged, we need to make sure the existing examples parse correctly into an acceptable AST shape, that we can later lower into MLIR (#117). However, these tasks are not dependent on each other, as only enough of the AST is needed to convert, while this task is to extend that support.
There are already a few tests that work:
./resolution/compile-pass/circular.verona
./resolution/compile-pass/interleaved.verona
./reachability/compile-pass/visit-variable-renaming-type.verona
./reachability/compile-pass/visit-receiver.verona
./features/compile-pass/library/sibling.verona
./features/compile-pass/cell.verona
./typechecking/compile-pass/field-write.verona
while others emit errors when trying to parse a class
declaration.
The idea is to discuss, from the semantics, what syntax we want, and change either parser or examples to fit the consensus.
Acceptance criteria:
Optional:
Not included:
With the token tree in hand, produce an AST from it. If writing it in opaque MLIR is a good idea (see #101), then output opaque MLIR in a recognisable shape.
Acceptance criteria:
Not included:
Submodule at path src/rt/external/snmalloc
is at revision8304ded
, whereas upstream is at revision eaeb2aa
.
You can update the submodule by running:
git submodule update --remote src/rt/external/snmalloc
We'll soon need LLVM/MLIR src/build dir, and GitHub can cache build dirs of specific builds, we need to set that up and keep it current for all other Verona's CI builds.
First of all, thank you so much for making this project open-source 😄 .
I have a clarification question about the verona runtime :
(from verona/docs/explore.md)
With Project Verona, we are carving out an area of system programming, "infrastructure programming", that has important performance and predictability requirements, without needing raw access to the machine.
I initially assumed that since Verona is a Rust-inspired language focusing on system programming, it would probably aim to have a minimal or no runtime.
Is the Verona runtime solely for debugging purposes, or is it something that is required to be running in the backend for all verona programs?
Without extra -DRT_TESTS=ON
passed to cmake
, I managed to follow the README to configure, build/install verona and successfully run provided examples.
Yet with -DRT_TESTS=ON
I get compilation error (details below). Am I doing something wrong?
HEAD: 7bdf0f84f58f844411b093e34f57acd54ebd8a40
$ uname -a
Darwin MacBook-Air.local 19.2.0 Darwin Kernel Version 19.2.0: Sat Nov 9 03:47:04 PST 2019; root:xnu-6153.61.1~20/RELEASE_X86_64 x86_64
$ cmake --version
cmake version 3.16.2
$ ninja --version
1.9.0
$ mkdir build_ninja && cd build_ninja
$ cmake .. -GNinja -DCMAKE_BUILD_TYPE=Debug -DRT_TESTS=ON
-- The CXX compiler identification is AppleClang 11.0.0.11000033
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Version: 5.3.1
-- Build type: Debug
-- CXX_STANDARD: 17
-- Performing Test has_std_17_flag
-- Performing Test has_std_17_flag - Success
-- Performing Test has_std_1z_flag
-- Performing Test has_std_1z_flag - Success
-- Performing Test SUPPORTS_VARIADIC_TEMPLATES
-- Performing Test SUPPORTS_VARIADIC_TEMPLATES - Success
-- Performing Test SUPPORTS_USER_DEFINED_LITERALS
-- Performing Test SUPPORTS_USER_DEFINED_LITERALS - Success
-- Performing Test FMT_HAS_VARIANT
-- Performing Test FMT_HAS_VARIANT - Success
-- Performing Test HAS_NULLPTR_WARNING
-- Performing Test HAS_NULLPTR_WARNING - Success
-- Looking for open
-- Looking for open - found
-- The C compiler identification is AppleClang 11.0.0.11000033
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Performing Test CXX_FILESYSTEM
-- Performing Test CXX_FILESYSTEM - Success
-- Looking for _LIBCPP_VERSION
-- Looking for _LIBCPP_VERSION - found
CMake Warning at src/rt/external/snmalloc/CMakeLists.txt:54 (message):
Not generating clangformat target, no clang-format tool found
Call Stack (most recent call first):
src/CMakeLists.txt:28 (clangformat_targets)
-- Found Python3: /usr/local/Frameworks/Python.framework/Versions/3.7/bin/python3.7 (found version "3.7.4") found components: Interpreter
-- Configuring done
-- Generating done
-- Build files have been written to: <snip>/verona/build_ninja
$ ninja install
[36/153] Building CXX object src/rt/CMakeFiles/func-con-cowngc4.dir/test/func/cowngc4/cowngc4.cc.o
FAILED: src/rt/CMakeFiles/func-con-cowngc4.dir/test/func/cowngc4/cowngc4.cc.o
/Library/Developer/CommandLineTools/usr/bin/c++ -DSNMALLOC_CHEAP_CHECKS -DUSE_EXECINFO -DUSE_FLIGHT_RECORDER -DUSE_QUICK_EXIT -I../src/. -I../src/rt/. -I../src/rt/external/snmalloc/src -g -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -Wsign-conversion -Wall -Wextra -Werror -Wundef -mcx16 -march=native -std=gnu++1z -MD -MT src/rt/CMakeFiles/func-con-cowngc4.dir/test/func/cowngc4/cowngc4.cc.o -MF src/rt/CMakeFiles/func-con-cowngc4.dir/test/func/cowngc4/cowngc4.cc.o.d -o src/rt/CMakeFiles/func-con-cowngc4.dir/test/func/cowngc4/cowngc4.cc.o -c ../src/rt/test/func/cowngc4/cowngc4.cc
../src/rt/test/func/cowngc4/cowngc4.cc:384:11: error: no matching member function for call to 'run'
harness.run(test_cown_gc<RegionType::Trace>, forward, ring, &rand);
~~~~~~~~^~~
../src/rt/./test/harness.h:73:8: note: candidate template ignored: deduced conflicting types for parameter 'Args' (<unsigned long long, unsigned long, PRNG *> vs. <unsigned long, unsigned long, PRNG *>)
void run(void f(Args...), Args... args)
^
../src/rt/test/func/cowngc4/cowngc4.cc:385:11: error: no matching member function for call to 'run'
harness.run(test_cown_gc<RegionType::Arena>, forward, ring, &rand);
~~~~~~~~^~~
../src/rt/./test/harness.h:73:8: note: candidate template ignored: deduced conflicting types for parameter 'Args' (<unsigned long long, unsigned long, PRNG *> vs. <unsigned long, unsigned long, PRNG *>)
void run(void f(Args...), Args... args)
^
2 errors generated.
[41/153] Building CXX object src/rt/CMakeFiles/func-sys-cowngc3.dir/test/func/cowngc3/cowngc3.cc.o
ninja: build stopped: subcommand failed.
The following crashes the compiler:
class B {
foo(self: mut) {}
}
class Main {
main() {
var b = new B;
var mutb = mut-view b;
while (1) {
mutb.foo();
};
}
}
with error:
Unhandled case N6verona8compiler20VariableRenamingTypeE in visitor N6verona8compiler19ReachabilityVisitor16CallReachabilityE.
The follow program:
use "list.verona"
class A {
_id: U64 & imm;
create(id: U64 & imm): A & iso {
var result = new A;
result._id = id;
result
}
final(self: mut) {
Builtin.print1("Finalising A: {:#}\n", self)
}
}
class Cell {
_pool: cown[Pool] & imm;
_f: A & iso;
create(pool: cown[Pool] & imm, f: A & iso): Cell & iso {
var result = new Cell;
result._pool = pool;
result._f = f;
result
}
final(self: mut) {
Builtin.print1("Finalising Cell: {:#}\n", self);
Pool.return(self._pool, self._f = A.create(1));
}
}
class Pool {
_f: A & iso;
create(): cown[Pool] & imm {
var result = new Pool;
cown.create(result)
}
return(pool: cown[Pool] & imm, f: A & iso) {
Builtin.print1("Returning A: {:#}\n", mut-view f);
when (var p = pool) {
// FIXME: at this point A stops existing
Builtin.print1("A: {:#}\n", mut-view f);
p._f = f;
}
}
}
class Main {
main() {
var pool = Pool.create();
var c = Cell.create(pool, A.create(0));
}
}
When run gives the following output:
Finalising Cell: Cell { cown[Pool](0x7fde1b010020), A { 0 } }
Returning A: A { 0 }
Finalising A: A { 0 }
fish: '../verona/build/dist/veronac ./…' terminated by signal SIGSEGV (Address boundary error)
The value passed to return
is finalised before it's passed into the behaviour.
The current strategy is to build all components (ast, mlir, opt, codegen, runtime) as libraries that can be used by different tools, and to have a number of executables that will use those libraries in different ways to achieve different tasks.
With each library with its own test driver, plus the main tools for compilation, we'll end up with a large number of tools, each with its own sub-set of options. To simplify testing and not confuse developers, it would be good if we had a clear set of goals regarding their command line interface.
-o
for output file, -S
for "text format", --emit-llvm
for LLVM bitcode, etc.--emit-mlir
and --emit-llvm
) and behave similarly (ex. the effect of -S
on both should be the same).-doThing
but --do-thing
).Have you ever heard of Nim language? It is also has a great c++ interface and Nim 1.0 came out recent time.
https://en.m.wikipedia.org/wiki/Venona_project
It's very similar. Is Microsoft using Rust to out spies in hi-tech research labs? 😅
It'd be nice to have an initial set of tests for the parser, from the existing examples in testsuite
that we kinda agree yields an "ok" AST, and add some checks for it.
Even if they're not what we ultimately want, any changes in the grammar or parser would come alongside changes in the test checks, which will make explicit the effect of the change.
This can avoid having to write long commit messages or referring to unlinked documentation.
As the AST dump is a simple tree, we can easily use OutputCheck
instead of FileCheck
for them, so no big changes needed.
First iteration using opaque MLIR and see what comes out of it. We still won't be able to lower that in any helpful way before we have a minimal dialect and the type inference working.
Acceptance criteria:
Not included:
Submodule at path src/rt/external/snmalloc
is at revision0f5cc16
, whereas upstream is at revision a43773c
.
You can update the submodule by running:
git submodule update --remote src/rt/external/snmalloc
Verona types are complex. Non-trivial pointer provenance, region encoding, mutability, data isolation, conditional execution all complicate the system. The language shouldn't have any more notation that is required to accurately infer all types at compile time, but that means a more complex type inference engine is needed.
Before we design that engine, we need to know how those types will be represented in MLIR, on both opaque and explicit representations.
Acceptance criteria:
Not included:
The final step in the conversion process is to turn Verona dialect into LLVM IR. The path is through a series of lowering steps, from Verona dialect to standard dialects, to LLVM dialect, to LLVM IR.
There shouldn't be many optimisation passes in between, mostly mechanical transformations and cleanups, but we may need to lower some special Verona cases directly into LLVM dialect, if standard MLIR dialects can't support it (for ex. printf and strings).
Acceptance criteria:
Potential goals:
Not included:
#include <test/harness.h>
struct MyCown : VCown<MyCown> {};
struct Msg : VAction<Msg>
{
MyCown* m;
Msg(MyCown* m) : m(m) {}
void f()
{
Systematic::cout() << "Msg on " << m << std::endl;
}
};
void run_test()
{
MyCown* t = new MyCown;
// HERE: the weak RC is never released.
t->weak_acquire();
Cown::schedule<Msg, YesTransfer>(t, t);
}
int main(int argc, char** argv)
{
SystematicTestHarness h(argc, argv);
h.run(run_test);
}
I would expect a debug_is_empty: found non-empty allocator
error. Instead the cown is collected twice, once when the strong RC hits zero and once on "teardown".
You can see in the log below the two Collecting: 0x10c010220
lines:
--cores 4
Seed: 5489
0 Set fair: 1
0 Object state :0x10c0100a0: EPOCH_NONE -> SCANNED
0 Object state :0x10c010100: EPOCH_NONE -> SCANNED
0 Object state :0x10c010160: EPOCH_NONE -> SCANNED
0 Object state :0x10c0101c0: EPOCH_NONE -> SCANNED
0 Epoch change on: 0x10c010220 : 7 -> EPOCH_A
0 Object state :0x10c010220: EPOCH_NONE -> EPOCH_A
0 0x10c0010a0 state change EPOCH_NONE -> EPOCH_NONE
0 MultiMessage 0x10c0010a0 payload 0x0 (EPOCH_NONE)
0 Schedule behaviour of type: 3Msg
0 MultiMessageBody 0x10c020020
0 0x10c0010d0 state change EPOCH_NONE -> EPOCH_A
0 MultiMessage 0x10c0010d0 payload 0x10c020020 (EPOCH_A)
0 MultiMessage 0x10c0010d0 index 0 fast requesting 0x10c010220
0 MultiMessage fast acquired 0x10c010220
0 MultiMessage fast send completed, rescheduling cown 0x10c010220
0 LIFO Scheduled Cown: 0x10c010220
0 Unpausing other threads.
0 Starting all threads
3 Pausing
4 Popped cown:0x10c010220
4 Scheduled Cown: 0x10c010220 (EPOCH_A)
4 Running Cown: 0x10c010220
4 Running Message 0x10c0010d0 on 0x10c010220
4 MultiMessage 0x10c0010d0 index 0 acquired 0x10c010220 epoch EPOCH_A
4 Msg on 0x10c010220
4 MultiMessage 0x10c0010d0 completed and running on 0x10c010220
4 Queue empty
4 Rescheduling Cown: 0x10c010220 (EPOCH_A)
2 Pausing
4 Scheduled Cown: 0x10c010220 (EPOCH_A)
4 Running Cown: 0x10c010220
4 Cown has no work this time:0x10c010220
4 Cown release: 0x10c010220
4 Cown dealloc: 0x10c010220
4 Collecting: 0x10c010220
4 Weak release 0x10c010220
4 Unscheduling Cown: 0x10c010220
4 Stub collect: 0x10c010220
4 Cown 0x10c010220 not outdated.
4 Cown 0x10c010220 has weak refs.
4 Pausing
1 Pausing
1 Teardown beginning
1 Teardown: all threads stopped
1 cv_notify_all() for teardown
1 Teardown: all threads beginning teardown
1 Begin teardown (phase 1)
1 End teardown (phase 1)
Unpausing
4 Begin teardown (phase 1)
4 Collecting: 0x10c010220
Unpausing
The program then ends with a SIGABRT, mostly from reading already freed garbage data.
Currently the implementation does not have many language features that are essential for building more realistic programs, and exercising the research. This issue is for tracking and prioritising new language features:
The following have been implemented in the runtime, but not exposed in the language:
class A { }
class B {
f: C & iso;
}
class C { }
class Main
{
main() {
var x = new A;
var y = new B in x;
y.f = new C;
Builtin.freeze(x);
}
}
Running this gives a debug_is_empty: found non-empty allocator
error.
The B and C objects are unreachable and should be collected on freeze. However only B is being collected.
From a quick look at rt/region/freeze.h
, there should be a find_iso_fields
call somewhere, probably in the while (!to_dealloc.empty())
loop, followed by a loop to release the found regions.
Hi everybody,
maybe this is not the right place to ask, but I want know where I can find the specifications for the programming language. Thank you.
With both dialect and type system designed, we can start coding the conversion infrastructure from opaque MLIR to Verona dialect. This task will probably need to occur in parallel with #105, and after #104 and #106.
We should have a series of partial conversions, with or without type inference, and finish with a pure Verona dialect (with possible embedded std/loop dialects) graph.
Acceptance criteria:
Not included:
The solver randomly goes into an infinite loop for this test.
I'm disabling it in #78 in order to not block the rest of the work.
I tried to build Verona today and all tests failed on master, which doesn't make sense.
When running ctest -R <test> -V
I noticed the error was:
9: CMake Error at .../verona/testsuite/common.cmake:11 (message):
9: To run tests you must build the install target.
It's not a widespread practice to require ninja install
before testing, but it's not uncommon either.
The best "fix" would be to create a ninja check
which depends on install
and just runs ctest
.
Another option would be to not require installing and take the binaries and libraries from the build directory.
Hello,
is there any timeline available for the language?
The bot to check if contributors have signed the Microsoft Contributor License Agreement isn't configured on this repo.
Happy hacking!
Title says a lot:
~/g/verona_test verona-interpreter main.verona
terminate called after throwing an instance of 'std::logic_error'
what(): Instruction overflow 6 8307
fish: “verona-interpreter main.verona” terminated by signal SIGABRT (Abort)
class Main {
main() {
Builtin.print2("Hello World!");
}
}
Weigh the pros and cons of using opaque MLIR as our AST instead of having a hand-coded AST structure.
Acceptance criteria:
Not included:
This issue contains the WIP gaps for the type system. There are several aspects of the formalisation that have not been implemented. This issues is intended to track the main omissions in the implementation:
mut
fields, should ensure the regions are the same, this is not currently enforced.iso
field, must invalidate references to sub-regions, and calls to trace
must invalidate references into the traced region, and sub-regions.readonly
capability.mut-view
: certain borrowing like operations are implicit, but not all. Currently, we require the programmer to insert mut-view
in many places to handle the lack of inference of capability coercions.Compiling the following:
class Cell[T] {
foo(self: mut) {}
}
class B {
bar(): Cell[U64Obj] & iso {
new Cell
}
baz() {
var a = B.bar();
(mut-view a).foo();
}
}
class Main {
main() {
B.baz();
}
}
gives the error:
terminate called after throwing an instance of 'std::out_of_range'
what(): map::at
It looks like the type Cell hasn't been visited in the Reachabilty pass as the following doesn't cause the same issue:
baz() {
baz() {
var c = new Cell;
B.refine(mut-view c);
var a = B.bar();
(mut-view a).foo();
}
}
Submodule at path src/rt/external/snmalloc
is at revision3775a62
, whereas upstream is at revision 8723ae2
.
You can update the submodule by running:
git submodule update --remote src/rt/external/snmalloc
Submodule at path src/rt/external/snmalloc
is at revision 49667c9
, whereas upstream is at revision dbfb645
.
Looking for ways to inject in Verona a DSL enabling concurrent neural network executions for:
jgraph/drawio#715
Exciting to see this released! Thanks for all the work so far.
In the interest of integrating with the Verona runtime from another compiler, I aim to wrap the API for C, so that it can be easily invoked from LLVM IR.
I have already started work on such a wrapper on my fork. It's nowhere near finished, but it's a start as I begin to digest the design of the runtime's API and figure out how to properly wrap it.
Is there interest in eventually merging such a C API into this repository after it's reasonably functional and polished up? If so, we could keep this ticket open and I could track my work here before eventually filing the PR.
If not, then we can close this ticket and I suppose I can make my own repo to house that work with this one as a submodule. Obviously I'd prefer to get it integrated here if possible, but I'll understand if you say it's beyond the focus of this repository at this time.
Thanks again!
We have other repos with documents, and some more spread across people's machines, none of which are easily accessible. We should create a docs
directory in this repository and add the relevant documents, even if still in draft, with a CMake step to build them (using something like texlive) and have them as a target for ninja docs
.
This could be part of the main build, or a separate step in the CI, it doesn't matter. As long as we can test that it convert docs to readable formats, it should be fine.
I'd go even further, and, after the document is reasonably stable (post-draft, pre-alpha), we could move the discussion of semantics online, as pull requests on Latex files. A pre-commit build would pick it up and build all the docs, as expected, and we could then download those pre-built documents to compare and review.
It would make for a much nicer review process, where we have the time to go through the changes before the meetings and only discuss the core points and questions. It would also make it easier to separate concerns for different block of changes into the same PR.
My environments are follows:
OS: Windows 10 Pro x64
System Locale: Japanese (CP932)
Compiler: Visual Studio 2019 16.4.3
CMAKE_CXX_FLAGS includes "/WX" in MSVC environment. It cause some compilation issues.
In my environment (CP932 encoding), cl.exe needs "/utf-8" option to compile sources.
6>F:\project\verona\src\.\compiler/ir/ir.h(1,1): error C2220: the following warning is treated as an error
6>F:\project\verona\src\.\compiler/ir/ir.h(1,1): warning C4819: The file contains a character that cannot be represented in the current code page (932). Save the file in Unicode format to prevent data loss
Exception handling needs "/EHsc" option.
8>F:\project\verona\external\CLI11\include\CLI/Option.hpp(674,1): error C2220: the following warning is treated as an error
8>F:\project\verona\external\CLI11\include\CLI/Option.hpp(674,1): warning C4530: C++ exception handler used, but unwind semantics are not enabled. Specify /EHsc
8>Done building project "interpreter-sys-bin.vcxproj" -- FAILED.
CMake configure with -DCMAKE_CXX_FLAGS="/EHsc /utf-8"
works fine for me.
// foo.verona
use "bar.verona"
// bar.verona
class A {
m(x: T) { } // Use a non-existent type T
}
Compiling foo.verona
will cause an error, as expected, but the location is incorrect:
$ ./dist/veronac foo.bar
foo.verona:0:6: error: Cannot find value for symbol 'T'
// foo.verona
^
1 error generated
The error should be at bar.verona:3:8
, but foo.verona:0:6
is reported instead.
Prior to #6 snmalloc was forced to be the latest version for any update to Verona by CI failing if it was not. This will not feasible going forward, so we should develop an alternative mechanism for ensuring we are tracking the latest version.
@plietar suggestion is to automatically create an issue if we are not up-to-date, so we can track the debt, and it can be resolved.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.