Once we have changed to a server architecture, we can implement the language server protocol, to allow popular editors such as Visual Studio Code to use our workbench.

Currently RemoveConstantPool comes after some simplified bytecode delta's that use the constant pool. I'd rather have these delta's make advantage of the RemoveConstantPool delta.

However, things like getSignature in InstructionC and getStacType in TypeInstance also dependent on the constant pool. Maybe all of these could switch to the inline constant pool, since RemoveConstantPool would be the first transformation.

Allow command line invocation of compilation. Include support for multiple file projects.

1. Change JavaLang so it only runs once.
2. Run a profiler
3. Add a workbench cache that contains workbench servers, so equivalent workbenches don't have to get created twice.

Ignored produce grammars only have an effect when printing, not when parsing, so it's nicer to optimize them out of the parser. An example of a grammar that produces this scenario is ~~:

def ~~(right: BiGrammar): BiGrammar = {
    (this <~ space) ~ right
}

The space between this and right has no effect for parsing.

Add a small optimization that among other things does:
Optimize(IgnoreRight(x, Produce y)) => Optimize(x)

1. Create a separate repository for the equation layout library
2. Upgrade the library by allowing the equations to be separated into equation sets, that can depend on other sets, but where the set dependencies form a DAG, so the end result is solvable. The equation set mechanism allows arbitrary function to operate on the results of one equation set before being fed into another equation set. This allows us to use max as we need for a table layout.
3. Deduce minimal equation sets from a big set of equations, by doing a bottom-up search for solvable sets.

This depends on the name binding language

Extend and change the bytecode syntax so it mimics javap, so when printing it contains constant and instruction indices, and when parsing also allows these indices.

Hey there! I noticed some possible problems in some code in this repo. A quick summary of a few of them is below, but let me know if you're interested in seeing a full report or talking about cloud security in general.

severity: serious

filename: ./modularLanguages/src/test/resources/AutoScalingMultiAZWithNotifications.yaml

line number(s): [503]

resource(s):

Missing egress rule means all traffic is allowed outbound. Make this explicit if it is desired configuration

severity: warning

filename: ./modularLanguages/src/test/resources/AutoScalingMultiAZWithNotifications.yaml

line number(s): [488]

resource(s):

Elastic Load Balancer should have access logging enabled

severity: warning

filename: ./modularLanguages/src/test/resources/AutoScalingMultiAZWithNotifications.yaml

line number(s): [503]

resource(s):

Security group rules without a description obscure their purpose and may lead to bad practices in ensuring they only allow traffic from the ports and sources/destinations required.

severity: warning

filename: ./modularLanguages/src/test/resources/AutoScalingMultiAZWithNotifications.json

line number(s): [430]

resource(s):

Security group rules without a description obscure their purpose and may lead to bad practices in ensuring they only allow traffic from the ports and sources/destinations required.

severity: warning

filename: ./languageServer/src/test/resources/AutoScalingMultiAZWithNotifications.yaml

line number(s): [488]

resource(s):

Elastic Load Balancer should have access logging enabled

If you do (x ^ y) ~ z then the printer prints in reading order x z y, while the parser parses x y z.

The semantics of the parser are incorrect here. The underlying issue is that Grammar (used for parsing) sees text as one-dimensional while Document (used for printing) has a two-dimensional concept of text.

If we add two-dimensional parsing capabilities to Grammar, then we can repair the BiGrammar invariant that parse(print(X)) = X

Another advantage of two-dimensional parsing is that it'll allow for parsing some context-sensitive languages that depend on indentation, like Haskell or Python.

There are instructions missing and also parts of the class file definition such as exceptions.

Enable defining delta's like this:

trait Compilation {
  def visit<T>(action: T => ()) {
  }
  def getFactory<T>: T
}

trait BlockStatementFactory {
  def create(statements: Seq[Node]): BlockStatement
}

trait BlockStatement {
  def statements: Seq[Node]
  def statements_(statements: Seq[Node]): Unit
}

object BlockDelta extends LanguageExtension {

  def factoryType = typeTag[BlockStatementFactory]

  def tranformProgram(compilation: Compilation) {
    val blockFactory = compilation.getFactory(factoryType)
    program.visit<BlockStatement>(block => {
      val newStatements = blockFactory.create(Seq.empty[Node])
      block.statements = Seq(newStatements);
    });
  }
}

Change the default bytecode syntax to be similar to javap, although without redundant information such a indices before the constants, or indices before instructions.

Enable dynamic method resolution for Node objects. Possibly this can even find singleton NodeField objects, by storing those in a registry that allows string based lookup. If that's not possible, we can go the route of a NodeFieldFromString.

Because we can, as it is useful outside of compilers

Add error-recovery to the parser. This allows a parse to continue after it has encountered an error, allowing multiple parse errors to be found in one document.

This builds on issue Add basic javap

Javap has a nice table format when printing, where the constants section uses columns for indices, constant names, and constant arguments. We don't currently support this in our printing because we don't have a BiGrammar for tables.

Add the Jasmin language as an example.

Jasmin is a simplified bytecode language, defined here: https://en.wikipedia.org/wiki/Jasmin_(software)

It serves as a great intermediate language for Java compilation.

First something unrelated to Delta's:

trait Language[Program] {
  def syntax: Syntax[Program]
  def semantics: Semantics[Program]
}

trait Pipeline[In, Core, Out] {
  def language: Language[Core]
  def inputSyntax: Syntax[In]
  def outputSyntax: Syntax[Out]
  def desugaring: In => Core
  def compilation: Core => Out
}

trait Syntax[Program] {
  def getParser
  def getPrinter
}

And then adding Delta's

trait LanguageDelta {
  def extendSyntax
  def extendSemantics
}

trait CompilationDelta {
  def transformOutputSyntax

  // Pass is added to the end of the pipeline
  def transformProgram
}

trait DesugaringDelta {
  def transformInputSyntax

  // Pass is added to the start of the pipeline
  def transformProgram
}

Currently all the workbench's capabilities are ran by starting the workbench, running the capability and then shutting down the workbench.

Switch this to a server architecture, where first the workbench is started, and then capabilities can be invoked on the running workbench. This has two performance advantages:

• Firstly, the language building phase, where the delta's are combined to create a language, only has to be run once.
• Secondly, the server can maintain a cache, which can be used to implement incremental compilation.

On to the future

Grammar path traversal is slow because:

• Methods like findAllPathsToKey traverse the entire graph
• It depends on reflection

Add an optimization that removes pop instructions when possible.

For example the following two instructions can be removed entirely, since all they do is load a value into the stack and then remove it from the stack:

iconst_0
pop

It's important to have an optimization that removes unnecessary pops, because they are generated by Java expressions used as statements, such as method calls with a side-effect where the return value is not used.

Doing this optimization completely and correctly is not trivial! It probably requires constructing a data-dependency graph and finding pairs of pop-pushy instructions in there to eleminate.

Slightly more complicated example than the first:

iconst_0
iconst_1
call method with side-effect that consumes 1 argument from the stack
pop

Desired result:

iconst_1
call method with side-effect that consumes 1 argument from the stack

If this optimization is implemented succesfully make sure to add a demonstration program.

88e9fe6d5b81:Miksilo$ git status
On branch master
Your branch is up-to-date with 'origin/master'.

nothing to commit, working tree clean

88e9fe6d5b81:Miksilo$ java -version
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

88e9fe6d5b81:Miksilo $ sbt run
[info] Loading settings from assembly.sbt ...
[info] Loading project definition from /Users/miksilo/Miksilo/project
[info] Loading settings from build.sbt ...
[info] Set current project to miksilo (in build file:/Users/miksilo/Miksilo/)
[error] java.lang.RuntimeException: No main class detected.
[error] at scala.sys.package$.error(package.scala:27)
[error] at sbt.Defaults$.$anonfun$bgRunTask$4(Defaults.scala:1163)
[error] at scala.Option.getOrElse(Option.scala:121)
[error] at sbt.Defaults$.$anonfun$bgRunTask$3(Defaults.scala:1163)
[error] at scala.Function1.$anonfun$compose$1(Function1.scala:44)
[error] at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:39)
[error] at sbt.std.Transform$$anon$4.work(System.scala:66)
[error] at sbt.Execute.$anonfun$submit$2(Execute.scala:262)
[error] at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
[error] at sbt.Execute.work(Execute.scala:271)
[error] at sbt.Execute.$anonfun$submit$1(Execute.scala:262)
[error] at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:174)
[error] at sbt.CompletionService$$anon$2.call(CompletionService.scala:36)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error] at java.lang.Thread.run(Thread.java:748)
[error] (Compile / bgRun) No main class detected.
[error] Total time: 0 s, completed Sep 22, 2018 3:08:05 PM

88e9fe6d5b81:Miksilo $ uname -a
Darwin 88e9fe6d5b81.my-mac.my-company.com 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 21 20:07:39 PDT 2018; root:xnu-3789.73.14~1/RELEASE_X86_64 x86_64

Add a delta that inlines the constant pool to get a simpler bytecode langauge.

The Jasmin language also does this.

Colouring schemes are often based on name binding, so maybe this should be integrated with the NaBL language.

Add incremental parsing, which allows for fast re-parsing on partial textual changes. This is required for IDE support. See this paper for more details.

Add a DSL similar to the NaBL language.

Work has already been done in: https://github.com/keyboardDrummer/NaBLClone

Given a Delta chain with missing dependencies, automatically add the missing ones. However, for this to work well it might be that more information than just the dependencies property is required.

Looks like we need to use a specific scala version?

88e9fe6d5b81:playground avidane$ sbt run
[warn] No sbt.version set in project/build.properties, base directory: /Users/avidane/miksilo/Miksilo/playground
[info] Set current project to playground (in build file:/Users/avidane/miksilo/Miksilo/playground/)
[info] Updating ...
[info] Done updating.
[info] Compiling 50 Scala sources and 1 Java source to /Users/avidane/miksilo/Miksilo/playground/target/scala-2.12/classes ...
[error] /Users/avidane/miksilo/Miksilo/playground/src/main/scala/application/Program.scala:10:12: object mxgraph is not a member of package com
[error] import com.mxgraph.swing.mxGraphComponent
[error]            ^
[error] /Users/avidane/miksilo/Miksilo/playground/src/main/scala/application/Program.scala:11:12: object oxbow is not a member of package org
[error] import org.oxbow.swingbits.dialog.task.TaskDialogs
[error]            ^
[error] /Users/avidane/miksilo/Miksilo/playground/src/main/scala/application/Program.scala:13:14: object swing is not a member of package scala
[error] import scala.swing.{Component, MainFrame, SimpleSwingApplication}

See this wiki page for the specification