Giter VIP home page Giter VIP logo

mill's Introduction

Mill Logo Mill

Build and Release Gitter Chat Mill Patreon

Your shiny new Scala build tool! Confused by SBT? Frustrated by Maven? Perplexed by Gradle? Give Mill a try!

Table of Contents

Documentation

If you want to use Mill in your own projects, check out our documentation:

Here is some quick example, so that you can imagine how it looks:

import mill._, scalalib._

object foo extends ScalaModule {
  def scalaVersion = "3.3.4"
}

object bar extends ScalaModule {
  def moduleDeps = Seq(foo)
  def scalaVersion = "3.3.4"
}

If you use Mill and like it, you will probably enjoy the following book by the Author:

The remainder of this readme is developer-documentation targeted at people who wish to work on Mill’s own codebase. The developer docs assume you have read through the user-facing documentation linked above. It’s also worth spending a few minutes reading the following blog posts to get a sense of Mill’s design & motivation:

Mill is profiled using the JProfiler Java Profiler, by courtesy of EJ Technologies.

Mill is built using Mill. To begin, you just need a JVM installed, and the ./mill script will be sufficient to bootstrap the project.

IntelliJ Setup

If you are using IntelliJ IDEA to edit Mill’s Scala code, you can create the IntelliJ project files via:

./mill -j 0 mill.idea.GenIdea/idea

Manual & Automated Testing

The following table contains the main ways you can test the code in com-lihaoyi/mill, via manual testing or automated tests:

Config

Automated Testing

Manual Testing

Manual Testing CI

In-Process Tests

main.__.test, scalalib.test, contrib.buildinfo.test, etc.

Sub-Process w/o packaging/publishing

example.__.local, integration.__.local

dev.run

test-mill-dev.sh

Sub-Process w/ packaging/publishing

example.__.server, integration.__.server

dev.assembly

test-mill-release.sh

Bootstrapping: Building Mill with your current checkout of Mill

installLocal

test-mill-bootstrap.sh

In-Process Tests

In-process tests live in the .test sub-modules of the various Mill modules. These range from tiny unit tests, to larger integration tests that instantiate a TestUtil.BaseModule in-process and a TestEvaluator to evaluate tasks on it.

Most "core" tests live in main.__test; these should run almost instantly, and cover most of Mill’s functionality that is not specific to Scala/Scala.js/etc.. Tests specific to Scala/Scala.js/Scala-Native live in scalalib.test/scalajslib.test/scalanativelib.test respectively, and take a lot longer to run because running them involves actually compiling Scala code.

The various contrib modules also have tests in this style, e.g. contrib.buildinfo.test

Note that the in-memory tests compile the BaseModule together with the test suite, and do not exercise the Mill script-file bootstrapping, transformation, and compilation process.

Sub-Process Tests without Packaging/Publishing

example.__.local and integration.__.local tests run Mill end-to-end in a subprocess, but without the expensive/slow steps of packaging the core packages into an assembly jar and publishing the remaining packages to ~/.ivy2/local. You can reproduce these tests manually using ./mill dev.run <test-folder-path> <command>.

example tests are written in a single build.sc file, with the test commands written in a comment with a bash-like syntax together with the build code and comments that explain the example. These serve three purposes:

  1. Basic smoke-tests to make sure the functionality works at all, without covering every edge case

  2. User-facing documentation, with the test cases, test commands, and explanatory comments included in the Mill documentation site

  3. Example repositories, that Mill users can download to bootstrap their own projects

The integration tests are similar to example tests and share most of their test infrastructure, but with differences:

  1. integration tests are meant to test features more thoroughly then example tests, covering more and deeper edge cases even at the expense of readability

  2. integration tests are written using a Scala test suite extending IntegrationTestSuite, giving more flexibility at the expense of readability

You can reproduce any of the tests manually using dev.run, e.g.

Automated Test

./mill "example.basic[1-simple].local"

Manual Test

./mill dev.run example/basic/1-simple run --text hello

Manual Test using Launcher Script

./mill dev.launcher && (cd example/basic/1-simple && ../../../out/dev/launcher.dest/run run --text hello)

Sub-Process Tests with Packaging/Publishing

example.__.server, integration.__.server, example.__.fork and integration.__.fork cover the same test cases as the .local tests described above, but they perform packaging of the Mill core modules into an assembly jar, and publish the remaining modules to ~/.ivy2/local. This results in a more realistic test environment, but at the cost of taking tens-of-seconds more to run a test after making a code change.

You can reproduce these tests manually using dev.assembly:

./mill dev.assembly && (cd example/basic/1-simple && ../../../out/dev/assembly.dest/mill run --text hello)

There are two flavors of these tests:

  1. .server test run the test cases with the default configuration, so consecutive commands run in the same long-lived background server process

  2. .fork test run the test cases with --no-server, meaning each command runs in a newly spawned Mill process

In general you should spend most of your time working with the .local version of the example and integration tests to save time, and only run .fork or .server once .local is passing.

Bootstrapping: Building Mill with your current checkout of Mill

To test bootstrapping of Mill’s own Mill build using a version of Mill built from your checkout, you can run

./mill installLocal
ci/patch-mill-bootstrap.sh

This creates a standalone assembly at target/mill-release you can use, which references jars published locally in your ~/.ivy2/local cache, and applies any necessary patches to build.sc to deal with changes in Mill between the version specified in .config/mill-version that is normally used to build Mill and the HEAD version your assembly was created from. You can then use this standalone assembly to build & re-build your current Mill checkout without worrying about stomping over compiled code that the assembly is using.

This assembly is design to work on bash, bash-like shells and Windows Cmd. If you have another default shell like zsh or fish, you probably need to invoke it with sh ~/mill-release or prepend the file with a proper shebang.

If you want to install into a different location or a different Ivy repository, you can set its optional parameters.

Install into /tmp
$ ./mill installLocal --binFile /tmp/mill --ivyRepo /tmp/millRepo
...
Published 44 modules and installed /tmp/mill

Troubleshooting

In case of troubles with caching and/or incremental compilation, you can always restart from scratch removing the out directory:

os.remove.all -rf out/

Project Layout

The Mill project is organized roughly as follows:

Core modules that are included in the main assembly

  • runner, main.*, scalalib, scalajslib, scalanativelib.

These are general lightweight and dependency-free: mostly configuration & wiring of a Mill build and without the heavy lifting.

Heavy lifting is delegated to the worker modules (described below), which the core modules resolve from Maven Central (or from the local filesystem in dev) and load into isolated classloaders.

Worker modules that are resolved from Maven Central

  • scalalib.worker, scalajslib.worker[0.6], scalajslib.worker[1.0]

These modules are where the heavy-lifting happens, and include heavy dependencies like the Scala compiler, Scala.js optimizer, etc.. Rather than being bundled in the main assembly & classpath, these are resolved separately from Maven Central (or from the local filesystem in dev) and kept in isolated classloaders.

This allows a single Mill build to use multiple versions of e.g. the Scala.js optimizer without classpath conflicts.

Contrib modules

  • contrib/bloop/, contrib/flyway/, contrib/scoverage/, etc.

These are modules that help integrate Mill with the wide variety of different tools and utilities available in the JVM ecosystem.

These modules are not as stringently reviewed as the main Mill core/worker codebase, and are primarily maintained by their individual contributors. These are maintained as part of the primary Mill Github repo for easy testing/updating as the core Mill APIs evolve, ensuring that they are always tested and passing against the corresponding version of Mill.

Compatibility & Stability

Mill maintains backward binary compatibility for each major version (major.minor.point), enforced with Mima, for the following packages:

  • mill.api

  • mill.util

  • mill.define

  • mill.eval

  • mill.resolve

  • mill.scalalib

  • mill.scalajslib

  • mill.scalanativelib

Other packages like mill.runner, mill.bsp, etc. are on the classpath but offer no compatibility guarantees.

Currently, Mill does not offer compatibility guarantees for mill.contrib packages, although they tend to evolve slowly. This may change over time as these packages mature over time.

Project Maintenance

Pull Requests

  • Changes to the main branch need a pull request. Exceptions are preparation commits for releases, which are meant to be pushed with tags in one go

  • Merged pull request (and closed issues) need to be assigned to a Milestone

  • Pull requests are typically merged via "Squash and merge", so we get a linear and useful history

  • Larger pull request, where it makes sense to keep single commits, or with multiple authors may be committed via merge commits.

Commit Messages

  • The title should be meaningful and may contain the pull request number in parentheses (typically automatically generated on GitHub)

  • The description should contain additional required details, which typically reflect the content of the first PR comment

  • A full link to the pull request should be added via a line: Pull request: <link>

  • If the PR has multiple authors but is merged as merge commit, those authors should be included via a line for each co-author: Co-authored-by: <author>

  • If the message contains links to other issues or pull requests, you should use full URLs to reference them

Changelog

0.11.9 - 2024-07-18

Changes since 0.11.8:

New features

  • First class support for Java projects #3261

  • Add coarse grained lock around BSP request to avoid deadlocks #3243

0.11.8 - 2024-06-21

Changes since 0.11.7:

New features

  • Support Scala Native 0.5.0 #3054 #3107 #3120

  • Add jvmOptions to docker contrib configuration #3079

  • Pass auxiliary class files to zinc, so they are deleted together #3072

  • BSP: Handle new JvmCompileClasspath request #3086

  • Add support for Cobertura XML report task to help integration #3093

  • Support Scala.js minify via scalaJSMinify: Target[String] #3094

  • Restructure TestModule, add RunModule #3064

  • Move run-targets into RunModule #3090

  • TestModule: Support generation of JUnit-compatible xml report #3099 #3172 #3135 #3184

  • Use docker image hash to determine if build should rerun #3124

  • Add multi platform support to contrib.docker through docker buildx #3143

  • Add ability to re-write ESModule imports at link time #3109

  • Adding sonatype central support #3130 #3187

  • Add TestModule.discoveredTestClasses target to ease test inspection #3191

  • Support "pom" packaging type in PublishModule #3222

Fixes and Improvements

  • Restructure VersionFinder, increase speed, added ticker messages #3014

  • Fix bsp compile classpath inconsistencies #3017

  • Bring more bsp-specific targets in line with their non-bsp versions #3018

  • Make BSP task processing more resilient #3022

  • Update example build.sc with current library versions #3055

  • Scoverage: Do not include the scalac plugin in the reporting classpath #3060

  • Add Scala 2.13.13 and 2.12.19 to supported bridges list #3066

  • Fix BuildInfo static Java file generator #3074

  • Scoverage inner module defaults to skipIdea = outer.skipIdea #3098

  • Deduplicate input tasks in inspect command #3102

  • CI: add scalafix-check to precondition to publish #3095

  • Docs for Continuous cache invalidation #3096

  • Isolate scoverage modules from their parent modules #3118

  • Scoverage improve classpath override to the minimal #3122

  • Improve internal BSP API and docs, fix a match error condition #3111

  • Retry example integration tests that didn’t finish after 5 minutes #3125

  • Fix mill script for parameter starting with -i #3127

  • Fixed semanticdb file copying issue, added more tests #3080

  • Detect assemblies with too many entries to fail shell script prepending #3140

  • Exclude mill provided dependencies in meta builds #3189 #3221

  • Fix Scala.js toolchain logs in server-client mode #3196

  • Fix compiler bridge build setup and build essential versions in CI #3179

  • Add Scala 2.13.14 to bridgeScalaVersions #3166

  • GenIdea improvements #3153

  • GenIdea: Use synthetic scala-SDK entry for compiler setup #3154

  • Fix classpath regression in Bloop export #1918 #3211

  • Support build.sc in Bloop #3208

  • Fail if testOnly does not match any test case #3224

Updates and internal changes

  • Code cleanups

  • Documentation updates and fixes

  • Test and CI improvements

  • Updates: acyclic 0.3.12, Ammonite 3.0.0-M2-11-713b6963, asm 9.7, commons-compress 1.26.2, commons-io 2.16.1, coursier 2.1.10, fastparse 3.1.0, fansi 0.5.0, guava 33.2.1, jarjar-abrams-core 1.14.0, jline 3.26.2, junixsocket 2.9.1, log4j-core 2.23.1, mainargs 0.7.0 mill-mima 0.1.1, mill-scalafix 0.4.0, os-lib 0.10.2, Play 2.8.22 / 2.9.4 / 3.0.4, pprint 0.9.0, protobuf-java 3.25.3, Scala 2.12.19 / 2.13.14, Scala Native 0.4.17 / 0.5.3, scala-xml 2.3.0, scalacheck 1.18.0, scalatest 3.2.18, scoverage 2.1.1, semanticdb-java 0.9.10, semanticdb-scalac 4.9.7 requests 0.8.3, upickle 3.3.1, utest 0.8.3, zinc 1.10.0

For details refer to milestone 0.11.8 and the list of commits.

0.11.7 - 2024-02-05

Changes since 0.11.6:

New features

  • Support type selectors as path selection syntax in resolve #2997, #2998

  • Read coursier default config files to set up repositories and support mirror configuration #2886, #2917

  • Added support for type attribute when parsing dependencies #2994

  • Add new ScalaModule.scalacHelp command #2921

  • Add a ScalaModule.consoleScalacOptions target #2948

  • Enable colored output for Scala 2 compiler output #2950

  • Add publishLocalCached and publishM2LocalCached targets to PublishModule #2976

  • Support Scala Native build target. This allows to build static and dynamic libraries other than binaries #2898

  • Prepare the Scala.js integration for SIP-51 #2988

Fixes and Improvements

  • Better detect Windows Subsystem for Linux environments #2901

  • Avoid evaluating `T.input`s twice #2952

  • Deduplicate (anonymous) tasks in results #2959

  • Synchronize evaluateGroupCached to avoid concurrent access to cache #2980

  • Properly sanitize Windows reserved names and symbols in evaluator paths #2964, #2965

  • Detect colliding cross module values #2984

  • Mask forward slashes in cross values to fix their cache locations #2986

  • Re-enable test result summary (print done message from test framework to stdout) #2993

  • Fix overeager caching of cliImports value in generatedScriptSources #2977

  • Allow resolving moduleDeps with older Scala 3 versions #2877

  • GenIdea: Put module dependencies after library dependencies #2925

  • BSP: do not filter clean-requests for meta-builds #2931

  • BSP: Add JavaModule.bspBuildTargetData to make JavaModule reports workable BuildTarget #2930

  • BSP: Send logMessage instead of diagnostics when textDocument is unknown #2979

  • Scoverage inner module defaults new to skipIdea = true #2989

Updates and internal changes

  • Dependency updates: asm-tree 9.6, bsp4j 2.2.0-M1, coursier 2.1.8, jline 3.25.0, jna 5.14.0, junixsocket-core 2.8.3, log4j-core 2.22.1, mainargs 0.6.1, os-lib 0.9.3, scalafmt 3.7.15, Scala.js 1.15.0, scala-native 8.4.16, semanticdb-java 0.9.8, semanticdb-scala 4.8.15, upickle 3.1.4, zinc 1.9.6

  • Contrib dependency updates: Play 2.8.21, Play 2.9.1, Play 3.0.1

  • Documentation updates and new sections

  • More code cleanups, explicit result types and enforcement of some code quality metrics via mill-scalafix

For details refer to milestone 0.11.7 and the list of commits.

0.11.6 - 2023-11-21

Changes since 0.11.5:

  • Make PathRef robust against concurrent filesyste modifications #2832

  • Use logger error stream for informational messages #2839

  • Harden assembly logic against Zip-Slip vulnerabilities #2847

  • Add an option to disable incremental compilation with zinc #2851

  • Add check for right Tests traits in ScalaJS and Native #2874

  • Attempt to recover from client/server connection errors to #2879

  • Fix discovery of targets whose names get mangled #2883

  • Make mill show skip -j prefixes to ensure machine readability #2884

For details refer to milestone 0.11.6 and the list of commits.

0.11.5 - 2023-10-04

Changes since 0.11.4:

  • Support for Java 21 #2768

  • Various BSP improvements #2814, #2813, #2810, #2771

  • The T.workspace context path now always points to the top-level project directory, also for meta builds #2809

  • Mill now better detects and reports logical cycles in inter-module dependencies #2790

  • Fixed semanticDB data generation for meta builds #2809

  • The prepareOffline command also fetches relevant compiler-bridges for Scala modules #2791

  • Improved ScalaJSModule and added support for IRFileCache #2783

  • The JavaModule.zincReportCachedProblems configuration can now also customized via a Java system property #2775

  • Fixed a file truncation issue in protobuf module and print a warning when proto file get overwritten #2800

  • Documentation improvements

  • Dependency updates: bsp4j 2.1.0-M7, castor 0.3.0, coursier-interface 1.0.19, jarjarabrams 1.9.0, jline 3.23.0, junitsocket 2.8.1, mainargs 0.5.4, scalafmt 3.7.14, Scala.js 1.14.0, semanticdb-java 0.9.6, semanticdb-scala 4.8.10

  • Various other improvements and cleanups

For details refer to milestone 0.11.5 and the list of commits.

0.11.4 - 2023-09-19

Changes since 0.11.3:

  • Fix binary incompatibility issue with Discover macro’s generated generic code #2749

  • Support the release-size mode in ScalaNativeModule #2754

For details refer to milestone 0.11.4 and the list of commits.

0.11.3 - 2023-09-17

Changes since 0.11.2:

  • Allow Mill CLI to select the meta-build frame it operates on via --meta-level <n> #2719

  • Improve the mill resolve suggestion when a user specifies a target in the wrong module #2731

  • Fix conflicting dependencies between upstream JavaModules #2735

  • Fix the scala-library dependency for (generic) platform modules #2739

  • Fix terminal forwarding in .console and .repl commands #2743

For details refer to milestone 0.11.3 and the list of commits.

0.11.2 - 2023-08-28

Changes since 0.11.1:

  • Target invalidation after making changes to build.sc is now done at a fine-grained method-level granularity, using callgraph reachability analysis to see which targets depend on the code that was changed. See #2417 for more details

  • Fix redirection of stdout stream to stderr when using show #2689

  • Fix line numbers in errors for scripts starting with leading comments or whitespace #2686

  • Fix init command and support runnig Mill without existing build.sc file #2662

  • Fixes for BSP editor integration sometimes using the wrong output folder for meta-build metadata, causing subsequent builds from the command line to fail #2692

  • Added a new mill.idea.GenIdea/idea command to generate IntelliJ IDE metadata, improving-upon and replacing the older mill.scalalib.GenIdea/idea command which is now deprecated #2638

  • Update Coursier to 2.1.6 to mitigate CVE CVE-2022-46751 #2705

For details refer to milestone 0.11.2 and the list of commits.

0.11.1 - 2023-06-23

Changes since 0.11.0:

  • mill.define.Cross: Introduced default cross segments and some CLI convenience

  • mill.testrunner: Fixed exception when running ScalaTest or ZIOTest frameworks

  • mill.runner: Removed some obsolete / defunct cli options

  • mill.runner: Properly distinct-ify commands when resolving wide matching target patterns

  • mill.scalajslib: Restored correct defaults for esFeature

  • mill.bsp: Fixed a MatchError in the buildtarget/scalaTestClasses request

  • mill.contrib.bloop: Corrected accidentally changed package name resulting in non-functional plugin

  • mill.contrib.scoverage: Fixed defunct plugin due to a missing type annotation

  • Various internal improvements

  • Dependency updates: Ammonite 3.3.0-M0-32-96e851cb, bsp4j 2.1.0-M5, zinc 1.9.1

  • Mill is now build with Mill 0.11

For details refer to milestone 0.11.1 and the list of commits.

0.11.0 - 2023-06-06

This release is binary incompatible to 0.11.0-M11.

Changes since 0.11.0-M11:

  • 0.11.0 is the next breaking version after the 0.10.x series, with a large number of improvements. See the changelog below for 0.11.0-M1 to 0.11.0-M11 for a full list of user-facing changes.

For details refer to milestone 0.11.0 and the list of commits.

Older releases

0.11.0-M11 - 2023-06-04

This release is binary incompatible to 0.11.0-M10.

Changes since 0.11.0-M10:

  • Make foo.test command run tests with user code in the boot classloader, rather than in a sub-classloader #2561

  • Mill backend server timeout is now configurable #2550

  • Mill assembly is now distributed via Maven Central, rather than Github release assets, to remove an unnecessary single point of failure #2560

  • Tests inner trait was removed, to avoid trait shadowing which will be removed in Scala 3. Please use ScalaTests, ScalaJSTests, or ScalaNativeTests instead #2558

For details refer to milestone 0.11.0-M11 and the list of commits.

0.11.0-M10 - 2023-05-24

This release is binary incompatible to 0.11.0-M9.

Changes since 0.11.0-M9:

  • Make mill.define.Module a trait to allow abstract/virtual modules to be traits rather than classes #2536

  • Move mill.BuildInfo to mill.main.BuildInfo to avoid name conflicts with mill.contrib.buildinfo.BuildInfo #2537

  • PlatformScalaModule now exposes platformScalaSuffix for user code to use #2534

  • Add Agg.when operator to simplify common workflow of adding optional flags or command line parameters #2353

  • Generalize handling of test module source folder layout, such that they always match the folder layout of the enclosing module #2531

For details refer to milestone 0.11.0-M10 and the list of commits.

0.11.0-M9 - 2023-05-18

This release is binary incompatible to 0.11.0-M8.

Changes since 0.11.0-M8:

  • Overhauled target resolution logic. It is now significantly lazier, resulting in less of the module tree being un-necessarily instantiated, and also more precise and predictable #2453 #2511

  • Allow tasks to be passed as the CLI parameters of the run command, allowing run to be easily used in the implementation of other tasks #2452

  • T.inputs are now watched properly with --watch, and trigger re-evaluations when the watched value changes #2489

  • Support for Java 20 #2501

  • Broke up mill.modules package #2513, with functionality re-distributed to mill.util and mill.scalalib

  • Overhaul BSP-related code, for improved fidelity and correctness #2415 #2414 #2518 #2521

  • Enabled compilation warnings in build.sc #2519

  • Print out the CLI flags when inspecting T.commands #2522

For details refer to milestone 0.11.0-M9 and the list of commits.

0.11.0-M8 - 2023-04-24

This release is binary incompatible to 0.11.0-M7.

Changes since 0.11.0-M7:

  • Added an example/ folder in the Mill repo, containing common build setups demonstrating Mill features with explanations of how each feature works

  • Pre-compiled Scala incremental compiler interface #2424, to speed up clean builds

  • Add some helpers to simplify cross-version/cross-platform modules {#2406}[#2406]

  • You can now override T{…​} Targets with T.source or T.sources, and vice versa #2402

  • Removed the Ammonite script runner dependency used to evaluate build.sc files and instead compile them using Mill #2377

  • Add TestModule.ZioTest #2432

  • Caching fixes for external modules #2419

  • Overhaul of the Mill BuildInfo plugin, moving the key-value into resources to avoid needing to re-compile your module when the values change, adding JavaModule support, and allowing Javadoc/Scaladoc comments to be associated with the generated BuildInfo constants #2425

  • Global Configuration via ~/.mill/ammonite/predefScript.sc is no longer supported in this version. If that breaks your workflow, please report and tell us your use case so we can provide sufficient replacement or support for your use case before Mill 0.11.

  • Overhaul of the documentation. Created many executable example projects which are included in the documentation and run/tested on CI

  • Change cross module definitions to be traits instead of classes, for greater regularity and less builerplate at call sites. This change requires slight modification to existing build scripts that define cross modules.

For details refer to milestone 0.11.0-M8 and the list of commits.

0.11.0-M7 - 2023-03-30

This release is binary incompatible to 0.11.0-M6.

Changes since 0.11.0-M6:

  • Introduced automatic PathRef validation for cached targets; default-enabled it for CoursierModule.resolveDeps and various resolvedIvyDeps targets

  • bsp: Update Protocol version to 2.1.0-M4

  • bsp: Support new mainClasses field in run and test environments

  • bsp: Fixed handling of Mill plugins and other improvements

  • scalanativelib: new nativeDump setting in ScalaNativeModule

  • contrib.twirllib: Use newer scala-parser-combinators version when used with Scala 3

  • contrib.scalapblib: Added new flag to search for proto files in dependencies

  • Various refactorings to improve binary compatibility

  • Updated dependencies: Ammonite 3.0.0-M0-5 coursier 2.1.0, scala native tools 0.4.12, semanticdb 4.7.6, trees 4.7.6, upickle 3.0.0

  • DX improvements

For details refer to milestone 0.11.0-M7 and the list of commits.

0.11.0-M6 - 2023-03-09

This release is binary incompatible to 0.11.0-M5.

Changes since 0.11.0-M5:

  • main: Re-added missing --color and predef cli arguments.

For details refer to milestone 0.11.0-M6 and the list of commits.

0.11.0-M5 - 2023-03-09

This release is binary incompatible to 0.11.0-M4.

Changes since 0.11.0-M4:

  • Cross is no longer adding the cross parameters to the millSourcePath.

    You should review your cross modules setups to avoid build issues like incorrect source paths or missing files. CrossScalaModule is not affected by this change.

  • API refactorings: PathRef, moved JarManifest to mill.main

  • No longer inherit the Ammonite CLI config

  • scalalib: Fixed loosing customized mapDependencies when ScalaModule get mixed in after

  • scalalib: New TestModule.Weaver

  • scalajslib: New JsEnvConfig.Selenium

  • testrunner: Fixed concurrency issue with test event reporting

  • Updated dependencies: ammonite 3.0.0-M0-3, coursier 2.1.0-RC6, jarajar-abrams-core 1.8.2, lambdatest 0.8.0, log4j-core 2.20.0, os-lib 0.9.1, scoverage 2.0.8, semanticdb-scalac 4.7.5, trees 4.7.5

  • Documentation updates

For details refer to milestone 0.11.0-M5 and the list of commits.

0.11.0-M4 - 2023-02-10

This release is binary incompatible to 0.11.0-M3.

Changes since 0.11.0-M3:

  • scalalib: New configuration target zincReportCachedProblems which, when true, enables reporting of all found problems, even for files which were not re-compiled

  • scalalib: Improved SemanticDB support for Java and Scala

  • scalalib: Mitigate another coursier download issue

  • scalajslib: Fetch more tooling dependencies in prepareOffline

  • scalanativelib: Fetch more tooling dependencies in prepareOffline

  • scalanativelib: Updated tools to version 0.4.10

  • bsp: Improved support for the mill-build module which should improve the editing experience for build.sc and other build files

  • Cleanups and internal improvements

  • Updated dependencies: Ammonite 3.0.0-M0-1, junixsocket 2.6.2, semanticdb-java 0.8.13, upickle 3.0.0-M2

For details refer to milestone 0.11.0-M4 and the list of commits.

0.11.0-M3 - 2023-01-23

This release is binary incompatible to 0.11.0-M2.

Changes since 0.11.0-M2:

  • Slightly changed the out/ folder layout for overridden and private tasks. Instead of foo.overridden we now use a foo.super path to hold the metadata and scratch files.

  • Fixed the caching for private targets with same name but defined in different super traits.

  • Fixed non-functional clean command when used with arguments denoting modules.

  • scalalib: Fixed GenIdea issues on Windows, when the build uses plugins or additional libraries.

  • scalajslib: ScalaJSModule.ScalaJSTests now extends ScalaModule.ScalaTests which improves consistency, better default values and compatibility with other modules like ScoverageModule.

  • scalanativelib: ScalaNativeModule.ScalaNativeTests now extends ScalaModule.ScalaTests which improves consistency, better default values and compatibility with other modules.

  • contrib.gitlab: Improved error handling for token lookup and documentation.

  • Updated dependencies: coursier 2.1.0-RC5, jna 5.13.0, semanticdb-scala 4.7.3, trees 4.7.3

  • Documentation improvements

For details refer to milestone 0.11.0-M3 and the list of commits.

0.11.0-M2 - 2023-01-13

This release is binary incompatible to 0.11.0-M1.

Changes since 0.11.0-M1:

  • Splitted BSP module into mill.bsp and mill.bsp.worker and removed various dependencies (guava, bsp4j, xtends, lsp4j, …​) from Mill API

  • scalalib: Added support to generate semanticdb data for Java source files

  • scalajslib: Added support for scalaJSOutputPattern

  • scalajslib: Added suport for scalaJSSourceMap

  • scalajslib: Dropped support for Scala.js 0.6

  • Updated dependencies: ammonite 2.5.6, coursier 2.1.0-RC4, semanticdb 4.7.1, requests 0.8.0, scalafmt 3.6.1, trees 4.7.1, upickle 3.0.0-M1, utest 0.8.1

  • Various internal cleanups and improvements

  • Documentation improvements

For details refer to milestone 0.11.0-M2 and the list of commits.

0.11.0-M1 - 2022-12-17

This release is binary incompatible to 0.10.10.

Changes since 0.10.10:

  • Greatly improved dependency resolution between modules, making it possible to mix Java and Scala modules

  • main: Restrict Cross parameter to be of type Module

  • scalalib: Performance improvements in Zinc worker module

  • scalalib: Resources are no longer added to compileClasspath.

  • scalalib: Added new compileResources which will end up in compileClasspath

  • scalalib: Consolidated artifact and platform suffix handling

  • scalajslib : Performance improvements in worker module

  • scalanativelib : Performance improvements in worker module

  • Updated dependencies: coursier 2.1.0-RC3, os-lib 0.9, scala 2.12.17, trees 4.7.0

  • Removed lots of deprecated API

  • Various internal cleanups and improvements

For details refer to milestone 0.11.0-M1 and the list of commits.

0.10.15 - 2024-01-07

Changes since 0.10.13:

  • Make semanticDbEnablePluginScalacOptions protected and thus accessible for downstream use and customization

For details refer to milestone 0.10.15 and the list of commits.

0.10.13 - 2023-11-12

Changes since 0.10.12:

This version especially aids the transition from Mill 0.10 to Mill 0.11.

  • Backport of Java 20 compatibility

  • Deprecated mill.define.SelectMode.Single

  • Backport of mill.eval.Evaluator.evalOrThrow

  • Deprecated all inner Tests traits of modules derived from JavaModule and provide type aliases to use their successors

  • Dependency updates: Ammonite 2.5.11, Coursier 2.1.7, jarjar-abrams 1.9.0, Junixxocket 2.8.2, Play 2.8.21, Scalameta Trees 4.8.12, Scala Native 0.4.16, Scoverage 2.0.11, Zinc 1.9.5

For details refer to milestone 0.10.13 and the list of commits.

0.10.12 - 2023-03-20

Changes since 0.10.11:

  • scalalib: Mitigate another coursier download issue (backported from 0.11.0-M4)

  • testrunner: Fixed a concurrency issue with test event reporting (backported from 0.11.0-M5)

  • scalajslib: Deprecated fastOpt and fullOpt

  • Updated dependencies: coursier 2.1.0, jarjar-abrams 1.8.2, jna 5.13.0, junixsocket 2.6.2, log4j-core 2.20.0, scalafmt-dynamic 3.6.1, trees 4.7.6

For details refer to milestone 0.10.12 and the list of commits.

0.10.11 - 2023-01-24

Changes since 0.10.10:

  • Fixed non-functional clean command when used with arguments denoting modules.

  • scalalib: Fixed GenIdea issues on Windows, when the build uses plugins or additional libraries.

  • scalajslib: Performance improvements in worker module

  • scalajslib: Pass more settings from ScalaJSModule to its Tests

    If you have issues after the update (like org.scalajs.jsenv.UnsupportedInputException, #2300), check your settings in the test modules for consistency.

  • Some internal improvements backported from 0.11.0-M3

For details refer to milestone 0.10.11 and the list of commits.

0.10.10 - 2022-12-06

After this release, the main branch is open for development of the next major version of Mill, which is probably 0.11. Further maintenance work will happen in the 0.10.x branch.

Changes since 0.10.9:

  • Lots of documentation improvements, additions and restructurings

  • core: Fixed some rare issues with reading the .mill-jvm-opts file

  • core: We made slight adaptions to the T.dest location of target defined in included files, to fix potential colliding cache locations

  • scalalib: JAR archives created by Mill now by default contain directory entries

  • scalalib: Updated zinc to 1.8.0

  • scalajslib: Added support for more JsEnv providers

    If you get some issues (like connect ECONNREFUSED, #2204) after the update, review your settings for JsEnv providers.

  • scalanativelib: Support for incremental compilation since Scala Native 0.4.9

  • contrib.testng: The TestNG library is no longer provided transitively; you need to provide it explicitly

  • contrib.bloop: Fixed re-generation of bloop config dir; previously, it could happen, that there where no config files at all

  • BSP: implement buildTarget/OutputPaths request

  • Various version bumps and internal improvements

For details refer to milestone 0.10.10 and the list of commits.

0.10.9 - 2022-11-10

Changes since 0.10.8:

  • Hardened Maven and Ivy artifact handling to prevent path-traversal attacks (see CVE-2022-37866), also updated coursier dependency to a non-vulnerable version

  • Decoupled mill-moduledefs module into a separate project, to better suite compiler plugin packaging and improve support for various Scala versions

  • Applied more workarounds to coursier concurrency issues to make parallel builds more robust

  • Added support for newer Scala versions in GenIdea (2.2 and 2.3)

  • Fixed an issue where PublishModule dropped module dependencies when mixed-in after other trait which overrode moduleDeps

  • new JMH contrib plugin

  • Lots of internal improvements and dependency updates

For details refer to milestone 0.10.9 and the list of commits.

0.10.8 - 2022-10-10

Changes since 0.10.7:

  • Improvements for better Scala 3.2 support

  • Fixed non-working default commands in cross modules

  • CoursierModule: mitigate more download failure situations (e.g. checksum failures)

  • PublishModule: properly show gpg output in server mode

  • BSP: Better compiler message handling (logMessage instead of showMessage) and support for diagnostic code

  • ScoverageModule: Support for Scoverage 2.x

  • New contrib module GitlabPublishModule

  • Various internal improvements and version bumps

  • Documentation improvements

For details refer to milestone 0.10.8 and the list of commits.

0.10.7 - 2022-08-24

Changes since 0.10.6:

  • Don’t print unwanted debug messages from zinc

For details refer to milestone 0.10.7 and the list of commits.

0.10.6 - 2022-08-24

Changes since 0.10.5:

  • PathRefs sigs are now more reproducible and independent of the used filesystem

  • JavaModule can now use a non-local Java Compiler to support all options

  • Logger: new debugEnabled to detect whether debug logging is enabled

  • New testkit module, to use TestEvaluator in external projects

  • Fixed reading of .mill-jvm-opts in server mode

  • BSP: Automatic SemanticDB enablement to improve Metals support

  • mill.twirllib.TwirlModule - new mandatory target twirlScalaVersion to configure the Scala version used by Twirl compiler, and support for newer versions

  • Lots of documentation updates and additions

  • Upgraded to Zinc 1.7.1 and various other dependency updates

For details refer to milestone 0.10.6 and the list of commits.

0.10.5 - 2022-07-01

Changes since 0.10.4:

  • New init command, to create new Mill projects from Gitter8 (g8) templates

  • PathRef now gracefully ignores socket files. This also fixes an annoying issue when Mill was previously ran in a source directory.

  • Fixed a bug with using mill --repl without any further arguments, which was introduced in Mill 0.10.2 and prevented proper no-server mode

  • Fixed the visualize command which wasn’t working on some newer JREs

  • Improved Mill server support for Windows 11 and some Windows Server versions

  • ScalaModule / ZincWorkerModule: Fixed incremental compilation issue with JRE 17 (and probably others)

  • TestModule now better supports JUnit5 test suites

  • ScalaJsModule: internal improvements to the stability of the Scala.js linker

  • ScalaNativeModule: Added support for nativeEmbedResources

  • BSP: improved handling of the mill-build module when the BSP client is IntelliJ IDEA

  • Documentation updates and link fixes

  • Various dependency updates

For details refer to milestone 0.10.5 and the list of commits.

0.10.4 - 2022-05-06

Changes since 0.10.3:

  • Switched from ipcsocket to junixsocket library to enhance the robustness of Mill client server communication. This should greatly improve the user experience on Windows.

  • Internal improvements and better support of GraalVM

  • The Mill Documentation site has now built-in search functionality

  • ScalaJsModule: New targets fastLinkJS and fullLinkJS and deprecated fastOpt and fullOpt

  • ScalaJsModule: Support for ModuleSplitStyle

  • BSP: Updated to protocol version 2.1.0-M1, added support for test framework names and support for the JVM extension

  • GenIdea: More consistent ordering of libraries and dependent modules and support for Scala 3.1 language level

  • Bloop: Added support for runtime dependencies

  • Enhanced test suite to cover Mill client server scenarios

  • Various dependency updates

For details refer to milestone 0.10.4 and the list of commits.

0.10.3 - 2022-04-11

Changes since 0.10.2:

  • Fixed import $file for files with hyphens and other symbols in its name

  • Fixed an issues with truncated output just before Mill finishes

  • Mill commands now support arguments of type Task[T], which can improve writing re-usable commands, especially, they can be called from other tasks more easily

  • JavaModule: Improved correctness and performance of compileClasspath and bspCompileClasspath targets. This fixes an issue with BSP for large projects. Also, compile-time dependencies no longer sneak into the classpath transitively.

  • JavaModule: Added docJarUseArgsFile target and fix issue with Windows command length restrictions in docJar

  • BSP and Bloop: Better detect foreign modules

  • Various internal API refinements and improvements

  • Reorganized integration test suite and build setup

For details refer to milestone 0.10.3 and the list of commits.

0.10.2 - 2022-03-18

Changes since 0.10.1:

  • Mill workers can now implement AutoCloseable to properly free resources

  • ScalaModule: Fixed repl start

  • CoursierModule: Fixed concurrent download issues with coursier (we detect and retry)

  • MainModule: Fixed potential match error with show and showNamed

  • Restructured contrib module documentation

  • Internal improvements

For details refer to milestone 0.10.2 and the list of commits.

0.10.1 - 2022-03-08

Changes since 0.10.0:

  • MillClienMain can now act as universal main to start mill with or without a server process

  • Improved show command that always returns valid JSON

  • Added showNamed command that includes the task names in the output

  • Implemented more granular cache invalidation which keeps more caches intact when you split your build setup over multiple .sc files with import $file.

  • The local .mill-jvm-opts file is no longer restricted to -X options

  • CoursierModule: Added coursierCacheCustomizer to support a FileCache customizers

  • JavaModule: the docJar target no longer includes hidden files

  • ScalaModule: Updated to latest zinc version

  • ScalaModule: Reworked scalac plugins handling for the better and to support Scala 3

  • ScalaNativeModule: fixed Dep.withDottyCompat behavior

  • ScalaJSModule: support for linking multiple modules

  • ScalafmtModule: Support for newer Scalafmt versions

  • Tool chain: Update to Mill 0.10.0

  • Tool chain: we no longer create files outside the mill project directory (e.g. ~/mill-release is now under target/mill-release)

  • Various dependency updates

  • Lots of internal improvements

For details refer to milestone 0.10.1 and the list of commits.

0.10.0 - 2022-01-14

Changes since 0.10.0-M5:

  • Changed structure of out directory, out/foo/dest is now out/foo.dest

  • Fixed issues with loading of predef scripts

  • --watch now supports manual re-runs by pressing enter

  • Improved subprocess handling

  • Published poms can now contain properties and versionScheme information

  • Improved Scala.js support, including more target configuration options and support for Node 17

  • Improved Scala Native for version > 0.4.2 and support Scala 3

  • Internal improvements, fixes and dependency version updates

For details refer to milestone 0.10.0 and the list of commits.

0.10.0-M5 - 2021-12-18

This release breaks binary compatibility for external plugins.

Changes since 0.10.0-M4:

  • Fixed Log4Shell security vulnerability in ZincWorkerModule (CVE-2021-44228)

  • Factored out the testrunner into a new module, which also fixes some potential classloader issues when executing tests (e.g. with JNA)

  • Removed the limitation of max 22 inputs for tasks

  • --watched commands can now re-run when pressing enter-key

  • task and arguments of commands can now have hyphens in their name

  • Reworked and decluttered the out-folder structure

  • prepareOffline now has a all flag to control if all or only some dependency should be prefetched

  • Made caching more effective for targets overridden in stackable-traits

  • Further BSP improvements, esp. for Metals and Scala 3

  • Lots of other internal improvements and fixes

  • Various dependency updates

For details refer to milestone 0.10.0-M5 and the list of commits.

0.10.0-M4 - 2021-11-08

Changes since 0.10.0-M3:

  • BSP support rework and overhaul of built-in BSP server

  • GenIdea: failures when inspecting and resolving the build are not properly reported

  • Coursier: we now implemented a workaround to tackle concurrent downloads issues

  • New + separator to provide multiple targets (with parameters) via cmdline

  • New --import cmdline option to run ad-hoc plugins without editing of build.sc

  • New T.ctx().workspace API to access the project root directory

  • Various internal improvements and bug fixes

  • Various refactorings and cleanups

For details refer to milestone 0.10.0-M4 and the list of commits.

0.10.0-M3 - 2021-09-29

This is a milestone release. This release breaks binary compatibility for external plugins build for mill 0.9.x. The API is suspected to change before a 0.10.0 releae.

Changes since 0.10.0-M2:

  • ScalaModule with PublishModule: the scala-library artifact is now always part of the dependencies in published `pom.xml`s and `ivy.xml`s

  • New JavaModule.mandatoryIvyDeps target to provide essential dependencies like scala-library without forcing the user to call super.ivyDeps

  • ScalaJSModule.scalaLibraryIvyDeps no longer contains the scala-js-library, but only the scala-library; if you need that, use ScalaJSModule.mandatoryIvyDeps instead.

  • import $ivy support $MILL_BIN_PLATFORM variable and a new sort notations for external plugins

  • We fixed and enabled lots of tests to run on Windows

  • Some generic targets like plan or path now also return their output

  • GenIdea: improved support for Scala 3 projects *

For details refer to milestone 0.10.0-M3 and the list of commits.

0.10.0-M2 - 2021-09-17

This is a early milestone release. This release breaks binary compatibility for external plugins build for mill 0.9.x. The API is suspected to change before a 0.10.0 releae.

Changes since 0.9.9:

  • Removed deprecated API

  • ScalaModule: added mandatoryScalacOptions to avoid the common issue that users forget to include mandatory options when defining their own.

  • Renamed toolsClasspath targets found in various modules to avoid hard to resolve clashes when mixing traits

  • Fixed and improved our test suite on Windows

  • Various fixes and improvements

  • Various dependency updates

For details refer to milestone 0.10.0-M2 and the list of commits.

0.9.12 - 2022-01-07

  • fixed parsing of command parameters in show command

  • zinc worker: Updated log4j2 to 2.17.1

For details refer to milestone 0.9.12 and the list of commits.

0.9.11 - 2021-12-15

  • zinc worker: Updated log4j2 to 2.16.0 to fix Log4Shell (CVE-2021-44228) vulnerability

For details refer to milestone 0.9.11 and the list of commits.

0.9.10 - 2021-11-13

  • Some feature backports from mill 0.10

  • New + separator to provide multiple targets (with parameters) via cmdline

  • New --import cmdline option to run ad-hoc plugins without editing of build.sc

  • import $ivy support $MILL_BIN_PLATFORM variable and a new sort notations for external plugins

For details refer to milestone 0.9.10 and the list of commits.

0.9.9 - 2021-07-15

Changes since 0.9.8:

  • BSP: Fixed/improved source item for root project

  • Bloop: Prevent compilation during bloop config generation

  • GenIdea: Fix content path of root project (mill-build)

  • Various version bumps

For details refer to milestone 0.9.9 and the list of commits.

0.9.8 - 2021-05-27

Changes since 0.9.7:

  • Fixed some potential binary incompatibilities with external plugins (builds against older os-lib versions)

  • Fixed location and configuration of mills home path (used for caching of build scripts)

  • Properly close jar resources - should fix issues in assembly, esp. on Windows where open resources are locked

  • BSP: Repaired mills BSP server

  • playlib: Fixed issues with the play-contrib module and added support for Play 2.8

  • GenIdea: changed dir for generated mill modules to .idea/mill_modules

  • Various version bumps, including Scala 2.13.5

For details refer to milestone 0.9.8 and the list of commits.

0.9.7 - 2021-05-14

Changes since 0.9.6:

  • ScalaModule: Support for Scala 3

  • CoursierModule: Support customized dependency resolution (needed to work with ScalaFX)

  • TestModule: Added new testFramework target and only support one test framework. Deprecated testFrameworks targets.

  • TestModule: Added new convenience traits to configure popular test frameworks, e.g. TestModule.Junit4, TestModule.ScalaTest, TestModule.Utest and many more

  • Bloop: Added support for foreign modules

  • Better support for Windows environments

  • Various internal improvements, cleanups, and deprecations

  • Various dependencies updates

  • Removed tut contrib module because of unmaintained/archived upstream dependency

For details refer to milestone 0.9.7 and the list of commits.

0.9.6 - 2021-04-03

The mill project home and repository has been moved to https://github.com/com-lihaoyi/mill.

  • repl and console targets now support forkArgs and forkEnv

  • Support for Scala 3 release candidates and new Scaladoc 3 tool

  • Support for Scala.js on Scala 3

  • Scala Native improvements

  • Test runner now uses an args file to support running tests on Windows

  • GenIdea: better supports source jars, full config contributions and provided/runtime dependencies

  • Various dependency updates

  • Documentation site reworked to support multiple release versions

  • Improved CI setup to better test mill on Windows

For details refer to milestone 0.9.6 and the list of commits.

0.9.5 - 2021-01-26

  • Updated zinc to 1.4.4

  • Support for Scala Native 0.4.0

  • Support for Scala.js ESModule (including Bloop support)

  • Inner Tests traits in modules like JavaModule, ScalaModule and others now have unique names (JavaModuleTests, ScalaTests, etc), to allow for easier customization

  • Various version bumps of dependencies

  • CI now runs all tests, it did miss some before

For details refer to milestone 0.9.5 and the list of commits.

0.9.4 - 2020-12-21

  • Implemented more BSP protocol commands and fixed some glitches with IntelliJ

  • Stabilized CI builds

  • Various fixes and improvements

  • Various version bumps

For details refer to milestone 0.9.4 and the list of commits.

0.9.3 - 2020-11-26

(We also tagged 0.9.0, 0.9.1, and 0.9.2, but due to release difficulties, we ask you not to use them.)

  • Replace the built in @main method functionality with the MainArgs library

  • Note that the MainArgs replacement has some backwards incompatibilities: Short flags like -i can no longer be passed via --i, the @doc("") is now @arg(doc = ""), Seq[T] parameters are now passed via repeated --foo flags rather than comma-separated.

  • Add the ability to relocate/shade files in .assembly #947

  • Twirl enhancements #952

  • Add scalacPluginClasspath to Tests #956

  • Add toMap methods to BuildInfo #958

  • Bump coursier to version 2.0.0 #973

  • Make BSP support a first-class citizen #969

  • Omit the suffix in artifactName in cross modules #953

  • Allow test classes with constructor parameters #982

  • Proguard contrib module #972

  • Support Scala.js useECMAScript2015 option and ModuleKind.ESModule #1004

  • Support Scala.js incremental linking #1007

For details refer to milestone 0.9.3 and the list of commits.

0.8.0 - 2020-07-20

  • Bump external dependencies: uPickle 1.2.0, Ammonite 2.2.0, etc.

  • Use default coursier repos (#931)

  • Work around relative paths issue on windows (#936)

  • Support Scala.js versions >1.0.0 (#934)

For details refer to milestone 0.8.0 and the list of commits.

0.7.4 - 2020-07-03

  • new command line options --repl and --no-server, deprecated --interactive option

  • Support for Scala.js 1.1

  • Fixed missing source maps for Scala.js 1.0 and 1.1

  • Improved BSP contrib module

For details refer to milestone 0.7.4 and the list of commits.

0.7.3

For details refer to milestone 0.7.3 and the list of commits.

0.7.2 - 2020-05-19

For details refer to milestone 0.7.2 and the list of commits.

0.7.1 - 2020-05-17

For details refer to milestone 0.7.1 and the list of commits.

0.7.0 - 2020-05-15

  • Greatly improved parallel builds via -j <n>/--jobs <n>, with better scheduling and utilization of multiple cores

  • build.sc files now uses Scala 2.13.2

  • Avoid duplicate target resolution with mill resolve __

  • Add ability to pass GPG arguments to publish via --gpgArgs

  • -w/--watch now works for T.source targets

For details refer to milestone 0.7.0 and the list of commits.

0.6.3 - 2020-05-10

  • Finished incomplete support to publish extra artifacts to IVY repositories (publishLocal)

  • Improved Sonatype uploads

  • GenIdea: improvements for shared source dirs and skipped modules

  • ScoverageModule: Some refactorings to allow better customization

  • More robust classpath handling under Windows

For details refer to milestone 0.6.3 and the list of commits.

0.6.2 - 2020-04-22

  • Mill can now execute targets in parallel. This is experimental and need to be enabled with --jobs <n> option.

  • PublishModule: new publishM2Local to publish into local Maven repositories

  • PublishModule: enhanced publishLocal to specify to ivy repository location

  • Windows: Fixed windows launcher and more robust classpath handling

  • ScalaNativeModule: improved compiling and linking support

  • new contrib module VersionFile

  • Dependency: improved dependency update checker and expose results for programmatic use

  • ǹew contrib module Bintray

  • ǹew contrib module Artifactory

  • fixed testCached support in various modules

  • GenIdea: improvements, esp. related to source jars

For details refer to milestone 0.6.2 and the list of commits.

0.6.1 - 2020-02-24

  • Bugfix: Mill now no longer leaks open files (version bump to uPickle 1.0.0)

  • New --version option

  • Added Support for Scala.js 1.0.0+

  • Added Support for Scala Native 0.4.0-M2

  • JavaModule: Enhanced ivyDepsTree to optionally include compile-time and runtime-time dependencies

  • JavaModule: allSourceFiles no longer include Scala sources

  • JavaModule: assembly supports configurable separator when merging resources

  • ScoverageModule: respect unmanagedClasspath, added console reporter

  • ScalaPBModule: added more configuration options

  • Bloop: Fixed inconsistent working directory when executing tests via bloop (forces -Duser.dir when generating bloop config)

For details refer to milestone 0.6.1 and the list of commits.

0.6.0 - 2020-01-20

  • Support for METALS 0.8.0 in VSCode

For details refer to milestone 0.6.0 and the list of commits.

0.5.9 - 2020-01-14

  • Bump library versions again

  • Alias T.ctx. functions to T.: T.dest, T.log, etc.

  • Bump Mill’s client-connect-to-server timeout, to reduce flakiness when the server is taking a moment to start up

For details refer to the list of commits.

Version 0.5.8 has some binary compatibility issues in requests-scala/geny and should not be used.

0.5.7 - 2019-12-28

  • Bump library versions: Ammonite 2.0.1, uPickle 0.9.6, Scalatags 0.8.3, OS-Lib 0.6.2, Requests 0.4.7, Geny 0.4.2

For details refer to milestone 0.5.7 and the list of commits.

0.5.5 / 0.5.6 - 2019-12-20

(we skipped version 0.5.4 as we had some publishing issues)

  • Bump library versions: Ammonite 1.9.2, uPickle 0.9.0, Scalatags 0.8.2, OS-Lib 0.5.0, Requests 0.3.0, Geny 0.2.0, uTest 0.7.1

  • Fixed a long standing issue that output of sub-processes are only shown when -i option was used. Now, you will always seen output of sub-process.

  • Mill now properly restarts it’s server after it’s version has changed

  • PublishModule: added ability to publish into non-staging repositories

  • ScalaPBModule: added extra include path option

For details refer to milestone 0.5.5 and the list of commits.

0.5.3 - 2019-12-07

  • GenIdea/idea: improved support for generated sources and use/download sources in more cases

  • ScalaJS: improvements and support for ScalaJS 0.6.29+ and 1.0.1.RC1

  • Introduced new CoursierModule to use dependency management independent from a compiler

  • ScoverageModule: better handling of report directories

  • ScalaPBModule: more configuration options

  • various other fixes and improvements

For details refer to milestone 0.5.3 and the list of commits.

0.5.2 - 2019-10-17

  • TestModule: new `testCached`target, which only re-runs tests after relevant changes

  • TestModule.test: fixed issue when stacktraces have no filename info

  • Dependency/updates: fixed issue with reading stale dependencies

  • GenIdea/idea: no longer shared output directories between mill and IntelliJ IDEA

  • support for Dotty >= 0.18.1

  • Fixed backwards compatibility of mill wrapper script

  • Mill now support the Build Server Protocol 2.0 (BSP) and can act as a build server

  • bloop: removed semanticDB dependency

  • Documentation updates

For details refer to milestone 0.5.2 and the list of commits.

0.5.1 - 2019-09-05

  • GenIdea: Bug fixes

  • GenIdea: Support for module specific extensions (Facets) and additional config files

  • Add ability to define JAR manifests

  • Dotty support: Updates and support for binary compiler bridges

  • Ivy: improved API to create optional dependendies

  • Interpolate $MILL_VERSION in ivy imports

  • Zinc: Fixed logger output

  • Scoverage: Upgrade to Scoverage 1.4.0

  • Flyway: Upgrade to Flyway 6.0.1

  • Bloop: Updated semanticDB version to 4.2.2

  • Documentation updates

  • Improved robustness in release/deployment process

For details refer to milestone 0.5.1 and the list of commits.

0.5.0 - 2019-08-08

  • Mill now supports a ./mill bootstrap script, allowing a project to pin the version of Mill it requires, as well as letting contributors use ./mill …​ to begin development without needing to install Mill beforehand.

  • Support for a .mill-version file or MILL_VERSION environment variable for Overriding Mill Versions

  • Fix scoverage: inherit repositories from outer project #645

0.4.2 - 2019-06-30

  • Improvements to IntelliJ project generation #616

  • Allow configuration of Scala.js' JsEnv #628

0.4.1 - 2019-06-13

  • Fixes for scala native test suites without test frameworks #627

  • Fix publication of artifacts by increasing sonatype timeouts

  • Bug fixes for Scoverage integration #623

0.4.0 - 2019-05-20

  • Publish compileIvyDeps as provided scope (535)

  • Added contrib modules to integrate Bloop, Flyway, Play Framework, Scoverage

  • Allow configuration of GPG key names when publishing (530)

  • Bump Ammonite version to 1.6.7, making Requests-Scala available to use in your build.sc

  • Support for Scala 2.13.0-RC2

  • ScalaFmt support now uses the version specified in .scalafmt.conf

0.3.6 - 2019-01-17

  • Started to splitting out mill.api from mill.core

  • Avoid unnecessary dependency downloading by providing fetches per cache policy

  • Added detailed dependency download progress to the progress ticker

  • Fixed internal code generator to support large projects

  • Zinc worker: compiler bridge can be either pre-compiled or on-demand-compiled

  • Zinc worker: configurable scala library/compiler jar discovery

  • Zinc worker: configurable compiler cache supporting parallelism

  • Version bumps: ammonite 1.6.0, scala 2.12.8, zinc 1.2.5

  • Mill now by default fails fast, so in case a build tasks fails, it exits immediately

  • Added new -k/--keep-going commandline option to disable fail fast behaviour and continue build as long as possible in case of a failure

0.3.5 - 2018-11-18

  • Bump uPickle to 0.7.1

0.3.4 - 2018-11-06

  • Mill is now bundled with OS-Lib, providing a simpler way of dealing with filesystem APIs and subprocesses

0.3.3 - 2018-11-02

  • Added new debug method to context logger, to log additional debug info into the task specific output dir (out/<task>/log)

  • Added --debug option to enable debug output to STDERR

  • Fix ScalaModule#docJar task when Scala minor versions differ 475

0.3.2 - 2018-10-19

  • Automatically detect main class to make ScalaModule#assembly self-executable

0.3.0 - 2018-10-19

  • Bump Ammonite to 1.3.2, Fastparse to 2.0.4

  • Sped up ScalaModule#docJar task by about 10x, greatly speeding up publishing

  • Add a flag JavaModule#skipIdea you can override to disable Intellij project generation #458

  • Allow sub-domains when publishing #441

0.2.8 - 2018-09-21

  • mill inspect now displays out the doc-comment documentation for a task.

  • Avoid shutdown hook failures in tests #422

  • Ignore unreadable output files rather than crashing #423

  • Don’t compile hidden files #428

0.2.7 - 2018-08-27

  • Add visualizePlan command

  • Basic build-info plugin in mill-contrib-buildinfo

  • ScalaPB integration in mill-contrib-scalapblib

  • Fixes for Twirl support, now in mill-contrib-twirllib

  • Support for building Dotty projects #397

  • Allow customization of run/runBackground working directory via forkWorkingDir

  • Reduced executable size, improved incremental compilation in #414

0.2.6 - 2018-07-30

  • Improve incremental compilation to work with transitive module dependencies

  • Speed up hot compilation performance by properly re-using classloaders

  • Speed up compilation time of build.sc files by removing duplicate macro generated routing code

0.2.5 - 2018-07-22

  • Add .runBackground and .runMainBackground commands, to run something in the background without waiting for it to return. The process will keep running until it exits normally, or until the same .runBackground command is run a second time to spawn a new version of the process. Can be used with -w for auto-reloading of long-running servers.

  • Scala-Native support. Try it out!

  • Add --disable-ticker to reduce spam in CI

  • Fix propagation of --color flag

0.2.4 - 2018-07-07

  • Fix resolution of scala-{library,compiler,reflect} in case of conflict

  • Allow configuration of JavaModule and ScalafmtModule scala workers

  • Allow hyphens in module and task names

  • Fix publishing of ScalaJS modules to properly handle upstream ScalaJS dependencies

0.2.3 - 2018-06-03

  • Added the mill show visualize command, making it easy to visualize the relationships between various tasks and modules in your Mill build.

  • Improve Intellij support (351): better jump-to-definition for third-party libraries, no longer stomping over manual configuration, and better handling of import $ivy in your build file.

  • Support for un-signed publishing and cases where your GPG key has no passphrase (346)

  • Basic support for Twirl, Play Framework’s templating language (271)

  • Better performance for streaming large amounts of stdout from Mill’s daemon process.

  • Allow configuration of append/exclude rules in ScalaModule#assembly (309)

0.2.2 - 2018-05-20

  • Preserve caches when transitioning between -i/--interactive and the fast client/server mode (329)

  • Keep Mill daemon running if you Ctrl-C during -w/--watch mode (327)

  • Allow mill version to run without a build file (328)

  • Make docJar (and thus publishing) robust against scratch files in the source directories (334) and work with Scala compiler options (336)

  • Allow passing Ammonite command-line options to the foo.repl command (333)

  • Add mill clean (315) to easily delete the Mill build caches for specific targets

  • Improve IntelliJ integration of `MavenModule`s/`SbtModule`s' test folders (298)

  • Avoid showing useless stack traces when foo.test result-reporting fails or foo.run fails

  • ScalaFmt support (308)

  • Allow ScalaModule#generatedSources to allow single files (previous you could only pass in directories)

0.2.0 - 2018-04-10

  • Universal (combined batch/sh) script generation for launcher, assembly, and release (#264)

  • Windows client/server improvements (#262)

  • Windows repl support (note: MSYS2 subsystem/shell will be supported when jline3 v3.6.3 is released)

  • Fixed Java 9 support

  • Remove need for running publishAll using --interactive when on OSX and your GPG key has a passphrase

  • First-class support for `JavaModule`s

  • Properly pass compiler plugins to Scaladoc (#282)

  • Support for ivy version-pinning via ivy"…​".forceVersion()

  • Support for ivy excludes via ivy"…​".exclude() (#254)

  • Make ivyDepsTree properly handle transitive dependencies (#226)

  • Fix handling of runtime-scoped ivy dependencies (#173)

  • Make environment variables available to Mill builds (#257)

  • Support ScalaCheck test runner (#286)

  • Support for using Typelevel Scala (#275)

  • If a module depends on multiple submodules with different versions of an ivy dependency, only one version is resolved (#273)

0.1.7 - 2018-03-26

  • Support for non-interactive (client/server) mode on Windows.

  • More fixes for Java 9

  • Bumped the Mill daemon timeout from 1 minute to 5 minutes of inactivity before it shuts down.

  • Avoid leaking Node.js subprocesses when running ScalaJSModule tests

  • Passing command-line arguments with spaces in them to tests no longer parses wrongly

  • ScalaModule#repositories, scalacPluginIvyDeps, scalacOptions, javacOptions are now automatically propagated to Tests modules

  • ScalaJSModule linking errors no longer show a useless stack trace

  • ScalaModule#docJar now properly uses the compileClasspath rather than runClasspath

  • Bumped underlying Ammonite version to 1.1.0, which provides the improved Windows and Java 9 support

0.1.6 - 2018-03-13

  • Fixes for non-interactive (client/server) mode on Java 9

  • Windows batch (.bat) generation for launcher, assembly, and release

0.1.5 - 2018-03-13

  • Introduced the mill plan foo.bar command, which shows you what the execution plan of running the foo.bar task looks like without actually evaluating it.

  • Mill now generates an out/mill-profile.json file containing task-timings, to make it easier to see where your mill evaluation time is going

  • Introduced ScalaModule#ivyDepsTree command to show dependencies tree

  • Rename describe to inspect for consistency with SBT

  • mill resolve now prints results sorted alphabetically

  • Node.js configuration can be customised with ScalaJSModule#nodeJSConfig

  • Scala.js fullOpt now uses Google Closure Compiler after generating the optimized Javascript output

  • Scala.js now supports NoModule and CommonJSModule module kinds

  • Include compileIvyDeps when generating IntelliJ projects

  • Fixed invalid POM generation

  • Support for Java 9 (and 10)

  • Fixes for Windows support

  • Fixed test classes discovery by skipping interfaces

  • Include "optional" artifacts in dependency resolution if they exist

  • out/{module_name} now added as a content root in generated IntelliJ project

0.1.4 - 2018-03-04

  • Speed up Mill client initialization by another 50-100ms

  • Speed up incremental `assembly`s in the common case where upstream dependencies do not change.

  • Make ScalaJSModule#run work with main-method discovery

  • Make ScalaWorkerModule user-defineable, so you can use your own custom coursier resolvers when resolving Mill’s own jars

  • Simplify definitions of SCM strings

  • Make the build REPL explicitly require -i/--interactive to run

  • Log a message when Mill is initializing the Zinc compiler interface

0.1.3 - 2018-02-26

  • Greatly reduced the overhead of evaluating Mill tasks, with a warm already-cached mill dev.launcher now taking ~450ms instead of ~1000ms

  • Mill now saves compiled build files in ~/.mill/ammonite, which is configurable via the --home CLI arg.

  • Fixed linking of multi-module Scala.js projects

0.1.2 - 2018-02-25

  • Mill now keeps a long-lived work-daemon around in between commands; this should improve performance of things like compile which benefit from the warm JVM. You can use -i/--interactive for interactive consoles/REPLs and for running commands without the daemon

  • Implemented the ScalaModule#launcher target for easily creating command-line launchers you can run outside of Mill

  • ScalaModule#docJar no longer fails if you don’t have scala-compiler on classpath

  • Support for multiple testFrameworks in a test module.

0.1.1 - 2018-02-19

  • Fixes for foo.console

  • Enable Ammonite REPL integration via foo.repl

0.1.0 - 2018-02-18

  • First public release

mill's People

Contributors

adadima avatar ajrnz avatar alexarchambault avatar atty303 avatar baccata avatar carlosedp avatar chikei avatar ckipp01 avatar ggrossetie avatar heksesang avatar jeantil avatar jkstrauss avatar joan38 avatar jodersky avatar lefou avatar lihaoyi avatar lolgab avatar nafg avatar nightscape avatar nrktkt avatar nvander1 avatar robby-phd avatar rockjam avatar romain-gilles-ultra avatar rtimush avatar sake92 avatar samvel1024 avatar scala-steward avatar sequencer avatar smarter avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

gitter-badger he-pin tindzk piotrkwiecinski rockjam dwijnand baccata ranahasan sailfish009 akshay-sri123 zoosky ramsinha soc ryan-williams stephennancekivell vinayakpathak pleira pietrotull ajrnz h2000 bugthesystem aeons jjcipher masseguillaume 3tty0n freewind-forks sidnt ucbjrl scalway nornagon sake92 sunlightjoe dohrayme lptk francisdb dennis4b witchlove wegtam aywengo lolgab shanielh aleks-sidorenko benjaminfrank internityfoundation lvauthrin vdeclerk kyw-iduce gabro guersam shengc msoule empia nomadblacky ggrossetie andersonvom julianmichael grebe artempyanykh caeus runeblaze cvnb narma ohze qingwei91 jkstrauss bbarker ouspark bobo1239 keynmol mdedetrich sullis gavares mkrajc aosagie yeleywale av-elier nightscape jonas-depop oyvindberg smarter davidgregory084 manuzhang gillisvandenbroeke johnduffell dbuschman7 nh13 vic omnipresent07 renkai lefou rowhit henrywoo yurique bbjubjub2494 vitaliis leonardehrenfried gehnaphore lihaoyi-databricks kongo2002 cremboc

mill's Issues

Support for downloading & caching a binary as part of a build

Necessary to fully support the better-files build

Generally useful for all sorts of other things too.

We probably can just ignore the problem of expiration: once the file is downloaded, we keep it forever unless someone explicitly deletes it.

This is basically already the behavior of Targets e.g. def foo = T{...}, so we can just define a helper method:

def downloadedJar = T{ Util.download("https://github.com/williamfiset/FastJavaIO/releases/download/v1.0/fastjavaio.jar") }

Where Util.download is a Task that downloads the file to the T.ctx().dest path and returns a PathRef that we can feed into downstream Targets, eg. ScalaModule#depClasspath

Write a minimal unit test for scalaplugin/

#2 will be an integration test for the module, but we really should have a truly "hello world" unit test that exercises it's functionality in as little code/complexity as possible.

Once we have unit tests, we can start adding more functionality to the scalaplugin/ package, which we currently cannot do since testing it is manual and annoying. e.g. TestModule helpers which come with test pre-defined, SbtModule helpers which come with main+test in one bundle.

Figure out some formal logging policy

Our logging right now is a bit of a mess, we need to answer:

  • How do we remove the verbosity of things like Running Core.test.upstreamCompileOutput, which hardly ever do any work?

  • What goes to stdout and what goes to stderr?

  • Where does --show fit into this, since a user calling --show is likely to want to capture the result as a clean JSON blob without all the misc log spam?

Ammonite's policy is that "misc" low-priority output (compiling files, blah) goes to stderr, leaving stdout for things that a user explicitly prints, so users can foo.sc > out.txt and be sure `out.txt doesn't contain any random trash.

Parallel Task Evaluation

The current Evaluator is sequential: tasks are evaluated one by one in a linear topological order.

Because we have the dependency graph between tasks, it should be straightforward to implement a parallel evaluator that evaluates tasks N-ways parallel. This would not require any change to the user-code implementing tasks and targets.

Open questions include how such a parallel evaluator will fit into the existing codebase (we probably want to keep the sequential evaluator around as an option indefinitely) and how to make sense of the interleaved log-spam from the parallel-executing tasks (we could provide a T.ctx().println method that will prefix the stdout with the name of the target being evaluated, and ask task writers to use that)

Log task output to files

Currently, every task Ctx exposes a logger, and we also capture stdout/stderr from task body evaluation and push it through the logger. This is useful for e.g. running Mill evaluations silently in unit tests, to avoid logspam

We should also log the output related to each task to a log file. Each task foo.bar already has a out/foo/bar folder to use for output files and out/foo/bar.mill.json to cache its return value. We can stream the logs to out/foo/bar.log as the task executed, so a user can easily find the logs related to each task.

This is especially important once we parallelize task execution, and the console output of Mill becomes all interleaved, and the logfiles will be the only place to find non-interleaved task output.

Bust caches when Mill classpath changes

Currently we bust caches every time the inputs of a Target group change: only if the inputs change do we consider the output change able to change, and thus the Target group needing to recompute.

This is insufficient: if the Target group's implementation itself changes, the output could change, and the Target should recompute. There is no way to account for this at a fine-grained level, but we can be conservative: we should incorporate the classpath-signature of the Mill build into our cache keys. If the classpath of the Mill build changes, we invalidate all caches.

We already have code in Ammonite (which we already depend on for the Scala script-running capability) that flushes Ammonite's script-compile caches when the classpath changes. We can re-use the same code to flush Mill's caches

This will flush things caches every time someone changes the build, even when it's unnecessary. Nevertheless, it's probably the right thing to do, especially when you consider builds change much more rarely than the code being built/run/tested, and stale caches are a pain in the neck

Support forked execution for ScalaModule#run and TestScalaModule#test

Currently, ScalaModule#run runs in a subprocess and TestScalaModule#test runs in-process in a Classloader. This is arbitrary and inconsistent

We should standardize on a way to let either method run either subprocessed or in-process, and make this available to the user of Mill if they want to run any arbitrary JVM classpath as part of any task.

Create a "release" artifact that is configured to pull necessary jars from Ivy/Maven

Currently, Mill depends on a number of compiler-bridge jars which are created as part of the Mill build. Both sbt scalaplugin/test and the executable generated by sbt scalaplugin/test:assembly reference these jars, by hard-coding their absolute paths on disk. Thus the executable built by sbt scalaplugin/test:assembly would only work on the same computer you ran the SBT command on. The number of locally-built jars that Mill needs to reference programmatically is likely to grow over time (e.g. TestRunner will likely move into it's own jar, along with the ScalaModule zinc integration)

Obviously, this will not do if we want to make Mill usable by anyone on the internet. What we need is a "release" build of the Mill executable that will resolve the necessary compiler-bridge (and other) jars from Maven Central (or from your on-disk publishLocal maven cache, for testing) instead of hard-coding absolute paths to recently-built jars on disk.

Create simple task that downloads and caches dependencies

Currently, forcing a mill to update dependencies requires the user to execute a task named externalCompileDepClasspath which is hard to remember and hard to type.

Similar to sbt's update, we need to create a simply named task that forces dependency resolution.
Suggestions

  • update
  • resolve
  • deps

Write unit tests for GenIdea

This is pretty fragile code that is reasonably entangled with ScalaModule's internals, and has broken in the past. We should have at least a simple unit test that does GenIdea on a trivial build and checks the output against some hardcoded XML files, just to avoid accidental regressions and breakage

Only one `Task` in a group should be allowed to use the `T.ctx().dest` directory

Tasks are grouped together under their downstream Target before running, and every Task in that group has the same T.ctx().dest which is defined by the path to the downstream Target. Currently there is nothing stopping multiple Tasks from reading/writing from dest, which could allow for unintended interference between them.

We should fail with an error if more than one Task attempts to use the dest dir, for any purpose. How to provide a nice error message, given that Tasks don't need to have names, is a bit of an open question: maybe the stack trace of the points which use .dest are good enough?

Support long-lived worker processes

Currently, Mill builds are stateless: build results are persisted to disk, but all in-memory state lives on the call-stack of evaluate, and is discarded after every run.

There is a large class of build tooling that works better with long-lived worker processes. A Scala Zinc compile daemon which can keep the JVM hot & fast is one example, but there are countless others: Javascript bundlers e.g. Webpack, websocket servers for browser-reloading e.g. Workbench, and many others.

We should come up with a way to support long lived build processes that can support all these cases, keeping the worker processes around even as the Mill process repeatedly starts and exits.

Build REPL

Since we are using Ammonite to run build.sc scripts, it shouldn't be a stretch to also use Ammonite as the build REPL: to keep Mill alive and run multiple commands with a warm JVM, and possibly to inspect and query the build programmatically.

I'd like to be able to build targets using the Build REPL the same way you can refer to in build.sc files:

Core.jar()
bridges("2.10.6").compile()

Where () forces the target to be build & evaluated. This is in addition to running arbitrary Scala code.

This will take some changes to the way () is handled, so we can use () for the applicative T{...} syntax in build files, () for "evaluate now" functionality in the REPL, but not let people use () in build files for "evaluate now" (they should only use it in T{...} syntax within build files)

Add nice prettyprints/toString for the Build REPL

Currently, the toStrings you get inspecting tasks/modules in the REPL is pretty bare:

lihaoyi mill$ scalaplugin/target/mill --repl
Loading...
@ build
res0: build.type = build

@ build.Core
res1: Core.type = ammonite.predef.$up.build$Core$@33ce764b

@ build.Core.compile
res2: T[scalaplugin.CompilationResult] = mill.define.Persistent@ddd029e

While you can poke around with autocomplete or typeOf(build).members, it would be nice to hook into Ammonite's pretty-printer and use it to show nicer output at the REPL:

lihaoyi mill$ scalaplugin/target/mill --repl
Loading...
@ build
res0: build.type = build
Children:
- build.Core
- build.ScalaPlugin
- build.bridges

@ build.Core
res1: Core.type = build.Core
Children:
- build.Core.assembly
- build.Core.basePath
- build.Core.classpath
- build.Core.compile
- build.Core.compileDepClasspath
- build.Core.console
...

@ build.Core.compile
res2: T[scalaplugin.CompilationResult] = build.Core.compile
Inputs:
- build.Core.scalaVersion
- build.Core.allSources
- build.Core.compileDepClasspath
- build.Core.scalaCompilerClasspath
- build.Core.compilerBridgeClasspath
- build.Core.scalacOptions
- build.Core.scalacPluginClasspath
- build.Core.javacOptions
- build.Core.upstreamCompileOutput

This can be done by hooking into the Ammonite REPL pretty-printer:

repl.pprinter() = repl.pprinter().copy(additionalHandlers = ...)

We can make use of the Discovered mirror within the additionalHandlers, which should contain the information needed for labelling all the Tasks

It might be harder to hook into toString output, since we can't make use of the Discovered mirror within toString, but maybe there's still something we can do there

Dependencies resolution doesn't fail when artifact not found.

When dependency not found - for example, such version doesn't exist, or we don't have right resolver - dependency resolution doesn't fail, and we get error at compile time with error message like:

[error] application/src/main/scala/Main.scala:12:13: object circe is not a member of package io
[error]   import io.circe._

Write a `TestScalaModule` class to make it easier to specify test suites

Currently specifying a test suite is kind of annoying

object CoreTests extends ScalaModule {
  def scalaVersion = "2.12.4"
  override def projectDeps = Seq(Core)
  def basePath = pwd / 'scalaplugin
  override def sources = pwd/'core/'src/'test/'scala
  override def ivyDeps = Seq(
    Dep("com.lihaoyi", "utest", "0.6.0")
  )

  def test() = T.command{
    TestRunner.apply(
      "mill.UTestFramework",
      runDepClasspath().map(_.path) :+ compile().path,
      Seq(compile().path)
    )
  }
}

TestScalaModule would encapsulate all the standard stuff into

object CoreTests extends TestScalaModule {
  def projectDeps = Seq(Core) // scalaVersion inferred from projectDeps
  override def sources = pwd/'core/'src/'test/'scala
  override def ivyDeps = Seq(Dep("com.lihaoyi", "utest", "0.6.0"))
  def testFramework = "mill.UTestFramework"
}

Add an SbtScalaModule for easy migration from SBT project layouts

Currently, ScalaModule is a bit low-level: it defaults to your Scala code being in src/ (which differs from SBT's convention of src/main/scala/) and you need to wire up a val test = new Tests suite yourself.

While I think there's value in providing a "simpler" ScalaModule without the decades of SBT/Maven/Ivy baggage, we should also provide a SbtScalaModule to ease the conversion of projects from SBT to Mill. SbtScalaModule should only require you to define your basePath, and automatically set:

  • source code in src/main/scala/
  • resources in src/main/resource/
  • test code in src/test/scala/
  • test resources in src/test/resource/

We should also figure out a story for cross-SbtScalaModules build against multiple Scala versions (with version-specific sources) as these are very common in open-source libraries

Sonatype PublishSigned support?

One of the major things Mill is currently missing is the ability to publishSigned artifacts (along with source jars, etc.) to Maven central.

We already have the ability to create Jars, e.g. mill run ScalaPlugin.assembly. #3 will allow us to make them executable. The last step is to let us publish them to some artifact repository: Maven Central, Bintray, etc.

I'm not familiar with what would go into uploading a Jar to maven central, but some good resources may be:

Basically whatever SBT-PGP is doing, we should be able to do the same

Watch mode doesn't handle new files

Currently watch mode (mill --watch proj.compile ) reacts only on changes from existed at startup moment files.
At least it should react on file creation. I'm not sure about removing, maybe there are some inotify limitations (comparing to sbt - it reacts on new files, but ignore removing)

Get rid of uPickle

Ammonite uses it, and we use it too for convenience, but it's basically unmaintained and should go away.

I tried using play-json for a while but that added a non-trivial amount of startup time (200-300ms?) just from classloading Jackson, which for some reason is a lot of classfiles.

The correct thing to do is probably to strip out all the gnarly recursive-implicit-derivation part of uPickle (which isn't needed for how we are using it here), keep the implicits and the non-implicit caseR/caseW/caseRW macro, and just use that instead.

We should also push this upstream to Ammonite to also use this stripped-down uPickle

Allow other kinds of input tasks which recompute every time

Currently, of all Tasks, only T.source has a non-constant sideHash that forces it to re-evaluate whenever the contents of the filesystem path it represents changes. This is necessary because otherwise, a Task with no inputs will never get re-evaluated.

We should generalize this to allow other non-filesystem-related Tasks to force re-evaluation every time: use cases include Tasks that load the sys.env("JAVA_HOME"), or which shell out to git to load the current commit hash. In these cases, the operation is fast enough I'm fine with never caching it ever.

We should provide a T.input task, used as such:

val currentGitHash = T.input{ %%("git", "rev-parse", "head").out }
val javaHome = T.input{ sys.env("JAVA_HOME") }

These T.inputs can be used in downstream Tasks and Targets. The body of the T.input will be re-evaluated every time, and if the result changes, it will bust caches of all downstream Tasks and Targets and force them to re-evaluate too

More flexible query language for running multiple tasks

Currently running multiple tasks in Mill is a bit of a pain:

mill run Core.test && mill run ScalaPlugin.test

Even though the second task will benefit from caching of the first, the tasks run sequentially. Mill's execution engine is actually happy to evaluate multiple tasks in parallel, we just need to expose it to the user.

uTest has a similar problem (users want to select multiple paths of tests to run) and allows syntax like:

testOnly -- test.examples.NestedTests.outer1.{inner1,inner2}
testOnly -- test.examples.NestedTests.{outer1.inner1,outer2.inner3}
testOnly -- {test.examples.HelloTests.test1,test.examples.NestedTests.outer2}

We could copy that, but we'd probably also want more flexibility, e.g.

mill run {Core,ScalaPlugin}.test
# Run all tasks ending in `.test`; might need to pick a different syntax for bash-compatibility
mill run *.test

Notably, whatever syntax we choose should be able to play nicely with tasks taking command-line arguments

mill run Core.test mill.define.ApplicativeTests

And hopefully we'll be able to find something with less awkward quoting than SBT's

sbt "core/testOnly -- mill.define.ApplicativeTests"

Cross-compiling across minor versions of Scala is broken

Apply this patch:

lihaoyi mill$ git diff
diff --git a/scalaplugin/src/test/scala/mill/scalaplugin/AcyclicTests.scala b/scalaplugin/src/test/scala/mill/scalaplugin/AcyclicTests.scala
index 7ba8b26..89ed57c 100644
--- a/scalaplugin/src/test/scala/mill/scalaplugin/AcyclicTests.scala
+++ b/scalaplugin/src/test/scala/mill/scalaplugin/AcyclicTests.scala
@@ -9,7 +9,7 @@ import utest._
 import mill.util.JsonFormatters._
 object AcyclicBuild{
   val acyclic =
-    for(crossVersion <- Cross("2.10.6", "2.11.8", "2.12.4"))
+    for(crossVersion <- Cross("2.10.6", "2.11.8", "2.12.3"))
     yield new ScalaModule{outer =>
       def basePath = AcyclicTests.workspacePath
       def organization = "com.lihaoyi"
@@ -46,7 +46,7 @@ object AcyclicTests extends TestSuite{

     'scala210 - check("2.10.6")
     'scala211 - check("2.11.8")
-    'scala212 - check("2.12.4")
+    'scala212 - check("2.12.3")

     val allBinaryVersions = Seq("2.10", "2.11", "2.12")
     def check(scalaVersion: String) = {

Run the test:

sbt "scalaplugin/test-only -- mill.scalaplugin.AcyclicTests.scala212"

See it blow up:

------------- Running Tests mill.scalaplugin.AcyclicTests.scala212 -------------
[info] Compiling 5 Scala sources to /Users/lihaoyi/Dropbox/Workspace/mill/target/workspace/acyclic/acyclic/2.12.3/compile/classes ...
[error] ## Exception when compiling 5 sources to /Users/lihaoyi/Dropbox/Workspace/mill/target/workspace/acyclic/acyclic/2.12.3/compile/classes
[error] vtable stub
[error] java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:488)
[error] java.lang.StringBuilder.append(StringBuilder.java:166)
[error] xsbt.ClassName.$anonfun$classNameAsSeenIn$1(ClassName.scala:83)
[error] scala.reflect.internal.SymbolTable.enteringPhase(SymbolTable.scala:237)
[error] xsbt.ClassName.classNameAsSeenIn(ClassName.scala:78)
[error] xsbt.ClassName.classNameAsSeenIn$(ClassName.scala:76)
[error] xsbt.ExtractAPI.classNameAsSeenIn(ExtractAPI.scala:49)
[error] xsbt.ExtractAPI.mkClassLike(ExtractAPI.scala:637)
[error] xsbt.ExtractAPI.$anonfun$classLike$1(ExtractAPI.scala:620)
[error] scala.collection.mutable.HashMap.getOrElseUpdate(HashMap.scala:82)
[error] xsbt.ExtractAPI.classLike(ExtractAPI.scala:620)
[error] xsbt.ExtractAPI.extractAllClassesOf(ExtractAPI.scala:605)
[error] xsbt.API$TopLevelHandler.class(API.scala:65)
[error] xsbt.API$TopLevelTraverser.traverse(API.scala:73)
[error] xsbt.API$TopLevelTraverser.traverse(API.scala:69)
[error] scala.reflect.api.Trees$Traverser.$anonfun$traverseStats$2(Trees.scala:2498)
[error] scala.reflect.api.Trees$Traverser.atOwner(Trees.scala:2507)
[error] scala.reflect.api.Trees$Traverser.$anonfun$traverseStats$1(Trees.scala:2498)
[error] scala.reflect.api.Trees$Traverser.traverseStats(Trees.scala:2497)
[error] scala.reflect.internal.Trees.itraverse(Trees.scala:1337)
[error] scala.reflect.internal.Trees.itraverse$(Trees.scala:1211)
[error] scala.reflect.internal.SymbolTable.itraverse(SymbolTable.scala:16)
[error] scala.reflect.internal.SymbolTable.itraverse(SymbolTable.scala:16)
[error] scala.reflect.api.Trees$Traverser.traverse(Trees.scala:2475)
[error] xsbt.API$TopLevelTraverser.traverse(API.scala:75)
[error] xsbt.API$TopLevelTraverser.traverse(API.scala:69)
[error] scala.reflect.api.Trees$Traverser.$anonfun$traverseStats$2(Trees.scala:2498)
[error] scala.reflect.api.Trees$Traverser.atOwner(Trees.scala:2507)
[error] scala.reflect.api.Trees$Traverser.$anonfun$traverseStats$1(Trees.scala:2498)
[error] scala.reflect.api.Trees$Traverser.traverseStats(Trees.scala:2497)
[error] scala.reflect.internal.Trees.itraverse(Trees.scala:1337)
[error] scala.reflect.internal.Trees.itraverse$(Trees.scala:1211)
[error] scala.reflect.internal.SymbolTable.itraverse(SymbolTable.scala:16)
[error] scala.reflect.internal.SymbolTable.itraverse(SymbolTable.scala:16)
[error] scala.reflect.api.Trees$Traverser.traverse(Trees.scala:2475)
[error] xsbt.API$TopLevelTraverser.traverse(API.scala:75)
[error] xsbt.API$TopLevelTraverser.traverse(API.scala:69)
[error] scala.reflect.api.Trees$Traverser.apply(Trees.scala:2513)
[error] xsbt.API$ApiPhase.processScalaUnit(API.scala:43)
[error] xsbt.API$ApiPhase.processUnit(API.scala:35)
[error] xsbt.API$ApiPhase.apply(API.scala:33)
[error] scala.tools.nsc.Global$GlobalPhase.$anonfun$applyPhase$1(Global.scala:426)
[error] scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:419)
[error] scala.tools.nsc.Global$GlobalPhase.$anonfun$run$1(Global.scala:390)
[error] scala.tools.nsc.Global$GlobalPhase.$anonfun$run$1$adapted(Global.scala:390)
[error] scala.collection.Iterator.foreach(Iterator.scala:929)
[error] scala.collection.Iterator.foreach$(Iterator.scala:929)
[error] scala.collection.AbstractIterator.foreach(Iterator.scala:1417)
[error] scala.tools.nsc.Global$GlobalPhase.run(Global.scala:390)
[error] xsbt.API$ApiPhase.run(API.scala:27)
[error] scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1431)
[error] scala.tools.nsc.Global$Run.compileUnits(Global.scala:1416)
[error] scala.tools.nsc.Global$Run.compileSources(Global.scala:1412)
[error] scala.tools.nsc.Global$Run.compile(Global.scala:1515)
[error] xsbt.CachedCompiler0.run(CompilerInterface.scala:131)
[error] xsbt.CachedCompiler0.run(CompilerInterface.scala:106)
[error] xsbt.CompilerInterface.run(CompilerInterface.scala:32)
[error] sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.lang.reflect.Method.invoke(Method.java:498)
[error] sbt.internal.inc.AnalyzingCompiler.call(AnalyzingCompiler.scala:237)
[error] sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:111)
[error] sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:90)
[error] sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$3(MixedAnalyzingCompiler.scala:83)
[error] scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
[error] sbt.internal.inc.MixedAnalyzingCompiler.timed(MixedAnalyzingCompiler.scala:134)
[error] sbt.internal.inc.MixedAnalyzingCompiler.compileScala$1(MixedAnalyzingCompiler.scala:74)
[error] sbt.internal.inc.MixedAnalyzingCompiler.compile(MixedAnalyzingCompiler.scala:117)
[error] sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1(IncrementalCompilerImpl.scala:305)
[error] sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1$adapted(IncrementalCompilerImpl.scala:305)
[error] sbt.internal.inc.Incremental$.doCompile(Incremental.scala:101)
[error] sbt.internal.inc.Incremental$.$anonfun$compile$4(Incremental.scala:82)
[error] sbt.internal.inc.IncrementalCommon.recompileClasses(IncrementalCommon.scala:117)
[error] sbt.internal.inc.IncrementalCommon.cycle(IncrementalCommon.scala:64)
[error] sbt.internal.inc.Incremental$.$anonfun$compile$3(Incremental.scala:84)
[error] sbt.internal.inc.Incremental$.manageClassfiles(Incremental.scala:129)
[error] sbt.internal.inc.Incremental$.compile(Incremental.scala:75)
[error] sbt.internal.inc.IncrementalCompile$.apply(Compile.scala:70)
[error] sbt.internal.inc.IncrementalCompilerImpl.compileInternal(IncrementalCompilerImpl.scala:309)
[error] sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileIncrementally$1(IncrementalCompilerImpl.scala:267)
[error] sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError(IncrementalCompilerImpl.scala:158)
[error] sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally(IncrementalCompilerImpl.scala:237)
[error] sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:68)
[error] mill.scalaplugin.ScalaModule$.compileScala(ScalaModule.scala:129)
[error] mill.scalaplugin.ScalaModule.$anonfun$compile$3(ScalaModule.scala:327)
[error] mill.eval.Result$.create(Result.scala:6)
[error] mill.scalaplugin.ScalaModule.$anonfun$compile$2(ScalaModule.scala:328)
[error] mill.define.Applicative$Applyer.$anonfun$zipMap$7(Applicative.scala:43)
[error] mill.define.Task$MappedDest.evaluate(Task.scala:186)
[error] mill.eval.Evaluator.$anonfun$evaluateGroup$7(Evaluator.scala:167)
[error] mill.eval.Evaluator.$anonfun$evaluateGroup$7$adapted(Evaluator.scala:156)
[error] scala.collection.Iterator.foreach(Iterator.scala:929)
[error] scala.collection.Iterator.foreach$(Iterator.scala:929)
[error] scala.collection.AbstractIterator.foreach(Iterator.scala:1417)
[error] scala.collection.IterableLike.foreach(IterableLike.scala:71)
[error] scala.collection.IterableLike.foreach$(IterableLike.scala:70)
[error] scala.collection.AbstractIterable.foreach(Iterable.scala:54)
[error] mill.eval.Evaluator.evaluateGroup(Evaluator.scala:156)
[error] mill.eval.Evaluator.evaluateGroupCached(Evaluator.scala:107)
[error] mill.eval.Evaluator.$anonfun$evaluate$2(Evaluator.scala:33)
[error] mill.eval.Evaluator.$anonfun$evaluate$2$adapted(Evaluator.scala:29)
[error] scala.collection.Iterator.foreach(Iterator.scala:929)
[error] scala.collection.Iterator.foreach$(Iterator.scala:929)
[error] scala.collection.AbstractIterator.foreach(Iterator.scala:1417)
[error] mill.eval.Evaluator.evaluate(Evaluator.scala:29)
[error] mill.scalaplugin.TestEvaluator$.eval(TestEvaluator.scala:21)
[error] mill.scalaplugin.AcyclicTests$.eval$1(AcyclicTests.scala:43)
[error] mill.scalaplugin.AcyclicTests$.check$1(AcyclicTests.scala:64)
[error] mill.scalaplugin.AcyclicTests$.$anonfun$tests$76(AcyclicTests.scala:49)
[error] utest.framework.StackMarker$.dropOutside(StackMarker.scala:13)
[error] utest.framework.TestCallTree.run(Model.scala:35)
[error] utest.framework.TestCallTree.run(Model.scala:33)
[error] utest.TestRunner$.$anonfun$runAsync$5(TestRunner.scala:74)
[error] utest.framework.Executor.utestWrap(Executor.scala:12)
[error] utest.framework.Executor.utestWrap$(Executor.scala:10)
[error] utest.TestSuite.utestWrap(TestSuite.scala:18)
[error] utest.TestRunner$.$anonfun$runAsync$4(TestRunner.scala:71)
[error] utest.framework.StackMarker$.dropOutside(StackMarker.scala:13)
[error] utest.TestRunner$.$anonfun$runAsync$2(TestRunner.scala:71)
[error] utest.TestRunner$.evaluateFutureTree(TestRunner.scala:170)
[error] utest.TestRunner$.$anonfun$evaluateFutureTree$2(TestRunner.scala:173)
[error] scala.concurrent.Future$.$anonfun$traverse$1(Future.scala:841)
[error] scala.collection.IndexedSeqOptimized.foldLeft(IndexedSeqOptimized.scala:56)
[error] scala.collection.IndexedSeqOptimized.foldLeft$(IndexedSeqOptimized.scala:64)
[error] scala.collection.mutable.ArrayBuffer.foldLeft(ArrayBuffer.scala:48)
[error] scala.concurrent.Future$.traverse(Future.scala:841)
[error] utest.TestRunner$.evaluateFutureTree(TestRunner.scala:173)
[error] utest.TestRunner$.runAsync(TestRunner.scala:98)
[error] utest.runner.BaseRunner.runSuite(BaseRunner.scala:160)
[error] utest.runner.BaseRunner.$anonfun$makeTask$1(BaseRunner.scala:171)
[error] utest.runner.Task.execute(Task.scala:20)
[error] sbt.TestRunner.runTest$1(TestFramework.scala:76)
[error] sbt.TestRunner.run(TestFramework.scala:85)
[error] sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202)
[error] sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202)
[error] sbt.TestFramework$.sbt$TestFramework$$withContextLoader(TestFramework.scala:185)
[error] sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202)
[error] sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202)
[error] sbt.TestFunction.apply(TestFramework.scala:207)
[error] sbt.Tests$.sbt$Tests$$processRunnable$1(Tests.scala:239)
[error] sbt.Tests$$anonfun$makeSerial$1.apply(Tests.scala:245)
[error] sbt.Tests$$anonfun$makeSerial$1.apply(Tests.scala:245)
[error] sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44)
[error] sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44)
[error] sbt.std.Transform$$anon$4.work(System.scala:63)
[error] sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
[error] sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
[error] sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
[error] sbt.Execute.work(Execute.scala:237)
[error] sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
[error] sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
[error] sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
[error] sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
[error] java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error] java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error] java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error] java.lang.Thread.run(Thread.java:748)
[error]
X mill.scalaplugin.AcyclicTests.scala212 18260ms
  scala.MatchError: Left(Exception(java.lang.IncompatibleClassChangeError: vtable stub)) (of class s
  cala.util.Left)
    mill.scalaplugin.AcyclicTests$.check$1(AcyclicTests.scala:64)
    mill.scalaplugin.AcyclicTests$.$anonfun$tests$76(AcyclicTests.scala:49)
Tests: 1, Passed: 0, Failed: 1
[error] Failed tests:
[error] 	mill.scalaplugin.AcyclicTests
[error] (scalaplugin/test:testOnly) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 46 s, completed 29 Nov, 2017 7:53:48 PM

Build Gitbucket using Mill

https://github.com/gitbucket/gitbucket seems like a nice project to shoot for: an open source application (rather than a library), but a relatively simple one with a relatively simple build (v.s. something like https://github.com/ornicar/lila which has a bajillion submodules), that uses some SBT plugins but not too many.

We should write a Mill build for Gitbucket, both to stretch Mill's capabilities (and discover any missing features that we need to support) as well as as an example build for the silent majority of Scala programmers building applications to do real work, rather than playing with open-source libraries.

ScalaTest tests do not appear to be running 

sbt "~integration/testOnly -- mill.integration.JawnTests.scala2123"

Running jawn[2.12.3].Parser.test.test
CharBuilderSpec:
ChannelSpec:
JNumIndexCheck:
SyntaxCheck:

We should figure out why and get it running

Task Sandboxing

Bazel restricts tasks being run to their given work directory: using sandbox-exec on OS-X, and LXC Containers on Linux. This is extremely useful to ensure that

  • Badly-configured builds don't "accidentally" pass due to stale state lying around the filesystem
  • Those builds don't fail mysteriously when that stale state is removed
  • The presence of stale state doesn't cause other builds to fail mysteriously

We can do the same thing, at least to a best effort: using Java SecurityManagers to limit file access in Mill JVM code, and forcing people to use a "blessed" subprocess interface that restricts subprocesses using the same OS tools that Bazel uses.

Builds becoming brittle because parts of it accidentally/implicitly depend on other parts that "have to"/"have always been" run before-hand is a common problem, and this would fix it.

Changes in build don't invalidate caches

Minimal reproducible:

mkdir mill-repr
cd mill-repr

echo 'import mill._
import mill.scalaplugin._
import ammonite.ops._

object Invalidate extends ScalaModule {
  def scalaVersion = "2.12.4"
  def basePath = pwd
  override def mainClass = Some("Main")

  override def ivyDeps = Seq(
    Dep("io.circe", "circe-core", "0.8.0"),
    Dep("io.circe", "circe-parser", "0.8.0")
  )
}' > build.sc

mkdir -p src/main/scala

echo 'object Main extends App {
  println(io.circe.parser.parse("""{"a": 2}"""))
}' > src/main/scala/Main.scala

mill run Invalidate.run

now change circe version to 0.9999.0 and run mill run Invalidate.run once again.
Expected behavior: mill should invalidate build caches and fail at resolving/compilation.
Actual behavior: build ran as the previous one

If you rm -rf out and run mill run Invalidate.run once again - you'll get error at compilation

Tighten up Ammonite script integration

Currently, the integration with Ammonite is pretty hacky and fragile. e.g. compile errors in build.sc files are treated badly:

lihaoyi mill$ git diff
diff --git a/build.sc b/build.sc
index f58c160..28d5706 100755
--- a/build.sc
+++ b/build.sc
@@ -18,7 +18,7 @@ trait MillModule extends ScalaModule{ outer =>
     def testFramework = "mill.UTestFramework"
   }
 }
-
+?
 object Core extends MillModule {
   override def compileIvyDeps = Seq(
     Dep.Java("org.scala-lang", "scala-reflect", scalaVersion())
lihaoyi mill$ scalaplugin/target/mill run Core.test
Compiling /Users/lihaoyi/Dropbox/Workspace/mill/build.sc
build.sc:21: not found: value ?
val res_4 = ?
            ^
(Failure(Compilation Failed),ArrayBuffer((/Users/lihaoyi/.ammonite/predef.sc,0), (/Users/lihaoyi/Dropbox/Workspace/mill/build.sc,1512792521000)))
Exception in thread "main" scala.NotImplementedError: an implementation is missing
	at scala.Predef$.$qmark$qmark$qmark(Predef.scala:284)
	at mill.Main.run(Main.scala:286)
	at mill.Main$.main(Main.scala:227)
	at mill.Main.main(Main.scala)

Ammonite's own CLI script-running functionality is rock solid, even in failure cases. We should make sure Mill's is of similar quality: informative error messages, no uncaught stack traces, and well-behaved in all cases including things like watch-&-rerun.

bug: console is not working

The new definition of mill.modules.Jvm.subprocess doesn't seem to work with interactive processes like scala console. Old way to run subprocess seems to work fine:

def subprocess(mainClass: String,
               classPath: Seq[Path],
               options: Seq[String] = Seq.empty) = {
  import ammonite.ops.ImplicitWd._
  %("java", "-cp", classPath.mkString(":"), mainClass, options)
}

If we are okay with two implementations, here is the fix

Incremental Scala compile with Zinc isn't working right

To repro:

git clean -xdf
sbt scalaplugin/test:assembly
scalaplugin/target/mill run ScalaPlugin.assembly
echo " " >> core/src/main/scala/mill/define/Graph.scala
scalaplugin/target/mill run ScalaPlugin.assembly

The last command will run into a bunch of compile errors, when it should compile successfully

Set up CI infrastructure

Sooner or later, we'll need a CI for mill. We could start it sooner, to have more confidence when merging PR's

Scala.js testing framework support

Tests for ScalaJSModule should be executed with JavaScript engine. We probably don't need to support all JavaScript engines available in SBT, but having DOM support looks like a necessity.

Commands should be given a `dest` directory

Currently T.ctx().dest only works for labelled Targets, which are publicly accessible. Tasks (which don't have a label) and overriden Targets are not given dest directories, because it's not clear what path that dest directory would have.

Currently, Commands do not have a dest directory. Given that the Commands do have a unique path by which they are run, they too should have a dest path assigned to them for them to use for scratch & output files. This would let us avoid creating temp dirs as we do here https://github.com/lihaoyi/mill/blob/bb61c05217671e80ba381b2cac869130c306baa8/scalaplugin/src/main/scala/mill/scalaplugin/ScalaModule.scala#L20

Idea task should not depend on compilation success

It was a common problem in the days of sbt-idea and now we have the same issue in Mill — idea task depends on the compilation results, so if my code is broken I cannot import a project into IDEA to fix it.

Extract Target-selection logic out of Main and write some unit tests

https://github.com/lihaoyi/mill/blob/1382e27191e662da2be0ec904c6266c7f21bc68d/core/src/main/scala/mill/Main.scala#L25-L112
This is currently a bit messy, but the logic is complex enough (with enough error conditions) that it deserves some proper testing.

Cross-building with mill Core.cross[a].printIt works, but mill Core.cross[doesntExist].printIt fails with an ugly exception, and it would be great if we could tighten up error handling and lock it down with unit tests as part of this effort

Watch & re-run functionality in the REPL

You currently can use Ammonite's --watch flag to watch & re-run targets from the Bash shell. When used to run the REPL, --watch only serves to re-start the REPL if you exited it after making changes.

We should provide a watch(Core.compile) function that can be used to watch & re-run a one (or more) targets from Build REPL

Enable just-in-time compilation of compiler-bridge

The Zinc incremental compiler requires versions of org.scala-sbt::compiler-bridge compiled for exactly the scala-compiler version you are using. Currently we pre-compile a few versions of compiler-bridge in our own build and use those, but we don't pre-compile all the publicly released versions (because it would take very long...) and we can't have pre-compiled versions for snapshots or other odd versions of Scala that may be floating around (type-level, release candidates, milestones, ...)

The correct thing to do is to download and compile compiler-bridge on-demand as part of a build, and cache/re-use it there-after.

Investigate Mill vs SBT performance

My own subjective feeling is that Mill is slower at the watch-edit-recompile-test workflow than SBT is. Previously this was largely due to Mill not keeping around warm copies of Scalac, but that should have been fixed already. Another point is that Mill doesn't currently run independent tasks in parallel, while SBT does.

This task is to investigate the relative performance of Mill vs SBT at the common watch-edit-recompile-test, to understand what the performance differences are and exactly what is causing them. There's no fundamental reason we should be any slower, and understanding our performance is the first step in making sure we're up to par

Implement `eval` function to let user build multiple targets in the REPL

Currently, you can only select one target at a time:

Core.compile()
Core.test.compile()
Core.test.test()
ScalaPlugin.assembly()

Mill's evaluate function already lets you select a Vector of Tasks you want to build. We should expose that to the user in the REPL:

val (a, b) = eval(Core.compile, Core.test.compile)
val (a, b, c) = eval(Core.compile, Core.test.compile, ScalaPlugin.assembly)

`mill.Module`s should have a default `basePath`

e.g. Core's should default to Core/ (or maybe core/?), bridges[2.10.6] should default to bridges/2.10.6/, and so on.

This can be used to automatically have e.g. ScalaModule pick up sources in the right place, rather than having to specify the basePath manually each time. For now Modules would be able to continue picking up sources outside the basePath, but this at least let's people provide a nice default, and later on if we wanted we could restrict Modules to only picking up sources within their allocated basePath

Pass analysis of dependent projects to Zinc

Zinc needs a lookup implementation that maps directories on the classpath to an Analysis object. This allows Zinc to "know" when to recompile sources when they depend on other sub-projects in the build. Right now, dependent projects would always be recompiled, regardless of what changed or not. See eclipse implementation

Code-generate .zipMap functions to support Applicative macro

Currently, these are defined manually, up to a maximum arity of 7. That means a T{...} call can only depend on up to 7 other tasks: an arbitrary limitation.

We should use source-code-generation to generate the zipMap functions up to the "standard" arity of 22, which is the limit on how many args the function literal can have. In doing so, we'd also need to flesh out the source-code-generation story for ScalaModule, making sure it's easy to generate and add arbitrary sources to the build (which would require changes to compileScala)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.