Giter VIP home page Giter VIP logo

util's Introduction

Twitter Util

Build Status Project status Gitter Maven Central

A bunch of idiomatic, small, general purpose tools.

See the Scaladoc here or check out the user guide.

Status

This project is used in production at Twitter (and many other organizations), and is being actively developed and maintained.

Releases

Releases are done on an approximately monthly schedule. While semver is not followed, the changelogs are detailed and include sections on public API breaks and changes in runtime behavior.

Contributing

We feel that a welcoming community is important and we ask that you follow Twitter's Open Source Code of Conduct in all interactions with the community.

The release branch of this repository contains the latest stable release of Util, and weekly snapshots are published to the develop branch. In general pull requests should be submitted against develop. See CONTRIBUTING.md for more details about how to contribute.

Using in your project

An example SBT dependency string for the util-core library would look like this:

val utilCore = "com.twitter" %% "util-core" % "23.11.0"

Units

Time

import com.twitter.conversions.DurationOps._

val duration1 = 1.second
val duration2 = 2.minutes
duration1.inMillis // => 1000L

Space

import com.twitter.conversions.StorageUnitOps._
val amount = 8.megabytes
amount.inBytes // => 8388608L
amount.inKilobytes // => 8192L

Futures

A Non-actor re-implementation of Scala Futures.

import com.twitter.conversions.DurationOps._
import com.twitter.util.{Await, Future, Promise}

val f = new Promise[Int]
val g = f.map { result => result + 1 }
f.setValue(1)
Await.result(g, 1.second) // => this blocks for the futures result (and eventually returns 2)

// Another option:
g.onSuccess { result =>
  println(result) // => prints "2"
}

// Using for expressions:
val xFuture = Future(1)
val yFuture = Future(2)

for {
  x <- xFuture
  y <- yFuture
} {
  println(x + y) // => prints "3"
}

Future interrupts

Method raise on Future (def raise(cause: Throwable)) raises the interrupt described by cause to the producer of this Future. Interrupt handlers are installed on a Promise using setInterruptHandler, which takes a partial function:

val p = new Promise[T]
p.setInterruptHandler {
  case exc: MyException =>
    // deal with interrupt..
}

Interrupts differ in semantics from cancellation in important ways: there can only be one interrupt handler per promise, and interrupts are only delivered if the promise is not yet complete.

Object Pool

The pool order is FIFO.

A pool of constants

import scala.collection.mutable
import com.twitter.util.{Await, SimplePool}

val queue = new mutable.Queue[Int] ++ List(1, 2, 3)
val pool = new SimplePool(queue)

// Note that the pool returns Futures, it doesn't block on exhaustion.
assert(Await.result(pool.reserve()) == 1)
pool.reserve().onSuccess { item =>
  println(item) // prints "2"
}

A pool of dynamically created objects

Here is a pool of even-number generators. It stores 4 numbers at a time:

import com.twitter.util.{Future, FactoryPool}

val pool = new FactoryPool[Int](4) {
  var count = 0
  def makeItem() = { count += 1; Future(count) }
  def isHealthy(i: Int) = i % 2 == 0
}

It checks the health when you successfully reserve an object (i.e., when the Future yields).

Hashing

util-hashing is a collection of hash functions and hashing distributors (eg. ketama).

To use one of the available hash functions:

import com.twitter.hashing.KeyHasher

KeyHasher.FNV1_32.hashKey("string".getBytes)

Available hash functions are:

FNV1_32
FNV1A_32
FNV1_64
FNV1A_64
KETAMA
CRC32_ITU
HSIEH

To use KetamaDistributor:

import com.twitter.hashing.{KetamaDistributor, KetamaNode, KeyHasher}

val nodes = List(KetamaNode("host:port", 1 /* weight */, "foo" /* handle */))
val distributor = new KetamaDistributor(nodes, 1 /* num reps */)
distributor.nodeForHash("abc".##) // => client

Time and Duration

Like arithmetic on doubles, Time and Duration arithmetic is now free of overflows. Instead, they overflow to Top and Bottom values, which are analogous to positive and negative infinity.

Since the resolution of Time.now has been reduced (and is also more expensive due to its use of system time), a new Stopwatch API has been introduced in order to calculate durations of time.

It's used simply:

import com.twitter.util.{Duration, Stopwatch}
val elapsed: () => Duration = Stopwatch.start()

which is read by applying elapsed:

val duration: Duration = elapsed()

License

Copyright 2010 Twitter, Inc.

Licensed under the Apache License, Version 2.0: https://www.apache.org/licenses/LICENSE-2.0

util's People

Contributors

aliciavargas avatar bierbaum avatar bonniee avatar cacoco avatar dotordogh avatar enbnt avatar freels avatar grimreaper avatar hamdiallam avatar isabelmartin avatar j3h avatar jcrawford13 avatar jcrossley avatar joybestourous avatar jyanjing avatar kevinoliver avatar luciferous avatar mariusae avatar mattdickinson5 avatar mjeffryes avatar mosesn avatar patliu85 avatar ryanoneill avatar sprsquish avatar stevegury avatar tigerlily-he avatar travisbrown avatar vkostyukov avatar wisechengyi avatar yufangong avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

util's Issues

ScribeHandler's flush operation shall be async

I believe ScribeHandler's flush operation shall be async otherwise the while-loop inside the flush method might take a long time thus cause the public logging method to take very long.

    while (offset < response.length) {
      val n = inStream.read(response, offset, response.length - offset)
      if (n < 0) {
        throw new IOException("End of stream")
      }
      offset += n
      if (!archaicServer && (offset > 0) && (response(0) == 0)) {
        archaicServer = true
        close()
        lastConnectAttempt = Time.epoch
        log.error("Scribe server is archaic; retrying with old protocol.")
        throw new Retry
      }
    }

I'm using ScribeHandler to write logs to a scribe server, I'm observing that occasionally there are tens of milliseconds delay between two next-to-each-other log.debug() clauses. This happens very rarely but do repeat.

Thanks.

~Dong

MessageFormat not used when formatting LogRecord

If you're calling Java code that uses JUL you may find the log records are formatted incorrectly. ie.

public static MyJavaDep
{
    private static Logger logger= Logger.getLogger( getClass().getName() );

    public void sayHello( String name )
    {
        logger.log( Level.INFO, "Hello {0}, how are you?", name );
    }
}

If called by Java code you'll see "Hello Bob, how are you?". When called from Scala the log message will not resolve {0} to be name and you'll see "Hello {0}, how are you?".

Internally it appears like util-logging uses the StringFormat instead of MessageFormat. It would be great if we could use the StringFormat notation when logging via util-logging, but still have dependent Java code log via MessageFormat as expected.

deleting individual util api docs

We have unified api docs now, so there isn't a need for individual util api docs. There shouldn't be that many links to them anymore, but if someone is depending heavily on them, just be aware that they're disappearing soon.

Duration.Top is not preserved by .fromNanoseconds

scala> import com.twitter.util.Duration
import com.twitter.util.Duration

Duration.Top is preserved by .fromMilliseconds (relevant code):

scala> Duration.fromMilliseconds(Duration.Top.inMilliseconds + 1)
res2: com.twitter.util.Duration = Duration.Top

scala> Duration.fromMilliseconds(Duration.Top.inMilliseconds + 1) == Duration.Top
res4: Boolean = true

But not by .fromNanoseconds (relevant code):

scala> Duration.fromNanoseconds(Duration.Top.inNanoseconds)
res6: com.twitter.util.Duration = 106751.days+23.hours+47.minutes+16.seconds+854.milliseconds+775.microseconds+807.nanoseconds

scala> Duration.fromNanoseconds(Duration.Top.inNanoseconds) == Duration.Top
res7: Boolean = false

scala> Duration.fromNanoseconds(Duration.Top.inNanoseconds).inNanoseconds == Duration.Top.inNanoseconds
res8: Boolean = true

sbt-package-dist;1.0.5 not found when building locally

When loading up in SBT, I get an unresolved dependency:

[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: com.twitter#sbt-package-dist;1.0.5: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn] 
[warn]  Note: Some unresolved dependencies have extra attributes.  Check that these dependencies exist with the requested attributes.
[warn]      com.twitter:sbt-package-dist:1.0.5 (sbtVersion=0.11.3, scalaVersion=2.9.1)
[warn] 
[error] {file:/home/dev/util/project/}default-d2018f/*:update: sbt.ResolveException: unresolved dependency: com.twitter#sbt-package-dist;1.0.5: not found

Removal of Var.memo() in 6.7.0 breaks compatibility with Finagle 6.7.1

While using Finagle 6.7.1 in my project I've bumped the util-core dependency to the latest 6.7.0. The result is

Exception in thread "main" java.lang.NoSuchMethodError: com.twitter.util.Var.memo()Lcom/twitter/util/Var;
    at com.twitter.finagle.Group$class.set(Group.scala:46)
    at com.twitter.finagle.Group$$anon$2.set(Group.scala:153)
    at com.twitter.finagle.Group$$anon$3.<init>(Group.scala:63)
    at com.twitter.finagle.Group$class.collect(Group.scala:60)
    at com.twitter.finagle.Group$$anon$2.collect(Group.scala:153)
    at com.twitter.finagle.Group$class.map(Group.scala:53)
    at com.twitter.finagle.Group$$anon$2.map(Group.scala:153)
    at com.twitter.finagle.client.DefaultClient$$anonfun$19.apply(DefaultClient.scala:186)
    at com.twitter.finagle.client.DefaultClient$$anonfun$19.apply(DefaultClient.scala:164)
    at scala.Function1$$anonfun$compose$1.apply(Function1.scala:49)
    at com.twitter.finagle.factory.Refinery.<init>(Refinery.scala:20)
    at com.twitter.finagle.client.DefaultClient$$anonfun$21.apply(DefaultClient.scala:199)
    at com.twitter.finagle.client.DefaultClient$$anonfun$21.apply(DefaultClient.scala:199)
    at com.twitter.finagle.builder.ClientBuilder.buildFactory(ClientBuilder.scala:815)
    โ€ฆ

Formally, the removed Var.memo() was in public API, so either it should have been a major release, or, preferrably, the method should have been marked as deprecated, in case if anyone has already used it.

RichU64String.toU64Long "parses" single digit negative numbers

This is extremely unimportant and may even "not a bug", but I was surprised when I did:

val longVal = "-1".toU64Long

In this case, longVal is 255. Seems to me like this should probably throw NumberFormatException, since it isn't a valid unsigned 64-bit string? Other example values that return numbers:

-f
-f00
10-f00

I'm happy to submit a patch if that makes sense.

Support scala 2.11

We would like to support scala 2.11.

2.11 needs to be added to the list of cross-built scala versions, which is simple. The bigger problem is that not every scala library has 2.11 versions yet, or will have 2.11 versions in the future.

Blockers

scalatest

scalatest has only published one version against 2.11 at the time of this issue, and it's from the scalatest 2 major version. Either we'll have to ask the scalatest maintainers to publish an old version of scalatest against 2.11, or go figure out how to upgrade. Because of the way that util is used inside of twitter, it will require a good deal of behind the scenes work to upgrade the scalatest version if we're using features which have broken api compatibility.

specs

specs hasn't published a new 2.11 version yet, and has been deprecated for a while in favor of specs2. There are two ways forward--we can upgrade to specs2, since there's a clear migration path for specs to specs2. The other way forward is to switch from specs to scalatest. In the long term, we want all of our tests to be in scalatest. On the other hand, we have a ton of tests in specs. In util alone, I found 55.

Please reach out if you would like to help, or if you've found another blocker.

util-eval: Question about classpath in executable jar

Hello.

At https://github.com/twitter/util/blob/master/util-eval/src/main/scala/com/twitter/util/Eval.scala#L338 it uses assumption that executable jar have just one, but on my debian server with last oracle jdk i have such class path:

List(List(/opt/staging/webapp/webapp/dist/webapp_2.10-0.1.jar, /opt/staging/webapp/newrelic/newrelic.jar), List(/usr/lib/jvm/j2sdk1.7-oracle/jre/lib/ext/localedata.jar, /usr/lib/jvm/j2sdk1.7-oracle/jre/lib/ext/sunpkcs11.jar, /usr/lib/jvm/j2sdk1.7-oracle/jre/lib/ext/dnsns.jar, /usr/lib/jvm/j2sdk1.7-oracle/jre/lib/ext/sunjce_provider.jar, /usr/lib/jvm/j2sdk1.7-oracle/jre/lib/ext/zipfs.jar, /usr/lib/jvm/j2sdk1.7-oracle/jre/lib/ext/sunec.jar))

I want to point that java agent (-javaagent) also in classpath - of course mentioned assumption is wrong.
Because of this my app could not have access in Eval to other jars.

Could you advise something to this?

util-logging: Guide for configuring in README is deprecated

This code in README:

import com.twitter.logging.config._

val config = new LoggerConfig {
  node = ""
  level = Logger.INFO
  handlers = new FileHandlerConfig {
    filename = "/var/log/example/example.log"
    roll = Policy.SigHup
  }
}
config()

gives compilation warning:

class LoggerConfig in package config is deprecated: use LoggerFactory
class FileHandlerConfig in package config is deprecated: use HandlerFactory

Did util-core not support 2.11.4 yet?

Did twitter util-core not support 2.11.4 yet?
I got this exception when I add "com.twitter" %% "util-core" % "6.12.1" to my build.sbt.

[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: com.twitter#util-core_2.11;6.12.1: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
sbt.ResolveException: unresolved dependency: com.twitter#util-core_2.11;6.12.1: not found

here is my build.sbt

name := "hello_fsm"

version := "1.0"

scalaVersion := "2.11.4"

resolvers += "Twitter" at "http://maven.twttr.com"

libraryDependencies ++= Seq(
  "com.typesafe.akka" %% "akka-actor" % "2.3.5",
  "com.typesafe.akka" %% "akka-slf4j" % "2.3.5",
  "com.typesafe.akka" %% "akka-testkit" % "2.3.5",
  "com.twitter" %% "util-core" % "6.12.1",
  "org.scalatest" % "scalatest_2.11" % "2.2.1" % "test"
)

libraryDependencies += "ch.qos.logback" % "logback-classic" % "1.0.13"

java.lang.UnsupportedOperationException: Current thread CPU time measurement is not supported.

Was trying to deploy some service using finagle on FreeBSD, and Scheduler throws

WARNING: Unhandled exception in connection with /192.168.93.33:63593 , shutting down connection
java.lang.UnsupportedOperationException: Current thread CPU time measurement is not supported.
        at sun.management.ThreadImpl.verifyCurrentThreadCpuTime(ThreadImpl.java:206)
        at sun.management.ThreadImpl.getCurrentThreadCpuTime(ThreadImpl.java:213)
        at com.twitter.concurrent.LocalScheduler$Activation.submit(Scheduler.scala:93)
        at com.twitter.concurrent.LocalScheduler.submit(Scheduler.scala:140)
        at com.twitter.concurrent.Scheduler$.submit(Scheduler.scala:54)
        at com.twitter.util.Promise.continue(Promise.scala:524)
        at com.twitter.util.Promise$Responder$class.transform(Promise.scala:135)
        at com.twitter.util.Promise.transform(Promise.scala:224)
        at com.twitter.util.Future.flatMap(Future.scala:670)
        at com.twitter.finagle.ServiceFactory$$anon$5.apply(Service.scala:134)
        at com.twitter.finagle.Filter$$anon$3.apply(Filter.scala:81)
        at com.twitter.finagle.server.DefaultServer.serveTransport(DefaultServer.scala:138)
        at com.twitter.finagle.server.DefaultServer$$anon$1$$anonfun$8.apply(DefaultServer.scala:160)
        at com.twitter.finagle.server.DefaultServer$$anon$1$$anonfun$8.apply(DefaultServer.scala:159)
        at com.twitter.finagle.netty3.ServerBridge.channelOpen(server.scala:289)
        at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:60)
        at com.twitter.finagle.thrift.ThriftFrameCodec.handleUpstream(ThriftFrameCodec.scala:17)
        at org.jboss.netty.channel.Channels.fireChannelOpen(Channels.java:170)
        at org.jboss.netty.channel.socket.nio.NioAcceptedSocketChannel.<init>(NioAcceptedSocketChannel.java:42)
        at org.jboss.netty.channel.socket.nio.NioServerBoss.registerAcceptedChannel(NioServerBoss.java:137)
        at org.jboss.netty.channel.socket.nio.NioServerBoss.process(NioServerBoss.java:104)
        at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
        at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)

It seems FreeBSD does not support ThreadMXBean.getCurrentThreadCpuTime(), any chance to add checking call ThreadMXBean.isCurrentThreadCpuTimeSupported() before ThreadMXBean.getCurrentThreadCpuTime() ?

failed download com.twitter#finagle-core;2.0.0!finagle-core.jar

[info] == util-zk-common / update ==
[info] downloading http://maven.twttr.com/com/twitter/finagle-core/2.0.0/finagle-core-2.0.0.jar ...
[warn] [FAILED ] com.twitter#finagle-core;2.0.0!finagle-core.jar: Downloaded file size doesn't match expected Content Length for http://maven.twttr.com/com/twitter/finagle-core/2.0.0/finagle-core-2.0.0.jar. Please retry. (52945ms)
[warn] [FAILED ] com.twitter#finagle-core;2.0.0!finagle-core.jar: Downloaded file size doesn't match expected Content Length for http://maven.twttr.com/com/twitter/finagle-core/2.0.0/finagle-core-2.0.0.jar. Please retry. (52945ms)
[warn] ==== twitter.com: tried
[warn] http://maven.twttr.com/com/twitter/finagle-core/2.0.0/finagle-core-2.0.0.jar
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: FAILED DOWNLOADS ::
[warn] :: ^ see resolution messages for details ^ ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: com.twitter#finagle-core;2.0.0!finagle-core.jar
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[info]
[warn] :: problems summary ::
[warn] :::: WARNINGS
[warn] [FAILED ] com.twitter#finagle-core;2.0.0!finagle-core.jar: Downloaded file size doesn't match expected Content Length for http://maven.twttr.com/com/twitter/finagle-core/2.0.0/finagle-core-2.0.0.jar. Please retry. (52945ms)
[warn] [FAILED ] com.twitter#finagle-core;2.0.0!finagle-core.jar: Downloaded file size doesn't match expected Content Length for http://maven.twttr.com/com/twitter/finagle-core/2.0.0/finagle-core-2.0.0.jar. Please retry. (52945ms)
[warn] ==== twitter.com: tried
[warn] http://maven.twttr.com/com/twitter/finagle-core/2.0.0/finagle-core-2.0.0.jar
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: FAILED DOWNLOADS ::
[warn] :: ^ see resolution messages for details ^ ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: com.twitter#finagle-core;2.0.0!finagle-core.jar
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[info]
[info] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
[error] sbt.ResolveException: download failed: com.twitter#finagle-core;2.0.0!finagle-core.jar

util-eval refers to classes that are removed or deprecated in scala 2.11.x

The issue is with util-eval/src/main/scala/com/twitter/util/Eval.scala, line 81:

classPathOfClass("scala.ScalaObject")

This fails because ScalaObject has been removed, after being deprecated in 2.10.

Likewise, line 75:

classPathOfClass("scala.tools.nsc.Interpreter")

refers to a class that is deprecated in 2.11.

(It's a bit odd that that this wasn't flagged by the unit tests)

Problems with transitive dependencies

I added

"com.twitter" %% "util" % "1.12.12",

to my build.sbt, but this introduced problems with transitive dependencies. thrift#libthrift can't be found (are you missing the .pom?) and jmxtools/jmxri could not be fetched.

Complete output below.

$ sbt
[info] Loading project definition from /home/yang/pod/sales/scala/project
[info] Compiling 1 Scala source to /home/yang/pod/sales/scala/project/target/scala-2.9.1/sbt-0.11.1/classes...
[info] Set current project to pod (in build file:/home/yang/pod/sales/scala/)
[info] Updating {file:/home/yang/pod/sales/scala/}pod...
[info] Resolving org.scala-lang#scala-library;2.9.1 ...
[info] Resolving com.google.protobuf#protobuf-java;2.4.1 ...
[info] Resolving org.scalaquery#scalaquery_2.9.0;0.9.4 ...
[info] Resolving postgresql#postgresql;9.0-801.jdbc4 ...
[info] Resolving com.jolbox#bonecp;0.7.1.RELEASE ...
[info] Resolving com.google.guava#guava;r08 ...
[info] Resolving org.slf4j#slf4j-api;1.5.10 ...
[info] Resolving ru.circumflex#circumflex-orm;2.1-SNAPSHOT ...
[info] Resolving ru.circumflex#circumflex-core;2.1 ...
[info] Resolving commons-io#commons-io;1.4 ...
[info] Resolving org.scala-lang#scala-compiler;2.9.1 ...
[info] Resolving org.slf4j#slf4j-api;1.6.1 ...
[info] Resolving c3p0#c3p0;0.9.1.2 ...
[info] Resolving net.sf.ehcache#ehcache-core;2.4.3 ...
[info] Resolving javax.transaction#jta;1.1 ...
[info] Resolving org.scalatra#scalatra_2.9.1;2.1.0-SNAPSHOT ...
[info] Resolving org.scalatra#scalatra-scalate_2.9.1;2.1.0-SNAPSHOT ...
[info] Resolving org.fusesource.scalate#scalate-core;1.5.3 ...
[info] Resolving org.fusesource.scalate#scalate-util;1.5.3 ...
[info] Resolving org.scalatra#scalatra-fileupload_2.9.1;2.1.0-SNAPSHOT ...
[info] Resolving commons-fileupload#commons-fileupload;1.2.1 ...
[info] Resolving commons-io#commons-io;2.1 ...
[info] Resolving org.fusesource.scalate#scalate-jruby;1.5.0 ...
[info] Resolving org.jruby#jruby-complete;1.5.6 ...
[info] Resolving org.fusesource.scalamd#scalamd;1.5 ...
[info] Resolving org.mortbay.jetty#jetty;6.1.22 ...
[info] Resolving org.mortbay.jetty#jetty-util;6.1.22 ...
[info] Resolving org.mortbay.jetty#servlet-api;2.5-20081211 ...
[info] Resolving net.debasishg#sjson_2.9.0;0.12 ...
[info] Resolving org.objenesis#objenesis;1.2 ...
[info] Resolving net.databinder#dispatch-json_2.9.0;0.8.1 ...
[info] Resolving com.lambdaworks#scrypt;1.2.0 ...
[info] Resolving net.sf.opencsv#opencsv;2.1 ...
[info] Resolving org.apache.commons#commons-math;2.2 ...
[info] Resolving org.apache.commons#commons-lang3;3.0 ...
[info] Resolving ch.qos.logback#logback-classic;0.9.29 ...
[info] Resolving ch.qos.logback#logback-core;0.9.29 ...
[info] Resolving org.scalatest#scalatest_2.9.0;1.6.1 ...
[info] Resolving com.h2database#h2;1.3.158 ...
[info] Resolving pentaho.weka#pdm-3.7-ce;SNAPSHOT ...
[info] Resolving net.java.dev.jna#jna;3.3.0 ...
[info] Resolving org.scalala#scalala_2.9.0;1.0.0.RC2-SNAPSHOT ...
[info] Resolving jline#jline;0.9.94 ...
[info] Resolving org.apache.xmlgraphics#xmlgraphics-commons;1.3.1 ...
[info] Resolving commons-logging#commons-logging;1.0.4 ...
[info] Resolving jfree#jfreechart;1.0.13 ...
[info] Resolving jfree#jcommon;1.0.16 ...
[info] Resolving netlib#netlib-java;0.9.3 ...
[info] Resolving netlib#arpack-combo;0.1 ...
[info] Resolving com.lowagie#itext;2.1.5 ...
[info] Resolving bouncycastle#bcmail-jdk14;138 ...
[info] Resolving bouncycastle#bcprov-jdk14;138 ...
[info] Resolving rhino#js;1.7R2 ...
[info] Resolving junit#junit;4.9 ...
[info] Resolving org.hamcrest#hamcrest-core;1.1 ...
[info] Resolving org.apache.commons#commons-email;1.2 ...
[info] Resolving javax.mail#mail;1.4.1 ...
[info] Resolving javax.activation#activation;1.1 ...
[info] Resolving commons-validator#commons-validator;1.3.1 ...
[info] Resolving commons-beanutils#commons-beanutils;1.7.0 ...
[info] Resolving commons-digester#commons-digester;1.6 ...
[info] Resolving commons-collections#commons-collections;2.1 ...
[info] Resolving xml-apis#xml-apis;1.0.b2 ...
[info] Resolving oro#oro;2.0.8 ...
[info] Resolving org.scala-tools.time#time_2.9.1;0.5 ...
[info] Resolving joda-time#joda-time;1.6.2 ...
[info] Resolving com.carrotsearch#hppc;0.4.1 ...
[info] Resolving com.twitter#util_2.9.1;1.12.12 ...
[info] Resolving com.twitter#util-codec_2.9.1;1.12.12 ...
[info] Resolving commons-codec#commons-codec;1.5 ...
[info] Resolving org.apache.commons#commons-parent;20 ...
[info] Resolving org.apache#apache;9 ...
[info] Resolving com.twitter#util-core_2.9.1;1.12.12 ...
[info] Resolving com.twitter#util-zk_2.9.1;1.12.12 ...
[info] Resolving org.apache.zookeeper#zookeeper;3.3.4 ...
[info] Resolving log4j#log4j;1.2.15 ...
[info] Resolving javax.jms#jms;1.1 ...
[info] Resolving com.sun.jdmk#jmxtools;1.2.1 ...
[info] Resolving com.sun.jmx#jmxri;1.2.1 ...
[info] Resolving com.twitter#util-collection_2.9.1;1.12.12 ...
[info] Resolving commons-collections#commons-collections;3.2.1 ...
[info] Resolving org.apache.commons#commons-parent;9 ...
[info] Resolving org.apache#apache;4 ...
[info] Resolving com.google.guava#guava;r09 ...
[info] Resolving com.google#google;5 ...
[info] Resolving com.twitter#util-logging_2.9.1;1.12.12 ...
[info] Resolving com.twitter#util-reflect_2.9.1;1.12.12 ...
[info] Resolving asm#asm;3.3.1 ...
[info] Resolving asm#asm-parent;3.3.1 ...
[info] Resolving cglib#cglib;2.2 ...
[info] Resolving asm#asm-commons;3.3.1 ...
[info] Resolving asm#asm-parent;3.3.1 ...
[info] Resolving asm#asm-tree;3.3.1 ...
[info] Resolving asm#asm-parent;3.3.1 ...
[info] Resolving asm#asm-util;3.3.1 ...
[info] Resolving asm#asm-parent;3.3.1 ...
[info] Resolving com.twitter#util-zk-common_2.9.1;1.12.12 ...
[info] Resolving com.twitter.common#zookeeper;0.0.31 ...
[info] Resolving com.google.guava#guava;10.0.1 ...
[info] Resolving com.google.guava#guava-parent;10.0.1 ...
[info] Resolving org.sonatype.oss#oss-parent;7 ...
[info] Resolving com.google.code.findbugs#jsr305;1.3.9 ...
[info] Resolving javax.inject#javax.inject;1 ...
[info] Resolving com.twitter.common#args;0.1.19 ...
[info] Resolving com.twitter.common#args-apt;0.0.8 ...
[info] Resolving commons-lang#commons-lang;2.4 ...
[info] Resolving org.apache.commons#commons-parent;9 ...
[info] Resolving com.twitter.common#base;0.0.28 ...
[info] Resolving com.twitter.common#quantity;0.0.20 ...
[info] Resolving com.twitter.common#collections;0.0.23 ...
[info] Resolving com.twitter.common#util-system-mocks;0.0.15 ...
[info] Resolving com.twitter.common#net-util;0.0.19 ...
[info] Resolving com.twitter.common#io-json;0.0.3 ...
[info] Resolving com.google.code.gson#gson;1.6 ...
[info] Resolving com.twitter.common#io;0.0.13 ...
[info] Resolving com.twitter.common#io-thrift;0.0.11 ...
[info] Resolving org.apache.thrift#libthrift;0.5.0 ...
[info] Resolving commons-lang#commons-lang;2.5 ...
[info] Resolving org.apache.commons#commons-parent;12 ...
[info] Resolving org.apache#apache;4 ...
[info] Resolving org.slf4j#slf4j-jdk14;1.6.1 ...
[info] Resolving org.slf4j#slf4j-parent;1.6.1 ...
[info] Resolving com.twitter.common#dynamic-host-set;0.0.10 ...
[info] Resolving com.twitter.common#net-pool;0.0.15 ...
[info] Resolving com.twitter.common#stats;0.0.28 ...
[info] Resolving com.twitter.common#stats-provider;0.0.11 ...
[info] Resolving com.twitter.common#application-action;0.0.13 ...
[info] Resolving com.google.inject#guice;3.0 ...
[info] Resolving com.google.inject#guice-parent;3.0 ...
[info] Resolving com.google#google;5 ...
[info] Resolving aopalliance#aopalliance;1.0 ...
[info] Resolving org.sonatype.sisu.inject#cglib;2.2.1-v20090111 ...
[info] Resolving org.sonatype.forge#forge-parent;6 ...
[info] Resolving com.twitter.common#util-sampler;0.0.12 ...
[info] Resolving com.twitter.common#util;0.0.28 ...
[info] Resolving com.twitter.common#util-executor-service-shutdown;0.0.3 ...
[info] Resolving com.twitter.common#jdk-logging;0.0.8 ...
[info] Resolving com.twitter.common#service-thrift;0.0.13 ...
[info] Resolving com.twitter#util-hashing_2.9.1;1.12.12 ...
[info] Resolving com.twitter#util-thrift_2.9.1;1.12.12 ...
[info] Resolving org.codehaus.jackson#jackson-mapper-asl;1.8.1 ...
[info] Resolving org.codehaus.jackson#jackson-core-asl;1.8.1 ...
[info] Resolving thrift#libthrift;0.5.0 ...
[warn]  module not found: thrift#libthrift;0.5.0
[warn] ==== local: tried
[warn]   /home/yang/.ivy2/local/thrift/libthrift/0.5.0/ivys/ivy.xml
[warn] ==== POD: tried
[warn]   https://dev.partyondata.com/deps/thrift/libthrift/0.5.0/libthrift-0.5.0.pom
[warn] ==== Twitter: tried
[warn]   http://maven.twttr.com/thrift/libthrift/0.5.0/libthrift-0.5.0.pom
[warn] ==== Scala-Tools Snapshots: tried
[warn]   http://scala-tools.org/repo-snapshots/thrift/libthrift/0.5.0/libthrift-0.5.0.pom
[warn] ==== Sonatype OSS Snapshots: tried
[warn]   http://oss.sonatype.org/content/repositories/snapshots/thrift/libthrift/0.5.0/libthrift-0.5.0.pom
[warn] ==== Sonatype OSS releases: tried
[warn]   http://oss.sonatype.org/content/repositories/releases/thrift/libthrift/0.5.0/libthrift-0.5.0.pom
[warn] ==== ScalaNLP: tried
[warn]   http://repo.scalanlp.org/repo/thrift/libthrift/0.5.0/libthrift-0.5.0.pom
[warn] ==== Pentaho: tried
[warn]   http://repo.pentaho.org/artifactory/pentaho/thrift/libthrift/0.5.0/libthrift-0.5.0.pom
[warn] ==== FuseSource snapshots: tried
[warn]   http://repo.fusesource.com/nexus/content/repositories/snapshots/thrift/libthrift/0.5.0/libthrift-0.5.0.pom
[warn] ==== JBoss: tried
[warn]   https://repository.jboss.org/nexus/content/repositories/thirdparty-releases/thrift/libthrift/0.5.0/libthrift-0.5.0.pom
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/thrift/libthrift/0.5.0/libthrift-0.5.0.pom
[warn] ==== Scala-Tools Maven2 Repository: tried
[warn]   http://scala-tools.org/repo-releases/thrift/libthrift/0.5.0/libthrift-0.5.0.pom
[info] Resolving com.twitter#util-eval_2.9.1;1.12.12 ...
[info] Resolving org.clapper#grizzled-scala_2.9.1;1.0.9 ...
[info] Resolving com.googlecode.linkedin-j#linkedin-j-core;1.0.416 ...
[info] Resolving com.googlecode.linkedin-j#linkedin-j;1.0.416 ...
[info] Resolving org.sonatype.oss#oss-parent;7 ...
[info] Resolving oauth.signpost#signpost-core;1.2.1.1 ...
[info] Resolving oauth.signpost#oauth-signpost;1.2.1.1 ...
[info] Resolving net.sf.kxml#kxml2;2.3.0 ...
[info] Resolving commons-cli#commons-cli;1.2 ...
[info] Resolving org.apache.commons#commons-parent;11 ...
[info] Resolving javax.servlet#servlet-api;2.5 ...
[info] downloading http://oss.sonatype.org/content/repositories/snapshots/org/scalatra/scalatra_2.9.1/2.1.0-SNAPSHOT/scalatra_2.9.1-2.1.0-SNAPSHOT.jar ...
[info]  [SUCCESSFUL ] org.scalatra#scalatra_2.9.1;2.1.0-SNAPSHOT!scalatra_2.9.1.jar (3327ms)
[info] downloading http://oss.sonatype.org/content/repositories/snapshots/org/scalatra/scalatra-scalate_2.9.1/2.1.0-SNAPSHOT/scalatra-scalate_2.9.1-2.1.0-SNAPSHOT.jar ...
[info]  [SUCCESSFUL ] org.scalatra#scalatra-scalate_2.9.1;2.1.0-SNAPSHOT!scalatra-scalate_2.9.1.jar (680ms)
[info] downloading http://oss.sonatype.org/content/repositories/snapshots/org/scalatra/scalatra-fileupload_2.9.1/2.1.0-SNAPSHOT/scalatra-fileupload_2.9.1-2.1.0-SNAPSHOT.jar ...
[info]  [SUCCESSFUL ] org.scalatra#scalatra-fileupload_2.9.1;2.1.0-SNAPSHOT!scalatra-fileupload_2.9.1.jar (2687ms)
[info] downloading http://scala-tools.org/repo-releases/org/clapper/grizzled-scala_2.9.1/1.0.9/grizzled-scala_2.9.1-1.0.9.jar ...
[info]  [SUCCESSFUL ] org.clapper#grizzled-scala_2.9.1;1.0.9!grizzled-scala_2.9.1.jar (2907ms)
[info] downloading http://oss.sonatype.org/content/repositories/releases/com/googlecode/linkedin-j/linkedin-j-core/1.0.416/linkedin-j-core-1.0.416.jar ...
[info]  [SUCCESSFUL ] com.googlecode.linkedin-j#linkedin-j-core;1.0.416!linkedin-j-core.jar (1381ms)
[info] downloading http://maven.twttr.com/com/twitter/util-codec_2.9.1/1.12.12/util-codec_2.9.1-1.12.12.jar ...
[info]  [SUCCESSFUL ] com.twitter#util-codec_2.9.1;1.12.12!util-codec_2.9.1.jar (2045ms)
[info] downloading http://maven.twttr.com/com/twitter/util-zk_2.9.1/1.12.12/util-zk_2.9.1-1.12.12.jar ...
[info]  [SUCCESSFUL ] com.twitter#util-zk_2.9.1;1.12.12!util-zk_2.9.1.jar (140ms)
[info] downloading http://maven.twttr.com/com/twitter/util-logging_2.9.1/1.12.12/util-logging_2.9.1-1.12.12.jar ...
[info]  [SUCCESSFUL ] com.twitter#util-logging_2.9.1;1.12.12!util-logging_2.9.1.jar (148ms)
[info] downloading http://maven.twttr.com/com/twitter/util-reflect_2.9.1/1.12.12/util-reflect_2.9.1-1.12.12.jar ...
[info]  [SUCCESSFUL ] com.twitter#util-reflect_2.9.1;1.12.12!util-reflect_2.9.1.jar (96ms)
[info] downloading http://maven.twttr.com/com/twitter/util-zk-common_2.9.1/1.12.12/util-zk-common_2.9.1-1.12.12.jar ...
[info]  [SUCCESSFUL ] com.twitter#util-zk-common_2.9.1;1.12.12!util-zk-common_2.9.1.jar (121ms)
[info] downloading http://maven.twttr.com/com/twitter/util-core_2.9.1/1.12.12/util-core_2.9.1-1.12.12.jar ...
[info]  [SUCCESSFUL ] com.twitter#util-core_2.9.1;1.12.12!util-core_2.9.1.jar (270ms)
[info] downloading http://maven.twttr.com/com/twitter/util-hashing_2.9.1/1.12.12/util-hashing_2.9.1-1.12.12.jar ...
[info]  [SUCCESSFUL ] com.twitter#util-hashing_2.9.1;1.12.12!util-hashing_2.9.1.jar (121ms)
[info] downloading http://maven.twttr.com/com/twitter/util-thrift_2.9.1/1.12.12/util-thrift_2.9.1-1.12.12.jar ...
[info]  [SUCCESSFUL ] com.twitter#util-thrift_2.9.1;1.12.12!util-thrift_2.9.1.jar (84ms)
[info] downloading http://maven.twttr.com/com/twitter/util-eval_2.9.1/1.12.12/util-eval_2.9.1-1.12.12.jar ...
[info]  [SUCCESSFUL ] com.twitter#util-eval_2.9.1;1.12.12!util-eval_2.9.1.jar (130ms)
[info] downloading http://maven.twttr.com/com/twitter/util-collection_2.9.1/1.12.12/util-collection_2.9.1-1.12.12.jar ...
[info]  [SUCCESSFUL ] com.twitter#util-collection_2.9.1;1.12.12!util-collection_2.9.1.jar (133ms)
[info] downloading http://repo1.maven.org/maven2/commons-codec/commons-codec/1.5/commons-codec-1.5.jar ...
[info]  [SUCCESSFUL ] commons-codec#commons-codec;1.5!commons-codec.jar (1036ms)
[info] downloading http://repo1.maven.org/maven2/org/apache/zookeeper/zookeeper/3.3.4/zookeeper-3.3.4.jar ...
[info]  [SUCCESSFUL ] org.apache.zookeeper#zookeeper;3.3.4!zookeeper.jar (1233ms)
[info] downloading http://repo1.maven.org/maven2/log4j/log4j/1.2.15/log4j-1.2.15.jar ...
[info]  [SUCCESSFUL ] log4j#log4j;1.2.15!log4j.jar (998ms)
[info] downloading https://repository.jboss.org/nexus/content/repositories/thirdparty-releases/javax/jms/jms/1.1/jms-1.1.jar ...
[info]  [SUCCESSFUL ] javax.jms#jms;1.1!jms.jar (264ms)
[warn]  [NOT FOUND  ] com.sun.jdmk#jmxtools;1.2.1!jmxtools.jar (0ms)
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/com/sun/jdmk/jmxtools/1.2.1/jmxtools-1.2.1.jar
[warn]  [NOT FOUND  ] com.sun.jmx#jmxri;1.2.1!jmxri.jar (0ms)
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/com/sun/jmx/jmxri/1.2.1/jmxri-1.2.1.jar
[info] downloading http://repo1.maven.org/maven2/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar ...
[info]  [SUCCESSFUL ] commons-collections#commons-collections;3.2.1!commons-collections.jar (815ms)
[info] downloading http://repo1.maven.org/maven2/asm/asm/3.3.1/asm-3.3.1.jar ...
[info]  [SUCCESSFUL ] asm#asm;3.3.1!asm.jar (994ms)
[info] downloading http://repo1.maven.org/maven2/cglib/cglib/2.2/cglib-2.2.jar ...
[info]  [SUCCESSFUL ] cglib#cglib;2.2!cglib.jar (1014ms)
[info] downloading http://repo1.maven.org/maven2/asm/asm-commons/3.3.1/asm-commons-3.3.1.jar ...
[info]  [SUCCESSFUL ] asm#asm-commons;3.3.1!asm-commons.jar (984ms)
[info] downloading http://repo1.maven.org/maven2/asm/asm-util/3.3.1/asm-util-3.3.1.jar ...
[info]  [SUCCESSFUL ] asm#asm-util;3.3.1!asm-util.jar (1002ms)
[info] downloading http://repo1.maven.org/maven2/asm/asm-tree/3.3.1/asm-tree-3.3.1.jar ...
[info]  [SUCCESSFUL ] asm#asm-tree;3.3.1!asm-tree.jar (998ms)
[info] downloading http://maven.twttr.com/com/twitter/common/zookeeper/0.0.31/zookeeper-0.0.31.jar ...
[info]  [SUCCESSFUL ] com.twitter.common#zookeeper;0.0.31!zookeeper.jar (215ms)
[info] downloading http://oss.sonatype.org/content/repositories/releases/com/google/guava/guava/10.0.1/guava-10.0.1.jar ...
[info]  [SUCCESSFUL ] com.google.guava#guava;10.0.1!guava.jar (1142ms)
[info] downloading http://repo1.maven.org/maven2/javax/inject/javax.inject/1/javax.inject-1.jar ...
[info]  [SUCCESSFUL ] javax.inject#javax.inject;1!javax.inject.jar (936ms)
[info] downloading http://repo1.maven.org/maven2/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar ...
[info]  [SUCCESSFUL ] com.google.code.findbugs#jsr305;1.3.9!jsr305.jar (1149ms)
[info] downloading http://maven.twttr.com/com/twitter/common/args/0.1.19/args-0.1.19.jar ...
[info]  [SUCCESSFUL ] com.twitter.common#args;0.1.19!args.jar (146ms)
[info] downloading http://maven.twttr.com/com/twitter/common/base/0.0.28/base-0.0.28.jar ...
[info]  [SUCCESSFUL ] com.twitter.common#base;0.0.28!base.jar (151ms)
[info] downloading http://maven.twttr.com/com/twitter/common/io-json/0.0.3/io-json-0.0.3.jar ...
[info]  [SUCCESSFUL ] com.twitter.common#io-json;0.0.3!io-json.jar (132ms)
[info] downloading http://maven.twttr.com/com/twitter/common/io-thrift/0.0.11/io-thrift-0.0.11.jar ...
[info]  [SUCCESSFUL ] com.twitter.common#io-thrift;0.0.11!io-thrift.jar (145ms)
[info] downloading http://maven.twttr.com/com/twitter/common/net-util/0.0.19/net-util-0.0.19.jar ...
[info]  [SUCCESSFUL ] com.twitter.common#net-util;0.0.19!net-util.jar (129ms)
[info] downloading http://maven.twttr.com/com/twitter/common/dynamic-host-set/0.0.10/dynamic-host-set-0.0.10.jar ...
[info]  [SUCCESSFUL ] com.twitter.common#dynamic-host-set;0.0.10!dynamic-host-set.jar (110ms)
[info] downloading http://maven.twttr.com/com/twitter/common/net-pool/0.0.15/net-pool-0.0.15.jar ...
[info]  [SUCCESSFUL ] com.twitter.common#net-pool;0.0.15!net-pool.jar (125ms)
[info] downloading http://maven.twttr.com/com/twitter/common/quantity/0.0.20/quantity-0.0.20.jar ...
[info]  [SUCCESSFUL ] com.twitter.common#quantity;0.0.20!quantity.jar (150ms)
[info] downloading http://maven.twttr.com/com/twitter/common/util/0.0.28/util-0.0.28.jar ...
[info]  [SUCCESSFUL ] com.twitter.common#util;0.0.28!util.jar (134ms)
[info] downloading http://maven.twttr.com/com/twitter/common/service-thrift/0.0.13/service-thrift-0.0.13.jar ...
[info]  [SUCCESSFUL ] com.twitter.common#service-thrift;0.0.13!service-thrift.jar (161ms)
[info] downloading http://maven.twttr.com/com/twitter/common/args-apt/0.0.8/args-apt-0.0.8.jar ...
[info]  [SUCCESSFUL ] com.twitter.common#args-apt;0.0.8!args-apt.jar (123ms)
[info] downloading http://maven.twttr.com/com/twitter/common/util-system-mocks/0.0.15/util-system-mocks-0.0.15.jar ...
[info]  [SUCCESSFUL ] com.twitter.common#util-system-mocks;0.0.15!util-system-mocks.jar (98ms)
[info] downloading http://maven.twttr.com/com/twitter/common/collections/0.0.23/collections-0.0.23.jar ...
[info]  [SUCCESSFUL ] com.twitter.common#collections;0.0.23!collections.jar (139ms)
[info] downloading http://oss.sonatype.org/content/repositories/releases/com/google/code/gson/gson/1.6/gson-1.6.jar ...
[info]  [SUCCESSFUL ] com.google.code.gson#gson;1.6!gson.jar (694ms)
[info] downloading http://maven.twttr.com/com/twitter/common/io/0.0.13/io-0.0.13.jar ...
[info]  [SUCCESSFUL ] com.twitter.common#io;0.0.13!io.jar (95ms)
[info] downloading http://maven.twttr.com/org/apache/thrift/libthrift/0.5.0/libthrift-0.5.0.jar ...
[info]  [SUCCESSFUL ] org.apache.thrift#libthrift;0.5.0!libthrift.jar (269ms)
[info] downloading http://repo1.maven.org/maven2/org/slf4j/slf4j-jdk14/1.6.1/slf4j-jdk14-1.6.1.jar ...
[info]  [SUCCESSFUL ] org.slf4j#slf4j-jdk14;1.6.1!slf4j-jdk14.jar (808ms)
[info] downloading http://repo1.maven.org/maven2/commons-lang/commons-lang/2.5/commons-lang-2.5.jar ...
[info]  [SUCCESSFUL ] commons-lang#commons-lang;2.5!commons-lang.jar (2142ms)
[info] downloading http://maven.twttr.com/com/twitter/common/stats/0.0.28/stats-0.0.28.jar ...
[info]  [SUCCESSFUL ] com.twitter.common#stats;0.0.28!stats.jar (155ms)
[info] downloading http://maven.twttr.com/com/twitter/common/stats-provider/0.0.11/stats-provider-0.0.11.jar ...
[info]  [SUCCESSFUL ] com.twitter.common#stats-provider;0.0.11!stats-provider.jar (97ms)
[info] downloading http://maven.twttr.com/com/twitter/common/application-action/0.0.13/application-action-0.0.13.jar ...
[info]  [SUCCESSFUL ] com.twitter.common#application-action;0.0.13!application-action.jar (145ms)
[info] downloading http://maven.twttr.com/com/twitter/common/util-sampler/0.0.12/util-sampler-0.0.12.jar ...
[info]  [SUCCESSFUL ] com.twitter.common#util-sampler;0.0.12!util-sampler.jar (98ms)
[info] downloading http://oss.sonatype.org/content/repositories/releases/com/google/inject/guice/3.0/guice-3.0.jar ...
[info]  [SUCCESSFUL ] com.google.inject#guice;3.0!guice.jar (915ms)
[info] downloading http://repo1.maven.org/maven2/aopalliance/aopalliance/1.0/aopalliance-1.0.jar ...
[info]  [SUCCESSFUL ] aopalliance#aopalliance;1.0!aopalliance.jar (809ms)
[info] downloading http://repo1.maven.org/maven2/org/sonatype/sisu/inject/cglib/2.2.1-v20090111/cglib-2.2.1-v20090111.jar ...
[info]  [SUCCESSFUL ] org.sonatype.sisu.inject#cglib;2.2.1-v20090111!cglib.jar (1666ms)
[info] downloading http://maven.twttr.com/com/twitter/common/util-executor-service-shutdown/0.0.3/util-executor-service-shutdown-0.0.3.jar ...
[info]  [SUCCESSFUL ] com.twitter.common#util-executor-service-shutdown;0.0.3!util-executor-service-shutdown.jar (144ms)
[info] downloading http://maven.twttr.com/com/twitter/common/jdk-logging/0.0.8/jdk-logging-0.0.8.jar ...
[info]  [SUCCESSFUL ] com.twitter.common#jdk-logging;0.0.8!jdk-logging.jar (119ms)
[info] downloading http://repo1.maven.org/maven2/org/codehaus/jackson/jackson-mapper-asl/1.8.1/jackson-mapper-asl-1.8.1.jar ...
[info]  [SUCCESSFUL ] org.codehaus.jackson#jackson-mapper-asl;1.8.1!jackson-mapper-asl.jar (1157ms)
[info] downloading http://repo1.maven.org/maven2/org/codehaus/jackson/jackson-core-asl/1.8.1/jackson-core-asl-1.8.1.jar ...
[info]  [SUCCESSFUL ] org.codehaus.jackson#jackson-core-asl;1.8.1!jackson-core-asl.jar (993ms)
[info] downloading http://repo1.maven.org/maven2/oauth/signpost/signpost-core/1.2.1.1/signpost-core-1.2.1.1.jar ...
[info]  [SUCCESSFUL ] oauth.signpost#signpost-core;1.2.1.1!signpost-core.jar (1003ms)
[info] downloading http://repo1.maven.org/maven2/net/sf/kxml/kxml2/2.3.0/kxml2-2.3.0.jar ...
[info]  [SUCCESSFUL ] net.sf.kxml#kxml2;2.3.0!kxml2.jar (994ms)
[info] downloading http://repo1.maven.org/maven2/commons-cli/commons-cli/1.2/commons-cli-1.2.jar ...
[info]  [SUCCESSFUL ] commons-cli#commons-cli;1.2!commons-cli.jar (998ms)
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: thrift#libthrift;0.5.0: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::              FAILED DOWNLOADS            ::
[warn]  :: ^ see resolution messages for details  ^ ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: com.sun.jdmk#jmxtools;1.2.1!jmxtools.jar
[warn]  :: com.sun.jmx#jmxri;1.2.1!jmxri.jar
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[error] {file:/home/yang/pod/sales/scala/}pod/*:update: sbt.ResolveException: unresolved dependency: thrift#libthrift;0.5.0: not found
[error] download failed: com.sun.jdmk#jmxtools;1.2.1!jmxtools.jar
[error] download failed: com.sun.jmx#jmxri;1.2.1!jmxri.jar
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore?

FutureTransformer (and other Java-friendly APIs?) needs @throws annotations

Scaladoc says it's allowed to throw exceptions from FutureTransformer.map/flatMap/handle/rescue, but since they aren't annotated with @throws checked exceptions can't be thrown from implementations. In this particular case there is a workaround with calling super.handle(throwable) instead of throw throwable, but it's a bit ugly.

java.lang.NoClassDefFoundError: scala/collection/JavaConversions$JMapWrapper

I have a project which is using the LruMap:

// LRU
import com.twitter.util.LruMap

The library is the latest version:

val collUtils   = "com.twitter"          %  "util-collection"     % "6.3.4"

Everything was fine until I upgraded my project to Scala 2.10 - here's the weirdness at the console:

scala> import scala.collection.JavaConversions.JMapWrapper
import scala.collection.JavaConversions.JMapWrapper
scala> import com.snowplowanalytics.maxmind.geoip.IpGeo
import com.snowplowanalytics.maxmind.geoip.IpGeo
scala> val ipGeo = IpGeo(dbFile = "/opt/maxmind/GeoLiteCity.dat", memCache = false, lruCache = 20000)
java.lang.NoClassDefFoundError: scala/collection/JavaConversions$JMapWrapper

IpGeo makes use of LruMap, which in turn uses scala.collection.JavaConversions.JMapWrapper.

I've done some digging and it looks like JMapWrapper underwent a big change between 2.9.3 and 2.10: it changed from a case class to a deprecated type alias:

https://github.com/scala/scala/blob/v2.9.3/src/library/scala/collection/JavaConversions.scala#L792

=>

https://github.com/scala/scala/blob/v2.10.0/src/library/scala/collection/JavaConversions.scala#L65

Could this be causing the problem?

Thread-local LocalScheduler$Activation objects are never garbage-collected

LocalScheduler has thread-local Activation objects.

class LocalScheduler(lifo: Boolean) extends Scheduler {
  ...
  @volatile private[this] var activations = Set[Activation]()
  private[this] val local = new ThreadLocal[Activation] {
    override def initialValue = null
  }

And when a thread-local Activation is created, it is added to activations var.

private class Activation extends Scheduler with Iterator[Runnable] {
  ...
  private[this] def get(): Activation = {
    ...
    local.set(new Activation)
    synchronized { activations += local.get() }
    local.get()
  }

But there is no code to remove Activation objects from activations. So these are never garbage-collected, because they are referenced by activations even after their owner threads are terminated.

This is not a problem if using only fixed-size threads, but if using a dynamic thread pool, it can lead to a memory leak.

ExecutorServiceFuturePool fatal exceptions silently leave dangling Futures

When a FuturePool task throws a fatal exception (like an OOME), the task exits silently and the future never resolves. I think this is because apply tries to update its promise with Try(f), but Try(f) doesn't catch fatal exceptions in f, and so they bubble up and kill the runnable in the underlying executor service. Nothing is logged and the promise is never updated.

Code showing the behavior:

val pool = FuturePool.unboundedPool
val future = pool {
  throw new OutOfMemoryError()
}
Await.result(future) // blocks forever, silently

`sbt update` has to time out at https://m2proxy.atlassian.com/repository/public/

When running sbt update on a clone, I get multiple errors like

[info] == util-reflect_2.9.1 / update ==
[error] Server access Error: Connection timed out url=https://m2proxy.atlassian.com/repository/public/asm/asm-parent/3.3.1/asm-parent-3.3.1.jar
[error] Server access Error: Connection timed out url=https://m2proxy.atlassian.com/repository/public/asm/asm-commons/3.3.1/asm-commons-3.3.1-sources.jar
[error] Server access Error: Connection timed out url=https://m2proxy.atlassian.com/repository/public/asm/asm-commons/3.3.1/asm-commons-3.3.1-src.jar
[error] Server access Error: Connection timed out url=https://m2proxy.atlassian.com/repository/public/asm/asm-commons/3.3.1/asm-commons-3.3.1-javadoc.jar
[error] Server access Error: Connection timed out url=https://m2proxy.atlassian.com/repository/public/asm/asm-tree/3.3.1/asm-tree-3.3.1-sources.jar
[error] Server access Error: Connection timed out url=https://m2proxy.atlassian.com/repository/public/asm/asm-tree/3.3.1/asm-tree-3.3.1-src.jar
[error] Server access Error: Connection timed out url=https://m2proxy.atlassian.com/repository/public/asm/asm-tree/3.3.1/asm-tree-3.3.1-javadoc.jar
[error] Server access Error: Connection timed out url=https://m2proxy.atlassian.com/repository/public/asm/asm/3.3.1/asm-3.3.1-sources.jar

Since the connection takes a long time to time out, update takes a very long time. It can be worked around by adding

override def repositories = super.repositories - ("atlassian" at "https://m2proxy.atlassian.com/repository/public/")

to Project and ProjectDefaults.

TrySpec

I guess the spec

...
"Try" should {
"rescue" in {
val myException = new MyException
Return(1) rescue { case _ => Return(2) } mustEqual Return(1)
Throw(e) rescue { case _ => Return(2) } mustEqual Return(2)
Throw(e) rescue { case _ => Throw(e) } mustEqual Throw(e) //THIS PART
}
...

THIS PART should be Throw(myException) rescue ...
Just FYI

Future deadlock

Hi,

I am working on upgrading storehaushaus-kafka to work with Kafka 0.8.+. One of the issue I am running into is that Kafka 0.8 has introduced some blocking operations and when these operation get called inside a Future/FuturePool they end up in intermittent deadlocks. In particular this snippet of code is deadlocking when wrapped inside a Future

def send(messages: KeyedMessage[K,V]*) {
    lock synchronized {
      if (hasShutdown.get)
        throw new ProducerClosedException
      recordStats(messages)
      sync match {
        case true => eventHandler.handle(messages)
        case false => asyncSend(messages)
      }
    }
  }

Is there anything that can be done to prevent getting into deadlock? or the only option is to ask Kafka team to not use synchronize block in their api?
Thanks

java.lang.UnsupportedOperationException: Position.line

In trying to eval a config file that has an error in it I've seen that Eval.scala throws an UnsupportedOperationException.

FATAL: Error in config file: ../config.scala
java.lang.UnsupportedOperationException: Position.line
        at scala.reflect.internal.util.Position.line(Position.scala:205)
        at com.twitter.util.Eval$StringCompiler$$anon$1.display(Eval.scala:444)
        at scala.tools.nsc.reporters.AbstractReporter.info0(AbstractReporter.scala:49)
        at scala.tools.nsc.reporters.Reporter$$anonfun$error$1.apply$mcV$sp(Reporter.scala:70)
        at scala.tools.nsc.reporters.Reporter$$anonfun$error$1.apply(Reporter.scala:70)
        at scala.tools.nsc.reporters.Reporter$$anonfun$error$1.apply(Reporter.scala:70)
        at scala.tools.nsc.reporters.Reporter.withoutTruncating(Reporter.scala:42)
        at scala.tools.nsc.reporters.Reporter.error(Reporter.scala:70)
        at scala.tools.nsc.Global.globalError(Global.scala:228)
        at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1556)
        at scala.tools.nsc.Global$Run.compileSources(Global.scala:1544)
        at com.twitter.util.Eval$StringCompiler.apply(Eval.scala:530)
        at com.twitter.util.Eval$StringCompiler$$anonfun$apply$2.apply(Eval.scala:544)
        at com.twitter.util.Eval$StringCompiler$$anonfun$apply$2.apply(Eval.scala:543)
        at scala.Option.getOrElse(Option.scala:120)
        at com.twitter.util.Eval$StringCompiler.apply(Eval.scala:543)
        at com.twitter.util.Eval.applyProcessed(Eval.scala:196)
        at com.twitter.util.Eval.apply(Eval.scala:167)
        at com.twitter.ostrich.admin.RuntimeEnvironment.loadConfig(RuntimeEnvironment.scala:230)
        at com.twitter.ostrich.admin.RuntimeEnvironment.loadRuntimeConfig(RuntimeEnvironment.scala:256)
        at io.angstrom.hiveworker.Main$.main(Main.scala:11)
        at io.angstrom.hiveworker.Main.main(Main.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at scala_maven_executions.MainHelper.runMain(MainHelper.java:164)
        at scala_maven_executions.MainWithArgsInFile.main(MainWithArgsInFile.java:26)

As you can see this happens on line 444 in Eval.scala. On that line, Eval.scala is calling scala.reflect.internal.util.Position.line. Looking at the source for Position.line, it is explicitly not implemented:

def line: Int = throw new UnsupportedOperationException("Position.line")

Along with Position.column:

def column: Int = throw new UnsupportedOperationException("Position.column")

which is also used in Eval.scala on line 447.

This is preventing me from figuring out the real reason the config file evaluation is failing as the UnsuportedOperationException happens in the building of the error message to return. In order to move forward, I rebuilt util-eval replacing the call to

messages += (severityName + "line " + (pos.line - lineOffset) + ": " + message) ::

with

messages += (severityName + "line " + (0 - lineOffset) + ": " + message) ::

which works in giving me the underlying error. So it seems to be an issue that Eval.scala is calling Position.line (and Position.column).

I am using version 6.1.0 of com.twitter:util-eval_2.10 (pulled in by com.twitter:finagle-ostrich4_2.10:6.2.1) but I've verified that the same thing happens in version 6.2.4 of com.twitter:util-eval_2.10.

[INFO] +- com.twitter:finagle-ostrich4_2.10:jar:6.2.1:compile
[INFO] |  \- com.twitter:ostrich_2.10:jar:9.1.0:compile
[INFO] |     +- com.twitter:util-eval_2.10:jar:6.1.0:compile
[INFO] |     |  \- org.scala-lang:scala-compiler:jar:2.10.0:compile
[INFO] |     \- com.twitter:scala-json_2.10:jar:3.0.1:compile

Provide a way to shutdown Closable.collectorThread

When running in a plugin environment, it is necessary to shutdown all threads a plugin has started.

Closable.collectorThread isn't exposed and thus cannot be shutdown via any lifecycle API.

A workaround is to throw a fatal error from a Closable that is registered using the 'Closable.closeOnCollect(Closable, Object)` method, but this has no ordering guarantees.

A more friendly approach would use an appropriate ExecutorService that only created a thread if one wasn't available, and died when not in use.

Eval dies when run under Tomcat 7 versioned deployment

I've prepared a trivial example to show the behavior: here

Build it: mvn package

Deploy w/o version

curl -T - -u tomcat:tomcat 'http://localhost:8080/manager/text/deploy?update=true&path=/' < target/ROOT.war

result:

Sep 6, 2011 5:21:24 PM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive ROOT.war
0 [Thread-13] INFO bootstrap.liftweb.Boot - Started

Deploy with version

curl -T - -u tomcat:tomcat 'http://localhost:8080/manager/text/deploy?update=true&path=/&version=01' < target/ROOT.war

result:

Sep 6, 2011 5:23:12 PM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive ROOT##01.war
1 [Thread-14] ERROR net.liftweb.http.provider.HTTPProvider - Failed to Boot! Your application may not run properly
scala.tools.nsc.MissingRequirementError: object scala not found.
        at scala.tools.nsc.symtab.Definitions$definitions$.getModuleOrClass(Definitions.scala:517)
        at scala.tools.nsc.symtab.Definitions$definitions$.ScalaPackage(Definitions.scala:37)
        at scala.tools.nsc.symtab.Definitions$definitions$.ScalaPackageClass(Definitions.scala:38)
        at scala.tools.nsc.symtab.Definitions$definitions$.UnitClass(Definitions.scala:83)
        at scala.tools.nsc.symtab.Definitions$definitions$.init(Definitions.scala:789)
        at scala.tools.nsc.Global$Run.(Global.scala:604)
        at com.twitter.util.Eval$StringCompiler.apply(Eval.scala:487)
        at com.twitter.util.Eval$StringCompiler$$anonfun$apply$3.apply(Eval.scala:504)
        at com.twitter.util.Eval$StringCompiler$$anonfun$apply$3.apply(Eval.scala:503)
        at scala.Option.getOrElse(Option.scala:104)
        at com.twitter.util.Eval$StringCompiler.apply(Eval.scala:503)
        at com.twitter.util.Eval.applyProcessed(Eval.scala:172)
        at com.twitter.util.Eval.applyProcessed(Eval.scala:165)

Best,
Alexander Azarov

util.Eval's class loader should be configurable

util.Eval has an issue in when used in environments with custom class loaders (such as in Play framework).

We have a set of libraries using util.Eval and we are using those libraries in a Play application. Basically, there are two class loaders in action at the same time.

While new Eval().apply[AnyRef](configFileContents) correctly loads a MyConfig object, that object can't be casted to MyConfig inside the Play app because it's been loaded using a different class loader than the one Play uses.

The problem is on lines 470 and 487 of Eval.scala.

If those lines could be changed to

classLoader = new AbstractFileClassLoader(target, baseClassLoader)

where baseClassLoader is an overridable method (or a constructor argument, whichever you prefer), that would be great!

StorageUnit needs to implement hashCode

StorageUnit implements equals, but does not bother to implement hashCode which is a bad idea.

scala> import com.twitter.conversions.storage._
import com.twitter.conversions.storage._

scala> val x1 = 4.megabytes
x1: com.twitter.util.StorageUnit = 4194304.bytes

scala> val x2 = 4.megabytes
x2: com.twitter.util.StorageUnit = 4194304.bytes

scala> x1.hashCode
res0: Int = 1573818776

scala> x2.hashCode
res1: Int = 770910585

Duration does it right, so we should do something like

override def hashCode = inBytes.hashCode

Rescue from TimeoutException

I'm trying to write a method to execute a blocking method (legacy HTTP GET, for that matter) asynchronously, with retries on timeout. I'd like to avoid catching the TimeoutException, so I'm trying to use Future.rescue instead.

Follows is a specs2 test attempting to do just that; for some reason which I can't really understand, the test fails. Am I missing something, or is there a bug somewhere?

class FutureRescueTimeout extends SpecificationWithJUnit {

  import com.twitter.util._
  import java.util.concurrent.Executors

  val pool = FuturePool(Executors.newSingleThreadExecutor)
  implicit val timer = new JavaTimer
  val timeout = new RichWholeNumber(100).millis // because of implicit conversion collision between specs and twitter time

  "rescue" should {
    "recover from timeout immediately" in {
      val f: Future[Boolean] = pool {
        Thread.sleep(500)
        true
      }
      .within(timeout)
      .rescue {
        case e: TimeoutException => Future(true)
      }

      f.get must beTrue
    }

    "recover from timeout using another future" in {
      val f = recursiveRetry(List(150, 30), timeout)
      f.get must beTrue
     }

  }

  def recursiveRetry(sleep: List[Long], timeout: Duration): Future[Boolean] = pool {
      Thread.sleep(sleep.head)
      true
    }
    .within(timeout)
    .rescue {
      case e: TimeoutException if !sleep.tail.isEmpty => recursiveRetry(sleep.tail, timeout)
    }
}

Event[T]() creates non-threadsafe Event instance

Event[T]() creates an instance that maintains a list of registered witnesses in @volatile var registerrs. The updates to this var are neither atomic nor synchronized. As a result adding/closing a witness is not guaranteed to execute successfully.

Here's a sample implementation that I currently use instead:

  def event[A](): Event[A] with Witness[A] = new Event[A] with Witness[A] {
    private type W = Witness[A]
    private[this] val witnesses = new AtomicReference(Vector.empty[W])

    @tailrec private def cas(f: Vector[W] => Vector[W]): Unit = {
      val old = witnesses.get
      if (!witnesses.compareAndSet(old, f(old))) cas(f)
    }

    def register(w: Witness[A]) = {
      cas(_ :+ w)
      Closable.make { _ => cas(_.filterNot(_ eq w)); Future.Unit }
    }

    def notify(note: A) = synchronized { witnesses.get.foreach(_.notify(note)) }
  }

Eval generates broke classpath when current classpath includes filename with spaces

To reproduce:

  • Create an SBT project in a directory that includes an escaped space (%20).
  • Evaluate Scala source code that includes a reference to class in your current classpath.
  • Observe failure:
com.twitter.util.Eval$CompilerException: Compiler exception error: line 2: object turing is not a member of package com
import com.turing.madea.MadeaConfig
           ^
    at com.twitter.util.Eval$StringCompiler.apply(Eval.scala:529)
    at com.twitter.util.Eval$StringCompiler$$anonfun$apply$3.apply(Eval.scala:540)
    at com.twitter.util.Eval$StringCompiler$$anonfun$apply$3.apply(Eval.scala:539)
    at scala.Option.getOrElse(Option.scala:108)
    at com.twitter.util.Eval$StringCompiler.apply(Eval.scala:539)
    at com.twitter.util.Eval.applyProcessed(Eval.scala:198)
    at com.twitter.util.Eval.applyProcessed(Eval.scala:191)
    at com.twitter.util.Eval.apply(Eval.scala:137)

Migration 6.9.0 from 6.8.1 [util-zk] fails to see dependency junit 4.8.1, mockito-all,

Here is our build.sbt : https://github.com/indykish/megam_common/blob/master/build.sbt
We use sbt 0.13.0, scala 2.10.3

Any thing else we should do different for using 6.9.0 ? It works well with 6.8.1.

It tries to download junit-sources.src.

[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[warn]  [FAILED     ] junit#junit;4.8.1!junit.src:  (0ms)
[warn] ==== local: tried
[warn]   /home/ram/.ivy2/local/junit/junit/4.8.1/srcs/junit-sources.src
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/junit/junit/4.8.1/junit-4.8.1-sources.src
[warn] ==== Twitter Repo: tried
[warn]   http://maven.twttr.com/junit/junit/4.8.1/junit-4.8.1-sources.src
[warn] ==== Typesafe Repo: tried
[warn]   http://repo.typesafe.com/typesafe/releases/junit/junit/4.8.1/junit-4.8.1-sources.src
[warn] ==== Sonatype Releases: tried
[warn]   https://oss.sonatype.org/content/public/junit/junit/4.8.1/junit-4.8.1-sources.src
[warn] ==== Typesafe Snapshots: tried
[warn]   http://repo.typesafe.com/typesafe/snapshots/junit/junit/4.8.1/junit-4.8.1-sources.src
[warn] ==== Sonatype OSS Snapshots: tried
[warn]   https://oss.sonatype.org/content/repositories/snapshots/junit/junit/4.8.1/junit-4.8.1-sources.src
[warn] ==== Scala-Tools Maven2 Snapshots Repository: tried
[warn]   http://scala-tools.org/repo-snapshots/junit/junit/4.8.1/junit-4.8.1-sources.src
[warn] ==== JBoss: tried
[warn]   https://repository.jboss.org/nexus/content/groups/public/junit/junit/4.8.1/junit-4.8.1-sources.src
[warn]  [FAILED     ] org.scala-tools.testing#specs_2.10;1.6.9!specs_2.10.src:  (0ms)
[warn] ==== local: tried
[warn]   /home/ram/.ivy2/local/org.scala-tools.testing/specs_2.10/1.6.9/srcs/specs_2.10-sources.src
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/org/scala-tools/testing/specs_2.10/1.6.9/specs_2.10-1.6.9-sources.src
[warn] ==== Twitter Repo: tried
[warn]   http://maven.twttr.com/org/scala-tools/testing/specs_2.10/1.6.9/specs_2.10-1.6.9-sources.src
[warn] ==== Typesafe Repo: tried
[warn]   http://repo.typesafe.com/typesafe/releases/org/scala-tools/testing/specs_2.10/1.6.9/specs_2.10-1.6.9-sources.src
[warn] ==== Sonatype Releases: tried
[warn]   https://oss.sonatype.org/content/public/org/scala-tools/testing/specs_2.10/1.6.9/specs_2.10-1.6.9-sources.src
[warn] ==== Typesafe Snapshots: tried
[warn]   http://repo.typesafe.com/typesafe/snapshots/org/scala-tools/testing/specs_2.10/1.6.9/specs_2.10-1.6.9-sources.src
[warn] ==== Sonatype OSS Snapshots: tried
[warn]   https://oss.sonatype.org/content/repositories/snapshots/org/scala-tools/testing/specs_2.10/1.6.9/specs_2.10-1.6.9-sources.src
[warn] ==== Scala-Tools Maven2 Snapshots Repository: tried
[warn]   http://scala-tools.org/repo-snapshots/org/scala-tools/testing/specs_2.10/1.6.9/specs_2.10-1.6.9-sources.src
[warn] ==== JBoss: tried
[warn]   https://repository.jboss.org/nexus/content/groups/public/org/scala-tools/testing/specs_2.10/1.6.9/specs_2.10-1.6.9-sources.src
[warn]  [FAILED     ] org.mockito#mockito-all;1.8.5!mockito-all.src:  (0ms)
[warn] ==== local: tried
[warn]   /home/ram/.ivy2/local/org.mockito/mockito-all/1.8.5/srcs/mockito-all-sources.src
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/org/mockito/mockito-all/1.8.5/mockito-all-1.8.5-sources.src
[warn] ==== Twitter Repo: tried
[warn]   http://maven.twttr.com/org/mockito/mockito-all/1.8.5/mockito-all-1.8.5-sources.src
[warn] ==== Typesafe Repo: tried
[warn]   http://repo.typesafe.com/typesafe/releases/org/mockito/mockito-all/1.8.5/mockito-all-1.8.5-sources.src
[warn] ==== Sonatype Releases: tried
[warn]   https://oss.sonatype.org/content/public/org/mockito/mockito-all/1.8.5/mockito-all-1.8.5-sources.src
[warn] ==== Typesafe Snapshots: tried
[warn]   http://repo.typesafe.com/typesafe/snapshots/org/mockito/mockito-all/1.8.5/mockito-all-1.8.5-sources.src
[warn] ==== Sonatype OSS Snapshots: tried
[warn]   https://oss.sonatype.org/content/repositories/snapshots/org/mockito/mockito-all/1.8.5/mockito-all-1.8.5-sources.src
[warn] ==== Scala-Tools Maven2 Snapshots Repository: tried
[warn]   http://scala-tools.org/repo-snapshots/org/mockito/mockito-all/1.8.5/mockito-all-1.8.5-sources.src
[warn] ==== JBoss: tried
[warn]   https://repository.jboss.org/nexus/content/groups/public/org/mockito/mockito-all/1.8.5/mockito-all-1.8.5-sources.src
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::              FAILED DOWNLOADS            ::
[warn]  :: ^ see resolution messages for details  ^ ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: junit#junit;4.8.1!junit.src
[warn]  :: org.scala-tools.testing#specs_2.10;1.6.9!specs_2.10.src
[warn]  :: org.mockito#mockito-all;1.8.5!mockito-all.src
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[trace] Stack trace suppressed: run last *:update for the full output.
[error] (*:update) sbt.ResolveException: download failed: junit#junit;4.8.1!junit.src

Compiler issue with Futures

I'm seeing some strange behavior in scala 2.10.2 with Future.

Welcome to Scala version 2.10.2 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_40).
Type in expressions to have them evaluated.
Type :help for more information.

scala> import com.twitter.util.Future
import com.twitter.util.Future
^

scala> def a(default: Int, f: Future[Int]) = {
| f handle { case _ => default }
| }
a: (default: Int, f: com.twitter.util.Future[Int])com.twitter.util.Future[Any]

scala> def a(default: Int, f: Future[Int]): Future[Int] = {
| f handle { case _ => default }
| }
:9: error: type mismatch;
found : A1 => B1
required: Int
f handle { case _ => default }

scala> def a(defaultValue: Int, f: Future[Int]): Future[Int] = {
| f handle { case _ => defaultValue }
| }
a: (defaultValue: Int, f: com.twitter.util.Future[Int])com.twitter.util.Future[Int]

Same test in 2.9.2 compiles without error.

setReadLength on ThriftCodec

Once we upgrade to Thrift 0.7.0, we can prevent the TProtocol from allocating huge amounts of memory by setting a read length limit:

override def invert(bytes: Array[Byte]) = {
    val obj = prototype.deepCopy
    val stream = new ByteArrayInputStream(bytes)
    val protocol = factory.getProtocol(new TIOStreamTransport(stream))
    // set upper bound on bytes available so that protocol does not try
    // to allocate and read large amounts of data in case of corrupt input
    protocol.setReadLength(bytes.length)
    obj.read(protocol)
    obj.asInstanceOf[T]
  }

[util-eval] scala 2.10 java.lang.NoSuchMethodError

scala> import com.twitter.util._
import com.twitter.util._

scala> val eval = new com.twitter.util.Eval(Some(new java.io.File(System.getProperty("java.io.tmpdir"))))
eval: com.twitter.util.Eval = com.twitter.util.Eval@2ce8dc6e

scala> eval("1+1")
java.lang.NoSuchMethodError: scala.Predef$.augmentString(Ljava/lang/String;)Lscala/collection/immutable/StringOps;
at com.twitter.util.Eval$IncludePreprocessor.apply(Eval.scala:387)
at com.twitter.util.Eval$IncludePreprocessor.apply(Eval.scala:384)
at com.twitter.util.Eval$$anonfun$sourceForString$1.apply(Eval.scala:117)
at com.twitter.util.Eval$$anonfun$sourceForString$1.apply(Eval.scala:116)
at scala.collection.LinearSeqOptimized$class.foldLeft(LinearSeqOptimized.scala:110)
at scala.collection.immutable.List.foldLeft(List.scala:78)
at com.twitter.util.Eval.sourceForString(Eval.scala:116)
at com.twitter.util.Eval.apply(Eval.scala:134)
at .(:12)

SyslogFormatter doesn't handle multiline messages well

The SyslogFormatter overrides lineterminator (to be ""), which means logs go to syslog as a single line. However, each sub-line (what would be a line in the default logger) still gets the priority & timestamp prefix, which makes for a very unreadable line.

For instance, here's my config:

new SyslogHandlerConfig {
  formatter = new SyslogFormatterConfig{
    serverName = "myserver"
  }
}

The following lines look fine:

Jan 22 19:43:35 app01.sea01.staging.abc [myserver] admin: Admin HTTP interface started on port 9900.
Jan 22 19:43:35 app01.sea01.staging.abc [myserver] http: Starting service

However, a stacktrace looks pretty bad (notice that the "<11>timestamp host..." prefix is repeated several times):

Jan 22 19:45:01 app01.sea01.staging.abc [myserver] stats: Error occurred recording stats<11>2014-01-22T19:45:01 app01.sea01.staging.abc [myserver] stats: com.abc.app.managers.DatabaseNotFoundException: Database with key 99229cd4e78a98357107814cc60 does not exist<11>2014-01-22T19:45:01 app01.sea01.staging.abc [myserver] stats:     at com.abc.app.managers.DatabaseManager$$anonfun$getByKey$1.apply(DatabaseManager.scala:48)<11>2014-01-22T19:45:01 app01.sea01.staging.abc [myserver] stats:     at com.abc.app.managers.DatabaseManager$$anonfun$getByKey$1.apply(DatabaseManager.scala:44)<11>2014-01-22T19:45:01 app01.sea01.staging.abc [myserver] stats:     at com.twitter.util.Try$.apply(Try.scala:13)<11>2014-01-22T19:45:01 app01.sea01.staging.abc [myserver] stats:     at com.twitter.util.ExecutorServiceFuturePool$$anon$2.run(FuturePool.scala:75)<11>2014-01-22T19:45:01 app01.sea01.staging.abc [myserver] stats:     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)<11>2014-01-22T19:45:01 app01.sea01.staging.abc [myserver] stats:     at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)<11>2014-01-22T19:45:01 app01.sea01.staging.abc [myserver] stats:     at java.util.concurrent.FutureTask.run(FutureTask.java:166)<11>2014-01-22T19:45:01 app01.sea01.staging.abc [myserver] stats:     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)<11>2014-01-22T19:45:01 app01.sea01.staging.abc [myserver] stats:     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)<11>2014-01-22T19:45:01 app01.sea01.staging.abc [myserver] stats:     at java.lang.Thread.run(Thread.java:701)<11>2014-01-22T19:45:01 app01.sea01.staging.abc [myserver] stats:     at com.abc.util.concurrent.ErrorLoggedThread.run(NamedThreadFactory.scala:25)

EDIT - code blocks to show <11>'s and spacing

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.