Comments (10)
Following up on chat on https://gitter.im/typelevel/general?at=57136f222c9711166432a27e
I'm seeing constant local build time of around 2h 30 min, mainly achieved by using artifactory with virtual repositories, as per #152 (comment)
The setup is really rather easy:
Just add the extra repositories listed in common.conf
to artifactory as Maven/Ivy/sbt remote repositories (Make sure "Suppress POM Consistency Checks" is ticked). At this point, you have just proxied some extra repos.
Then create three virtual repositories, such as (CB-Ivy, CB-Maven, CB-Sbt) and add the current repositories listed in common.conf
to the appropriate Virtual repo, in the same order. The idea here is that all the drop down through the repos is "pre-calculated"
Your repositories in common.conf
is then something like:
local
cb-ivy: YOUR-URL/artifactory/CB-Ivy, [organization]/[module]/(scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext]
cb-maven: YOUR-URL/artifactory/CB-Maven
cb-sbt: YOUR-URL/artifactory/CB-Sbt, [organization]/[module]/(scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext]
If needed, you could always create the Virtual repos per config - eg "CB-Ivy_2.12.x" and so on.
I already had Artifactory setup, but not used by default and it did help adding it to dbuild, but the dbuilds were still really slow. ( as in Ctrl-C slow...).
Adding the Virtual repos was the cure :)
from community-build.
Should be resolved by #243 and #244 (via scala/scala-jenkins-infra#171)
from community-build.
Note that this moves all the resolver config into artifactory, whose config isn't under source control... (scala/scala-jenkins-infra#172)
from community-build.
ugh, well, it looks like now the slowness turned into resolver issues
from community-build.
Looking into this, it seems the maven.twttr.com repo is returning jars using the text/html mimetype? E.g., https://maven.twttr.com/com/twitter/common/objectsize/0.0.10/objectsize-0.0.10.jar is showing up as text in my browser, and artifactory refuses to cache it. (https://scala-ci.typesafe.com/artifactory/twitter-maven/com/twitter/common/objectsize/0.0.10/objectsize-0.0.10.jar says:
{
"errors" : [ {
"status" : 409,
"message" : "Error fetching https://maven.twttr.com/com/twitter/common/objectsize/0.0.10/objectsize-0.0.10.jar: Mismatching mime types found in request: [application/java-archive] and response: text/html, and this repo blocks text/html on mismatch (remote response: 409: null)"
} ]
}
)
from community-build.
@stuhood is this something someone at Twitter could address? serving up artifacts with the wrong MIME type seems like something that could affect a variety of downstream tools, not just the Scala community build in particular
from community-build.
the problem on the Twitter end still exists afaik (I didn't check again just now), but we don't care so much at our end now since #243, so I'm closing this
from community-build.
fyi @mosesn @kevinoliver
from community-build.
@SethTisue thanks, we're aware of the problem but don't have an excellent solution yet.
from community-build.
recently: a team at Twitter "took it upon themselves to migrate their dependencies to maven central. The need to use maven.twttr.com should go away entirely soon"
Adriaan: "We'll drop the maven.twttr.com repo from our artifactory cache once the move is complete"
new ticket: scala/scala-jenkins-infra#197
from community-build.
Related Issues (20)
- add JDK 17 build HOT 8
- radically simplify the build HOT 6
- Add Cats Effect 3 to the community build HOT 6
- dbuild launcher JARs seem to have gone missing HOT 3
- policy for advancing project SHAs? HOT 1
- Move the JDK 17 build off early-access, onto 17 final HOT 3
- install latest scala-cli on behemoths HOT 2
- Delete and archive the Gitter room HOT 1
- Run the build on JDK 20 HOT 15
- Upgrade to sbt 1.8.0 HOT 1
- Upgrade to JDK 20 final once Temurin is available HOT 2
- fix failing sbt 0.13 builds in 2.12 build HOT 2
- Add JDK 21 to build matrix HOT 22
- add more scalafix projects HOT 12
- scalafmt failing HOT 12
- Scaladoc generation failing in Scalafix repo after recent updates HOT 2
- Add JDK 22-ea to build matrix HOT 4
- Add Akka resolver so I can unfreeze some downstream-from-Akka repos
- Upgrade behemoths to JDK 22 final HOT 1
- Add JDK 23 to matrix HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from community-build.