lemonlabsuk / scala-uri Goto Github PK

View Code? Open in Web Editor NEW

305.0 305.0 34.0 1.68 MB

Simple scala library for building and parsing URIs

License: Other

Scala 99.87% Shell 0.13%

scala scala-uri uri url url-parsing

scala-uri's People

Contributors

Stargazers

Watchers

scala-uri's Issues

Any chance this could be released for ScalaJS?

I am trying to parse and rewrite uris based on the current state in my webapp. This would greatly help!

ScalaJS linking error when calling addPathPart

scala-uri version: 2.2.2
ScalaJS version: 1.0.1
Scala version: 2.13.1

Describe the bug
When addPathPart and other methods that modify URLs such as "addParams" are used, fastOptJS task returns linking errors:

[error] Referring to non-existent class io.lemonlabs.uri.typesafe.PathPart$Ops
[error]   called from io.lemonlabs.uri.typesafe.PathPart$AllOps
[error]   called from io.lemonlabs.uri.typesafe.PathPart$ops$.toAllPathPartOps(java.lang.Object,io.lemonlabs.uri.typesafe.PathPart)io.lemonlabs.uri.typesafe.PathPart$AllOps
[error]   called from io.lemonlabs.uri.UrlPath.addPart(java.lang.Object,io.lemonlabs.uri.typesafe.PathPart)io.lemonlabs.uri.UrlPath
[error]   called from io.lemonlabs.uri.Url.addPathPart(java.lang.Object,io.lemonlabs.uri.typesafe.PathPart)io.lemonlabs.uri.Url
[error]   called from io.lemonlabs.Main$.writeUrl(io.lemonlabs.uri.Url)void
[error]   called from io.lemonlabs.Main$.urlTextTyped()void
[error]   called from io.lemonlabs.Main$.main([java.lang.String)void
[error]   called from static io.lemonlabs.Main.main([java.lang.String)void
[error]   called from core module module initializers
[error] involving instantiated classes:
[error]   io.lemonlabs.uri.typesafe.PathPart$ops$
[error]   io.lemonlabs.Main$
[error] Referring to non-existent method io.lemonlabs.uri.typesafe.PathPart$AllOps.path()java.lang.String
[error]   called from io.lemonlabs.uri.UrlPath.addPart(java.lang.Object,io.lemonlabs.uri.typesafe.PathPart)io.lemonlabs.uri.UrlPath
[error]   called from io.lemonlabs.uri.Url.addPathPart(java.lang.Object,io.lemonlabs.uri.typesafe.PathPart)io.lemonlabs.uri.Url
[error]   called from io.lemonlabs.Main$.writeUrl(io.lemonlabs.uri.Url)void
[error]   called from io.lemonlabs.Main$.urlTextTyped()void
[error]   called from io.lemonlabs.Main$.main([java.lang.String)void
[error]   called from static io.lemonlabs.Main.main([java.lang.String)void
[error]   called from core module module initializers
[error] involving instantiated classes:
[error]   io.lemonlabs.Main$
[error] Referring to non-existent class io.lemonlabs.uri.typesafe.PathPart$Ops
[error]   called from io.lemonlabs.uri.typesafe.PathPart$ops$$anon$2.path()java.lang.String
[error]   called from io.lemonlabs.uri.UrlPath.addPart(java.lang.Object,io.lemonlabs.uri.typesafe.PathPart)io.lemonlabs.uri.UrlPath
[error]   called from io.lemonlabs.uri.Url.addPathPart(java.lang.Object,io.lemonlabs.uri.typesafe.PathPart)io.lemonlabs.uri.Url
[error]   called from io.lemonlabs.Main$.writeUrl(io.lemonlabs.uri.Url)void
[error]   called from io.lemonlabs.Main$.urlTextTyped()void
[error]   called from io.lemonlabs.Main$.main([java.lang.String)void
[error]   called from static io.lemonlabs.Main.main([java.lang.String)void
[error]   called from core module module initializers
[error] involving instantiated classes:
[error]   io.lemonlabs.uri.typesafe.PathPart$ops$$anon$2
[error]   io.lemonlabs.Main$
[error] Referring to non-existent method io.lemonlabs.uri.typesafe.PathPart$Ops.path()java.lang.String
[error]   called from io.lemonlabs.uri.typesafe.PathPart$ops$$anon$2.path()java.lang.String
[error]   called from io.lemonlabs.uri.UrlPath.addPart(java.lang.Object,io.lemonlabs.uri.typesafe.PathPart)io.lemonlabs.uri.UrlPath
[error]   called from io.lemonlabs.uri.Url.addPathPart(java.lang.Object,io.lemonlabs.uri.typesafe.PathPart)io.lemonlabs.uri.Url
[error]   called from io.lemonlabs.Main$.writeUrl(io.lemonlabs.uri.Url)void
[error]   called from io.lemonlabs.Main$.urlTextTyped()void
[error]   called from io.lemonlabs.Main$.main([java.lang.String)void
[error]   called from static io.lemonlabs.Main.main([java.lang.String)void
[error]   called from core module module initializers
[error] involving instantiated classes:
[error]   io.lemonlabs.uri.typesafe.PathPart$ops$$anon$2
[error]   io.lemonlabs.Main$
[error] Referring to non-existent class io.lemonlabs.uri.typesafe.PathPart$Ops
[error]   called from constructor io.lemonlabs.uri.typesafe.PathPart$ops$$anon$2.<init>(java.lang.Object,io.lemonlabs.uri.typesafe.PathPart)void
[error]   called from io.lemonlabs.uri.typesafe.PathPart$ops$.toAllPathPartOps(java.lang.Object,io.lemonlabs.uri.typesafe.PathPart)io.lemonlabs.uri.typesafe.PathPart$AllOps
[error]   called from io.lemonlabs.uri.UrlPath.addPart(java.lang.Object,io.lemonlabs.uri.typesafe.PathPart)io.lemonlabs.uri.UrlPath
[error]   called from io.lemonlabs.uri.Url.addPathPart(java.lang.Object,io.lemonlabs.uri.typesafe.PathPart)io.lemonlabs.uri.Url
[error]   called from io.lemonlabs.Main$.writeUrl(io.lemonlabs.uri.Url)void
[error]   called from io.lemonlabs.Main$.urlTextTyped()void
[error]   called from io.lemonlabs.Main$.main([java.lang.String)void
[error]   called from static io.lemonlabs.Main.main([java.lang.String)void
[error]   called from core module module initializers
[error] involving instantiated classes:
[error]   io.lemonlabs.uri.typesafe.PathPart$ops$
[error]   io.lemonlabs.Main$
[error] Referring to non-existent method io.lemonlabs.uri.typesafe.PathPart$Ops.$init$()void
[error]   called from constructor io.lemonlabs.uri.typesafe.PathPart$ops$$anon$2.<init>(java.lang.Object,io.lemonlabs.uri.typesafe.PathPart)void
[error]   called from io.lemonlabs.uri.typesafe.PathPart$ops$.toAllPathPartOps(java.lang.Object,io.lemonlabs.uri.typesafe.PathPart)io.lemonlabs.uri.typesafe.PathPart$AllOps
[error]   called from io.lemonlabs.uri.UrlPath.addPart(java.lang.Object,io.lemonlabs.uri.typesafe.PathPart)io.lemonlabs.uri.UrlPath
[error]   called from io.lemonlabs.uri.Url.addPathPart(java.lang.Object,io.lemonlabs.uri.typesafe.PathPart)io.lemonlabs.uri.Url
[error]   called from io.lemonlabs.Main$.writeUrl(io.lemonlabs.uri.Url)void
[error]   called from io.lemonlabs.Main$.urlTextTyped()void
[error]   called from io.lemonlabs.Main$.main([java.lang.String)void
[error]   called from static io.lemonlabs.Main.main([java.lang.String)void
[error]   called from core module module initializers
[error] involving instantiated classes:
[error]   io.lemonlabs.uri.typesafe.PathPart$ops$
[error]   io.lemonlabs.Main$
[error] There were linking errors
[error] (Compile / fastOptJS) There were linking errors

To Reproduce

git clone https://github.com/lemonlabsuk/scala-uri-scalajs-example.git (commit lemonlabsuk/scala-uri-scalajs-example@3175f58)
in src/main/scala/io/lemonlabs/Main.scala file after line 62 add url.addPathPart("part").
Start sbt and execute fastOptJS task.

Become complaint with WHATWG url standard

Description

There are many tests that already exist for checking parsing URL's. For example web-platform-tests (see test cases into *.json files). We can reuse these tests for free, for improving the quality of this project. What do you think?

Updated:
Does 'scala-uri' parser support https://url.spec.whatwg.org/?

Return Option[Url] from .parse() instead of potentially throwing exception

I was surprised that the Url.parse() method didn't return an Option or Either, as this is a potentially error-prone operation, then I found UrlParser.getOrThrow() which explains the unwrapped return type. Instead of doing this and making the caller handle the exception, it would be better IMHO to return an Option[Url] or Either[Url]. You could also do this in a new method called Url.parseOption if you wanted to maintain current code.

Code Example:

val url = Url.parseOption("somethingBogus")
url mustEqual None

file system path doesn't work with relative paths

Describe the bug
If you create a Uri("file://pathPart1/file") I would expect the resulting Url to have a path "pathPart1/file" but it turns out as "/pathPart1/file" which makes Uri("file://pathPart1/file") == Uri("file:///pathPart1/file")

To Reproduce

val uri = Uri("file://pathPart1/file")
uri.toString.startsWith("/") must beFalse

Abstract over JSON library or depend on a different one

Description
The current version of this library depends on a spray-json library, that is not popular anymore. The solutions can be:

abstraction over the JSON library
depend on smth more popular (for instance Circe)

It is a great library that I would really like to use, but the spray-json dependency makes it not that attractive.

I would like to hear what are your thoughts about it, thanks!

GIT urls should be treated specifically

Thanks for a nice library. Unfortunately it cannot handle GIT URIs, could you fix that please?

@ Uri.parse("git+ssh://[email protected]:org/project.git")
io.lemonlabs.uri.parsing.UriParsingException: Invalid URL could not be parsed. Invalid input 'o', expected _int (line 1, column 26):
git+ssh://[email protected]:org/project.git
                         ^
  io.lemonlabs.uri.parsing.UrlParser$$anonfun$mapParseError$1.applyOrElse(UrlParser.scala:231)
  io.lemonlabs.uri.parsing.UrlParser$$anonfun$mapParseError$1.applyOrElse(UrlParser.scala:228)
  scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:38)
  scala.util.Failure.recoverWith(Try.scala:236)
  io.lemonlabs.uri.parsing.UrlParser.mapParseError(UrlParser.scala:228)
  io.lemonlabs.uri.parsing.UrlParser.parseUrl(UrlParser.scala:271)
  io.lemonlabs.uri.parsing.UrlParser$.parseUrl(UrlParser.scala:322)
  io.lemonlabs.uri.parsing.UriParser$.$anonfun$parseUri$1(UriParser.scala:27)
  scala.util.Failure.orElse(Try.scala:224)
  io.lemonlabs.uri.parsing.UriParser$.parseUri(UriParser.scala:27)
  io.lemonlabs.uri.Uri$.$anonfun$parseTry$2(Uri.scala:81)
  scala.util.Success.flatMap(Try.scala:251)
  io.lemonlabs.uri.Uri$.parseTry(Uri.scala:81)
  io.lemonlabs.uri.Uri$.parse(Uri.scala:87)
  ammonite.$sess.cmd9$.<init>(cmd9.sc:1)
  ammonite.$sess.cmd9$.<clinit>(cmd9.sc)


@ Url.parse("git+ssh://[email protected]:org/project.git")
io.lemonlabs.uri.parsing.UriParsingException: Invalid URL could not be parsed. Invalid input 'o', expected _int (line 1, column 26):
git+ssh://[email protected]:org/project.git
                         ^
  io.lemonlabs.uri.parsing.UrlParser$$anonfun$mapParseError$1.applyOrElse(UrlParser.scala:231)
  io.lemonlabs.uri.parsing.UrlParser$$anonfun$mapParseError$1.applyOrElse(UrlParser.scala:228)
  scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:38)
  scala.util.Failure.recoverWith(Try.scala:236)
  io.lemonlabs.uri.parsing.UrlParser.mapParseError(UrlParser.scala:228)
  io.lemonlabs.uri.parsing.UrlParser.parseUrl(UrlParser.scala:271)
  io.lemonlabs.uri.parsing.UrlParser$.parseUrl(UrlParser.scala:322)
  io.lemonlabs.uri.Url$.parseTry(Uri.scala:560)
  io.lemonlabs.uri.Url$.parse(Uri.scala:554)
  ammonite.$sess.cmd10$.<init>(cmd10.sc:1)
  ammonite.$sess.cmd10$.<clinit>(cmd10.sc)

@ Url.parse("[email protected]:org/project.git")
res11: Url = RelativeUrl(RootlessPath(Vector("[email protected]:org", "project.git")), QueryString(Vector()), None)

AbsoluteUrl cannot be cast to RelativeUrl

Code Example:

val url = Url.parse("http://example.com/somePart/subPart?q=1&p=2")
println(url.toRelativeUrl)

Results in

Exception in thread "main" java.lang.ClassCastException: AbsoluteUrl cannot be cast to RelativeUrl
	at io.lemonlabs.uri.Url$class.toRelativeUrl(Uri.scala:496)
	at io.lemonlabs.uri.AbsoluteUrl.toRelativeUrl(Uri.scala:795)
	at Test$.delayedEndpoint$Test$1(Test.scala:5)
	at Test$delayedInit$body.apply(Test.scala:3)
	at scala.Function0$class.apply$mcV$sp(Function0.scala:34)
	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
	at scala.App$$anonfun$main$1.apply(App.scala:76)
	at scala.App$$anonfun$main$1.apply(App.scala:76)
	at scala.collection.immutable.List.foreach(List.scala:381)
	at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
	at scala.App$class.main(App.scala:76)
	at Test$.main(Test.scala:3)
	at Test.main(Test.scala)

while I'm expecting to get /somePart/subPart?q=1&p=2

scala 2.11.8
scala-uri 1.1.5

More benchmarks

Public suffixes
URL building (addParam, addPathPart)
Rendering (toString)

Become cats friendly

Description
Provide instances of Eq, Order, Show, etc for all the Uri types such as Uri, Url, AbsoluteUrl, Host, etc.

Make Functor instances for mapping path parts, query params, etc?

support absolute file system paths

currently AbsoluteUrl does not support "file:///path/to/file" which is a 100% valid URI

import java.net.URI
import io.lemonlabs.uri.AbsoluteUrl
val path = "file:///path/to/file"
val a = new URI(path)
val b = AbsoluteUrl.parse(path)

Handling generic type of uris

Hello

I wanted to use the library, but as for 0.4.17 i experienced some strange issues:

For example for ftp:// it could work

val ftp = "ftp://t49e0444/FilewatcherFS/1990_00_copy/".uri
ftp.path
ftp.protocol
ftp.hostParts

ftp: com.netaporter.uri.Uri = ftp://t49e0444/FilewatcherFS/1990_00_copy/
res0: String = /FilewatcherFS/1990_00_copy/
res1: Option[String] = Some(ftp)
res2: Seq[String] = Vector(t49e0444)

But for file?

val file = "file:/Downloads/".uri
file.path
file.protocol
file.hostParts

file: com.netaporter.uri.Uri = /file:/Downloads/
res3: String = /file:/Downloads/
res4: Option[String] = None
res5: Seq[String] = Vector()

Or worse

val file2 = "file://Downloads/".uri
file2.path
file2.protocol
file2.hostParts

file2: com.netaporter.uri.Uri = file://Downloads/
res6: String = /
res7: Option[String] = Some(file)
res8: Seq[String] = Vector(Downloads)

it should work with one /, but it does not.
And with // it is just strange.

How to approach it?

Large port numbers are not accepted

Describe the bug
The parser seems to fail port numbers after 10 digits even though RFC 3981 imposes no limit on the length of a port number string.

To Reproduce

Tested on Scala 2.11, scala-uri 1.4.10 (let me know if this has already been fixed in 2.0)

import io.lemonlabs.uri.Url

val url = "hNu.u+m+-+q+://_%Ad:h%b1+%69:%E7@m'%c4&~%c2-%15%75.)%15c;%5a%FC$)*:10653127493172/*Q$F/%6A)%2a%9d@:@%6C/)@/%59%Fe@%49::/!!;:+@$z%e6:j%0c.%81::!%06z/@':+:o~@c%0D@@%06=%c3@:/,%16%6c:~@:@@%c3+:@@%60%F4s~-/%55:%b5@%BD%9e_:@:z:%a3%53/*%b7$//l:l@=%33%E5%B1:@.f@;%5b/%EA%35:j@%Ca,::xu/e:/r):@u'@%e3/%68u$&$;@*--%d4%c2$)::~%A3:M/%5a@@.:%3B,:(:@:::%b0/,@%57!-)@')@/q@@;%78:/@~&:~%5c(*(:@%C1-%3b:?//:?@/v#??:@?://@/"

Url.parse(url)

[info]   UriParsingException was thrown during property evaluation.
[info]     Message: Invalid URL could not be parsed. Invalid input '3', expected _path_for_authority, _maybe_query_string, _fragment or 'EOI' (line 1, column 78):
[info]   hNu.u+m+-+q+://_%Ad:h%b1+%69:%E7@m'%c4&~%c2-%15%75.)%15c;%5a%FC$)*:10653127493172/*Q$F/%6A)%2a%9d@:@%6C/)@/%59%Fe@%49::/!!;:+@$z%e6:j%0c.%81::!%06z/@':+:o~@c%0D@@%06=%c3@:/,%16%6c:~@:@@%c3+:@@%60%F4s~-/%55:%b5@%BD%9e_:@:z:%a3%53/*%b7$//l:l@=%33%E5%B1:@.f@;%5b/%EA%35:j@%Ca,::xu/e:/r):@u'@%e3/%68u$&$;@*--%d4%c2$)::~%A3:M/%5a@@.:%3B,:(:@:::%b0/,@%57!-)@')@/q@@;%78:/@~&:~%5c(*(:@%C1-%3b:?//:?@/v#??:@?://@/

The parser crashes on the 10th digit of the number 10653127493172.

The test input is originally from a ScalaCheck generator and ScalaCheck is not shrinking it properly, hence the crazy URL.

mdoc

Typeclass support in non-DSL API

Description
Investigate including the typeclasses added in #51 in the core API

For example, so you can do something like:

import io.lemonlabs.uri.Url
final case class Foo(a: Int, b: String)
 object Foo {
  implicit val traversableParams: TraversableParams[Foo] = TraversableParams.product
}

val foo = Foo(a = 1, b = "bar")
Url.parse("http://theon.github.com/scala-uri").withParams(foo)

opaque URI handling compared with java.net.URI

Hi there,

I've been playing with your nice library for parsing URI string and noticed a behavior not consistent with java.net.URI for opaque URI which do not have paths (https://docs.oracle.com/javase/8/docs/api/java/net/URI.html).

scala> new java.net.URI("mailto:[email protected]").getPath
res0: String = null

scala> parse("mailto:[email protected]").path
res1: String = /mailto:[email protected]

Is it intentional?

Url parsing garbled problem

Describe the bug
url decode garbled

Example:

val urlString = "/v1/popplan/?v=6.2.5507.400&did=%e6%44%9b%ca%94%15%9f%0f%3e%aa%16%b7%ca%94%02%e6%8f%a1%9d%5a%b2%9a%78%b7%45%bf%81%93%9b%42%70%c6%36%87%2c%63%5c%fb%f0%7a%67%28%ff%ea%b0%31%7b%27&p=%bf%27%fb%c0%29%7f%14%9a%d0%c2%f3%9e%f9%19%66%fc%6a%8e%45%ee%98%85%0f%9a%52%09%ce%3c%c4%d1%b1%6d%01%5c%d0%fa%5d%74%be%c2%9c%5c%8a%ea%68%f3%9c%c5%df%97%98%09%6f%ed%e6%95%db%76%1b%5d%66%02%4e%11%ae%9c%4d%8d%59%03%24%68%64%ea%dc%3e%5a%5c%0e%8a%1a%eb%71%c9%eb%08%1a%82%fd%93%13%aa%2e%38%d6%64%64%39%07%aa%5b%58%c8%f6%bb%7a%50%e7%97%51%77%a8%81%bc%93%38%df%c8%99%b6%cf%78%91%c5%f9%ae%44%d6%73%bb%f8%ed%05%e3%76%ce%e2%9a%91%63%bf%5c%b1%77%0d%71%45%4f%02%b3%83%db%a9%21%01%04%0b%0a%a8%c7%48%95%b7%82%38%ec%f8%7c%ce%7b%08%f6%78%b6%e4%d3%8a%fa%cc%be%80%06%48%25%d9%45%06%5f%dc%6c%b4%bc%53%d7%8d%d3%a1%4c%a9%7c%75%b2%1f%89%9f%eb%96%2a"
  implicit val c = UriConfig(
    decoder = PercentDecoder(ignoreInvalidPercentEncoding = true)
  )
  val decoderUrl = Url.parse(urlString).toStringRaw.toLowerCase
  println(decoderUrl)

result is

(qeo���۩!���
��h���8��|��x��ӊ�̾��h%�e�_�l��s׍ӡl�|u�����*  )

Can you help me?
thanks

scala-uri_sjs0.6_2.11-0.5.0.pom not published to maven central

Pulling down the code and running sbt +publishLocal generates the sjs 2.11 jars, so it may be as simple as running sbt +publish with the right credentials.

ignoreInvalidPercentEncoding encodes %s of correctly encoded URLs

It seems that ignoreInvalidPercentEncoding encodes the%s of correctly encoded URLs.

Code Example:

import io.lemonlabs.uri.Url
import io.lemonlabs.uri.config.UriConfig
import io.lemonlabs.uri.decoding.PercentDecoder

implicit val c = UriConfig(
  decoder = PercentDecoder(ignoreInvalidPercentEncoding = true)
)
val url = Url.parse("example.com/path%20with%20space")
url.toString // "example.com/path%2520with%2520space"

"%20" becomes "%2520"

Use scala-steward?

Description

Scala Steward is a robot that helps you keeping library dependencies and sbt plugins up-to-date.

https://github.com/fthomas/scala-steward

Example
All you have to do is edit this repos and send PR.

Scoverage issues in production

Hello,

I'm having issues with scoverage being used at runtime in a Play app in production mode. It actually tries to write into /work/scala-uri/jvm/target/scala-2.12/scoverage-data/ (which obviously does not exist so it fails, badly).

Looks like one should not define coverageEnabled in the build.sbt file as written here: scoverage/sbt-scoverage#84 (comment)

Can you fix this and publish a version without coverage enabled?

Failed to parse url that starts with long number

The heuristics that determine the presence of the IP address in the autority part of URL, is incorrect.
Capturing too many numbers leads to java.lang.NumberFormatException

How to reproduce?

import io.lemonlabs.uri.Url

Url.parse("https://%1.0fexample.com/" format math.pow(2, 31))

Deprecate Old DSL

Description

Thanks to the new typesafe DSL provided in #51, we should now deprecate the old DSL.

2.0.0 Benchmarking

Prior to releasing 2.0.0, I would like to get scala-uri-benchmarks migrated over to sbt-jmh and do a comparison against previous versions

Check conformity to WHATWG URL standard

Description
Check scala-uri conforms to the following WHATWG standard:
https://url.spec.whatwg.org/

Support dotty as a build target

Description
It would be great if this project also cross compiled to dotty.

Add Scala 2.13 support

Question: Support for IDNs as punycode

Hi, thanks for all the work on this library, it's been a big help so far.

The only "issue" I'm currently facing are internationalized domain names (or at least their representation in ASCII as punycode (RFC 3492)).

The problem is that domains have to be encoded in punycode, while the path has to be urlencoded, for most HTTP clients to work. There are libraries that can convert IDNs to punycode, but need the separate domain. scala-uri can separate domain from a URL, but doesn't convert the domain to punycode itself.

The workaround is short, but feels wrong/ugly:

import java.net.IDN

val url = UrlParser.parseAbsoluteUrl("https://はじめよう.みんな/howto.html").get
val punycodeUrl = url.withHost(IDN.toASCII(url.host.toString))

Are there plans to support conversion to punycode in scala-uri itself?

Support zone indicies for IPv6 hosts

https://en.wikipedia.org/wiki/IPv6_address#Use_of_zone_indices_in_URIs

Colons in path segments are not interpreted correctly

Describe the bug
Tested on Scala 2.11, scala-uri 1.4.10 (let me know if this has already been fixed in 2.0)

To Reproduce

val url = n+Pw+yb-k.+czhe+p.d.kgx-.s-8++..t+-.6-.s+-+-gi://32.60.46.58:3469/@@%f2@%4e::n@=:@:.:v*$%43*:%65@@:@/:@%89%c3,z@%4e;@c~+%08@_':'::%B2%e0@$@(o%84J%7f+::%81::%25@%2be-%ED+_~::%62.%62%23%BF/:'+:::@=i::=-%A0:%05@:_::@@!@;%fb):g:+@!*7%1B@:)@)x@c%34:)%f2@:%36)/%F1%5b:.%cd!::%35&@@%eaw.::m%9B:,%Ca:e%44::(@:/+@:@@:@@*.-(+%69r'%C1@n-0::E:/*%36%67,%4f'%A3%f9@@%27:=@@!@6t%30@:*_~@$@@%df():.@:x'_%02:@~@!::%63%7B%B4@%d0%39-_%e8@:/@'D:@-%56@%cF@%f7v:%c7(!;~)@_:y:d*::_@+%6d%E8@%1D:&@@.(:%DE:@=g(@://@@%a6@x@.&&@-%80':-::@_b%b9~%c9:%d1)@&:@,$@(*%0Ef@@~):%63'%30:!@@!:q:%4A%44%c8/)7-%42)g%a9::%A1/5%2b%cb'%aF::@/:L%1D:_%A4:@%76@::@%64@:/==Nt,&@-:@w~@,!,:%7Bm@i@@%01:@@:@@$:%b5@:@@&+f:@@:=@::@@:-%0c@::/%E5h%e4@3%61).@:%67%4E:@y%c2$@@%85:@r@@=W%bE%b5%1C@*%3e%f4%10_:%B1_@::+:%20::@_@(:(/:=~:::=~,++%25@:$:h~%3C@%DD_:%4es@:@@@.-:%c1%5D@@()~.-:%C2:n%79:&/@@%C9@)/e@@@:@%9D!::!@@@&$@%BA/:@$:(,@:@_$_%12:&.:@%63@&,~%d7%08:-@y@$@@%53(:x%9E;@::%6E:~%9E%88(;*o%71/w:@%aa-%D0%DA.%5d~%83:=':%2f:g@%34@%DC;@@%d7@h~_:,:::@@:%f7%73:~:c%42~:s%65:&l@+_/@%88:%48@:@::,/!/@:z,-@%3b:@%82*@@&%63%E5~:+,::,@@%9eg@;!'@::%DC@%44'/:w::%5ea%F9~%31$:*%0fc,@@:':%c9_:(=%fF2::%02-$b%42M/:)x8::,@~_@+U:-_/%Fa@)_@),%D1):@:,='@:_1@;:@+%33*:%cB/%98%bA)'@@),%cF(.$@&%44)%d7'!::%9A_:*-%9A:%89!$,:;~@%E0%9f$$;:*!:l%13@@-*c!:u:,/=@/%0E%6C::%EF%0A+s@(@%EEwi@@:%36%e93u/.q~J.(&%C9%ec_@@~%21.+@(-%59%7dK;@%ee@_:/%AC1.:*$(.:@'@*:&%80.::@@:+%86@%33@*%61::$_:M%53@@@)=%3e!+':;$o/@1$~@@r%E4_z::&%78:@%e0@v@:%66@@/:i*:~~%49~!%1E@:_@.&%Cc:%78:e/_@_%43%aC$l@x@x::;~(@=%01:r%2F:7@:o:@@@j@-@%eC%08!6:~@=!_~/@%83@%86:::%19/@%cC%93-@3/:_@'::@%Fek:%36@:(!;:%71@&:@@(%e2::/:@:/+&m@5@&@$::@%C3%2d@.:)v@+m~:::-%1F@yI_@%48n@%5a::%43@;@p/*@.%72--::='_:%4A~:l(:@.:%3b,g!:~:$-_;@+=%75k%e5:@;l*@@g/m:&:@,_::z-:6~:%F9a=/:-%e3%Ec%80@@)@='@.+x~&%a9%e9'%35@+-:+z@/%c8:%5d%52@%ca_~x@@:a%41!@./$:w,,(%3f@/'%bE@_%99@%b2/=mrd%3dy%3a:/:%85%30:@(%a0%FF@:9=%1c::@,%5F@q:~@@/%c7:$%86c/%10/_%e4-,::)%a8%ab:(__:@:%B3@k@(%9Da:@:@.u:@$&x@-:%1eh)::%94@%9b%34@:/_%3e:%0B%65%d0%1dC:@d@@@%bbe.%CC:p@::@@@%96::%8a.%8e$@%2B%C3:Z;.;.@@:@i!.+@@z$=:/@%17:@+:~%aF=%d8&@O@@@:-::@::o/@:.@.::'%63!_%F6m:%c4?//s//?%EF/?@?!@/:?~?*.?/:?//?&%48////?,?//??@/#%80?//:/,.???y//:=?/:@//:??:;/?*????/@&/@@k?/%b5?*?@

Url.parse(url)

[info]     Message: Invalid URL could not be parsed. Invalid input ':', expected _int (line 1, column 77):
[info]   n+Pw+yb-k.+czhe+p.d.kgx-.s-8++..t+-.6-.s+-+-gi://32.60.46.58:3469/@@%f2@%4e::n@=:@:.:v*$%43*:%65@@:@/:@%89%c3,z@%4e;@c~+%08@_':'::%B2%e0@$@(o%84J%7f+::%81::%25@%2be-%ED+_~::%62.%62%23%BF/:'+:::@=i::=-%A0:%05@:_::@@!@;%fb):g:+@!*7%1B@:)@)x@c%34:)%f2@:%36)/%F1%5b:.%cd!::%35&@@%eaw.::m%9B:,%Ca:e%44::(@:/+@:@@:@@*.-(+%69r'%C1@n-0::E:/*%36%67,%4f'%A3%f9@@%27:=@@!@6t%30@:*_~@$@@%df():.@:x'_%02:@~@!::%63%7B%B4@%d0%39-_%e8@:/@'D:@-%56@%cF@%f7v:%c7(!;~)@_:y:d*::_@+%6d%E8@%1D:&@@.(:%DE:@=g(@://@@%a6@x@.&&@-%80':-::@_b%b9~%c9:%d1)@&:@,$@(*%0Ef@@~):%63'%30:!@@!:q:%4A%44%c8/)7-%42)g%a9::%A1/5%2b%cb'%aF::@/:L%1D:_%A4:@%76@::@%64@:/==Nt,&@-:@w~@,!,:%7Bm@i@@%01:@@:@@$:%b5@:@@&+f:@@:=@::@@:-%0c@::/%E5h%e4@3%61).@:%67%4E:@y%c2$@@%85:@r@@=W%bE%b5%1C@*%3e%f4%10_:%B1_@::+:%20::@_@(:(/:=~:::=~,++%25@:$:h~%3C@%DD_:%4es@:@@@.-:%c1%5D@@()~.-:%C2:n%79:&/@@%C9@)/e@@@:@%9D!::!@@@&$@%BA/:@$:(,@:@_$_%12:&.:@%63@&,~%d7%08:-@y@$@@%53(:x%9E;@::%6E:~%9E%88(;*o%71/w:@%aa-%D0%DA.%5d~%83:=':%2f:g@%34@%DC;@@%d7@h~_:,:::@@:%f7%73:~:c%42~:s%65:&l@+_/@%88:%48@:@::,/!/@:z,-@%3b:@%82*@@&%63%E5~:+,::,@@%9eg@;!'@::%DC@%44'/:w::%5ea%F9~%31$:*%0fc,@@:':%c9_:(=%fF2::%02-$b%42M/:)x8::,@~_@+U:-_/%Fa@)_@),%D1):@:,='@:_1@;:@+%33*:%cB/%98%bA)'@@),%cF(.$@&%44)%d7'!::%9A_:*-%9A:%89!$,:;~@%E0%9f$$;:*!:l%13@@-*c!:u:,/=@/%0E%6C::%EF%0A+s@(@%EEwi@@:%36%e93u/.q~J.(&%C9%ec_@@~%21.+@(-%59%7dK;@%ee@_:/%AC1.:*$(.:@'@*:&%80.::@@:+%86@%33@*%61::$_:M%53@@@)=%3e!+':;$o/@1$~@@r%E4_z::&%78:@%e0@v@:%66@@/:i*:~~%49~!%1E@:_@.&%Cc:%78:e/_@_%43%aC$l@x@x::;~(@=%01:r%2F:7@:o:@@@j@-@%eC%08!6:~@=!_~/@%83@%86:::%19/@%cC%93-@3/:_@'::@%Fek:%36@:(!;:%71@&:@@(%e2::/:@:/+&m@5@&@$::@%C3%2d@.:)v@+m~:::-%1F@yI_@%48n@%5a::%43@;@p/*@.%72--::='_:%4A~:l(:@.:%3b,g!:~:$-_;@+=%75k%e5:@;l*@@g/m:&:@,_::z-:6~:%F9a=/:-%e3%Ec%80@@)@='@.+x~&%a9%e9'%35@+-:+z@/%c8:%5d%52@%ca_~x@@:a%41!@./$:w,,(%3f@/'%bE@_%99@%b2/=mrd%3dy%3a:/:%85%30:@(%a0%FF@:9=%1c::@,%5F@q:~@@/%c7:$%86c/%10/_%e4-,::)%a8%ab:(__:@:%B3@k@(%9Da:@:@.u:@$&x@-:%1eh)::%94@%9b%34@:/_%3e:%0B%65%d0%1dC:@d@@@%bbe.%CC:p@::@@@%96::%8a.%8e$@%2B%C3:Z;.;.@@:@i!.+@@z$=:/@%17:@+:~%aF=%d8&@O@@@:-::@::o/@:.@.::'%63!_%F6m:%c4?//s//?%EF/?@?!@/:?~?*.?/:?//?&%48////?,?//??@/#%80?//:/,.???y//:=?/:@//:??:;/?*????/@&/@@k?/%b5?*?@

Excuse the confusing URL, it was auto-generated by ScalaCheck and the shrinking is not working correctly.

The error occurs on the first occurrence of ::, when trying to parse the second colon; it seems to think it is parsing a port number even though there is a port number earlier in the URL.

It may think it is parsing a path-noscheme which would not allow colon characters; however, this URL has the scheme n+Pw+yb-k.+czhe+p.d.kgx-.s-8++..t+-.6-.s+-+-gi, which is valid. So it should be parsing a path-abempty which would interpret a colon as just another character in the path segment.

Ensure compatiblity with Graal

Hey there,

first of all i would like to thank for developing this software! I am a backend engineer at Prisma. I am opening this issue as we are a happy user of this library and would like it to be compatible with Graal.

What is Graal?

Graal is a collection of tools. For one it is a JVM replacement that does not only run Java Bytecode but also other languages like Javascript or Ruby. It also comes with a tool called native image. This allows to compile a JAR into a standalone binary.

We would like to turn our Prisma Server into a lightweight binary with Graal.

However the Graal native image tool comes with some limitations. Some features of the JVM are not supported, e.g. the support for reflection is limited because it is not a good fit with ahead of time compilation. Therefore we created a test suite to check the compatibility of our used dependencies with Graal. For each dependency we tried to come up with a super small program and see whether it compiles with Graal or not.

The program to test compatibility with this library can be found here.

What's the problem we have hit?
When running with -H:+ReportUnsupportedElementsAtRuntime:

Exception in thread "main" java.lang.NullPointerException
	at scala.io.Source$.fromURL(Source.scala:141)
	at scala.io.Source$.fromURL(Source.scala:136)
	at io.lemonlabs.uri.inet.PublicSuffixSupportImpl$.trie$lzycompute(PublicSuffixSupportImpl.scala:12)
	at io.lemonlabs.uri.inet.PublicSuffixSupportImpl$.trie(PublicSuffixSupportImpl.scala:10)
	at io.lemonlabs.uri.inet.PublicSuffixSupportImpl.publicSuffix(PublicSuffixSupportImpl.scala:28)
	at io.lemonlabs.uri.inet.PublicSuffixSupportImpl.publicSuffix$(PublicSuffixSupportImpl.scala:27)
	at io.lemonlabs.uri.DomainName.publicSuffix(Host.scala:59)
	at io.lemonlabs.uri.DomainName.longestSubdomain(Host.scala:115)
	at io.lemonlabs.uri.DomainName.subdomain(Host.scala:72)
	at io.lemonlabs.uri.Authority.subdomain(Authority.scala:46)
	at io.lemonlabs.uri.UrlWithAuthority.subdomain(Uri.scala:692)
	at io.lemonlabs.uri.UrlWithAuthority.subdomain$(Uri.scala:691)
	at io.lemonlabs.uri.AbsoluteUrl.subdomain(Uri.scala:780)
	at Main$.main(Main.scala:9)
	at Main.main(Main.scala)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:177)

When running without -H:+ReportUnsupportedElementsAtRuntime:

Exception in thread "main" java.lang.NullPointerException
	at scala.io.Source$.fromURL(Source.scala:141)
	at scala.io.Source$.fromURL(Source.scala:136)
	at io.lemonlabs.uri.inet.PublicSuffixSupportImpl$.trie$lzycompute(PublicSuffixSupportImpl.scala:12)
	at io.lemonlabs.uri.inet.PublicSuffixSupportImpl$.trie(PublicSuffixSupportImpl.scala:10)
	at io.lemonlabs.uri.inet.PublicSuffixSupportImpl.publicSuffix(PublicSuffixSupportImpl.scala:28)
	at io.lemonlabs.uri.inet.PublicSuffixSupportImpl.publicSuffix$(PublicSuffixSupportImpl.scala:27)
	at io.lemonlabs.uri.DomainName.publicSuffix(Host.scala:59)
	at io.lemonlabs.uri.DomainName.longestSubdomain(Host.scala:115)
	at io.lemonlabs.uri.DomainName.subdomain(Host.scala:72)
	at io.lemonlabs.uri.Authority.subdomain(Authority.scala:46)
	at io.lemonlabs.uri.UrlWithAuthority.subdomain(Uri.scala:692)
	at io.lemonlabs.uri.UrlWithAuthority.subdomain$(Uri.scala:691)
	at io.lemonlabs.uri.AbsoluteUrl.subdomain(Uri.scala:780)
	at Main$.main(Main.scala:9)
	at Main.main(Main.scala)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:177)

Note: The Graal compilation terminates on the first encountered error. Therefore there might be more problems that need to be addressed.

How can incompatibilities with Graal be fixed?

Here is a blog post by the Graal team that explains how the native image tool is working and how they were able to compile a server built on top of netty into a binary.

We ran this test with Graal 1.0.0-rc5. We used scala-uri version 1.1.1.

Scala.js linkingErrors on Uri.addParams(map)

Describe the bug

scala-uri version: 2.2.2
ScalaJS version: 1.0.1 & 1.1.0
Scala version: 2.13.2

Uri.addParams(map) worked on Scala.js 0.6.x, but on Scala.js 1.0.0/1.1.0, it start failiing when js-linkning (fastOptJS/fullOptJS) time.

Not sure, but maybe related to #152

To Reproduce

val params = Map.empty[String,String]
val withParams = Url.parse("/foo/bar").addParams(params)

stack trace

[error] Referring to non-existent class io.lemonlabs.uri.typesafe.TraversableParams$Ops
[error]   called from io.lemonlabs.uri.typesafe.TraversableParams$AllOps
[error]   called from io.lemonlabs.uri.typesafe.TraversableParams$ops$.toAllTraversableParamsOps(java.lang.Object,io.lemonlabs.uri.typesafe.TraversableParams)io.lemonlabs.uri.typesafe.TraversableParams$AllOps
[error]   called from io.lemonlabs.uri.QueryString.addParams(java.lang.Object,io.lemonlabs.uri.typesafe.TraversableParams)io.lemonlabs.uri.QueryString
[error]   called from io.lemonlabs.uri.Url.addParams(java.lang.Object,io.lemonlabs.uri.typesafe.TraversableParams)io.lemonlabs.uri.Url
[error]   called from Foo$.main([java.lang.String)void
[error]   called from static Foo.main([java.lang.String)void
[error]   called from core module module initializers
[error] involving instantiated classes:
[error]   io.lemonlabs.uri.typesafe.TraversableParams$ops$
[error]   io.lemonlabs.uri.QueryString
[error]   Foo$
[error] Referring to non-existent method io.lemonlabs.uri.typesafe.TraversableParams$AllOps.toSeq()scala.collection.immutable.Seq
[error]   called from io.lemonlabs.uri.QueryString.addParams(java.lang.Object,io.lemonlabs.uri.typesafe.TraversableParams)io.lemonlabs.uri.QueryString
[error]   called from io.lemonlabs.uri.Url.addParams(java.lang.Object,io.lemonlabs.uri.typesafe.TraversableParams)io.lemonlabs.uri.Url
[error]   called from Foo$.main([java.lang.String)void
[error]   called from static Foo.main([java.lang.String)void
[error]   called from core module module initializers
[error] involving instantiated classes:
[error]   io.lemonlabs.uri.QueryString
[error]   Foo$
[error] Referring to non-existent class io.lemonlabs.uri.typesafe.TraversableParams$Ops
[error]   called from constructor io.lemonlabs.uri.typesafe.TraversableParams$ops$$anon$11.<init>(java.lang.Object,io.lemonlabs.uri.typesafe.TraversableParams)void
[error]   called from io.lemonlabs.uri.typesafe.TraversableParams$ops$.toAllTraversableParamsOps(java.lang.Object,io.lemonlabs.uri.typesafe.TraversableParams)io.lemonlabs.uri.typesafe.TraversableParams$AllOps
[error]   called from io.lemonlabs.uri.QueryString.addParams(java.lang.Object,io.lemonlabs.uri.typesafe.TraversableParams)io.lemonlabs.uri.QueryString
[error]   called from io.lemonlabs.uri.Url.addParams(java.lang.Object,io.lemonlabs.uri.typesafe.TraversableParams)io.lemonlabs.uri.Url
[error]   called from Foo$.main([java.lang.String)void
[error]   called from static Foo.main([java.lang.String)void
[error]   called from core module module initializers
[error] involving instantiated classes:
[error]   io.lemonlabs.uri.typesafe.TraversableParams$ops$
[error]   io.lemonlabs.uri.QueryString
[error]   Foo$
[error] Referring to non-existent method io.lemonlabs.uri.typesafe.TraversableParams$Ops.$init$()void
[error]   called from constructor io.lemonlabs.uri.typesafe.TraversableParams$ops$$anon$11.<init>(java.lang.Object,io.lemonlabs.uri.typesafe.TraversableParams)void
[error]   called from io.lemonlabs.uri.typesafe.TraversableParams$ops$.toAllTraversableParamsOps(java.lang.Object,io.lemonlabs.uri.typesafe.TraversableParams)io.lemonlabs.uri.typesafe.TraversableParams$AllOps
[error]   called from io.lemonlabs.uri.QueryString.addParams(java.lang.Object,io.lemonlabs.uri.typesafe.TraversableParams)io.lemonlabs.uri.QueryString
[error]   called from io.lemonlabs.uri.Url.addParams(java.lang.Object,io.lemonlabs.uri.typesafe.TraversableParams)io.lemonlabs.uri.Url
[error]   called from Foo$.main([java.lang.String)void
[error]   called from static Foo.main([java.lang.String)void
[error]   called from core module module initializers
[error] involving instantiated classes:
[error]   io.lemonlabs.uri.typesafe.TraversableParams$ops$
[error]   io.lemonlabs.uri.QueryString
[error]   Foo$
[error] Referring to non-existent class io.lemonlabs.uri.typesafe.TraversableParams$Ops
[error]   called from io.lemonlabs.uri.typesafe.TraversableParams$ops$$anon$11.toSeq()scala.collection.immutable.Seq
[error]   called from io.lemonlabs.uri.QueryString.addParams(java.lang.Object,io.lemonlabs.uri.typesafe.TraversableParams)io.lemonlabs.uri.QueryString
[error]   called from io.lemonlabs.uri.Url.addParams(java.lang.Object,io.lemonlabs.uri.typesafe.TraversableParams)io.lemonlabs.uri.Url
[error]   called from Foo$.main([java.lang.String)void
[error]   called from static Foo.main([java.lang.String)void
[error]   called from core module module initializers
[error] involving instantiated classes:
[error]   io.lemonlabs.uri.typesafe.TraversableParams$ops$$anon$11
[error]   io.lemonlabs.uri.QueryString
[error]   Foo$
[error] Referring to non-existent method io.lemonlabs.uri.typesafe.TraversableParams$Ops.toSeq()scala.collection.immutable.Seq
[error]   called from io.lemonlabs.uri.typesafe.TraversableParams$ops$$anon$11.toSeq()scala.collection.immutable.Seq
[error]   called from io.lemonlabs.uri.QueryString.addParams(java.lang.Object,io.lemonlabs.uri.typesafe.TraversableParams)io.lemonlabs.uri.QueryString
[error]   called from io.lemonlabs.uri.Url.addParams(java.lang.Object,io.lemonlabs.uri.typesafe.TraversableParams)io.lemonlabs.uri.Url
[error]   called from Foo$.main([java.lang.String)void
[error]   called from static Foo.main([java.lang.String)void
[error]   called from core module module initializers
[error] involving instantiated classes:
[error]   io.lemonlabs.uri.typesafe.TraversableParams$ops$$anon$11
[error]   io.lemonlabs.uri.QueryString
[error]   Foo$
[error] There were linking errors
[error] (Compile / fastOptJS) There were linking errors
[error] Total time: 1 s, completed 2020/06/02 11:44:59

Easy way

I created a repo for minimum reproduction code with Scala.js 1.1.0 setup.

git clone https://github.com/exoego/scalajs1-errors.git --branch scala-uri-2.2.2
cd scalajs1-errors
sbt fastOptJS

parseOption throws NullPointerException instead of returning None for null input

parseOption throws NullPointerException instead of returning None for null input

Public suffix com.bd

There is no public suffix com.bd In PublicSuffixes sets

Not Validating the authority for Absolute Urls

AbsoluteUrl parsing is not validating the domain name in authority to only allow LDH ("letter digit hyphen") characters. Where as standard Java URI honors it. This is what it is mentioned in RFC-3986

Such a name consists of a sequence of domain labels separated by ".", each domain label starting and ending with an alphanumeric character and possibly also containing "-" characters

To Reproduce

import io.lemonlabs.uri.AbsoluteUrl
import java.net.URI

scala> println(AbsoluteUrl.parse("https://www.goog le.com?q=i+am+invalid"))
https://www.goog le.com?q=i%2Bam%2Binvalid

scala> println(new URI("https://www.goog le.com?q=i+am+invalid").toURL)
java.net.URISyntaxException: Illegal character in authority at index 8: https://www.goog le.com?q=i+am+invalid
  at java.net.URI$Parser.fail(URI.java:2848)
  at java.net.URI$Parser.parseAuthority(URI.java:3186)
  at java.net.URI$Parser.parseHierarchical(URI.java:3097)
  at java.net.URI$Parser.parse(URI.java:3053)
  at java.net.URI.<init>(URI.java:588)
  ... 36 elided

Does not accept ipv4 address as last 2 segments of ipv6 address

Describe the bug
RFC 3986 allows the least-significant two segments of an IPv6 address to be represented in IPv4 address textual format, but the URL parser does not accept this style.

To Reproduce

val url = "s://@[f4E0:F65b:734E:dc04:026b:8629:226.252.129.66]:3298571?"
Url.parse(url)

[info]     Message: Invalid URL could not be parsed. Invalid input '.', expected HexDigit or ':' (line 1, column 40):
[info]   s://@[f4E0:F65b:734E:dc04:026b:8629:226.252.129.66]:3298571?

Url parsing regression.

Hi,

We're trying to update scala-uri to the latest version (from 0.4.x), and we're running into a regression for some URLs (admittedly, pretty strange ones 😄):

soundcloud://sounds:78237871?target_url=https%3A%2F%2Fsoundcloud.com%2Fosfdjskj-dkfsdfa%2Fofsfhsdlfsf

Interestingly, this works in the scala js example website: https://lemonlabs.io/scala-uri-scalajs-example/index.html

This likely got introduce between 1.0.0-rc1 and 1.1.4.

Code Example:

val url = Url.parse("soundcloud://sounds:476949372?target_url=https%3A%2F%2Fsoundcloud.com%2Fosvaldo-j-figueroa%2Fosvval-tat-freestyle")

Results in the following:

[info] - should result in a valid Uri object *** FAILED ***
[info]   io.lemonlabs.uri.parsing.UriParsingException: Invalid URL could not be parsed. Unexpected end of input, expected [^@] or '@' (line 1, column 102):
[info] soundcloud://sounds:78237871?target_url=https%3A%2F%2Fsoundcloud.com%2Fosfdjskj-dkfsdfa%2Fofsfhsdlfsf
[info]                                                                                                      ^
[info]   at io.lemonlabs.uri.parsing.UrlParser.getOrThrow(UrlParser.scala:230)
[info]   at io.lemonlabs.uri.parsing.UrlParser.parseUrl(UrlParser.scala:274)
[info]   at io.lemonlabs.uri.parsing.UrlParser$.parseUrl(UrlParser.scala:325)
[info]   at io.lemonlabs.uri.Url$.parse(Uri.scala:545)
[info]   at io.lemonlabs.uri.ParsingTests.$anonfun$new$7(ParsingTests.scala:61)
[info]   at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
[info]   at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
[info]   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info]   at org.scalatest.Transformer.apply(Transformer.scala:22)
[info]   at org.scalatest.Transformer.apply(Transformer.scala:20)
[info]   ...

I'm having a look now, but maybe you know exactly where to look 😃

Unexpected Url parsing

Describe the bug
In this particular example, during the url parsing there appears an extra character:

To Reproduce

val str = """data/some__user--data!@#$%^&*()`~{}[]\|=+,?';<>;/files/my-data.tif"""
val uri = Uri(str) 
//> data/some__user--data!@#$%^&*()`~{}[]\|=+,?';<>;/files/my-data.tif:
// ^ an extra : appeared

The question is how to properly handle some URIs with extra characters? Percent encoding during the parsing step?

addParam when parameter is already present

Shouldn't this be rejected or overwritten? I see that a duplicate param is added..

Drop Scala.js 0.6

Our dependencies have all dropped Scala.js 0.6, only continuing with 1.x, so we will need to follow suit

/ & ? # are all doing the same thing if the parameter is a String

Hi,

So I have some code like this: url & s"buildId=${n.i}"
Where
url=https://teamcity.jetbrains.com/viewType.html?buildTypeId=ApacheAnt_BuildAntUsingMave n.i=1255520
and
the result is: https://teamcity.jetbrains.com/viewType.html/buildId=1255520?buildTypeId=ApacheAnt_BuildAntUsingMave

Now that's clearly not what I want. I think the problem is the combination of the implicit conversion of String to Uri and the fact that & is implemented like this:

def /(other: Uri) = merge(other)
def ?(other: Uri) = merge(other)
def `#`(other: Uri) = merge(other)
def &(other: Uri) = merge(other)

I guess the implicit conversion assumes it's a Path and then the merge will do the wrong thing. I guess an overload with String would solve the problem. What do you think?

Percent-sign followed by only one character leads to `StringIndexOutOfBoundsException` when appearing at end of component (e.g., path or query-var)

It should be raising a UriDecodeException (presumably), as it would do with an invalid % sequence, like %zz for example.

Here's a test-case:

  it should "give proper exception when percent-sign is followed by less than 2 characters" in {
    Seq("/?x=%3", "/%3", "/?a=%3&b=whatever", "/?%3=okay").foreach { part =>
      try Uri.parse("http://test.com" + part)
      catch { case _: com.netaporter.uri.decoding.UriDecodeException => /* No problem! */}
    }
  }

[feature request] Scala.js 1.0.0 support

Description
I would like to use scala-uri with Scala.js 1.0.0.

Scala.js 1.0.0 is not officially announcend, but was released to Maven Centeral and available now.
Scala.js requires artifacts published for each major version of Scala.js (0.6.x / 1.x).

Currently scala-uri is published only for Scala.js 0.6.

I think it requires time until all Scala.js users migrates to Scala.js 1.0.0, so it would be great if scala-uri cross-publishes for Scala.js 0.6/1.0.
(Even scalatest is not published for Scala.js 1.0.0 yet !).

Possibly move PublicSuffixes into a separate module?

Description
Since #75 the scala-uri JAR has gone from 456K to 1.6M and a compiled minimal Scala.js project has gone from 1.65MB to 16.3MB!

It might be worth moving this public suffixes functionality into a separate module so that users who do not need this functionality can have a scala-uri dependency with a smaller file size

Url.addParam has incorrect return type

Describe the bug

See

scala-uri/shared/src/main/scala/io/lemonlabs/uri/Uri.scala

Line 283 in 6b5bb33

def addParam[A: QueryKeyValue](a: A): Url =

The return type is Url, when it should be Self

To Reproduce

scala> import io.lemonlabs.uri.AbsoluteUrl
import io.lemonlabs.uri.AbsoluteUrl

scala> AbsoluteUrl.parse("http://github.com").addParam("a" -> "b")
res0: io.lemonlabs.uri.Url = http://github.com?a=b

res0 should be a io.lemonlabs.uri.AbsoluteUrl

Drop Scala 2.11 support

Circe may be dropping 2.11 support, which means we probably should as well: circe/circe#1176

Use Mergify for automatically merge scala-steward's PRs?

Description
Mergify is often used for projects that use scala-steward.

Install Mergify app and write .mergify.yml like this

pull_request_rules:
  - name: automatically merge scala-steward's PRs
    conditions:
      - author=scala-steward
      - status-success=continuous-integration/travis-ci/pr      
      - body~=labels:.*semver-patch
    actions:
      merge:
        method: merge

If the CI is successful and the update is semver-patch, the scala-steward PR will be merged automatically.

Missing maven release v0.5.0 for Scala 2.12

As it is documented, version 0.5.0 should be used for Scala 2.12, although only 0.4.17 and 0.4.16 are available in Maven repository: https://mvnrepository.com/artifact/io.lemonlabs/scala-uri_2.12

Including scala-uri your project

scala-uri 0.5.x is currently built with support for scala 2.12.x, 2.11.x
Release builds are available in maven central. For SBT users just add the following dependency:

"io.lemonlabs" %% "scala-uri" % "0.5.0"

Can you please fix this by publishing this version there?
Thanks.

support for data scheme

Description
It would be great if it would support data:// urls (https://en.wikipedia.org/wiki/Data_URI_scheme)

Example

val dataUrl = Url.parse("data:text/vnd-example+xyz;foo=bar;base64,R0lGODdh")
val mediaType: List[(String, String)] = dataUrl.mediaType
val rawContent: Array[Byte] = dataUrl.rawData
// and maybe something more structured such as:
val myDataType = dataUrl.dataAsJson[ExpectedPayloadType]

Easy way to omit parameter when using ?

I found the 0.5.7 behaviour of omitting the query parameter altogether if the value was None very useful.

Code Example 0.5.7:

scala> import com.netaporter.uri.dsl._
import com.netaporter.uri.dsl._

scala> val optional: Option[String] = None
optional: Option[String] = None

scala> "foo" ? ("optional" -> optional)
res0: com.netaporter.uri.Uri = foo

Would it be possible to revert to the 0.5.7 behaviour or add it as an alternative?

With 1.0.0 it seems I'd have to write the following ugly code to omit the parameter when the value is None:

val optional: Option[String] = ???
val url = optional.fold("http://example.com/foo" ? ("bar" -> bar))(_ => "http://example.com/foo" ? ("bar" -> bar) & ("optional" -> optional)))

Play interprets a query parameter without a value as having a value of the empty string, and the URL I'm generating is being passed to a Play app.

When I discovered that scala-uri omitted the parameter altogether when the value was None it was a real 😍 moment so I'd love to have this feature back.

lemonlabsuk / scala-uri Goto Github PK

scala-uri's People

Contributors

Stargazers

Watchers

Forkers

scala-uri's Issues

Describe the bug

To Reproduce

Easy way

Recommend Projects

Recommend Topics

Recommend Org