Introduce support for nested properties about zio-config HOT 17 CLOSED

@jdegoes
When it comes to nested structure, should write back functionality be considering it writing back to nested structure ?

On the append and zero usage to flatten it back to map, does that consider when the value is list?

from zio-config.

This looks really interesting, and hopefully we can have a PR soon to discuss further.

from zio-config.

@jdegoes
would you mind helping us on integrating it with ConfigSource?

from zio-config.

The simplest possible modification to ConfigSource:

def getConfigValue(path: String): IO[ReadError, PropertyTree[String, String]]

Runner-up:

def getConfigValue(paths: List[String]): IO[ReadError, String]

Both have enough structure to read nested configs. The latter is a little more "elemental".

We'll have to modify some other places to support nesting when defining properties, etc.

from zio-config.

The only immediate issue that I could find is:
Getting a map of entire system env may go through security management exception.
while system.env(path) returning Option may still be successful.

This means forming a sys env to Property[String, string] (or even map[String, String]) can be a bit of trouble?

As of now we use zio.system.env and that works well. it either give a ReadError (fatal error/ missing value) or the actual value , given a path.

from zio-config.

Or is it that, an instance of PropertyTree[String, String] is valid only for 1 path?

EDIT: I got it when I had a look at the proposal in detail today. Thanks @jdegoes

from zio-config.

What could be a proper user interface to describe a nested config ?

A very verbose user interface, but really flexible: (A less verbose is given in next comment)

    case class Database(port: Int, dbUrl: String)
    case class Prod(ldap: String, database: Database)
 
  val config  
     (nested("auth") >~> string("ldap") |@| 
      nested("database") >~> 
       (int("PORT") |@| string("DB_URL"))(Database.apply, Database.unapply)
   )(Prod.apply, Prod.unapply)

// Given a valid hoccon source.
// Ex: 
  {
   "auth" : { "ldap" : "sample1" } 
   "database": {
        "PORT" : "sample2",
        "DB_URL": "sample3"
     }
}

val result = read(config).provide(hocconSource) === Prod(...)

write(config, result) == 

List(
   PropertyTree.Record(
      "auth", 
       Map("ldap" -> PropertyType.Leaf("sample1")),
   PropertyTree.Record(
     "database",
      Map("PORT" -> PropertyType.Leaf("sample2"), "DB_URL" -> PropertyType.Lead("sample3")
   )
)

This allows user to write a valid hoccon, that could be read by some other application expecting the hoccon source.

from zio-config.

An alternative approach to describing such a config is:

  string(List("auth", "ldap")) |@| (int(List("database", "PORT")) |@| string(List("database", "DB_URL")))......

// Pretty much same as   string("auth.ldap") |@| (int("database.PORT")) |@| string("database.DB_URL"))).....

In this case, Writing it back to a PropertyTree (hoccon, for instance) can turn out to be a bit fragile?

from zio-config.

@jdegoes What I am trying to say with the above example is, the config descriptor itself clearly indicates, we are talking about reading from something like a hoccon, yaml etc. Or in other words, only those sources will be able to provide use with proper PropertyTree(mentioned in your comment) that can serve the config descriptor. Does it make sense? It's still initial thinking for me

from zio-config.

In both approaches, it will be PropertyTree that's our initial structure.

from zio-config.

You need something like:

nested("foo") { /* inside here is a ConfigDescriptor */ } |@| ...

The type will be:

def nested[A](path: String)(cd: ConfigDescriptor[A]): ConfigDescriptor[A]

It will require a new case in the ConfigDescriptor enumeration.

from zio-config.

By the way, we can do nested even with flat properties.

I used to see this format all the time with Java properties:

application.settings.connectionUrl = ...
application.settings.backupUrl = ...

With environmental variables you sometimes see JAVA_ENV, JAVA_HOME, etc.

Because ConfigDescriptor is just data, we can translate the nesting into any format we want.

from zio-config.

The only immediate issue that I could find is:
Getting a map of entire system env may go through security management exception.
while system.env(path) returning Option may still be successful.

This implies ConfigSource should not support enumeration, e.g. be like:

def getConfigValue(paths: List[String]): IO[ReadError, String]

from zio-config.

@jdegoes
Sure. In any case, it seems we need a new node Nested in ConfigDescriptor. Is that reasonable?

from zio-config.

@afsalthaj Yes, that's a great & safe start, since we know we'll need it and what the type should be.

from zio-config.

Whenever program encounters Nested, it accumulates the path which is then used to go to deep into the PropertyTree, to fetch the final Leaf.

Will need to start scratching with the code now.

from zio-config.

Fixed in #78

from zio-config.