cies / htoml Goto Github PK

View Code? Open in Web Editor NEW

38.0 3.0 13.0 326 KB

TOML file format parser in Haskell

Home Page: https://hackage.haskell.org/package/htoml

License: Other

Haskell 5.17% HTML 94.77% Shell 0.05%

stackage aeson haskell parsec parser toml ghc hackage json

htoml's Introduction

htoml

A TOML parser library in Haskell.

TOML is the obvious, minimal configuration language by Tom Preston-Werner. It is an alternative to the XML, YAML and INI formats mainly for the purpose of configuration files. Many will find that XML and YAML are too heavy for the purpose of configuration files prupose while INI is underspecified. TOML is to configuration files, like what Markdown is for rich-text.

This library aims to be compatible with the latest version of the TOML spec. Compatibility between htoml version and TOML (as proven by BurntSushi's language agnostic TOML test suite) is as follows:

TOML v0.4.0 is implemented by htoml >= 1.0.0.0
(currently only one item in this mapping, more will follow)

Documentation

Apart from this README, documentation for this package may (or may not) be found on Hackage.

Quick start

Installing htoml is easy. Either by using Stack (recommended):

stack install htoml

Or by using Cabal:

cabal install htoml

In order to make your project depend on it you can add it as a dependency in your project's .cabal file, and since it is not yet on Stackage you will also have to add it to the extra-deps section of your stack.yaml file when using Stack.

To quickly show some features of htoml we use Stack to start a GHCi-based REPL. It picks up configuration from the .ghci file in the root of the repository.

git clone https://github.com/cies/htoml.git
cd htoml
stack init
stack --install-ghc ghci

Add a --resolver flag to the stack init command to specify a specific package snapshot, e.g.: --resolver lts-4.1.

In case you have missing dependencies (possibly file-embed), they can be added to the extra-deps in stack.yaml automatically with:

stack solver --update-config

We can now start exploring htoml from a GHCi REPL. From the root of this repository run:

stack ghci

Now read a .toml file from the benchmark suite, with:

txt <- readFile "benchmarks/example.toml"
let r = parseTomlDoc "" txt
r

...which prints:

Right (fromList [("database",VTable (fromList [("enabled",VBoolean True),("po [...]

Then convert it to Aeson (JSON), with:

let Right toml = r
toJSON toml

...which prints:

Object (fromList [("database",Object (fromList [("enabled",Bool True),("po [...]

Finally trigger a parse error, with:

let Left err = parseTomlDoc "" "== invalid toml =="
err

...it errors out (as it should), showing:

(line 1, column 1):
unexpected '='
expecting "#", "\n", "\r\n", letter or digit, "_", "-", "\"", "'", "[" or end of input

Note: Some of the above outputs are truncated, indicated by [...].

How to pull data from a TOML file after parsing it

Once you have sucessfully parsed a TOML file you most likely want to pull some piecces of data out of the resulting data structure.

To do so you have two main options. The first is to use pattern matching. For example let's consider the following parseResult:

Right (fromList [("server",VTable (fromList [("enabled",VBoolean True)] ) )] )

Which could be pattern matched with:

case parseResult of
  Left  _ -> "Could not parse file"
  Right m -> case m ! "server" of
    VTable mm -> case mm ! "enabled" of
      VBoolean b -> "Server is " ++ (if b then "enabled" else "disabled")
      _ -> "Could not parse server status (Boolean)"
    _ -> "TOML file does not contain the 'server' key"

The second main option is to use the toJSON function to transform the data to an Aeson data structure, after which you can use your Aeson toolbelt to tackle the problem. Since TOML is intended to be a close cousin of JSON this is a very practical approach.

Other ways to pull data from a parsed TOML document will most likely exist; possible using the lens library as documented here.

Compatibility

Currently we are testing against several versions of GHC with Travis CI as defined in the env section of our .travis.yml. lts-2 implies GHC 7.8.4, lts-3 implies GHC 7.10.2, lts-4/lts-5 imply GHC 7.10.3, and nightly is build with a regularly updated version of GHC.

Version contraints of `htoml`'s dependencies

If you encounter any problems because htoml's dependecies are constrained either too much or too little, please file a issue for that. Or off even better submit a PR.

Tests and benchmarks

Tests are build and run with:

stack test

BurntSushi's language agnostic test suite is embedded in the test suite executable. Using a shell script (that lives in test/BurntSushi) the latest tests can be fetched from its Github repository.

The benchmarks, that use the amazing criterion library, are build and run with:

stack build :benchmarks

Contributions

Most welcome! Please raise issues, start discussions, give comments or submit pull-requests. This is one of the first Haskell libraries I wrote, feedback is much appreciated.

Features

Compatibility to the TOML spec is proven by an extensive test suite
Incorporates BurntSushi's language agnostic test suite
Has an internal representation that easily maps to JSON
Provides an Aeson-style JSON interface (suggested by Greg Weber)
Useful error messages (thanks to using Parsec over Attoparsec)
Understands arrays as described in this issue
Fails on mix-type arrays (as per spec)
Comes with a benchmark suite to make performance gains/regressions measurable
Tries to be well documented (please raise an issue if you find documentation lacking)
Available on Stackage (see top of this README for badges indicating TOMLs inclusion in Stackage status)

Todo

More documentation and start to use the proper Haddock idioms
Add property tests with QuickCheck (the internet says it's possible for parsers)
Extensively test error cases (probably improving error reporting along the way)
See how lenses may (or may not) fit into this package, or an additional package
Consider moving to one of the more modern parser combinators in Haskell (megaparsec maybe?) -- possibly wait until a clear winner shows

Do you see todo that looks like fun thing to implement and you can spare the time? Please knoe that PRs are welcome :)

Acknowledgements

Originally this project started off by improving the toml package by Spiros Eliopoulos.

HuwCampbell helped a lot by making tests pass and implementing "explicitness tracking" in Parsec's parser state.

Copyright and licensing

This package includes BurntSushi's language agnostic TOML tests, which are WTFPL licensed.

The TOML examples that are used as part of the benchmarks are copied from Tom Preston-Werner's TOML spec which is MIT licensed.

For all other files in this project the copyrights are specified in the htoml.cabal file, they are distributed under the BSD3 license as found in the LICENSE file.

htoml's People

Contributors

Stargazers

Watchers

Forkers

erebe huwcampbell david-hoze tmcgilchrist gurkenglas erikd barreyo triggernz glimming expectocode notbrianzach mirokuratczyk

htoml's Issues

fails to install with stackage lts-3.0 / ghc 7.10

This package looks useful, thanks. stack install failed for me:

$ stack install htoml 
unbounded-delays-0.1.0.9: configure
logict-0.6.0.2: download
regex-tdfa-rc-1.1.8.3: download
unbounded-delays-0.1.0.9: build
logict-0.6.0.2: configure
logict-0.6.0.2: build
regex-tdfa-rc-1.1.8.3: configure
unbounded-delays-0.1.0.9: install
regex-tdfa-rc-1.1.8.3: build
logict-0.6.0.2: install
smallcheck-1.1.1: download
smallcheck-1.1.1: configure
smallcheck-1.1.1: build
smallcheck-1.1.1: install
regex-tdfa-rc-1.1.8.3: install
tasty-0.10.1.2: download
tasty-0.10.1.2: configure
tasty-0.10.1.2: build
tasty-0.10.1.2: install
tasty-hunit-0.9.2: download
tasty-hunit-0.9.2: configure
tasty-quickcheck-0.8.3.2: download
tasty-smallcheck-0.8.0.1: download
tasty-hunit-0.9.2: build
tasty-quickcheck-0.8.3.2: configure
tasty-hunit-0.9.2: install
tasty-quickcheck-0.8.3.2: build
tasty-smallcheck-0.8.0.1: configure
tasty-smallcheck-0.8.0.1: build
tasty-quickcheck-0.8.3.2: install
tasty-smallcheck-0.8.0.1: install
tasty-hspec-1.1: download
tasty-hspec-1.1: configure
tasty-hspec-1.1: build
tasty-hspec-1.1: install
htoml-0.1.0.2: download
htoml-0.1.0.2: configure
htoml-0.1.0.2: build
Completed all 10 actions.

--  While building package htoml-0.1.0.2 using:
      /Users/simon/.stack/setup-exe-cache/setup-Simple-Cabal-1.22.4.0-x86_64-osx-ghc-7.10.2 --builddir=.stack-work/dist/x86_64-osx/Cabal-1.22.4.0/ build --ghc-options  -ddump-hi -ddump-to-file
    Process exited with code: ExitFailure 1
    Logs have been written to: /Users/simon/src/hledger/.stack-work/logs/htoml-0.1.0.2.log

    Configuring htoml-0.1.0.2...
    Building htoml-0.1.0.2...
    Preprocessing library htoml-0.1.0.2...
    [1 of 3] Compiling Text.Toml.Types  ( src/Text/Toml/Types.hs, .stack-work/dist/x86_64-osx/Cabal-1.22.4.0/build/Text/Toml/Types.o )
    [2 of 3] Compiling Text.Toml.Parser ( src/Text/Toml/Parser.hs, .stack-work/dist/x86_64-osx/Cabal-1.22.4.0/build/Text/Toml/Parser.o )

    src/Text/Toml/Parser.hs:83:5:
        Non type-variable argument in the constraint: Stream s m Char
        (Use FlexibleContexts to permit this)
        When checking that ‘twoChar’ has the inferred type
          twoChar :: forall s u (m :: * -> *).
                     Stream s m Char =>
                     Char -> ParsecT s u m [Char]
        In an equation for ‘tableArrayHeader’:
            tableArrayHeader
              = between (twoChar '[') (twoChar ']') headerValue
              where
                  twoChar c = count 2 (char c)

    src/Text/Toml/Parser.hs:177:25:
        Couldn't match expected type ‘time-1.5.0.1:Data.Time.Format.Locale.TimeLocale’
                    with actual type ‘System.Locale.TimeLocale’
        NB: ‘time-1.5.0.1:Data.Time.Format.Locale.TimeLocale’
              is defined in ‘Data.Time.Format.Locale’ in package ‘time-1.5.0.1’
            ‘System.Locale.TimeLocale’
              is defined in ‘System.Locale’ in package ‘old-locale-1.0.0.7’
        In the first argument of ‘parseTime’, namely ‘defaultTimeLocale’
        In the expression:
          parseTime defaultTimeLocale (iso8601DateFormat $ Just "%X") d

fails to build with aeson-2.0

htoml                          > Building library for htoml-1.0.0.3..
htoml                          > [1 of 3] Compiling Text.Toml.Types
htoml                          > 
htoml                          > /tmp/stack-89f3a8327c6d452d/htoml-1.0.0.3/src/Text/Toml/Types.hs:193:23: error:
htoml                          >     • Couldn't match type: HashMap Text Value
htoml                          >                      with: Data.Aeson.KeyMap.KeyMap Value
htoml                          >       Expected: HashMap Text v -> Object
htoml                          >         Actual: HashMap Text v -> HashMap Text Value
htoml                          >     • In the second argument of ‘(.)’, namely ‘M.map toBsJSON’
htoml                          >       In the expression: Object . M.map toBsJSON
htoml                          >       In an equation for ‘toBsJSON’: toBsJSON = Object . M.map toBsJSON
htoml                          >     |
htoml                          > 193 |   toBsJSON = Object . M.map toBsJSON
htoml                          >     |                       ^^^^^^^^^^^^^^

Bug with nested tables and arrays

[[fruit.blah]]
  name = "apple"

  [fruit.blah.physical]
    color = "red"
    shape = "round"

[[fruit.blah]]
  name = "banana"

  [fruit.blah.physical]
    color = "yellow"
    shape = "bent"

From my reading of the toml spec, that should work, and give rise to the following JSON:

{  
   "fruit":{  
      "blah":[  
         {  
            "name":"apple",
            "physical":{  
               "color":"red",
               "shape":"round"
            }
         },
         {  
            "name":"banana",
            "physical":{  
               "color":"yellow",
               "shape":"bent"
            }
         }
      ]
   }
}

And tomlv accepts it:

$ tomlv -types fruit.toml                                                                                                                         ~
    fruit.blah                         ArrayHash
        fruit.blah.name                String
        fruit.blah.physical            Hash
            fruit.blah.physical.color  String
            fruit.blah.physical.shape  String
    fruit.blah                         ArrayHash
        fruit.blah.name                String
        fruit.blah.physical            Hash
            fruit.blah.physical.color  String
            fruit.blah.physical.shape  String

However attempting to parse with htoml gives Cannot redefine table ('fruit, blah, physical'.

Validation

How is one suppose to validate the toml file? Maybe, as recommendation, there is a library that goes well with htoml?

Floats with only exponent won't parse

Just looking at the code, your floats require a decimal point ( . character), while TOML accepts 3e9 as a valid float.

Suggestion for Node's data constructors

The Value data type is currently defined:

data Node = VTable    Table
          | VTArray   [Table]
          | VString   !Text
          | VInteger  !Int64
          | VFloat    !Double
          | VBoolean  !Bool
          | VDatetime !UTCTime
          | VArray    [Node]

The bangs are inconsistent. I would recommend putting them on all of them, as aeson does. Also, like in aeson, I would recommend changing VArray and VTArray to use Vectors instead of lists. This tends to be more performant, although I don't have any benchmarks to verify this.

Out of date

The latest version of TOML is now 4.0, not 3.1 as the README states.

Lenses

https://github.com/ambiata/x/blob/master/x-htoml/src/X/Text/Toml.hs

Write TOML to file

It would be handy to have a way to serialize an existing Table.

Why is there "tests" executable in addition to a test suite?

Because of that depending on htoml brings test-related libraries like hspec and tasty into clients' dependency trees.

Also plain cabal/stack install htoml populates your path with a confusing tests executable.

Changes to the Types, and therefor the public API

The new Node data structure as by @HuwCampbell's PR looks like this:

data Node = VTable    Table
          | VITable   Table
          | VTArray   [Table]
          | VString   Text
          | VInteger  Int64
          | VFloat    Double
          | VBoolean  Bool
          | VDatetime UTCTime
          | VArray    [Node]

While the old data struct that I came up with, is actually a combination of two data types, and looks like:

data Node = NTValue TValue
          | NTable  Table
          | NTArray [Table]
-- [...]
data TValue = VString   Text
            | VInteger  Int64
            | VFloat    Double
            | VBoolean  Bool
            | VDatetime UTCTime
            | VArray    [TValue]

Now my main questions regarding the new data structure are:

What is a VITable, how is it different from a VTable, and why do we need it?
The VArray can now contain anything of type Node; I guess in my old data structure I tried to make sure that VArrays can only contain values (TValue). I do not know if missing this bit of type safety is a real issue (I guess not). And I also wonder why we still need a difference between VTArray and VArray; but imagining how had it will be for modify the code in that direction hints that it is probably better this way.

Please know that this is one of my first proper Haskell project, so any feedback is welcome.

If you (any reader, but @HuwCampbell in particular) have any ideas on what could be improved to get this lib to higher standard please let me know (I'd like to push for inclusion in Stackage after I got some more feedback).

Cheers!