hpack does not seem to support the cc-options field for passing options to the c compiler:

cueball:/vol/hosts/cueball/workspace/projects/unicode-transforms$ stack build
WARNING: Ignoring unknown field "cc-options" in library section
WARNING: Ignoring unknown field "cc-options" in library section

At least gitlab and bitbucket shortcuts are needed, together with a mirror of source-repository stanza with type-location-... fields.

Is it possible to specify build-type: Custom somehow? I tried adding an appropriate entry to my package.yaml file, but hpack complained about an unsupported entry.

I haven't had a chance to get all the way to the root of this yet, but I'm logging this here so I don't forget it.

The copy of hpack that's built into stack exhibits a bug that doesn't occur with a vanilla hpack installation, namely displaying:

WARNING: Ignoring unknown field "c-sources" in library section

when building a library with a c-sources field.

I traced the difference to hpack defining

extra-deps:
- aeson-0.11.0.0

in its stack.yaml, which is not defined in Stack's stack.yaml. When I add that to Stack's stack.yaml the problem disappears.

/cc @mgsloan

https://github.com/toml-lang/toml

Any interest in adding support for accepting either package.yaml or package.toml?

It's a little more verbose than yaml (strings require quotes) but some people may prefer it. Example package.toml:

name       = "hpack"
version    = "0.15.0"
synopsis   = "An alternative format for Haskell packages"
maintainer = "Simon Hengel <[email protected]>"
license    = "MIT"
github     = "sol/hpack"
category   = "Development"

ghc-options = "-Wall"

dependencies = [
  "base >= 4.7 && < 5",
  "base-compat >= 0.8",
  "deepseq",
  "directory",
  "filepath",
  "Glob",
  "htoml",
  "text",
  "containers",
  "unordered-containers",
  "yaml",
  "aeson >= 0.11",
]

[library]
source-dirs = "src"
exposed-modules = [
  "Hpack",
  "Hpack.Config",
  "Hpack.Run",
  "Hpack.Yaml",
]

[executables.hpack]
main = "Main.hs"
source-dirs = "driver"
dependencies = ["hpack"]

[tests.spec]
cpp-options = "-DTEST"
main        = "Spec.hs"
source-dirs = ["test", "src"]
dependencies = [
  "hspec == 2.*",
  "QuickCheck",
  "temporary",
  "mockery >= 0.3",
  "interpolate",
  "aeson-qq",
]

From a package.yaml file like this:

ghc-options: -Wall -fno-warn-name-shadowing

dependencies:
  - base == 4.*
  - generics-sop
  - safe

library: {}

tests:
  spec:
    main: "Spec.hs"
    ghc-options: "-threaded -O0 -pgmL markdown-unlit"
    source-dirs:
      - test
      - examples

    dependencies:
      - getopt-generics
      - hspec
      - hspec-expectations
      - silently

I would expect the test-suite section of the generated cabal file to contain a ghc-options field with the value -Wall -fno-warn-name-shadowing -threaded -O0 -pgmL markdown-unlit but it has none.

When a field that takes a directory contains a dot, hpack's renderer will generate a line with nothing but a . in it. However, this is invalid in a cabal file, as dots on their single lines are meant to express empty-lines (I think, I can't find the documentation the source that does it is here).

A simple example of the bug is here: https://gist.github.com/0cc7f8f8e2cbc9ee7bbf82876a822e28

In there, source-dirs: . generates an invalid cabal file. Though this specific case isn't super common, without specifying source-dirs: ., hpack won't include the other-modules section for the modules in the current directory (I'd consider that another ticket).

My idea for a fix would be to output:

field: .
  , other-value
  , other-value

Whenever . is a value.

That would make codebase a bit more robust.

  2) Hpack.Config.getModules, when source directory is './.', ignores Setup
       expected: ["Foo"]
        but got: ["Foo","Setup"]

this because other dir in path is /private/var/... and other /var/... - i.e. src == dir check isn't enough. One would need to follow links

$ ls -l /
...
lrwxr-xr-x@   1 root  wheel    11 Oct 23  2014 var -> private/var

I have been using template haskell to create configuration files, and including those files in the data-files of a cabal package.

Hpack seems to check whether files exist before including them in the generated cabal file, which means I have to create stub files to get Hpack to work. This is annoying because I have to commit this stub to source control, and be sure to not commit generated files.

In the case where glob patterns are not used, I think the fix is as simple as not excluding a non-existant data file from the generated .cabal file.

I would like to be able to generate files of arbitrary name in template haskell, and have a glob pattern in the package.yaml file pick up each of those generated files, but I suspect this is impractical.

It would be helpful if hpack had shown anything when run from the command line, even if it's just Generating foo.cabal from package.yml or a usage string. The current behavior is confusing:

% hpack
% hpack --help
% hpack help
%

The overall idea is to have stack detect hpack files and automatically generate cabal files when the hpack files change. I'm not sure if everyone will be on board with the idea, but it seems like a good idea to me, lets discuss!

In commercialhaskell/stack#1568 , I lay out an approach to having multiple stack configurations inform wide version bounds for dependencies. One reason that I'm keen on hpack is that the stage where .cabal files are generated would be a great time to insert these version constraints.

I'm tempted to implement else-branches to conditional, e.g. accept something like the following, in addition to the current syntax:

when:
  - condition: os(windows)
    then:
      dependencies: Win32
    else:
      dependencies: unix

I think this could be nicer than navigating the README, for example when looking up the benchmarks syntax or more exotic features.

That would require setting cabal-version: >= 1.21

My license files typically have an extension. hpack only looks for a file called LICENSE with no extension. I tried to manually set the license-file, but that is an unknown field. I would like to be able to manually set the license file. It would be nice if license files with extensions could automatically be found.

Several major features (flags, conditionals) have been in master for months now, but to use them you have to reverse-engineer from source code or issue discussion. hpack's lack of docs is holding it back.

We still need to settle on some concrete syntax here. Ideas?

I can't manage to create a package.yaml that allows to add exposed-modules depending on a cabal flag. This is roughly what I tried:

  when:
  - condition: flag(with-servant-aeson-specs)
    dependencies:
    - servant-aeson-specs ==0.2.* || ==0.3.* || ==0.4.*
    exposed-modules:
    - Servant.MatrixParam.AesonSpecs

But I get the following message from hpack:

WARNING: Ignoring unknown field "exposed-modules" in library section
servant-matrix-param.cabal is up-to-date

And the cabal file is missing the exposed-modules.

Is there a way to do this with hpack?

(Here's the relevant project, if that helps: https://github.com/plow-technologies/servant-matrix-param)

We need to:

1. adapt hpack to allow the latest version of aeson
2. add upper bounds for aeson to all previous versions of hpack on Hackage

Currently, hpack complains:

package.yaml: when expecting a String, encountered Number instead

It seems impossible to specify a build tool with hpack. Am I missing something?

While it is clever to infer the package name from the directory, it causes issues when hpack files are present in hackage tarballs: commercialhaskell/stack#2254

I haven't tested, but I believe this will even be an issue when you do stack unpack on the package, as the unpacked dir defaults to having the package name as well as the package version.

This is a bit of a hack idea, but I think it could work well:

1. Based on source directories, it would discover haskell modules using the existing hpack logic. It'd also guess that the project name is the name of the current dir.

2. Generate an hpack file with these fields filled in, but just a dependency on base.

3. Iteratively run the compiler and parse out import resolution failures. Find the package in some Map ModuleName [PackageName], and ask the user when there's ambiguity.

The module name map could come from the stackage snapshot you've picked, or some other source like the latest hackage. For stack integration, the fanciest thing would be to fall back on looking it up in hackage, and add extra-deps as necessary.

It'd also be quite fancy if this could update an existing hpack file, perhaps based on errors the user encounters in their editor's ghci session

As in subject. We can do this fairly easily so I wonder if it makes sense for hpack to do so.

I think cabal warns itself later so if hpack is going fairly strictly for just translating yaml -> cabal then it's not that important. OTOH hpack already warns on missing source dirs…

GHCJS uses ghcjs-options, in a similar manner to ghc-options.

See https://www.reddit.com/r/haskell/comments/54knub/ghcjs_dedupe/d83vetj.

Perhaps it makes sense to be able to specify multiple packages in one hpack file? Then, common dependencies and flags can be factored out.

The only downside I can see to this is that it makes it easier to inadvertently add unused dependencies. Ideally, these'd be detected (commercialhaskell/stack#39)

So, this'd mean you'd potentially end up with a packages.yaml next to your `stack.yaml. Alternatively, I could even imagine stack supporting inline hpack in its packages list (related to #61). Not sure the complexity is worth it, or whether it'd be a positive effect on stack.yaml - just a thought!

Hackage requires a description for every package. If description is not specified, we could fallback to synopsis.

Something like this in hpack.yaml description field:

a
.
b

Gets collapsed to this in .cabal:
a . b

So everything becomes a single line. Is there something that I am missing or that's how it works?

I'd like to write a multi-line description in the yaml file, with long lines broken for readability. Ideally the line breaks would be preserved in the cabal file, for the same reason. In the yaml file I tried description: (complains if the text contains :), description: > and description: |. The last seemed to work best, but I think it is adding cabal's . lines for every line break; I think it should add these only for blank lines (paragraph separators).

I don't know if it's possible to preserve line breaks all the way through to the cabalish-haddock rendering on hackage - probably that's not desirable.

Complementing #106

My use case for this is that when running hpack from a script, it would be best to call it as:

hpack -i input.yaml -o output.cabal

Instead of having to cd to the directory where the input.yaml file is located, and having to “guess” the name of the output cabal file.

I am using stack and because of #104 I cannot keep a package.yaml. I want to be able to generate the cabal file from an arbitrary file say hpack.yaml. Otherwise I have to keep renaming to and from package.yaml to avoid #104. For such cases it will be useful to have a command line option to specify an input file such that:

hpack -f hpack.yaml

or read from stdin

hpack < hpack.yaml

Hpack looks really nice, great work! I think a good step towards adoption would be to support automatic conversion of cabal files to hpack format. This could be done in a new package, hpack-convert, or something, to avoid a dependency on Cabal in the main hpack library (maybe that dependency would be ok, though?).

say my library is "foo-driver"

i get Paths_foo-driver listed in the other-modules section

it should be Paths_foo_driver.

There were some threads about introducing a -XHaskell2016 or -XGlasgow2016, or perhaps a new language.

https://www.reddit.com/r/haskell/comments/4fsuvu/can_we_have_xhaskell2016_which_turns_on_the_most/
https://www.reddit.com/r/haskell/comments/4ggp8z/summary_of_the_xhaskell2016_feedback/

hpack seems like a great place to experiment with this. What if something like this was added as a feature of hpack? To avoid confusion and make it clear it's an experiment, could have something like language: hpack2016.

IMHO, ideally we'd carve out something that can be the seed of a new standard, and so should only include extensions that we expect to make it in to the standard. I think the following are good guidelines:

• Widely used with few unexpected consequences. We can use the poll results for this
• Avoid things that have a bunch of complexity, particularly if there are known unresolved loose ends. Avoid complex things that haven't been around for a long time and are fully proven.
• Avoid extensions that can cause some code to not compile when enabled. I have accumulated a list of these in the stack ghci code, to inform the user when they are loading multiple packages that have potentially conflicting extensions: https://github.com/commercialhaskell/stack/blob/master/src/Stack/Ghci.hs#L408

Here are the most popular "safe" extensions:

LambdaCase (71)
GADTSyntax (70)
RankNTypes (70)
ScopedTypeVariables (69)
DeriveGeneric (66)
TupleSections (59)
BangPatterns (59)
MultiParamTypeClasses (56)
FlexibleInstances (33)
FlexibleContexts (31)
MultiWayIf (31)
TypeFamilies, TypeOperators (29)
DuplicateRecordFields (17)
FunctionalDependencies (15)
DisambiguateRecordFields (14)
MonadComprehensions (12)
BinaryLiterals (11)
RecursiveDo (10)
Other safe extensions (there was no feedback against having them on by default):
ParallelListComp (7)
PartialTypeSignatures (6)
TypeApplications (4)
RecordWildCards (4)
PatternSynonyms (4)
EmptyCase (3)
InstanceSigs (3)
KindSignatures (3)

I would exclude the following:

• TypeApplications - too new
• DisambiguateRecordFields + DuplicateRecordFields - too new
• PatternSynonyms - too new
• TypeFamilies - as much as I like them, they seem too new. At once, they are popular and effective.
• KindSignatures - perhaps this is too stringent, but it seemed to me like we aren't really sure what the kind syntax will be (* vs Type). Has this been decided for sure?

Not sure:

• MonadComprehensions - seems like it could cause type ambiguity for code copied from somewhere that doesn't expect it.
• PartialTypeSignatures is new, but this is mostly a dev feature so it seems reasonable for its behavior to change a bit. Seems like something we will for sure want.
• DeriveGeneric - I'm not sure we want to codify this just yet. This one is really popular, though
• RankNTypes - complex to support
• RecordWildCards - it's nice at times, but not sure we want to codify something that makes local scope resolution require datatype reification.

So, without the ones I want to exclude and the ones I'm not so sure about, we get:

LambdaCase, GADTSyntax, ScopedTypeVariables, TupleSections, BangPatterns, MultiParamTypeClasses,
FlexibleInstances, FlexibleContexts, MultiWayIf, TypeOperators, FunctionalDependencies,
BinaryLiterals, RecursiveDo, ParallelListComp, PartialTypeSignatures, EmptyCase, InstanceSigs

hpack doesn't recognise extensions: or tested-with:. Also it should allow benchmarks to be specified (just like tests).

Just trying out this tool.. loving it so far!

With source-dirs: . , Setup.hs is included in exposed-modules which is undesirable.

Can we change

Maybe (HashMap String (CaptureUnknownFields (Section ExecutableSection)))

to

Maybe (Map String (CaptureUnknownFields (Section ExecutableSection)))

or alternatively traverse them in name order. Time from time ordering of multiple tests changes. I guess I'm unlucky, I got a hashclash.

(probably using ci.appveyor.com)

@phadej Are you interested to look into this?

Hpack does not seem to support the data-dir cabal option. Is there a reason for this? If not I would be happy to contribute a fix.

I've revised existing versions: http://hackage.haskell.org/package/hpack/revisions/

A new release is needed if you wish to restore backwards compatibility, otherwise I recommend adding a base >= 4.7 lower bound.

Citing from http://hydra.cryp.to/build/1306557/nixlog/2/raw:

Failures:

  test/Hpack/ConfigSpec.hs:64:
  1) Hpack.Config.parseJSON, when parsing a Dependency, when parsing fails, returns an error message
       expected: Left "Error in $: expected String or an Object, encountered Number"
        but got: Left "when expecting a String or an Object, encountered Number instead"

  test/Hpack/ConfigSpec.hs:69:
  2) Hpack.Config.parseJSON, when parsing a Dependency, when parsing fails, when ref is missing, produces accurate error messages
       expected: Left "Error in $: key \"ref\" not present"
        but got: Left "key \"ref\" not present"

  test/Hpack/ConfigSpec.hs:78:
  3) Hpack.Config.parseJSON, when parsing a Dependency, when parsing fails, when both git and github are missing, produces accurate error messages
       expected: Left "Error in $: neither key \"git\" nor key \"github\" present"
        but got: Left "neither key \"git\" nor key \"github\" present"

  test/Hpack/ConfigSpec.hs:643:
  4) Hpack.Config.readPackageConfig, when package.yaml is invalid, returns an error
       expected: Left "package.yaml: Error in $.executables.foo: failed to parse field executables: The key \"main\" was not found"
        but got: Left "package.yaml: The key \"main\" was not found"

  test/Hpack/UtilSpec.hs:71:
  5) Hpack.Util.List, when parsing single values, returns error messages from element parsing
       expected: Left "Error in $: neither key \"git\" nor key \"github\" present"
        but got: Left "neither key \"git\" nor key \"github\" present"

  test/Hpack/UtilSpec.hs:78:
  6) Hpack.Util.List, when parsing a list of values, propagates parse error messages of invalid elements
       expected: Left "Error in $[1]: neither key \"git\" nor key \"github\" present"
        but got: Left "neither key \"git\" nor key \"github\" present"

Subject says it all. 😄

A complete build log is at http://hydra.cryp.to/build/785838/log/raw.

I would love to be able to specify version numbers using npm's advanced range syntax. Stuff like 1.2.x, ~1.2.3, and ^1.2.3. I think the semver-range package can represent them, but I haven't ever played with it.

Some package.yaml file use e.g. driver, src as the value for source-dirs. This is not a valid yaml list value.

It still works most of the time because the field value gets inserted without quotes into the cabal file. For example: the package.yaml of this repo.

Should hpack warn about non-existing source directories?

The ghc-options line in cabal becomes too long if there are many options specified. It will help readability if the line is split into multiple lines whenever it gets longer than 80 chars.