ocaml / ocamlfind Goto Github PK

View Code? Open in Web Editor NEW

27.0 9.0 29.0 567 KB

The OCaml findlib library manager

License: Other

Makefile 4.99% Shell 3.37% OCaml 90.68% OpenEdge ABL 0.90% Standard ML 0.06%

library-management-system ocaml

ocamlfind's Introduction

Findlib

Project page with links to download tarballs and documentation
README.xml with changelog
Installation
License

ocamlfind's People

Contributors

Stargazers

Watchers

ocamlfind's Issues

The link to the quickstart in the user's guide is broken

On this page: http://projects.camlcity.org/projects/dl/findlib-1.9.6/doc/guide-html/index.html ...where it says:

Quickstart. See also the Quickstart page for instructions for the most common cases.

The link goes to a 404 page. It should point here instead: http://projects.camlcity.org/projects/dl/findlib-1.9/doc/QUICKSTART

ocamlfind output has problems on windows

On windows in unix context it should not output the extra \r otherwise it breaks its use in configure scripts: see ocaml/opam-repository#24841

doc/ref-man seems to be missing.

doc/ref-man, and along with it the man pages, seems to be missing from the sources.

opam install ocamlfind.1.9.5 fails with ocaml 4.00.1

15:58 ~/src/hol-light (master) ocamlc -v
The OCaml compiler, version 4.00.1
Standard library directory: /home/blanqui/.opam/4.00.1/lib/ocaml

15:57 ~/src/hol-light (master) opam install ocamlfind
The following actions will be performed:
  ∗ install ocamlfind 1.9.5

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
⬇ retrieved ocamlfind.1.9.5  (cached)
[ERROR] The compilation of ocamlfind.1.9.5 failed at "make opt".

#=== ERROR while compiling ocamlfind.1.9.5 ====================================#
# context     2.1.2 | linux/x86_64 | ocaml-base-compiler.4.00.1 | https://opam.ocaml.org#a887fb27
# path        ~/.opam/4.00.1/.opam-switch/build/ocamlfind.1.9.5
# command     ~/.opam/opam-init/hooks/sandbox.sh build make opt
# exit-code   2
# env-file    ~/.opam/log/ocamlfind-6441-f5e279.env
# output-file ~/.opam/log/ocamlfind-6441-f5e279.out
### output ###
# Options are:
# [...]
#   -dalloc  (undocumented)
#   -dreload  (undocumented)
#   -dscheduling  (undocumented)
#   -dlinear  (undocumented)
#   -dstartup  (undocumented)
#   - <file>  Treat <file> as a file name (even if it starts with `-')
#   -help  Display this list of options
#   --help  Display this list of options
# make[1]: *** [Makefile:173 : findlib_config.cmx] Erreur 2
# make[1] : on quitte le répertoire « /home/blanqui/.opam/4.00.1/.opam-switch/build/ocamlfind.1.9.5/src/findlib »
# make: *** [Makefile:18 : opt] Erreur 2



<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
┌─ The following actions failed
│ λ build ocamlfind 1.9.5
└─ 
╶─ No changes have been performed

1.9.1 Install script not parsing on MacOS

The install script for ocamlfind doesn't appear to be compatible with zsh.
I've tried running the same script with bash, and it does appear to parse.

$ opam install ocamlfind                          
The following actions will be performed:
  ∗ install ocamlfind 1.9.1
  ∗ install topkg     1.0.3
  ∗ install ctypes    0.18.0
===== ∗ 3 =====
Do you want to continue? [Y/n] y

<><> Gathering sources ><><><><><><><><><><><><><><><><><><><><><><><><><><>  🐫 
[ctypes.0.18.0] found in cache
[ocamlfind.1.9.1] found in cache

[topkg.1.0.3] found in cache

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><>  🐫 
[ERROR] The installation of ocamlfind failed at "make install".

#=== ERROR while installing ocamlfind.1.9.1 ===================================#
# context     2.0.5 | macos/x86_64 | ocaml-base-compiler.4.08.1 | https://opam.ocaml.org#d9074796
# path        ~/.opam/4.10.0/.opam-switch/build/ocamlfind.1.9.1
# command     ~/.opam/opam-init/hooks/sandbox.sh install make install
# exit-code   2
# env-file    ~/.opam/log/ocamlfind-77354-c743ac.env
# output-file ~/.opam/log/ocamlfind-77354-c743ac.out
### output ###
# [...]
# if [ "1" -eq 1 ]; then \
#     for x in camlp4 dbm graphics labltk num ocamlbuild; do \
#       if [ -f "/Users/ishaangandhi/.opam/4.10.0/lib/$x/META" ]; then \
#         if ! grep -Fq '[distributed with Ocaml]' "//Users/ishaangandhi/.opam/4.10.0/lib/$x/META"; then \
#           rm -f site-lib-src/$x/META; \
#         fi \
#       fi \
#     done; \
#     test -f "site-lib-src/num/META" || rm -f "site-lib-src/num-top/META"; \
#   fi
# zsh:1: parse error near `done'
# make: *** [check-installation] Error 1



<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><>  🐫 
┌─ The following actions failed
│ ∗ install ocamlfind 1.9.1
└─ 
╶─ No changes have been performed

Patch to make ocamlfind relocatable (via a binary installer on macOS, Windows, Snap)

I maintain the Coq Platform project, which provides binary installers for MacOS, Windows and Linux (snap) for INRIA Coq (a proof assistant). In version 8.16 (autumn 2022) Coq switched to use ocamlfind. Now ocamlfind was not designed with relocation by an installer in mind.

I patched ocamlfind such that it works with relocation.

The main changes are:

ocamlfind searches the directory structure upwards from the executable until it finds a file findlib.root or reaches /.
The OCAML_SITELIB, OCAMLFIND_CONF are augmented with a _RUN variant which allows to give e.g. relative path to the path of the findlib.root file (three leading dots are replaced with the path of findlib.root).

This mechanism has the advantage, that one need not patch files with directories during installation - one just has to put the findlib.root at the proper place, which is no issue on the supported platforms.

This system should have a few 1000 installations meanwhile on macOS, Windows and Snap and I did not hear anything negative. If this is desirable for other ocamlfind users is a different question.

Please let me know what you think and if you would like to see a PR or if you have a better idea I can also keep this as a Coq Platform local patch - it shouldn't kill me to port the patch.

Fails to build on OCaml 5.00 trunk

In ocaml/ocaml#10867, OCaml removed a load of deprecated functions. This prevents ocamlfind from installing:

#=== ERROR while compiling ocamlfind.1.9.1 ====================================#
# context     2.1.0 | linux/x86_64 | ocaml-variants.5.00.0+trunk | https://opam.ocaml.org#ffd4aedc
# path        ~/.opam/5.00.0+trunk/.opam-switch/build/ocamlfind.1.9.1
# command     ~/.opam/opam-init/hooks/sandbox.sh build make all
# exit-code   2
# env-file    ~/.opam/log/ocamlfind-93913-b17e9e.env
# output-file ~/.opam/log/ocamlfind-93913-b17e9e.out
### output ###
# [...]
# ocamlc -I +compiler-libs -opaque -g -c frontend.ml
# File "frontend.ml", line 859, characters 10-39:
# 859 |           Fl_package_base.package_users predicates1 packages1
#                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# Warning 6 [labels-omitted]: label preds was omitted in the application of this function.
# File "frontend.ml", line 1826, characters 16-29:
# 1826 |       let buf = String.create buflen in   (* FIXME: Bytes.create *)
#                        ^^^^^^^^^^^^^
# Error: Unbound value String.create
# make[1]: *** [Makefile:165: frontend.cmo] Error 2
# make[1]: Leaving directory '/home/user/.opam/5.00.0+trunk/.opam-switch/build/ocamlfind.1.9.1/src/findlib'
# make: *** [Makefile:14: all] Error 2

Applying the fix in the comment does indeed fix it.

export currently non-exported CMI files for findlib ?

Can I send you a PR to export the currently non-exported CMI files of findlib? E.g. findlib_config ?

This is useful for writers of add-on tools (like reinstall-if-diff) who want their code to be as portable as ocamlfind.

ocamlnat problems

When one invokes #use topfind this test here triggers a an error on macos 14.4.1 with ocamlfind 1.9.6.

While on ocaml 4.14.2 this seems harmless:

> ocamlnat -noinit
OCaml version 4.14.2 - native toplevel
Enter #help;; for help.

# #use "topfind";;
>> Fatal error: Toploop.dll_run dlopen(/var/folders/9p/bzrzxtv53zsdsb88p7lbcsk80000gn/T/camlTOP256189b.so, 0x000A): symbol not found in flat namespace '_camlTopfind'
File "_none_", line 1:
Warning 58 [no-cmx-file]: no cmx file was found in path for module TOP1, and its interface was not compiled with -opaque
- : unit = ()
Findlib has been successfully loaded. Additional directives:
  #require "package";;      to load a package
  #list;;                   to list the available packages
  #camlp4o;;                to load camlp4 (standard syntax)
  #camlp4r;;                to load camlp4 (revised syntax)
  #predicates "p,q,...";;   to set these predicates
  Topfind.reset();;         to force that packages will be reloaded
  #thread;;                 to enable threads

- : unit = ()

It seems that on 5.2.0~rc1 this results in a catastrophic error:

> ocamlnat -noinit
OCaml version 5.2.0~rc1 - native toplevel
Enter #help;; for help.

# #use "topfind";;
File "_none_", line 1:
Warning 58 [no-cmx-file]: no cmx file was found in path for module Topfind, and its interface was not compiled with -opaque
>> Fatal error: Toploop.dll_run dlopen(/var/folders/9p/bzrzxtv53zsdsb88p7lbcsk80000gn/T/camlTOP2a4cc58.so, 0x000A): symbol not found in flat namespace '_camlTopfind'
Error while loading /Users/dbuenzli/.opam/5.2.0~rc1+ocamlnat/lib/findlib/findlib.cma: error loading shared library: Failure("dlopen(/Users/dbuenzli/.opam/5.2.0~rc1+ocamlnat/lib/findlib/findlib.cma, 0x000A): tried: '/Users/dbuenzli/.opam/5.2.0~rc1+ocamlnat/lib/findlib/findlib.cma' (not a mach-o file), '/System/Volumes/Preboot/Cryptexes/OS/Users/dbuenzli/.opam/5.2.0~rc1+ocamlnat/lib/findlib/findlib.cma' (no such file), '/Users/dbuenzli/.opam/5.2.0~rc1+ocamlnat/lib/findlib/findlib.cma' (not a mach-o file)").
Error while loading /Users/dbuenzli/.opam/5.2.0~rc1+ocamlnat/lib/findlib/findlib_top.cma: error loading shared library: Failure("dlopen(/Users/dbuenzli/.opam/5.2.0~rc1+ocamlnat/lib/findlib/findlib_top.cma, 0x000A): tried: '/Users/dbuenzli/.opam/5.2.0~rc1+ocamlnat/lib/findlib/findlib_top.cma' (not a mach-o file), '/System/Volumes/Preboot/Cryptexes/OS/Users/dbuenzli/.opam/5.2.0~rc1+ocamlnat/lib/findlib/findlib_top.cma' (no such file), '/Users/dbuenzli/.opam/5.2.0~rc1+ocamlnat/lib/findlib/findlib_top.cma' (not a mach-o file)").

File "_none_", line 1:
Warning 58 [no-cmx-file]: no cmx file was found in path for module TOP1, and its interface was not compiled with -opaque

- : unit = ()

File "_none_", line 1:
Warning 58 [no-cmx-file]: no cmx file was found in path for module Topfind, and its interface was not compiled with -opaque
>> Fatal error: Toploop.dll_run dlopen(/var/folders/9p/bzrzxtv53zsdsb88p7lbcsk80000gn/T/camlTOP3a63122.so, 0x000A): symbol not found in flat namespace '_camlTopfind'

Exception: Misc.Fatal_error.

However going back to 4.14.2. It seems the support is broken (does it try to load the wrong objects ? it prints cmxa files instead of cmxs, not sure if it's a rendering bug or the actual bug):

# #require "cmdliner";;
/Users/dbuenzli/.opam/4.14.2+ocamlnat/lib/cmdliner: added to search path
/Users/dbuenzli/.opam/4.14.2+ocamlnat/lib/cmdliner/cmdliner.cmxa: loaded
# Cmdliner.Cmd.eval;;
>> Fatal error: Toploop.dll_run dlopen(/var/folders/9p/bzrzxtv53zsdsb88p7lbcsk80000gn/T/camlTOP44dc430.so, 0x000A): symbol not found in flat namespace '_camlCmdliner'
Exception: Misc.Fatal_error.

This is not an issue with the compilation object as omod can load it:

# #use "omod.top";;
Omod v0.0.4-3-gb0d2684 loaded. Type Omod.help () for more info.
# Omod.load "Cmdliner";;
[INC] /Users/dbuenzli/.opam/4.14.2+ocamlnat/lib/ocaml
[INC] /Users/dbuenzli/.opam/4.14.2+ocamlnat/lib/cmdliner
[OBJ] /Users/dbuenzli/.opam/4.14.2+ocamlnat/lib/cmdliner/cmdliner.cmxs
- : bool = true
# Cmdliner.Cmd.eval;;
- : ?help:Format.formatter ->
    ?err:Format.formatter ->
    ?catch:bool ->
    ?env:(string -> string option) ->
    ?argv:string array ->
    ?term_err:Cmdliner.Cmd.Exit.code ->
    unit Cmdliner.Cmd.t -> Cmdliner.Cmd.Exit.code
= <fun>

Some problems with warning/error in findlib

Gerd, I'm finding some problems with findlib's warning/error. Here's an example:

ocamlfind ocamlc -verbose -package camlp5.pa_o,camlp5.pr_o -syntax camlp5r -c foo.ml  "
Effective set of preprocessor predicates: preprocessor,syntax,camlp5r
Effective set of compiler predicates: pkg_camlp5.pa_o,pkg_camlp5.pr_o,syntax,autolink,byte

In the META file for camlp5, I have:

package "pa_o" (
  error(camlp5r) = "camlp5.pa_o cannot be used with syntax camlp5r"
...
}

But this is useless, because error/warning are evaluated against compiler predicates, and "camlp5o" is a syntax predicate. I think that, in addition to adding "syntax" to syntax_preds and predicates, it would be useful to add syntax_<syntax> (where <syntax> is the specified syntax, e.g. syntax_camlp5o).

What do you think?

It's a small change, and I can send you a PR, but wanted to get your thoughts first.

P.S. Obviously I can make this change in chetmurthy/not-ocamlfind and it will suffice for many of my needs, but I figured, this is a useful consistency-check for findlib itself. It will make error/warning variables much more useful.

remove special treatment of "num"

For discussion, see ocaml/opam-repository#10925

`compiler-libs.common` being loaded again in REPL

Hello all,

I am trying to use the Env module in compiler-libs.common (link: https://ocaml.org/p/ocaml-base-compiler/4.14.1/doc/Env/index.html) , which was expunged from the recent toplevel.
In order to use this, I am building a new REPL using ocamlmktop which links compiler-libs to the REPL, and this resolves the issue.
However, if I load some library, e.g., camlp5, using Topfind.load_deeply, it loads compiler-libs.common again, making the REPL unable to use Env:

# Topdirs.dir_use Format.std_formatter "topfind";;
(snip)
# Topfind.load_deeply ["camlp5"];;
.../_opam/lib/ocaml/compiler-libs: added to search path
.../_opam/lib/ocaml/compiler-libs/ocamlcommon.cma: loaded <- this again loads ocamlcommon.cma
(snip)
# let x = Env.lookup_value ~use:true ~loc:Location.none Longident.(Ldot(Lident "Bytes", "get")) !Toploop.toplevel_env;;
Exception:
Env.Error
 (Env.Lookup_error
   ({Location.loc_start =
      {Lexing.pos_fname = "_none_"; pos_lnum = 0; pos_bol = 0; pos_cnum = -1};
     loc_end =
      {Lexing.pos_fname = "_none_"; pos_lnum = 0; pos_bol = 0; pos_cnum = -1};
     loc_ghost = true},
   <abstr>,
   Env.Cannot_scrape_alias (Longident.Lident "Bytes", Path.Pident <abstr>))).

ocaml/ocaml#12271 has more details about the error.

In order to resolve this error, I am relying on Topfind.don't_load ["compiler-libs.common"]. It works, but it would be great if compiler-libs.common is not loaded by default because it is already available in the toplevel.

an "ocamlfind reinstall-if-diff" command

[this is almost certainly not one I think you'll want to add to ocamlfind, so I'm really asking you this to get your opinion whether this is a foolish idea, before I implement it]

Recently I've been building a project with a bunch of interrelated sub-packages that need to be built in a particular order. At full-project-install time, I install a single findlib package with a bunch of sub-packages. Durin the build, each sub-package is built and then installed in a local repository. So sub-package C can depend on sub-package A,B, by depending on their findlib directories, and instead of a mess of include-directories and references to cmo/cmi/cma files, the references are always to other findlib packages. It's very clean and neat.

There is only one problem: the last step in building a sub-package is to "ocamlfind install" it to the local repository. But if the package hasn't actually changed, then the act of installing it will update the insalled package. So if I rerun the build on package B, this will cause C to think that B has changed, and trigger a complete rebuild of C.

If there were an "ocamlfind reinstall-if-diff " command that would (a) check that the list of files provided was identical to the contents of the package , and if so, then do nothing, and (b) if different, would delete package and reinstall it, then this would be perfect.

I don't think this belongs in ocamlfind, but I do think this is a useful way to organize nontrivial ocaml projects with a bunch of internal dependencies.

What do you think? Am I missing something?

P.S. I believe that along this path will be an integration between ocamlfind and Google Bazel. That's sort of where I'm heading.

ocamlopt: cannot execute binary file [findlib_config.cmx] Error 126 on macOS

Hi I am on latest macOS 12.1 and am seeing the following failure:

$ opam list
# Packages matching: installed
# Name              # Installed # Synopsis
base-bigarray       base
base-threads        base
base-unix           base
conf-findutils      1           Virtual package relying on findutils
conf-gmp            3           Virtual package relying on a GMP lib system installation
ocaml               4.10.2      The OCaml compiler (virtual package)
ocaml-base-compiler 4.10.2      Official release 4.10.2
ocaml-config        1           OCaml Switch Configuration
$ opam install ocamlfind
The following actions will be performed:
  ∗ install ocamlfind 1.9.1

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><>  🐫
⬇ retrieved ocamlfind.1.9.1  (cached)
[ERROR] The compilation of ocamlfind.1.9.1 failed at "make opt".

#=== ERROR while compiling ocamlfind.1.9.1 ====================================#
# context     2.1.2 | macos/x86_64 | ocaml-base-compiler.4.10.2 | https://opam.ocaml.org#3bc4295d
# path        ~/.opam/__coq-platform.2021.11.0~8.13~2021.11/.opam-switch/build/ocamlfind.1.9.1
# command     ~/.opam/opam-init/hooks/sandbox.sh build make opt
# exit-code   2
# env-file    ~/.opam/log/ocamlfind-51444-a0e8a2.env
# output-file ~/.opam/log/ocamlfind-51444-a0e8a2.out
### output ###
# for p in findlib; do ( cd src/$p; /Library/Developer/CommandLineTools/usr/bin/make opt ) || exit; done
# ocamlopt -I +compiler-libs -g -opaque -c findlib_config.ml
# /Users/kartik/.opam/__coq-platform.2021.11.0~8.13~2021.11/bin/ocamlopt: /Users/kartik/.opam/__coq-platform.2021.11.0~8.13~2021.11/bin/ocamlopt: cannot execute binary file
# make[1]: *** [findlib_config.cmx] Error 126
# make: *** [opt] Error 2



<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><>  🐫
┌─ The following actions failed
│ λ build ocamlfind 1.9.1
└─
╶─ No changes have been performed
$

EDIT: it may be related to #25 cc: @arbipher. However, the error does not appear on arm64, just on x86_64.

ocamlfind doesn't work for wasicaml

three problems:

wasicaml has a bug and cannot run executables by non-absolute path (-> solve there, remixlabs/wasicaml#58)

/Users/gerd/biz/figly/protoquery/wasicaml/bin/ocamlfind ocamlmklib
ocamlfind: Package `threads' not found

This is due to a bug in findlib: frontend.ml, line 923, change this to:

  let type_of_threads =
    try package_property [] "threads" "type_of_threads"
    with No_such_package _ -> "ignore"

wasicaml doesn't support Unix.spawn, but you can use Sys.command. Alternate run_command implementation:

let run_command ?filter verbose cmd args =
  let printable_cmd =
    cmd ^ " " ^ String.concat " " (List.map escape_if_needed args) in
  ( match verbose with
      | Normal ->
          ()
      | Verbose ->
          print_endline ("+ " ^ printable_cmd);
          if filter <> None then
            print_string
              ("  (output of this command is filtered by ocamlfind)\n")
      | Only_show ->
          print_endline printable_cmd
  );
  flush stdout;

  if verbose <> Only_show then (
    let temp_file_opt =
      match filter with
        | None -> None
        | Some _ ->
            Some(Filename.temp_file "ocamlfind" ".out") in
    let cmd1 =
      match temp_file_opt with
        | None -> cmd
        | Some f -> cmd ^ " > " ^ Filename.quote f in
    let code =
      Sys.command cmd1 in
    if code <> 0 then (
      if verbose = Verbose then
        print_string (cmd ^ " returned with exit code " ^ string_of_int code ^ "\n");
      exit code
    );

    ( match filter, temp_file_opt with
        | Some filter_fun, Some temp_file ->
            let ch = open_in temp_file in
            ( try
                while true do
                  let line = input_line ch in
                  match filter_fun line with
                      None -> ()       (* Suppress line *)
                    | Some line' -> print_endline line'
                done;
                assert false
              with
                  End_of_file ->
                  close_in ch;
                  flush stdout
            )
        | _ -> ()
    )
  )
;;

configure fails to detect native dynlink

I'm using ocaml 4.14.0, built on x86-64 with world.opt and no special configure options. findlib 1.9.3 worked fine.

findlib 1.9.4's configure script fails to detect the native dynlink library. This is because check_library correctly detects the bytecode library in the standard location and sets dynlink_dir to ^, but the natdynlink test doesn't handle that value:

if [ -f "${ocaml_core_stdlib}/${dynlink_dir#+}/dynlink.cmxa" ]; then

Rewriting it as follows fixes the problem, but you may prefer a different fix:

if [ "$dynlink_dir" = '^' ]; then
    real_dynlink_dir="."
else
    real_dynlink_dir="${dynlink_dir#+}"
fi
if [ -f "${ocaml_core_stdlib}/$real_dynlink_dir/dynlink.cmxa" ]; then

Using `install` as `INSTALLFILE` sets the executable bit on all installed files

Discovered while updating the Debian package. As a workaround, I set INSTALLFILE to cp.

homepage is using http

findlib's home page is: http://projects.camlcity.org/projects/findlib.html
It's using http instead of https. Some web proxy will just block http://xxx.
I'm wondering if this situation could be improved a little bit.

opaque and g flags are not detected correctly on ocaml 5.2

Hi,
OCaml 5.2.0 contains ocaml/ocaml#12613 which will make ocamlc and ocamlopt error if no input files are passed.
The configure script relies on this do detect support for -opaque and -g:

ocamlfind/configure

Line 437 in f68ff34

ocamlc -opaque >/dev/null 2>/dev/null || opaque=""

ocamlfind/configure

Line 445 in f68ff34

ocamlopt -g >/dev/null 2>/dev/null || native_debugging_info=""

Concretely, building findlib in an opam switch before and after the ocaml change yields the following diff:

--- /tmp/opam-good/lib/findlib/Makefile.config
+++ /tmp/opam-bad/lib/findlib/Makefile.config
@@ -24,8 +24,8 @@
 SYSTEM=linux
 NUMTOP=
 SH=
-OPAQUE=-opaque
-OCAMLOPT_G=-g
+OPAQUE=
+OCAMLOPT_G=
 CHECK_BEFORE_INSTALL=0
 INSTALLDIR = install -d
 # change to INSTALLDIR = mkdir -p   when BSD install is unavavailable

This is visible to the end user because linking against findlib will now emit this warning:

Warning 58 [no-cmx-file]: no cmx file was found in path for module Findlib, and its interface was not compiled with -opaque

Thanks

ocamlfind will not build on OSX Catalina if CLICOLOR=1 and/or CLICOLOR_FORCE=1

tl;dr:

When building ocamlfind on OSX:

First: check to see if ls site-lib-src outputs colorized text.
- If the output is not colorized text, then keep going
- Otherwise, see if gls (gnu-ls) is installed.
  - If gls is installed, keep going
  - If gls is not installed, tell the user they need to install Gnu's ls, for instance, via brew install coreutils
Second: you need to unroll

ocamlfind/Makefile

Line 99 in 6c8438c

for x in `ls site-lib-src`; do test ! -f "site-lib-src/$$x/META" || echo $$x >> Makefile.packages.in; done

to properly capture the exit code from seeing if a META file exists in a sub-sub-directory of site-lib-src

STORY:

I come here after much messing around trying to get opam install csvtool to work on OSX Catalina on an Intel Mac.

It was failing on ocaml install topkg. Working with @kit-ty-kate and @dbuenzli starting at dbuenzli/topkg#119 (comment) we took a look at ocamlfind

This line in the Makefile appears to have an issue:

ocamlfind/Makefile

Line 99 in 6c8438c

 for x in `ls site-lib-src`; do test ! -f "site-lib-src/$$x/META" || echo $$x >> Makefile.packages.in; done 

If I'm understanding of that line correctly, the goal is:

Iterate over a list of site-lib-src directories; call the current iterated value x
If site-lib-src/x/META IS a regular file, return exit code 1; else return exit code 0
If the exit code is non-zero -- meaning that IT WAS a regular file -- then echo that x into Makefile.packages.in -- else go to next entry from the site-lib-src directory

After doing

./configure
make all
make opt
make install

I find there are 3 files that qualify for that META:

➜ find site-lib-src -name "META" -print
site-lib-src/bytes/META
site-lib-src/stdlib/META
site-lib-src/dynlink/META

And if I run test ! -f "site-lib-src/bytes/META" || echo "bytes" >> Makefile.packages.in, indeed I do get:

➜ cat Makefile.packages.in
bytes

The problem? As soon as one puts that test ! -f "site-lib-src/$$x/META" || echo $$x >> Makefile.packages.in line inside a for loop, OSX always considers the referenced META file returns 0.

Every. Single. Time.

...which means that the Makefile.packages.in file is never created...and the Makefile.packages file ends up just being:

➜ cat Makefile.packages
SITELIB_META = %

You would think the easy answer would be to -- within the for loop:

Run the test command by itself, sending the output to /dev/null
Assign $? to a variable
Now process accordingly

It gets even more complicated:

In OSX, if one is using a fancy-schmancy terminal program -- like iterm2 -- with a nice shell and prompt -- like oh-my-zsh with the spaceship-prompt -- then ls is colorized due to a forced alias of ls -G...

...and the ls site-lib-src output can't be properly understood by the test command because the characters are in blue.

...which means on OSX we need to de-colorize the ls site-lib-src output before using it.

"Oh", you think, "let's just use ls --color=never", right?

Doesn't work.

How about \ls?

Nope -- nor "ls" nor 'ls'

So far the only way I've found to do that do that is to brew install coreutils -- which then provides gls -- which then offers a non-colorized output of site-lib-src.

Unrolled bash script: https://gist.github.com/philCryoport/5565b8fae4b58b5a26823860f28691ad

Resulting Makefile.packages.in:

➜ cat Makefile.packages.in
bigarray
bytes
compiler-libs
dynlink
ocamldoc
stdlib
str
threads
unix

Please do fix when you get a moment?

Thanks.

Bad argument when running `ocamlfind printconf effmetadir`

The usage message for effmetadir is not supported (src).

May I also ask what is the prefix eff_ for?

An error in the computation of syntax packages?

In process_pp_spec, syntax packages are computed by taking the argument packages and running it thru a predicate that checks that they have a nonempty "archive" variable. But this means that a "meta-package" that requires N other packages and doesn't have its own archive, is not selectable as a syntax package. which means that the packages it requires are not thereby selectable, either.

The fix is trivial; I can send a PR, but I include it here only for perusal. I've also added some logging so that writers of syntax packages can debug their META files. If you're OK with this change, I can send you a PR; I can either include or not include the logging (I think it's useful, but it's no big deal either way).

    * [syntax_preds] + "byte".
    *)
 
+  let candidate_packages =
+    package_deep_ancestors syntax_preds packages in
+  if verbose then begin
+    prerr_string ("Preprocessor predicates: " ^
+		  String.concat "," syntax_preds ^ "\n");
+    prerr_string ("Candidate preprocessor packages: " ^
+		  String.concat "," candidate_packages ^ "\n");
+  end ;
+
   (* One packages must now have the variable "preprocessor", usually camlp4 *)
-  let cl_pp_packages = select_pp_packages syntax_preds packages in
+  let cl_pp_packages = select_pp_packages syntax_preds candidate_packages in
   let pp_packages =
     package_deep_ancestors syntax_preds cl_pp_packages in
 
+  if verbose then begin
+    prerr_string ("Selected preprocessor packages: " ^
+		  String.concat "," pp_packages ^ "\n");
+  end ;
+
   let preprocessor_cmds =
     List.flatten
       (List.map (fun pname ->

Improved support for build-time Fl_dynload ?

Dear findlib maintainers,

in Coq we have started to use Fl_dynload to allow the loading of Coq's plugins that have dependencies. Also, this means we identify our plugin namespace with findlib's namespace which I think is a good think.

However, we are facing some limitations to make this work at build-time with dune (so we can compose packages, etc...)

In particular, when generating the build rules for a .v file that depends on a plugin, we need to translate a findlib library name pkg.foo to the plugin .cmxs file, while we are in the build layout. We face two problems here:

findlib's package path is determined at initialization time, moreover it seems we can use some of the API without calling init and init will be called automatically. Should maybe the contract be made stronger? Would a more dynamic handling of the include path make sense?
a critical problem is that Dune generates _build/default/META.coq-core , however that file is not usable as it is meant to be installed in _build/install/lib/coq-core/META, thus findlib will complain that a top-level directory field is missing. Would making the API more flexible make sense?
an alternative choice would be to depend on installed plugins and libraries, however that may be problematic for Dune's build / install phase separation. On the other hand, maybe that is what we should do, as the scheme Coq has adopted requires plugins to be public (which is IMHO a bit unfortunate, but that's an orthogonal problem)

cc @Alizter @rgrinberg @gares

Gentoo - undefined reference to `io.c.880cdbd1

I am expriencing following output during build:
build.log

my gcc -v:

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-pc-linux-gnu/11.2.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /var/tmp/portage/sys-devel/gcc-11.2.0/work/gcc-11.2.0/configure --host=x86_64-pc-linux-gnu --build=x86_64-pc-linux-gnu --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/gcc-bin/11.2.0 --includedir=/usr/lib/gcc/x86_64-pc-linux-gnu/11.2.0/include --datadir=/usr/share/gcc-data/x86_64-pc-linux-gnu/11.2.0 --mandir=/usr/share/gcc-data/x86_64-pc-linux-gnu/11.2.0/man --infodir=/usr/share/gcc-data/x86_64-pc-linux-gnu/11.2.0/info --with-gxx-include-dir=/usr/lib/gcc/x86_64-pc-linux-gnu/11.2.0/include/g++-v11 --with-python-dir=/share/gcc-data/x86_64-pc-linux-gnu/11.2.0/python --enable-languages=c,c++,go,fortran --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --disable-libunwind-exceptions --enable-checking=yes --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 11.2.0 p1' --disable-esp --enable-libstdcxx-time --with-build-config=bootstrap-lto --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --enable-multilib --with-multilib-list=m32,m64 --disable-fixed-point --enable-targets=all --enable-libgomp --disable-libssp --disable-libada --disable-systemtap --disable-valgrind-annotations --disable-vtable-verify --disable-libvtv --with-zstd --enable-lto --with-isl --disable-isl-version-check --enable-default-pie --enable-default-ssp
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 11.2.0 (Gentoo 11.2.0 p1)

I have following ocaml language package installed:

 * Found these USE flags for dev-lang/ocaml-4.11.2-r2:
 U I
 - - emacs     : Add support for GNU Emacs
 - - flambda   : Enables the Flambda optimizer: A new intermediate representation (introduced in ocaml 4.03) in the depths of the compiler designed to allow for better inlining. 
 - - latex     : Add support for LaTeX (typesetting package)
 + + ocamlopt  : Enable ocamlopt support (ocaml native code compiler) -- Produces faster programs (Warning: you have to disable/enable it at a global scale)
 - - spacetime : Enables the Spacetime memory profiler. See https://caml.inria.fr/pub/docs/manual-ocaml/spacetime.html for more information. 
 - - xemacs    : Add support for XEmacs

findlib: use install(1) + support DESTDIR

The Makefiles for findlib use mkdir and cp for installing files.
Please use install(1) instead (install -d for directories, install for files, one at a time).

Also, it would be great if you could support DESTDIR. Package managers want to install into a temporary location and create the binary package from there.

So instead of

mkdir -p "$(prefix)$(OCAMLFIND_BIN)"

use

install -d "$(DESTDIR)$(prefix)$(OCAMLFIND_BIN)"

and instead of

cp doc/ref-man/ocamlfind.1 "$(prefix)$(OCAMLFIND_MAN)/man1"

use

install -c doc/ref-man/ocamlfind.1 "$(DESTDIR)$(prefix)$(OCAMLFIND_MAN)/man1"

Thanks!

configure sets invalid variable name

Configuring findlib-1.9.4, using bash 5.1.16 as the shell, gives this harmless error in the middle of the configure output:

./configure: line 497: compiler-libs_dir=^: command not found

compiler-libs_dir isn't a legal sh variable name. This is coming from check_library; should it special-case compiler-libs?

Can not compile ocamlfind 1.9.6 under Cygwin /opam 2.2.0 /ocaml 5.2.0

# tools/patch '@sitelib@' '/home/Kevin/.opam/5.2.0+trunk/lib' |
# tools/patch '@FINDLIB_PATH@' '/home/Kevin/.opam/5.2.0+trunk/lib/ocaml;/home/Kevin/.opam/5.2.0+trunk/lib' -p >findlib.conf
# sed: -e expression #1, char 58: unknown option to `s'

In the patch script, the delimiter used by sed is ;, which is generally a safe choice. However, since the replacement value contains ;, we need to choose a delimiter that won't appear in the paths.

Here, I've changed the delimiter from ; to |, as | is less likely to appear in file paths. So, the final line of the patch script should look like this:

sed -e 's|'"$varname"'|'"$varvalue"'|g'

would an "ocamlfind preprocess" subcommand be useful?

It seems like, often there's a use for getting one's hands on the output of the preprocessing step of an "ocamlfind ocamlc ..." command. Both for debugging, but also for interfacing with dune. I can't find a way to do this right now ..... but I'm guessing that I could figure out how to implement it in not much time. Would this be useful? Is there something I'm missing?

Thanks in advance.

1.9.5 fails to install on NixOS

While installing ocamlfind 1.9.5 in NixOS 22.05:

$ opam install "ocamlfind=1.9.5"
...
#=== ERROR while installing ocamlfind.1.9.5 ===================================#
# context     2.1.2 | linux/x86_64 | ocaml.4.13.1 | https://opam.ocaml.org#e1c20992
# path        ~/.opam/default/.opam-switch/build/ocamlfind.1.9.5
# command     ~/.opam/opam-init/hooks/sandbox.sh install make install
# exit-code   2
# env-file    ~/.opam/log/ocamlfind-103265-a0c709.env
# output-file ~/.opam/log/ocamlfind-103265-a0c709.out
### output ###
# [...]
# install -d "`dirname \"/home/ryan/.opam/default/lib/findlib.conf\"`"
# test -f "/home/ryan/.opam/default/lib/findlib.conf" || install -c findlib.conf "/home/ryan/.opam/default/lib/findlib.conf"
# make[1]: Leaving directory '/home/ryan/.opam/default/.opam-switch/build/ocamlfind.1.9.5'
# for p in findlib; do ( cd src/$p; make install ); done
# make[1]: Entering directory '/home/ryan/.opam/default/.opam-switch/build/ocamlfind.1.9.5/src/findlib'
# install -d "/home/ryan/.opam/default/lib/findlib"
# install -d "/home/ryan/.opam/default/bin"
# install -d "/nix/store/sx7m5a7xwh76jzb05g07f3nbhl44qw4z-ocaml-4.13.1/lib/ocaml"
# install: cannot change permissions of ‘/nix/store/sx7m5a7xwh76jzb05g07f3nbhl44qw4z-ocaml-4.13.1/lib/ocaml’: Read-only file system
# make[1]: *** [Makefile:126: install] Error 1
# make[1]: Leaving directory '/home/ryan/.opam/default/.opam-switch/build/ocamlfind.1.9.5/src/findlib'
# make: *** [Makefile:24: install] Error 2
...

Note that in nix, /nix/store is read only to ensure reproducibility.

I've found that the last available version on opam, 1.9.3, works fine.