OCaml's -w option should be supported, but I don't know how. There is no corresponding value in the Clflags module.

Here's a utop session showing the behavior.

First, I first define a show function because Outcome.eval is abstract:

utop # open Oloop;;
utop # let show (x : 'a Outcome.t) = match x with
  | `Eval x -> `Eval (Outcome.result x, Outcome.stdout x, Outcome.warnings x)
  | `Uneval _ -> `Uneval x;;
val show : 'a Outcome.t -> [> `Eval of Outcometree.out_phrase * string * (Location.t * Warnings.t) list | `Uneval of 'a Outcome.t ] =
  <fun>

Now, create an oloop toplevel:

utop # let t = create Outcome.merged >>| ok_exn;;
val t : Outcome.merged t Deferred.t = <abstr>

Evaluate one phrase. Seems okay.

utop # t >>= fun t -> eval t "#use \"topfind\"" >>| show;;
- : [> `Eval of Outcometree.out_phrase * string * (Location.t * Warnings.t) list | `Uneval of Outcome.merged Outcome.t ] =
`Eval
(Outcometree.Ophr_eval (Outcometree.Oval_constr (Outcometree.Oide_ident "()", []), Outcometree.Otyp_constr (Outcometree.Oide_ident "unit", [])),    
   "Findlib has been successfully loaded. Additional directives:\n  #require \"package\";;      to load a package\n  #list;;                   to list the available packages\n  #camlp4o;;                to load camlp4 (standard syntax)\n  #camlp4r;;                to load camlp4 (revised syntax)\n  #predicates \"p,q,...\";;   to set these predicates\n  Topfind.reset();;         to force that packages will be reloaded\n  #thread;;                 to enable threads\n\n",
[])    

Evaluate a second phrase. There is a problem. We're not seeing the output.

utop # t >>= fun t -> eval t "#thread" >>| show;;
- : [> `Eval of Outcometree.out_phrase * string * (Location.t * Warnings.t) list | `Uneval of Outcome.merged Outcome.t ] =
`Eval
(Outcometree.Ophr_eval (Outcometree.Oval_constr (Outcometree.Oide_ident "()", []), Outcometree.Otyp_constr (Outcometree.Oide_ident "unit", [])),    
   "",
[])

Evaluate third phrase, and now you get the output that should have been returned above.

utop # t >>= fun t -> eval t "let _ = 2" >>| show;;
- : [> `Eval of Outcometree.out_phrase * string * (Location.t * Warnings.t) list | `Uneval of Outcome.merged Outcome.t ] =
`Eval
(Outcometree.Ophr_eval (Outcometree.Oval_int 2, Outcometree.Otyp_constr (Outcometree.Oide_ident "int", [])),                                        
   "/Users/ashish/.opam/rwo/lib/ocaml/threads: added to search path\n/Users/ashish/.opam/rwo/lib/ocaml/unix.cma: loaded\n/Users/ashish/.opam/rwo/lib/ocaml/threads/threads.cma: loaded\n",
[])

Here's one example of an unexpected failure:

$ cat a.ml 
#use "topfind";;
#thread;;
#camlp4o;;
#require "core";;
#require "core.syntax";;
open Core_kernel.Std;;

$ oloop a.ml -silent-directives
(* part 0 *)
# #use "topfind";;

# #thread;;

# #camlp4o;;

# #require "core";;
Error: Reference to undefined global `Lazy'

# #require "core.syntax";;
Error: Reference to undefined global `Lazy'

# open Core_kernel.Std;;
Error: Unbound module Core_kernel

I don't use ocamlbuild or oasis. Is there a way to have the tmp/ directory ignored? We ignore it in .gitignore, and I use it to write throwaway code. But if any build files are there, ocamlbuild complains about hygiene violiations.

Outcome.uneval includes a structured value representing warnings, but we don't actually provide them yet. They have to be parsed out of stderr. This will become easier in future versions of OCaml due to ocaml/ocaml#171.

$ cat async.topscript 
#use "topfind";;    
#thread;;
#camlp4o;;
#require "core";;
#require "core.syntax";;
#require "async";;
open Core.Std;;
open Async.Std;;

let maybe_raise =
  let should_fail = ref false in
  fun () ->
    let will_fail = !should_fail in
    should_fail := not will_fail;
    after (Time.Span.of_sec 0.5)
    >>= fun () ->
    if will_fail then raise Exit else return ()
;;

maybe_raise ();;
maybe_raise ();;

$ oloop async.topscript -silent-directives -determine-deferred
(* part 0 *)
# #use "topfind";;  

# #thread;;

# #camlp4o;;

# #require "core";;

# #require "core.syntax";;

# #require "async";;

# open Core.Std;;

# open Async.Std;;

# let maybe_raise =
  let should_fail = ref false in
  fun () ->
    let will_fail = !should_fail in
    should_fail := not will_fail;
    after (Time.Span.of_sec 0.5)
    >>= fun () ->
    if will_fail then raise Exit else return ()
;;
val maybe_raise : unit -> unit Async_kernel.Deferred.t = <fun>

# maybe_raise ();;
- : unit = ()

# maybe_raise ();;
(Invalid_argument "output_value: abstract value (outside heap)")
oloop-top: sending the phrase eval outcome failed because: output_value: abstract value (outside heap)

Simplifying the example doesn't give the error. For instance, if you just always raise Exit, then you don't see this error.

Some code has been taken from the ocaml.org project and some from utop's uTop_main.ml file.

This library should provide enough functionalities so that a program for PythonTeX to support OCaml can easily be developed.

José Romildo Malaquias [email protected] posted in the OCaml list:

PythonTeX [https://www.ctan.org/pkg/pythontex] is a LaTeX package that
allows Python code entered within a TeX document to be executed, and the
output to be included in the original document. It supports other
languagens beside Python.

I have just requested inclusion of OCaml support in PythonTeX:
gpoore/pythontex#62

The author of PythonTeX says it could be possible, and he explained what
would be needed from the OCaml side. See the discussion in issue #62 in
the previous link.

Basically, it would need a program which reads lines from a text file,
and these lines are given as input to an interactive session, giving
back input interspersed with output. He gives more details in the issue
discussion. Please read the issue discussion in the above link.

Is there already such a program? If not, can it be easily written?

I appreciate any help in providing this program.

We currently define type phrase, but the definition isn't really accurate. It assumes a separation of stderr and stdout, but actually these outputs should be interleaved. Also exceptions aren't mentioned. The correct type to represent outputs is probably something like:

type output = [`Stdout of string | `Stderr of string] list * exn option

This captures all stdout and stderr in the order that output occurs. It also includes an exception that may have been raised.

The harder part is to:

• Adjust the script to retain stdout and stderr in the order they are output.
• Robustly capture any exception raised by the evaluated expression, being sure not to conflate that with exceptions raised by the code evaluating the code.
• If possible, also distinguish between the output that is a rendering of the evaluated phrase vs output that is a side effect of evaluating the phrase. In this case, see if something better than string should be used to represent the result. Maybe some type from the Outcometree or Toploop modules.

Right now the return type of eval is (Outcometree.out_phrase * 'a Output.t, error * string) Result.t, which I feel is a misuse of the Error value. Normally if a function returns an Error, it means the function failed to execute correctly. However, in this case the Error (err,msg) value is actually a correct result from Oloop's perspective. The eval did its job successfully. It called the toploop and found that the given phrase had some error. As I explain in the documentation of Oloop_script, one could very well want to use Oloop to show examples of incorrect OCaml input.

I plan to define the following type:

type 'a outcome = [
| `Ophr_val of out_value * out_type * 'a Oloop.Output.t
| `Ophr_signature of (out_sig_item * out_value option) list * 'a Oloop.Output.t
| `Ophr_exception of exn * out_val * 'a Oloop.Output.t
| `Lexer of Lexer.error * Location.t * string
| `Syntaxerr of Syntaxerr.error * string
| `Typedecl of Location.t * Typedecl.error * string
| `Typetexp of Location.t * Env.t * Typetexp.error * string
| `Typecore of Location.t * Env.t * Typecore.error * string
| `Symtable of Symtable.error * string
]

The first 3 are semantically equivalent to Outcometree.out_phrase * 'a Output.t, and the rest are equivalent to error * string. Putting them all in a flat list now forces the user to decide what is considered an error or not, which I think is the only option. We really cannot know if evaluation of a script should stop on the first lexer error, for example. Maybe all the subsequent phrases make perfect sense still.

Note that eval could truly fail, e.g. oloop-top shutdown since it was launched. The current implementation doesn't capture this, and maybe it's okay to leave this as an exception.

Add options to the command line app that match those of the OCaml compiler, especially -short-paths and -safe-string.

I don't get the output of the file:

$ ./test_deferred.native 
# #use "topfind";;
# #thread;;
# #require "core";;
# #require "async";;
# open Core.Std;;

In Oloop, we factor out eval_error from error purely because with sexp doesn't work on exn. Is there really no workaround? How come it is possible to convert exn to and from Error.t, and Error.t supports sexp?

Right now we represent with type (float * string) list, which associates part numbers with the content of that part. We probably also want to detect which phrases are directives vs regular OCaml code.

The _oasis file lists all but the main Oloop module as InternalModules. This is causing some typing problems, which I resolved temporarily by exposing all modules in c465761.

To reproduce the problem, path pin the repo and git checkout 95fa79a, the commit before c465761. Then try the following two cases:

cd to your working directory and launch utop from there, which will cause the checked-in .ocamlinit file to be used:

utop # let f (x:Oloop.Script.Evaluated.phrase) = match x.Oloop.Script.Evaluated.outcome with `Eval _ | `Uneval _ -> ();;
val f : Oloop.Script.Evaluated.phrase -> unit = <fun>

Now, opam install oloop and try the same thing with the installed version of oloop:

utop # #require "oloop";;
utop # let f (x:Oloop.Script.Evaluated.phrase) = match x.Oloop.Script.Evaluated.outcome with `Eval _ | `Uneval _ -> ();;
Error: This pattern matches values of type [? `Eval of 'a ]
          but a pattern was expected which matches values of type
          Oloop.Outcome.merged Oloop_outcome.t 

$ cat ocamlinit 
#use "topfind";;
#thread;;
#require "core async";;

$ utop

utop # Oloop.with_toploop ~init:"ocamlinit" Oloop.Outcome.merged ~f:(fun t -> eval t "open Core_kernel.Std" >>| fun x -> Ok x);;
- : Oloop_outcome.merged Outcome.t Core_kernel.Std.Or_error.t =
Core_kernel.Result.Ok
  (`Uneval
    (`Typetexp
       ({Location.loc_start =
          {Lexing.pos_fname = "//toplevel//"; pos_lnum = 1; pos_bol = 0; pos_cnum = 5};
         loc_end = {Lexing.pos_fname = "//toplevel//"; pos_lnum = 1; pos_bol = 0; pos_cnum = 20};
         loc_ghost = false},
        <abstr>, Typetexp.Unbound_module (Longident.Lident "Core_kernel")),
     "Error: Unbound module Core_kernel\n"))

It's not an issue with relative paths. Passing the absolute path to init doesn't change the behavior.

To avoid having oloop-top depend on sexplib, we avoid adding with sexp to the definition of type error in Oloop_types. However, we want the same definition in Oloop with with sexp. This ends up causing us to duplicate the definitions, but there is no compiler enforcement that they are identical. Is there a way to make it so? If we used regular variants, we could use manifest type:

module A = struct
  type t = A | B
end

type u = A.t = A | B
with sexp

but the analogous idea doesn't work with polymorphic variants. This is a syntax error:

module A = struct
  type t = [`A | `B]
end

type u = A.t = [`A | `B]
with sexp

In our opam file, I've now added "cp" "opam/files/oloop.install" "./" to the install step. But is it necessary, or does the command "ocaml" "setup.ml" "-install" already do that anyway? Or is there some other better solution?

More generally, we used oasis2opam to generate the opam/ directory, but then modified it manually. That seems wrong. If the opam/ directory is auto-generated, maybe we shouldn't include it in the repo. But that wouldn't allow us to do an opam pin, so that also seems wrong. Should we just document that the opam/ directory is auto-generated and should not be manually modified? Is that correct?

Although one can always include the desired directives and module open commands in the input file, it might be convenient to pass these as command line options. For example, someone might want to always load core and async and open Core.Std and Async.Std, without having to write this at the top of their files.

If I opam install oloop and try to use it from utop, it works fine:

$ utop
utop # #require "oloop";;
utop # open Core.Std;;
utop # open Async.Std;;
utop # let f() = Oloop.(with_toploop Output.merged ~f:(fun t -> eval t "2+3" >>| fun _ -> Ok ()));;
val f : unit -> unit Async_kernel.Std.Deferred.Or_error.t = <fun>

However, launching utop from within my working directory, I get a fatal error. Note, this ends up using the .ocamlinit file checked into the repo.

$ utop
utop # let f() = Oloop.(with_toploop Output.merged ~f:(fun t -> eval t "2+3" >>| fun _ -> Ok ()));;
>> Fatal error: f unbound at toplevel
Fatal error: exception Misc.Fatal_error
Raised at file "map.ml", line 117, characters 16-25 

If oloop-top isn't found, we just hang forever. Should immediately get an error.

Right now the script always echos results to stdout and prints them to files. Make the echoing optional since it is annoying when processing many files.

@Chris00 AFAICT oasis2opam is now a dependency and should be added to the opam file. Is that right?

The outcome is empty when giving an invalid directive, e.g.

# Oloop.with_toploop Oloop.Outcome.merged
  ~f:(fun t -> Oloop.eval t "#asdf;;" >>| fun x -> Ok x)
  >>| ok_exn
  >>| function
      | `Uneval _ -> assert false
      | `Eval e -> Oloop.Outcome.(result e, stdout e, warnings e)
;;
- : Outcometree.out_phrase * string * (Location.t * Warnings.t) list = (Outcometree.Ophr_signature [], "", [])

OCaml's toplevel gives:

# #asdf;;
Unknown directive `asdf'.

Oloop ends up hanging if you call it too many times. Here's a session in utop exhibiting the behavior:

utop # let f (x:int) : unit Or_error.t Deferred.t =
  printf "run %d\n" (x+1);
  Oloop.(with_toploop Output.merged ~f:(fun t ->
    eval t "2+3" >>| fun _ -> Ok ()
  ));;
val f : int -> unit Or_error.t Deferred.t = <fun>

utop # Deferred.Or_error.List.iter ~f (List.init 1000 ~f:Fn.id);;
run 1
...
run 93

At this point it hangs. You have to hit Ctrl-C. Then if you call the same line again, you see a resource leak:

utop # Deferred.Or_error.List.iter ~f (List.init 1000 ~f:Fn.id);;
run 1
- : unit Core_kernel.Std.Or_error.t =
Core_kernel.Result.Error                                                                                                              
 (Unix.Unix_error "Too many open files"
 "create_process: stderr->parent pipe creation failed"
 "((prog /Users/ashish/.opam/rwo/bin/oloop-top) (args (--redirect-stderr --sock /var/folders/qd/480pwlg91dg1ggqth6zxl1hm0000gn/T/oloop58c8c7.fifo)) (env (Extend ())))")

We get different errors if #camlp4o is issued or not. IIUC, we are not catching the correct error when camlp4 is used.

Without camlp4o:

$ cat a.ml 
let x-plus-y = x + y;;

$ oloop a.ml
(* part 0 *)
# let x-plus-y = x + y;;
Error: Syntax error

With camlp4o:

$ cat a.ml 
#use "topfind";;
#camlp4o;;
let x-plus-y = x + y;;

$ oloop a.ml -silent-directives
(* part 0 *)
# #use "topfind";;

# #camlp4o;;

# let x-plus-y = x + y;;
Pervasives.Exit

Note the specific error also differs in OCaml's toplevel:

$ ocaml -noinit
        OCaml version 4.02.1

# let x-plus-y = x + y;;
Error: Syntax error

versus:

$ ocaml -noinit
        OCaml version 4.02.1

# #use "topfind";;
# #camlp4o;;
# let x-plus-y = x + y;;
Error: Parse error: [fun_binding] expected after [ipatt] (in [let_binding])

-noinit is not available on 4.01.0. We should delete it since it is anyway not working as mentioned in #25.

# let u : Oloop.Outcome.uneval =
  `Typetexp
    (
      Location.none,
      Env.empty,
      Typetexp.Unbound_value (Longident.Ldot (Longident.Lident "M", "f"))
    )
;;

# Oloop.Outcome.report_uneval Format.str_formatter u;;
Exception: Not_found.

The issue arises only when the identifier includes Ldot. For example, the following works fine:

# let u : Oloop.Outcome.uneval =
  `Typetexp
    (
      Location.none,
      Env.empty,
      Typetexp.Unbound_value (Longident.Lident "f")
    )
;;

# Oloop.Outcome.report_uneval Format.str_formatter u;;
- : string = "File \"_none_\", line 1:\nError: Unbound value f\n" 

Right now we generate 1 output file per part of each input file. Perhaps it would be more convenient to output a single file.

You often don't want to see the output of #require and other directives. Add an option to the command line app that will omit this output.

In oloop_outcome.mli, I claim `Internal_error is returned when the OCaml compiler itself raises an exception. I suspect I'm wrong. @Chris00 can you please let me know the correct description.

I pushed 2b1f495, but the feature doesn't work. Setting working_dir causes oloop to hang.

$ cat stack.ml 
class ['a] stack init = object
  val mutable v = init

  method pop = 
    match v with
    | hd :: tl -> 
      v <- tl;
      Some hd
    | [] -> None

  method push hd = 
    v <- hd :: v
end ;;

$ ./app.native stack.ml 
((pid 85452) (thread_id 0)
 ((human_readable 2015-05-18T15:57:48-0400)
  (int63_ns_since_epoch 1431979068026404000))
 "unhandled exception in Async scheduler"
 ("unhandled exception"
  ((lib/monitor.ml.Error_
    ((exn "Assert_failure lib/conv.ml:487:15")
     (backtrace
      ("Raised at file \"lib/conv.ml\", line 487, characters 15-27"
       "Called from file \"lib/conv.ml\", line 215, characters 26-43"
       "Called from file \"lib/conv.ml\", line 229, characters 17-44"
       "Called from file \"lib/conv.ml\", line 234, characters 8-27"
       "Called from file \"lib/exn.ml\", line 38, characters 50-65"
       "Called from file \"list.ml\", line 73, characters 12-15"
       "Called from file \"list.ml\", line 73, characters 12-15"
       "Called from file \"lib/core_list.ml\", line 328, characters 16-30"
       "Called from file \"lib/result.ml\", line 56, characters 16-19"
       "Called from file \"lib/deferred.ml\", line 15, characters 64-67"
       "Called from file \"lib/job_queue.ml\", line 164, characters 6-47" ""))
     (monitor
      (((name main) (here ()) (id 1) (has_seen_error true)
        (is_detached false) (kill_index 0))))))
   (Pid 85452))))

The outcome of this script ends up being an Uneval of an Internal_error, and the printing of such a value is the source of the above error. If we comment out the exception printer here, then the problem goes away:

$ ./app.native stack.ml 
(* part 0 *)
# class ['a] stack init = object
  val mutable v = init

  method pop = 
    match v with
    | hd :: tl -> 
      v <- tl;
      Some hd
    | [] -> None
  method push hd = 
    v <- hd :: v
end ;;
oloop-top: sending the phrase eval outcome failed because: output_value: functional value

A second bug is that the specific error seems wrong. The expected outcome is:

Error: Some type variables are unbound in this type:
         class ['a] stack :
           'b list ->
           object
             val mutable v : 'b list
             method pop : 'b option
             method push : 'b -> unit
           end
       The method pop has type 'b option where 'b is unbound

When the compilation of a phrase results in a warning or an error message, add information of the location where it was detected.