Giter VIP home page Giter VIP logo

Comments (6)

mgree avatar mgree commented on June 6, 2024

These are amazing (a/k/a awful) performance issues, thanks for finding them!

from libdash.

thurstond avatar thurstond commented on June 6, 2024

Another source of inefficiency in parse_tilde is that OCaml by default has eager evaluation, which means implode will be called at every per-character iteration of parse_tilde, instead of just once when the ret value is actually needed
i.e., for an n-character input, implode will be about n times, leading to O(n^2) runtime.

Code snippet where this happens in ast.ml:

parse_tilde acc = 
  let ret = if acc = [] then None else Some (implode acc) in
  function
  | [] -> (ret , [])
  ...
  | c::s' -> parse_tilde (acc @ [c]) s'  

To show this, I changed dash.ast to add a printf to the implode function:

let implode l =
  let s = Bytes.create (List.length l) in
  let rec imp i l =
    match l with
    | []  -> ()
    | (c::l) -> (Bytes.set s i c; imp (i+1) l)
  in
  imp 0 l;
  Printf.printf "implode created: %s\n" s;
  Bytes.unsafe_to_string s

Output:

/pash/compiler/parser# echo "~lovecraft" | ./parse_to_json.native
implode created: l
implode created: lo
implode created: lov
implode created: love
implode created: lovec
implode created: lovecr
implode created: lovecra
implode created: lovecraf
implode created: lovecraft
["Command",[1,[],[[["T",["Some","lovecraft"]]]],[]]]

There is a "Lazy" module in OCaml, or just use Haskell 🙃

from libdash.

mgree avatar mgree commented on June 6, 2024

😬 😅

from libdash.

thurstond avatar thurstond commented on June 6, 2024

Linear-time versions of parse_tilde and to_assign (changing list append to list prepend + one-off reverse, and removing the eager implode in parse_tilde):

and maybe_implode_rev acc =
  if acc = [] then None else Some (implode (List.rev acc))

and parse_tilde acc =
  function
  | [] -> (maybe_implode_rev acc, [])
  (* CTLESC *)
  | '\129'::_ as s -> None, s
  (* CTLQUOTEMARK *)
  | '\136'::_ as s -> None, s
  (* terminal: CTLENDVAR, /, : *)
  | '\131'::_ as s -> maybe_implode_rev acc, s
  | ':'::_ as s -> maybe_implode_rev acc, s
  | '/'::_ as s -> maybe_implode_rev acc, s
  (* ordinary char *)
  (* TODO 2019-01-03 only characters from the portable character set *)
  | c::s' -> parse_tilde (c :: acc) s'
  • The copy-and-pasted maybe_implode_rev acc is not as pretty as having ret (or a lazy version of it), but it's fast.
and to_assign v = function
  | [] -> failwith ("Never found an '=' sign in assignment, got " ^ implode (List.rev v))
  | C '=' :: a -> (implode (List.rev v),a)
  | C c :: a -> to_assign (c :: v) a
  | _ -> failwith "Unexpected special character in assignment"

from libdash.

mgree avatar mgree commented on June 6, 2024

Relatedly, parse_arg and arg_char are using the call stack when they shouldn't need to.

from libdash.

mgree avatar mgree commented on June 6, 2024

After #17 merges, the only remaining issue is the pervasively expensive string concatenation. Converting to_string to use Buffer should resolve this last issue.

from libdash.

Related Issues (17)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.