Comments (6)
These are amazing (a/k/a awful) performance issues, thanks for finding them!
from libdash.
Another source of inefficiency in parse_tilde
is that OCaml by default has eager evaluation, which means implode
will be called at every per-character iteration of parse_tilde
, instead of just once when the ret
value is actually needed
i.e., for an n-character input, implode
will be about n times, leading to O(n^2) runtime.
Code snippet where this happens in ast.ml:
parse_tilde acc =
let ret = if acc = [] then None else Some (implode acc) in
function
| [] -> (ret , [])
...
| c::s' -> parse_tilde (acc @ [c]) s'
To show this, I changed dash.ast
to add a printf
to the implode
function:
let implode l =
let s = Bytes.create (List.length l) in
let rec imp i l =
match l with
| [] -> ()
| (c::l) -> (Bytes.set s i c; imp (i+1) l)
in
imp 0 l;
Printf.printf "implode created: %s\n" s;
Bytes.unsafe_to_string s
Output:
/pash/compiler/parser# echo "~lovecraft" | ./parse_to_json.native
implode created: l
implode created: lo
implode created: lov
implode created: love
implode created: lovec
implode created: lovecr
implode created: lovecra
implode created: lovecraf
implode created: lovecraft
["Command",[1,[],[[["T",["Some","lovecraft"]]]],[]]]
There is a "Lazy" module in OCaml, or just use Haskell 🙃
from libdash.
😬 😅
from libdash.
Linear-time versions of parse_tilde
and to_assign
(changing list append to list prepend + one-off reverse, and removing the eager implode in parse_tilde):
and maybe_implode_rev acc =
if acc = [] then None else Some (implode (List.rev acc))
and parse_tilde acc =
function
| [] -> (maybe_implode_rev acc, [])
(* CTLESC *)
| '\129'::_ as s -> None, s
(* CTLQUOTEMARK *)
| '\136'::_ as s -> None, s
(* terminal: CTLENDVAR, /, : *)
| '\131'::_ as s -> maybe_implode_rev acc, s
| ':'::_ as s -> maybe_implode_rev acc, s
| '/'::_ as s -> maybe_implode_rev acc, s
(* ordinary char *)
(* TODO 2019-01-03 only characters from the portable character set *)
| c::s' -> parse_tilde (c :: acc) s'
- The copy-and-pasted
maybe_implode_rev acc
is not as pretty as havingret
(or a lazy version of it), but it's fast.
and to_assign v = function
| [] -> failwith ("Never found an '=' sign in assignment, got " ^ implode (List.rev v))
| C '=' :: a -> (implode (List.rev v),a)
| C c :: a -> to_assign (c :: v) a
| _ -> failwith "Unexpected special character in assignment"
from libdash.
Relatedly, parse_arg
and arg_char
are using the call stack when they shouldn't need to.
from libdash.
After #17 merges, the only remaining issue is the pervasively expensive string concatenation. Converting to_string
to use Buffer should resolve this last issue.
from libdash.
Related Issues (17)
- Incorrect handling of single-quoted escape sequences in `ast.ml`
- Add OPAM testing to CI HOT 1
- Abandon dynamic linking HOT 6
- Unify AST with Smoosh
- Prompt callbacks HOT 1
- Better pretty printing HOT 2
- AST mishandles for loop arguments
- Build on macOS produces funny .so files HOT 1
- Proper documentation of AST
- Migrating python bindings from PaSh
- Shell AST server HOT 1
- Failed building wheel for libdash on Apple Silicon
- Completion of error handling HOT 1
- Use shasta to return python AST objects
- Move to binpash organization
- Add `CTLMBCHAR` support
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from libdash.