StackOverflowError when parsing many unclosed tags in series about hickory HOT 8 CLOSED

I also tested this with clojure 1.8.0 and the problem exists there as well. The stack trace is identical except for line numbers.

from hickory.

Hm, yeah, that's gonna do that. The problem is that a protocol is used to convert between different representations of the data. The element node has to first call this protocol function recursively on its children, in order to get the value to add to itself. I will think about this some more, but for right now, I don't see a way to get around this easily.

Typically when parsing a potentially infinitely recursive thing, you could use a stack on the heap and manually manage it, instead of implicitly managing it on the actual stack frames of the functions. But with clojure's immutable data structures, it makes you need to have all the elements in a given data structure already finished before you create the thing that holds them. Maybe there is something we could do with zippers here, or maybe we'd need to go into Java and use mutability for this. I'll need to think about it more.

Let me know if you want to work on this.

from hickory.

In 135faba, I replaced the recursive protocol implementation with an iterative one using zippers as you suggested. This passes all the existing unit tests. So far, this is just for hickory, but I think I could easily do the same for hiccup.

What do you think?

from hickory.

It looks there are some other areas we'd have to convert from recursive to iterative, like hickory.render/hickory-to-html and maybe the hickory.select/select* fns.

from hickory.

Yeah, no doubt. Did you see my comment on your commit?

from hickory.

I did, thanks. I'll have a look at integrating back into the protocol this weekend.

from hickory.

By the way, I was wrong about needing to change the hickory.select/select* fns; they're already iterative and seem to work fine with this pathological case:

(->> (apply str (repeat 2048 "<font>abc"))
     hickory/parse
     hickory/as-hickory
     (select/select (select/tag :font))
     count)
;;=> 2048

from hickory.

I updated #51 with a solution that better preserves the protocol, I think (2d21efa). Let me know what you think.

from hickory.