We might want to write up some docs that present functionality broken down the same way that jQuery's docs do it (which I find totally useful and obvious):

• core (parsing)
• selectors
• attributes
• traversing
• manipulation

Microsoft Word notably generates this kind of HTML

I haven't worked out a way to create a simple example of this, but when I tried to use webrat (which requires nokogiri) I found that my existing tests (about 10 test classes run from rake test:integration) when run together (not singly) caused a seg fault or a bus error (this was before I wrote any specific webrat tests).

So as far as I understand Nokogiri shouldn't even be being called. And when I could deduce where the error occurred it was somewhere deep in the rails framework (but suspiciously in an html parsing context). I tried step-debugging the code, but this gave me an error in a different place.

Running on a OSX 10.5.6, Ruby 1.8.6

My workaround was to downgrade nokogiri to 1.2.2

Sorry this wasn't such a useful issue, but maybe you can give me some guidance as to what information could help narrow down the possible cause.

We’re using the Builder in Babylon to build XML stanzas (XMPP).

After running for a few hours each of our application was gaining a lot of weight… It took some hours to track the memory leaks downs (with Dike), and it seems that it comes from the Nokogiri::XML::Builder.

It seems that all the variables in the context of the caller are “wasted”, not cleaned after execution.

Could that be possible?

Please feel free to contact me if you have any question about this (julien.genestoux AT gmail.com for Gtalk ;) )

Make sure the SAX parsers have RDoc

This is a test.

The Reader API kind of sucks. Make it better.

Hello!

When building a node, it seems that special characters are encoded :

irb(main):004:0> doc = Nokogiri::XML::Document.new
=> <?xml version="1.0"?>

irb(main):005:0> n = Nokogiri::XML::Node.new("toto", doc)
=> <toto/>

irb(main):013:0> n["salut"]= "hum&"
=> "hum&"

irb(main):014:0> doc
=> <?xml version="1.0"?>
<toto salut="hum&amp;"/>

In our application, we actually use a XML Push Parser to parse this content... here is a small bit of the implementtation (see the full implementation here :

def start_element(qname, attributes = [])
  e = Nokogiri::XML::Element.new(qname, @doc)
  add_namespaces_and_attributes_to_node(attributes, e)
  [...]
end
def add_namespaces_and_attributes_to_node(attrs, node) 
  (attrs.size / 2).times do |i|
    name, value = attrs[2 * i], attrs[2 * i + 1]
    node.set_attribute name, value
end

The problem is that the push parser doesn't "unescape" the content of the attributes... and then, when we recreate a new node, these attributes are escaped again. The solution would probabluy be to unescape the content of these attributes before adding them to the node, but I have no idea on how to do this!

Thanks a bunch

This code segfaults ruby and I think that's pretty charming. Haven't dug into it, yet.

h = Nokogiri::HTML.parse("<p></p>")
node = h.at('p')
node.add_next_sibling(node.children)

We should probably rewrite this in C for a speed boost:

http://xmlsoft.org/html/libxml-xpathInternals.html#xmlXPathNodeSetMerge

see http://jira.codehaus.org/browse/JRUBY-3636 for details

There seems to be a problem with parsing if the html tags are irregular.
For example, in an iframe tag, if the 'height' attribute is present but
no value is written for it, then an error occurs saying that "odd number
of arguments for hash".
Please suggest a solution for this problem.

Nokogiri::XML::Reader.from_memory needs access to its first argument for the whole lifetime of the Reader, but doesn't tell the garbage collector about it. SEGFAULTs may or may not happen afterwards, depends how the memory is reused.

> > fx = Nokogiri::XML)
> > fx.xpath(‘/rdf:RDF/item’)
> > => nil
> > fx.xpath(’rdf:RDF/.each do |tag|
> > p tag.name
> > p xchg.path
> > end
> > “channel”
> > “/rdf:RDF/_¹”
> > “item”
> > “/rdf:RDF/_²”
> > “item”
> > "/rdf:RDF/³"
> > “item”
> > “/rdf:RDF/*⁴”
> > …

Nodes can be compared as to where they are in the document tree. Implementing spaceship would let us sort a node set by appearance in the document.

http://xmlsoft.org/html/libxml-xpath.html#xmlXPathCmpNodes

Those methods would be handy for adding attributes to a node.

Add meta encoding getters and setters for HTML::Document

http://xmlsoft.org/html/libxml-HTMLtree.html#htmlSetMetaEncoding

Nokogiri::HTML should return an empty document

This might be nice:

http://xmlsoft.org/html/libxml-xpathInternals.html#xmlXPathHasSameNodes

add_namespace fails when it is passed an empty string. It works when nil though, so maybe you should consider that "" is equivalent to no prefix?

Thx,

Hello,

I have posted the message in Google Groups - http://groups.google.com/group/ruby-talk-google/browse_thread/thread/fde1cde1094e0549/ea7e7d0fe029744d

I wrote the test which now fails in my test case: http://gist.github.com/100313

P.S. Nokogiri::LIBXML_VERSION outputs "2.7.3", Nokogiri::VERSION - "1.2.3".

Adding namespaces to child nodes should work inside default namespaces.

This code raises a RuntimeError with the message "Could not reparent node (xmlDocCopyNode)":

h = Nokogiri::HTML.parse("<p>sometext</p>")
n = h.at('p')
n.add_next_sibling(n.children)

We should probably pick one of these two things to do instead:

1. raise an error that says "Hey, dummy, that's a NodeSet, not a Node. We need you to add just n.children[0] through n.children.last one by one with add_next_sibling"
2. flatten the NodeSet out for the user

We should implement NodeSet intersection:

http://xmlsoft.org/html/libxml-xpathInternals.html#xmlXPathIntersection

Write an rdoc test to make sure that everything has RDoc.

With an XSLT stylesheet that's not well-formed, this result_doc = xslt.transform(doc) brings a segfault (at least in this specific case).

Is there a way to attach a file? For now, it's here:
http://nokogiri.lighthouseapp.com/attachments/116557/noko.rb

Make the HTML SAX interface match the xml interface

we should open JIRA tickets for vararg support in FFI callbacks

then we should format the libxml error messages properly in the error/exception callbacks

We recently switched from Builder to Nokogiri for XML generation, with big speed improvements.

And now we're looking to update XML reading, however, legacy data has some standard incorrect fields that worked with Rails XmlSimple (which uses REXML) but does not work with Nokogiri. Here is an example:

>> xml = "<dummy><data_multiple><1><name>Bob</name></1><2><name>Joe</name></2></data_multiple></dummy>"
>> XmlSimple.xml_in(xml)
=> {"data_multiple"=>[{"1"=>[{"name"=>["Bob"]}], "2"=>[{"name"=>["Joe"]}]}]}
>> Nokogiri::XML(xml)
=> <?xml version="1.0"?>
<dummy><data_mutiple>1&gt;<name>Bob</name></data_mutiple>1&gt;2&gt;<name>Joe</name></dummy>

Note the <1></1> tags are being escaped and thus breaking things.

I'm not sure if this is excepted behavior, or a parsing issue.

copy/paste/run the following code. should explain itself...

require 'rubygems'
require 'nokogiri'

@builder = Nokogiri::HTML::Builder.new do |doc|
doc.html {
doc.head {
doc.title "FOOBAR"
}
doc.body {
# THIS WORKS LIKE A CHARM
doc.send(:h1, "A", { :class => 'foo'})
# THIS ONLY WORKS IF I 'undef p' FIRST
doc.send(:p, "B", { :class => 'bar'})
}
}
end

puts @builder.doc

Well, I have been trying to install nokogiri on my Ubuntu 8.10 which has libxml2 version 2.6.32. Is there a version of nokogiri that works with that libxml2 version?

$ cat test.rb
require "rubygems"
require "nokogiri"

doc = Nokogiri::HTML(File.read(ARGV[0]))
p doc

$ jruby -w test.rb data.html
/Users/francois/Library/Java/JRuby/jruby-1.2.0/lib/ruby/gems/1.8/gems/nokogiri-1.2.3-java/lib/nokogiri/xml/node.rb:180: undefined method next_sibling' for classNokogiri::XML::Node' (NameError)
from /Users/francois/Library/Java/JRuby/jruby-1.2.0/lib/ruby/gems/1.8/gems/nokogiri-1.2.3-java/lib/nokogiri/xml/node.rb:31:in require' from /Users/francois/Library/Java/JRuby/current/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:inrequire'
from /Users/francois/Library/Java/JRuby/jruby-1.2.0/lib/ruby/gems/1.8/gems/nokogiri-1.2.3-java/lib/nokogiri/xml.rb:3
from /Users/francois/Library/Java/JRuby/jruby-1.2.0/lib/ruby/gems/1.8/gems/nokogiri-1.2.3-java/lib/nokogiri/xml.rb:31:in require' from /Users/francois/Library/Java/JRuby/current/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:inrequire'
from /Users/francois/Library/Java/JRuby/jruby-1.2.0/lib/ruby/gems/1.8/gems/nokogiri-1.2.3-java/lib/nokogiri.rb:10
from /Users/francois/Library/Java/JRuby/jruby-1.2.0/lib/ruby/gems/1.8/gems/nokogiri-1.2.3-java/lib/nokogiri.rb:36:in require' from /Users/francois/Library/Java/JRuby/current/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:inrequire'
from test.rb:2

$ jruby -S gem list -l nokogiri

*** LOCAL GEMS ***

nokogiri (1.2.3)

$ jruby --version
jruby 1.2.0 (ruby 1.8.6 patchlevel 287) (2009-03-16 rev 9419) [i386-java]

frequently getting the following error when running on MRI FFI:

/home/mike/code/nokogiri/test/helper.rb:11: libxml version info:     {"nokogiri"=>"1.2.4", "warnings"=>[], "libxml"=>{"platform"=>"ruby",     "binding"=>"ffi", "loaded"=>"2.6.31"}}
Loaded suite -e
Started
...
Finished in 3.285882 seconds.

  1) Error:
test_attribute_roundtrip(TestReader):
RangeError: 0xdb7eb02a is recycled object
    /home/mike/code/nokogiri/lib/nokogiri/ffi/structs/common_node.rb:11:in     `_id2ref'
    /home/mike/code/nokogiri/lib/nokogiri/ffi/structs/common_node.rb:11:in     `ruby_node'
    /home/mike/code/nokogiri/lib/nokogiri/ffi/xml/node.rb:252:in     `wrap'
    /home/mike/code/nokogiri/lib/nokogiri/ffi/xml/node.rb:107:in     `attribute_nodes'
    /home/mike/code/nokogiri/lib/nokogiri/ffi/xml/reader.rb:42:in     `attribute_nodes'
    /home/mike/code/nokogiri/lib/nokogiri/xml/reader.rb:52:in     `attributes'
    ./test/test_reader.rb:142:in `test_attribute_roundtrip'
    /home/mike/code/nokogiri/lib/nokogiri/xml/reader.rb:61:in `call'
    /home/mike/code/nokogiri/lib/nokogiri/xml/reader.rb:61:in `each'
    ./test/test_reader.rb:141:in `test_attribute_roundtrip'

544 tests, 1367 assertions, 0 failures, 1 errors
rake aborted!
Command failed with status (1): [/usr/bin/ruby1.8 -w     -Ilib:ext:bin:test -e ...]

Add html tag information lookup. We should be able to look up html tag information.

http://xmlsoft.org/html/libxml-HTMLparser.html#htmlTagLookup

This might be pretty nice as well:

http://xmlsoft.org/html/libxml-xpathInternals.html#xmlXPathNodeSetContains

We’re duping a node, but not assigning a GC function.

Add documentation to Nokogiri::XML::SAX::Parser

Get nokogiri working with JRuby via FFI

testing

id2ref is slow and may be turned off by default in JRuby 1.4.

discussed with wmeissner, and the probable path is to build an API into FFI that is an hash table containing address => weakref(ruby_object).

Nokogiri should have DTD validation support

i wish i could delete this ticket

From Wayne Meissner:

I noticed in nokogiri, in Document.read_io that you use LibXML.memcpy to transfer data from a ruby string to a native memory buffer.

It'll be much faster to use AbstractMemory#put_bytes(), as the memcpy mapping will triple copy the string data on JRuby.

basically, what happens behind the scene is:

1. The string data is copied to a temporary native memory buffer
2. The temp memory is passed as the parameter
3. memcpy copies the data from the temp memory to the destination memory you specified
4. The temporary native memory is copied back to the ruby string.

compared to put_bytes:

1. The string data is copied once from the ruby byte array backing the ruby string into the destination memory address.
2. There is no step 2.

For large files, this might make a difference.

this code, run under valgrind, demonstrates:

doc = Nokogiri::XML <<-EOHTML
<root>
 <a>
   <b/>
 </a>
</root>
EOHTML

root = doc.at("root")
a = root.at("a")
b = a.at("b")
a.add_next_sibling(b.unlink)

puts doc.to_s

We should probably alias #clone to #dup.

NodeSet#dup is broken. The duplicate set should contain the same nodes as the duplicated set.

I've got two nodesets. When I try:

nodeset1 << nodeset2

... Nokogiri laughs at me with:

ArgumentError: node must be a Nokogiri::XML::Node
    from (irb):29:in `<<'
    from (irb):29
    from :0

Ironically, the elements of a NodeSet are Nokogiri::XML::Elements, which is just an empty subclass of Nokogiri::XML::Node.

It looks like the offending bit is line 52 of xml_node_set.c:

static VALUE push(VALUE self, VALUE rb_node)
{
  xmlNodeSetPtr node_set;
  xmlNodePtr node;

  if(! rb_funcall(rb_node, rb_intern("is_a?"), 1, cNokogiriXmlNode)) // <-- THIS ONE
    rb_raise(rb_eArgError, "node must be a Nokogiri::XML::Node");

  Data_Get_Struct(self, xmlNodeSet, node_set);
  Data_Get_Struct(rb_node, xmlNode, node);
  xmlXPathNodeSetAdd(node_set, node);
  return self;
}