Rails CI using Ruby 3.2.0-dev fails https://buildkite.com/rails/rails/builds/91200#0184aae9-a971-4423-8bb6-60e7a14ec3fb/1048-1057

Investigated this failure and it is likely due to cgi behavior change between 0.3.3 and 0.3.4 because cgi default gem version has bumped to 0.3.5 recently https://www.ruby-lang.org/en/news/2022/11/22/http-response-splitting-in-cgi-cve-2021-33621/

Steps to reproduce

require 'cgi'
CGI::Cookie.new('domain'=>'.example.com', 'name'=>'name1')

Expected behavior

It returns []

% gem install cgi -v 0.3.3
Fetching cgi-0.3.3.gem
Building native extensions. This could take a while...
Successfully installed cgi-0.3.3
Parsing documentation for cgi-0.3.3
Installing ri documentation for cgi-0.3.3
Done installing documentation for cgi after 0 seconds
1 gem installed
% irb
irb(main):001:0> require 'cgi'
irb(main):002:0> CGI::Cookie.new('domain'=>'.example.com', 'name'=>'name1')
=> []
irb(main):003:0>

Actual behavior

It raises ArgumentError

% gem install cgi -v 0.3.4
Fetching cgi-0.3.4.gem
Building native extensions. This could take a while...
Successfully installed cgi-0.3.4
Parsing documentation for cgi-0.3.4
Installing ri documentation for cgi-0.3.4
Done installing documentation for cgi after 0 seconds
1 gem installed
% irb
irb(main):001:0> require 'cgi'
irb(main):002:0> CGI::Cookie.new('domain'=>'.example.com', 'name'=>'name1')
/Users/yahonda/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/cgi-0.3.4/lib/cgi/cookie.rb:128:in `domain=': invalid domain: ".example.com" (ArgumentError)
	from /Users/yahonda/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/cgi-0.3.4/lib/cgi/cookie.rb:95:in `initialize'
	from (irb):2:in `new'
	from (irb):2:in `<main>'
	from /Users/yahonda/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/irb-1.4.2/exe/irb:11:in `<top (required)>'
	from /Users/yahonda/.rbenv/versions/3.1.2/bin/irb:25:in `load'
	from /Users/yahonda/.rbenv/versions/3.1.2/bin/irb:25:in `<main>'
irb(main):003:0>

The release of Ruby 3.0.2 shipped changes to the CGI library, including (but not limited to) #4, changes that came from a JRuby contributor. However, Ruby 3.0.2 claims to be shipping the 0.2.0 released gem.

The gem needs to be updated and released.

JRuby only sources gem-based standard libraries from released gems, and when the released gem does not match what the Ruby version claims, we have no way to install the correct library.

I looked at the following 2 specifications and got the following information:

2.2 Basic Rules

4.1.1. Syntax

The token and path cannot contain CTL. The regular expressions of the two are as follows:

TOKEN_RE = %r"\A[[!-~]&&[^()<>@,;:\\\"/?=\[\]{}]]+\z"
PATH_VALUE_RE = %r"\A[[ -~]&&[^;]]*\z"

What's the difference between the following two uses? How do you match CTLs?

[!-~]
[ -~]

The rspec tests of our Rails app started failing with an ArgumentError after upgrading to 2.7.7. On inspection, the issue seems to be caused by CGI::Cookie.domain=:

def domain=(str)
      if str and ((str = str.b).bytesize > 255 or !DOMAIN_VALUE_RE.match?(str))
        raise ArgumentError, "invalid domain: #{str.dump}"
      end
      @domain = str
    end

Setting a breakpoint:

0> str
=> ".example.com"

0> DOMAIN_VALUE_RE
=> /\A(?(?!-)[-A-Za-z0-9]+(?<!-))(?:.\g)*\z/

0> DOMAIN_VALUE_RE.match?(str)
=> false

0> DOMAIN_VALUE_RE.match?('example.com')
=> true

Not sure if this is the right place to post this but I haven't seen any other record of this occurring.

We're running a Rails 7 app and after updating from Ruby 3.1.2 to 3.1.3 a number of our integration specs failed with the following:

visit login_path # Example calling code.

ArgumentError: invalid domain: ".example.com"
from /Users/me/.rbenv/versions/3.1.3/lib/ruby/3.1.0/cgi/cookie.rb:128:in `domain='

Reverting back to Ruby 3.1.2 produces no such error.

Again, this might actually be an issue with rspec or capybara or something in between but since the error is being thrown from cgi, I thought I would post it here.

Let me know what I can do to help test and resolve.

Thanks!

Joshua

We were informed today of the new CVE-2021-33621 and want to update the copy of cgi shipped with JRuby 9.3. Unfortunately Ruby 2.6 has not been patched and I cannot find the related branches for older versions of CGI anywhere.

What patch went into 0.1.0.2 and friends? Where are the branches for those release lines?

The changes in v0.1.0.2 do not appear to pass RubySpec. Either the library needs to be fixed or the specs need to be updated.

See https://github.com/jruby/jruby/actions/runs/3535817028/jobs/5934245109 for an example job that failed on JRuby after updating to v0.1.0.2.

ext/cgi/escape.c uses C99 features, which cannot use before ruby 3.0.

• Designated Initializers for html_escape_table
• Variable declaration after code in optimized_escape_html

The latter is easy to make C89-compliant but the former needs to expand the table.

The class of CGI.escapeHTML result is not consistent for escaped/non-escaped cases.

require 'cgi/escape'
class S<String;end
p %w[a &].map {|c|[c, CGI.escapeHTML(S.new(c)).class]}.to_h #=> {"a"=>S, "&"=>String}

Probably, should be String always?

the gem claims to support ruby 2.5+, but on ruby 2.5/2.6 it fails on missing dependencies

make: *** No rule to make target '/opt/hostedtoolcache/Ruby/2.6.8/x64/include/ruby-2.6.0/ruby/assert.h', needed by 'escape.o'.  
Stop.

actually, the gem doesn't compile extensions at all because there's a missing line in cgi.gemspec

spec.extensions    = ["ext/cgi/escape/extconf.rb"]

I think it's a bug or is it intentional?

The test added in 107a0c6 contains a test that uses `*h to merge two string hashes. This does not work under Ruby 2.6 and earlier, making it impossible to test v0.1.0.2 on any Ruby 2.6-compatible implementation.

Error: test_cgi_cookie_new_with_domain(CGICookieTest): TypeError: hash key "name" is not a Symbol
/home/enebo/work/gems/cgi/test/cgi/test_cgi_cookie.rb:65:in `test_cgi_cookie_new_with_domain'
     62: 
     63:   def test_cgi_cookie_new_with_domain
     64:     h = {'name'=>'name1', 'value'=>'value1'}
  => 65:     cookie = CGI::Cookie.new('domain'=>'a.example.com', **h)
     66:     assert_equal('a.example.com', cookie.domain)
     67: 
     68:     cookie = CGI::Cookie.new('domain'=>'1.example.com', **h)