sparklemotion / http-cookie Goto Github PK
View Code? Open in Web Editor NEWA Ruby library to handle HTTP cookies in a way both compliant with RFCs and compatible with today's major browsers
License: MIT License
A Ruby library to handle HTTP cookies in a way both compliant with RFCs and compatible with today's major browsers
License: MIT License
Checking that the uri instance descends from URI::HTTPS is insufficient:
http-cookie/lib/http/cookie.rb
Line 590 in 405a48b
Instead, this needs to check the actual scheme of the uri instance (or something else more meaningful). Faraday, for example, uses a URI::HTTP instance with scheme set to https for its secure connections and as a result can't work with this library and secure cookies.
the cookies are kept by 3 domains:
a.com
x.a.com
y.a.com
by calling jar.save
"a.com" and "x.a.com" are saved, but 'y.a.com' is left.
what shall i submit to describe the problem precisely?
Hi!
I'm trying to package http-cookie for arch linux. During rake test --trace
I get the following error:
/usr/bin/ruby -w -I"lib" /usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/rake_test_loader.rb "test/test_http_cookie.rb" "test/test_http_cookie_jar.rb"
/build/ruby-http-cookie/src/http-cookie-1.0.4/test/test_http_cookie.rb:752: warning: ambiguous first argument; put parentheses or a space even after `-' operator
Loaded suite /usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/rake_test_loader
Started
................................................E
===============================================================================
Error: test_yaml_expires(TestHTTPCookie): Psych::DisallowedClass: Tried to load unspecified class: HTTP::Cookie
/usr/lib/ruby/gems/3.0.0/gems/psych-4.0.3/lib/psych/class_loader.rb:99:in `find'
/usr/lib/ruby/gems/3.0.0/gems/psych-4.0.3/lib/psych/class_loader.rb:28:in `load'
/usr/lib/ruby/gems/3.0.0/gems/psych-4.0.3/lib/psych/visitors/to_ruby.rb:424:in `resolve_class'
/usr/lib/ruby/gems/3.0.0/gems/psych-4.0.3/lib/psych/visitors/to_ruby.rb:213:in `visit_Psych_Nodes_Mapping'
/usr/lib/ruby/gems/3.0.0/gems/psych-4.0.3/lib/psych/visitors/visitor.rb:30:in `visit'
/usr/lib/ruby/gems/3.0.0/gems/psych-4.0.3/lib/psych/visitors/visitor.rb:6:in `accept'
/usr/lib/ruby/gems/3.0.0/gems/psych-4.0.3/lib/psych/visitors/to_ruby.rb:35:in `accept'
/usr/lib/ruby/gems/3.0.0/gems/psych-4.0.3/lib/psych/visitors/to_ruby.rb:318:in `visit_Psych_Nodes_Document'
/usr/lib/ruby/gems/3.0.0/gems/psych-4.0.3/lib/psych/visitors/visitor.rb:30:in `visit'
/usr/lib/ruby/gems/3.0.0/gems/psych-4.0.3/lib/psych/visitors/visitor.rb:6:in `accept'
/usr/lib/ruby/gems/3.0.0/gems/psych-4.0.3/lib/psych/visitors/to_ruby.rb:35:in `accept'
/usr/lib/ruby/gems/3.0.0/gems/psych-4.0.3/lib/psych.rb:335:in `safe_load'
/usr/lib/ruby/gems/3.0.0/gems/psych-4.0.3/lib/psych.rb:370:in `load'
/build/ruby-http-cookie/src/http-cookie-1.0.4/test/test_http_cookie.rb:1083:in `test_yaml_expires'
1080: assert_equal false, cookie.session?
1081: assert_equal nil, cookie.max_age
1082:
=> 1083: ycookie = YAML.load(cookie.to_yaml)
1084: assert_equal false, ycookie.session?
1085: assert_equal nil, ycookie.max_age
1086: assert_in_delta cookie.expires, ycookie.expires, 1
===============================================================================
With the following trace:
Command failed with status (1): [ruby -w -I"lib" /usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/rake_test_loader.rb "test/test_http_cookie.rb" "test/test_http_cookie_jar.rb" ]
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/testtask.rb:130:in `block (3 levels) in define'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/file_utils.rb:57:in `sh'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/file_utils.rb:104:in `ruby'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/testtask.rb:117:in `block (2 levels) in define'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/file_utils_ext.rb:58:in `verbose'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/testtask.rb:111:in `block in define'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `each'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `synchronize'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/task.rb:188:in `invoke'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:160:in `invoke_task'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `each'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block in top_level'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:125:in `run_with_threads'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:110:in `top_level'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:83:in `block in run'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:186:in `standard_exception_handling'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:80:in `run'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/usr/bin/rake:23:in `load'
/usr/bin/rake:23:in `<main>'
Tasks: TOP => test
Do you have any clue what's broken here?
We discovered a bug in http-cookie while investigating jruby/jruby#5910.
In AbstractStore
, (and also in AbstractSaver
I believe) you have code like this:
...
class << self
@@class_map = {}
# Gets an implementation class by the name, optionally trying to
# load "http/cookie_jar/*_store" if not found. If loading fails,
# IndexError is raised.
def implementation(symbol)
@@class_map.fetch(symbol)
rescue IndexError
begin
require 'http/cookie_jar/%s_store' % symbol
@@class_map.fetch(symbol)
rescue LoadError, IndexError
raise IndexError, 'cookie store unavailable: %s' % symbol.inspect
end
end
def inherited(subclass) # :nodoc:
@@class_map[class_to_symbol(subclass)] = subclass
end
def class_to_symbol(klass) # :nodoc:
klass.name[/[^:]+?(?=Store$|$)/].downcase.to_sym
end
end
...
This is not thread-safe in any implementation and is the true cause of our variable object widths.
The problem here is that the inherited
hook is called immediately once the HashStore
and other subclasses extend AbstractStore
. In other words, the class goes into this @@class_map
before any of its methods have been defined! As a result, another thread might see and try to instantiate the class before it is fully initialized.
This is the reason we had unpredictable results in jruby/jruby#5910; depending on when the second thread starts inspecting the class, it will see none, some, or all of HashStore
's methods defined, giving us a different view of instance variables.
It actually gets worse, though, because it's possible for a second thread to start instantiating HashStore
before it is fully defined, which causes AbstractStore
's initialize to call the default version of default_options
that returns nil, and you will get errors like this:
...
NoMethodError: undefined method `each_pair' for nil:NilClass
initialize at /Users/headius/projects/jruby/lib/ruby/gems/shared/gems/http-cookie-1.0.3/lib/http/cookie_jar/abstract_store.rb:52
repro.rb at repro.rb:5
Similar to #6:
NoMethodError: undefined method `each_pair' for nil:NilClass
http/cookie_jar/abstract_store.rb:50:in `initialize'
default_options.each_pair { |key, default|
http/cookie_jar.rb:38:in `new'
base.implementation(value).new(*args)
http/cookie_jar.rb:38:in `get_impl'
base.implementation(value).new(*args)
http/cookie_jar.rb:74:in `initialize'
@store = get_impl(AbstractStore, opthash[:store], opthash)
I think it's the automatic loading that is the problem. We should have an option to disable it, for applications that prefer stability, or better yet IMHO, get rid of it altogether.
I've had an issue, when a cookie contains quotes wrapped around the value, the library is stripping out the quotes. Which is causing my authentication to fail. I've created a monkey in my fork, however when I went to create the tests and submit. It seems there is a few failing tests in this area and a bit more complex than I original thought.
def scan_value
''.tap { |s|
case
when scan(/[^,;"]+/)
s << matched
def scan_value
''.tap { |s|
case
when scan(/[^,;]+/)
s << matched
I'll try and get to the bottom of the failing tests, I'm running Ruby 2.1.3
Perhaps it would be worth adding the tests into Travis. Would be more than happy to update if desired?
Hi there,
In yaml_saver.rb
you require psych
for all 1.9.x Rubies but don't declare it in the gemspec.
I don't see a reasonable way of declaring psych for specific Rubies since you won't know what version it'll ultimately be running under, so it might be best to declare it explicitly for everyone, as right now it's exploding at runtime.
Failure/Error: agent.cookie_jar.load("cookies.yaml")
NoMethodError:
undefined method `acceptable?' for #<Hash:0x007f9b4220cf30>
# /Users/UsEr/.rvm/gems/ruby-2.3.0@osso/gems/http-cookie-1.0.3/lib/http/cookie_jar.rb:108:in `add'
# /Users/UsEr/.rvm/gems/ruby-2.3.0@osso/gems/mechanize-2.7.5/lib/mechanize/cookie_jar.rb:22:in `add'
...
I am getting this when trying to run my specs:
/Users/plundberg/.rvm/gems/ruby-2.3.3/gems/http-cookie-1.0.3/lib/http/cookie_jar.rb:18: warning: loading in progress, circular require considered harmful - /Users/plundberg/.rvm/gems/ruby-2.3.3/gems/http-cookie-1.0.3/lib/http/cookie_jar/abstract_store.rb
from /Volumes/extra/git/ecraft/ecraft.uxfactory.time-workers/workers/_lib/trello.rb:21:in `block in load_trello_data'
from /Volumes/extra/git/ecraft/ecraft.uxfactory.time-workers/workers/_lib/trello.rb:55:in `fetch_boards'
from /Volumes/extra/git/ecraft/ecraft.uxfactory.time-workers/workers/_lib/trello.rb:113:in `perform_get_request'
from /Users/plundberg/.rvm/gems/ruby-2.3.3/gems/rest-client-2.0.0/lib/restclient.rb:67:in `get'
from /Users/plundberg/.rvm/gems/ruby-2.3.3/gems/rest-client-2.0.0/lib/restclient/request.rb:52:in `execute'
from /Users/plundberg/.rvm/gems/ruby-2.3.3/gems/rest-client-2.0.0/lib/restclient/request.rb:52:in `new'
from /Users/plundberg/.rvm/gems/ruby-2.3.3/gems/rest-client-2.0.0/lib/restclient/request.rb:136:in `initialize'
from /Users/plundberg/.rvm/gems/ruby-2.3.3/gems/rest-client-2.0.0/lib/restclient/request.rb:391:in `process_cookie_args!'
from /Users/plundberg/.rvm/gems/ruby-2.3.3/gems/rest-client-2.0.0/lib/restclient/request.rb:391:in `new'
from /Users/plundberg/.rvm/gems/ruby-2.3.3/gems/http-cookie-1.0.3/lib/http/cookie_jar.rb:74:in `initialize'
from /Users/plundberg/.rvm/gems/ruby-2.3.3/gems/http-cookie-1.0.3/lib/http/cookie_jar.rb:18:in `const_missing'
from /Users/plundberg/.rvm/gems/ruby-2.3.3/gems/http-cookie-1.0.3/lib/http/cookie_jar.rb:18:in `require'
/Users/plundberg/.rvm/gems/ruby-2.3.3/gems/http-cookie-1.0.3/lib/http/cookie_jar/abstract_store.rb:18: warning: loading in progress, circular require considered harmful - /Users/plundberg/.rvm/gems/ruby-2.3.3/gems/http-cookie-1.0.3/lib/http/cookie_jar/hash_store.rb
from /Volumes/extra/git/ecraft/ecraft.uxfactory.time-workers/workers/_lib/trello.rb:21:in `block in load_trello_data'
from /Volumes/extra/git/ecraft/ecraft.uxfactory.time-workers/workers/_lib/trello.rb:55:in `fetch_boards'
from /Volumes/extra/git/ecraft/ecraft.uxfactory.time-workers/workers/_lib/trello.rb:113:in `perform_get_request'
from /Users/plundberg/.rvm/gems/ruby-2.3.3/gems/rest-client-2.0.0/lib/restclient.rb:67:in `get'
from /Users/plundberg/.rvm/gems/ruby-2.3.3/gems/rest-client-2.0.0/lib/restclient/request.rb:52:in `execute'
from /Users/plundberg/.rvm/gems/ruby-2.3.3/gems/rest-client-2.0.0/lib/restclient/request.rb:52:in `new'
from /Users/plundberg/.rvm/gems/ruby-2.3.3/gems/rest-client-2.0.0/lib/restclient/request.rb:136:in `initialize'
from /Users/plundberg/.rvm/gems/ruby-2.3.3/gems/rest-client-2.0.0/lib/restclient/request.rb:391:in `process_cookie_args!'
from /Users/plundberg/.rvm/gems/ruby-2.3.3/gems/rest-client-2.0.0/lib/restclient/request.rb:391:in `new'
from /Users/plundberg/.rvm/gems/ruby-2.3.3/gems/http-cookie-1.0.3/lib/http/cookie_jar.rb:74:in `initialize'
from /Users/plundberg/.rvm/gems/ruby-2.3.3/gems/http-cookie-1.0.3/lib/http/cookie_jar.rb:38:in `get_impl'
from /Users/plundberg/.rvm/gems/ruby-2.3.3/gems/http-cookie-1.0.3/lib/http/cookie_jar/abstract_store.rb:15:in `implementation'
from /Users/plundberg/.rvm/gems/ruby-2.3.3/gems/http-cookie-1.0.3/lib/http/cookie_jar/abstract_store.rb:18:in `rescue in implementation'
from /Users/plundberg/.rvm/gems/ruby-2.3.3/gems/http-cookie-1.0.3/lib/http/cookie_jar/abstract_store.rb:18:in `require'
Can you pleas detach URI calls into a configurable proc or adapter, as some sites has UTF-8 (non standard) chars inside URL
Som my temporary fix is:
invalid_uri_fix = Module.new do
def URI(url)
encoded_url = URI.encode(url.to_s)
URI.parse(encoded_url)
end
end
HTTP::CookieJar.prepend(invalid_uri_fix)
HTTP::Cookie.extend(invalid_uri_fix)
The perfect solution i see will be the HTTP::Cookie.default_uri_parser
accessor:
# default
HTTP::Cookie.default_uri_parser ->(value) do
URI.parse(value)
end
# user
HTTP::Cookie.default_uri_parser ->(value) do
Addressable::URI.parse(value)
end
It is possible for a cookie to contains a comma in its value.
This is the real header I found in the wild:
Set-Cookie: token=2e732538-f38e-ae67-8217-5330686854ee,35cfb541def868b6778c0dc45e368322,hWiqTu7qtWt6wBbcUzWqK018Zb1iA5CeHPafl+7nFyAlE3XzPFCFsav2QrmFcSTLlnYNFvlS/7PCsJf9u+ychIZQFY4JGZb4dSoEIgaLtNKWEcD+/hHrAnUEaAtOg7ChfTbN42PS3wPhxOtAI5RcLw==; expires=Thu, 21-Jan-2021 05:20:32 GMT; path=/; secure; HttpOnly
Chrome parses it as
name: token
value: 2e732538-f38e-ae67-8217-5330686854ee,35cfb541def868b6778c0dc45e368322,hWiqTu7qtWt6wBbcUzWqK018Zb1iA5CeHPafl+7nFyAlE3XzPFCFsav2QrmFcSTLlnYNFvlS/7PCsJf9u+ychIZQFY4JGZb4dSoEIgaLtNKWEcD+/hHrAnUEaAtOg7ChfTbN42PS3wPhxOtAI5RcLw==
However, this gem parses it as
name: 35cfb541def868b6778c0dc45e368322,hWiqTu7qtWt6wBbcUzWqK018Zb1iA5CeHPafl+7nFyAlE3XzPFCFsav2QrmFcSTLlnYNFvlS/7PCsJf9u+ychIZQFY4JGZb4dSoEIgaLtNKWEcD+/hHrAnUEaAtOg7ChfTbN42PS3wPhxOtAI5RcLw
value: =
Which cause errors.
Installing ri documentation for http-cookie-1.0.2...
Before reporting this, could you check that the file you're documenting
has proper syntax:
/Users/republicofapps/.rvm/rubies/ruby-1.9.3-p484/bin/ruby -c lib/http/cookie.rb
RDoc is not a full Ruby parser and will fail when fed invalid ruby programs.
The internal error was:
(NoMethodError) undefined method `last' for nil:NilClass
ERROR: While generating documentation for http-cookie-1.0.2
... MESSAGE: undefined method `last' for nil:NilClass
... RDOC args: --ri --op /Users/republicofapps/.rvm/gems/ruby-1.9.3-p484@herokucedar/doc/http-cookie-1.0.2/ri lib README.md LICENSE.txt --title http-cookie-1.0.2 Documentation --quiet
test_erroneous_store
and test_erroneous_saver
fail, including the message
"/private/var/folders/6q/m7fn2jj52y52cfcy2qc588jw0000gp/T/d20181127-8321-1k21rfr/http/cookie_jar/erroneous_saver.rb"]> was expected to include
<"/var/folders/6q/m7fn2jj52y52cfcy2qc588jw0000gp/T/d20181127-8321-1k21rfr/http/cookie_jar/erroneous_saver.rb">.
Because the cookie jar uses a Monitor, it cannot be marshaled.
[3] pry(main)> Marshal.dump(HTTP::CookieJar.new)
TypeError: no _dump_data is defined for class Monitor
from (pry):3:in `dump'
[4] pry(main)>
I filed a bug with the ruby regarding the error for Marshaling Monitor. They expressed that this is intended behavior and Monitor cannot be properly Marshaled, which means that cookie-jar should probably handle this error
https://bugs.ruby-lang.org/issues/17334
I'm using mechanize for some automation purposes and noticed that a Cookie value is not correctly received on Tomcat 7.
Mechanize sends:
Cookie: COOKIE_NAME=/context/UI/Login?xyz=abcd
Tomcat 7 treats ? as a cookie separator while parsing and thus only receives COOKIE_NAME => /context/UI/Login
.
Current browsers treat ? also as separator and send the cookie value quoted:
COOKIE_NAME="/context/UI/Login?xyz=abcd"
Mechanize/http-cookie only treats some control characters and ,;\ as delimiters to determine whether cookie values should be quoted:
http-cookie/lib/http/cookie/scanner.rb
Line 13 in 405a48b
It seems the cookie handling is a complex topic and the delimiters are not clearly specified. When I look at Tomcat's cookie source code, they have different scenarios where they treat even more characters as delimiters (i.e. all HTTP RFC2616 token delimiters, which would include ?/(){} etc.)
I suggest we add these token delimiters in the RE_BAD_CHAR
regexp so containing strings get quoted; I think it won't break things if we foresightfully add some more quotes (I don't see a case where additional quotes would cause a problem).
For now, I'm monkey patching the cookie library to work around this:
require 'mechanize'
HTTP::Cookie::Scanner::RE_BAD_CHAR = /([\x00-\x20\x7F",;\\\?])/
Thanks for your great work!
I found the following error in my heroku logfiles:
NoMethodError: undefined method `implementation' for HTTP::CookieJar::AbstractStore:Class
/app/vendor/bundle/ruby/2.2.0/gems/http-cookie-1.0.2/lib/http/cookie_jar.rb:38:in `get_impl'
/app/vendor/bundle/ruby/2.2.0/gems/http-cookie-1.0.2/lib/http/cookie_jar.rb:74:in `initialize'
/app/vendor/bundle/ruby/2.2.0/gems/mechanize-2.7.3/lib/mechanize/http/agent.rb:135:in `new'
/app/vendor/bundle/ruby/2.2.0/gems/mechanize-2.7.3/lib/mechanize/http/agent.rb:135:in `initialize'
/app/vendor/bundle/ruby/2.2.0/gems/mechanize-2.7.3/lib/mechanize.rb:182:in `new'
/app/vendor/bundle/ruby/2.2.0/gems/mechanize-2.7.3/lib/mechanize.rb:182:in `initialize'
Unfortunately I did not log the uri that was queried by my rake task, and the problem did not occur in the following hours. I still hope it's enough info to find the culprit.
domain_name transitively pulls in unf which in turn pulls in unf_ext. We've gotten some complaints about this in http.rb:
Is it really necessary, and is the transitive dependency on a native extension worth it?
There are tests which check for a valid domain, e.g.: https://github.com/sparklemotion/http-cookie/blob/master/lib/http/cookie.rb#L572
Thats good, but it is a blocker if http is used over a unixsocket.
Is there a possiblity to work around this test on a proper way?
For quick testing I just replaced the "raise" in line 572 with "true", which "fixes" the issue.
Can you guys describe how to update this type of code? It's not longer possible to use the dump_cookiestxt
and load_cookiestxt
methods
def self.get_cookies(agent)
s = StringIO.new
agent.cookie_jar.dump_cookiestxt(s)
s.string
end
#Load +agent+ with cookies from +cookie_string+
def self.load_cookies(agent, cookie_string)
agent.cookie_jar.load_cookiestxt(StringIO.new(cookie_string)) if !cookie_string.blank?
end
What I used to do was pass around cookies in txt form. How would I rework these functions to be able to do that again?
Just found out this gem, seems that it one of my dependences for quite a long time and used more than 100k repositories according to GitHub
I think integration rubocop
in CI and fixing majority of issues will be positive thing for this library?
If product owners are not against this idea I can try to work on this task
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.