marty90 / pychromedevtools Goto Github PK
View Code? Open in Web Editor NEWPyChromeDevTools is a python module that allows one to interact with Google Chrome using Chrome DevTools Protocol within a Python script.
License: Apache License 2.0
PyChromeDevTools is a python module that allows one to interact with Google Chrome using Chrome DevTools Protocol within a Python script.
License: Apache License 2.0
I'm getting following error for recent versions of WebView.
Call: PyChromeDevTools.ChromeInterface(host='127.0.0.1', port=37461, timeout=5)
Stack trace:
websock.connect(url, **options)
File "/home/.../.pyenv/versions/venv/lib/python3.10/site-packages/websocket/_core.py", line 253, in connect
self.handshake_response = handshake(self.sock, url, *addrs, **options)
File "/home/.../.pyenv/versions/venv/lib/python3.10/site-packages/websocket/_handshake.py", line 57, in handshake
status, resp = _get_resp_headers(sock)
File "/home/.../.pyenv/versions/venv/lib/python3.10/site-packages/websocket/_handshake.py", line 147, in _get_resp_headers
raise WebSocketBadStatusException("Handshake status %d %s", status, status_message, resp_headers)
websocket._exceptions.WebSocketBadStatusException: Handshake status 403 Forbidden
It sems server is not accepting connection from the localhost. When I've added suppress_origin=True
to create_connection
call it worked.
If I want to fetch real-time http or https traffic, but create a new tag, I can't get the traffic of the new tag using chrome.wait_message()
chrome = ChromeInterface()
chrome.Network.enable()
chrome.Page.enable()
count = 0
while True:
messages = chrome.wait_message()
if messages != None:
try:
print(messages["params"]["response"]["url"])
except:
pass
Question is title. Thanks
Thank you for the code. I would like to ask that how to use it to interact with Chrome console using .py file?
With Chrome in general, I can see the websocket data from Chrome dev tools - network tab - websocket request - Frame tab
.
Is it possible to get these websocket data via PyChromeDevTools
If i go to the url where using websocket?
I often use PyChromeDevTools to debug Android Webview pages, which is an excellent open source library.
Recently, I discovered that due to the upgrade of the Android Webview kernel, PyChromeDevTools cannot connect to Android Webview normally because of the security policy --remote-allow-origins.
However, I couldn't specify Android to use this command line parameter to start Webview, and encountered a similar error: #28.
After a long period of research and attempts, I think PyChromeDevTools needs to be optimized:
In the PyChromeDevTools source code _init_.py, lines 60, 67, and 71,
websocket.create_connection(wsurl)
should be changed to
websocket.create_connection(wsurl, suppress_origin=True)
.
Now let me explain, the modified code blocks the Origin header in the first http connection for WebSocket connection. Chromium does not seem to be restricted by the parameter --remote-allow-origins when the Origin header does not exist.
After this modification, my Android Webview can use PyChromeDevTools to restore normal connection.
The above is my experience sharing, I hope it can help people in need:)
Hi, I open the chrome browser and try to do anything after connecting to the dev tools, but anything I try results in 'socket is already closed'...
Any suggestions?? (I have tried all of those lines and all of them end up throwing the socket closed error.. chrome = PyChromeDevTools.ChromeInterface() - seems to be working fine.
subprocess.call(['open', '-a', "/Applications/Google Chrome.app", '--args', '--remote-debugging-port=9222'])
chrome = PyChromeDevTools.ChromeInterface()
chrome.Network.enable()
# chrome.Page.enable()
# chrome.Page.navigate(url="https://www.google.com")
# chrome.wait_event("Page.loadEventFired", timeout=60000)```
How can i catch ajax request?
Example: Google Maps
After installing the packages as described, tried executing your code in a local py file and my terminal keeps returning "ModuleNotFoundError: No module named 'PyChromeDevTools'". Is there something I'm missing to import the lib?
1、Network.enable().
2、open a webpage , the webpage has a looped video.
3、try Page.getFrameTree()
The code is stuck in an endless loop at self.parent.pop_messages () of generic_function。In pop_messages, it will always receive Network.dataReceived Events
C:/work/harChrome/harCapturer.py:50: DeprecationWarning: use options instead of chrome_options
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()),
Traceback (most recent call last):
File "C:\Python310\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
python-BaseException
Process finished with exit code -1073741510 (0xC000013A: interrupted by Ctrl+C)
Sometimes, I want to visit various pages at the same time.
Used the exact same example from notes. (python 2.7.3) Every website I go to takes 2 seconds to load.
Tested with www.httpbin.org/delay/8 that should take a 8s delay and it also loaded in 2 seconds. how can I troubleshoot to understand what is going on?
chrome.Runtime.evaluate(expression="alert(1111)")
I want to callback do it
ep:
chrome.Page.handleJavaScriptDialog(accept=True)
will be close the alert
Please see title. I know these features are experimental, I was wondering if more information could be given, such as whether these will be included in the final version and if so, what would the estimated time of their completion be?
Thank you, I appreciate your work.
Traceback (most recent call last):
File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
raise err
File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 61] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/urllib3/connectionpool.py", line 356, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output
self.send(msg)
File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send
self.connect()
File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect
conn = self._new_conn()
File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x10b4652b0>: Failed to establish a new connection: [Errno 61] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
timeout=timeout
File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/urllib3/connectionpool.py", line 649, in urlopen
_stacktrace=sys.exc_info()[2])
File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/urllib3/util/retry.py", line 388, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1', port=1234): Max retries exceeded with url: /json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x10b4652b0>: Failed to establish a new connection: [Errno 61] Connection refused',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/aldoec/PycharmProjects/untitled/index.py", line 3, in <module>
chrome = PyChromeDevTools.ChromeInterface(host="127.0.0.1", port=1234)
File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/PyChromeDevTools-0.1-py3.6.egg/PyChromeDevTools/__init__.py", line 43, in __init__
File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/PyChromeDevTools-0.1-py3.6.egg/PyChromeDevTools/__init__.py", line 51, in connect
File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/PyChromeDevTools-0.1-py3.6.egg/PyChromeDevTools/__init__.py", line 46, in get_tabs
File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/requests/api.py", line 72, in get
return request('get', url, params=params, **kwargs)
File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/requests/api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/requests/sessions.py", line 502, in request
resp = self.send(prep, **send_kwargs)
File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/requests/sessions.py", line 612, in send
r = adapter.send(request, **kwargs)
File "/Users/aldoec/.virtualenvs/DjangoTest/lib/python3.6/site-packages/requests/adapters.py", line 504, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=1234): Max retries exceeded with url: /json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x10b4652b0>: Failed to establish a new connection: [Errno 61] Connection refused',))
I am working on a macbook pro Retina, 13-inch, Late 2012. High Sierra OS verision 10.13.15 (17F77)
My problem is that I am attempting to open Chrome with the remote debugging port option and the program stalls in terminal. I have to press ^C, killing the process, because otherwise it just sits there. Here is what I see in my terminal:
(3rdPartyInstagramApp) iphone:debugger-protocol-viewer forestdewberry$ /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --remote-debugging-port=9222 --user-data-dir=remote-profile
DevTools listening on ws://127.0.0.1:9222/devtools/browser/25b29bd9-3b71-4f5a-b2f8-bc97d0ba2ed5
2018-07-15 16:26:42.302 Google Chrome[48614:4393942] *** Owner supplied to -[NSTrackingArea initWithRect:options:owner:userInfo:] referenced a deallocating object. Tracking area behavior is undefined. Break on NSTrackingAreaDeallocatingOwnerError to debug.
2018-07-15 16:26:56.076 Google Chrome[48614:4393942] *** Owner supplied to -[NSTrackingArea initWithRect:options:owner:userInfo:] referenced a deallocating object. Tracking area behavior is undefined. Break on NSTrackingAreaDeallocatingOwnerError to debug.
^C(3rdPartyInstagramApp) iphone:debugger-protocol-viewer forestdewberry
What is causing this error and how do I fix it?
Hi Marty90,
When an Adblock Block Plus extension is added in chrome, the Adblock console can be viewed using inpspect. I would like to get logs of Adblock Plus using your tool. Is it possible to integrate that?
Basically, the code will person:
inpsect > Adblock > select (show blocked, items of all) type and logs the requests to a file.
Thanks!
Errors in the protocol responses should be turned into Python exceptions, so that devs can see details and fix their code.
This happens before the timeout, and leads to the except/break.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "project.py", line 200, in some_method
loadEvent, _ = self._chrome.wait_event('Page.loadEventFired', timeout=60)
File "/path/to/project/PyChromeDevTools/__init__.py", line 84, in wait_event
message = self.ws.recv()
File "/path/to/project/PyChromeDevTools/websocket/_core.py", line 293, in recv
opcode, data = self.recv_data()
File "/path/to/project/PyChromeDevTools/websocket/_core.py", line 310, in recv_data
opcode, frame = self.recv_data_frame(control_frame)
File "/path/to/project/PyChromeDevTools/websocket/_core.py", line 323, in recv_data_frame
frame = self.recv_frame()
File "/path/to/project/PyChromeDevTools/websocket/_core.py", line 357, in recv_frame
return self.frame_buffer.recv_frame()
File "/path/to/project/PyChromeDevTools/websocket/_abnf.py", line 340, in recv_frame
self.recv_header()
File "/path/to/project/PyChromeDevTools/websocket/_abnf.py", line 288, in recv_header
header = self.recv_strict(2)
File "/path/to/project/PyChromeDevTools/websocket/_abnf.py", line 375, in recv_strict
bytes_ = self.recv(min(16384, shortage))
File "/path/to/project/PyChromeDevTools/websocket/_core.py", line 427, in _recv
return recv(self.sock, bufsize)
File "/path/to/project/PyChromeDevTools/websocket/_socket.py", line 83, in recv
raise WebSocketTimeoutException(message)
PyChromeDevTools.websocket._exceptions.WebSocketTimeoutException: timed out
The tool work well when I use it to scrapy the AJAX web. However, I have to click links in the ajax page to load the data I needed. So, I wonder whether there is a way to automatically navigate the page, like what chromedriver and selenium do.
I read Chrome DevTools protocol docs.
After v1.3 it support gracefully close method in Browser domain.
but it doesn't work with PyChromeDevTools.
My chrome version is 71.0.3578.98
Thanks.
Hey,
I just wanted to ask, if it is possible for you to publish this package to PyPi? This would ease the installation process. As you have already a setup.py file, this should not be to complicated.
Hi, trying to geolocate the browser doesn't seem to work.
print(chrome.Emulation.canEmulate())
returns {'id': 3, 'result': {'result': True}} - OK
chrome.Emulation.setDeviceMetricsOverride(width=2436, height=1125, fitWindow=True, deviceScaleFactor=1, mobile=True)
returns a landscaped bigscreen - OK
but...
chrome.Emulation.setGeolocationOverride(latitude=27, longitude=78, accuracy=100)
then
chrome.Page.navigate(url="https://www.google.com/maps")
returns Google maps at @40.4992353,-3.8787972,16z, my IP coordinates.
Is it possible to override geolocation with the protocol viewer?
Thank you
You can connect directly to a targetID using the method connect_targetID. You must pass the targetID as parameter to this function. In this case, you avoid querying the json and finding the tab id.
the "tab id" is webSocketDebuggerUrl?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.