aprikyan / google-books-downloader Goto Github PK
View Code? Open in Web Editor NEWAn open-source utility to scrape Google Books
License: MIT License
An open-source utility to scrape Google Books
License: MIT License
I didn't get a cookie? What could have went wrong? How to debug further?
(P.S. I updated Chromedriver to 94 for myself)
Traceback (most recent call last):
File "C:\google-books-downloader-master\gbd.py", line 248, in
step2()
File "C:\google-books-downloader-master\gbd.py", line 216, in step2
cookie = get_cookie(list(all_pages.items())[0][1])
File "C:\google-books-downloader-master\gbd.py", line 140, in get_cookie
return cookies[0]
IndexError: list index out of range
Traceback (most recent call last):
File "C:\google-books-downloader\gbd.py", line 244, in
driver = webdriver.Chrome("chromedriver.exe", options=chrome_options)
File "C:\Python39\lib\site-packages\seleniumwire\webdriver.py", line 211, in init
super().init(*args, **kwargs)
File "C:\Python39\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 76, in init
RemoteWebDriver.init(
File "C:\Python39\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 157, in init
self.start_session(capabilities, browser_profile)
File "C:\Python39\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "C:\Python39\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Python39\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot find Chrome binary
hi there
thank you for this repo
unfortunately i can not run this script due to running it on mac os
i get this error which about chromedriver.exe
would you mind fixing this for mac os too?
selenium.common.exceptions.WebDriverException: Message: 'chromedriver.exe' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home
best
Hi, I'm stuck at Install, step 2.
$ python --version
Python 2.7.17
$ uname -srm
Linux 4.15.0-20-generic x86_64
$ sudo -H /usr/local/bin/pip3 install -r requirements.txt
Requirement already satisfied: regex in /usr/local/lib/python3.6/dist-packages (from -r requirements.txt (line 1)) (2021.3.17)
Requirement already satisfied: progressbar in /usr/lib/python3/dist-packages (from -r requirements.txt (line 2)) (2.3)
Requirement already satisfied: selenium==3.141.0 in /usr/local/lib/python3.6/dist-packages (from -r requirements.txt (line 3)) (3.141.0)
Collecting selenium-wire>=2.1.0
Downloading selenium_wire-4.6.2-py3-none-any.whl (238 kB)
|████████████████████████████████| 238 kB 2.1 MB/s
Requirement already satisfied: urllib3 in /usr/lib/python3/dist-packages (from selenium==3.141.0->-r requirements.txt (line 3)) (1.22)
Collecting dataclasses>=0.7
Downloading dataclasses-0.8-py3-none-any.whl (19 kB)
Collecting kaitaistruct>=0.7
Downloading kaitaistruct-0.9.tar.gz (5.5 kB)
Preparing metadata (setup.py) ... done
Collecting zstandard>=0.14.1
Downloading zstandard-0.17.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
|████████████████████████████████| 2.5 MB 436 kB/s
Collecting pyparsing>=2.4.2
Downloading pyparsing-3.0.7-py3-none-any.whl (98 kB)
|████████████████████████████████| 98 kB 885 kB/s
Requirement already satisfied: pysocks>=1.7.1 in /usr/local/lib/python3.6/dist-packages (from selenium-wire>=2.1.0->-r requirements.txt (line 4)) (1.7.1)
Collecting h2>=4.0
Downloading h2-4.1.0-py3-none-any.whl (57 kB)
|████████████████████████████████| 57 kB 581 kB/s
Collecting pyOpenSSL>=19.1.0
Downloading pyOpenSSL-22.0.0-py2.py3-none-any.whl (55 kB)
|████████████████████████████████| 55 kB 471 kB/s
Collecting brotli>=1.0.9
Downloading Brotli-1.0.9-cp36-cp36m-manylinux1_x86_64.whl (357 kB)
|████████████████████████████████| 357 kB 3.3 MB/s
Requirement already satisfied: blinker>=1.4 in /usr/lib/python3/dist-packages (from selenium-wire>=2.1.0->-r requirements.txt (line 4)) (1.4)
Collecting wsproto>=0.14
Downloading wsproto-1.0.0-py3-none-any.whl (24 kB)
Collecting hyperframe>=6.0
Downloading hyperframe-6.0.1-py3-none-any.whl (12 kB)
Requirement already satisfied: pyasn1>=0.3.1 in /usr/lib/python3/dist-packages (from selenium-wire>=2.1.0->-r requirements.txt (line 4)) (0.4.2)
Collecting certifi>=2019.9.11
Downloading certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
|████████████████████████████████| 149 kB 3.7 MB/s
Collecting hpack<5,>=4.0
Downloading hpack-4.0.0-py3-none-any.whl (32 kB)
Collecting cryptography>=35.0
Downloading cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
|████████████████████████████████| 3.6 MB 2.4 MB/s
Collecting h11<1,>=0.9.0
Downloading h11-0.13.0-py3-none-any.whl (58 kB)
|████████████████████████████████| 58 kB 779 kB/s
Collecting cffi>=1.12
Downloading cffi-1.15.0-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (405 kB)
|████████████████████████████████| 405 kB 3.1 MB/s
Collecting typing-extensions
Downloading typing_extensions-4.1.1-py3-none-any.whl (26 kB)
Collecting pycparser
Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
|████████████████████████████████| 118 kB 2.7 MB/s
Using legacy 'setup.py install' for kaitaistruct, since package 'wheel' is not installed.
Installing collected packages: pycparser, typing-extensions, dataclasses, cffi, hyperframe, hpack, h11, cryptography, zstandard, wsproto, pyparsing, pyOpenSSL, kaitaistruct, h2, certifi, brotli, selenium-wire
Attempting uninstall: cryptography
Found existing installation: cryptography 2.1.4
Uninstalling cryptography-2.1.4:
Successfully uninstalled cryptography-2.1.4
Running setup.py install for kaitaistruct ... done
Attempting uninstall: certifi
Found existing installation: certifi 2018.1.18
Uninstalling certifi-2018.1.18:
Successfully uninstalled certifi-2018.1.18
Successfully installed brotli-1.0.9 certifi-2021.10.8 cffi-1.15.0 cryptography-36.0.1 dataclasses-0.8 h11-0.13.0 h2-4.1.0 hpack-4.0.0 hyperframe-6.0.1 kaitaistruct-0.9 pyOpenSSL-22.0.0 pycparser-2.21 pyparsing-3.0.7 selenium-wire-4.6.2 typing-extensions-4.1.1 wsproto-1.0.0 zstandard-0.17.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
$ apt list --installed | grep chrome
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
chromium-chromedriver/bionic-updates,bionic-security,now 97.0.4692.71-0ubuntu0.18.04.1 amd64 [installed]
$ python gbd.py
File "gbd.py", line 34
return (f"https://books.google.com/books?{id_part}&pg=1&hl=en#v=onepage&q&f=false",
^
> SyntaxError: invalid syntax
$ cat gbd.py | grep '.exe'
driver = webdriver.Chrome("chromedriver.exe", options=chrome_options)
$ sed -iE '/chromedriver\.exe/ s/\.exe//' gbd.py
$ cat gbd.py | grep 'chromedriver'
driver = webdriver.Chrome("chromedriver", options=chrome_options)
I think it doesn't work with the latest Chrome.
google-books-downloader crash.log
message: unknown error: cannot find chrome binary
I completed all steps of creating the tool on my computer and came to the stage to enter the URL. I live in Australia and my Google book URL looks different to the sample you have given possibly from the US Google Books site. I tried this with a few books but same results. I have attached the crash log, and how I changed the book format to the sample given by you. Can you please assist me to overcome this issue?
Thanks, and regards
Chiran
Crash Log:
C:\Users\winds\AppData\Local\Programs\Python\Python311\google-books-downloader crash.log
A. This is the example that you have provided:
https://books.google.com/books?id=buc0AAAAMAAJ&printsec=frontcover&sa=X&ved=2ahUKEwj-y8T4r5vrAhWKLewKHaIQBnYQ6AEwAXoECAQQAg#v=onepage&f=false
B. This is the book that I want to Download:
https://www.google.com.au/books/edition/Rajarathnakaraya_%E0%B6%BB%E0%B7%8F%E0%B6%A2%E0%B6%BB%E0%B6%AD%E0%B7%8A%E0%B6%B1/va6gEAAAQBAJ?hl=en&gbpv=1
C. This is how I put it into your format With Square Brackets for clarity to show what I changed
https://books.google.com/books?id=[Rajarathnakaraya_%E0%B6%BB%E0%B7%8F%E0%B6%A2%E0%B6%BB%E0%B6%AD%E0%B7%8A%E0%B6%B1/va6gEAAAQBAJ?hl=en&gbpv]=onepage&f=false
What I pasted (without square brackets)
https://books.google.com/books?id=Rajarathnakaraya_%E0%B6%BB%E0%B7%8F%E0%B6%A2%E0%B6%BB%E0%B6%AD%E0%B7%8A%E0%B6%B1/va6gEAAAQBAJ?hl=en&gbpv=onepage&f=false
After installing the required dependencies: python gbd.py
gives the error:
Traceback (most recent call last):
File "google-books-downloader\gbd.py", line 6, in <modu
le>
from seleniumwire import webdriver
File "Python310\site-packages\seleniumwire
\webdriver.py", line 23, in <module>
from selenium.webdriver import EdgeOptions
ImportError: cannot import name 'EdgeOptions' from 'selenium.webdriver' (Python310\site-packages\selenium\webdriver\__init__
.py)
Սևանը չէ՞
Apr useric input vercnelu tex@ argparser avelacru, araga shat u aveli harmara, vor terminalic kanchi python gbd.py --save True xosqi https://docs.python.org/3/library/argparse.html
DISCLAIMER OF RESPONSIBILITY: The code is provided as-is without any further warranty. It is designed solely for legal usage; the author(s) of the code are not responsible for any illegal actions done by anyone using this code. Read the License for more information.
time module @ orinaki hamar standard lib um ka, iran petq chi install anel, regex nel vayte(vstah chem)
Traceback (most recent call last):
File "/usr/lib/python3.9/site-packages/seleniumwire/proxy/proxy2.py", line 91, in proxy_request
conn.request(self.command, path, req_body, dict(req.headers))
File "/usr/lib/python3.9/http/client.py", line 1255, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.9/http/client.py", line 1301, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.9/http/client.py", line 1250, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.9/http/client.py", line 1010, in _send_output
self.send(msg)
File "/usr/lib/python3.9/http/client.py", line 950, in send
self.connect()
File "/usr/lib/python3.9/site-packages/seleniumwire/proxy/proxy2.py", line 368, in connect
super().connect()
File "/usr/lib/python3.9/http/client.py", line 1417, in connect
super().connect()
File "/usr/lib/python3.9/http/client.py", line 921, in connect
self.sock = self._create_connection(
File "/usr/lib/python3.9/socket.py", line 843, in create_connection
raise err
File "/usr/lib/python3.9/socket.py", line 831, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
Error making request
I'm not on Windows. I believe this is the reason why I get this error:
~/bin $ python gbd.py
Google Books Downloader by @aprikyan, 2020.
. . . . . . . . . . .
Traceback (most recent call last):
File "/mnt/Data/bin/gbd.py", line 244, in <module>
driver = webdriver.Chrome("chromedriver.exe", options=chrome_options)
File "/home/simonapetra/.local/lib/python3.10/site-packages/seleniumwire/webdriver.py", line 206, in __init__
super().__init__(*args, **kwargs)
File "/home/simonapetra/.local/lib/python3.10/site-packages/selenium/webdriver/chrome/webdriver.py", line 73, in __init__
self.service.start()
File "/home/simonapetra/.local/lib/python3.10/site-packages/selenium/webdriver/common/service.py", line 72, in start
self.process = subprocess.Popen(cmd, env=self.env,
File "/usr/lib/python3.10/subprocess.py", line 966, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/lib/python3.10/subprocess.py", line 1842, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: 'chromedriver.exe'
Can you confirm or advice how to solve this error, please?
eg. https://books.google.nl/books?id=S-Fu4fGHnlAC
eg. https://books.google.nl/books?id=J2jC7gH_4xAC
can be solved by correcting the regex
def get_book_url():
if re.findall(r"id=[A-Za-z0-9-_]+", url):
id_part = re.findall(r"id=[A-Za-z0-9-_]+", url)[-1]
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.