selenium-python-aws-lambda's People
Forkers
codybushnell shafaypro renburnett w-chase104 pranavmk98 stevezieglerva jamesd000 hoangchunghien junouyang kdw9502 andrei-dolgolev alejandroalffer orseni rahulgunkar akaashhazarika biranchi2018 victor-war kevin-ayling leeshinyook butyear rus19023 arka2010 najibkado jjang750selenium-python-aws-lambda's Issues
chromedriver' executable needs to be in PATH.
I tried uploading your ZIP in Python Lambda
Python version 3.7, 3.8, and 3.6 gives following error
{
"errorMessage": "Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home\n",
"errorType": "WebDriverException",
"stackTrace": [
" File "/var/task/lambda_function.py", line 46, in lambda_handler\n driver = webdriver.Chrome(chrome_options=chrome_options)\n",
" File "/var/task/selenium/webdriver/chrome/webdriver.py", line 68, in init\n self.service.start()\n",
" File "/var/task/selenium/webdriver/common/service.py", line 81, in start\n raise WebDriverException(\n"
]
}
Zip/Unzip no longer works
I downloaded the ZIP, uploaded to AWS, and the lambda ran successfully. If I simply download the zip > extract all > send to compressed file, it no longer works when I upload it. I tracked back to this after trying upload edited code and having it fail.
Is there some step/procedure that I am missing? Some other way of extracting/zipping? I tried through direct upload and S3.
After unzipping and rezipping I get the unable to import lambda_function error (when this python file is still in the folder and unchanged)
{
"errorMessage": "Unable to import module 'lambda_function'"
}
Thanks for any help!
Returning Empty HTML
Hello,
I am able to return the html for the webpage that I am trying to scrape. However, the div that contains all of the information that is shown on the webpage is empty. When I run the code on my local machine with a different chromedriver, it returns all of the information from the page. Would you happen to know what the issue might be?
Thanks
Invalid URL
I'm having problems running test events. In my test event I have the following
{
"url":"www.google.com"
}
when I run test it gives me the following message
{
"errorMessage": "Message: unknown error: unhandled inspector error: {\"code\":-32000,\"message\":\"Cannot navigate to invalid URL\"}\n (Session info: headless chrome=62.0.3202.94)\n (Driver info: chromedriver=2.35.528139 (47ead77cb35ad2a9a83248b292151462a66cd881),platform=Linux 4.14.88-90.76.amzn2.x86_64 x86_64)\n",
"errorType": "WebDriverException",
"stackTrace": [
[
"/var/task/lambda_function.py",
49,
"lambda_handler",
"driver.get(event['url'])"
],
[
"/var/task/selenium/webdriver/remote/webdriver.py",
332,
"get",
"self.execute(Command.GET, {'url': url})"
],
[
"/var/task/selenium/webdriver/remote/webdriver.py",
320,
"execute",
"self.error_handler.check_response(response)"
],
[
"/var/task/selenium/webdriver/remote/errorhandler.py",
242,
"check_response",
"raise exception_class(message, screen, stacktrace)"
]
]
}
along with
START RequestId: ce5f4ee0-0263-484e-95cd-3909c4390241 Version: $LATEST
Starting google.com
Message: unknown error: unhandled inspector error: {"code":-32000,"message":"Cannot navigate to invalid URL"}
(Session info: headless chrome=62.0.3202.94)
(Driver info: chromedriver=2.35.528139 (47ead77cb35ad2a9a83248b292151462a66cd881),platform=Linux 4.14.88-90.76.amzn2.x86_64 x86_64)
: WebDriverException
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 49, in lambda_handler
driver.get(event['url'])
File "/var/task/selenium/webdriver/remote/webdriver.py", line 332, in get
self.execute(Command.GET, {'url': url})
File "/var/task/selenium/webdriver/remote/webdriver.py", line 320, in execute
self.error_handler.check_response(response)
File "/var/task/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: unhandled inspector error: {"code":-32000,"message":"Cannot navigate to invalid URL"}
(Session info: headless chrome=62.0.3202.94)
(Driver info: chromedriver=2.35.528139 (47ead77cb35ad2a9a83248b292151462a66cd881),platform=Linux 4.14.88-90.76.amzn2.x86_64 x86_64)
END RequestId: ce5f4ee0-0263-484e-95cd-3909c4390241
REPORT RequestId: ce5f4ee0-0263-484e-95cd-3909c4390241 Duration: 5548.74 ms Billed Duration: 5600 ms Memory Size: 256 MB Max Memory Used: 198 MB
Any idea what could be happening here? I followed all the steps properly as far as I can. Even tried updating the bin files and changing the permission rights.
Timed Out Error
Hi 🙂
I'm trying to run this file on lambda, but this error showed up
{ "errorMessage": "2021-02-16T01:48:40.456Z c9ee96ed-c1dd-46c5-bbed-fd12d512d186 Task timed out after 20.01 seconds" }
This is the log
START RequestId: c9ee96ed-c1dd-46c5-bbed-fd12d512d186 Version: $LATEST
Starting google.com
END RequestId: c9ee96ed-c1dd-46c5-bbed-fd12d512d186
REPORT RequestId: c9ee96ed-c1dd-46c5-bbed-fd12d512d186 Duration: 20009.43 ms Billed Duration: 20000 ms Memory Size: 256 MB Max Memory Used: 256 MB Init Duration: 56.57 ms
2021-02-16T01:48:40.456Z c9ee96ed-c1dd-46c5-bbed-fd12d512d186 Task timed out after 20.01 seconds
I didn't even unzip and re-zip the file, I uploaded the very file I downloaded here.
The runtime is Python 3.6 and I also created the PATHs.
If you have any clue, please do not hesitate to let me know. Thanks!
OSError: [Errno 8] Exec format error
Hello,
I am testing the code on my mac and it returns this information.
Is the code made for pc ?
SAM local run Error
Starting google.com
Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home
: WebDriverException
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 53, in lambda_handler
driver = webdriver.Chrome(chrome_options=chrome_options)
File "/var/task/selenium/webdriver/chrome/webdriver.py", line 68, in __init__
self.service.start()
File "/var/task/selenium/webdriver/common/service.py", line 83, in start
os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home
END RequestId: 32e1ecb4-3781-4c7b-8898-52a82b4c625c
REPORT RequestId: 32e1ecb4-3781-4c7b-8898-52a82b4c625c Duration: 67 ms Billed Duration: 100 ms Memory Size: 256 MB Max Memory Used: 21 MB
"errorMessage": "Message: 'chromedriver' executable may have wrong permissions.
Hello Manivannan,
Thanks for sharing the code, in the first instance I had no problems, but to the following code fragment of "lambda_function.py":
Original:
driver = webdriver.Chrome (chrome_options = chrome_options)
page_data = ""
if 'url' en event.keys ():
driver.get (evento ['url'])
page_data = driver.page_source
print (page_data)
driver.close ()
volver page_data
New:
driver = webdriver.Chrome (chrome_options = chrome_options)
page_data = ""
if 'url' en event.keys ():
driver.get (evento ['url'])
page_data = driver.title
print (page_data)
driver.close ()
volver page_data
When testing the lambda I am getting the following error:
{
"errorMessage": "Message: 'chromedriver' executable may have wrong permissions. Please see https://sites.google.com/a/chromium.org/chromedriver/home\n",
"errorType": "WebDriverException",
"stackTrace": [
[
"/var/task/lambda_function.py",
46,
"lambda_handler",
"driver = webdriver.Chrome(chrome_options=chrome_options)"
],
[
"/var/task/selenium/webdriver/chrome/webdriver.py",
68,
"init",
"self.service.start()"
],
[
"/var/task/selenium/webdriver/common/service.py",
88,
"start",
"os.path.basename(self.path), self.start_error_message)"
]
]
}
Any ideas? Please your help.
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable may have wrong permissions.
Hello!
When i run the functions, the following error appears:
_
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable may have wrong permissions.
_
Not Run
sir, How can i run this code, can you tell me step by step because every time this code shows me error.
How to add additional packages like boto3
I would like to copy the downloaded file into S3. How to add boto3?
Won't work after change.
Hey there. I tried to make it work. If I just upload your ZIP to lambda and run the function, it works.
So I decided to change a bit of code in lambda_function.py, and that's only to edit line 49,
driver.get('http://www.geneva.info/')
after this change, I pack it back to ZIP, upload and run test. I get this error:
START RequestId: f6bd2ac2-e78c-11e8-86d6-19ee2b8cd09b Version: $LATEST
Starting google.com
Message: 'chromedriver' executable may have wrong permissions. Please see https://sites.google.com/a/chromium.org/chromedriver/home
: WebDriverException
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 46, in lambda_handler
driver = webdriver.Chrome(chrome_options=chrome_options)
File "/var/task/selenium/webdriver/chrome/webdriver.py", line 68, in __init__
self.service.start()
File "/var/task/selenium/webdriver/common/service.py", line 88, in start
os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable may have wrong permissions. Please see https://sites.google.com/a/chromium.org/chromedriver/home
I use Selenium on Windows, but I am fairly new to AWS. What is a problem here and how to make it work?
Thank you.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.