Giter VIP home page Giter VIP logo

heroku-integrated-firefox-geckodriver's Introduction

heroku-integrated-firefox-geckodriver

Build Status License: MIT

Now supports Heroku-20, and legacy versions Heroku-16 and Heroku-18.

Available for download at The Heroku Elements Marketplace. Search heroku-integrated-firefox-geckodriver on the official Heroku Elements Marketplace to access the buildpack.

Buildpack heroku-integrated-firefox-geckodriver enables your application or client code - running in any high-level language such as Python, Ruby or Node.js - to access Firefox along with Geckodriver (the Selenium driver for Firefox) in a Heroku slug and enables the driver objects to perform automated operations defined in the source code.

Version compatibility as follows:

  • Firefox v82.0.3
  • Geckodriver v0.28.0

Installation:

To install and integrate the buildpack with your application running on Heroku's dyno:

$ heroku create --buildpack https://github.com/pyronlaboratory/heroku-integrated-firefox-geckodriver

# or if your app is already created:
$ heroku buildpacks:add https://github.com/pyronlaboratory/heroku-integrated-firefox-geckodriver

$ git push heroku master

Configurations:

Update Heroku's environment variables to store the following path strings.

FIREFOX_BIN: /app/vendor/firefox/firefox

Alternatively, you can even use /app/vendor/firefox/firefox-bin

GECKODRIVER_PATH: /app/vendor/geckodriver/geckodriver

LD_LIBRARY_PATH: /usr/local/lib:/usr/lib:/lib:/app/vendor

PATH: /usr/local/bin:/usr/bin:/bin:/app/vendor/

These configuration variables can be updated via Heroku CLI as follows:

Executable command: heroku config:set <ENV_VARIABLE>=<ABSOLUTE_PATH>

$ heroku config:set FIREFOX_BIN=/app/vendor/firefox/firefox

Setting FIREFOX_BIN and restarting python-app... done, v6
FIREFOX_BIN: '/app/vendor/firefox/firefox'

Implementation:

Python

app.py

import os
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary

def  load_driver():
	options = webdriver.FirefoxOptions()
	
	# enable trace level for debugging 
	options.log.level = "trace"

	options.add_argument("-remote-debugging-port=9224")
	options.add_argument("-headless")
	options.add_argument("-disable-gpu")
	options.add_argument("-no-sandbox")

	binary = FirefoxBinary(os.environ.get('FIREFOX_BIN'))

	firefox_driver = webdriver.Firefox(
		firefox_binary=binary,
		executable_path=os.environ.get('GECKODRIVER_PATH'),
		options=options)

	return firefox_driver

def  start():
	driver = load_driver()
	driver.get("https://www.google.com/")
	print(driver.title)
	driver.close()

if  __name__ == "__main__":
	start()

NOTE: Make sure to add a requirements.txt file with relevant packages, and add the following entry to your Procfile:

worker: python app.py

Ruby

app.rb

require  'selenium-webdriver'

options = Selenium::WebDriver::Firefox::Options.new

options.add_argument('--remote-debugging-port=9222')
options.add_argument('--headless')
options.add_argument('--disable-gpu')
options.add_argument('--no-sandbox')

Selenium::WebDriver::Firefox::Binary.path=ENV['FIREFOX_BIN']
Selenium::WebDriver::Firefox::Service.driver_path=ENV['GECKODRIVER_PATH']
  
# use argument `:debug` instead of `:info` for detailed logs in case of an error
Selenium::WebDriver.logger.level = :info 

driver = Selenium::WebDriver.for :firefox, options: options
driver.get "https://www.google.com"
puts  "#{driver.title}"
driver.quit

A minimalist Gemfile

source "https://rubygems.org"
gem 'selenium-webdriver', '3.142.7'

and Gemfile.lock file

GEM
  remote: https://rubygems.org/
  specs:
    childprocess (3.0.0)
    rubyzip (2.3.0)
    selenium-webdriver (3.142.7)
      childprocess (>= 0.5, < 4.0)
      rubyzip (>= 1.2.2)

PLATFORMS
  x64-mingw32

DEPENDENCIES
  selenium-webdriver (= 3.142.7)

BUNDLED WITH
   2.2.19

NOTE: Run heroku run bash from your local machine to test your script. Execute command ruby app.rb via bash to see result.

Node.js

app.js

const { Builder } = require('selenium-webdriver');
const { Options, firefox } = require('selenium-webdriver/firefox');

const  options = new  Options()
 .headless()
 .setBinary(`${process.env.FIREFOX_BIN}`);

(async  function  example() {
	let  driver = new  Builder()
	 .forBrowser('firefox')
	 .setFirefoxOptions(options)
	 .build();

	try {
		await  driver.get('https://www.google.com/');
		console.log(await  driver.getTitle());
	}
	catch(err) {
		console.log(err.message);
	}
	finally {
		await  driver.quit();
	}
})();

NOTE: Make sure to add a package.json file with relevant packages, and add the following entry to your Procfile:

worker: node app.js

Go ahead and scrape the universe. Drop a star if you like this small project!

Buy me a Ko-fi, will ya?!

heroku-integrated-firefox-geckodriver's People

Contributors

pyronlaboratory avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

heroku-integrated-firefox-geckodriver's Issues

Mismatch of Firefox & Geckodriver versions?

Hi, i added this buildpack to heroku but keep getting this error:

selenium.common.exceptions.WebDriverException: Message: invalid argument: can't kill an exited process

After searching about it, it seems this is caused by incompatible Firefox & Geckodriver versions. Any help on how to solve this error? Thank you.

Differences in Heroku behavior

I'm trying to deploy a new staging environment with the buildpack. However, it is not working as throws error "Selenium::WebDriver::Error::UnknownError (invalid argument: can't kill an exited process)", despite I have the same configuration on both heroku machines. They have the same Firefox, Ubuntu and Geckodriver version.

lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.4 LTS
Release:	18.04
Codename:	bionic

Looking at the build log, I noticed some differences in the libraries installed (see here highlighted differences). Why are they not installing the same libraries? How can we be sure to install correctly all of them?

Problem deploying to Heroku

remote: -----> Firefox
remote: geckodriver app detected
remote: /tmp/codon/tmp/buildpacks/768374e866288447126d1d75779462919bfbaab4/bin/compile: line 16: cd: /tmp/codon/tmp/buildpacks/768374e866288447126d1d75779462919bfbaab4/fonts: No such file or directory
remote: /tmp/codon/tmp/buildpacks/768374e866288447126d1d75779462919bfbaab4/bin/compile: line 82: syntax error near unexpected token `;'
remote: ! Push rejected, failed to compile Firefox
remote: geckodriver app.
remote:
remote: ! Push failed

build error on heroku for dajngo app

remote: -----> Installing requirements with pip
remote: -----> Firefox
remote: geckodriver app detected
remote: /tmp/codon/tmp/buildpacks/768374e866288447126d1d75779462919bfbaab4/bin/compile: line 16: cd: /tmp/codon/tmp/buildpacks/768374e866288447126d1d75779462919bfbaab4/fonts: No such file or directory
remote: /tmp/codon/tmp/buildpacks/768374e866288447126d1d75779462919bfbaab4/bin/compile: line 82: syntax error near unexpected token `;'
remote: ! Push rejected, failed to compile Firefox
remote: geckodriver app.
remote:
remote: ! Push failed

Unable to find Mozilla geckodriver

Thanks for the buildpack!

I'm getting this error on heroku while trying to use selenium with firefox:

Selenium::WebDriver::Error::WebDriverError:  Unable to find Mozilla geckodriver. Please download the server from https://github.com/mozilla/geckodriver/releases and place it somewhere on your PATH. More info at https://developer.mozilla.org/en-US/docs/Mozilla/QA/Marionette/WebDriver.

Here's my init code:

profile = Selenium::WebDriver::Firefox::Profile.new
profile['browser.download.dir'] = download_dir
profile['browser.download.folderList'] = 2
profile['browser.helperApps.neverAsk.saveToDisk'] = 'images/jpeg, application/pdf, application/octet-stream'
profile['browser.acceptInsecureCerts'] = true
profile['pdfjs.disabled'] = true

caps = Selenium::WebDriver::Remote::Capabilities.firefox
caps['acceptInsecureCerts'] = true

driver = Selenium::WebDriver.for :firefox, profile: profile, desired_capabilities: caps

Steps in README do not work on a new created app

Hi! I recently created a new app in Heroku, installed the buildpack, added the ENV variables and it is not working.

I tried different things commented on other issues (I'm using Ruby):

Selenium::WebDriver::Firefox::Binary.path = ENV['FIREFOX_BIN']
Selenium::WebDriver::Firefox.driver_path = ENV['GECKODRIVER_PATH']
options = Selenium::WebDriver::Firefox::Options.new(args: ['-headless', '--disable-gpu', '--no-sandbox', '--remote-debugging-port=9224'])
browser = Selenium::WebDriver.for(:firefox, options: options)

But I'm getting

Selenium::WebDriver::Error::SessionNotCreatedError (Expected browser binary location, but unable to find binary in default location, no 'moz:firefoxOptions.binary' capability provided, and no binary flag set on the command line)

My env variables are:

FIREFOX_BIN=/app/vendor/firefox/firefox
GECKODRIVER_PATH=/app/vendor/geckodriver/geckodriver
LD_LIBRARY_PATH=/usr/local/lib:/usr/lib:/lib:/app/vendor
PATH=/usr/local/bin:/usr/bin:/bin:/app/vendor/

I also tried removing the buildpack from the heroku webapp, installing it from the command line and restarting the dynos (as commented #3 (comment) but it didn't work)

Thanks!

Node.js moz:firefoxOptions.binary - SessionNotCreatedError

Hello,

I created a nodejs project which use selenium with firefox headless. My project works locally but not on heroku.

SessionNotCreatedError: Expected browser binary location, but unable to find binary in default location, no 'moz:firefoxOptions.binary' capability provided, and no binary flag set on the command line
const {Builder, By, until} = require('selenium-webdriver');
const firefox = require('selenium-webdriver/firefox');

const screen = {
  width: 1920,
  height: 1080
};

let options = new firefox.Options();
//Below arguments are critical for Heroku deployment
options.addArguments("--headless");
options.addArguments("--disable-gpu");
options.addArguments("--no-sandbox");
options.windowSize(screen);


let driver = new Builder()
  .forBrowser('firefox')
  .setFirefoxOptions(options)
  .build();

Sans titre

My package.json

{
  "name": "untitled",
  "version": "1.0.0",
  "description": "",
  "scripts": {
    "start": "node index.js"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "express": "latest",
    "selenium-webdriver": "latest",
    "geckodriver": "^1.19.1"
  },
  "engines": {
    "node": "13.6.0"
  }
}

Python - invalid argument: can't kill an exited process

Hi guys, I am trying to use Geckodriver in my Python application hosted on Heroku and I'm getting the following exception:

"invalid argument: can't kill an exited process"

My code:
from selenium.webdriver.firefox.options import Options

options = Options()
options.headless = True
options.binary_location = os.environ.get("FIREFOX_BIN")
driver = webdriver.Firefox(options=options, executable_path=os.environ.get("GECKODRIVER_PATH"))

driver.get("https://my-url")

Any ideas? I already installed the buildpacks and set all the server variables.

Incorrect Permission Error When Deploying to Heroku-18

I created a fork here ( https://github.com/checktest82/heroku-firefox-geckofork ) to fix an error that was stated here: #16

I deployed my application to test here: https://github.com/checktest82/test-package

I added all environment variables to my slug: https://i.imgur.com/FUtgcLl.png

Yet when I run it, Im thrown this error:

{"error":"Message: 'geckodriver' executable may have wrong permissions. \n","status":0}

Which you can see live: https://test66711.herokuapp.com/get

How do I fix?

Also to add, Heroku-20 does not work with this (throws an error if you try to deploy to a slug with heroku-20)

Build error on heroku

I've managed to run firefox on heroku for one week. Unfortunately, the builds started to fail for no apparent reason. I went back to the last working commit but it keeps logging the following message:

-----> Firefox
geckodriver app detected
/app/tmp/buildpacks/fad1b53606e93324bf7b116b7ae1ae05f6e204d621459cc0273846dc4311d64ba9fb45071b7a13f31167eff2ba9ab5990b670c4d0658fcb71ab5e63cc1dd1e6a/bin/compile: line 16: cd: /app/tmp/buildpacks/fad1b53606e93324bf7b116b7ae1ae05f6e204d621459cc0273846dc4311d64ba9fb45071b7a13f31167eff2ba9ab5990b670c4d0658fcb71ab5e63cc1dd1e6a/fonts: No such file or directory
/app/tmp/buildpacks/fad1b53606e93324bf7b116b7ae1ae05f6e204d621459cc0273846dc4311d64ba9fb45071b7a13f31167eff2ba9ab5990b670c4d0658fcb71ab5e63cc1dd1e6a/bin/compile: line 36: syntax error near unexpected token `<<<'
 !     Push rejected, failed to compile Firefox
geckodriver app.
 !     Push failed

ruby '2.5.1'
gem 'rails', '~> 5.1.4'
heroku-18

failed to compile Firefox

I just used this repo to install firefox and geckodriver but getting this error and I know little about heroku and scripts . Hope this helps.

Installing requirements with pip -----> Firefox geckodriver app detected /tmp/codon/tmp/buildpacks/768374e866288447126d1d75779462919bfbaab4/bin/compile: line 16: cd: /tmp/codon/tmp/buildpacks/768374e866288447126d1d75779462919bfbaab4/fonts: No such file or directory /tmp/codon/tmp/buildpacks/768374e866288447126d1d75779462919bfbaab4/bin/compile: line 82: syntax error near unexpected token ;'
! Push rejected, failed to compile Firefox
geckodriver app.
! Push failed`

'geckodriver' executable may have wrong permissions.

As you can likely tell, I'm attempting to use https://github.com/St3veR0nin/cleverbot_aio_free with this, and I have run into several issues (of which my extreme lack of knowledge prevents me from knowing the origin).

I was consistently encountering Unable to find a matching set of capabilities, though after adding (and later removing) some code, I now only encounter this issue.

2020-05-06T11:40:28.897936+00:00 app[worker.1]:     cb = cleverbotfree.cbfree.Cleverbot()
2020-05-06T11:40:28.897948+00:00 app[worker.1]:   File "/app/.heroku/python/lib/python3.6/site-packages/cleverbotfree/cbfree.py", line 42, in __init__
2020-05-06T11:40:28.898084+00:00 app[worker.1]:     self.browser = webdriver.Firefox(options=self.opts)
2020-05-06T11:40:28.898088+00:00 app[worker.1]:   File "/app/.heroku/python/lib/python3.6/site-packages/selenium/webdriver/firefox/webdriver.py", line 164, in __init__
2020-05-06T11:40:28.898306+00:00 app[worker.1]:     self.service.start()
2020-05-06T11:40:28.898311+00:00 app[worker.1]:   File "/app/.heroku/python/lib/python3.6/site-packages/selenium/webdriver/common/service.py", line 88, in start
2020-05-06T11:40:28.898451+00:00 app[worker.1]:     os.path.basename(self.path), self.start_error_message)
2020-05-06T11:40:28.898490+00:00 app[worker.1]: selenium.common.exceptions.WebDriverException: Message: 'geckodriver' executable may have wrong permissions. 
2020-05-06T11:40:28.898493+00:00 app[worker.1]: ```

What do I do?

Error: The geckodriver executable could not be found on the current PATH

Thank you for the buildpack!

I've followed the instructions from here and also made sure to add all the environmental variables, but i still get this error

2022-03-30T20:22:27.706157+00:00 app[worker.1]:   Error: The geckodriver executable could not be found on the current PATH. Please download the latest version from https://github.com/mozilla/geckodriver/releases/ and ensure it can be found on your PATH.
2022-03-30T20:22:27.706158+00:00 app[worker.1]:       at findGeckoDriver (/app/node_modules/selenium-webdriver/firefox.js:463:11)
2022-03-30T20:22:27.706158+00:00 app[worker.1]:       at new ServiceBuilder (/app/node_modules/selenium-webdriver/firefox.js:556:22)
2022-03-30T20:22:27.706159+00:00 app[worker.1]:       at Function.createSession (/app/node_modules/selenium-webdriver/firefox.js:612:21)
2022-03-30T20:22:27.706159+00:00 app[worker.1]:       at createDriver (/app/node_modules/selenium-webdriver/index.js:147:33)
2022-03-30T20:22:27.706160+00:00 app[worker.1]:       at Builder.build (/app/node_modules/selenium-webdriver/index.js:720:16)
2022-03-30T20:22:27.706160+00:00 app[worker.1]:       at Function.<anonymous> (/app/dist/modules/dump.js:32:18)
2022-03-30T20:22:27.706161+00:00 app[worker.1]:       at Generator.next (<anonymous>)
2022-03-30T20:22:27.706161+00:00 app[worker.1]:       at /app/dist/modules/mod.js:8:71
2022-03-30T20:22:27.706161+00:00 app[worker.1]:       at new Promise (<anonymous>)
2022-03-30T20:22:27.706162+00:00 app[worker.1]:       at __awaiter (/app/dist/modules/mod.js:4:12)
2022-03-30T20:22:27.706165+00:00 app[worker.1]: 
2022-03-30T20:22:27.707939+00:00 app[worker.1]: Failed to process updates. Error: The geckodriver executable could not be found on the current PATH. Please download the latest version from https://github.com/mozilla/geckodriver/releases/ and ensure it can be found on your PATH.
2022-03-30T20:22:27.707940+00:00 app[worker.1]:     at findGeckoDriver (/app/node_modules/selenium-webdriver/firefox.js:463:11)
2022-03-30T20:22:27.707941+00:00 app[worker.1]:     at new ServiceBuilder (/app/node_modules/selenium-webdriver/firefox.js:556:22)
2022-03-30T20:22:27.707941+00:00 app[worker.1]:     at Function.createSession (/app/node_modules/selenium-webdriver/firefox.js:612:21)
2022-03-30T20:22:27.707942+00:00 app[worker.1]:     at createDriver (/app/node_modules/selenium-webdriver/index.js:147:33)
2022-03-30T20:22:27.707942+00:00 app[worker.1]:     at Builder.build (/app/node_modules/selenium-webdriver/index.js:720:16)
2022-03-30T20:22:27.707942+00:00 app[worker.1]:     at Function.<anonymous> (/app/dist/modules/mod.js:32:18)
2022-03-30T20:22:27.707943+00:00 app[worker.1]:     at Generator.next (<anonymous>)
2022-03-30T20:22:27.707943+00:00 app[worker.1]:     at /app/dist/modules/mod.js:8:71
2022-03-30T20:22:27.707943+00:00 app[worker.1]:     at new Promise (<anonymous>)
2022-03-30T20:22:27.707944+00:00 app[worker.1]:     at __awaiter (/app/dist/modules/mod.js:4:12)

I've check out a simular issue, but I couldn't figure out how it would help me.

Any ideas?

deploy error

geckodriver app detected
/tmp/codon/tmp/buildpacks/768374e866288447126d1d75779462919bfbaab4/bin/compile: line 16: cd: /tmp/codon/tmp/buildpacks/768374e866288447126d1d75779462919bfbaab4/fonts: No such file or directory
/tmp/codon/tmp/buildpacks/768374e866288447126d1d75779462919bfbaab4/bin/compile: line 82: syntax error near unexpected token `;'
! Push rejected, failed to compile Firefox
geckodriver app.
! Push failed

Couldn't load XPCOM

Thanks for the buildpack ๐Ÿ˜ƒ

I'm getting this error when trying to start a new instance of Selenium Webdriver:

XPCOMGlueLoad error for file /app/vendor/firefox/libxul.so:
libXt.so.6: cannot open shared object file: No such file or directory
Couldn't load XPCOM.

Any ideas ? I've set all the environment variables in the README. Here's my configuration :

Selenium::WebDriver::Firefox::Binary.path = ENV['FIREFOX_BIN']
Selenium::WebDriver::Firefox.driver_path = ENV['GECKODRIVER_PATH']
Selenium::WebDriver.logger.level = :debug

Thanks !

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.