Giter VIP home page Giter VIP logo

spectre's Introduction

Spectre is no longer being maintained here

Spectre hasn't seen active development for some time. We've found a lot more mileage from https://percy.io which is easy to use and zero maintenance.

Alternatively try a fork with active development:

Spectre

Build Status

Spectre is a web application to diff screenshots. It's heavily influenced by VisualReview, BackstopJS and Wraith. Read more about how we use it at Friday in our blog post: How we do visual regression testing.

Spectre! Spectre!

Running the app

You can either run the app using docker, or you can run it natively on your machine – there are instructions for both docker, and running on macOS below.

Alternatively, you can run the application on Heroku.

Running using docker

Install docker

Setup the database (only needs to be done once):

docker-compose run --rm app bundle exec rake db:setup

To run the application:

docker-compose up

When you see WEBrick::HTTPServer#start: pid=2 port=3000, the app will be running at http://localhost:3000

Running natively on macOS

Prerequisites

  • Ruby (doesn't currently work with v2.4.0)
  • Postgres
  • Imagemagick

On a Mac, the easiest way to install the above, is to use homebrew.

  1. install homebrew using these instructions
  2. brew install rbenv
  3. rbenv install 2.7.1
  4. rbenv init and follow the instructions it prints
  5. brew install imagemagick@6
  6. brew install postgresql and follow the instructions it prints about starting the postgresql server

Setup

  • Clone the repo
  • bundle install && bundle exec rake db:setup
  • bundle exec rails s

Running on Heroku:

Deploy to Heroku

Submitting tests

A "test" is a screenshot and associated metadata. A test is categorised under a Project, which in turn has (test) Suites. A test is submitted and associated with a "run" of a suite.

First you should create a new "run". The JSON response will contain the run_id to submit with each subsequent test.

POST /runs
  project: My Project Name
  suite: My Suite Name

Then you can submit a screenshot!

POST /tests
  test:
    run_id: {run_id from above},
    name: Homepage,
    platform: OSX,
    browser: PhantomJS,
    size: 1024,
    screenshot: <File>,
    crop_area: '640x480+50+100'
  • name is a friendly name of your test. It should describe the template, component or state of the thing you've screenshotted
  • platform is the OS/platform that the screenshot was taken on (e.g. OSX, Windows, iOS, Android etc.)
  • browser is the browser that was used to render the screenshot. This will usually be a headless webkit such as Phantom, but if using Selenium you may have used a "real" browser
  • size is the screenshot size
  • screenshot is the image itself. PNGs are preferred
  • crop_area allows to specify a bounding box to crop from uploaded screenshot ("x+<top_left_x>+<top_left_y>"). If not specified, full screenshot is used for comparison. Can be used to perform visual diffs for specific page elements.

Integration with Rake tasks or Cucumber

Most of the time you'll want to use your own rake task to control Selenium and take screenshots, or take screenshots during cucumber step definitions. There's a handy spectre_client gem to upload screenshots to your Spectre gem.

Example test run

An example test run can be executed using:

docker-compose run --rm app bin/demo_test_run http://app:3000

or, if not using docker, first install phantomjs (brew install phantomjs), then run:

bin/demo_test_run http://localhost:3000

If you've deployed the app on heroku, you can replace the URL with the hosted one, e.g.:

docker-compose run --rm app bin/demo_test_run https://your-spectre-install.herokuapp.com
# or
bin/demo_test_run https://your-spectre-install.herokuapp.com

Administration

Spectre doesn't provide a UI or API to edit or delete content. We've included rails_admin, so head to /admin for this. By default there is no password.

Contributing

Tests

Rspec and Cucumber are included in the project. Test coverage is minimal but please don't follow our lead, write tests for anything you add. Use rake to run the existing tests.

spectre's People

Contributors

alkesh avatar bczegeny avatar brainrake avatar darrylb-github avatar dependabot[bot] avatar fionnbharra avatar firefishy avatar jchuerva avatar klamping avatar nickdunn avatar tk8817 avatar vgrigoruk avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

spectre's Issues

docker network

Hey,
you should use networks instead of links, links are outdated and will be removed in the future

Hangup under Mac OSx with Docker

Hi,

If want to start it via Docker, the things hang up..

Building app   
Step 1/11 : FROM ruby:2.3   
 ---> 03ff8cebf24f   
Step 2/11 : WORKDIR /opt    
 ---> Using cache
 ---> d414ae489c27
Step 3/11 : RUN curl -Ls https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2 | tar -jxf -
 ---> Using cache
 ---> 20e3c36c3bf1
Step 4/11 : RUN apt-get -qq update && apt-get -qq install fontconfig
 ---> Using cache
 ---> b24c1d7747ab
Step 5/11 : RUN ln -s /opt/phantomjs-2.1.1-linux-x86_64/bin/phantomjs /usr/local/bin/
 ---> Using cache
 ---> cff9be0fd825
Step 6/11 : WORKDIR /app
 ---> Using cache
 ---> 8e34591895e6
Step 7/11 : ADD Gemfile* /app/
 ---> Using cache
 ---> c983e83e7359
Step 8/11 : RUN gem install bundler && bundle config build.nokogiri --use-system-libraries && bundle install --quiet --jobs 16 --retry 5 --without test
 ---> Running in bc46bd296e0d
Successfully installed bundler-1.16.1
1 gem installed
The gemspec at
/usr/local/bundle/bundler/gems/spectre_client-a83cbca5f054/spectre_client.gemspec
is not valid. Please fix this gemspec.
The validation error was '"TODO: Put your gem's website or public repo URL
here." is not a valid HTTP URI'
ERROR: Service 'app' failed to build: The command '/bin/sh -c gem install bundler && bundle config build.nokogiri --use-system-libraries && bundle install --quiet --jobs 16 --retry 5 --without test' returned a non-zero code: 1

Docker Test run unable to find phantom

When I run docker-compose run --rm app bin/demo_test_run http://app:3000 I get the following:

Created Spectre run
Traceback (most recent call last):
	12: from bin/demo_test_run:156:in `<main>'
	11: from bin/demo_test_run:67:in `demo_run_with_screenshots'
	10: from /usr/local/bundle/gems/capybara-3.18.0/lib/capybara/dsl.rb:51:in `block (2 levels) in <module:DSL>'
	 9: from /usr/local/bundle/gems/capybara-3.18.0/lib/capybara/session.rb:277:in `visit'
	 8: from /usr/local/bundle/gems/poltergeist-1.18.1/lib/capybara/poltergeist/driver.rb:100:in `visit'
	 7: from /usr/local/bundle/gems/poltergeist-1.18.1/lib/capybara/poltergeist/driver.rb:27:in `browser'
	 6: from /usr/local/bundle/gems/poltergeist-1.18.1/lib/capybara/poltergeist/driver.rb:47:in `client'
	 5: from /usr/local/bundle/gems/poltergeist-1.18.1/lib/capybara/poltergeist/client.rb:16:in `start'
	 4: from /usr/local/bundle/gems/poltergeist-1.18.1/lib/capybara/poltergeist/client.rb:16:in `new'
	 3: from /usr/local/bundle/gems/poltergeist-1.18.1/lib/capybara/poltergeist/client.rb:53:in `initialize'
	 2: from /usr/local/bundle/gems/cliver-0.3.2/lib/cliver.rb:24:in `detect!'
	 1: from /usr/local/bundle/gems/cliver-0.3.2/lib/cliver/dependency.rb:117:in `detect!'
/usr/local/bundle/gems/cliver-0.3.2/lib/cliver/dependency.rb:151:in `raise_version_mismatch!': Could not find an executable 'phantomjs' that matched the requirements '>= 1.8.1', '< 3.0'. Found versions were {"/usr/local/bin/phantomjs"=>"140385087622784"}. (Cliver::Dependency::VersionMismatch)

A project and run are created but no screenshot is sent to the app.

Provide documentation on optional parameters

Hello,

I am looking for information on the different parameters and config

eg: what percentage difference equals a pass or fail, and how to tweak those percentages
eg: what the fuzz_level parameter does
etc.

Does anyone have this information, is there a plan to add it to the documentation here?

Ignore element e.g. image carousel

Hi,

Is there a way to set an element, like an image carousel, to be ignored in the comparison? I'm running into false positives (false negatives?) when running against pages with these - not the end of the world but any reason but would be great to be able to reduce overhead on checking the failed pages.

Thanks!

ERROR: Service 'app' failed to build

$ docker-compose run --rm app bundle exec rake db:setup
.........
Step 3/11 : RUN curl -Ls https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2 | tar -jxf -
---> Running in 3d5320149fde

bzip2: Compressed file ends unexpectedly;
perhaps it is corrupted? Possible reason follows.
bzip2: Inappropriate ioctl for device
Input file = (stdin), output file = (stdout)

It is possible that the compressed file(s) have become corrupted.
You can use the -tvv option to test integrity of such files.

You can use the `bzip2recover' program to attempt to recover
data from undamaged sections of corrupted files.

tar: Child returned status 2
tar: Error is not recoverable: exiting now
ERROR: Service 'app' failed to build: The command '/bin/sh -c curl -Ls https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2 | tar -jxf -' returned a non-zero code: 2
(base) ✘-1 ~/QA/SPECTRE/spectr

Has the "width" parameter been replaced/renamed to "size"? (POST /tests)

Hi,

The README.md says that you should include a "width" parameter (among other parameters) when making a POST request to /tests (to submit a screenshot). However, I'm getting a validation error that "Size can't be blank" when following those instructions. When I look into tests_controller.rb, then I don't see any "width" parameter being required, but I see a "size" parameter which is required. So I renamed my "width" parameter to "size" and my POST request was successful. Should the readme be updated?

From the README.md:
Then you can submit a screenshot!

POST /tests
      test:
        run_id: {run_id from above},
        name: Homepage,
        platform: OSX,
        browser: PhantomJS,
        width: 1024,
        screenshot: <File>

heroku failed deploy

Hi guys,
so I tried the heroku deploy, it starts ok but after a couple of minutes it shows:

FATAL: permission denied for database "postgres"
DETAIL: User does not have CONNECT privilege.
Couldn't create database for {"adapter"=>"postgresql", "encoding"=>"utf8", "username"=>"jxrztvwrupgrcc", "password"=>"cb9fd014fa11513fea2f8c044f2a1497c07f9f20af39e19433c973d0509a10f6", "pool"=>5, "timeout"=>5000, "port"=>5432, "database"=>"d4mfl79gs0n3vd", "host"=>"ec2-23-21-66-88.compute-1.amazonaws.com"}
rake aborted!
PG::ConnectionBad: FATAL: permission denied for database "postgres"
DETAIL: User does not have CONNECT privilege.
/app/vendor/bundle/ruby/2.5.0/gems/pg-1.1.4/lib/pg.rb:56:in initialize' /app/vendor/bundle/ruby/2.5.0/gems/pg-1.1.4/lib/pg.rb:56:in new'
/app/vendor/bundle/ruby/2.5.0/gems/pg-1.1.4/lib/pg.rb:56:in connect' /app/vendor/bundle/ruby/2.5.0/gems/activerecord-5.0.7.2/lib/active_record/connection_adapters/postgresql_adapter.rb:673:in connect'
/app/vendor/bundle/ruby/2.5.0/gems/activerecord-5.0.7.2/lib/active_record/connection_adapters/postgresql_adapter.rb:217:in initialize' /app/vendor/bundle/ruby/2.5.0/gems/activerecord-5.0.7.2/lib/active_record/connection_adapters/postgresql_adapter.rb:37:in new'
/app/vendor/bundle/ruby/2.5.0/gems/activerecord-5.0.7.2/lib/active_record/connection_adapters/postgresql_adapter.rb:37:in postgresql_connection' /app/vendor/bundle/ruby/2.5.0/gems/activerecord-5.0.7.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:729:in new_connection'
/app/vendor/bundle/ruby/2.5.0/gems/activerecord-5.0.7.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:773:in checkout_new_connection' /app/vendor/bundle/ruby/2.5.0/gems/activerecord-5.0.7.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:752:in try_to_checkout_new_connection'
/app/vendor/bundle/ruby/2.5.0/gems/activerecord-5.0.7.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:713:in acquire_connection' /app/vendor/bundle/ruby/2.5.0/gems/activerecord-5.0.7.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:490:in checkout'
/app/vendor/bundle/ruby/2.5.0/gems/activerecord-5.0.7.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:364:in connection' /app/vendor/bundle/ruby/2.5.0/gems/activerecord-5.0.7.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:883:in retrieve_connection'
/app/vendor/bundle/ruby/2.5.0/gems/activerecord-5.0.7.2/lib/active_record/connection_handling.rb:128:in retrieve_connection' /app/vendor/bundle/ruby/2.5.0/gems/activerecord-5.0.7.2/lib/active_record/connection_handling.rb:91:in connection'
/app/vendor/bundle/ruby/2.5.0/gems/activerecord-5.0.7.2/lib/active_record/tasks/postgresql_database_tasks.rb:6:in connection' /app/vendor/bundle/ruby/2.5.0/gems/activerecord-5.0.7.2/lib/active_record/tasks/postgresql_database_tasks.rb:15:in create'
/app/vendor/bundle/ruby/2.5.0/gems/activerecord-5.0.7.2/lib/active_record/tasks/database_tasks.rb:109:in create' /app/vendor/bundle/ruby/2.5.0/gems/activerecord-5.0.7.2/lib/active_record/tasks/database_tasks.rb:129:in block in create_current'
/app/vendor/bundle/ruby/2.5.0/gems/activerecord-5.0.7.2/lib/active_record/tasks/database_tasks.rb:295:in block in each_current_configuration' /app/vendor/bundle/ruby/2.5.0/gems/activerecord-5.0.7.2/lib/active_record/tasks/database_tasks.rb:292:in each'
/app/vendor/bundle/ruby/2.5.0/gems/activerecord-5.0.7.2/lib/active_record/tasks/database_tasks.rb:292:in each_current_configuration' /app/vendor/bundle/ruby/2.5.0/gems/activerecord-5.0.7.2/lib/active_record/tasks/database_tasks.rb:128:in create_current'
/app/vendor/bundle/ruby/2.5.0/gems/activerecord-5.0.7.2/lib/active_record/railties/databases.rake:27:in block (2 levels) in <top (required)>' /app/vendor/bundle/ruby/2.5.0/gems/rake-12.3.2/exe/rake:27:in <top (required)>'
/app/vendor/bundle/ruby/2.5.0/gems/bundler-1.15.2/lib/bundler/cli/exec.rb:74:in load' /app/vendor/bundle/ruby/2.5.0/gems/bundler-1.15.2/lib/bundler/cli/exec.rb:74:in kernel_load'
/app/vendor/bundle/ruby/2.5.0/gems/bundler-1.15.2/lib/bundler/cli/exec.rb:27:in run' /app/vendor/bundle/ruby/2.5.0/gems/bundler-1.15.2/lib/bundler/cli.rb:360:in exec'
/app/vendor/bundle/ruby/2.5.0/gems/bundler-1.15.2/lib/bundler/vendor/thor/lib/thor/command.rb:27:in run' /app/vendor/bundle/ruby/2.5.0/gems/bundler-1.15.2/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in invoke_command'
/app/vendor/bundle/ruby/2.5.0/gems/bundler-1.15.2/lib/bundler/vendor/thor/lib/thor.rb:369:in dispatch' /app/vendor/bundle/ruby/2.5.0/gems/bundler-1.15.2/lib/bundler/cli.rb:20:in dispatch'
/app/vendor/bundle/ruby/2.5.0/gems/bundler-1.15.2/lib/bundler/vendor/thor/lib/thor/base.rb:444:in start' /app/vendor/bundle/ruby/2.5.0/gems/bundler-1.15.2/lib/bundler/cli.rb:10:in start'
/app/vendor/bundle/ruby/2.5.0/gems/bundler-1.15.2/exe/bundle:30:in block in <top (required)>' /app/vendor/bundle/ruby/2.5.0/gems/bundler-1.15.2/lib/bundler/friendly_errors.rb:121:in with_friendly_errors'
/app/vendor/bundle/ruby/2.5.0/gems/bundler-1.15.2/exe/bundle:22:in <top (required)>' /app/bin/bundle:3:in load'
/app/bin/bundle:3:in `

'
Tasks: TOP => db:create
(See full trace by running task with --trace)

Good example of a working project?

Can anyone share a dump of their configuration files for this?

I'm at a loss of even where to start to try and use this.

I want to do Visual Regression testing of a website between two public facing URLS, a DEV url and a PROD url, and have it scan a collection of paths in each, at various break points, and then show me the diffs with errors based on a % difference...

Does anyone have something I can start from?

Thank you.

Describe POST /test example parameters

readme says "screenshot: screenshot is the image itself. PNGs are preferred"
^^ what type is expected? should it be path to the image on local hdd?

  • if yes, how to put the images to spectre before sending the request? (Assuming spectre is on separate server than tests)
  • if not, then what format should it be? base64 encoded ?

docker-compose up fails

Any ideas?

λ docker-compose up
Starting spectre_db_1 ... done
Starting spectre_app_1 ... done
Attaching to spectre_db_1, spectre_app_1
db_1   |
db_1   | PostgreSQL Database directory appears to contain a database; Skipping initialization
db_1   |
db_1   | LOG:  database system was shut down at 2020-07-28 15:09:59 UTC
db_1   | LOG:  MultiXact member wraparound protections are now enabled
db_1   | LOG:  autovacuum launcher started
db_1   | LOG:  database system is ready to accept connections
app_1  | /usr/local/bundle/gems/mime-types-3.2.2/lib/mime/types/logger.rb:30: warning: `_1' is reserved for numbered parameter; consider another name
app_1  | /usr/local/bundle/gems/mime-types-3.2.2/lib/mime/types/logger.rb:30: warning: `_2' is reserved for numbered parameter; consider another name
app_1  | /usr/local/bundle/gems/mime-types-3.2.2/lib/mime/types/logger.rb:30: warning: `_3' is reserved for numbered parameter; consider another name
app_1  | /usr/local/bundle/gems/tzinfo-1.2.5/lib/tzinfo/ruby_core_support.rb:142: warning: Using the last argument as keyword parameters is deprecated
app_1  | /usr/local/bundle/gems/tzinfo-1.2.5/lib/tzinfo/ruby_core_support.rb:142: warning: Using the last argument as keyword parameters is deprecated
app_1  | /usr/local/bundle/gems/activerecord-5.0.7.2/lib/active_record/type.rb:22: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call
app_1  | /usr/local/bundle/gems/activerecord-5.0.7.2/lib/active_record/type/adapter_specific_registry.rb:7: warning: The called method `add_modifier' is defined here
app_1  | /usr/local/bundle/gems/actionpack-5.0.7.2/lib/action_dispatch/middleware/stack.rb:35: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call
app_1  | /usr/local/bundle/gems/actionpack-5.0.7.2/lib/action_dispatch/middleware/static.rb:113: warning: The called method `initialize' is defined here
app_1  | rake aborted!
app_1  | Errno::EEXIST: File exists @ dir_s_mkdir - /app/tmp/cache/assets/sprockets/v3.0/lf
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/cache/file_store.rb:84:in `set'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/cache.rb:212:in `set'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/cache.rb:136:in `set'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/loader.rb:246:in `store_asset'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/loader.rb:185:in `load_from_unloaded'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/loader.rb:60:in `block in load'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/loader.rb:317:in `fetch_asset_from_dependency_cache'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/loader.rb:44:in `load'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/cached_environment.rb:20:in `block in initialize'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/cached_environment.rb:47:in `load'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/bundle.rb:23:in `block in call'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/utils.rb:200:in `dfs'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/bundle.rb:24:in `call'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/processor_utils.rb:75:in `call_processor'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/processor_utils.rb:57:in `block in call_processors'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/processor_utils.rb:56:in `reverse_each'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/processor_utils.rb:56:in `call_processors'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/loader.rb:134:in `load_from_unloaded'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/loader.rb:60:in `block in load'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/loader.rb:317:in `fetch_asset_from_dependency_cache'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/loader.rb:44:in `load'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/cached_environment.rb:20:in `block in initialize'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/cached_environment.rb:47:in `load'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/base.rb:66:in `find_asset'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/base.rb:73:in `find_all_linked_assets'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/manifest.rb:134:in `block in find'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/manifest.rb:133:in `each'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/manifest.rb:133:in `find'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/sprockets/manifest.rb:186:in `compile'
app_1  | /usr/local/bundle/gems/sprockets-rails-3.2.1/lib/sprockets/rails/task.rb:68:in `block (3 levels) in define'
app_1  | /usr/local/bundle/gems/sprockets-3.7.2/lib/rake/sprocketstask.rb:147:in `with_logger'
app_1  | /usr/local/bundle/gems/sprockets-rails-3.2.1/lib/sprockets/rails/task.rb:67:in `block (2 levels) in define'
app_1  | Tasks: TOP => assets:precompile
app_1  | (See full trace by running task with --trace)
spectre_app_1 exited with code 1

Error while accessing Spectre application on browser

Error while accessing Spectre application on browser

[fb146069-19e1-4d29-b142-ec9a5f241978] activesupport (5.0.7.2) lib/active_support/tagged_logging.rb:69:in `block in tagged'

[fb146069-19e1-4d29-b142-ec9a5f241978] activesupport (5.0.7.2) lib/active_support/tagged_logging.rb:26:in `tagged'

[fb146069-19e1-4d29-b142-ec9a5f241978] activesupport (5.0.7.2) lib/active_support/tagged_logging.rb:69:in `tagged'

[fb146069-19e1-4d29-b142-ec9a5f241978] railties (5.0.7.2) lib/rails/rack/logger.rb:24:in `call'

[fb146069-19e1-4d29-b142-ec9a5f241978] actionpack (5.0.7.2) lib/action_dispatch/middleware/request_id.rb:24:in `call'

[fb146069-19e1-4d29-b142-ec9a5f241978] rack (2.2.2) lib/rack/method_override.rb:24:in `call'

[fb146069-19e1-4d29-b142-ec9a5f241978] rack (2.2.2) lib/rack/runtime.rb:22:in `call'

[fb146069-19e1-4d29-b142-ec9a5f241978] activesupport (5.0.7.2) lib/active_support/cache/strategy/local_cache_middleware.rb:28:in `call'

[fb146069-19e1-4d29-b142-ec9a5f241978] dragonfly (1.0.12) lib/dragonfly/cookie_monster.rb:9:in `call'

[fb146069-19e1-4d29-b142-ec9a5f241978] actionpack (5.0.7.2) lib/action_dispatch/middleware/executor.rb:12:in `call'

[fb146069-19e1-4d29-b142-ec9a5f241978] actionpack (5.0.7.2) lib/action_dispatch/middleware/static.rb:136:in `call'

[fb146069-19e1-4d29-b142-ec9a5f241978] rack (2.2.2) lib/rack/sendfile.rb:110:in `call'

[fb146069-19e1-4d29-b142-ec9a5f241978] railties (5.0.7.2) lib/rails/engine.rb:522:in `call'

[fb146069-19e1-4d29-b142-ec9a5f241978] puma (3.12.2) lib/puma/configuration.rb:227:in `call'

[fb146069-19e1-4d29-b142-ec9a5f241978] puma (3.12.2) lib/puma/server.rb:674:in `handle_request'

[fb146069-19e1-4d29-b142-ec9a5f241978] puma (3.12.2) lib/puma/server.rb:476:in `process_client'

[fb146069-19e1-4d29-b142-ec9a5f241978] puma (3.12.2) lib/puma/server.rb:334:in `block in run'

[fb146069-19e1-4d29-b142-ec9a5f241978] puma (3.12.2) lib/puma/thread_pool.rb:135:in `block in spawn_thread'

I, [2020-11-05T03:50:00.427707 #1] INFO -- : [70176e1d-f42a-4129-b2d9-6a6fdd7c058a] Started GET "/" for 172.18.0.1 at 2020-11-05 03:50:00 +0000

I, [2020-11-05T03:50:00.434823 #1] INFO -- : [7d6ddc74-7e7e-4055-b3a8-378179663360] Started GET "/projects" for 172.18.0.1 at 2020-11-05 03:50:00 +0000

I, [2020-11-05T03:50:00.435655 #1] INFO -- : [7d6ddc74-7e7e-4055-b3a8-378179663360] Processing by ProjectsController#index as HTML

I, [2020-11-05T03:50:00.436505 #1] INFO -- : [7d6ddc74-7e7e-4055-b3a8-378179663360] Rendering projects/index.html.erb within layouts/application

D, [2020-11-05T03:50:00.438361 #1] DEBUG -- : [7d6ddc74-7e7e-4055-b3a8-378179663360] Project Load (0.9ms) SELECT "projects".* FROM "projects"

I, [2020-11-05T03:50:00.498963 #1] INFO -- : [7d6ddc74-7e7e-4055-b3a8-378179663360] Rendered projects/index.html.erb within layouts/application (62.3ms)

I, [2020-11-05T03:50:00.499171 #1] INFO -- : [7d6ddc74-7e7e-4055-b3a8-378179663360] Completed 500 Internal Server Error in 63ms (ActiveRecord: 0.9ms)

F, [2020-11-05T03:50:00.500258 #1] FATAL -- : [7d6ddc74-7e7e-4055-b3a8-378179663360]

F, [2020-11-05T03:50:00.500361 #1] FATAL -- : [7d6ddc74-7e7e-4055-b3a8-378179663360] ActionView::Template::Error (PG::UndefinedTable: ERROR: relation "projects" does not exist

LINE 1: SELECT "projects".* FROM "projects"

^

: SELECT "projects".* FROM "projects"):

F, [2020-11-05T03:50:00.500560 #1] FATAL -- : [7d6ddc74-7e7e-4055-b3a8-378179663360] 13:

[7d6ddc74-7e7e-4055-b3a8-378179663360] 14:

[7d6ddc74-7e7e-4055-b3a8-378179663360] 15:

[7d6ddc74-7e7e-4055-b3a8-378179663360] 16: <% @projects.includes(:suites).each do |project| %>

[7d6ddc74-7e7e-4055-b3a8-378179663360] 17: <% project.suites.each_with_index do |suite, i| %>

[7d6ddc74-7e7e-4055-b3a8-378179663360] 18:

[7d6ddc74-7e7e-4055-b3a8-378179663360] 19: <%= project.name %>

F, [2020-11-05T03:50:00.501057 #1] FATAL -- : [7d6ddc74-7e7e-4055-b3a8-378179663360]

F, [2020-11-05T03:50:00.501117 #1] FATAL -- : [7d6ddc74-7e7e-4055-b3a8-378179663360] app/views/projects/index.html.erb:16:in `_app_views_projects_index_html_erb___657709373241675594_60940'

tiny typo

It's heavily influence**d** ... in README.

Java Client for Spectre - Getting 500 error on submit tests

I am working on a java client for Spectre.
After getting the run_id successfully , when I submit the tests I get 500 Internal Server Error

Completed 200 OK in 39ms (Views: 0.1ms | ActiveRecord: 23.5ms) I, [2017-11-08T04:26:08.282721 #14] INFO -- : Started POST "/tests/" for 172.18.0.1 at 2017-11-08 04:26:08 +0000 I, [2017-11-08T04:26:08.284353 #14] INFO -- : Processing by TestsController#create as HTML I, [2017-11-08T04:26:08.284861 #14] INFO -- : Parameters: {"test"=>{"run_id"=>"13", "name"=>"Homepage", "platform"=>"OSX", "browser"=>"Firefox", "size"=>"1289", "screenshot"=>{"path"=>"/var/folders/0v/9tqxsqpn38n64cwv8b6mdh9w3d7_zq/T/screenshot8143117615798594514.png"}, "crop_area"=>"640x480+50+100", "source_url"=>"http://mysite.dev/"}} I, [2017-11-08T04:26:08.286149 #14] INFO -- : Completed 500 Internal Server Error in 1ms (ActiveRecord: 0.0ms) F, [2017-11-08T04:26:08.287157 #14] FATAL -- : NoMethodError (undefined methodpath' for nil:NilClass):
app/controllers/tests_controller.rb:20:in create'

Cannot call Spectre API endpoint `POST /runs`

After get Spectre run with some effort as reported here, I can run the web app at localhost:3000

Though, when I call the API endpoint POST /runs at localhost:3000/runs with POST method using Postman, it results in error

We're sorry, but something went wrong.
If you are the application owner check the logs for more information.

The web app localhost:3000 also fail now with the same error.

So, how can I get the API work?

when screenshot differences are small, the baseline is replaced with the new screenshot, but should remain unchanged

when screenshot differences are small, the baseline is replaced with the new screenshot, but should remain unchanged

STR:

  1. Replace demo_test_run with the one in the zip file
    demo_test_run.zip
    attached to this issue.
  2. Invoke demo_test_run with the argument "baseline" (e.g., docker-compose run --rm app bin/demo_test_run http://app:3000 baseline) and view the screenshot comparison in the Spectre UI.
    image
  3. Invoke demo_test_run with the argument "fail_small" (e.g., docker-compose run --rm app bin/demo_test_run http://app:3000 fail_small) and view the screenshot comparison in the Spectre UI. Note that the difference is not big enough to cause the test to fail, but it is noted as "0.05% difference".
    image
  4. Invoke demo_test_run again with the argument "fail_small" and view the screenshot comparison in the Spectre UI.
    image
    Observed: The newly created screenshot is shown as the baseline.
    Expected: The original baseline screenshot will still be shown as the baseline.

Notes:

I noticed this because I ran a test where the output in a failed case differed only slightly (two words were translated incorrectly) in such a way that the difference was not large enough to be considered a comparison failure. My Selenium test compared the text and the test also used Spectre screen comparison. When I ran the test again, the test still failed because the text was still different, but this time the screen comparison stated that there was no difference at all, not even the small (e.g., 0.03%) difference that had originally been reported.

A scenario in which this bug could manifest itself with very bad effect would be the following: each time a test is run, a particular piece of text shifts by one pixel further to the right each time. Each time, the baseline gets replaced with the new (shifted baseline), and so eventually, the text is in a completely different location. If it were compared to the original baseline, it would be noted as a failure, but since a new baseline is computed on each trial, the failure is never reported.

I have only observed this unexpected baseline replacement to occur on "passes". If I run demo_test_run with "baseline" (run twice or set as baseline as necessary) "fail_large", and "fail_larger", and "fail_larger" the original baseline is, as expected, not replaced in any of these runs.

Meaningful Heroku Error Messages

I did deploy the app on Heroku which was very easy and I'm also able to submit tests and screenshot.
However, I do receive sometimes generic error messages when accessing test runs.

Error message:
We're sorry, but something went wrong.
If you are the application owner check the logs for more information.
image

I checked the logs, but those are empty -> log/production.log
There is also no error message in the Heroku console.

How can I find out what went wrong?
Can you enhance the error messages to give a hint? If it is a database probleme, this would allow us to fix it.

I could not track down the error reason so far. Sometimes there is no response or a timeout from the screenshot endpoint, but this does not always cause the error descriped above.

Thanks!

Add turbolinks

Might suit the UX now turbolinks has a better loading bar, and all Spectre image loads are lazy.

Docker installation fails:

I followed the instructions for docker install on mac OS X 10.12, but I met the following error:

$ docker-compose run --rm app bundle exec rake db:setup
Starting spectre_db_1 ... done
Building app
Step 1/12 : FROM ruby:2.5.3
 ---> 72aaaee1eea4
Step 2/12 : WORKDIR /opt
 ---> Using cache
 ---> 2040027e8836
Step 3/12 : RUN curl -Ls https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2 | tar -jxf -
 ---> Running in 031c07f66495

bzip2: Compressed file ends unexpectedly;
	perhaps it is corrupted?  *Possible* reason follows.
bzip2: Inappropriate ioctl for device
	Input file = (stdin), output file = (stdout)

It is possible that the compressed file(s) have become corrupted.
You can use the -tvv option to test integrity of such files.

You can use the `bzip2recover' program to attempt to recover
data from undamaged sections of corrupted files.

tar: Child returned status 2
tar: Error is not recoverable: exiting now
ERROR: Service 'app' failed to build: The command '/bin/sh -c curl -Ls https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2 | tar -jxf -' returned a non-zero code: 2

Ty in advance for any help.

Doesn't delete thumbnails

When deleting a test, the three screenshots are deleted (Dragonfly cleans up) but the extra thumbnails stored on disk are not. The delete callback should purge these.

DB Migration PostgreSQL 9.6

Hey,
after the last update, i cannot start the Docker Container properly, because the data directory was initialized with postgre 9.4, which is not compatible with 9.6.12

Can't see past the previous 5 runs

Each test suite only shows the previous 5 runs and there doesn't seem to be a way to view earlier runs using the id in the url.

Is there anyway to do this as it would be really useful to view historic changes especially for cross browsing testing?

Filter by Projects

Hello,

I am not sure how easily this can be done, but it would be great to be able to filter by projects, like how you can filter by suites.

Example, you have three projects, each with 5 suites. The dashboard currently shows all the projects and suites in one. Ideally, tapping the project name would bring the user to a page with suite that just belong to that project (instead of all projects' suites).

On a suite page (as well as on test run pages) it shows something like the following near the top: "Projects > ProjectName > SuiteName". Right now, clicking a project name brings the user back to the root /projects page. Ideally, it would bring the user to "https://specre_url/projects/{projectname}/suites/" or similar, so we can see all the suites just for that project.

I am not a ruby guy so it would be harder for me to contribute, but if you can point me in the right direction I would be glad to help with it too.

Cannot run Spectre via docker-compose from Ubuntu Desktop 16

I follow the running guide here which executes the command

docker-compose run --rm app bundle exec rake db:setup

I got the error which say something like

kwargs_from_env() got an unexpected keyword argument 'environment'

My google search leads me this post and gives me the below command that fixes the issue.

sudo pip install --upgrade 'docker-py>=1.9.0' #fix error ref. https://github.com/docker/compose/issues/3973#issuecomment-249300447

My question, is it correct way to run Spectre?

Make it work with ruby 2.4.0

If you try and run spectre using ruby 2.4.0,
the db:schema:load fails with:

SystemStackError: stack level too deep

Ruby 2.3.0 works fine (I haven't checked with 2.3.3)

License

Hey wearefriday, thanks for this project. Our team is exploring using spectre for visual regression testing paired with screenshots from https://saucelabs.com/. One of the things we were curious about is if you are planning to add a license (MIT, Apache-2.0, ISC, etc)?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.