Giter VIP home page Giter VIP logo

console-fail-test's Introduction

console-fail-test

Gently fails test runs if the console was used during them. πŸ“’

All Contributors: 4 πŸ‘ͺ Codecov Test Coverage Contributor Covenant License: MIT Sponsor: On GitHub Style: Prettier TypeScript: Strict npm package version Contributor Covenant

Why?

Logging to the console during tests can be a sign of:

  • 🚫 warnings from third-party libraries such as React for improper usage
  • πŸ€• temporary code that shouldn't be checked into your project
  • πŸ“’ unnecessary spam in your tests window

This little library throws an error after each test if a console method was called during it. It's got some nifty features:

  • πŸ“Š Summary of which methods are called with calling arguments
  • πŸ›« Failures are thrown after tests finish, so your tests will fail normally if they should
stdout | src/index.test.ts > index > example test that console.logs
Whoopsies!

 ❯ src/index.test.ts (4)
   ❯ index (4)
     Γ— example test that console.logs
       β ™ [ afterEach ]
     βœ“ example test that does not console.log

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯- Failed Tests 1 ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯-

 FAIL  src/index.test.ts > index > example test that console.logs
Error: Oh no! Your test called the following console method:
  * log (1 call)
    > Call 0: "Whoopsies!"

Usage

console-fail-test is meant to support any (test framework) & (spy library) combination. It will auto-detect your combination if possible and use the most appropriate environment hooks and function spies it can find.

For example, in a Jest config:

// jest.config.js
module.exports = {
	setupFilesAfterEnv: ["console-fail-test/setup.js"],
};

Test Frameworks

See the Documentation link for each supported framework for how to set up console-fail-test with that framework.

Framework Support? API Request Documentation
Ava βœ…οΈ require("ava") Ava.md
Mocha βœ…οΈ ✨ "mocha" Mocha.md
Jasmine βœ…οΈ ✨ "jasmine" Jasmine.md
Jest βœ…οΈ ✨ "jest" Jest.md
lab βœ… exports.lab Lab.md
node-tap βœ…οΈ require("node-tap") NodeTap.md
Vitest βœ…οΈ ✨ "vitest" Vitest.md
Cypress βš™οΈ /issues/199
QUnit βš™οΈ /issues/19
Playwright βš™οΈ /issues/198
tape βš™οΈ /issues/17
TestCafe βš™οΈ /issues/15

Spy Libraries

If your test framework provides its own spy library, console-fail-test will by default use that library. If a supported spy library isn't detected, an internal fallback will be used to spy on console methods.

You can request a specific test library using the Node API with its API request:

require("console-fail-test").cft({
	spyLibrary: "sinon",
});
Library Support? API Request Spy Documentation
Jasmine βœ…οΈ "jasmine" jasmine.createSpy() Jasmine.md#spies
Jest βœ…οΈ "jest" jest.fn() Jest.md#spies
Sinon βœ…οΈ require("sinon") sinon.spy() Sinon.md#spies
Vitest βœ…οΈ "vitest" vi.fn() Vitest.md#spies

Ignoring console methods

By default, console-fail-test will error on any called console method. If you'd like allow certain methods, pass a console object to the cft API when you set it up:

require("console-fail-test").cft({
	console: {
		warn: true, // won't error on any instance of console.warn
	},
});

Development

Requires:

After forking the repo from GitHub:

git clone https://github.com/<your-name-here>/console-fail-test
cd console-fail-test
yarn

Contribution Guidelines

We'd love to have you contribute! Check the issue tracker for issues labeled accepting prs to find bug fixes and feature requests the community can work on. If this is your first time working with this code, the good first issue label indicates good introductory issues.

Please note that this project is released with a Contributor Covenant. By participating in this project you agree to abide by its terms. See CODE_OF_CONDUCT.md.

Contributors

Colin MacKenzie
Colin MacKenzie

πŸ’» πŸ€”
Dimitri Kopriwa
Dimitri Kopriwa

πŸ’»
Josh Goldberg
Josh Goldberg

πŸ”§ πŸ’» 🚧 πŸš‡ πŸ€”
SUZUKI Sosuke
SUZUKI Sosuke

πŸ’»

πŸ’™ This package is based on @JoshuaKGoldberg's create-typescript-app.

console-fail-test's People

Contributors

allcontributors[bot] avatar dependabot[bot] avatar gitter-badger avatar greenkeeper[bot] avatar jakemhiller avatar joshuakgoldberg avatar kopax-polyconseil avatar renovate[bot] avatar sosukesuzuki avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

console-fail-test's Issues

πŸ“ Documentation: Revamp getting started guides & internal docs

Bug Report Checklist

Overview

It's been a while since I wrote this package. It hasn't seen very much adoption in that time. I think part of the problem is that it's a bit confusing to read through.

Assigning this task to myself to audit the flow of comprehension:

  • README.md / first run experience
  • Internal code docs, especially on adding new frameworks / libraries

Additional Info

No response

TypeError: Cannot read property 'match' of undefined

    TypeError: Cannot read property 'match' of undefined

      at exports.formatComplaintLineArg (node_modules/console-fail-test/src/complaining.js:16:31)
          at Array.map (<anonymous>)
      at Object.exports.formatComplaintCall (node_modules/console-fail-test/src/complaining.js:13:51)
      at exports.formatComplaintLineWithIndex (node_modules/console-fail-test/src/complaining.js:12:70)
          at Array.map (<anonymous>)
      at formatMethodComplaint (node_modules/console-fail-test/src/complaining.js:6:73)
          at Array.map (<anonymous>)
      at Object.exports.createComplaint (node_modules/console-fail-test/src/complaining.js:20:47)
      at node_modules/console-fail-test/src/cft.js:35:39
      at Object.<anonymous> (node_modules/console-fail-test/src/environments/jest.js:10:17)

This happens when the arg passed to formatComplaintLineArg is an Error.

[Jest27] Cannot add a hook after tests have started running. Hooks must be defined synchronously.

I was forced to update my test dependencies and it seems the module console-fail-test cannot work anymore:

    Cannot add a hook after tests have started running. Hooks must be defined synchronously.

      32 |   server.listen()
      33 |   // eslint-disable-next-line @typescript-eslint/no-unsafe-call
    > 34 |   consoleFailTestModule.cft({
         |                         ^
      35 |     testFramework: 'jest',
      36 |     spyLibrary: 'jest',
      37 |     console: allowConsoleRuntimeConfig,

Any clue how we can fix this?

Document the difference between Jest setup files

Jest has two ways to run setup files:

  • setupFiles: run before the global jest variable is created
  • setupFilesAfterEnv: run after ☝️

It's confusing that you must use setupFilesAfterEnv and not setupFiles. This should be documented in Jest.md.

How to reconfigure fail on console feature ?

I have the following in setupTests.js:

const allowConsoleDefaultConfig = {
  debug: true,
  error: true,
  log: true,
  warn: true,
}

global.allowConsole = function (config = allowConsoleDefaultConfig) {
  const { debug, error, log, warn } = { ...allowConsoleDefaultConfig, ...config }
  consoleFailTestModule.cft({
    testFramework: 'jest',
    spyLibrary: 'jest',
    console: {
      debug,
      error,
      log,
      warn,
    },
  })
}

global.allowConsole({
  debug: false,
  error: false,
  log: false,
  warn: false,
})

Within a test, I run at the top of the file :

allowConsole()

I expect the test to allow console logging.

Instead, I have the initial configuration and my test fail.

How can I reconfigure the fail on console feature ?

πŸš€ Feature: Add support for Playwright

Bug Report Checklist

  • I have tried restarting my IDE and the issue persists.
  • I have pulled the latest main branch of the repository.
  • I have searched for related issues and found none that matched my issue.

Overview

Playwright is awesome. Let's add support for it!

Additional Info

Note that this library wouldn't validate a lack of console logs in the browser's console. It would just verify that the running tests don't log. I think a separate package dedicated to Playwright would be best for that.

Missing LICENSE file

The license is listed as MIT in package.json, but it's convenient to have an explicit file.

Investigate ways to allow users to provide options

Auto-detection of frameworks is nifty and all, but:

  • It's only a matter of time before folks will want to specify their own libraries
  • Some libraries are proving rather difficult to auto-detect (e.g. #10)

What things will users want to provide as an option?

  • Test framework?
  • Spy library?

How should they be able to provide these options?

Allow non-global Sinon to be used if requested

If the user requests Sinon without making sinon.spy globally available:

require("console-fail-test").cft({
    spyLibrary: "sinon",
});

...then they should still be allowed to use the framework. It's not great practice to ask folks to create globals.

How about making two Sinon spy factories? One that's called upon spyLibrary: "sinon" request that does the require/import, and one that attempts global auto-detection?

This is blocked by figuring out how to work with browser-only test frameworks such as QUnit, where the entire console-fail-test framework might need to be transpiled into roughly a single .js file to be included via a script tag.

Allow ignoring consoles on a case by case basis

There can be cases where we're console.erroring on purpose (e.g. my component tries to fetch some data -> if that fails, I'd like to log it). These will still warn in our tests though. Would be nice to have some ignore syntax, perhaps like eslint:

// console-fail-ignore-next-line
console.error('I know what I'm doing')

CI isn't running anymore

There are a couple lint errors in #76 that aren't being caught. In fact, there's no CI running on the PR at all! I'll have to take a look.

Allow banning only some methods on console

I'm trying to enable this on a large codebase with many console calls of various types. It's a bit of a mess. πŸ˜‚

In order to help users do this, let's add an API option to disable methods on the console:

import { cft } from "console-fail-test";

cft({
    console: {
        log: false,
    }
});

...where methods are banned by default.

πŸ› Bug: crash when logging circular object

Bug Report Checklist

  • I have tried restarting my IDE and the issue persists.
  • I have pulled the latest main branch of the repository.
  • I have searched for related issues and found none that matched my issue.

Expected

const x = { prop: {} };
x.prop = x;
console.log(x);

Expected:
library works

Actual

    TypeError: Converting circular structure to JSON
        at JSON.stringify (<anonymous>)

Additional Info

Found this when logging out one of @typescript-eslint's ast objects:

    TypeError: Converting circular structure to JSON
        --> starting at object with constructor 'Object'
        |     property 'object' -> object with constructor 'Object'
        --- property 'parent' closes the circle
        at JSON.stringify (<anonymous>)

      at formatComplaintLineArg (../../node_modules/console-fail-test/src/complaining.js:20:17)
          at Array.map (<anonymous>)
      at formatComplaintCall (../../node_modules/console-fail-test/src/complaining.js:17:49)
      at formatComplaintLineWithIndex (../../node_modules/console-fail-test/src/complaining.js:15:92)
          at Array.map (<anonymous>)
      at formatMethodComplaint (../../node_modules/console-fail-test/src/complaining.js:9:10)
          at Array.map (<anonymous>)
      at createComplaint (../../node_modules/console-fail-test/src/complaining.js:24:47)
      at ../../node_modules/console-fail-test/src/cft.js:36:59
      at afterEachCallback (../../node_modules/console-fail-test/src/environments/jest.js:22:39)
      at Object.<anonymous> (../../node_modules/console-fail-test/src/environments/jest.js:15:9)

πŸš€ Feature: avoid omission for new lines

Bug Report Checklist

  • I have tried restarting my IDE and the issue persists.
  • I have pulled the latest main branch of the repository.
  • I have searched for related issues and found none that matched my issue.

Overview

For example, typescript-eslint warns like below:

=============
    
WARNING: You are currently running a version of TypeScript which is not officially supported by @typescript-eslint/typescript-estree.
    
You may find that it works just fine, or you may not.
    
SUPPORTED TYPESCRIPT VERSIONS: >=3.3.1 <5.0.0
    
YOUR TYPESCRIPT VERSION: 5.0.1-rc
    
Please only submit bug reports when using the officially supported version.

=============

In this case, if tests are failed for the above reason, console-fail-test prints like below:

Oh no! Your test called the following console method:
  * log (1 call)
    > Call 0: "=============...

I couldn't understand what happened from the message.

So I think it is better to print all printed value that includes new lines characters.

Additional Info

No response

Allow overriding cft settings within individual blocks

It's a reasonable use case to want to configure cft differently in separate groups of tests. For example, in test frameworks that use describe("name", () => ... blocks, having a single describe block allow a particular console method (#35 / #36) would be good.

I think this would imply there needs to be a stack of configurations with abstracted ways to push & pop from it... Haven't done much digging yet. It'd be great to see a proposal here!

An in-range update of prettier is breaking the build 🚨

The devDependency prettier was updated from 1.18.2 to 1.19.0.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

prettier is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details
  • ❌ build_and_end_to_end: * build - Failed

Release Notes for Prettier 1.19: Long awaited Vue option, TypeScript 3.7 and new JavaScript features

diff

πŸ”— Release Notes

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

πŸš€ Feature: Better support for test assertions

Bug Report Checklist

  • I have tried restarting my IDE and the issue persists.
  • I have pulled the latest main branch of the repository.
  • I have searched for related issues and found none that matched my issue.

Overview

I've found myself writing code like 1-4 of the following spy instances:

let mockConsoleError: SpyInstance;
let mockConsoleInfo: SpyInstance;
let mockConsoleLog: SpyInstance;
let mockConsoleWarn: SpyInstance;

describe("...", () => {
	beforeEach(() => {
		mockConsoleError = vi
			.spyOn(console, "error")
			.mockImplementation(() => undefined);
		mockConsoleInfo = vi
			.spyOn(console, "info")
			.mockImplementation(() => undefined);
		mockConsoleLog = vi
			.spyOn(console, "log")
			.mockImplementation(() => undefined);
		mockConsoleWarn = vi
			.spyOn(console, "warn")
			.mockImplementation(() => undefined);
	});

	// ...
});

Surely there's got to be a more clean way of doing this?

I don't it's 100% doable right now, but I'd love to investigate getting it to work... vaguely:

import { console } from "console-fail-test";

// ...

expect(console.log.mock.calls).toMatchInlineSnapshot();

Additional Info

No response

Investigate TestCafe support

https://github.com/DevExpress/testcafe

Basic support for TestCafe should be fairly simple.

Advanced support would be very nifty, and should probably come in as separate followup issues & pull requests once this is supported:

  • Also checking browser console logs?
  • Per-environment settings? To enable just one or the other kind of log?

πŸ› Bug: [email protected] causes crash after tests

Bug Report Checklist

  • I have tried restarting my IDE and the issue persists.
  • I have pulled the latest main branch of the repository.
  • I have searched for related issues and found none that matched my issue.

Expected

console-fail-test should support the latest versions of Vitest, which is 0.30.X right now.

Actual

From JoshuaKGoldberg/create-typescript-app#373 -> https://github.com/JoshuaKGoldberg/template-typescript-node-package/actions/runs/4853883547/jobs/8650573662?pr=373

 FAIL  src/greet.test.ts > greet > logs a specified number of times when times is provided
TypeError: spy.restore is not a function
 ❯ node_modules/.pnpm/[email protected]/node_modules/console-fail-test/src/cft.js:32:17
 ❯ node_modules/.pnpm/[email protected]/node_modules/console-fail-test/src/environments/vitest.js:12:79

Additional Info

The culprit is vitest-dev/vitest#3069. Turns out spy.calls and spy.restore were undocumented meant-to-be-private properties.

πŸš€ Feature: Add support for Cypress

Bug Report Checklist

  • I have tried restarting my IDE and the issue persists.
  • I have pulled the latest main branch of the repository.
  • I have searched for related issues and found none that matched my issue.

Overview

Cypress is quite nice. Let's add support for it!

Additional Info

Note that this library wouldn't validate a lack of console logs in the browser's console. It would just verify that the running tests don't log. I think a separate package dedicated to Playwright would be best for that.

Document package releases

For reference, all you need to do to publish a new release is increase the version in package.json on the master branch; CircleCI will pick up that there's a new release.

This could be documented in Development.md.

Allow being used inside Jest beforeAll

Hello, we are trying to ugprade jest to latest and encounter issue in our upgrade branch with consol-fail-test: pass-culture/pass-culture-app-native#3271

example:

  ● Test suite failed to run

    Cannot add a hook after tests have started running. Hooks must be defined synchronously.

      30 |
      31 | global.beforeAll((done) => {
    > 32 |   consoleFailTestModule.cft({
         |                         ^
      33 |     testFramework: 'jest',
      34 |     spyLibrary: 'jest',
      35 |     console: allowConsoleRuntimeConfig,

      at eventHandler (node_modules/jest-circus/build/eventHandler.js:113:11)
      at Object.after (node_modules/console-fail-test/src/environments/jest.js:10:13)
      at Object.<anonymous>.exports.cft (node_modules/console-fail-test/src/cft.js:20:21)
      at Object.cft (src/tests/setupTests.js:32:25)

It seems it does not support the latest jest version, is there a workaroud to keep using console-fail-test or a way we can help?

πŸ› Bug: unnecessary files published

Bug Report Checklist

  • I have tried restarting my IDE and the issue persists.
  • I have pulled the latest main branch of the repository.
  • I have searched for related issues and found none that matched my issue.

Expected

The npm package contains the minimal files required to run the code

Actual

https://unpkg.com/browse/[email protected]/

image

Additional Info

Can either use .npmignore to ignore files, or package.json files: string[] to specify which files to publish.

Add example projects under docs/

It'd be good to have a few example projects to show how to use console-fail-test. I'm thinking at least a couple for each supported environment:

  • One basic one with calling cft() at the top of a test file
  • One more advanced one with calling cft(...) (providing some settings) in a setupTests.js file or equivalent

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.