sproutapp / pavlov Goto Github PK
View Code? Open in Web Editor NEWA BDD framework for your Elixir projects
License: MIT License
A BDD framework for your Elixir projects
License: MIT License
Example:
defmodule Something.SomethingElse do
def do_it do
IO.puts "Doing it!"
42
end
end
defmodule MyTest do
use Pavlov.Case, async: true
alias Something.SomethingElse, as: Potato
describe "something" do
let :result, do: Potato.do_it
it "can't find Potato" do
assert result == 42
end
end
end
Expected to pass, but get:
** (UndefinedFunctionError) undefined function Potato.do_it/0 (module Potato is not available)
let
subjects should be able to be shared between child contexts. Maybe an Agent that stores each subject and then redefines it for contexts that are substrings of the original context could work?
I am using pavlov to test controllers in my phoenix app.
By default, phoenix generates test/support/conn_case.ex
that sets @endpoint MyApp.Endpoint
in using
block in the following way:
defmodule MyApp.ConnCase do
@moduledoc """
This module defines the test case to be used by
tests that require setting up a connection.
Such tests rely on `Phoenix.ConnTest` and also
imports other functionality to make it easier
to build and query models.
Finally, if the test case interacts with the database,
it cannot be async. For this reason, every test runs
inside a transaction which is reset at the beginning
of the test unless the test case is marked as async.
"""
use ExUnit.CaseTemplate
using do
quote do
# Import conveniences for testing with connections
use Phoenix.ConnTest
alias MyApp.Repo
import Ecto.Model
import Ecto.Query, only: [from: 2]
import MyApp.Router.Helpers
# The default endpoint for testing
@endpoint MyApp.Endpoint
end
end
setup tags do
unless tags[:async] do
Ecto.Adapters.SQL.restart_test_transaction(MyApp.Repo, [])
end
:ok
end
end
Thanks to this you have to set endpoint once.
Unfortunately, when I use contexts in pavlov-based tests it seems that I have to set @endpoint
again and again in each of contexts. This will work:
defmodule MyApp.UserControllerTest do
use MyApp.ConnCase, async: true
use Pavlov.Case, async: true
import Pavlov.Syntax.Expect
let(:base_url) do
"/api/user"
end
describe "POST (base_url)" do
context "if no Authorization header was passed" do
@endpoint MyApp.Endpoint
it "returns status code of 401 Unauthorized" do
response = post conn(), base_url
expect response.status |> to_eq 401
end
end
end
end
But this won't:
defmodule MyApp.UserControllerTest do
use MyApp.ConnCase, async: true
use Pavlov.Case, async: true
import Pavlov.Syntax.Expect
let(:base_url) do
"/api/user"
end
describe "POST (base_url)" do
@endpoint MyApp.Endpoint
context "if no Authorization header was passed" do
it "returns status code of 401 Unauthorized" do
response = post conn(), base_url
expect response.status |> to_eq 401
end
end
end
end
or
defmodule MyApp.UserControllerTest do
use MyApp.ConnCase, async: true
use Pavlov.Case, async: true
import Pavlov.Syntax.Expect
let(:base_url) do
"/api/user"
end
@endpoint MyApp.Endpoint
describe "POST (base_url)" do
context "if no Authorization header was passed" do
it "returns status code of 401 Unauthorized" do
response = post conn(), base_url
expect response.status |> to_eq 401
end
end
end
end
It will end up with the following error:
** (RuntimeError) no @endpoint set in test case
stacktrace:
(phoenix) lib/phoenix/test/conn_test.ex:183: Phoenix.ConnTest.dispatch/5
test/controllers/user_controller_test.exs:34
I guess that context/describe macros change scope in such a way that @endpoint
is no longer accessible, but it's extremely unconvinient to define it multiple times and this breaks DRY principle.
Obviously you can say that this is not pavlov issue, but it's just wrong that Phoenix.ConnTest depends on @endpoint
but well, I think it's pavlov that should somehow address this issue because module attributes are quite common in Elixir world.
When upgrading to Elixir 1.2, because of the new with
keyword, the following compile error is thrown:
** (CompileError) test/github/watcher_test.exs:2: cannot import Pavlov.Mocks.Matchers.with/2 because it conflicts with Elixir special forms
(elixir) src/elixir_import.erl:92: :elixir_import.calculate/6
(elixir) src/elixir_import.erl:22: :elixir_import.import/4
expanding macro: Pavlov.Mocks.__using__/1
test/github/watcher_test.exs:2: SlackCoder.Github.WatcherTest (module)
(elixir) expanding macro: Kernel.use/1
test/github/watcher_test.exs:2: SlackCoder.Github.WatcherTest (module)
expanding macro: Pavlov.Case.__using__/1
test/github/watcher_test.exs:2: SlackCoder.Github.WatcherTest (module)
(elixir) expanding macro: Kernel.use/2
test/github/watcher_test.exs:2: SlackCoder.Github.WatcherTest (module)
(elixir) lib/code.ex:363: Code.require_file/2
(elixir) lib/kernel/parallel_require.ex:47: anonymous fn/2 in Kernel.ParallelRequire.spawn_requires/5
Putting import Kernel, except: [with: 1]
before use Pavlov.Case
does not seem to remedy the issue...
It should be memoized in the same example, but not across different examples.
Deeply related to #19, can probably be fixed via a similar implementation.
I have a couple of suggestions with regards to the mocking section.
When mocking multiple functions on the same module, the syntax currently looks like:
allow(Mockable) |> to_receive(do_something: fn -> :error end)
Mockable |> to_receive(do_something_else: fn -> :success end)
If you attempt the following:
allow(Mockable) |> to_receive(do_something: fn -> :error end)
allow(Mockable) |> to_receive(do_something_else: fn -> :success end)
Then an error will be raised:
** (ErlangError) erlang error: {:already_started, #PID<0.343.0>}
stacktrace:
src/meck_proc.erl:96: :meck_proc.start(Fixtures.Mockable, [:no_link])
This is because :meck.new
assigns the name Elixir.Fixtures.Mockable_meck
to the process it starts. Since it is a named process, there is a collision.
I have a potential fix for this issue if you wish to use it then I can raise a pull request. This fix looks like:
def allow(module, opts \\ [:no_link]) do
case Process.whereis(String.to_atom("#{to_string(module)}_meck")) do
nil -> setup_meck(module, opts)
_ -> module
end
end
defp setup_meck(module, opts) do
:meck.new(module, opts)
# Unload the module once the test exits
on_exit fn ->
:meck.unload(module)
end
module
end
The second proposal is to allow chaining from to_receive
e.g.
allow(Mockable)
|> to_receive(do_something: fn -> :error end)
|> to_receive(do_something_else: fn -> :success end)
A naive implementation of this is to simply return the module
instead of the {module, mock, value}
tuple that is currently returned.
A more future proof implementation may be to return a struct for the module from both allow
and to_receive
and change the first argument of to_receive
to be the struct and pattern match the module out of it. This means that the other parameters will still be available if any additional functions require them in the future.
Let statements don't retain their memoization when referenced inside of a before statement
Here's a failing spec:
context "Callbacks" do
setup_all do
Agent.start_link(fn -> 0 end, name: :memoized_let)
:ok
end
before :each do
Agent.update(:memoized_let, fn acc -> 0 end)
something
:ok
end
it "only invokes the letted block once" do
assert Agent.get(:memoized_let, fn acc -> acc end) == 1
something
assert Agent.get(:memoized_let, fn acc -> acc end) == 1
end
end
It fails with the following message:
1) .let, Callbacks, only invokes the letted block once (:"Elixir.PavlovCaseTest.let.Callbacks")
test/case/case_test.exs:91
Assertion with == failed
code: Agent.get(:memoized_let, fn acc -> acc end) == 1
lhs: 2
rhs: 1
stacktrace:
test/case/case_test.exs:94
.................................
Finished in 1.1 seconds (1.0s on load, 0.1s on tests)
91 tests, 1 failure, 5 skipped
Randomized with seed 69531
This doesn't follow the behavior in rspec
It seems that internally in pavlov context and test names are concatenated and converted into atom.
Unfortunately it leads to case in which it's super simple to hit internal erlang limit of atom length (255 characters) just with simple nesting.
Test case:
iex(2)> :erlang.binary_to_atom("Elixir.MyApp.UserTest.POST (base_url).if Authorization header was passed.and it uses Bearer scheme.and the token passed was granted.and access token owner has permissions to create new record of this type.but no POST data was passed.and something something happens", :utf8)
** (SystemLimitError) a system limit has been reached
:erlang.binary_to_atom("Elixir.MyApp.UserTest.POST (base_url).if Authorization header was passed.and it uses Bearer scheme.and the token passed was granted.and access token owner has permissions to create new record of this type.but no POST data was passed.and something something happens", :utf8)
Should support:
xit
xdescribe
xcontext
Obviously xit
is the most important feature to support, but the others would be nice to have.
My tests were working fine on 0.1.2. Upgrade to 0.2.2 has compile issues when using aliases in callbacks.
The following test runs on 0.1.2, but gives the following compile error on 0.2.2:
==> example
Compiled lib/example.ex
Generated example.app
** (CompileError) nofile:7: One.__struct__/0 is undefined, cannot expand struct One
(elixir) src/elixir_map.erl:44: :elixir_map.translate_struct/4
(stdlib) lists.erl:1352: :lists.mapfoldl/3
(stdlib) lists.erl:1353: :lists.mapfoldl/3
# lib/example.ex
defmodule Example do
end
defmodule Example.One do
defstruct one: 1
def get, do: inspect(__MODULE__)
end
# test/example_test.exs
defmodule ExampleTest do
use Pavlov.Case, async: true
alias Example.One
before :each do
{:ok, one: %One{one: 3}}
end
it "gets Example.One" do
assert One.get == "Example.One"
end
describe "nested" do
end
end
before_each
, run before every example in a batchbefore_all
, run before a batch of examplesafter_each
, run after every example in a batchafter_all
, run after a batch of examplesIt would be nice to be able to pattern match to succeed:
expect(%{abc: 123, def: 234}) |> to_match(%{abc: 123})
would pass but:
expect(%{abc: 345, def: 234}) |> to_match(%{abc: 123})
would not
When I use let
I receive one error like this
test/models/channel/input/stream/rtp_test.exs:1: warning: this clause cannot match because a previous clause at line 174 always matches
per each let
used in the test case.
It does not affect running tests but it floods the console.
Hello
I am trying to do the following
describe ".start_link/0" do
before(:each) do
allow Supervisor |> to_receive(start_link: fn(options) -> {:ok, self()} end)
end
it "calls Supervisor.start_link" do
This.start_link
expect Supervisor |> to_have_received :start_link
end
end
but I get the following error ** (FunctionClauseError) no function clause matching in Pavlov.Mocks.to_receive/2
I am using master.
Would it be possible to add Gherkin support like Cucumber, so that we can have test cases in human readable .feature files and the actual code would be just step definiotions?
I am using pavlov to test Phoenix Framework controllers.
Consider the following example:
defmodule MyApp.UserTest do
use MyApp.ConnCase, async: true # default ConnCase generated by Phoenix
use Pavlov.Case, async: true
import Pavlov.Syntax.Expect
let(:base_url) do
"/api/user"
end
describe "POST (base_url)" do
context "if no Authorization header was passed" do
@endpoint MyApp.Endpoint
subject do
conn() |> post base_url
end
it "returns status code of 401 Unauthorized" do
expect subject.status |> to_eq 401
end
end
end
end
It will end up with
** (CompileError) test/controllers/user_test.exs:52: function conn/0 undefined
(stdlib) lists.erl:1337: :lists.foreach/2
test/controllers/user_test.exs:48: (module)
test/controllers/user_test.exs:47: (module)
If I add module names:
defmodule MyApp.UserTest do
use MyApp.ConnCase, async: true # default ConnCase generated by Phoenix
use Pavlov.Case, async: true
import Pavlov.Syntax.Expect
let(:base_url) do
"/api/user"
end
describe "POST (base_url)" do
context "if no Authorization header was passed" do
@endpoint MyApp.Endpoint
subject do
Phoenix.ConnTest.conn() |> Phoenix.ConnTest.post base_url # Here we add module names
end
it "returns status code of 401 Unauthorized" do
expect subject.status |> to_eq 401
end
end
end
end
it will end up with the following error:
** (CompileError) nofile:52: you must require Phoenix.ConnTest before invoking the macro Phoenix.ConnTest.post/2
(elixir) expanding macro: Kernel.|>/2
nofile:52: :"Elixir.MyApp.UserControllerTest.POST (base_url).if no Authorization header was passed".subject/0
(elixir) lib/module.ex:360: Module.eval_quoted/4
(elixir) lib/enum.ex:543: anonymous fn/3 in Enum.each/2
(elixir) lib/enum.ex:1275: anonymous fn/3 in Enum.reduce/3
(elixir) lib/stream.ex:717: Stream.do_transform_each/3
The same happens if I try to use let
.
That makes let/subject
syntax useless for testing Phoenix controllers.
let/subject
IMO should be always evaluated in the scope of the calling function.
I'd like to add my own matchers, it seems to load them by looping functions in Pavlov.Matchers. Is it possible to add my own?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.