Comments (4)
Thanks a lot to reporting this... Yes ... sure it could be useful but I can see some trouble on implementing this check: what about if the value is an expression that return the same value? we cannot know the function value a compile time (i.e. could be a random value) or check something like 4 + 1 == 3 + 2
.
So.... I don't see a good way to implement this check without any false positive. I way could be introduce an annotation to enable duplicate cases but introduce a new syntax seams too much.
from rstest.
I'm still trying to wrap my head around how rstest works, but are you saying that at no point during the runtime of rstest does it know the (evaluated) values of expressions? If so, duplicate detection would indeed be rather difficult to implement...
Assuming rstest does know the values:
On the surface, even when the case
s evaluate to the same value I think I'd like to know about it (and eliminate the duplication). Whether the value is hardcoded or obtained by evaluating an expression seems irrelevant to me - if it's the same test case it's the same test case.
This would admittedly not quite work for randomized expressions and would lead to flaky behavior(/false positives)1. I agree having a way to disable/enable duplicate detection would be necessary. I also agree introducing new syntax is a bit much - I'm wondering if passing argument to #[rstest]
could be an option instead (and wouldn't be consider "syntax").
Footnotes
-
I'd personally be inclined to say that would be a problem with the testing approach rather than rstest, but I can definitely see why one would still want to support the use case. So I understand the concern. ↩
from rstest.
Yes, it could be possible to check it a run time but...
- Not every type implement
Eq
orPartialEq
- Every test run in a separate thread and implement a check need a shared R/W structure to implement it: this make independent tests not really isolated
- Random input is just a trivial example but I can imagine function that return values based o some state : not a good test design but a valid use case
So my taste is implement it with:
- Lexicographic expression matching and not runtime value matching in order to raise an explicit and a clear compile time error
- Add an annotation
#[allow_duplicate]
to disable this check
from rstest.
Every test run in a separate thread and implement a check need a shared R/W structure to implement it: this make independent tests not really isolated
Hadn't considered this, I agree this severely limits the desirability of checking duplicates at runtime.
So my taste is implement it with:
- Lexicographic expression matching and not runtime value matching in order to raise an explicit and a clear compile time error
- Add an annotation
#[allow_duplicate]
to disable this check
Lexicographic expression matching sounds reasonable to me and I agree a #[allow_duplicate]
annotation makes sense (the opposite, something like #[check_duplicates]
makes less sense I think).
Let me add one more alternative to the discussion: what about some optional mode/tool for detecting duplicates that is entirely separate from using rstest normally. For example, this could be a "debug" mode of operation for detecting duplicates where (simplified:) tests are ran in one thread and duplicate values will be reported.
The reason I'm suggesting this is because I believe looking at values rather than lexicographic expressions will result in fewer false negatives (see details below). Admittedly this would come at the cost of convenience as it would require some extra command/flag/etc. to find duplicates...
-
Lexicographic expressions
negative positive true 1 + 1 != 1 + 2
1 + 2 == 1 + 2
false 2 + 1 != 1 + 2
rand() == rand()
**: it would always falsy report this. Even though two evaluations of
rand()
might return the same value, they should probably be considered unique in practice. -
Values
negative positive true 1 + 1 != 1 + 2
1 + 2 == 1 + 2
false rand() == rand()
***rand() == rand()
****: it might falsy report this if
rand()
happens to return the same value twice. The likelihood of this would depend on the output distribution ofrand()
.
***: it might fail report ifrand()
has, say, 2 possible return values and happens to return both of them. While in practice this might be rare becauserand()
is much more likely to return one of the possible return values.
Not every type implement
Eq
orPartialEq
Just for the record, I'd imagine such types should never be considered equal for the purposes of duplicate detection.
from rstest.
Related Issues (20)
- Access files in parent directory to a repo HOT 9
- Generate tests from json files HOT 2
- [question] Test with different specializations of a generic type? HOT 4
- '#[once]' cleanup after tests executed HOT 1
- `#[ignore]` at the top only applies to the first case HOT 3
- Consider supporting `wasm-bindgen-test` for async test HOT 3
- Allow customizing the "length" of the test case names from the path
- Separating test input data & expected values? HOT 1
- Unable to silence lints HOT 1
- New #[files] feature does not recompile on file change HOT 5
- Consider allowing to rename the crate by passing an attribute `crate = ...` to the proc-macro HOT 9
- vscode/rust-analyzer "debug overlay action" not working anymore HOT 2
- Question: tests failed by memory allocation of x bytes failure` when only running many cases once HOT 2
- Allow `clippy::too_many_arguments` HOT 6
- Please clearly state the MSRV in the README and documentation and use the "rust-version" flag. HOT 2
- Add capability to ignore arguments HOT 3
- [vscode] text wrapping for run/debug test buttons
- Misleading error messaging when defining a test that includes a lifetime argument HOT 1
- Support argument destructuring for `#[values(...)]` HOT 1
- Injecting static types HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rstest.