Comments (11)
I found the same defect opened in last year. IDK why allure(integration) stores so much information(
from allure-csharp.
Hi, @EvgenyMarchuk !
If a test has complicated parameters (objects with lots of transitive references, large collections, ORM entities. etc), there might be issues like that.
Could you please narrow the issue to a single test and describe its parameters' types? That will help us develop better constraints to impose on our conversion algorithm. That will also help me to come up with a workaround for you.
from allure-csharp.
Hi @delatrie
I get a similar error when passing Mock objects (based on the Moq library - https://github.com/devlooped/moq) for my services or repositories as parameters in my test.
Serialization of such parameters leads to the fact that the test report (json) can be more than 700 MB in size, and the resulting html in single-file mode takes up more than 800 MB and does not open in the browser.
from allure-csharp.
@TonEnfer
Could you please show how you parametrize your tests (just one example, with personal data stripped)?
from allure-csharp.
@delatrie
It's quite difficult for me to prepare an example so that it does not contain personal data, but later I will try to prepare an example with which it is possible to reproduce the problem
from allure-csharp.
Thank you. It will help me better understand some edgy use cases when our serialization algorithm performs poorly.
At the moment, you may use the type formatters API to workaround the issue:
using Allure.Net.Commons;
class ToStringFormatter<T> : TypeFormatter<T>
{
public string Format(T? value) => value?.ToString() ?? "null";
}
using Allure.Net.Commons;
using NUnit.Framework;
using Moq;
[SetUpFixture]
class AllureSetup // Should be in the root namespace of the test assembly or without a namespace at all
{
[OneTimeSetUp]
public void SetupAllure()
{
AllureLifecycle.Instance.AddTypeFormatter(new ToStringFormatter<Mock<IMyRepository>>());
/* Formatters for other mocks and types that cause troubles */
}
}
from allure-csharp.
@delatrie
Here is the minimal code with which I can reproduce the problem.
Running this test produces a report of 138 MB in size. The parameter string is very (VERY!) long. Unfortunately, my computer is not powerful enough to analyze it (or Iβm using the wrong tools, but VSCode and Notepad++ canβt handle it)
If the Setup call is removed from the body of the Test
method, the report file takes on an acceptable size.
Calls to the Bar
and VerifyAll
methods do not affect , but are left for the overall picture
.csproj
:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net8.0</TargetFramework>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Allure.Xunit" Version="2.12.0" />
<PackageReference Include="AutoFixture" Version="4.18.1" />
<PackageReference Include="AutoFixture.AutoMoq" Version="4.18.1" />
<PackageReference Include="AutoFixture.Xunit2" Version="4.18.1" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.9.0" />
<PackageReference Include="Moq" Version="4.20.70" />
<PackageReference Include="xunit" Version="2.7.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.5.7">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
</ItemGroup>
</Project>
Class1.cs
;
using AutoFixture;
using AutoFixture.AutoMoq;
using AutoFixture.Xunit2;
using Moq;
using Xunit;
namespace Allure482;
public class Class1
{
[
Theory,
AutoMockData
]
public void Test(
[Frozen] Mock<IFoo> testInterface
)
{
testInterface.Setup(x => x.Bar());
testInterface.Object.Bar();
testInterface.VerifyAll();
}
}
public interface IFoo
{
void Bar();
}
public class AutoMockDataAttribute(): AutoDataAttribute(() => new Fixture().Customize(new AutoMoqCustomization()));
from allure-csharp.
A much easier way to reproduce the problem:
public static IEnumerable<object[]> GetMembers => [[() => { }]];
[Theory, MemberData(nameof(GetMembers))]
public void Test(Action checkResultAction)
{
checkResultAction();
}
This code produces a JSON report slightly larger than 12.5 MB.
This does not cause the exception described in the Issue, but it does a good job of showing what is causing the problem.
It seems that Allure should include formatters for at least Func
and Action
by default.
However this will not solve the problem when Func
or Action
is a property on an object that is serialized to JSON so it seems that a JSON converter for these types is required
To resolve the issue, you can specify the following ContractResolver
as the ContractResolver
in the SerializerSettings
of the FormatFunctions
class:
private class NewtonsoftContractResolver: DefaultContractResolver
{
protected override JsonContract CreateContract(Type objectType)
{
if (typeof(Delegate).IsAssignableFrom(objectType))
{
return base.CreateStringContract(objectType);
}
return base.CreateContract(objectType);
}
}
This solution will serialize all delegates as a string.
P.S.
There is one more nuance that I would like to mention, but which is not directly related to this problem - using the current version of Allure significantly slows down the execution of tests - in my CI/CD the test execution time (about 1000 different tests) increased by about 1.5 times. I didn't investigate the problem because I don't have time for it, but I can assume that this may be due to JSON serialization in general and the not very high performance of Newtonsoft.Json in particular.
P.S.S.
I was very excited when I first saw Allure, and extremely disappointed when I first tried it.
I am very upset by allure-csharp's long release cycle, as well as the lack of testing, which leads to new bugs after each release.
Because of these problems, I have not been able to use Allure for many months.
from allure-csharp.
Hello! When is it going to be fixed?
Thanks!
from allure-csharp.
The problem is that given
[Test]
public void Test(T a) { /* ... */ }
we can't decide whether to include a
in the report or not for all possible values of T
. The same goes for nested properties/fields of type T. Sometimes, people put unpredictably complicated data in test parameters. While I personally see it more as an antipattern (Allure aside, that makes reporting of any sort much more complex), we still need a solution that covers all cases.
We can do some heuristics to exclude some types that obviously don't help us in the report and only blow up its size. As suggested by @TonEnfer, we can safely exclude any delegate types (including, but not limited to, Action
, Func<T>
, and variants with args). If someone wants to contribute on that matter, I will gladly accept those PRs.
However, heuristics only partially solves the problem. From my perspective, we need the following (in addition to heuristics):
- Limit parameter values displayed in the report (a digest of the original value could be added as a new hidden attribute to help Allure put apart test results). That will help reduce the size but doesn't fix performance issues.
- Provide flexible opt-in/opt-out mechanics for tests, fixtures, and parameters that can be tuned via the configuration/API to give users more control over what they want to see in the report. That needs to be designed appropriately as, to the best of my knowledge, we don't have that in other adapters either.
- Allure event listeners for scenarios that can't be implemented with the above mechanisms. That's something we already have in allure-java and will eventually be implemented here as well.
Sorry for not giving you any specific dates. I want you to please be sure that this issue will be my top priority as soon as I have some time for allure-csharp.
from allure-csharp.
Xunit:
[ModuleInitializer]
public static void ConfigureAllure()
=> AllureLifecycle.Instance.AddTypeFormatter(new CompactStringTypeFormatter<CreateRequest>());
Custom logic to reduce size
public class CompactStringTypeFormatter<T> : TypeFormatter<T>
{
public override string Format(T value)
{
var type = typeof(T);
Debug.WriteLine(JsonConvert.SerializeObject(value));
var values = type.GetProperties()
.Where(x => x.CanRead && (!x.PropertyType.IsClass || x.PropertyType.IsPrimitive || x.PropertyType == typeof(string)))
.Select(property => new { property.Name, Value = property.GetValue(value) })
.Where(x => x.Value != null)
.Take(5)
.Select(x => $"{x.Name}:{x.Value}");
return string.Join(',', values);
}
}
from allure-csharp.
Related Issues (20)
- AllureIssue and AllureTMS annotations will not build up the full URL by getting allureConfig.json
- An adapter for Reqnroll HOT 6
- ArgumentNullException when calling a local function with a captured value and [AllureStep]
- Add support for async step functions returning ValueTask
- Class Cleanup method ClaimsDocumentforPolicyFeature.FeatureTearDown Failed
- Interpolate class fields and properties in attribute-based step names
- Allure.NUnit: Report exceptions from OneTimeSetUp methods of class fixtures HOT 2
- Random exception during test execution HOT 6
- Implement Allure event listeners
- Support SelectiveRun on ARM processors HOT 2
- Allow custom (re)initialization for Allure lifecycle
- Allure matches 2 error categories and duplicates failure for both of them
- Exception: Cannot access a disposed object
- Serialize more types with ToString HOT 2
- Update Allure.Reqnroll adapter to the latest (2.0.0) Reqnroll version
- Exception in AllureDisplayIgnoredAttribute if a test with cases marked with [Ignore]
- Console logs from OneTimeSetup methods are not displayed in the report
- The test case failed using FluentAssertions or Playwright are shown as broken instead of failed HOT 5
- There is no way to debug a method marked as AllureStep HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from allure-csharp.