После обновления allure adapter в отчете не отображается русская локализация шагов:

До обновления адаптера:

Версия specflow и фича файл не изменялись

• Allure version: 2.2.0
• Test framework: [email protected]
• Allure adaptor: [email protected]

AllureLifecycle.Instance.AllureConfiguration properties are accessible for changing:
unickq/allure-nunit#13

Allure instance should be configured via json.

Not able attach a file in Allurereport using Nunit and .netcore

here is the code i am using.
AllureLifecycle.Instance.AddAttachment("Attachment Title", "application/txt", @"C:\Testing\AllureTest.txt");
it gives the follwoing exception

Value cannot be null. Parameter name: key

StackTrace:
at System.Collections.Concurrent.ConcurrentDictionary2.ThrowKeyNullException() at System.Collections.Concurrent.ConcurrentDictionary2.TryGetValue(TKey key, TValue& value)
at System.Collections.Concurrent.ConcurrentDictionary`2.get_Item(TKey key)
at Allure.Commons.Storage.AllureStorage.Get[T](String uuid)
at Allure.Commons.AllureLifecycle.AddAttachment(String name, String type, Byte[] content, String fileExtension)
at Allure.Commons.AllureLifecycle.AddAttachment(String name, String type, String path)
at
)

• Allure version: Nunit.Allure(1.0.5)
• Test framework: Nunit(3.11.0)

Also, i used the code below. This also gives the same error

AllureLifecycle.AddAttachment("Attachment Title", "application/txt", @"C:\Testing\AllureTest.txt");

So how does an attachment can be added?

I'm submitting a ...

• bug report

The following test fails because it doesn't wait when the step finishes.

[AllureNUnit]
[TestFixture]
public class AllureAsyncStepTest
{
	private bool _isStep1Finished;

	[AllureStep("Step1")]
	public async Task Step1()
	{
		_isStep1Finished = false;
        
		await Task.Delay(500);

		_isStep1Finished = true;
	}

	[Test]
	public async Task SimpleStepTest1()
	{
		await Step1();
        
		Assert.That(_isStep1Finished, Is.True);
	}
}

I expect that the async step should finish before assertion because I use await.

If I comment the attribute [AllureStep("Step1")] then the test works as expected. Like this:

[AllureNUnit]
[TestFixture]
public class AllureAsyncStepTest
{
	private bool _isStep1Finished;

	// [AllureStep("Step1")]
	public async Task Step1()
	{
		_isStep1Finished = false;
        
		await Task.Delay(500);

		_isStep1Finished = true;
	}

	[Test]
	public async Task SimpleStepTest1()
	{
		await Step1();
        
		Assert.That(_isStep1Finished, Is.True);
	}
}

• Test framework: [email protected]
• Allure adaptor: [email protected]

I want to be able to mark my SpecFlow scenarios with a tag @Severity:critical/blocker/etc.
I looked at SpecFlow repo and they don't have any allure code.
So I suppose I need to add it to Allure specflow plugin. Please advice where I can do it.

I'm submitting a ...

• bug report
• feature request
• support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

At the moment there is no way to add link pattern.

What is the expected behavior?

See usage examples at allure-framework/allure-java#1

Please tell us about your environment:

Hello,
Created a simple framework with Scenario Outline tests, it can be found here. Copied allureConfig.json file into bin/debug directory. Nuget versions:

• Nunit 3.9.0;
• SpecFlow 2.3.0;
• SpecFlow.Allure 2.2.0.11;
• Net framework 4.6.2.
  After tests execution, using search, I found test-results directory in the next location:

C:\Users\Valentins\AppData\Local\Temp\allure-results

And it contains just .json files, like:

52bb84abaec342e1bf8e7fbc3e46a181-container

and

9ab91b6fc6e84eafbda735c49ccfcb1c-result

Can anyone help to understand, how to make this folder to appear in the tests and how to normally open Test results to see it in Html file?

Thanks in advance,
Valentins.

Hey Guys,
Thank you for last help.
Is it possible to adjust allure for .NET 4.5 or maybe it is difficult?

Cheers

I'm submitting a ...

• bug report
• feature request
• support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

Tests that are Ignored(Skipped) using Assert.Ignored() in BeforeScenario hook marked as broken in result.json file in allure-results folder

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

1. Create any test using Specflow and mark it as @Skipped, e.g:
   @Skipped
   Scenario: Test skipped test (Bug-test1)
   Given I go to any portal
   When I set "123456789060000" value for field
   And I click check button
   Then I should see something
2. Create BeforeScenarioHook. e.g:
   [BeforeScenario(Order = 1)]
   [Scope(Tag = "@Skipped")]
   public void BeforeScenarioSkipped()
   {
   Assert.Ignore("reason");
   }
3. Run tests and see output in allure-results folder:
   there is result.json file with data:
   .........
   .........
   {
   "links": [],
   "name": "Test skipped test (Bug-test1)",
   "status": "broken",
   "statusDetails": {
   "known": false,
   "muted": false,
   "flaky": false,
   "message": "Bug-test1",
   "trace": "NUnit.Framework.IgnoreException: Bug-test1\r\n at NUnit.Framework.Assert.Ignore(String message, Object[] args)\r\n at ......Hooks.ScenarioHooks.BeforeScenarioSkipped....\r\n at TechTalk.SpecFlow.Bindings.BindingInvoker.InvokeBinding(IBinding binding, IContextManager contextManager, Object[] arguments, ITestTracer testTracer, TimeSpan& duration)\r\n at Allure.SpecFlowPlugin.AllureBindingInvoker.InvokeBinding(IBinding binding, IContextManager contextManager, Object[] arguments, ITestTracer testTracer, TimeSpan& duration)"
   },
   "stage": "finished",
   "steps": [],
   "attachments": [],
   "parameters": [],
   "start": 1531925761229,
   "stop": 1531925761252
   }

What is the expected behavior?

"status": "skipped"

What is the motivation / use case for changing the behavior?

In such cases when we are using it with Jenkins during Publishing Allure Report build is marked as Unstable. Because when allure report is generated using allure-result folder there is a result.json file with status for test as broken ( for skipped/ignored test).

Please tell us about your environment:

• Allure version: Allure.Commons (2.3.64.1)
• Test framework: Specflow
• Allure adaptor: SpecFlow Adapter (SpecFlow.Allure 2.3.64.1)
• Generate report using: AllureCommandline (2.6.0)

Other information

NOTE: if we use standart @ignore tag provided by Specflow, the Specflow.Allure doesn't generate result.json at all.

Hey, please have a look at the error below - it looks like SpecFlow made some breaking changes to the API making Allure incompatible:

Works fine with versions 3.5.*

I'm submitting a ...

• bug report
• feature request
• support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

Test report generated from .NET trx file had incorrect timestamp: "Allure Report 1/19/1970". This issue may related to allure-framework/allure1#667.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

1. Running a simple unit test class by MSTest
2. Use allure-ci command to generate report allure generate TestResults , where "TestResults" is the MSTest result folder
3. Report is created successfully
4. Open index.html file
5. Report timestamp is incorrect: "1/19/1970"

What is the expected behavior?

Expected to have a correct timestamp.

What is the motivation / use case for changing the behavior?

Please tell us about your environment:

• Allure version: 2.12.1
• Test framework: MSTest v14.0.23107.0
• Allure adaptor: allure-csharp
• Generate report using: allure-ci

Other information

When trying to run a parameterized test using [AllureXunitTheory] attribute along with [MemberData] in Test Explorer, I get the following error in the output:
No test is available in . Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.

I have followed the identical solution as provided in https://github.com/allure-framework/allure-csharp/blob/main/Allure.XUnit.Examples/ExampleParameterisedTests.cs

However, there is no issue when running another non-parameterized test with [AllureXUnit] and I can execute and generate the report appropriately.

Please note that in my current solution I am using:
Allure version: 2.19.0
Allure.XUnit: 2.1.3
xunit: 2.4.1
xunit.runner.visualstudio: 2.4.5
Microsoft.AspNetCore.Mvc.Testing: 6.0.8
Microsoft.NET.Test.Sdk: 17.3.1
Visual Studio version: 4.8.04084
.NET 6

Other information

I'm submitting a ...

• bug report

What is the current behavior?

This library is outputting to console each start of a step, the step name and the result of each step.

seems to me some leftover debug logging.
this issue create lots of console logs for tests with many steps and make it difficult to find other relevant information that may have been ouputed to the console.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

any test with a step can generate this on the version Allure.NUnit 2.9.2-preview.1.

 [AllureStep("isLogedinValidationStep")]
    private void isLoggedin()
    {
    }
    [AllureStep("LoginSubStep")]
    private void sublogin()
    {
    }
    [AllureStep("LoginStep")]
    private void login()
    {
        sublogin();
    }

    [Test]
    [AllureName("test1")]
    public void Test1()
    {
        login();
        isLoggedin();
    }

What is the expected behavior?

I believe that the allure library should output nothing to the console by default.

What is the motivation / use case for changing the behavior?

cleaner console output

Please tell us about your environment:

• Test framework: [email protected]
• Allure adaptor: [email protected]

Other information

seems to me that the console logs are generated here:

I'm submitting a ...

• bug report

What is the current behavior?

When files for a report are generated, links are translated to lower case.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

1. Write simple test with specflow
2. Add to test tag @ TEST-1234
3. Change allureConfig.json:
   a) set "allure -> links -> issue" to "https://myEXample.com/{issue}"
   b) set "specflow -> links -> issue" to "^(TEST-\ \d+)" (no space between / and / )
4. Build project with test
5. Run a test in the console (i'm using a Package Manager Console)
6. Check allure-results file named like 45rydh4u4urudheh3hwjajeh-result.json:
   "links": [
   {
   "name": "TEST-1234",
   "url": "https://myexample.com/TEST-1234"
   ...
   ]

I have longer link, like "https://jira.domain.ru/secure/Tests.jspa#/testCase/TEST1234" and upper case is critical for me, because otherwise the link does not work in the allure report

What is the expected behavior?

Expected that register wasn't changed

"links": [
...
    "url": "https://myEXample.com/TEST-1234"
...
]

What is the motivation / use case for changing the behavior?

Link doesnt work correctly in allure report

Please tell us about your environment:

• Specflow.Allure version: 2.4.1.1
• Test framework: Specflow 2.4.0 + Specflow.NUnit 2.4.0
• Allure commons: 2.4.1.1
• Generate report using: Allure commandline 2.8.1

Other information

For any question please contact by mail: [email protected]
I'm interested in a quick fix.

Please note that the test works fine. I get this error only if Specflow.allure is installed.

Package:
"Allure.Commons" version="2.4.1.1"
"CsvHelper" version="7.1.1"
"SpecFlow" version="2.4.0"
"SpecFlow.Allure" version="2.4.1.1"
"SpecFlow.xUnit" version="2.4.0"
"System.ValueTuple" version="4.5.0"
"xunit" version="2.4.0"

Please find the error details:

Test Name: Verify the the Powered by link
Test FullName: Automation.Framework.Features.LoginPageFeature.VerifyTheThePoweredByLink
Test Source: C:\Users\swati.sharma\source\repos\Automation.Framework\Automation.Framework\Features\Login.feature : line 5
Test Outcome: Failed
Test Duration: 0:00:00.001

Result StackTrace:
----- Inner Stack Trace #1 (System.TypeInitializationException) -----
at Allure.SpecFlowPlugin.AllureTestTracerWrapper..ctor(ITraceListener traceListener, IStepFormatter stepFormatter, IStepDefinitionSkeletonProvider stepDefinitionSkeletonProvider, SpecFlowConfiguration specFlowConfiguration)
----- Inner Stack Trace -----
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.StreamReader..ctor(String path, Encoding encoding, Boolean detectEncodingFromByteOrderMarks, Int32 bufferSize, Boolean checkHost)
at System.IO.File.InternalReadAllText(String path, Encoding encoding, Boolean checkHost)
at System.IO.File.ReadAllText(String path)
at Allure.Commons.AllureLifecycle..ctor(String jsonConfigurationFile)
at Allure.Commons.AllureLifecycle..ctor()
at Allure.Commons.AllureLifecycle.get_Instance()
at Allure.SpecFlowPlugin.AllureTestTracerWrapper..cctor()
----- Inner Stack Trace #2 (Xunit.Sdk.TestClassException) -----
Result Message:
System.AggregateException : One or more errors occurred.
---- System.TypeInitializationException : The type initializer for 'Allure.SpecFlowPlugin.AllureTestTracerWrapper' threw an exception.
-------- System.IO.FileNotFoundException : Could not find file 'C:\Users\swati.sharma\AppData\Local\Temp\61766ac7-10ce-4e46-9566-141538edfcc9\61766ac7-10ce-4e46-9566-141538edfcc9\assembly\dl3\25915518\007786f7_d54fd401\allureConfig.json'.
---- The following constructor parameters did not have matching fixture data: FixtureData fixtureData

I'm submitting a ...

• bug report

Добрый день!
В классе с тестами поставлен атрибут AllureDisplayIgnored.
Тесты заигноренные через атрибут Ignore выводятся в отчете как скипнутые.

Но если тест был заигнорен в TestCaseData:

yield return new TestCaseData(AgentStub.AgentWithoutBankAuthAndBranchWithoutContracts,
                "Внимание, вы не авторизованы ни в одном банке! " +
                "Обратитесь к ответственному за работу с банками в дивизионе.")
                .SetName("{m}_NoBankAuth_BranchNoBankContracts")
                .SetDescription("Авторизация должна упасть с ошибкой - нет авторизации в банках и у филиала нет контрактов с банками.")
                .Ignore("2019/07/30 - Логика еще не реализована");

то в отчете он не выводится.

Hello Guys,
I'm fighting how to use your framework in my project. I'm using:

• C# - .NET 4.62
• NUnit - 3.8.1
• Installed Allure.SpecFlow

and I read your tutorial on GitHub page ofc, it doesn't work...
From your page, it's only necessary to install Allure.SpecFlow should I do yet?
It's very important to me. So please help me.
Cheers,
Michał

I'm submitting a ...

• bug report
• feature request
• support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

The NuGet package SpecFlow.Allure has only a lower boundary on version number.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

You can install the NuGet package to a project with a newer SpecFlow version that is not compatible with the one used to create the SpecFlow plugin.

What is the expected behavior?

Having a boundary for max level. In the current case SpecFlow < 3.4

What is the motivation / use case for changing the behavior?

When we at SpecFlow don't guarantee that a plugin works for multiple minor versions. They will work for all patch versions of a minor version.
So please always limit your NuGet packages to the supported minor version. Your users are trying to upgrade SpecFlow and then get errors because of your plugin.

Examples:
https://stackoverflow.com/questions/64647958/allure-error-after-updating-specflow-package/64649316#64649316
https://stackoverflow.com/a/64132740/3155323

Hi!

I'm submitting a ...

• bug report
• feature request

What is the current behavior?

Each test case of a parameterized test is interpreted as an independent new test without parameters.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

The minimal demo to reproduce is already in the examples :)

In the generated report for this test, there are two independent tests with different name/fullName and without parameters:

{
  "uuid": "1c3834b2-cdec-4ade-85e1-ce98d530e550-TestTheory(a: 1, b: 1)",
  "name": "TestTheory(a: 1, b: 1)",
  "fullName": "TestTheory(a: 1, b: 1)",
  ...
}

{
  "uuid": "5b7b5215-5a38-404d-b246-6b9a63f32f48-TestTheory(a: 1, b: 2)",
  "name": "TestTheory(a: 1, b: 2)",
  "fullName": "TestTheory(a: 1, b: 2)",
  ...
}

although, here is one test simply with different parameter values.

What is the expected behavior?

I would like to be able to get a report with the same names and with the listed parameters:

{
  "uuid": "1c3834b2-cdec-4ade-85e1-ce98d530e550-TestTheory(a: 1, b: 1)",
  "name": "TestTheory",
  "fullName": "TestTheory",
  "parameters": [
    {
      "name": "a",
      "value": "1"
    },
    {
      "name": "b",
      "value": "1"
    }
  ],
  ...
}

{
  "uuid": "5b7b5215-5a38-404d-b246-6b9a63f32f48-TestTheory(a: 1, b: 2)",
  "name": "TestTheory",
  "fullName": "TestTheory",
  "parameters": [
    {
      "name": "a",
      "value": "1"
    },
    {
      "name": "b",
      "value": "2"
    }
  ],
  ...
}

What is the motivation / use case for changing the behavior?

The current behavior is acceptable when the parameter values are immutable. However, when random generation of test data is used (e.g. AutoFixture), it becomes a problem, because the parameter values are always unique, which means there is no test history, and the number of unique tests increases with each publication of the report.

Please tell us about your environment:

• Test framework: xunit 2.4.1
• Generate report using: Allure.XUnit 2.9.2-preview.1

Other information

To generate random test data, I use AutoFixture.Xunit2 4.17.0

JFYI

Archived - https://github.com/unickq/allure-nunit
Added Suggested Alternatives - https://www.nuget.org/packages/NUnit.Allure/ and https://www.nuget.org/packages/NUnit.Allure.Steps/

Ping me if anything else needed for deprecation on my side

What is the current behavior?

There is no way to create custom labels. The tag was created on any custom labels (ex.: '@layer:e2e')

What is the expected behavior?

Custom label containing only 1 colon char should be recognised as custom label (ex.: '@layer:e2e' should create label named 'layer' with value 'e2e')

I'm submitting a ...

• bug report
• feature request
• support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

What is the expected behavior?

What is the motivation / use case for changing the behavior?

Please tell us about your environment:

• Allure version: 2.1.0
• Test framework: [email protected]
• Allure adaptor: [email protected]
• Generate report using: [email protected]

Other information

I would like to know when will have plugin to generate reports through XUni

I'm submitting a ...

• bug report
• feature request
• support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

Cannot use edit and continue on debug mode when allureStepAttribute is used

see this video of the failure

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

in the code bellow put a break point on the test
run in debug mode
edit the string
continue execution
it fails

code example

What is the expected behavior?

we should be able to use edit and continue since it is such a great tool for development

What is the motivation / use case for changing the behavior?

making edit and continue work as expected.

Please tell us about your environment:

<TargetFramework>net7.0</TargetFramework>

<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.3.2" />
<PackageReference Include="NUnit" Version="3.13.3" />
<PackageReference Include="NUnit3TestAdapter" Version="4.2.1" />
<PackageReference Include="NUnit.Analyzers" Version="3.3.0" />
<PackageReference Include="NUnit.Allure" Version="1.2.1.1" />

Other information

I believe to fix this it is only necessary to update to AspectInjector 2.7

as a temporary fix i copied the code for AllureStepAttribute in my project and use the latest aspectInjector and that works for me.

the following allure elements' support is required:

• static label Feature
• static label Epic
• static label Story
• static label Tag
• static label Owner
• static label description
• statit label DisplayName
• description with the source from the documentation (see Java's @description(useJavaDoc = true))
• steps
• support of setUp blocks (with steps)
• support of tearDown blocks (with steps)
• support of static labels (see Java' @CustomLabel("static value"))
• support of dynamic labels (see java's Allure.label("key", "value")
• support of attachments
• support of links
• support of parameterized tests
• spport of the exclusion of a parameter from the historyID
• support of test selective run

I'm submitting a ...

• bug report

What is the current behavior?

I am using specflow 3.5.5 and Specflow allure 3.5.0.6 with MSTest, when I run any test cases, it throws an error saying System.ArgumentNullException: 'Value cannot be null. Parameter name: key'

if I remove specflow Allure from project, test cases execute without any issues

Please tell us about your environment:

• Allure version: 3.5.0.6
• Test framework: Specflow MSTest
• Specflow : 3.5.5

you can find my code here https://github.com/allure-framework/allure-csharp

Hey there.

This is possible duplicate of #83. But I don't see example code there, so not sure.

The issue here - test with await TestError throws error when Step is trying to find it's parent in AllureStorage.cs line58

System.ArgumentNullException : Value cannot be null.
Parameter name: key
   at System.Collections.Concurrent.ConcurrentDictionary`2.TryGetValue(TKey key, TValue& value)
   at System.Collections.Concurrent.ConcurrentDictionary`2.get_Item(TKey key)
   at Allure.Commons.Storage.AllureStorage.Get[T](String uuid)
   at Allure.Commons.Storage.AllureStorage.AddStep(String parentUuid, String uuid, StepResult stepResult)
   at Allure.Commons.AllureLifecycle.StartStep(String parentUuid, String uuid, StepResult stepResult)
   at Allure.Commons.AllureLifecycle.StartStep(String uuid, StepResult result)

Below are code examples you can reproduce it.

public class Tests {
 private AllureWrapper allure = new AllureWrapper();

 [SetUp]
 public void SetUp() {
  AllureLifecycle.Instance.CleanupResultDirectory();
  allure.SetUp();
 }

 [TearDown]
 public void TearDown() {
  allure.TearDown();
 }

 // The test which is failing
 [Test]
 public async Task TestError() {
  await Task.Delay(1000);
  allure.Step();
 }

 // Same test without error
 [Test]
 public async Task TestNoError() {
  Task.Delay(1000).GetAwaiter().GetResult();
  allure.Step();
 }
}

public class AllureWrapper {
 private TestResultContainer testResultContainer = new TestResultContainer {
  uuid = "cont"
 };
 private TestResult testResult = new TestResult {
  uuid = "test"
 };


 public void SetUp() {
  AllureLifecycle.Instance.StartTestContainer(testResultContainer);
  AllureLifecycle.Instance.StartTestCase(testResultContainer.uuid, testResult);
 }


 public void TearDown() {
  AllureLifecycle.Instance.StartTestCase(testResult);
  AllureLifecycle.Instance.StopTestContainer(testResultContainer.uuid);

  AllureLifecycle.Instance.WriteTestContainer(testResultContainer.uuid);
  AllureLifecycle.Instance.WriteTestCase(testResult.uuid);
 }

 public void Step(string name = "Step") {
  var step = new StepResult() {
   name = name
  };
  AllureLifecycle.Instance.StartStep(Guid.NewGuid().ToString(), step);
  AllureLifecycle.Instance.StopStep();
 }
}

Allure.Commons 3.0.0.10

Hi @Bakanych ,

Current implementation of AllureLifecycle.Instance expects to find allureConfig.json in the same dir Allure.Commons.dll is. But

public AllureLifecycle() : this(Path.Combine(Path.GetDirectoryName(typeof (AllureLifecycle).Assembly.Location), AllureConstants.CONFIG_FILENAME))
{
}

Just tested 2.4.1.1 with .netCore 2.1 and found that Allure.Commons.dll is not in outDir, so allureConfig.json written in VS couldn't be found. Nuget path:
C:\Users\%USER%\.nuget\packages\allure.commons\2.4.1.1\lib\netstandard2.0\Allure.Commons.dll

So there need to be a way to set a path of allure config, since you disabled config property setters #16

Maybe smth like:

AllureLifecycle.SetInstance(new AllureLifecycle("json"))
AllureLifecycle.SetInstance(new AllureConfiguration())

I'm submitting a feature request

What is the current behavior?

It is not possible to have selective test runs from Allure TestOps.

What is the expected behavior?

Select test cases in Allure TestOps and run them in the Jenkins pipeline selectively.

Please tell us about your environment:

• Allure version: 4.6.1
• Test framework: Specflow
• Allure adaptor: Allure.SpecFlow 2.9.2-preview.1

Hello guys. I have a project on .net core and specflow 3 with NUnit provider. Without Allure.SpecFlow dependency added my tests run successfully.

But as soon as I add allure in the dependency, I get the following errors:
Message: OneTimeSetUp: System.MissingMethodException : Method not found: 'Void BoDi.ObjectContainer.RegisterTypeAs(System.String)'.

This is my code with bindings for passing the browser to test methods. Somewhere here (or elsewhere) I need to do something for Allure to work:

using BoDi;
using TechTalk.SpecFlow;

namespace InternalAutomation.Helpers
{
    [Binding]
    class Hooks
    {
        private Browser _browser;
        private readonly IObjectContainer _objectContainer;

        public Hooks(IObjectContainer objectContainer)
        {
            _objectContainer = objectContainer;
        }

        [BeforeScenario]
        public void Setup()
        {
            _browser = new Browser();
            _objectContainer.RegisterInstanceAs(_browser);
        }
        
        [AfterScenario]
        public void TearDown()
        {
            _browser.Termination();
        }
    }
}

I could not find a working example of a repository with similar dependencies.
Please help me figure it out. I really want to get a nice test report.

Hello!
After dicussion on webinar you told me about adding issue for Support SpecFlow 3.9 and more

Could you please resolve the current version SpecFlow supportin problem?

I'm submitting a

• bug report

What is the current behavior?

There is no console output attachment after tests

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

1. Run test:
   [Test] public void WriteOutputTest() { Console.WriteLine("Hello world!"); }

2. Generate report.

3. Text output attachment is missing

What is the expected behavior?

There is attachment in test

Please tell us about your environment:

• Allure version: Allure.NUnit 2.9.2-preview.1
• Test framework: NUnit 3.13.3

Hi,
Please add support for Specflow 3.0
https://specflow.org/2019/updating-to-specflow-3/

Thank you

Hi there,

Do we really need to read properties from .json file?

It's not really comfortable to use allure-commons with non-standard .NET because you need to download the whole .net st 2.0 dependencies.

Removed JSON configuration - and it still looks good ;)

@Bakanych can you please comment something about it? Would happy to chat with you when you have time

Hello there, I've been playing with new versions (not released ones) and
I've faced an issue, when I'm trying to use AllureStep inside a OneTimeSetUp method, it fails with following error:

OneTimeSetUp: System.ArgumentNullException : Value cannot be null. (Parameter 'key')

I'm submitting a ...

• bug report

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

    [AllureNUnit]
    [AllureParentSuite("Root Suite")]
    public class BaseTest
    {
        [OneTimeSetUp]
        public void CleanupResultDirectory()
        {
            AllureExtensions.WrapSetUpTearDownParams(() => { AllureLifecycle.Instance.CleanupResultDirectory(); },
                "Clear Allure Results Directory");
            Method(); // FAILS
        }

        [SetUp]
        public void Setup()
        {
            Method(); // doesnt fail
        }

        [AllureStep()]
        private void Method()
        {
        }
    }

What is the expected behavior?

I can use Step inside OneTimeSetUp

Please tell us about your environment:

• Allure version: 2.9.1-preview.6-nunit-fixtures

Other info

Looks like the problem in AllureNUnitAttribute class, the BeforeTest method is called only after OneTimeSetUp, so Allure thinks there is no tests started yet when we call a method with Step attribute inside OneTimeSetUp method, that's why it fails.

maybe align configuration with https://github.com/allure-framework/allure-js/blob/master/packages/allure-cucumberjs/README.md ?

Originally posted by @baev in #278 (comment)

I'm submitting a ...

• bug report

What is the current behavior?

Nothing happens, no folder, no files

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

Execute this code

[AllureNUnit]
class AllureFailOneTimeSetUp
{
[OneTimeSetUp]
public void Init()
{
throw new Exception("Something bad happened");
}
[Test]
public void SimpleTest()
{
Assert.True(true);
}
}

What is the expected behavior?

Folder with result of execution

What is the motivation / use case for changing the behavior?

If fail appears in the start of the program, it hard to find that, and need to read output

Please tell us about your environment:

• Allure version: 2.13.3
• Test framework: [email protected]
• Allure adaptor: [email protected]
• Framework: NetCore 3.1

Other information

I'm submitting a ...

• bug report

What is the current behavior?

When the 'SpecFlow.Plus.LivingDocPlugin' library is installed in a test project - you can't see 'test body' in the test result.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

1. Create a specflow test project and add the specflow.allure to the project, as recommended in the documentation1 documentation2
2. Add the 'SpecFlow.Plus.LivingDocPlugin' library to your test project

What is the expected behavior?

We should see Test body area in any cases

What is the motivation / use case for changing the behavior?

Incorrect behavior

Please tell us about your environment:

• Specflow.Allure: 3.5.0.4
• Specflow.NUnit: 3.5.14

Just tried to use https://github.com/allure-framework/allure-csharp/tree/main/Allure.SpecFlowPlugin

Scenario:
#language: ru
@123456 @tms:42 @link:http://example.org

Функция: Просто тест что все вместе работает
Сценарий: Очень простой сценарий
Дано Давайте что-то откроем

Step:

[Given(@"Давайте что-то откроем")]
public async void Givensomething()
{
    Response = await Client.GetAsync(_endpoint);
    Assert.Equal((HttpStatusCode.OK), Response.StatusCode);

}

Looks at the test output:
Xunit.Sdk.EqualException
Assert.Equal() Failure
Expected: OK
Actual: Unauthorized
at net_api_cukes.UsersResourceSteps.Givensomething() in

And data of resulting json:

{
"uuid": "9c83ed2f1a854e5f879883e797bf23ca",
"historyId": "Очень простой сценарий",
"fullName": "Очень простой сценарий",
"labels": [
{
"name": "thread",
"value": ".NET Long Running Task"
},
{
"name": "host",
"value": "0000NBB007WQ6M0"
},
{
"name": "feature",
"value": "Просто тест что все вместе работает"
},
{
"name": "tag",
"value": "123456"
},
{
"name": "tag",
"value": "tms:42"
},
{
"name": "tag",
"value": "link:http://example.org"
}
],
"links": [],
"name": "Очень простой сценарий",
"status": "passed",
"statusDetails": {
"known": false,
"muted": false,
"flaky": false
},
"stage": "finished",
"steps": [
{
"name": "Дано Давайте что-то откроем",
"status": "passed",
"statusDetails": {
"known": false,
"muted": false,
"flaky": false
},
"stage": "finished",
"steps": [],
"attachments": [],
"parameters": [],
"start": 1669807961943,
"stop": 1669807961984
}
],
"attachments": [],
"parameters": [],
"start": 1669807961939,
"stop": 1669807962109
}

What is the current behavior?

It is not possible to attach a file according to the documentation advice:
Documentation link: https://github.com/allure-framework/allure-csharp/tree/main/Allure.SpecFlowPlugin#attachments

Please fix documentation with:

AllureLifecycle.Instance.AddAttachment(path, "Attachment Title");

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem
Create any test using Specflow
Scenario: Test skipped test
Given I go to any portal
When I set "123456789060000" value for field
And I click check button
Then I should see something
Create BeforeScenarioHook. e.g:
[BeforeScenario(Order = 1)]
public void BeforeScenarioSkipped()
{
Assert.Fail("reason");
}
Please tell us about your environment:
Allure.Commons.2.3.67.1
Test framework: Specflow

Hi guys
I'm creating a new project using NET CORE 5.0
I'm using the latest versions of Specflow that supports net5.0. Everything was working fine until I've added SpecFlow.Allure 3.5.0.4
Here my dependencies

  <ItemGroup>
    <PackageReference Include="NUnit" Version="3.12.0" />
    <PackageReference Include="SpecFlow" Version="3.5.14" />
    <PackageReference Include="SpecFlow.Allure" Version="3.5.0.4" />
    <PackageReference Include="SpecFlow.Tools.MsBuild.Generation" Version="3.5.14" />
    <PackageReference Include="SpecRun.Runner" Version="3.5.8" />
    <PackageReference Include="SpecRun.SpecFlow" Version="3.5.8" />
  </ItemGroup>

I have my allureConfig.json with copy always and also I have my specflow.json with the assembly "assembly": "Allure.SpecFlowPlugin"
Once I've added Specflow.Allure I tried to execute my tests and returns this error:

The type initializer for 'Allure.SpecFlowPlugin.AllureBindingInvoker' threw an exception. -> Couldn't find Allure configuration file. Please either specify full path to allureConfig.json in the ALLURE_CONFIG environment variable or place allureConfig.json to the 'C:\Users\fescobar\.nuget\packages\allure.commons\3.5.0.4\lib\netstandard2.0' folderSystem.TypeInitializationException: The type initializer for 'Allure.SpecFlowPlugin.AllureBindingInvoker' threw an exception.
 ---> System.IO.FileNotFoundException: Couldn't find Allure configuration file. Please either specify full path to allureConfig.json in the ALLURE_CONFIG environment variable or place allureConfig.json to the 'C:\Users\fescobar\.nuget\packages\allure.commons\3.5.0.4\lib\netstandard2.0' folder
   at Allure.Commons.AllureLifecycle.GetDefaultJsonConfiguration()
   at Allure.Commons.AllureLifecycle..ctor(String jsonConfigurationFile)
   at Allure.Commons.AllureLifecycle.get_Instance()
   at Allure.SpecFlowPlugin.AllureBindingInvoker..cctor()
   --- End of inner exception stack trace ---

I've created this small project to reproduce the issue
https://github.com/fescobar/SpecFlow3Net5

If you remove the "SpecFlow.Allure" package and remove the Allure assembly configuration in the specflow.json the project works perfectly.

Thank you in advance.

I'm submitting a:

• bug report
• feature request
• support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

when we use AddAttachment(filePath) method in [AfterStep] specflow Hook then attachments aren't shown under each step. All attachments are added after all feature steps.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

Create simple Specflow project where screen can be made:
1) Add feature file:
@regression
Feature: Sample test
In order to verify mobile app
As logged in user
I want to see correct behavior

Scenario: Sample test
Given I click on "Sample button"
When I Swipe Left to "sample element" element
Then I should see "sample name" is "sample value" for card

    [StepDefinition(@"I click on ""(.*)""")]
    public void GivenIClickOnElement(string elementName)
    {
    }

    [StepDefinition(@"I Swipe Left to ""(.*)"" element")]
    public void SwipeLeft(string elementName)
    {
    }

    [Then(@"I should see ""(.*)"" is ""(.*)"" for card)]
    public void ThenIShouldSeeIsForCardWithLastDigits(string elementName, string expectedValue, string cardDigits)
    {
    }

2)
Add [AfterStep] hook
[Binding]
public class BaseHook {
private readonly ScenarioContext _scenarioContext;
public BaseHook(ScenarioContext scenarioContext)
{
_scenarioContext = scenarioContext;
}
[BeforeTestRun]
public static void RegisterServices()
{
//there run browser, or mobile app
}
[AfterStep]
public void TakeScreenShot()
{
var screenShotFilePath = ScreenShotHelper.SaveScreenShot(_scenarioContext);
AllureLifecycle.Instance.AddAttachment(screenShotFilePath, _scenarioContext.StepContext.StepInfo.Text);
}
}

What is the expected behavior?

Attachments are shown under each step in case of using [AfterStep] specflow Hook Attribute

What is the motivation / use case for changing the behavior?

It's more convenient to see screens under each step. Because if we have 20 steps for example it will be a bit messy

Please tell us about your environment:

NUnit.Allure Version 1.0.6
Allure.Commons Version 2.4.2.4
Specflow.Allure Version 2.4.2.4
Specflow Version 2.4.1
SpecFlow.NUnit Version 2.4.1

allure serve allure-results

Other information

In .json result I see:
"steps": [
{
"name": "Given I click on "Sample Element"",
"status": "passed",
"statusDetails": {
"known": false,
"muted": false,
"flaky": false
},
"stage": "finished",
"steps": [],
"attachments": [],
"parameters": [],
"start": 1570178164235,
"stop": 1570178164284
},
{
"name": "When I Swipe Left to "Sample" element",
"status": "passed",
"statusDetails": {
"known": false,
"muted": false,
"flaky": false
},
"stage": "finished",
"steps": [],
"attachments": [],
"parameters": [],
"start": 1570178164870,
"stop": 1570178164871
},
{
"name": "Then I should see "Sample Name" is "Samle" for card "",
"status": "passed",
"statusDetails": {
"known": false,
"muted": false,
"flaky": false
},
"stage": "finished",
"steps": [],
"attachments": [],
"parameters": [],
"start": 1570178165161,
"stop": 1570178165162
}
],
"attachments": [
{
"name": "I click on "Sample element"",
"source": "7b8d735728114f0c87ea75e8324933de-attachment.png",
"type": "image/png"
},
{
"name": "I Swipe Left to "Sample" element",
"source": "642e3d30d3944fce9b9cf10b2622d1f6-attachment.png",
"type": "image/png"
},
{
"name": "I should see "Sample name" is "Sample" for card"",
"source": "e7e4d3a8c6fc4e76900a05329084fb73-attachment.png",
"type": "image/png"
},
],
"parameters": [],
"start": 1570178164227,
"stop": 1570178165426
}
NOTE
If we add attachment in each step definition everything works fine.
[StepDefinition]
public void AddAttachment() {
var fileNameBase = $"{TestContext.CurrentContext.WorkerId}{DateTime.Now:yyyyMMdd_HHmmss}";
var artifactDirectory = Path.Combine(EnvironmentManager.AssemblyDirectoryPath, "Results");
var fileName = Path.Combine(artifactDirectory, fileNameBase + "_log.txt");
using (File.Create(fileName)) { }
AllureLifecycle.Instance.AddAttachment(fileName);
}

I'm submitting a ...

• [x ] bug report

What is the current behavior?

Testscenarios with two similar but different suite names are grouped twice under one suitename

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

Create a testscenario in a Specflow feature file. Add two tags on the top of file. (This is a specflow feature file with dutch language 🙂 )

@ThisIsMyTag @ThisIsMyTag2
Functionaliteit: This is my feature file

Scenario: Testing this
Gegeven im on the homepage
Als i click on the button
Dan i see the form

I then add these tags in my allureConfig.json file
"suites": { "parentSuite": "^(ThisIsMyTag|ThisIsMyTag2)", "suite": "^suite:?(.+)", "subSuite": "^subSuite:?(.+)" },

Now, when this testcase fails and I go to my Allure report. Under Suites I see my defect, shown twice under suitename ThisIsMyTag. ThisIsMyTag2 is not shown.

What is the expected behavior?

I would expect the failed testcase to show up under ThisIsMyTag once and under ThisIsMyTag2 once. Instead of twice under ThisIsMyTag, which is happening right now.

What is the motivation / use case for changing the behavior?

This is not expected behaviour, and looks like a problem in how the tags (suites) are processed.

Please tell us about your environment:

• Allure version: Allure Commons 2.4.2.4
• Test framework: nUnit 3.11.0
• Allure adaptor: Specflow Allure 2.4.2.4
• Generate report using: allure-commandline-2.7.0

Other information

I'm submitting a ...

• bug report
• feature request
• support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

Only one test result is displayed for multiple rows in Example table.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

1. Run a Scenario Outline, which contains more then one row (5 for example) in Example table

What is the expected behavior?

More then one result (5 for example) is displayed in Allure report.

What is the motivation / use case for changing the behavior?

Wrong information in Test report, 1 result on 5 tests (if 5 rows in Example table)

Please tell us about your environment:

• Allure version: 2.3.52.1
• Test framework: SpecFlow
• Allure adaptor: SpecFlow.Allure 2.3.52.1
• Generate report using: allure-commandline 2.6.0

Other information

I'm submitting a feature request

Currently, I'm not able to utilize all advantages from Allure TestOps on the C# stack.
For instance, the Test Layer, AllureId attributes, and custom fields aren't possible to set from the code.
So, I need to do a lot of manual work to assign that from Allure TestOps.

I need something as LabelAnnotation from allure-java-commons

environment:

• Allure version: Allure TestOps
• Test framework: nunit 3
• Allure adaptor: NUnit.Allure

I'm submitting a ...

• bug report
• feature request
• support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

We execute a test and it raises and TechTalk.SpecFlow.BindingException but it isn't catched by the AllureLifCycle object and it is marked as passed.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

We are using Specflow 2 with NUnit 3.
In order to reproduce the scenario is: you have to decored two step methods with the same specflow test. Beacause we have did it in our test suite by error.

    [When(@"I save and back")]
    public void WhenIGoBackCancelingAnyChanges()
    {
        ...
    }


    [When(@"I save and back")]
    public void WhenISaveAndBack()
    {
       ...
    }

When you do this by error, NUnit catches the TechTalk.SpecFlow.BindingException and only it's logged by NUnit. The ScenarioContext don't have logged the exception and specflow don't excute the BeforeStep hook.

In allure results files the failing step has the status none:
"name": "And I save and back",
"status": "none",

And the next steps are marked as skipped. But the information about the fail is in the TestResult object of NUnit.

What is the expected behavior?

We expect that the test was marked as Broken.

What is the motivation / use case for changing the behavior?

Please tell us about your environment:

• Allure version: 2.4.2.4
• Test framework: Specflow 2 and NUnit 3
• Allure adaptor: [email protected]
• Generate report using: Allure-Docker-Service

Other information

I'm submitting a ...

• bug report
• feature request
• support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

Tests won't run with latest version of specflow

Please tell us about your environment:

• Allure version: 3.1.0.6
• Test framework: [email protected]
• Specflow: 3.4.3
• Allure adaptor: 3.1.0.6

Other information

I tried to update all actual dependencies on the project. At this moment latest available SpecFlow version is 3.4.3 and SpecFlow.Allure 3.1.0.6

An attempt to run any test produces an error

OneTimeSetUp: System.TypeInitializationException : The type initializer for 'Allure.SpecFlowPlugin.AllureTestTracerWrapper' threw an exception.
----> System.TypeInitializationException : The type initializer for 'Allure.SpecFlowPlugin.PluginHelper' threw an exception.
----> System.MissingMethodException : Method not found: 'Void TechTalk.SpecFlow.FeatureInfo..ctor(System.Globalization.CultureInfo, System.String, System.String, System.String[])'.

Reason of the failure is the change of the API on SpecFlow side
SpecFlowOSS/SpecFlow@1a614f2#diff-f48c953716c89e50045366bbfdfd3fddL15
Since new version of FeatureInfo constructor now takes additional mandatory argument, its initialization fails

I'm submitting a ...

• bug report
• feature request
• support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

I get an error when using Allure with async/await

What is the expected behavior?

Allure should works as without async/await

What is the motivation / use case for changing the behavior?

I think using AsyncLocal instead ThreadLocal in stepContext will fix it like in NUnit framework
https://github.com/nunit/nunit/blob/master/src/NUnitFramework/framework/Internal/TestExecutionContext.cs

Please tell us about your environment:

• Allure version: 2.1.0
• Test framework: [email protected]
• Allure adaptor: [email protected]
• Generate report using: [email protected]

Other information

Latest SpecFlow 2.4 has been changed a little bit - plugin v2.3.67 doesn't work anymore.