thwaitesy / mstesthacks Goto Github PK

Most team members have no problems running the unit-tests. Yet when I run them on my machine, I get this exception and stacktrace.

• The Domain.Tests.dll.config file is present

• The MSTestHacks\debug.txt is present and written to
  [23-01-18 02:48:37] ------------------
  [23-01-18 02:48:37] Starting Execution
  [23-01-18 02:48:38] Successfully created datasource: Domain.Tests.Core.Request.Aggregate.StatusTests.FinalizationTests.RequestTypes, Iteration Count: 2, Elapsed Time : 00:00:00.3647961

• The MSTestHacks\RuntimeDataSources folder and files are there and written to

Result StackTrace:

at System.Security.AccessControl.Win32.SetSecurityInfo(ResourceType type, String name, SafeHandle handle, SecurityInfos securityInformation, SecurityIdentifier owner, SecurityIdentifier group, GenericAcl sacl, GenericAcl dacl)
   at System.Security.AccessControl.NativeObjectSecurity.Persist(String name, SafeHandle handle, AccessControlSections includeSections, Object exceptionContext)
   at System.Security.AccessControl.NativeObjectSecurity.Persist(String name, AccessControlSections includeSections, Object exceptionContext)
   at System.Security.AccessControl.NativeObjectSecurity.Persist(String name, AccessControlSections includeSections)
   at System.Security.AccessControl.FileSystemSecurity.Persist(String fullPath)
   at System.IO.File.SetAccessControl(String path, FileSecurity fileSecurity)
   at System.Configuration.Internal.WriteFileContext.DuplicateTemplateAttributes(String source, String destination)
   at System.Configuration.Internal.WriteFileContext.DuplicateFileAttributes(String source, String destination)
   at System.Configuration.Internal.WriteFileContext.Complete(String filename, Boolean success)
   at System.Configuration.Internal.InternalConfigHost.StaticWriteCompleted(String streamName, Boolean success, Object writeContext, Boolean assertPermissions)
   at System.Configuration.Internal.InternalConfigHost.System.Configuration.Internal.IInternalConfigHost.WriteCompleted(String streamName, Boolean success, Object writeContext, Boolean assertPermissions)
   at System.Configuration.Internal.InternalConfigHost.System.Configuration.Internal.IInternalConfigHost.WriteCompleted(String streamName, Boolean success, Object writeContext)
   at System.Configuration.Internal.DelegatingConfigHost.WriteCompleted(String streamName, Boolean success, Object writeContext)
   at System.Configuration.UpdateConfigHost.WriteCompleted(String streamName, Boolean success, Object writeContext)
   at System.Configuration.MgmtConfigurationRecord.SaveAs(String filename, ConfigurationSaveMode saveMode, Boolean forceUpdateAll)
   at System.Configuration.Configuration.SaveAsImpl(String filename, ConfigurationSaveMode saveMode, Boolean forceSaveAll)
   at System.Configuration.Configuration.Save(ConfigurationSaveMode saveMode)
   at MSTestHacks.RuntimeDataSource.AttachRuntimeDataSources..cctor() in c:\projects\mstesthacks\MSTestHacks\RuntimeDataSource\AttachRuntimeDataSources.cs:line 163

Result Message: Unable to create instance of class Domain.StatusTests.CancelTests. Error: System.UnauthorizedAccessException: Attempted to perform an unauthorized operation..

I have an App.config (it already contains some configuration information, but I've tried it empty also). Using VS2015 RC and latest Nuget package. I also tried the examples from your repo, but attempting to open one of the source files for the tests crashed VC2015 RC.

Cannot work out what is happening.

I'm receiving the following error when I try and run under VS2013:
Message: Data source 'Namespace.Unters1.Stuff' cannot be found in the test configuration settings

I created a class directly from the source in your example, see below :

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using MSTestHacks;

namespace Namespace
{
[TestClass]
public class UnitTest1 : TestBase
{
private IEnumerable Stuff
{
get
{
//This could do anything, get a dynamic list from anywhere....
return new List { 1, 2, 3 };
}
}

[TestMethod]
[DataSource("Namespace.UnitTest1.Stuff")]
public void TestMethod1()
{
  var number = this.TestContext.GetRuntimeDataSourceObject<int>();

  Assert.IsNotNull(number);
}

}
}

When I attempt to run I get the error listed above. Any suggestions?

Hi,
I Have installed MSTestHawks 2.3.2.0 in VS2017, with .net core 2.0 as the target framework and created a new test project.
I used your code but got the error as:

Error CS0012 The type 'TestContext' is defined in an assembly that is not referenced. You must add a reference to assembly 'Microsoft.VisualStudio.QualityTools.UnitTestFramework, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.

I noticed that the message "Package 'MSTestHacks 2.3.20' was restored using '.NetFramework, Version=v4.6.1' instead of the project target framework '.NetCoreApp, Version=v2.0' This package may not be fully compatible with your project" displayed on my Solution Explore.

I have installed following packages from NuGet in my project:

"Microsoft.NET.Test.Sdk" Version="15.5.0"
"MSTest.TestAdapter" Version="1.2.0"
"MSTest.TestFramework" Version="1.2.0"
"MSTestHacks" Version="2.3.20"

I also tried to install "Microsoft.VisualStudio.QualityTools.UnitTestFramework v11.0.50727.1", which the problem still exists. And "Microsoft.VisualStudio.QualityTools.UnitTestFramework.Update v15.0.26228" (This package is not compatible with netcoreapp2.0)

The code I used is as below:

[TestClass]
public class UnitTest1 : TestBase
{
  [TestMethod]
  [DataSource("Namespace.UnitTest1.Stuff")]
  public void TestMethod1()
  {
      var number = this.TestContext.GetRuntimeDataSourceObject< int >();

      Assert.IsNotNull(number);
  }

  private IEnumerable< int > Stuff
  {
      get
      {
          //This could do anything, fetch a dynamic list from anywhere....
          return new List< int > { 1, 2, 3 };
      }
  }
}

So, is this really a compatibility problem of .Net Core 2.0 or do I miss something to get it be right?
Thanks.

Using NUnit with parametrized test if the parameters are different (I think based on ToString() of the paraters passed to the TestMethod) in the TestExplorer I can see N test as the number of the rows.
So that I can run each test as a single test. (while using MStest I need to run every time all the tests)
Is there any way for doing this?

Edit:

Problem is found. You can close the Issue.

You main example doesn't work for me. I have VS 2013 Update 4 with Resharper and various other plugins.

1. I created a new C# Unit Test project called MsTestHacksExample.
2. Installed MsTestHacks from NuGet.
3. Made sure app.config exists. (It was created by your NuGet package. Well, actually Newonsoft.Json created it).
4. Added this test.

using System;
using System.Collections.Generic;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using MSTestHacks;

namespace MsTestHacksExample
{
    [TestClass]
    public class UnitTest1 : TestBase
    {
        private IEnumerable<int> Stuff
        {
            get
            {
                //This could do anything, fetch a dynamic list from anywhere....
                return new List<int> { 1, 2, 3 };
            }
        }

        [TestMethod]
        [DataSource("MsTestHacksExample.UnitTest1.Stuff")]
        public void TestMethod1()
        {
            var number = this.TestContext.GetRuntimeDataSourceObject<int>();

            Assert.IsNotNull(number);
        }
    }
}

1. Pressed Ctrl + R, Ctrl + T and the Unit Test ran, but this was the result:

Test Name:  TestMethod1
Test FullName:  MsTestHacksExample.UnitTest1.TestMethod1
Test Source:    c:\Users\jbarneck\Documents\Visual Studio 2013\Projects\tmp\MsTestHacksExample\MsTestHacksExample\UnitTest1.cs : line 23
Test Outcome:   Failed
Test Duration:  0:00:00

Result Message: The unit test adapter failed to connect to the data source or to read the data. For more information on troubleshooting this error, see "Troubleshooting Data-Driven Unit Tests" (http://go.microsoft.com/fwlink/?LinkId=62412) in the MSDN Library. Error details: Object reference not set to an instance of an object.

I am going to keep troubleshooting. Hopefully there is a simple step I am missing or something I've installed is interfering.

Trying to install on test project that target .net4.0 I get this error:

Install-Package : Could not install package 'MSTestHacks 2.2.22.0'. You are trying to install this package into a project that targets '.NETFramework,Version=v4.0', but the package does not contain any assembly references or
content files that are compatible with that framework. For more information, contact the package author.

I suggest to provide examples of DataSources, than can be used out of the box, e.g. the simple one
[DataSource("TestDataSourceExamples.TrueOrFalse")]

public IEnumerable TrueOrFalse
{
get
{
return new List< bool> { true, false};
}
}

Hi,
I Have installed MSTestHawks 2.3.2.0 in Visual studio 10, with .net 4.0 and created a new test project.
I used exactly the same code as you have shown in the Example.
I also have a blank or default App.config in this project.

But the test does not run and it gives the following error :
Unit Test Adapter threw exception:
Data source 'Namespace.UnitTest1.Stuff' cannot be found in the test configuration settings..

Can you please guide on the solution to this problem.

I can't see the parameters or a custom message for the parameters that represent each DataRow.
It could be useful if it would be like NUnit as it call the ToString() method for the parameters that are passed to the test.

I was using ExceptionAssert.Throws and even done some improvements(see pull request #21) . But recently I found that FluentAssertions(that we are using anyway) have very similar ShouldThrow
https://github.com/dennisdoomen/fluentassertions/wiki#exceptions with many variances,
E.g.
Action act = () => subject.Foo(null));
act.ShouldThrow().Where(e => e.Message.StartsWith("did"));

Could you advice is your ExceptionAssert has any advantages compare to FluentAssertions ShouldThrow ?

The MSTestHacks library is not strongly-named. This may not be relevant in most cases (as MSTestHacks is used almost solely in testing code, not in "production" code), but in some is a pain.
The problem is only when you need your testing assembly to be strongly named. In that case you can't use MSTestHacks as it's not strongly named.
Why would I need my testing assembly to be strongly named? In a rare case, when I need to give it access to internal classes of a strongly named assembly (e.g. one under tests).
Why wouldn't I add the strong name signature to the library on my own? For two reasons:

1. I would either have to not use the NuGet package of MSTestHacks, so my signed version is not overwritten when the package is refreshed from repo., or have some kind of additional automated step which ensures that the MSTestHacks library is strongly named.
2. The strong name signing operation should be done during compilation (or rather linking), so to perform it on a compiled library I get from NuGet, I'd have to decompile it first, and then recompile with a signature. This may actually alter the library, depending on the (original) compilation options.

Are there any drawbacks of having the MSTestHacks library strongly named? Yes. The main one is that the private key used to sign the assembly should be protected from unauthorized access, otherwise anyone could produce a library of their own, which would be signed just as the original one, circumventing all the security measures that the strong name brings. The secondary one is that the MSTestHacks will not be able to rely on other libraries, that are not strongly named. It's not the case now, but could turn out to be a problem in the future.
Will I have to also sign my assemblies with a strong name, when using a strongly named version of the MSTestHacks library? No. A strongly named assembly can be freely used by both signed and not signed assemblies.

When the combination of the test folder path and the type name exceeds the Windows 255 character limit, AttachRuntimeDataSources fails. This is because the source data gets stored in an xml file whose name is formed from type holding the data.

I've found that a partial solution is to use a hash of the data type name instead of the full type name as the name of the xml file. This doesn't solve problems with length of the test folder path but if that's a problem you are seeing with your test project this solution at least won't make the problem worse.

If you are seeing this problem as we were due to very long fully qualified type names, this should help you.

The change I made was to replace line 97 in AttachRuntimeDataSources.cs with these lines:

var saltedDataSourceName = "saltedDataSourceName" + dataSourceName + "saltedDataSourceName"; var dataSourceFilePath = Path.Combine(DATASOURCES_PATH, saltedDataSourceName.GetHashCode().ToString("X") + ".xml");

(Salting the data source name may not be necessary here but I was concerned about data source type names that are really short possibly leading to hash collisions. I'm not really an expert in the area of hashing so this may not be necessary.)

I would make a pull request for this change but I don't know how to do this with Git so I'm hoping one of the project contributors will incorporate this change for me.

Do you consider to enable github wiki?
I was trying to invoke method like the following
ExceptionAssert.Throws<Exception>(MyMethod(myParameters), validatorForException: x => x.Message == "This is silly");

but the correct way to call is

ExceptionAssert.Throws<ApplicationException>(() =>
   MyMethod(myParameters) ,
    validatorForException: x => x.Message == expectedMessage);

It will be useful to highlight in your documentation.