Giter VIP home page Giter VIP logo

mstesthacks's Introduction

Build status

Overview

Just a bunch of hacks to get around the deficiencies of MSTest.

Hopefully for those of us that have to work inside the constrainsts of MSTest, this library should ease our pain. (Just a little)

Check out the tests project for a few samples

Features

Runtime DataSource

A runtime data driven test as opposed to compile time. Just point your datasource to a property, field or method name that returns an IEnumerable and at runtime it will loop through the collection and just act like normal. (Think NUnit's TestCaseSource)

Exception Assert

Provides a much easier way to throw exceptions and verify the type, and information within the exception itself.

Getting Started

Install NuGet. Then, install MSTestHacks from the package manager console:

PM> Install-Package MSTestHacks

###Runtime DataSource Note: App.config or Web.config must be in the project to provide the dynamic linking.

1) You MUST inherit your test class from TestBase

[TestClass]
public class UnitTest1 : TestBase
{ }

2) Create a Property, Field or Method, that returns an IEnumerable

[TestClass]
public class UnitTest1 : TestBase
{
    private IEnumerable<int> Stuff
    {
        get
        {
            //This could do anything, fetch a dynamic list from anywhere....
            return new List<int> { 1, 2, 3 };
        }
    }
}

3) Add the DataSource attribute to your test method, pointing back to the IEnumerable name created earlier. This needs to be fully qualified.

[TestMethod]
[DataSource("Namespace.UnitTest1.Stuff")]
public void TestMethod1()
{
    var number = this.TestContext.GetRuntimeDataSourceObject<int>();
    
    Assert.IsNotNull(number);
}

Explanation When each TestBase inherited class is initialised, a process gets run to create an XML file for each datasource attribute. It then dynamically links each datasource up to the XML file. So when each test executes it loops over the datasource like it would normally in MSTest. The "GetRuntimeDataSourceObject" extension method is just a convenient helper to get the object back out of the datasource using JSON deserialisation. Simple really :)

###Exception Assert

//Verify the type and message.
[TestMethod]
public void MethodThrowsSpecificExceptionWithExpectedExceptionMessage()
{
	var expectedMessage = "crap.";
	var action = new Action(() => { throw new StackOverflowException(expectedMessage); });

	ExceptionAssert.Throws<StackOverflowException>(action, expectedMessage);
}

//Verify the type and anything about the exception with the 'validator'
[TestMethod]
public void MethodThrowsExceptionAndTheValidatorWorks()
{
	// Arrange
	Action action = () => { throw new Exception("This is silly"); };

	// Act & Assert
	ExceptionAssert.Throws<Exception>(action, validatorForException: x => x.Message == "This is silly");
}

Changelog

2.3.8

  • Fixed datasource file paths too long issues

2.3.1

  • Added support for .net 40 framework

2.2.19

  • Added Exception Assert Feature
  • Removed CodedUI Support (See AFrame)

2.1.0

  • Made references to MS TestTools not point to "Specific" versions e.g. VS2012 references

2.0.0

  • More logging for runtime datasource around timing etc
  • Introduced CodedUI Jquery controls for finding controls via jquery selectors

1.1.2

  • Added a fix so the datasources could point to an class that didnt inherit TestBase

1.1.1

  • Removed the need for [AttachRuntimeDataSources(typeof(ClassName))] - dynamically finds the datasources.
  • Added some rudimentary debugging
  • Moved all outputed data down 1 directory to be self contained in a directory called MSTestHacks
  • GetRuntimeDataSourceObject extenstion method is now in same namespace as testbase, so no need now for the using statement: using MSTestHacks.RuntimeDataSources;
  • Made code a little more efficient

1.1.0 - (BREAKING CHANGE)

  • Creating a datasource file per datasource, simplifies life
  • All datasources now have to be fully qualified, pointing to the IEnumerable. This creates a unique datasource that was required in some instances.

1.0.2

  • Inject the ConnectionString automatically
  • Add a couple more tests

1.0.1

  • Fixes the issue with stale iterations. Each run was not getting deleted from the xml file and was building up.

1.0.0

  • Replaced the database backend with an xml one
  • Production Ready

0.0.2

  • Simplified NuGet and added project links etc

0.0.1

  • Initial release

Contributors

Sam Thwaites
Corey Warner

Licence

See LICENCE

mstesthacks's People

Contributors

cjwarner avatar mach22 avatar michael-freidgeim-webjet avatar rcatley avatar thwaites avatar thwaitesy avatar timabell avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mstesthacks's Issues

Data Source Example TrueOrFalse

I suggest to provide examples of DataSources, than can be used out of the box, e.g. the simple one
[DataSource("TestDataSourceExamples.TrueOrFalse")]

public IEnumerable TrueOrFalse
{
get
{
return new List< bool> { true, false};
}
}

Is ExceptionAssert.Throws better than FluentAssertions.ShouldThrow ?

I was using ExceptionAssert.Throws and even done some improvements(see pull request #21) . But recently I found that FluentAssertions(that we are using anyway) have very similar ShouldThrow
https://github.com/dennisdoomen/fluentassertions/wiki#exceptions with many variances,
E.g.
Action act = () => subject.Foo(null));
act.ShouldThrow().Where(e => e.Message.StartsWith("did"));

Could you advice is your ExceptionAssert has any advantages compare to FluentAssertions ShouldThrow ?

Long type names in deep directory structures cause AttachRuntimeDataSources to fail

When the combination of the test folder path and the type name exceeds the Windows 255 character limit, AttachRuntimeDataSources fails. This is because the source data gets stored in an xml file whose name is formed from type holding the data.

I've found that a partial solution is to use a hash of the data type name instead of the full type name as the name of the xml file. This doesn't solve problems with length of the test folder path but if that's a problem you are seeing with your test project this solution at least won't make the problem worse.

If you are seeing this problem as we were due to very long fully qualified type names, this should help you.

The change I made was to replace line 97 in AttachRuntimeDataSources.cs with these lines:

var saltedDataSourceName = "saltedDataSourceName" + dataSourceName + "saltedDataSourceName"; var dataSourceFilePath = Path.Combine(DATASOURCES_PATH, saltedDataSourceName.GetHashCode().ToString("X") + ".xml");

(Salting the data source name may not be necessary here but I was concerned about data source type names that are really short possibly leading to hash collisions. I'm not really an expert in the area of hashing so this may not be necessary.)

I would make a pull request for this change but I don't know how to do this with Git so I'm hoping one of the project contributors will incorporate this change for me.

System.UnauthorizedAccessException: Attempted to perform an unauthorized operation.

Most team members have no problems running the unit-tests. Yet when I run them on my machine, I get this exception and stacktrace.

  • The Domain.Tests.dll.config file is present

  • The MSTestHacks\debug.txt is present and written to
    [23-01-18 02:48:37] ------------------
    [23-01-18 02:48:37] Starting Execution
    [23-01-18 02:48:38] Successfully created datasource: Domain.Tests.Core.Request.Aggregate.StatusTests.FinalizationTests.RequestTypes, Iteration Count: 2, Elapsed Time : 00:00:00.3647961

  • The MSTestHacks\RuntimeDataSources folder and files are there and written to

Result StackTrace:

at System.Security.AccessControl.Win32.SetSecurityInfo(ResourceType type, String name, SafeHandle handle, SecurityInfos securityInformation, SecurityIdentifier owner, SecurityIdentifier group, GenericAcl sacl, GenericAcl dacl)
   at System.Security.AccessControl.NativeObjectSecurity.Persist(String name, SafeHandle handle, AccessControlSections includeSections, Object exceptionContext)
   at System.Security.AccessControl.NativeObjectSecurity.Persist(String name, AccessControlSections includeSections, Object exceptionContext)
   at System.Security.AccessControl.NativeObjectSecurity.Persist(String name, AccessControlSections includeSections)
   at System.Security.AccessControl.FileSystemSecurity.Persist(String fullPath)
   at System.IO.File.SetAccessControl(String path, FileSecurity fileSecurity)
   at System.Configuration.Internal.WriteFileContext.DuplicateTemplateAttributes(String source, String destination)
   at System.Configuration.Internal.WriteFileContext.DuplicateFileAttributes(String source, String destination)
   at System.Configuration.Internal.WriteFileContext.Complete(String filename, Boolean success)
   at System.Configuration.Internal.InternalConfigHost.StaticWriteCompleted(String streamName, Boolean success, Object writeContext, Boolean assertPermissions)
   at System.Configuration.Internal.InternalConfigHost.System.Configuration.Internal.IInternalConfigHost.WriteCompleted(String streamName, Boolean success, Object writeContext, Boolean assertPermissions)
   at System.Configuration.Internal.InternalConfigHost.System.Configuration.Internal.IInternalConfigHost.WriteCompleted(String streamName, Boolean success, Object writeContext)
   at System.Configuration.Internal.DelegatingConfigHost.WriteCompleted(String streamName, Boolean success, Object writeContext)
   at System.Configuration.UpdateConfigHost.WriteCompleted(String streamName, Boolean success, Object writeContext)
   at System.Configuration.MgmtConfigurationRecord.SaveAs(String filename, ConfigurationSaveMode saveMode, Boolean forceUpdateAll)
   at System.Configuration.Configuration.SaveAsImpl(String filename, ConfigurationSaveMode saveMode, Boolean forceSaveAll)
   at System.Configuration.Configuration.Save(ConfigurationSaveMode saveMode)
   at MSTestHacks.RuntimeDataSource.AttachRuntimeDataSources..cctor() in c:\projects\mstesthacks\MSTestHacks\RuntimeDataSource\AttachRuntimeDataSources.cs:line 163

Result Message: Unable to create instance of class Domain.StatusTests.CancelTests. Error: System.UnauthorizedAccessException: Attempted to perform an unauthorized operation..

MSTest data driven Test do not create a new test for Rows from DataSource

Using NUnit with parametrized test if the parameters are different (I think based on ToString() of the paraters passed to the TestMethod) in the TestExplorer I can see N test as the number of the rows.
So that I can run each test as a single test. (while using MStest I need to run every time all the tests)
Is there any way for doing this?

MSTestHawks 2.3.2.0 : Unit Test Adapter threw exception: Data source 'Namespace.UnitTest1.Stuff' cannot be found in the test configuration settings..

Hi,
I Have installed MSTestHawks 2.3.2.0 in Visual studio 10, with .net 4.0 and created a new test project.
I used exactly the same code as you have shown in the Example.
I also have a blank or default App.config in this project.

But the test does not run and it gives the following error :
Unit Test Adapter threw exception:
Data source 'Namespace.UnitTest1.Stuff' cannot be found in the test configuration settings..

Can you please guide on the solution to this problem.

How to specify method to test in ExceptionAssert.Throws

Do you consider to enable github wiki?
I was trying to invoke method like the following
ExceptionAssert.Throws<Exception>(MyMethod(myParameters), validatorForException: x => x.Message == "This is silly");

but the correct way to call is

ExceptionAssert.Throws<ApplicationException>(() =>
   MyMethod(myParameters) ,
    validatorForException: x => x.Message == expectedMessage);

It will be useful to highlight in your documentation.

Main example doesn't work

You main example doesn't work for me. I have VS 2013 Update 4 with Resharper and various other plugins.

  1. I created a new C# Unit Test project called MsTestHacksExample.
  2. Installed MsTestHacks from NuGet.
  3. Made sure app.config exists. (It was created by your NuGet package. Well, actually Newonsoft.Json created it).
  4. Added this test.
using System;
using System.Collections.Generic;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using MSTestHacks;

namespace MsTestHacksExample
{
    [TestClass]
    public class UnitTest1 : TestBase
    {
        private IEnumerable<int> Stuff
        {
            get
            {
                //This could do anything, fetch a dynamic list from anywhere....
                return new List<int> { 1, 2, 3 };
            }
        }

        [TestMethod]
        [DataSource("MsTestHacksExample.UnitTest1.Stuff")]
        public void TestMethod1()
        {
            var number = this.TestContext.GetRuntimeDataSourceObject<int>();

            Assert.IsNotNull(number);
        }
    }
}
  1. Pressed Ctrl + R, Ctrl + T and the Unit Test ran, but this was the result:
Test Name:  TestMethod1
Test FullName:  MsTestHacksExample.UnitTest1.TestMethod1
Test Source:    c:\Users\jbarneck\Documents\Visual Studio 2013\Projects\tmp\MsTestHacksExample\MsTestHacksExample\UnitTest1.cs : line 23
Test Outcome:   Failed
Test Duration:  0:00:00

Result Message: The unit test adapter failed to connect to the data source or to read the data. For more information on troubleshooting this error, see "Troubleshooting Data-Driven Unit Tests" (http://go.microsoft.com/fwlink/?LinkId=62412) in the MSDN Library. Error details: Object reference not set to an instance of an object.

I am going to keep troubleshooting. Hopefully there is a simple step I am missing or something I've installed is interfering.

Can't install nuget package on .net4.0

Trying to install on test project that target .net4.0 I get this error:

Install-Package : Could not install package 'MSTestHacks 2.2.22.0'. You are trying to install this package into a project that targets '.NETFramework,Version=v4.0', but the package does not contain any assembly references or
content files that are compatible with that framework. For more information, contact the package author.

Error CS0012: The type 'TestContext' is defined in an assembly that is not referenced. You must add a reference to assembly 'Microsoft.VisualStudio.QualityTools.UnitTestFramework, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.

Hi,
I Have installed MSTestHawks 2.3.2.0 in VS2017, with .net core 2.0 as the target framework and created a new test project.
I used your code but got the error as:

Error CS0012 The type 'TestContext' is defined in an assembly that is not referenced. You must add a reference to assembly 'Microsoft.VisualStudio.QualityTools.UnitTestFramework, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.

I noticed that the message "Package 'MSTestHacks 2.3.20' was restored using '.NetFramework, Version=v4.6.1' instead of the project target framework '.NetCoreApp, Version=v2.0' This package may not be fully compatible with your project" displayed on my Solution Explore.

I have installed following packages from NuGet in my project:

"Microsoft.NET.Test.Sdk" Version="15.5.0"
"MSTest.TestAdapter" Version="1.2.0"
"MSTest.TestFramework" Version="1.2.0"
"MSTestHacks" Version="2.3.20"

I also tried to install "Microsoft.VisualStudio.QualityTools.UnitTestFramework v11.0.50727.1", which the problem still exists. And "Microsoft.VisualStudio.QualityTools.UnitTestFramework.Update v15.0.26228" (This package is not compatible with netcoreapp2.0)

The code I used is as below:

[TestClass]
public class UnitTest1 : TestBase
{
  [TestMethod]
  [DataSource("Namespace.UnitTest1.Stuff")]
  public void TestMethod1()
  {
      var number = this.TestContext.GetRuntimeDataSourceObject< int >();

      Assert.IsNotNull(number);
  }

  private IEnumerable< int > Stuff
  {
      get
      {
          //This could do anything, fetch a dynamic list from anywhere....
          return new List< int > { 1, 2, 3 };
      }
  }
}

So, is this really a compatibility problem of .Net Core 2.0 or do I miss something to get it be right?
Thanks.

Data source 'Blah.Blah' cannot be found in the test configuration settings

I have an App.config (it already contains some configuration information, but I've tried it empty also). Using VS2015 RC and latest Nuget package. I also tried the examples from your repo, but attempting to open one of the source files for the tests crashed VC2015 RC.

Cannot work out what is happening.

No strong name

The MSTestHacks library is not strongly-named. This may not be relevant in most cases (as MSTestHacks is used almost solely in testing code, not in "production" code), but in some is a pain.
The problem is only when you need your testing assembly to be strongly named. In that case you can't use MSTestHacks as it's not strongly named.
Why would I need my testing assembly to be strongly named? In a rare case, when I need to give it access to internal classes of a strongly named assembly (e.g. one under tests).
Why wouldn't I add the strong name signature to the library on my own? For two reasons:

  1. I would either have to not use the NuGet package of MSTestHacks, so my signed version is not overwritten when the package is refreshed from repo., or have some kind of additional automated step which ensures that the MSTestHacks library is strongly named.
  2. The strong name signing operation should be done during compilation (or rather linking), so to perform it on a compiled library I get from NuGet, I'd have to decompile it first, and then recompile with a signature. This may actually alter the library, depending on the (original) compilation options.

Are there any drawbacks of having the MSTestHacks library strongly named? Yes. The main one is that the private key used to sign the assembly should be protected from unauthorized access, otherwise anyone could produce a library of their own, which would be signed just as the original one, circumventing all the security measures that the strong name brings. The secondary one is that the MSTestHacks will not be able to rely on other libraries, that are not strongly named. It's not the case now, but could turn out to be a problem in the future.
Will I have to also sign my assemblies with a strong name, when using a strongly named version of the MSTestHacks library? No. A strongly named assembly can be freely used by both signed and not signed assemblies.

Question - Issue with Studio 2013

I'm receiving the following error when I try and run under VS2013:
Message: Data source 'Namespace.Unters1.Stuff' cannot be found in the test configuration settings

I created a class directly from the source in your example, see below :

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using MSTestHacks;

namespace Namespace
{
[TestClass]
public class UnitTest1 : TestBase
{
private IEnumerable Stuff
{
get
{
//This could do anything, get a dynamic list from anywhere....
return new List { 1, 2, 3 };
}
}

[TestMethod]
[DataSource("Namespace.UnitTest1.Stuff")]
public void TestMethod1()
{
  var number = this.TestContext.GetRuntimeDataSourceObject<int>();

  Assert.IsNotNull(number);
}

}
}

When I attempt to run I get the error listed above. Any suggestions?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.