Tag Archive: Unit Testing

Tweet

< Part One: Medium Trust primer

< Part Two: Devising a solution

Welcome to part three! We’re going to get some work done on unit testing in Medium Trust. If you want a primer, check out the previous posts. If you came here from there, well hello there recursive sentence!

First things first: the code for the below is all in MIT-licensed Umbraco 5! As you may / may not know, Umbraco 5 will be shipping both a set of Framework assemblies in addition to the core CMS when we release later this year. You can use it all for free, isn’t that grand?

For today, here’s a big unmissable link to download a preview zip of Umbraco.Framework.Testing. It’s MIT licensed and you can use this in your tests real easy. I’ve put it on my DropBox for now – I want to get this post out quick and go to the pub! into your hands! In an upcoming release we’ll make Framework & CMS downloads available as individual zips on CodePlex.

A quick note: I only came up with this last week, so (ironically) Umbraco 5 does not yet support Medium Trust as of today’s Alpha 3 – but the point is, we’ll now be able to use this technique to test our code in partial trust and make it happen reliably, and soon.

If you’d like to download all of the Umbraco codebase and see what other goodies are in our Framework and CMS, visit the repository at http://umbraco.codeplex.com/SourceControl/list/changesets.

To use this in your NUnit tests

I’ll walk through the code in another post, but if you want to jump straight into using this, download the zip, add a reference to Umbraco.Framework and Umbraco.Framework.Testing and you’re set.

Note: Because your tests will call NUnit’s Assert methods, NUnit itself needs to be able to run in partial trust. The zip contains a rebuild of NUnit.Framework.dll – the only change from 2.5.10.11092 is adding the AllowPartiallTrustedCallersAttribute to their assembly. Don’t believe me? Check out my fork at https://bitbucket.org/boxbinary/nunit-2.5x

Your test runner does not need to be rebuilt, so this is still compatible with NUnit’s runner, ReSharper’s runner, TeamCity etc.; only your own code referencing the NUnit Framework needs to reference the assembly.

Once you’ve added the references, let’s make a new fixture called MyCoolFixture. Set the fixture so it inherits from AbstractPartialTrustFixture, like so:

  1. using NUnit.Framework;
  2. using Umbraco.Framework.Testing.PartialTrust;
  3. namespace PartialTrustTest
  4. {
  5. [TestFixture]
  6. public class MyCoolFixture : AbstractPartialTrustFixture<MyCoolFixture>
  7. {
  8. public override void TestSetup()
  9. {
  10. return;
  11. }
  12. public override void TestTearDown()
  13. {
  14. return;
  15. }
  16. }
  17. }



Those TestSetup / TestTearDowns are there for you to put relevant code or just return if nothing’s needed.

Next let’s add a test or two:

  1. using System;
  2. using System.IO;
  3. using System.Runtime.Serialization.Formatters.Binary;
  4. using NUnit.Framework;
  5. using Umbraco.Framework.Testing.PartialTrust;
  6. namespace PartialTrustTest
  7. {
  8. [TestFixture]
  9. public class MyCoolFixture : AbstractPartialTrustFixture<MyCoolFixture>
  10. {
  11. [Test]
  12. public void ThisShouldWork()
  13. {
  14. // Arrange, Act
  15. Console.WriteLine(“Hiyaaa”);
  16. // Assert
  17. Assert.Pass(“Yeah this is a bit pedantic”);
  18. }
  19. [Test]
  20. public void ThisShouldNotWork()
  21. {
  22. // Arrange
  23. var myObject = DateTimeOffset.Now;
  24. // Act
  25. var myClone = Clone(myObject);
  26. // Assert
  27. Assert.Fail(“Wait, why did we get here?”);
  28. }
  29. ///
  30. /// Makes a copy from the object.
  31. /// Doesn’t copy the reference memory, only data.
  32. ///
  33. /// Type of the return object.
  34. /// Object to be copied.
  35. /// Returns the copied object.
  36. public static T Clone(T item)
  37. {
  38. if (item != null)
  39. {
  40. var formatter = new BinaryFormatter();
  41. var stream = new MemoryStream();
  42. formatter.Serialize(stream, item);
  43. stream.Seek(0, SeekOrigin.Begin);
  44. T result = (T)formatter.Deserialize(stream);
  45. stream.Close();
  46. return result;
  47. }
  48. else
  49. return default(T);
  50. }
  51. public override void TestSetup()
  52. {
  53. return;
  54. }
  55. public override void TestTearDown()
  56. {
  57. return;
  58. }
  59. }
  60. }



Nothing crazy, just a simple thing which should work and a simple thing which should not.

What, pray tell, should not? Well, that there Clone method is a really handy, and extremely common extension method that uses binary serialization to give you an exact copy of an object. FluentNHibernate uses it, in fact. But BinaryFormatter tries to set private members, and as we now know you can’t do that in Medium Trust!

So in theory that second test should fail. Let’s run that to see if all my promises are true:

image

Check .. that .. OUT!

SetUp : Umbraco.Framework.Testing.PartialTrust.PartialTrustTestException : Test ThisShouldNotWork fails in a partial trust environment, due to: Request for the permission of type ‘System.Security.Permissions.SecurityPermission, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089′ failed.
—-> System.Security.SecurityException : Request for the permission of type ‘System.Security.Permissions.SecurityPermission, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089′ failed.

So let’s fix the test so TestShouldFail() appropriately asserts that … it should fail:

  1. using System;
  2. using System.IO;
  3. using System.Runtime.Serialization.Formatters.Binary;
  4. using System.Security;
  5. using NUnit.Framework;
  6. using Umbraco.Framework.Testing.PartialTrust;
  7. namespace PartialTrustTest
  8. {
  9. [TestFixture]
  10. public class MyCoolFixture : AbstractPartialTrustFixture<MyCoolFixture>
  11. {
  12. [Test]
  13. public void ThisShouldWork()
  14. {
  15. // Arrange, Act
  16. Console.WriteLine(“Hiyaaa”);
  17. // Assert
  18. Assert.Pass(“Yeah this is a bit pedantic”);
  19. }
  20. [Test]
  21. public void ThisShouldNotWork()
  22. {
  23. // Arrange
  24. var myObject = DateTimeOffset.Now;
  25. // Act & Assert
  26. Assert.Throws<SecurityException>(() => Clone(myObject));
  27. }
  28. ///
  29. /// Makes a copy from the object.
  30. /// Doesn’t copy the reference memory, only data.
  31. ///
  32. /// Type of the return object.
  33. /// Object to be copied.
  34. /// Returns the copied object.
  35. public static T Clone(T item)
  36. {
  37. if (item != null)
  38. {
  39. var formatter = new BinaryFormatter();
  40. var stream = new MemoryStream();
  41. formatter.Serialize(stream, item);
  42. stream.Seek(0, SeekOrigin.Begin);
  43. T result = (T)formatter.Deserialize(stream);
  44. stream.Close();
  45. return result;
  46. }
  47. else
  48. return default(T);
  49. }
  50. public override void TestSetup()
  51. {
  52. return;
  53. }
  54. public override void TestTearDown()
  55. {
  56. return;
  57. }
  58. }
  59. }



And run it:

image

Woo!

But what if I want some tests to literally only test something in full-trust?

Well, you can of course have full-trust tests in a separate fixture. OR, you could decorate the test with the [TestOnlyInFullTrust] attribute I’ve added to our library. I’ll change the test again to demonstrate:

  1. [Test]
  2. public void ThisShouldNotWorkInFullTrust()
  3. {
  4. // Arrange
  5. var myObject = DateTimeOffset.Now;
  6. // Act & Assert
  7. Assert.Throws<SecurityException>(() => Clone(myObject));
  8. }
  9. [Test]
  10. [TestOnlyInFullTrust]
  11. public void ThisShouldWorkOnlyInFullTrust()
  12. {
  13. // Arrange
  14. var myObject = DateTimeOffset.Now;
  15. // Act & impicitly Assert
  16. Clone(myObject);
  17. }



And run:

image

Sweet!

So, there you have it. Easy partial-trust testing by inheriting from one base class.

In the next post – which may be next week, at this rate – I’ll walk through the code and explain how it works, for example if you’d like not to inherit from another class and just want to spin up an individual test yourself in a partial trust context individually.

We can also look into making this work with other testing frameworks, if you’re really nice.

Hope this helps you; let me know if so!

Tweet

< Part One: Medium Trust primer

> Part Three: Download the code

Welcome to Part Two, wherein we will tackle the beast that is Medium Trust in our NUnit tests. If you just want a downloadable zip to get going, you can always jump to part three.

Based on what we learnt in the previous post around how the permissions are stored in configuration, a simple option is open to us. The basic premise is that in our unit test, we’re going to spin up an AppDomain and pass in the same permissions from configuration that the ASP.Net hosting engine uses. We’re then going to run the test method in that partially-trusted AppDomain, rather than in the full-trust AppDomain of the test runner.

Simples! The end. Bai!

Well, not quite. First, some gotchas.

Spinning up another AppDomain is called “sandboxing” for a reason: they are totally separate, and any communication between AppDomains has to be done via .NET Remoting using serializable objects, just like using your own Remoting channel, WCF or a good ol’ SOAP web service on another machine. Can you spot the first gotcha yet?

Gotcha 1: You can’t just use AppDomain.SetData to pass objects by value from your test to a Medium-Trust domain because it doesn’t have permissions to deserialize them fully.

Makes sense when I put it like that, but it took me an hour of wasted time before I facepalmed and almost quit coding forever.

There aren’t ways around this. Passing an object to another AppDomain, whether it’s via the SetData and GetData methods, requires the object to either be serializable (if you want to pass it by value) or inherit from MarshalByRefObject so that you can deal with the object as a regular Remoting proxy to the other AppDomain.

Part of the point of spinning up the other AppDomain is precisely to prevent certain code from running, you know, like serialization that can get/set private members.

Yeah, I wasn’t proud. A whole hour.

So the next idea was to use the power of MarshalByRefObject to instantiate something in the partially-trusted AppDomain, and control it from the original fully-trusted AppDomain of the test runner. This is pretty much how we’ll solve the problem, but wait:

Gotcha 2: It’s tempting, but we can’t just make our entire test fixture inherit from MarshalByRefObject.

Let’s agree that it would be a complete pain in the arse to have to replicate the nice, established features of a test runner. Agreed? OK, with that sorted: we can’t have our test fixture inherit from MarshalByRefObject to solve this problem.

We need the partially-trusted AppDomain to “own” the instance of the test, and run it there. If we’re going to try to use an existing test runner (NUnit’s, ReSharper’s, TeamCity’s, etc.) then our first point of entry into a test is when the fixture is already running.

The solution workflow

The concept I’ve arrived at is, as always once you’ve spent ages banging your head against a wall, quite simple in hindsight.

Here’s how the workflow will happen – again, this is just within a standard test runner:

  • During fixture setup:
    • Our fixture will get set up by running code marked with the [FixtureSetUp] attribute
    • We’ll then create the partially-trusted AppDomain
    • We’ll tell that AppDomain to create a new instance of a simple marshalling class that can run a method for us when given a MethodInfo class. By calling AppDomain.CreateInstanceAndUnwrap(..), we’ll get back a Remoting proxy to that object so that we can control it from our original test runner.
    • These will both live as fields on the test fixture that we can access from each test.
  • Before each test:
    • Out test setup code, still running in the full-trust AppDomain, will get run because it’s marked with the NUnit [SetUp] attribute
    • At this point, we’ll get the name of the current test from NUnit’s TestContext, and tell our simple marshalling class to do two things over in that partial trust AppDomain:
      • Run a TestSetup method, to make sure if it has “real” setup work to do, that’s still fine
      • Run the actual method. We get that by using Reflection in our full-trust AppDomain to grab the MethodInfo of the current test name from NUnit’s context.
    • Our marshaller will then run the exact test method in the partial trust domain.
    • If the test fails, we can figure out why, and if it’s to do with permissions, BAM our time has been officially saved

“But wait!” I hear someone wail at the back, “By running the test in the SetUp part, you’re now going to run the test again in Full Trust!”

Well, I thought of that.

NUnit has a funny thing called a SuccessException. It’s how it guarantees that if you call Assert.Pass() in your test, the rest of the test will likely not execute – pretty much what you’d expect.

So, in our “management” set-up code, we’re going to intentionally throw a SuccessException. I took a look at the NUnit codebase to clarify, and sure enough that prevents the test from executing a second time.

The code, where’s the code?

Well, I thought I would just give you theory. Kidding! I’m hilarious. The downloadable code is on the next post – I really need to make my blog more developer-friendly, so for now it’s a fresh new page for us.