When multiple unit tests copy the same file, running all unit tests fail

Question

Description

I am writing unit tests for a method, which copies a file from a source to a destination. Basically it includes this code:

public void MyMethod() 
{
    // ...
    File.Copy(source, destination, true);
    // ...
}

In my unit test project, I have a test file: (test.png), which is located in the Resources folder of my unit test project. And I've set the Copy to Output property to Always.

I have 3 unit tests which are testing this method.

When they hit the line of the code which copies the file: source = "Resources\\test.png".

Issue

When I run the unit test individually, they all pass and everything is fine. However, when I run All Tests in Visual Studio, I get this run time error and unit tests fail:

System.IO.DirectoryNotFoundException

Could not find a part of the path 'Resources\test.png'.

My Thoughts...(UPDATED)

• Probably because Visual Studio runs each unit tests simultaneously in a separate thread and they all accessing the same file at the same time?

• I think for every unit test, Visual Studio is cleaning bin/Debug and bin/Release folders. Then it copies all the required project files in that folder. This causes sometimes the file actually does not exist?

Question

How can I fix this problem?

Is there any settings of configurations to resolve this?

How can I run all unit tests in Visual Studio (and Team City) when multiple unit tests are accessing the same file?

Answer 1

You could try to rule out the multi-threading issue by following the instructions from MSDN: Executing Unit Tests in parallel on a multi-CPU/core machine, setting parallelTestCount to 1. If the tests now pass, you've narrowed down the problem.

However, if your tests are still failing when you run them in a group - and I think this is the more likely scenario -, then my advice would be to check for any state those tests are sharing. The pattern you describe (i.e. passes in isolation; fails when not in isolation) is a symptom typically exhibited by tests that are (incorrectly) sharing state, and that state is being modified by those tests, causing one or more tests to fail.

Answer 2

Accessing the same file should not be a problem. Make sure you don't have a cleanUp Fixture(TestSuite level) to delete the file. Because from exception it looks like the file is being deleted after running a test.

Also concurrent read operation is fine and perfectly legal. If your unit tests are overwriting the file then it's a problem.

Answer 3

What happened was, since I was using relative path to the testing files, for some reason when running the unit tests in batch, test runner working directory is different than when running individual tests, hence it couldn't find the directory.

So I used this function to build the absolute path to the testing files:

    private string GetFilePath([CallerFilePath] string path = "")
    {
        return path;
    }

Then:

    string projectDir = Path.GetDirectoryName(GetFilePath());
    string testFile = Path.Combine(projectDir, @"Resources\test.png";

Answer 4

I'd speculate that your problem is that one of the tested methods changes directory, given the explicit "Directory Not Found" exception. It's improbable that file locking or any concurrency problems would cause the behaviour described.

Answer 5

If unit testing you shouldn't really be testing whether File.Copy (or any of the File class methods) worked since you didn't write that code. Instead you should test whether your code interacts correctly with the File type (i.e. did it pass the correct source file name, desination file name and overwrite value when you called "Copy"). First create an interface for the File class and a wrapper for it that implements the interface;

public interface IFileWrapper
{
    void Copy(string sourceFileName,string destFileName,bool overwrite);
    //Other required file system methods and properties here...
}

public class FileWrapper : IFileWrapper
{
    public void Copy(string sourceFileName, string destFileName, bool overwrite)
    {
        File.Copy(sourceFileName, destFileName, overwrite);
    }
}

You should then make the class you are testing include an IFileWrapper parameter (dependency injection). In your unit tests you can then use a mocking framework such as Moq or you could write your own mock;

public class MockFileWrapper : IFileWrapper
{
    public string SoureFileName { get; set; }
    public string DestFileName { get; set; }
    public bool Overwrite { get; set; }
    public void Copy(string sourceFileName, string destFileName, bool overwrite)
    {
        SoureFileName = sourceFileName;
        DestFileName = destFileName;
        Overwrite = overwrite;
    }
}

In real implementations pass in FileWrapper as the IFileWrapper parameter but in your unit tests pass in MockFileWrapper. By checking the properties of mockFileWrapper in your unit tests you can now determine whether you class calls Copy and how it is called. Since you are no longer sharing a real file between your unit tests you will avoid the chance of the tests sharing state or potentially locking the file.

Answer 6

As you mentioned in your answer, the test framework does not always run tests with the working directory set to your build output folder.

To instruct the test framework to place build artifacts or other files from your build output into the test directory, you need to use DeploymentItemAttribute. For your case, you would do something like:

const string destination = "Destination.txt";
const string source = "MyData.txt";

[DeploymentItem(source)]
[TestMethod]
public void MyMethod() 
{
    // …
    File.Copy(source, destination, true);
    // …
}

[TestCleanup]
public void Cleanup()
{
    // Clean up the destination so that subsequent tests using
    // the same deploy don’t collide.
    File.Delete(destination);
}

Also ensure that your files are marked with a Build Action of Contents and Always Copy. Otherwise, they won’t be in the build output directory and won’t be able to be copied to the test directory.