NUnit: TestName causes tests to go undiscovered

Question

There doesn't seem to be any related post I can find here, at Nunit, or on MS sites that report any problems with the TestName parameter. Without this parameter, my tests are discovered and are run. But, if I add this parameter to any TestCase, then that specific test is not discovered, and the 'tests run' counter displays the reduced number. Then I remove the TestName parameter, and they are discovered and the test counter shows the expected number of test discovered.

Is this a known bug with VS2022? Or am I entering the values incorrectly?

• VS Enterprise 2022 v17.3.6
• .NET Framework v4.8.04084
• C# Tools 4.3.0-3.22470.13+80a8ce8d5fdb9ceda4101e2acb8e8eb7be4ebcea
• NuGet Package Manager 6.3.0

My tests all load and run as shown here, selecting parseAUR and clicking Run Tests, will then show four test discovered.

    [TestCase(noneAur, ExpectedResult = 0, Description = "Finds nothing")]
    [TestCase(oneAur,  ExpectedResult = 1, Description = "Finds only one aur in report.")]
    [TestCase(oneAurPlus, ExpectedResult = 1, Description = "Find one aur, ignores afr in report")]
    [TestCase(twoAURs, ExpectedResult = 2, Description = "Finds two aur's in report")]
    public int parseAUR(string source)
    {
        return ClaimStatusService.ProcessUndeliverableReport(source);
    }

However, if I add TestName to the attribute list, then the tests are not discovered and are never run.

    [TestCase(noneAur, ExpectedResult = 0, Description = "Finds nothing", TestName = "parseAUR(None)")]
    [TestCase(oneAur,  ExpectedResult = 1, Description = "Finds only one aur in report.", TestName = "parseAUR(One)")]
    [TestCase(oneAurPlus, ExpectedResult = 1, Description = "Find one aur, ignores afr in report", TestName = "parseAUR(OnePlus)")]
    [TestCase(twoAURs, ExpectedResult = 2, Description = "Finds two aur's in report", TestName = "parseAUR(Two)")]

Adding or removing TestName to any TestCase will alter that specific test's visibility in the TestExplorer and alters the discovered test count.

UPDATE: moving the TestName parameter from the end 'fixes' the issue. In fact, just moving one TestCase in this way cased all the TestCases to be discovered.

Answer 1

Is it possible that by adding the test name, you're moving it up a level in the hierarchy and so it's there, but not where you might expect it to be?

I tried to recreate this in VS 2022 with various runners.

I took the most basic reproduction I could:

public class Tests
{
    [TestCase("TestString", ExpectedResult = 1)]
    [TestCase("TestString2", ExpectedResult = 1)]
    [TestCase("TestString3", ExpectedResult = 1)]
    public int StringTest(string inputString)
    {
        return 1;
    }
}

My VS and NCrunch runners both show all 3 test cases -- here's an example side by side:

When I add a test name to one of the tests:

public class Tests
{
    [TestCase("TestString", ExpectedResult = 1, TestName = "CheckThisOut(MyThing)")]
    [TestCase("TestString2", ExpectedResult = 1)]
    [TestCase("TestString3", ExpectedResult = 1)]
    public int StringTest(string inputString)
    {
        return 1;
    }
}

Then the runners do show the test:

But -- note the placement of the test on the left-hand side (the VS Test Runner). The test is no longer under StringTest, it appears to be considered a sibling of that test by VS. It appears next to it, not within it.

In Visual Studio's Test Explorer window, try searching for your custom test name to see if it appears (in the screenshot below, I typed my custom name in and saw it come up):

Answer 2

After much investigation, I have discovered it's a configuration mismatch between the project and the installed nunit version. Turns our build master updated the project to use 4.3.1 but I only had 4.2.1 installed. Why this would cause the issue is unknown. But, updating the nunit version on my machine resolved several issues with test discovery. A different solution would not load any test until the version was resolved.