Proposal: UIDataTestMethod / Support for DataRows in UITestMethods

[marcelwgn]

The DataTestMethod attribute is really helpful and makes writing tests a lot easier. However when developing tests that need to run on the UI thread, we have to use the UITestMethod attribute. This however leaves us with having to implement data driven tests ourselfes. Having a UIDataTestMethod attribute would be great to have to fill the gap between UI tests and non UI tests.

AB#1408166

[nohwnd]

Maybe you could PR it?

[marcelwgn]

Where would this actually be contributed to? Does UITestMethod even live in this repository?

[Rosuavio]

@chingucoding it does live in this repo.

testfx/src/TestFramework/Extension.UWP/UITestMethodAttribute.cs

Lines 12 to 43 in ec18af6

	public class UITestMethodAttribute : TestMethodAttribute
	{
	/// <summary>
	/// Executes the test method on the UI Thread.
	/// </summary>
	/// <param name="testMethod">
	/// The test method.
	/// </param>
	/// <returns>
	/// An array of <see cref="TestResult"/> instances.
	/// </returns>
	/// Throws <exception cref="NotSupportedException"> when run on an async test method.
	/// </exception>
	public override TestResult[] Execute(ITestMethod testMethod)
	{
	var attrib = testMethod.GetAttributes<AsyncStateMachineAttribute>(false);
	if (attrib.Length > 0)
	{
	throw new NotSupportedException(FrameworkMessages.AsyncUITestMethodNotSupported);
	}
	TestResult result = null;
	Windows.ApplicationModel.Core.CoreApplication.MainView.CoreWindow.Dispatcher.RunAsync(
	Windows.UI.Core.CoreDispatcherPriority.Normal,
	() =>
	{
	result = testMethod.Invoke(new object[] { });
	}).AsTask().GetAwaiter().GetResult();
	return new TestResult[] { result };
	}
	}

Also on a similar note, I am working on the Windows Community Toolkit and we make heavy use of UITestMethod either creating this UIDataTestMethod would be very useful or maybe a better solution would be to make using UITestMethod and DataRow together work, rather than make a new attribute?

Currently using those attributes together causes this rather obscure error.

[marcelwgn]

Oh that's interesting, thanks for pointing that out @RosarioPulella!

I'm also not sure what the best move forward would be. I think having a UIDataTestMethod would be best for API simplicity since it extends on an existing pattern. I also got the issue you linked often and was confused what was happening. This error message should definitely be updated.

[marcelwgn]

@nohwnd What do you think is the best way forward here?

[Rosuavio]

I think having a UIDataTestMethod would be best for API simplicity since it extends on an existing pattern.

I am not in tune with a lot of the patterns of this API, but I see a lot of test methods decorated with DataRow and a simple TestMethod and it works fine. It also seems like there is an existing pattern to compose DataRow with other attributes than DataTestMethod.

Either way it goes, I would like something to open up for using DataRow in UI tests.

[marcelwgn]

Looks like you have a better view of how the properties work together. If DataRow works with normal TestMethod, why exactly is there a DataTestMethod attribute though? I though you need to have those and otherwise it wouldn't work.

Totally agree that no matter what we settle on, the situation needs to change. Having no DataRow support in UI tests is quite annoying.

[michael-hawker]

@chingucoding they DataTestMethod used to be a separate thing, but they merged them and just left the old one there for compatibility for folks. It should probably just be marked Obsolete so folks could move off of it and they could remove it in the future.

We hit the same problem with async as well, but using a TestMethod to just dispatch to the UI thread first things seems to be working. Same for doing a similar thing for DataRow as well. It's just a bit more verbose.

[Haplois]

DataTestMethodAttribute has no functionality, we keep it there for compability. It's same as TestMethodAttribute. Instead of adding a new method, we should fix the bug in UITestMethod.