Visual Studio Test Platform Primer

Visual Studio Test Platform Primer

The aim of this article is to provide a quick reference to Visual Studio Test Platform concepts and topics which are used in the Boost Test Unit Test Adapter project.

Classes

• Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter.ITestDiscoverer

Interface describing a test discoverer. Implementations are required to implement void DiscoverTests(IEnumerable<string> sources, IDiscoveryContext discoveryContext, IMessageLogger logger, ITestCaseDiscoverySink discoverySink). Given a set of fully-qualified paths (sources) of test modules, the discoverer should notify the test platform of any tests through the provided ITestCaseDiscoverySink instance.

Test discoverer classes are to be annotated using Microsoft.VisualStudio.TestPlatform.ObjectModel.FileExtensionAttribute, stating file extensions which are of interest to the discoverer and eventually provided by Visual Studio and Microsoft.VisualStudio.TestPlatform.ObjectModel.DefaultExecutorUriAttribute which should match a registered ITestExecutor URI.

• Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter.ITestExecutor

Interface describing a test executor. Implementations are required to implement void RunTests(IEnumerable<string> sources, IRunContext runContext, IFrameworkHandle frameworkHandle), void RunTests(IEnumerable<TestCase> sources, IRunContext runContext, IFrameworkHandle frameworkHandle) and void Cancel().

Test discoverer classes are to be annotated using Microsoft.VisualStudio.TestPlatform.ObjectModel.ExtensionUriAttribute which states the executor's URI. Such URIs are usually of the form: executor://<TestAdapterName>/v<Version>.

The Cancel method is called by Visual Studio when a user cancels test execution. The IEnumerable<string> overload of RunTests is called when a user selects Run All.... A common implementation of this method is to re-discover tests and call the IEnumerable<TestCase> overload. The IEnumerable<TestCase> overload is called by Visual Studio when a user selects particular tests which are to be executed.

• Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter.IDiscoveryContext

Provides access to the .runsettings configuration.

• Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter.IMessageLogger

Logging mechanism provided by the Visual Studio framework. All log messages are redirected to the logger in use which by default is the Tests section in the Output window.

• Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter.ITestCaseDiscoverySink

Sink for test cases. Discovered test cases are to be notified to the Visual Studio framework via such instances.

• Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter.IRunContext

An IDiscoveryContext extension which contains properties identifying the test execution state such as whether debug test execution was requested, code coverage etc.

• Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter.IFrameworkHandle

Contains helper functionality to execute processes in debug mode, attaching the current Visual Studio instance to the spawned process.

Execution Model

Once Visual Studio identifies a successful build, test discoverers annotated with a matching file extension are requested to discover tests. This method will execute in a separate vstest.discoveryengine process, child of the Visual Studio IDE process.

When a user decides to execute the tests, a separate vstest.executionengine process, child of the Visual Studio IDE process is spawned. This process will then call the appropriate RunTests overload depending whether Run All... tests is triggered or specific test cases are selected for execution.

Additional Notes

• Consider defining breakpoints programmatically within the code since it can get a bit difficult to debug the test adapter since it runs in a different process.
• It is possible to communicate additional details (using primitive data types) between the discovery and execution phases via TestPropertys. Refer to TestObject.GetPropertyValue and TestObject.SetPropertyValue.

References

Reference	Link
Visual Studio Test Platform	https://github.com/Microsoft/vstest
Visual Studio Test Platform Documentation	https://github.com/Microsoft/vstest-docs
Anatomy of Chutzpah	http://matthewmanela.com/blog/anatomy-of-the-chutzpah-test-adapter-for-vs-2012-rc/
Authoring a New Visual Studio Adapter	http://blogs.msdn.com/b/bhuvaneshwari/archive/2012/03/13/authoring-a-new-visual-studio-test-adapter.aspx
Chutzpah Test Adapter	https://github.com/mmanela/chutzpah
NUnit Test Adapter	https://github.com/nunit/nunit3-vs-adapter
Google Test Test Adapter [1]	https://github.com/markusl/GoogleTestRunner
Google Test Test Adapter [2]	https://github.com/csoltenborn/GoogleTestAdapter