Integration of automated tests into pipelines

In this lesson, you'll learn how to integrate automated tests into Azure DevOps pipelines using a sample C# application with MSTest. This approach ensures that every code change is automatically verified before it moves to testing, staging, or production—helping you catch bugs early in the development lifecycle.

1. Setting Up the Application

Begin with a basic C# class library in Visual Studio that performs unit conversions. The library includes methods to convert Celsius to Fahrenheit, meters to feet, and kilograms to pounds. Below is the implementation of the conversion methods:

namespace ConverterLib
{
    public class Converter
    {
        public double CelsiusToFahrenheit(double celsius)
        {
            return (celsius * (9.0 / 5.0)) + 32;
        }

        public double MetersToFeet(double meters)
        {
            return meters * 3.28084;
        }

        public double KilogramsToPounds(double kilograms)
        {
            return kilograms * 2.20462;
        }
    }
}

A simple runner application is used to manually test these conversions. The following is the project file for the runner:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>
  <ItemGroup>
    <ProjectReference Include="..\Converter\Converter.csproj" />
  </ItemGroup>
</Project>

This initial manual verification confirms that the conversions work correctly before you proceed with automated testing.

2. Configuring the Azure DevOps Pipeline

First, set up an Azure DevOps pipeline from your repository in Azure Repos Git. In this example, we are using a simple converter application repository. When creating your pipeline, choose the appropriate options for the project type such as a .NET Desktop-like project template.

Below is the YAML configuration for the pipeline:

trigger:
  - master

pool:
  vmImage: 'windows-latest'
  name: "Azure Pipeline"

variables:
  solution: '**/*.sln'
  buildPlatform: 'Any CPU'
  buildConfiguration: 'Release'

steps:
- task: NuGetToolInstaller@1
  inputs:
    solution: '$(solution)'

- task: NuGetCommand@2
  inputs:
    restoreSolution: '$(solution)'

- task: VSBuild@1
  inputs:
    solution: '$(solution)'
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

- task: VSTest@2
  inputs:
    platform: '$(buildPlatform)'

When you save and run this pipeline, it automatically locates the solution files, restores NuGet packages, builds the solution, and finally runs the tests.

A successful pipeline run confirms that your configuration is correctly building the application and running the tests.

3. Adding Unit Tests

Integrate automated tests using MSTest by creating unit tests for the conversion methods in the ConverterLib. The following code snippet demonstrates a comprehensive set of tests for converting Celsius to Fahrenheit, meters to feet, and kilograms to pounds:

using ConverterLib;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace Converter.Tests
{
    [TestClass]
    public class UnitTest1
    {
        private Converter _converter;

        [TestInitialize]
        public void Setup()
        {
            _converter = new Converter();
        }

        [TestMethod]
        public void TestCelsiusToFahrenheit()
        {
            double celsius = 0;
            double expected = 32;
            double result = _converter.CelsiusToFahrenheit(celsius);
            Assert.AreEqual(expected, result, 0.001, "0°C should be 32°F");

            celsius = 100;
            expected = 212;
            result = _converter.CelsiusToFahrenheit(celsius);
            Assert.AreEqual(expected, result, 0.001, "100°C should be 212°F");
        }

        [TestMethod]
        public void TestMetersToFeet()
        {
            double meters = 1;
            double expected = 3.28084;
            double result = _converter.MetersToFeet(meters);
            Assert.AreEqual(expected, result, 0.001, "1 meter should convert accurately to feet");
        }

        [TestMethod]
        public void TestKilogramsToPounds()
        {
            double kilograms = 10;
            double expected = 22.0462;
            double result = _converter.KilogramsToPounds(kilograms);
            Assert.AreEqual(expected, result, 0.001, "10 kilograms should equal 22.0462 pounds");
        }
    }
}

Running these tests locally should result in all tests passing provided the Converter class implements the logic correctly.

4. Simulating a Logical Error and Pipeline Failure

To highlight the importance of automated testing, consider simulating a logic error by modifying the ConverterLib. For instance, change the CelsiusToFahrenheit method so that it always returns 4 instead of calculating the correct value:

namespace ConverterLib
{
    public class Converter
    {
        public double CelsiusToFahrenheit(double celsius)
        {
            return 4;
            // Correct implementation:
            // return (celsius * (9.0 / 5.0)) + 32;
        }

        public double MetersToFeet(double meters)
        {
            return meters * 3.28084;
        }

        public double KilogramsToPounds(double kilograms)
        {
            return kilograms * 2.20462;
        }
    }
}

Manually running the application may display incorrect conversion outputs:

Hello, World!
The temperature is 30 degrees, or 4 degrees Fahrenheit

C:\Users\jeremyProjects\SimpleConverter\ConverterRunner\bin\Debug\net8.0\Converter.Runner.exe (process 29184) exited with code 0 (0x0).
...

Even though the build appears successful, the VSTest task in the pipeline will detect the error by failing the test due to the incorrect conversion result.

While the VS Build step may succeed, the overall pipeline will fail due to the failing unit tests, demonstrating the importance of automated testing in catching logic errors early in the development process.

5. Alternative Pipeline Configuration Without Test Tasks

In some cases, a pipeline might be configured only to build the solution and omit the test step. For example:

# .NET Desktop
# Build and run tests for .NET Desktop on Windows classic desktop solutions.
# Additional steps can be added to publish symbols, save build artifacts, etc.
# Learn more at: https://docs.microsoft.com/azure/devops/pipelines/apps/windows/dot-net
trigger:
  - master
pool:
  vmImage: 'windows-latest'
  name: 'Default'
variables:
  solution: '**/*.sln'
  buildPlatform: 'Any CPU'
  buildConfiguration: 'Release'
steps:
- task: NuGetToolInstaller@1
- task: NuGetCommand@2
  inputs:
    restoreSolution: '$(solution)'
- task: VSBuild@1
  inputs:
    solution: '$(solution)'
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'
- task: VSTest@2
  inputs:
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

Omitting the test task allows the build to complete successfully even if there are logical errors. It is critical to include comprehensive test tasks (like the VSTest task) to avoid deploying faulty applications to production.

6. Starter Pipeline for Customization

For developers starting with Azure DevOps pipelines, a minimal starter pipeline provides a good foundation that can be customized to include build, test, deployment, and artifact management tasks. Here's an example:

# Starter pipeline
# Customize this minimal pipeline to build and deploy your code.
trigger:
- master

pool:
  vmImage: ubuntu-latest

steps:
- script: echo Hello, world!
  displayName: 'Run a one-line script'
  
- script: |
    echo Add other tasks to build, test, and deploy your project.
    echo See https://aka.ms/yaml
  displayName: 'Run a multi-line script'

While pre-built templates can speed up initial setup, many developers prefer starting with a blank pipeline for full control over the process. Just remember to include unit testing tasks to catch any errors before deployment.

7. Summary

In this lesson, we integrated MSTest into an Azure DevOps pipeline for a simple C# converter application. By setting up the project, creating robust unit tests, and configuring the pipeline to automatically run tests, we ensure that any logic errors are caught early—prior to reaching staging or production. This demonstrates the essential role of automated testing within a Continuous Integration/Continuous Deployment (CI/CD) workflow.

Happy coding and best of luck with your DevOps journey!