Table of Contents

Creating a test package

In this tutorial, you create a basic low-code app, prepare a DataMiner test package to test it, and trigger a QAOps test run.

Expected duration: 15 to 25 minutes.

The content and screenshots for this tutorial were created using DataMiner 10.6.4 and DIS 3.1.20.

Prerequisites

Overview

Step 1: Create a new DataMiner test package project

  1. Open Visual Studio and select Create new project.

  2. Select the DataMiner Test Package Project template, and then click Next.

  3. Enter a project name, for example, MyFirstTestPackage.

  4. Choose a location, for example, C:\DataMiner Testing.

  5. Make sure Place solution and project in the same directory is not selected.

  6. Click Next.

  7. Enter your name or initials in the Author field.

  8. Verify that Create DataMiner Package options is selected.

  9. For this tutorial, keep GitHub Workflows set to None.

  10. Click Create.

After the project opens, you should see the Getting Started Markdown file.

Step 2: Create a low-code app to be tested

Using a different DataMiner System than QAOps Sandbox where you have the necessary rights, create a low-code app for this test:

  1. In a browser, go to the home page of your DataMiner system.

  2. Click + Create app.

  3. In the header bar, specify a name for your app, for example, QAOps tutorial.

  4. Click Create page.

  5. Hover the mouse pointer over the colored bar with + to open the visualizations panel.

  6. From the General section of the visualizations panel, drag the Text visualization onto your page to create a component that will display text.

    Text visualization in the visualizations pane

  7. From the Basic Controls section of the panel, drag the Text Input visualization onto your page to create a component where users will be able to enter text.

    Text input visualization in the visualizations pane

  8. Select your new Text component, go to Settings, and enter an expression similar to Welcome {COMPONENT."Current view"."Text input 2".Value.Texts.Value}.

    You can begin by writing an open brace after the Welcome and follow the provided Intellisense, for example:

    Settings pane for Text component showing how Intellisense helps you choose the next value

  9. Select your new Text Input component, go to Settings, and enable Value Change.

  10. Publish the app and verify that text entered in the input box appears in the "Welcome" text box.

    Example of the low-code app where someone has entered "NiceTutorial" in the text input field, and the text field displays "Welcome NiceTutorial"

  11. Copy the app identifier from the URL and save it in a text file.

    For example, copy "991d7084-cd0a-412b-bffe-0ea176fc5430" from https://localhost/app/991d7084-cd0a-412b-bffe-0ea176fc5430/Page.

Step 3: Create the DataMiner environment to be tested

  1. In Visual Studio, connect DIS to the DataMiner System where you created the low-code app.

  2. In the menu bar, select Extensions > DIS > DMA > Connect.

  3. Open Solution Explorer (shortcut: Ctrl+Alt+L).

  4. Expand Package Content.

  5. Right-click LowCodeApps, select Add, and then click Import DataMiner Low-Code App.

    Menu selection to import a low-code app into the test package project

  6. Select the app you have created in step 2.

Step 4: Write a test

  1. Open Solution Explorer (shortcut: Ctrl+Alt+L).

  2. Right-click Solution 'MyFirstTestPackage', select Add, and then select New Item (Ctrl+Shift+A).

  3. Select C# Class and name it "PlaywrightUITest.cs".

    Double-check that the name matches exactly, as this is important for this specific tutorial.

  4. Confirm that the file is added under Solution Items.

    Playwright test code in a QAOps test package project

  5. Replace the file content with the following code:

    #:property PublishAot=false
#:package Microsoft.Playwright@1.57.0
#:package Skyline.DataMiner.CICD.Tools.WinEncryptedKeys.Lib@2.1.0

using Microsoft.Playwright;

using Skyline.DataMiner.CICD.Tools.WinEncryptedKeys.Lib;

using System;
using System.IO;
using System.Linq;
using System.Text.RegularExpressions;
using System.Threading;
using System.Threading.Tasks;

const string Url = "https://localhost/app/991d7084-cd0a-412b-bffe-0ea176fc5430/Page";
const string InputValue = "Random Person Name";

try
{
    var playwrightFolder = Path.Combine(
    Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData),
    "ms-playwright");

    var chromiumInstalled =
        Directory.Exists(playwrightFolder) &&
        Directory.GetDirectories(playwrightFolder, "chromium-*").Any();

    if (!chromiumInstalled)
    {
        Microsoft.Playwright.Program.Main(new[] { "install", "chromium" });
    }

    using var playwright = await Playwright.CreateAsync();

    await using var browser = await playwright.Chromium.LaunchAsync(new BrowserTypeLaunchOptions
    {
        Headless = false,
    });

    var context = await browser.NewContextAsync(new BrowserNewContextOptions
    {
        IgnoreHTTPSErrors = true,
    });

    var page = await context.NewPageAsync();

    await page.GotoAsync(Url, new PageGotoOptions
    {
        WaitUntil = WaitUntilState.NetworkIdle,
    });

    await page.WaitForLoadStateAsync(LoadState.DOMContentLoaded);

    await LoginLocally(page);

    Thread.Sleep(5000);
    await page.WaitForLoadStateAsync(LoadState.DOMContentLoaded);

    var textInputComponent = await FindFirstComponentByDataComponentNameRegexAsync(
        page,
        new Regex(@"^Text Input \d+$", RegexOptions.IgnoreCase));

    if (textInputComponent is null)
        throw new InvalidOperationException("Could not find a component with data-component-name matching '^Text Input \\d+$'.");

    var input = textInputComponent.Locator("input[type='text']").First;
    await input.WaitForAsync();
    await input.FillAsync(InputValue);

    var textComponent = await FindFirstComponentByDataComponentNameRegexAsync(
        page,
        new Regex(@"^Text \d+$", RegexOptions.IgnoreCase));

    if (textComponent is null)
        throw new InvalidOperationException("Could not find a component with data-component-name matching '^Text \\d+$'.");

    var expectedText = $"Welcome {InputValue}";
    await Assertions.Expect(textComponent).ToContainTextAsync(expectedText);
    Thread.Sleep(5000);
    Console.WriteLine($"PLAYWRIGHT_TEST_SUCCESS: Validated text component contains '{expectedText}'.");
    Environment.Exit(0);
}
catch (Exception ex)
{
    Console.Error.WriteLine($"PLAYWRIGHT_TEST_FAILURE: {ex}");
    Environment.Exit(1);
}

static async Task<ILocator?> FindFirstComponentByDataComponentNameRegexAsync(IPage page, Regex regex)
{
    var components = page.Locator("dma-db-component[data-component-name]");
    var count = await components.CountAsync();

    for (var i = 0; i < count; i++)
    {
        var component = components.Nth(i);
        var name = await component.GetAttributeAsync("data-component-name");

        if (!string.IsNullOrWhiteSpace(name) && regex.IsMatch(name))
            return component;
    }

    return null;
}

static async Task LoginLocally(IPage page)
{
    if (!Keys.TryRetrieveKey("QAOpsDataMinerUser", out var userName) || string.IsNullOrWhiteSpace(userName))
        throw new InvalidOperationException("Could not retrieve QAOpsDataMinerUser.");

    if (!Keys.TryRetrieveKey("QAOpsDataMinerPassword", out var password) || string.IsNullOrWhiteSpace(password))
        throw new InvalidOperationException("Could not retrieve QAOpsDataMinerPassword.");

    await page.GetByPlaceholder("Username").Last.FillAsync(userName);
    await page.GetByPlaceholder("Password").Last.FillAsync(password);
    await page.Locator("dwa-cta-button").GetByText("Sign in").ClickAsync();
}

  6. Look up the following line:

    const string Url = "https://localhost/app/991d7084-cd0a-412b-bffe-0ea176fc5430/Page";

  7. Replace the identifier in the URL with the identifier you saved in step 2.

Note

In this example, the test is written as a .NET 10 C# file-based app. You can also implement tests in another language or by using a different testing framework. C# file-based apps can be run directly from a CLI by using dotnet run.

Step 5: Harvest the test

  1. Open Solution Explorer (shortcut: Ctrl+Alt+L).

  2. Expand TestPackageContent > TestHarvesting and open TestDiscovery.ps1.

  3. Replace the existing content with the following content:

    $ErrorActionPreference = 'Stop'

# Base path four levels up, cross-platform
$pathToSolutionRoot = Resolve-Path (Join-Path $PSScriptRoot '..\..\..\')

$pathToGeneratedTests = Join-Path $PSScriptRoot 'tests.generated'
$pathToGeneratedDependencies = Join-Path $PSScriptRoot 'dependencies.generated'
$pathToXmlAutomationTests = Join-Path $PSScriptRoot 'xmlautomationtests.generated'

# Clean up any previous output
if (Test-Path $pathToGeneratedTests) {
    Remove-Item -Recurse -Force $pathToGeneratedTests
}

if (Test-Path $pathToGeneratedDependencies) {
    Remove-Item -Recurse -Force $pathToGeneratedDependencies
}

if (Test-Path $pathToXmlAutomationTests) {
    Remove-Item -Recurse -Force $pathToXmlAutomationTests
}

New-Item -ItemType Directory -Force -Path $pathToGeneratedTests  | Out-Null
New-Item -ItemType Directory -Force -Path $pathToGeneratedDependencies  | Out-Null
New-Item -ItemType Directory -Force -Path $pathToXmlAutomationTests  | Out-Null

Write-Host "Looking for top-level .cs files in solution folder: $pathToSolutionRoot" -ForegroundColor Cyan

$topLevelCsFiles = Get-ChildItem -Path $pathToSolutionRoot -File -Filter '*.cs'

foreach ($file in $topLevelCsFiles) {
    Copy-Item $file.FullName (Join-Path $pathToGeneratedTests $file.BaseName) -Force
}

Write-Host "Copied $($topLevelCsFiles.Count) top-level .cs file(s)." -ForegroundColor Cyan

# Warning: Do not clean up the collected files here. The next step in the SDK will use these.
Write-Information "`n🎉 Script completed successfully!"
exit 0

Step 6: Write the test execution PowerShell script

  1. Open Solution Explorer (shortcut: Ctrl+Alt+L).

  2. Expand TestPackageContent > TestPackagePipeline and open 2.TestPackageExecution.ps1.

    Test harvesting and execution files in the test package

  3. Replace the existing content with the following content:

    param (
    [Parameter(Mandatory = $true)]
    [string]$PathToTestPackageContent
)

$ErrorActionPreference = 'Stop'

# Import common code module
Import-Module -Name (Join-Path $PSScriptRoot 'CommonCode.psm1')

$pathToTestHarvesting = Join-Path $PathToTestPackageContent 'TestHarvesting'
$pathToGeneratedTests = Join-Path $pathToTestHarvesting 'tests.generated'
$pathToGeneratedDependencies = Join-Path $pathToTestHarvesting 'dependencies.generated'
$pathToTests = Join-Path $PathToTestPackageContent 'Tests'
$pathToDependencies = Join-Path $PathToTestPackageContent 'Dependencies'
$pathToPlaywrightUiTest = Join-Path $pathToGeneratedTests 'PlaywrightUITest'

# Track script start time
$scriptStart = Get-Date

try {
    Write-Host "Running Test Package tests..." -ForegroundColor Cyan
    Write-Host "Executing Playwright UI test: $pathToPlaywrightUiTest" -ForegroundColor Cyan

    if (-not (Test-Path $pathToPlaywrightUiTest)) {
        throw "Playwright UI test file not found: $pathToPlaywrightUiTest"
    }

    Write-Host "Checking NuGet sources..." -ForegroundColor Cyan

    $nugetSourcesOutput = & dotnet nuget list source 2>&1
    $nugetSourcesExitCode = $LASTEXITCODE

    if ($nugetSourcesExitCode -ne 0) {
        throw "Failed to list NuGet sources. Output: $($nugetSourcesOutput | Out-String)"
    }

    $nugetSourcesText = ($nugetSourcesOutput | Out-String)

    if ($nugetSourcesText -notmatch '(?im)^\s*\d+\.\s+nuget\.org\s*\[Enabled\]') {
        Write-Host "nuget.org source not found. Adding nuget.org..." -ForegroundColor Yellow

        $addNugetSourceOutput = & dotnet nuget add source 'https://api.nuget.org/v3/index.json' --name 'nuget.org' 2>&1
        $addNugetSourceExitCode = $LASTEXITCODE

        if ($addNugetSourceExitCode -ne 0) {
            throw "Failed to add nuget.org NuGet source. Output: $($addNugetSourceOutput | Out-String)"
        }

        Write-Host "nuget.org source added successfully." -ForegroundColor Green
    }
    else {
        Write-Host "nuget.org source already exists." -ForegroundColor Green
    }

    Write-Host "Starting Playwright test..." -ForegroundColor Cyan

    # Create temporary .cs file
    $tempPlaywrightFile = "$pathToPlaywrightUiTest.cs"
    Copy-Item $pathToPlaywrightUiTest $tempPlaywrightFile -Force

    $playwrightOutput = & dotnet run $tempPlaywrightFile 2>&1
    $playwrightExitCode = $LASTEXITCODE

    Remove-Item $tempPlaywrightFile -Force

    $playwrightMessage = ($playwrightOutput | Out-String).Trim()

    Write-Host "Playwright output:" -ForegroundColor DarkGray
    Write-Host $playwrightMessage

    if ($playwrightExitCode -eq 0) {
        if ([string]::IsNullOrWhiteSpace($playwrightMessage)) { 
            $playwrightMessage = "Playwright UI test passed." 
        }

        Write-Host "Playwright UI test SUCCEEDED." -ForegroundColor Green

        try { Push-TestCaseResult -Outcome 'OK' -Name 'pipeline_PlaywrightUITest' -Duration ((Get-Date) - $scriptStart) -Message $playwrightMessage -TestAspect Assertion } catch {}
    }
    else {
        if ([string]::IsNullOrWhiteSpace($playwrightMessage)) { 
            $playwrightMessage = "Playwright UI test failed." 
        }

        Write-Host "Playwright UI test FAILED." -ForegroundColor Red

        try { Push-TestCaseResult -Outcome 'Fail' -Name 'pipeline_PlaywrightUITest' -Duration ((Get-Date) - $scriptStart) -Message $playwrightMessage -TestAspect Assertion } catch {}

        throw "Playwright UI test failed with exit code $playwrightExitCode."
    }

    Write-Host "Test Package execution finished successfully." -ForegroundColor Green

    try { Push-TestCaseResult -Outcome 'OK' -Name 'pipeline_TestPackageExecution' -Duration ((Get-Date) - $scriptStart) -Message 'Test Package execution finished.' -TestAspect Execution } catch {}
}
catch {
    Write-Host "Test Package execution FAILED: $($_.Exception.Message)" -ForegroundColor Red

    try { Push-TestCaseResult -Outcome 'Fail' -Name 'pipeline_TestPackageExecution' -Duration ((Get-Date) - $scriptStart) -Message "Exception during Test Package execution: $($_.Exception.Message)" -TestAspect Execution } catch {}

    exit 1
}

Step 7: Download, install, and verify the QAOps tool

  1. Open a Command Prompt, Bash, or PowerShell window.

  2. Check if you have nuget.org as a known NuGet source:

    dotnet nuget list source

  3. Verify that the output contains nuget.org [Enabled].

    Note

    The first time you run a dotnet command on a computer, you will see a welcome message. The output of your command is displayed below that message.

  4. If your sources do not contain nuget.org, add it with the following command. Otherwise, skip this step.

    dotnet nuget add source https://api.nuget.org/v3/index.json -n "nuget.org"

  5. Install the QAOps tool:

    dotnet tool install skyline.dataminer.qaops --global

  6. Verify that the tool is available:

    dataminer-qaops --help

    The command output will display a description of the tool and the available commands.

Note

If you see the exception "Unable to load the service index for source ...", one of your configured NuGet sources may be unreachable or may have expired credentials.

To continue this tutorial, you can temporarily remove the failing source and add it again later when needed:

dotnet nuget remove source "NameOfSource"

Step 8: Find the unique test and configuration identifiers

For more details about these entities, see QAOps configuration and QAOps test suite.

  1. In the QAOps User app (i.e., the green QAOps app), go to the Configurations page.

  2. Select Demo Configuration and Demo Test Suite.

  3. Copy the configuration ID and save it in a text file.

  4. Copy the test suite ID and save it in a text file.

QAOps Identifier Selections

Step 9: Create a token

  1. In the QAOps User app, go to the Tokens page.

  2. In the top-left corner, click Create Token.

  3. Enter a name for the token.

  4. Select the scope that matches the ".execute" scope for your selected configuration.

  5. Click Generate Token.

  6. Wait until the token value is shown.

  7. Copy the token value and save it in a text file.

    For production environments, use a key vault solution.

Step 10: Create and find the test package

  1. In Visual Studio, open Solution Explorer (shortcut: Ctrl+Alt+L).

  2. Right-click the solution and select ReBuild Solution.

  3. Right-click Solution 'MyFirstTestPackage' and select Open Folder in File Explorer.

  4. In File Explorer, go to the bin\Debug\net48\DataMinerBuild folder of your project.

  5. Copy the generated QAOps test package .dmtest file path and save it in a text file.

Step 11: Trigger the test run

  1. Open a Command Prompt, Bash, or PowerShell window.

  2. Run the following command, after replacing the placeholders as indicated below, making sure to keep the double quotes around the TOKEN and TESTFILEPATH values:

    dataminer-qaops test-run --token "TOKEN" -t TESTSUITE -c CONFIGURATION -tags MYNAME -san saqaopssandbox --test-packages "TESTFILEPATH"

    • TOKEN: The token value you copied earlier. Make sure this value is enclosed in double quotes.

    • TESTSUITE: The test suite ID you copied earlier.

    • CONFIGURATION: The configuration ID you copied earlier.

    • MYNAME: Your name, nickname, or another identifier that helps you find your request.

    • TESTFILEPATH: The test package filepath you copied earlier. Make sure this value is enclosed in double quotes.

    Note

    For production systems, leave out the -san argument. This argument specifies which QAOps system receives the command. In this example, it targets the QAOps sandbox system. The default target is the production QAOps system.

Step 12: Verify that the request was received

  1. In the QAOps User app, go to the Overview page.

  2. Locate your tag in the list.

  3. Use the filter at the top to find your request more quickly.

  4. Track the test run life cycle.

Tip

To view and interpret test results, see Viewing test results.

Improve this page