Using trend patterns to detect backup failures

This tutorial illustrates DataMiner's pattern matching feature in the context of Dashboards and Low-Code Apps. It will show you how to access patterns and occurrences within a dashboard and how to effectively handle missing pattern occurrences, using the example of server backups. By utilizing a trend pattern, DataMiner will detect the backups. The goal of the example dashboard in this tutorial is to generate a comprehensive report indicating whether daily backup patterns have been detected for all servers.

Estimated duration: 20 minutes.

Note

The content and screenshots for this tutorial were created in DataMiner 10.4.4.

Tip

• For more information, such as technical limitations of pattern matching, see Pattern matching.
• See also: Kata #41: Trend patterns in Cube - recap Empower and Kata #42: Trend patterns in Dashboards & LCA on DataMiner Dojo.

Prerequisites

• A DataMiner System connected to dataminer.services.
• DataMiner 10.4.4 or higher with Storage as a Service (STaaS) (recommended) or a self-managed Cassandra database with indexing.
• Pattern matching is enabled (in DataMiner Cube: System Center > System settings > analytics config).

Overview

The tutorial consists of the following steps:

• Step 1: Install the example package from the Catalog
• Step 2: Compile a list of the most recent backup pattern occurrences per server
• Step 3: Interpret the timestamp of every occurrence

Step 1: Install the example package from the Catalog

1. Go to https://catalog.dataminer.services/details/cc65fcf5-de87-4e3f-8209-fdd027928e56.

2. Deploy the Catalog item to your DataMiner Agent by clicking the Deploy button.

   This will create a DataMiner element named Kata - Server overview in your system, which will be used throughout the rest of the tutorial. The element will be located in the view Kata > Trend patterns in Dashboards and LCA. This will also create a trend pattern and generate occurrences. This might take a couple of minutes.

3. In DataMiner Cube, check whether the element Kata - Server overview has been created before you continue to the next step.

Step 2: Compile a list of the most recent backup pattern occurrences per server

1. Create a new dashboard.

2. Drag and drop the Dropdown visualization from the pane on the left to the top of the dashboard.

   

   This component will allow users to select a specific trend pattern, which will be used in a query to retrieve the occurrences.

3. Drag and drop the table visualization from the pane on the left to the dashboard, below the dropdown component.

   

   This will add a table component. In this component, you will show the output of a GQI query.

4. Drag the edges of the table component to resize it until it takes up the width of your screen.

5. In the data pane, open the QUERIES section and click the "+" icon to create a new query.

   

6. Name the new query Patterns.

7. Configure the query as follows:

   1. Select the Get trend data patterns data source.

   2. Add a Select operator and make sure to have both Pattern name and Pattern ID selected.

      

8. Drag the query to the dropdown component at the top of the dashboard.

   This will link the component to the query, allowing the user to select any of the patterns in the system.

9. In the dropdown component, select the pattern Kata backup.

   This pattern has been added by the package you installed earlier.

10. Create another query and name it Most recent pattern occurrences per server.

11. Drag the query to the table component in order to link the component to this query.

12. Configure the query as follows:

    1. Select the Get trend data pattern events data source.

    2. Add a Filter operator and configure it as follows:

       • Column: Pattern ID.

       • Filter method: Equals

       • Value: Click the link icon in the value box, then select your Dropdown feed and the property Pattern ID, and click Apply.

         

13. Add an operator to extract the server name from the table key, so you will be able to use this as a column name:

    1. At the bottom of the query, select Apply custom operator, and then select ParameterKeyToSeparateColumns.

    2. In the Parameter Key Column box, select Parameter ID.

    

14. Add another custom operator to rename the Table Index column to Server.

    1. Select Apply custom operator, and then select Rename column.

    2. Select the column Table Index and specify the new name Server.

    

15. Configure the query so that the most recent pattern occurrences per server will be retrieved:

    1. Add another custom operator to transform the values from the Start time field from DateTime to double format:

       1. Select Apply custom operator, and then select DateTimeToDouble.

       2. Select the DateTime column Start time.

       This transformation is necessary to be able to do an aggregate operation on the Start time column, as the Aggregate operator does not support DateTime columns yet.

    2. Add an Aggregate operator to get the most recent pattern occurrence per server:

       • Column: Start time (as double)
       • Method: Maximum

    3. Add a Group by operator, selecting the column Server.

    4. Add the DoubleToDateTime custom operator to transform the result back to DateTime format.

    Your query should now look like this:

    

16. Add another Rename column custom operator to rename the aggregated result column to Most recent pattern occurrence.

    

Step 3: Interpret the timestamp of every occurrence

Now that you have retrieved the most recent pattern occurrence per server, you can use a custom operator to make a judgment based on those timestamps. A custom operator provides a lot of flexibility, allowing you to code pretty much whatever you want. In this case, the custom operator DateIsOlderThan will be used, which is available in the Catalog and was also included in the package you installed earlier. It adds a column indicating if the date is older than a given number of days.

1. At the bottom of the Most recent pattern occurrences per server query, add the DateIsOlderThan custom operator, with the following configuration:

   • Date Time Column: Most recent pattern occurrence
   • Number of days: 1

   

2. Use the Template editor to configure a validation icon/color for the Most recent pattern occurrence column:

   1. Select the table, go to Layout > Column appearance, and select the Most recent pattern occurrence column.

   2. Below this, select Left.

   3. On the right side of box where you have selected the column, select ... > Customize preset to open the template editor.

      

   4. In the template editor, go to the Tools tab, select Icon, and drag the mouse pointer on the left side of the work area to draw the icon layer.

      While the icon is selected, you can use the Dimensions boxes on the right to resize and reposition it exactly the way you want.

   5. Select the text layer, and adjust the right margin via the lower-left Dimensions box on the right, so that the text is displayed next to the icon.

   6. Add a condition to the icon layer for when the validation is OK:

      1. Click + Add case.

      2. Click When condition, select Validation, and specify the filter Yes.

      3. Select Show icon, set the color to green (e.g. #85A864) and the icon to a check mark.

         When you select the icon, you can use the filter box to quickly find a specific icon.

   7. Add another condition to the icon layer for when the validation returns invalid:

      1. Click + Add case.

      2. Click When condition, select Validation, and specify the filter No.

      3. Select Show icon, set the color to red (e.g. #FF0000) and the icon to an "error" exclamation mark.

   8. Similarly, add conditions to the text layer that will turn the text color to green or red depending on the validation.

   

   Tip

   See also: Template editor.

3. Hide the columns in the table that no longer need to be shown:

   1. In the data pane, make sure you are no longer editing the Most recent pattern occurrences query, and expand the query so you can see the columns it contains.

   2. Drag and drop the columns Server and Most recent pattern occurrence to the table component, so that these are added as a filter.

   

The resulting dashboard could for instance look like this: