Table of Contents

Service & Resource Management benchmarks

Specifications of the test servers & clients

Metrics 1 - 21 & 24 - 48

  • Intel Xeon Silver 4210
  • 8 cores (16TH) VM
  • 32 GB RAM
  • SSD (NVMe)
  • Windows Server 2019 Standard

Metrics 22 - 23

  • Intel Xeon E5-2620 v3
  • 2 sockets
  • 32 GB RAM
  • HDD
  • Elasticsearch
  • Windows Server 2012 R2 Standard

Metrics 49 - 54

  • Intel Xeon Silver 4210
  • 10 CPU VM
  • 32 GB RAM
  • HDD
  • Elasticsearch
  • Windows Server 2019 Standard

Metric 55

  • Intel Core i7-9700 CPU @3.20 GHz
  • 8 cores
  • 32 GB RAM
  • SSD
  • Cassandra
  • Windows 10 Enterprise

Benchmarks

# Specification Scope Metric Remarks Configuration
1 GetEligibleResources, node by node, not filling the linkertable entries DMS 216.10 ms 1 protocol with 10 parameters (double/string, basic param (id, name, description, read and write)); 1 parameter per function; 10 functions per protocol;
10 main resource elements; 10 resources per element (functionguid, name, functionname, maindveDMAID, MainDVEElementID, LinkertableEntries, description, mode available, max concurrency maxvalue)
1 ServiceDefinition (name, description, empty graph, new GUID)
20 nodes in ServiceDefinition (id, position, configuration with random functionid)
20 overlapping ReservationInstances in the time range		 Clean DMA, no other data.
The metric indicates the average time for requesting the eligible resources for all 20 nodes in 20 requests.
2 GetEligibleResources, all resources in one request, not filling the linkertable entries DMS 170.91 ms 1 protocol with 10 parameters (double/string, basic param (id, name, description, read and write)); 1 parameter per function; 10 functions per protocol;
10 main resource elements; 10 resources per element (functionguid, name, functionname, maindveDMAID, MainDVEElementID, LinkertableEntries, description, mode available, max concurrency maxvalue)
1 ServiceDefinition (name, description, empty graph, new GUID)
20 nodes in ServiceDefinition (id, position, configuration with random functionid)
20 overlapping ReservationInstances in the time range;		 Clean DMA, no other data.
The metric indicates the average time for requesting the eligible resources for all 20 nodes in 1 request.
3 GetEligibleResources, node by node, filling linkertable entries DMS 666.97 ms 1 protocol with 10 parameters (double/string, basic param (id, name, description, read and write)); 1 parameter per function; 10 functions per protocol;
10 main resource elements; 10 resources per element (functionguid, name, functionname, maindveDMAID, MainDVEElementID, LinkertableEntries, description, mode available, max concurrency maxvalue)
1 ServiceDefinition (name, description, empty graph, new GUID)
20 nodes in ServiceDefinition (id, position, configuration with random functionid)
20 overlapping ReservationInstances in the time range;		 Clean DMA, no other data.
The metric indicates the average time for requesting the eligible resources for all 20 nodes in 20 requests.
4 GetEligibleResources, all resources in one request, filling linkertable entries DMS 176.64 ms 1 protocol with 10 parameters (double/string, basic param (id, name, description, read and write)); 1 parameter per function; 10 functions per protocol;
10 main resource elements; 10 resources per element (functionguid, name, functionname, maindveDMAID, MainDVEElementID, LinkertableEntries, description, mode available, max concurrency maxvalue)
1 ServiceDefinition (name, description, empty graph, new GUID)
20 nodes in ServiceDefinition (id, position, configuration with random functionid)
20 overlapping ReservationInstances in the time range;		 Clean DMA, no other data.
The metric indicates the average time for requesting the eligible resources for all 20 nodes in 1 request.
5 Filter ReservationInstance.ID DMS 19.92 ms Based on 200 non-overlapping ReservationInstances (name, serviceDefinitionID, status confirmed, enhancedserviceProtocolID, new GUID, serviceID)
6 events, basic Script:eventname		 reservationid
40 properties (random word);
1 ServiceDefinition (name, description, empty graph, new GUID)
10 nodes in ServiceDefinition (id, position, configuration with random functionid)		 200 ReservationInstances, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
6 Filter ReservationInstance.Name equals DMS 18.47 ms Based on 200 non-overlapping ReservationInstances (name, serviceDefinitionID, status confirmed, enhancedserviceProtocolID, new GUID, serviceID)
6 events, basic Script:eventname		 reservationid
40 properties (random word);
1 ServiceDefinition (name, description, empty graph, new GUID)
10 nodes in ServiceDefinition (id, position, configuration with random functionid)		 200 ReservationInstances, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
7 Filter ReservationInstance.Name contains DMS 26.02 ms Based on 200 non-overlapping ReservationInstances (name, serviceDefinitionID, status confirmed, enhancedserviceProtocolID, new GUID, serviceID)
6 events, basic Script:eventname		 reservationid
40 properties (random word);
1 ServiceDefinition (name, description, empty graph, new GUID)
10 nodes in ServiceDefinition (id, position, configuration with random functionid)		 200 ReservationInstances, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
8 Filter ReservationInstance.Start DMS 22.60 ms Based on 200 non-overlapping ReservationInstances (name, serviceDefinitionID, status confirmed, enhancedserviceProtocolID, new GUID, serviceID)
6 events, basic Script:eventname		 reservationid
40 properties (random word);
1 ServiceDefinition (name, description, empty graph, new GUID)
10 nodes in ServiceDefinition (id, position, configuration with random functionid)		 200 ReservationInstances, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
9 Filter ReservationInstance.End DMS 21.11 ms Based on 200 non-overlapping ReservationInstances (name, serviceDefinitionID, status confirmed, enhancedserviceProtocolID, new GUID, serviceID)
6 events, basic Script:eventname		 reservationid
40 properties (random word);
1 ServiceDefinition (name, description, empty graph, new GUID)
10 nodes in ServiceDefinition (id, position, configuration with random functionid)		 200 ReservationInstances, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
10 Filter ReservationInstance.Properties DMS 18.31 ms Based on 200 non-overlapping ReservationInstances (name, serviceDefinitionID, status confirmed, enhancedserviceProtocolID, new GUID, serviceID)
6 events, basic Script:eventname		 reservationid
40 properties (random word);
1 ServiceDefinition (name, description, empty graph, new GUID)
10 nodes in ServiceDefinition (id, position, configuration with random functionid)		 200 ReservationInstances, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
11 Filter ReservationInstance.ServiceDefinitionID DMS 19.09 ms Based on 200 non-overlapping ReservationInstances (name, serviceDefinitionID, status confirmed, enhancedserviceProtocolID, new GUID, serviceID)
6 events, basic Script:eventname		 reservationid
40 properties (random word);
1 ServiceDefinition (name, description, empty graph, new GUID)
10 nodes in ServiceDefinition (id, position, configuration with random functionid)		 200 ReservationInstances, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
12 Filter ServiceDefinition.ID DMS 16.29 ms Based on 200 ServiceDefinitions (name, description, empty graph, new GUID)
10 nodes in ServiceDefinition (id, position, configuration with random functionid)
40 properties (random word)		 200 ServiceDefinitions, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
13 Filter ServiceDefinition.Name equals DMS 17.16 ms Based on 200 ServiceDefinitions (name, description, empty graph, new GUID)
10 nodes in DeviceDefinition (id, position, configuration with random functionid)
40 properties (random word)		 200 ServiceDefinitions, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
14 Filter ServiceDefinition.Name contains DMS 21.24 ms Based on 200 ServiceDefinitions (name, description, empty graph, new GUID)
10 nodes in ServiceDefinition (id, position, configuration with random functionid)
40 properties (random word)		 200 ServiceDefinitions, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
15 Filter ServiceDefinition.Description equals DMS 17.74 ms Based on 200 ServiceDefinitions (name, description, empty graph, new GUID)
10 nodes in ServiceDefinition (id, position, configuration with random functionid)
40 properties (random word)		 200 ServiceDefinitions, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
16 Filter ServiceDefinition.Description contains DMS 22.92 ms Based on 200 ServiceDefinitions (name, description, empty graph, new GUID)
10 nodes in ServiceDefinition (id, position, configuration with random functionid)
40 properties (random word)		 200 ServiceDefinitions, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
17 Filter ServiceDefinition.CreatedAt DMS 19.11 ms Based on 200 ServiceDefinitions (name, description, empty graph, new GUID)
10 nodes in ServiceDefinition (id, position, configuration with random functionid)
40 properties (random word)		 200 ServiceDefinitions, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
18 Filter ServiceDefinition.LastModified DMS 19.43 ms Based on 200 ServiceDefinitions (name, description, empty graph, new GUID)
10 nodes in ServiceDefinition (id, position, configuration with random functionid)
40 properties (random word)		 200 ServiceDefinitions, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
19 Filter ServiceDefinition.Properties DMS 15.42 ms Based on 200 ServiceDefinitions (name, description, empty graph, new GUID)
10 nodes in ServiceDefinition (id, position, configuration with random functionid)
40 properties (random word)		 200 ServiceDefinitions, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
20 Filter ReservationDefinition.ID DMS 23.47 ms Based on 200 ReservationDefinitions (name, description, empty graph, new GUID)
10 nodes in Definition (id, position, configuration with random functionid)
40 properties (random word)		 200 ReservationDefinitions, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
21 Filter ReservationDefinition.Name equals DMS 21.94 ms Based on 200 ReservationDefinitions (name, description, empty graph, new GUID)
10 nodes in Definition (id, position, configuration with random functionid)
40 properties (random word)		 200 ReservationDefinitions, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
22 Time it takes to add 10,000 service definitions DMA 1,168 s
23 Time it takes to delete 10,000 service definitions DMA 309 s
24 Filter ReservationDefinition.Name contains DMS 23.94 ms Based on 200 ReservationDefinitions (name, description, empty graph, new GUID)
10 nodes in Definition (id, position, configuration with random functionid)
40 properties (random word)		 200 ReservationDefinitions, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
25 Filter ReservationDefinition.Properties DMS 22.23 ms Based on 200 ReservationDefinitions (name, description, empty graph, new GUID)
10 nodes in Definition (id, position, configuration with random functionid)
40 properties (random word)		 200 ReservationDefinitions, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
26 Filter ReservationDefinition.ServiceID DMS 22.72 ms Based on 200 ReservationDefinitions (name, description, empty graph, new GUID)
10 nodes in Definition (id, position, configuration with random functionid)
40 properties (random word)		 200 ReservationDefinitions, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
27 Filter SRMServiceInfo.ServiceID DMS 16.04 ms Based on 200 SRMServiceInfo items (Name, Description, ServiceDefinitionId, ServiceResources)
10 nodes in Definition (id, position, configuration with random functionid)
40 properties random word;		 200 SRMServiceInfo items, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
28 Filter SRMServiceInfo.Name equals DMS 16.32 ms Based on 200 SRMServiceInfo items (Name, Description, ServiceDefinitionId, ServiceResources)
10 nodes in Definition (id, position, configuration with random functionid)
40 properties random word;		 200 SRMServiceInfo items, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
29 Filter SRMServiceInfo.Name contains DMS 21.50 ms Based on 200 SRMServiceInfo items (Name, Description, ServiceDefinitionId, ServiceResources)
10 nodes in Definition (id, position, configuration with random functionid)
40 properties random word;		 200 SRMServiceInfo items, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
30 Filter SRMServiceInfo.Description equals DMS 16.38 ms Based on 200 SRMServiceInfo items (Name, Description, ServiceDefinitionId, ServiceResources)
10 nodes in Definition (id, position, configuration with random functionid)
40 properties random word;		 200 SRMServiceInfo items, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
31 Filter SRMServiceInfo.Description contains DMS 21.59 ms Based on 200 SRMServiceInfo items (Name, Description, ServiceDefinitionId, ServiceResources)
10 nodes in Definition (id, position, configuration with random functionid)
40 properties random word;		 200 SRMServiceInfo items, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
32 Filter SRMServiceInfo.CreatedAt DMS 17.79 ms Based on 200 SRMServiceInfo items (Name, Description, ServiceDefinitionId, ServiceResources)
10 nodes in Definition (id, position, configuration with random functionid)
40 properties random word;		 200 SRMServiceInfo items, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
33 Filter SRMServiceInfo.LastModifiedAt DMS 22.32 ms Based on 200 SRMServiceInfo items (Name, Description, ServiceDefinitionId, ServiceResources)
10 nodes in Definition (id, position, configuration with random functionid)
40 properties random word;		 200 SRMServiceInfo items, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
34 Filter SRMServiceInfo.Properties DMS 15.21 ms Based on 200 SRMServiceInfo items (Name, Description, ServiceDefinitionId, ServiceResources)
10 nodes in Definition (id, position, configuration with random functionid)
40 properties random word;		 200 SRMServiceInfo items, no other data.
The metric indicates the average hit time, i.e. "total time to filter" divided by "the number of items returned".
35 Creating 1,000 ReservationInstances DMS 179,063.43 ms ReservationInstances contain between 10 and 200 resources (id, description, name, maxconcurrency 1000) and have 4 events (OnStarting, OnStarted, OnStopping, OnStopped) configured. They are scheduled to start between 2 hours and 3 days. 200 resources, no other data.
36 Reading all 1,000 ReservationInstances DMS 11,881.99 ms ReservationInstances contain between 10 and 200 resources (id, description, name, maxconcurrency 1000) and have 4 events (OnStarting, OnStarted, OnStopping, OnStopped) configured. They are scheduled to start between 2 hours and 3 days. 200 resources, no other data.
37 Deleting 1,000 ReservationInstances DMS 157,858.89 ms ReservationInstances contain between 10 and 200 resources (id, description, name, maxconcurrency 1000) and have 4 events (OnStarting, OnStarted, OnStopping, OnStopped) configured. They are scheduled to start between 2 hours and 3 days. 200 resources, no other data.
38 Start ReservationInstances DMS 66.97 s Based on 200 ReservationInstances with the same start time; This is the time until all are started.
ReservationInstance configuration:
10 resources (Name, available, max capacity 999999), status confirmed, name		 Clean DMA, no other data.
Metric indicates the time until all ReservationInstances are started, measured using an information event output by the configured start script.
39 Stopping ReservationInstances DMS 21.76 s Based on 200 ReservationInstances with the same start time; This is the time until all are started.
ReservationInstance configuration:
10 resources (Name, available, max capacity 999999), status confirmed, name		 Clean DMA, no other data.
Metric indicates the time until all ReservationInstances are started, measured using an information event output by the configured stop script
40 Event start delays DMS 19.47 s Based on 200 ReservationInstances with the same start time; This is the time until all are started.
ReservationInstance configuration:
10 resources (Name, available, max capacity 999999), status confirmed, name		 Clean DMA, no other data.
Metric indicates the time until all ReservationInstances are started, measured using the time at which the ResourceManagerEventMessage with status "Ongoing" for the reservation is received.
41 Event stop delays DMS 18.99 s Based on 200 ReservationInstances with the same start time; This is the time until all are started.
ReservationInstance configuration:
10 resources (Name, available, max capacity 999999), status confirmed, name		 Clean DMA, no other data.
Metric indicates the time until all ReservationInstances are started, measured using the time at which the ResourceManagerEventMessage with status "Ended" for the reservation is received.
42 Create 500 ServiceDefinitions DMS 5.29 s The service definitions only have a name and an ID. Clean DMA, no other data.
43 Deleting 500 ServiceDefinitions DMS 15.48 s The service definitions only have a name and an ID. 500 ServiceDefinitions, no other data.
44 Create 500 ServiceDefinitions DMS 9.24 s The service definitions only have a name and an ID.
The reservation instances only have a name, an ID and a status, and are scheduled to start immediately after creation and take 1 day.		 250 ReservationInstances
45 Deleting 500 ServiceDefinitions DMS 24.77 s The service definitions only have a name and and ID.
The reservation instances only have a name, an ID and a status, and are scheduled to start immediately after creation and take 1 day.		 250 ReservationInstances and 500 ServiceDefinitions, no other data.
46 Create 1,000 Resources DMS 1.14 s The resources contain an ID, a name, a description and 1 capacity. Clean DMA, no other data.
47 Create 1,000 Resources DMS 1.22 s The resources contain an ID, a name, a description and 1 capacity. 1,000 existing resources, no other data.
48 Delete 1,000 Resources DMS 16.07 s The resources contain an ID, a name, a description and 1 capacity. 2,000 existing resources, no other data.
49 SRM standard solution: 15 single bookings with 18 resources - average creation time DMS 9.7 s
50 SRM standard solution: 5 contributing bookings with 4 resources - average creation time DMS 5.6 s
51 SRM standard solution: 5 contributing bookings with 4 resources - average convert time DMS 3.7 s
52 SRM standard solution: 5 main bookings with 2 resources & 1 contributing booking – average creation time DMS 6 s
53 SRM standard solution: Service creation delay DMS 28.4 s
54 SRM standard solution: Service deleting delay DMS 18.9 s
55 Bookings app with 5000 bookings in viewport/timerange DataMiner Cube 1 s Follow mode enabled (= update every second), initial loading around 2.5 s Running Automation script "RT_SRM_Resource_5000bookings_NoProperties"
56 Resources + bookings timeline on root view DataMiner Cube 8 s Loaded 1235 active resources from the system with +1000 ongoing permanent bookings.
57 All resources timeline on root view DataMiner Cube 42 s Loaded approx. 5620 active resources from the system. 80-95% of the execution happens externally (so no client code involved). Fetching and loading all resources + resource bands from the root view takes 2 to 3.5 s.
Propose changes