Executions
When a Test Scenario is run it creates a Test Execution. This is a record of the outcome of the Test Scenario run and how long it took to execute.
Get all Executions
Produce a list of all the Test Executions of a Project or a Scenario recorded by LM.
Request Format
| Aspect | Value |
|---|---|
| Endpoint URL | /api/behaviour/executions |
| HTTP Method | GET |
Request Query parameters
| Parameter | Description | Mandatory |
|---|---|---|
| scenarioId | Unique identifier of the Test Scenario about which to return execution information. | No |
| projectId | Unique identifier of the Test Project about which to return execution information about. | No |
| includeScenario | Indicates whether additional Test Scenario information should be returned as part of the Scenario Execution information. Supported values are “true” or “false” | No |
The “scenarioId” and “projectId” are mutually exclusive. One of these parameters must be supplied.
Response Format
| Aspect | Value |
|---|---|
| Response Code | 200 – OK 404 – Project or Scenario ID could not be found. |
Example Response Body
For a 200 Response Code:
Response Body content type: application/json
[
{
"id": "string",
"name": "string",
"createdAt": "2019-03-01T15:37:43.705Z",
"lastModifiedAt": "2019-03-01T15:37:43.705Z",
"startedAt": "2019-03-01T15:37:43.705Z",
"finishedAt": "2019-03-01T15:37:43.705Z",
"scenarioId": "string",
"scenarioSummary": {
"description": "string",
"name": "string"
},
"status": "PASS"
}
]
For a 40x, 500 or any other error response please see error responses
Response properties
| Property Name | Description | Mandatory |
|---|---|---|
| id | Unique identifier of the Test Execution | Yes |
| name | Textual name of the Test Execution | Yes |
| createdAt | Date and time the Test Execution was created | Yes |
| lastModifiedAt | Date and time the Test Execution was last modified | Yes |
| startedAt | Date and time the Test Execution was started | Yes |
| finishedAt | Date and time the Test Execution finished | Yes |
| status | The Pass/Fail status of the Test Execution. Supported values are “PASS” and “FAIL” | Yes |
| scenarioId | Unique Identifier of the Test Scenario that was executed | Yes |
| scenarioSummary | Additional information about the Test Scenario that was executed. Will only be included when the “includeScenario” query parameter is set to “true” | Yes |
| description | Textual description of the Test Scenario that was executed | Yes |
| name | Textual name of the Test Scenario | Yes |
Get a single Execution
Return the details of a single Test Scenario execution recorded by LM.
Request Format
| Aspect | Value |
|---|---|
| Endpoint URL | /api/behaviour/executions{executionId} |
| HTTP Method | GET |
Request parameters
| Parameter Name | Description | Mandatory |
|---|---|---|
| executionId | Unique identifier of the behavior Test Execution to be returned. This can be any text string. | Yes |
Request Query parameters
| Parameter | Description | Mandatory |
|---|---|---|
| includeScenario | Indicates whether additional Test Scenario information should be returned as part of the Scenario Execution information. Supported values are “true” or “false” | No |
Response Format
| Aspect | Value |
|---|---|
| Response Code | 200 – OK 404 – Execution ID could not be found. |
Example Response Body
For a 200 Response Code:
Response Body content type: application/json
{
"id": "string",
"name": "string",
"createdAt": "2019-03-01T16:03:54.278Z",
"lastModifiedAt": "2019-03-01T16:03:54.278Z",
"finishedAt": "2019-03-01T16:03:54.278Z",
"startedAt": "2019-03-01T16:03:54.278Z",
"status": "PASS"
"error": "string",
"executionOutline": "string",
"executionReport": "string",
"scenarioId": "string",
"scenarioSummary": {
"description": "string",
"name": "string"
},
"registeredAssemblies": [
{
"assemblyDescriptorName": "string",
"assemblyId": "string",
"assemblyName": "string",
"installed": true,
"uninstalled": true
}
],
"registeredMetrics": [
"string"
],
"stageReports": [
{
"name": "string",
"status": "PASS",
"steps": [
{
"endTime": "2019-03-01T16:03:54.278Z",
"error": "string",
"slices": [
"string"
],
"startTime": "2019-03-01T16:03:54.278Z",
"status": "PASS",
"stepDisplayName": "string",
"stepType": "PRECONDITION"
}
]
}
],
}
For a 40x, 500 or any other error response please see error responses
Response properties
| Property Name | Description | Mandatory |
|---|---|---|
| id | Unique identifier of the Test Execution | Yes |
| name | Textual name of the Test Execution | Yes |
| createdAt | Date and time the Test Execution was created | Yes |
| lastModifiedAt | Date and time the Test Execution was last modified | Yes |
| startedAt | Date and time the Test Execution was started | Yes |
| finishedAt | Date and time the Test Execution finished | Yes |
| status | The Pass/Fail status of the Test Execution. Supported values are “PASS” and “FAIL” | Yes |
| scenarioId | Unique Identifier of the Test Scenario that was executed | Yes |
| scenarioSummary | Additional information about the Test Scenario that was executed. Will only be included when the “includeScenario” query parameter is set to “true” | Yes |
| error | Any errors reported during the execution of the Test Scenario | Yes |
| executionOutline | Short textual report on the Test Scenario execution | Yes |
| executionReport | Textual report on the Test Scenario execution | Yes |
| registeredAssemblies | A list of the Test Assemblies that were used in the Test Execution | Yes |
| stageReports | A list of reports for each stage executed as part of the Test Execution | Yes |
| registeredMetrics | A list of textual strings representing the names of the metrics that were gathered as part of the Test Execution | Yes |
Contents of scenarioSummary:
| Property Name | Description | Mandatory |
|---|---|---|
| description | Textual description of the Test Scenario that was executed | Yes |
| name | Textual name of the Test Scenario | Yes |
Contents of registeredAssemblies:
| Property Name | Description | Mandatory |
|---|---|---|
| assemblyDescriptorName | Name of the Test Assembly template from which the any Assembly using this configuration will be created. | Yes |
| assemblyId | Unique ID of the Test Assembly used | Yes |
| assemblyName | Textual name of the Test Assembly used | Yes |
| installed | Indicates whether this Test Assembly was installed. Supported values are “true” and “false” | Yes |
| uninstalled | Indicates whether this Test Assembly was uninstalled. Supported values are “true” and “false” | Yes |
Contents of stageReports:
| Property Name | Description | Mandatory |
|---|---|---|
| name | Textual name of the Stage | Yes |
| status | The Pass/Fail status of the Test Stage. Supported values are “PASS” and “FAIL” | Yes |
| steps | List of the individual Test Steps executed as part of the Test Stage. | Yes |
Contents of steps:
| Property Name | Description | Mandatory |
|---|---|---|
| stepDisplayName | Textual name of the Test Step | Yes |
| stepType | The type of the Test Step | Yes |
| status | The Pass/Fail status of the Test Execution. Supported values are “PASS” and “FAIL” | Yes |
| startTime | Date and time the Test Step was started | Yes |
| endTime | Date and time the Test Step finished | Yes |
| error | Any error reported during the execution of the Test Step | Yes |
| slices | List of the BDD test clauses that were executed as part of the Test Step | Yes |
Execute a Scenario
Requests LM to run the specified behavior Test Scenario using a specific Test Assembly (which must already exist).
Request Format
| Aspect | Value |
|---|---|
| Endpoint URL | /api/behaviour/executions |
| HTTP Method | POST |
Example Request Body
Request Body content type: application/json
{
"scenarioId": "string",
"assemblyId": "string",
"assemblyName": "string"
}
Request parameters
| Parameter | Description | Mandatory |
|---|---|---|
| scenarioId | Unique identifier for the behavior Test Scenario. This can be any text string, it is not limited to GUIDs. | Yes |
| assemblyId | Unique ID of the Test Assembly used in the behavior Test Scenario | Yes |
| assemblyName | Textual name of the Test Assembly used in the behavior Test Scenario | Yes |
Response Format
| Aspect | Value |
|---|---|
| Response Code | 200 – OK 404 – Scenario or Assembly not found (see returned error message structure for details) |
For a 40x, 500 or any other error response please see section 4.1 above
Get Scenario Execution Progress
Obtain information about the current progress and status of a single Test Scenario execution.
Request Format
| Aspect | Value |
|---|---|
| Endpoint URL | /api/behaviour/executions/{executionId}/progress |
| HTTP Method | GET |
Request parameters
| Parameter | Description | Mandatory |
|---|---|---|
| executionId | Unique identifier of the behavior Test Execution to be returned. This can be any text string. | Yes |
Response Format
| Aspect | Value |
|---|---|
| Response Code | 200 – OK 404 – Test Execution ID could not be found |
Example Response Body
For a 200 Response Code:
Response Body content type: application/json
{
"error": "string",
"finishedAt": "2019-03-01T17:30:03.257Z",
"startedAt": "2019-03-01T17:30:03.257Z",
"status": "PASS"
"id": "string",
"name": "string",
"registeredAssemblies": [
{
"assemblyDescriptorName": "string",
"assemblyId": "string",
"assemblyName": "string",
"installed": true,
"uninstalled": true
}
],
"registeredMetrics": [
"string"
],
"stageReports": [
{
"name": "string",
"status": "PASS",
"steps": [
{
"endTime": "2019-03-01T17:30:03.257Z",
"error": "string",
"slices": [
"string"
],
"startTime": "2019-03-01T17:30:03.257Z",
"status": "PASS",
"stepDisplayName": "string",
"stepType": "PRECONDITION"
}
]
}
],
}
For a 40x, 500 or any other error response please see section 4.1 above
Response properties
| Property Name | Description | Mandatory |
|---|---|---|
| id | Unique identifier of the Test Execution | Yes |
| name | Textual name of the Test Execution | Yes |
| startedAt | Date and time the Test Execution was started | Yes |
| finishedAt | Date and time the Test Execution finished | Yes |
| status | The Pass/Fail status of the Test Execution. Supported values are “PASS” and “FAIL” | Yes |
| error | Any errors reported during the execution of the Test Scenario | Yes |
| registeredAssemblies | A list of the Test Assemblies that were used in the Test Execution | Yes |
| stageReports | A list of reports for each stage executed as part of the Test Execution | Yes |
| registeredMetrics | A list of textual strings representing the names of the metrics that were gathered as part of the Test Execution | Yes |
Contents of registeredAssemblies:
| Property Name | Description | Mandatory |
|---|---|---|
| assemblyDescriptorName | Name of the Test Assembly template from which the any Assembly using this configuration will be created. | Yes |
| assemblyId | Unique ID of the Test Assembly used | Yes |
| assemblyName | Textual name of the Test Assembly used | Yes |
| installed | Indicates whether this Test Assembly was installed. Supported values are “true” and “false” | Yes |
| uninstalled | Indicates whether this Test Assembly was uninstalled. Supported values are “true” and “false” | Yes |
Contents of stageReports:
| Property Name | Description | Mandatory |
|---|---|---|
| name | Textual name of the Stage | Yes |
| status | The Pass/Fail status of the Test Stage. Supported values are “PASS” and “FAIL” | Yes |
| steps | List of the individual Test Steps executed as part of the Test Stage. | Yes |
Contents of steps:
| Property Name | Description | Mandatory |
|---|---|---|
| stepDisplayName | Textual name of the Test Step | Yes |
| stepType | The type of the Test Step | Yes |
| status | The Pass/Fail status of the Test Execution. Supported values are “PASS” and “FAIL” | Yes |
| startTime | Date and time the Test Step was started | Yes |
| endTime | Date and time the Test Step finished | Yes |
| error | Any error reported during the execution of the Test Step | Yes |
| slices | List of the BDD test clauses that were executed as part of the Test Step | Yes |
Get all Metrics for a Scenario Execution
Return a list of all the Metrics information available within LM for a single Test Execution run.
Request Format
| Aspect | Value |
|---|---|
| Endpoint URL | /api/behavior/executions/{executionId}/metrics |
| HTTP Method | GET |
Request parameters
| Parameter | Description | Mandatory |
|---|---|---|
| executionId | Unique identifier of the behavior Test Execution to be returned. This can be any text string. | Yes |
Response Format
| Aspect | Value |
|---|---|
| Response Code | 200 – OK 404 – Test Execution ID could not be found |
Example Response Body
For a 200 Response Code:
Response Body content type: application/json
[
{
"data": {
"additionalProp1": "string",
"additionalProp2": "string",
"additionalProp3": "string"
},
"executionId": "string",
"id": "string",
"metricThresholds": [
{
"endTime": 0,
"startTime": 0,
"type": "less than",
"value": 0
}
],
"name": "string"
}
]
For a 40x, 500 or any other error response please see section 4.1 above
Response properties
| Property Name | Description | Mandatory |
|---|---|---|
| id | Unique identifier for the Metric information | Yes |
| Name | Textual name of the Metric | Yes |
| executionId | Unique identifier of the behavior Test Execution to which the Metric information relates. This can be any text string. | Yes |
| data | A list of Metric information presented as Name-Value pairs. | Yes |
| metricThresholds | A list of Metric threshold information | Yes |
Contents of metricThresholds:
| Property Name | Description | Mandatory |
|---|---|---|
| type | Textual representation of the threshold type | Yes |
| value | The threshold value | Yes |
| startTime | The start time for the threshold value | Yes |
| endTime | The stop time for the threshold value | Yes |
Get a single Scenario Execution Metric
Return all the Metric information available within LM for a single Metric generated as part of a Test Execution run.
Request Format
| Aspect | Value |
|---|---|
| Endpoint URL | /api/behaviour/executions/{executionId}/metrics/{metricId} |
| HTTP Method | PUT |
Request parameters
| Parameter | Description | Mandatory |
|---|---|---|
| executionId | Unique identifier of the behavior Test Execution to be returned. This can be any text string. | Yes |
| metricId | Unique identifier of the Test Execution Metric to be returned. | Yes |
Response Format
| Aspect | Value |
|---|---|
| Response Code | 200 – OK 404 – Test Execution or Metric ID could not be found |
Example Response Body
For a 200 Response Code:
Response Body content type: application/json
{
"data": {
"additionalProp1": "string",
"additionalProp2": "string",
"additionalProp3": "string"
},
"executionId": "string",
"id": "string",
"metricThresholds": [
{
"endTime": 0,
"startTime": 0,
"type": "less than",
"value": 0
}
],
"name": "string"
}
For a 40x, 500 or any other error response please see section 4.1 above
Response properties
| Property Name | Description | Mandatory |
|---|---|---|
| id | Unique identifier for the Metric information | Yes |
| Name | Textual name of the Metric | Yes |
| executionId | Unique identifier of the behavior Test Execution to which the Metric information relates. This can be any text string. | Yes |
| data | A list of Metric information presented as Name-Value pairs. | Yes |
| metricThresholds | A list of Metric threshold information | Yes |
Contents of metricThresholds:
| Property Name | Description | Mandatory |
|---|---|---|
| type | Textual representation of the threshold type | Yes |
| value | The threshold value | Yes |
| startTime | The start time for the threshold value | Yes |
| endTime | The stop time for the threshold value | Yes |