What's on this page

Executions

When a Test Scenario is run it creates a Test Execution. This is a record of the outcome of the Test Scenario run and how long it took to execute.

Get all Executions

Produce a list of all the Test Executions of a Project or a Scenario recorded by LM.

Request Format

Aspect Value
Endpoint URL /api/behaviour/executions
HTTP Method GET

Request Query parameters

Parameter Description Mandatory
scenarioId Unique identifier of the Test Scenario about which to return execution information. No
projectId Unique identifier of the Test Project about which to return execution information about. No
includeScenario Indicates whether additional Test Scenario information should be returned as part of the Scenario Execution information. Supported values are “true” or “false” No

The “scenarioId” and “projectId” are mutually exclusive. One of these parameters must be supplied.

Response Format

Aspect Value
Response Code 200 – OK 404 – Project or Scenario ID could not be found.

Example Response Body

For a 200 Response Code:

Response Body content type: application/json

[

  {

​    "id": "string",

​    "name": "string",

​    "createdAt": "2019-03-01T15:37:43.705Z",

​    "lastModifiedAt": "2019-03-01T15:37:43.705Z",

​    "startedAt": "2019-03-01T15:37:43.705Z",

​    "finishedAt": "2019-03-01T15:37:43.705Z",

​    "scenarioId": "string",

​    "scenarioSummary": {

​      "description": "string",

​      "name": "string"

​    },

​    "status": "PASS"

  }

]

For a 40x, 500 or any other error response please see error responses

Response properties

Property Name Description Mandatory
id Unique identifier of the Test Execution Yes
name Textual name of the Test Execution Yes
createdAt Date and time the Test Execution was created Yes
lastModifiedAt Date and time the Test Execution was last modified Yes
startedAt Date and time the Test Execution was started Yes
finishedAt Date and time the Test Execution finished Yes
status The Pass/Fail status of the Test Execution. Supported values are “PASS” and “FAIL” Yes
scenarioId Unique Identifier of the Test Scenario that was executed Yes
scenarioSummary Additional information about the Test Scenario that was executed. Will only be included when the “includeScenario” query parameter is set to “true” Yes
description Textual description of the Test Scenario that was executed Yes
name Textual name of the Test Scenario Yes

Get a single Execution

Return the details of a single Test Scenario execution recorded by LM.

Request Format

Aspect Value
Endpoint URL /api/behaviour/executions{executionId}
HTTP Method GET

Request parameters

Parameter Name Description Mandatory
executionId Unique identifier of the behavior Test Execution to be returned. This can be any text string. Yes

Request Query parameters

Parameter Description Mandatory
includeScenario Indicates whether additional Test Scenario information should be returned as part of the Scenario Execution information. Supported values are “true” or “false” No

Response Format

Aspect Value
Response Code 200 – OK 404 – Execution ID could not be found.

Example Response Body

For a 200 Response Code:

Response Body content type: application/json

{

  "id": "string",

  "name": "string",

  "createdAt": "2019-03-01T16:03:54.278Z",

  "lastModifiedAt": "2019-03-01T16:03:54.278Z",

  "finishedAt": "2019-03-01T16:03:54.278Z",

  "startedAt": "2019-03-01T16:03:54.278Z",

  "status": "PASS"

  "error": "string",

  "executionOutline": "string",

  "executionReport": "string",

  "scenarioId": "string",

  "scenarioSummary": {

​    "description": "string",

​    "name": "string"

  },

  "registeredAssemblies": [

​    {

​      "assemblyDescriptorName": "string",

​      "assemblyId": "string",

​      "assemblyName": "string",

​      "installed": true,

​      "uninstalled": true

​    }

  ],

  "registeredMetrics": [

​    "string"

  ],

  "stageReports": [

​    {

​      "name": "string",

​      "status": "PASS",

​      "steps": [

​        {

​          "endTime": "2019-03-01T16:03:54.278Z",

​          "error": "string",

​          "slices": [

​            "string"

​          ],

​          "startTime": "2019-03-01T16:03:54.278Z",

​          "status": "PASS",

​          "stepDisplayName": "string",

​          "stepType": "PRECONDITION"

​        }

​      ]

​    }

  ],

}

For a 40x, 500 or any other error response please see error responses

Response properties

Property Name Description Mandatory
id Unique identifier of the Test Execution Yes
name Textual name of the Test Execution Yes
createdAt Date and time the Test Execution was created Yes
lastModifiedAt Date and time the Test Execution was last modified Yes
startedAt Date and time the Test Execution was started Yes
finishedAt Date and time the Test Execution finished Yes
status The Pass/Fail status of the Test Execution. Supported values are “PASS” and “FAIL” Yes
scenarioId Unique Identifier of the Test Scenario that was executed Yes
scenarioSummary Additional information about the Test Scenario that was executed. Will only be included when the “includeScenario” query parameter is set to “true” Yes
error Any errors reported during the execution of the Test Scenario Yes
executionOutline Short textual report on the Test Scenario execution Yes
executionReport Textual report on the Test Scenario execution Yes
registeredAssemblies A list of the Test Assemblies that were used in the Test Execution Yes
stageReports A list of reports for each stage executed as part of the Test Execution Yes
registeredMetrics A list of textual strings representing the names of the metrics that were gathered as part of the Test Execution Yes

Contents of scenarioSummary:

Property Name Description Mandatory
description Textual description of the Test Scenario that was executed Yes
name Textual name of the Test Scenario Yes

Contents of registeredAssemblies:

Property Name Description Mandatory
assemblyDescriptorName Name of the Test Assembly template from which the any Assembly using this configuration will be created. Yes
assemblyId Unique ID of the Test Assembly used Yes
assemblyName Textual name of the Test Assembly used Yes
installed Indicates whether this Test Assembly was installed. Supported values are “true” and “false” Yes
uninstalled Indicates whether this Test Assembly was uninstalled. Supported values are “true” and “false” Yes

Contents of stageReports:

Property Name Description Mandatory
name Textual name of the Stage Yes
status The Pass/Fail status of the Test Stage. Supported values are “PASS” and “FAIL” Yes
steps List of the individual Test Steps executed as part of the Test Stage. Yes

Contents of steps:

Property Name Description Mandatory
stepDisplayName Textual name of the Test Step Yes
stepType The type of the Test Step Yes
status The Pass/Fail status of the Test Execution. Supported values are “PASS” and “FAIL” Yes
startTime Date and time the Test Step was started Yes
endTime Date and time the Test Step finished Yes
error Any error reported during the execution of the Test Step Yes
slices List of the BDD test clauses that were executed as part of the Test Step Yes

Execute a Scenario

Requests LM to run the specified behavior Test Scenario using a specific Test Assembly (which must already exist).

Request Format

Aspect Value
Endpoint URL /api/behaviour/executions
HTTP Method POST

Example Request Body

Request Body content type: application/json

{

  "scenarioId": "string",

  "assemblyId": "string",

  "assemblyName": "string"

}

Request parameters

Parameter Description Mandatory
scenarioId Unique identifier for the behavior Test Scenario. This can be any text string, it is not limited to GUIDs. Yes
assemblyId Unique ID of the Test Assembly used in the behavior Test Scenario Yes
assemblyName Textual name of the Test Assembly used in the behavior Test Scenario Yes

Response Format

Aspect Value
Response Code 200 – OK 404 – Scenario or Assembly not found (see returned error message structure for details)

For a 40x, 500 or any other error response please see section 4.1 above

Get Scenario Execution Progress

Obtain information about the current progress and status of a single Test Scenario execution.

Request Format

Aspect Value
Endpoint URL /api/behaviour/executions/{executionId}/progress
HTTP Method GET

Request parameters

Parameter Description Mandatory
executionId Unique identifier of the behavior Test Execution to be returned. This can be any text string. Yes

Response Format

Aspect Value
Response Code 200 – OK 404 – Test Execution ID could not be found

Example Response Body

For a 200 Response Code:

Response Body content type: application/json

{

  "error": "string",

  "finishedAt": "2019-03-01T17:30:03.257Z",

  "startedAt": "2019-03-01T17:30:03.257Z",

  "status": "PASS"

  "id": "string",

  "name": "string",

  "registeredAssemblies": [

​    {

​      "assemblyDescriptorName": "string",

​      "assemblyId": "string",

​      "assemblyName": "string",

​      "installed": true,

​      "uninstalled": true

​    }

  ],

  "registeredMetrics": [

​    "string"

  ],

  "stageReports": [

​    {

​      "name": "string",

​      "status": "PASS",

​      "steps": [

​        {

​          "endTime": "2019-03-01T17:30:03.257Z",

​          "error": "string",

​          "slices": [

​            "string"

​          ],

​          "startTime": "2019-03-01T17:30:03.257Z",

​          "status": "PASS",

​          "stepDisplayName": "string",

​          "stepType": "PRECONDITION"

​        }

​      ]

​    }

  ],

}

For a 40x, 500 or any other error response please see section 4.1 above

Response properties

Property Name Description Mandatory
id Unique identifier of the Test Execution Yes
name Textual name of the Test Execution Yes
startedAt Date and time the Test Execution was started Yes
finishedAt Date and time the Test Execution finished Yes
status The Pass/Fail status of the Test Execution. Supported values are “PASS” and “FAIL” Yes
error Any errors reported during the execution of the Test Scenario Yes
registeredAssemblies A list of the Test Assemblies that were used in the Test Execution Yes
stageReports A list of reports for each stage executed as part of the Test Execution Yes
registeredMetrics A list of textual strings representing the names of the metrics that were gathered as part of the Test Execution Yes

Contents of registeredAssemblies:

Property Name Description Mandatory
assemblyDescriptorName Name of the Test Assembly template from which the any Assembly using this configuration will be created. Yes
assemblyId Unique ID of the Test Assembly used Yes
assemblyName Textual name of the Test Assembly used Yes
installed Indicates whether this Test Assembly was installed. Supported values are “true” and “false” Yes
uninstalled Indicates whether this Test Assembly was uninstalled. Supported values are “true” and “false” Yes

Contents of stageReports:

Property Name Description Mandatory
name Textual name of the Stage Yes
status The Pass/Fail status of the Test Stage. Supported values are “PASS” and “FAIL” Yes
steps List of the individual Test Steps executed as part of the Test Stage. Yes

Contents of steps:

Property Name Description Mandatory
stepDisplayName Textual name of the Test Step Yes
stepType The type of the Test Step Yes
status The Pass/Fail status of the Test Execution. Supported values are “PASS” and “FAIL” Yes
startTime Date and time the Test Step was started Yes
endTime Date and time the Test Step finished Yes
error Any error reported during the execution of the Test Step Yes
slices List of the BDD test clauses that were executed as part of the Test Step Yes

Get all Metrics for a Scenario Execution

Return a list of all the Metrics information available within LM for a single Test Execution run.

Request Format

Aspect Value
Endpoint URL /api/behavior/executions/{executionId}/metrics
HTTP Method GET

Request parameters

Parameter Description Mandatory
executionId Unique identifier of the behavior Test Execution to be returned. This can be any text string. Yes

Response Format

Aspect Value
Response Code 200 – OK 404 – Test Execution ID could not be found

Example Response Body

For a 200 Response Code:

Response Body content type: application/json

[

  {

​    "data": {

​      "additionalProp1": "string",

​      "additionalProp2": "string",

​      "additionalProp3": "string"

​    },

​    "executionId": "string",

​    "id": "string",

​    "metricThresholds": [

​      {

​        "endTime": 0,

​        "startTime": 0,

​        "type": "less than",

​        "value": 0

​      }

​    ],

​    "name": "string"

  }

]

For a 40x, 500 or any other error response please see section 4.1 above

Response properties

Property Name Description Mandatory
id Unique identifier for the Metric information Yes
Name Textual name of the Metric Yes
executionId Unique identifier of the behavior Test Execution to which the Metric information relates. This can be any text string. Yes
data A list of Metric information presented as Name-Value pairs. Yes
metricThresholds A list of Metric threshold information Yes

Contents of metricThresholds:

Property Name Description Mandatory
type Textual representation of the threshold type Yes
value The threshold value Yes
startTime The start time for the threshold value Yes
endTime The stop time for the threshold value Yes

Get a single Scenario Execution Metric

Return all the Metric information available within LM for a single Metric generated as part of a Test Execution run.

Request Format

Aspect Value
Endpoint URL /api/behaviour/executions/{executionId}/metrics/{metricId}
HTTP Method PUT

Request parameters

Parameter Description Mandatory
executionId Unique identifier of the behavior Test Execution to be returned. This can be any text string. Yes
metricId Unique identifier of the Test Execution Metric to be returned. Yes

Response Format

Aspect Value
Response Code 200 – OK 404 – Test Execution or Metric ID could not be found

Example Response Body

For a 200 Response Code:

Response Body content type: application/json

{

  "data": {

​    "additionalProp1": "string",

​    "additionalProp2": "string",

​    "additionalProp3": "string"

  },

  "executionId": "string",

  "id": "string",

  "metricThresholds": [

​    {

​      "endTime": 0,

​      "startTime": 0,

​      "type": "less than",

​      "value": 0

​    }

  ],

  "name": "string"

}

For a 40x, 500 or any other error response please see section 4.1 above

Response properties

Property Name Description Mandatory
id Unique identifier for the Metric information Yes
Name Textual name of the Metric Yes
executionId Unique identifier of the behavior Test Execution to which the Metric information relates. This can be any text string. Yes
data A list of Metric information presented as Name-Value pairs. Yes
metricThresholds A list of Metric threshold information Yes

Contents of metricThresholds:

Property Name Description Mandatory
type Textual representation of the threshold type Yes
value The threshold value Yes
startTime The start time for the threshold value Yes
endTime The stop time for the threshold value Yes