Scripts allow you to define the steps and expected results for your Scenarios. They provide clear instructions for testers or automated frameworks, ensuring consistency in how scenarios are executed and making your tests more reliable and well-documented.
Since these scripts are data-driven and are based on a test model, they are more maintainable. Adding or removing parameter values, constraints, or forced interactions in your model updates the Scenarios data driving the Scripts, but there is often no need to adjust the script details.
You can quickly transform optimized test data as depicted in Figure 1 into customizable scripts (Figures 2 and 3).
Figure 1 - Scenarios Data
Figure 2 - Manual Scripts
Figure 3 - Automate Scripts
Let’s see how we can do that. To access this feature, open a model - e.g. click My Test Models (Figure 4 - 1) and then click any of them (Figure 4 - 2) - or create a new one.
Figure 4 - Open Test Model
A menu will appear on the left. There, click Scripts (Figure 5 - 1) and select one of the two options: Automate (Figure 5 - 2) or Manual (Figure 5 - 3).
Figure 5 - Scripts menu navigation
Automate
Automate (Figure 5 - 2) is a smart Gherkin feature file editor. It is “aware” of the parameters, values, and generated test cases of the current model as well as the Gherkin syntax, with highlighting and autocomplete features. Automate makes it easy to create a feature file with data-driven Scenario and/or Scenario Outline sections that are populated by the generated test cases of your model.
Note: We’ll try to make the distinction between the Scenarios screen - with its scriptless data table - and the Scenario/Scenario Outline options of Automate scripts as clear as possible.
Working on an Automate Script
Top ribbon
Figure 6 - Top ribbon functions in Automate
Work on your Gherkin feature in the editor panel on the left (Figure 6 - 1), and the generated Scenario sections or Examples data tables are shown in the preview panel on the right (Figure 8 - 2). Once you're done editing, click the Save button (Figure 6 - 3). This button will become active as soon as you make a change in the text field (Figure 6 - 1). You must generate test cases (Scenarios data table) first before the preview becomes active, then select the appropriate strength (Figure 6 - 4).
You can click the Usage button (Figure 6 - 5) for a refresher on the Automate functionality.
Feature and Background
Each Gherkin script has a Feature as its first keyword followed by : and a little bit of context to describe the scope, usually in the form of a user story (which is demonstrated in the placeholder text). This section does not support parameterization of any kind.
A Background section is useful when a set of initial setup steps is repeated before each Scenario and Scenario Outline in the feature file. Starting a new line in Background and further sections brings up a suggestion dialog with standard Gherkin keywords. Keep in mind that restricting a parameter in the Background (with {[]}, as explained in the Conditional Logic section below) is valid syntax and is used to apply only the selected value to all Scenarios or Scenario Outlines for a given Feature.
Scenario or Scenario Outline
Next in the feature file is the enumeration of different testing scenarios. One way to describe a test case is with a Scenario block. It contains sequential steps, expressed as plain English Given/When/Then statements, that describe the necessary actions for executing the automated test script. If the model generates 28 rows on the Scenarios screen, then you could think that 28 Gherkin Scenario sections would need to be written out to achieve the combinatorial coverage.
However, instead of writing out all 28 scripts individually, you leverage Automate to create a single data-driven Scenario that includes the <Parameter Name> syntax. Then the tool can generate all 28 needed Scenario sections in the exported feature file. You can check the number of associated test cases in the Preview panel (Figure 7).
Figure 7 - Scenario block example in Automate
A Scenario Outline behaves similarly to a data-driven Scenario. When using a Scenario Outline, instead of having 28 Scenario sections (one for each test case), the export has one data-driven Scenario Outline with a Gherkin Examples data table attached to it that contains 28 rows (again, one for each test case). The Preview panel shows the first 10 rows of the Examples data table. The order of parameters in the Examples is dictated by the script, not by the Parameters screen of your model.
Figure 8 - Scenario Outline block example in Automate
Note: where possible, using Scenario Outline sections is preferable to generating many data-driven Scenario ones as Outline is a more compact expression of the testing ideas of the model.
The basic principles of your work with an Automate script are similar to those in any other Gherkin text editor. Click the grey background in the left panel to edit the script's text and write anything you want. You can phrase the actions however you’d like and utilize any combination of Gherkin keywords. With that said, (in most cases) you must connect Scripts to the Scenarios table via the parameterized syntax.
Simply typing < will populate a list of autocomplete suggestions related to all of the parameters in the model. At the end of the list, we have system variables:
- Test Number (sequential number within the Scenario script block),
- Test Case (constant number of the row in the complete table on the Scenarios screen)
- Expected Outcome (from the Forced Interactions screen, if applicable)
Wherever the <Parameter Name> appears in the test steps it will be replaced by the specified parameter's value for the corresponding test case. For example, writing the step:
And they select <Color> for the car's color
means that in each generated Scenario script block, the <Color> portion of the step will be replaced with the value of the Color parameter for the corresponding test case:
And they select Deep Blue Metallic for the car's color
or
And they select Red for the car’s color
etc.
Note: it is common to put dynamic elements in quotes (e.g., And they select "<Color>" for the car's color) to make them more visually distinguishable.
As you insert parameters into the script, they will become greyed out in the dropdown you get from typing <. It’s a quick way to check that you haven’t accidentally omitted any important variables by the time the script is finished.
Automate - Reusable Steps
You can make a step reusable in other scripts by clicking the "+" button that appears to the left of a Gherkin step (the line must have an indent).
If the current test model is in a project, you will be prompted to select whether the step will be reusable within that project only or for every test model in your organization.
To leverage reusable steps, simply start typing on a step line in any Gherkin feature file. Automate will show matching reusable steps in the autocomplete dropdown as you type. Select the desired step from the list, and its content will be inserted at that position. Reusable steps can include parameters using angle bracket notation (e.g. <State>), just like regular steps. Reusable steps can also include value conditions (e.g. {State[West Virginia, Pennsylvania]}).
You can access the library of reusable steps by opening the model properties menu (click on the model name at the top) and selecting the "Automate Reusable Step Library" menu item. In the library, you can see the list of steps, when they were created, when they were last used, and who created them.
You can edit a reusable step from the library by clicking the step's name (or the "pencil" icon highlighted above). The name is the content that gets inserted when the step is used. You can press the "Enter" key to save the changes or "Esc" to cancel them. You can also delete a reusable step here by clicking the "trash" icon.
Editing a reusable step affects its content for all future uses, but existing scripts that have already used the step won't be affected by the change.
Automate - Conditional Logic
Oftentimes application flows may have more or fewer steps that need to be executed for specific test cases. For example, in Tesla's car configurator, if the user is configuring a new Tesla Model S, there is no option for the Interior Layout.
Let’s say our test model for the Tesla car configurator contains a bi-directional bound pair constraint tying Model = Model S and Interior Layout = N/A together. If we don’t do anything else and just pass <Model> and <Interior Layout> to an Automate script, some test cases will read:
And they select the N/A Interior Layout
While you could figure out what this means in manual testing or set up an extra code snippet in automated testing, it’s a bit suboptimal.
Automate provides a way to restrict a Scenario or Scenario Outline to test cases that contain a specific value using the {Parameter Name[Parameter Value]} syntax. Simply typing { will populate a list of autocomplete suggestions related to all of the parameter values in the model.
Figure 9 - Example of a conditional restriction in Automate
Including {Parameter Name[Parameter Value]} anywhere in the set of steps in a Scenario or Scenario Outline will restrict the test cases included in the generated Scenario blocks or Examples data table to only those that contain the specified parameter value.
So, for instance, you would create 1 Scenario block with {Model[Model X]} and with the Interior Layout step + 1 Scenario block with {Model[Model S]} and without the Interior Layout step.
As mentioned earlier, if all script blocks within an Automate tab follow the same conditional logic, you can restrict all of them in one action by using the {Parameter[Parameter Value]} syntax in the Background section
You can have more than 1 value per restriction, just separate them with commas. It is also possible to invert a restriction by adding an exclamation point just after the opening curly bracket, like so - {!Parameter[Value 1, Value 2]} This includes all the test cases in which "Parameter" is not "Value 1" or "Value 2".
Important: Please make sure that you cover all the generated rows from the Scenarios screen across the Value Restrictions, especially if you leverage multiple Automate tabs.
Automate - Multi-tab Approach
Each script tab in Automate corresponds to 1 .feature file (or 1 .csv file for transitions to certain test management tools like Xray).
An Automate tab can contain as many data-driven Scenario and/or Scenario Outline sections as needed to automate all the testing ideas and flows contained in the test model. For the avoidance of doubt, you can mix Scenario and Scenario Outline blocks on the same tab.
But sometimes putting all your script blocks on 1 tab can be overwhelming. Or perhaps one of the flows is not yet ready for execution, so you don’t want to mix it with the others for export purposes.
Figure 10 - New Automate script
You can create multiple tabs in Automate by clicking the + icon in the top ribbon (Figure 10 - 1), providing a unique name (Figure 10 - 2), selecting the type (Figure 10 - 3), and clicking “Create” (Figure 10 - 4).
When you click on any tab, you can click the pencil icon to edit the script's name, the trash icon to delete the script, or the copysheet icon to clone the entire tab. When you export into “Gherkin Feature File” or “Automated Test Framework”, you will have an option to select all scripts or only the specific ones.
Figure 11 - Automate script export
Manual
Manual (Figure 5 - 3) is a script editor in a grid format. It is aware of the parameters, values, and generated test cases of the current model. Manual makes it easy to create a .csv file with data-driven Action/Data/Expected Result sections - export will generate a script per Scenarios row, based on your Manual template structure.
Working on a Manual Script
The basic principles of your work with a Manual script are similar to those in test management tools.
Top ribbon
Figure 12 - Top ribbon functions in Manual
Work on your step content in the editor grid (Figure 12 - 1). Important - to save the whole script, you need to use the “Save” button at the top (Figure 12 - 2). I.e. “Done” from each step does not auto-save the whole tab.
Below the tabs, you can select one of the generated Scenario strengths and the specific Scenarios row to control your preview for the value replacement (Figure 12 - 3).
You can click the Usage button (Figure 12 - 4) for a refresher on the Manual Scripts functionality.
Step content
To define your steps, simply click the grey background in the row (e.g. between the words Action and Data) and provide the necessary text. The template format includes Action, Data, and Expected Result columns - this selection is not customizable, but you can fill only one of three columns for a step to be properly saved. You can phrase the content of all 3 columns however you’d like. With that said, (in most cases) you must connect Scripts to the Scenarios table via the parameterized syntax.
Figure 13 - Editing then saving the step (lower part of the image is the state after clicking “Done” in the upper part)
Simply typing < will populate a list of autocomplete suggestions related to all of the parameters in the model (Figure 13 - 1). For example, writing the step:
User selects <Color> for the car's color
means that in each generated Manual script (in preview or export), the <Color> portion of the step will be replaced with the value of the Color parameter for the corresponding test case:
User selects Deep Blue Metallic for the car's color
or
User selects Red for the car’s color
etc.
Parameterized syntax can be used in any of the 3 columns. The same parameter can be used multiple times in the same template. It is also possible to not use all the parameters in a given template.
Once your step-level edits are complete, you can click “Cancel” or “Done” in the bottom right (Figure 13 - 2).
As you insert parameters into the script, they will become greyed out in the dropdown you get from typing <. It’s a quick way to check that you haven’t accidentally omitted any important variables by the time the script is finished.
You can
- copy individual cells within the tab or across tabs (Figure 13 - 3; e.g. hover over “Action” and click the “copysheet” button that appears next to it, then paste elsewhere with Ctrl+V);
- clone the whole step (all 3 columns + conditions, if any) within the tab by clicking the “copysheet” button on the right side (Figure 13 - 4);
- delete individual steps by clicking the “trash bin” button located on the right side of each step (Figure 13 - 6);
- reorder steps by dragging them (six-dot icon to the left of each step) or clicking corresponding arrows (Figure 13 - 5);
- hover over a white space around an existing step and click “New step” to insert it directly into the desired place (Figure 13 - 7);
Note: “Relax” feature has been retired, at least for now, so to recreate its behavior you would need to manually clone the steps and change parameter syntax & references.
Manual - Reusable Step Blocks
You can save all the steps of a manual script as a block of steps that can be reused in other scripts. To do that, click the "Save for Reuse" button in the top ribbon (between "Save" and "Discard"). You are then prompted to create reusable step blocks for just the current project, or for everyone.
To use a block, you can open any Manual Scripts tab, then click "Reuse Step Block" (which appears alongside "New Step" buttons between steps). Select the desired block from the dropdown, and its steps will be inserted at that position.
You can access the library of reusable blocks by opening the model properties menu (click on the model name at the top) and selecting the menu item "Manual Reusable Step Library". In that library, you can see the list of blocks, the number of steps they contain, when they were created, when they were last used, and who created them.
You can edit a block from the library by clicking the block's name. The block will open in the model you have active, meaning that blocks are not tied to only the model they were created in. Editing an existing block affects its content for all future uses, so be mindful of uncoordinated changes that may only apply to some use cases. Existing models that have already reused the block won't be affected by a change.
If a model that originated the block is deleted, the block still remains in the library.
Manual - Conditional Logic
Oftentimes application flows may have more or fewer steps that need to be executed for specific test cases. Step Conditions are the backbone of making sure your Manual Scripts accurately reflect the Scenarios.
If a specific behavior doesn't apply to every test case, when you edit a step, you can click the “+” icon at the top, next to the word “Condition” (Figure 13 - 8). You can then specify the parameter value (or combination of them - just click “+” again) that would control whether the whole step (all 3 columns) will appear in the exported script for a given Scenarios row.
We’ll use the same example as in the Automate section - in Tesla's car configurator, if the user is configuring a new Tesla Model S, there is no option for the Interior Layout.
Figure 14 - Conditional steps in Manual Scripts
We set the condition to the <Model> parameter being not equal to “Model S” while editing the step (Figure 14 - 1). Then we click “Done”, and the color of the step will reflect its applicability to the test case number you selected for preview (Figure 12 - 3):
- Green (Figure 14 - 2) means that this step will appear in the export for the particular test case (when that test has a Model different from “Model S”).
- Red with the crossed-out condition (Figure 14 - 3) means that this step will NOT appear in the export for the particular test case (when that test has “Model S”).
Note: for the avoidance of doubt, the color of the same step will change based on the preview - the image above just collates different states together, but (Figure 14 - 2) and (Figure 14 - 3) are mutually exclusive and won’t show as duplicated step entries.
By default, the condition type is set “is”, you can change it to “is not” by clicking on that text. If you added a condition by mistake and now need to specify that the step applies to all flows, you can click the trashbin icon next to the condition (Figure 14 - 4).
If you have the same Action that can have different, value-dependent expected results, you have to create a step per unique expected result, explicitly adding all condition combinations at the step level, then hardcoding the corresponding expected result in each step (see a few more tips in the Nuances section later).
Keep in mind that you need to specify the smallest number of Values that will result in the step being applied or the expected result being triggered (i.e. if Expected Result XYZ appears for A1 + B1 + ! and A1 + B1 + #, and the third parameter only has those 2 values, then your Condition is only A1 + B1)
Important: Please make sure that you cover all the generated rows from the Scenarios screen across the Step Conditions, especially if you leverage multiple Manual tabs.
Manual - Multi-tab Approach
Each tab in Manual corresponds to 1 .csv file for transitions to certain test management tools and contains 1 script template. Each Manual script template can include as many steps as you need. Although, if you feel like the count becomes unmanageable, it can be a sign to split the script and/or the model.
But sometimes putting all your script variants on 1 tab can be overwhelming. Or perhaps one of the flows is not yet ready for execution, so you don’t want to mix it with the others for export purposes.
Figure 15 - New Manual script tab
You can create multiple tabs in Manual by clicking the + icon in the top ribbon (Figure 15 - 1), providing a unique name (Figure 15 - 2), and clicking “Create” (Figure 15 - 3).
When you click on any tab, you can click the pencil icon to edit the script's name, the trash icon to delete the script, or the copysheet icon to clone the entire tab.
You cannot save an empty tab, so if you want to reset the script after the tab has been edited, you will need to delete the tab.
When you export into a test management tool, you will have an option to select only the specific ones.
Figure 16 - Manual script export
You can now use the {scenario_name} name variable in the export of manual scripts, it represents the tab name.
Note: there is an extra option in the script selection dropdown for export options that support Manual Scripts. It’s called “Simple Parameters Script”, and it exports a step per parameter in the “Parameter Name = Parameter Value” format. It requires the Manual Scripts to be empty. If you just want to quickly move Scenarios into a test management tool for basic traceability, you can leverage this selection without actually touching Manual Scripts.
Implementation summary
The table below summarizes the broad use cases and their implementation in Manual or Automate Scripts:
| Use Case | What to do - Manual | What to do - Automate |
| I have a step that always happens and doesn’t include dynamic data | Hardcode text into the relevant columns, do not use <> syntax or conditions. | Hardcode text into the lines, do not use <> syntax or {[]} conditions. |
| I have a step that always happens but does include dynamic data (that I added to the model) | Hardcode the supporting text + use <> syntax to include Parameters in any of the columns; do not use conditions | Hardcode the supporting text + use <> syntax to include Parameters; do not use {[]} conditions |
| I have steps that sometimes happen (it may or may not include dynamic data) |
Apply the condition at the step level (the step content may or may not use <> syntax to include Parameters). Clone the step and change the Conditions to explicitly cover all unique possibilities. |
Apply the {[]} condition - can be inside the step, inside the Scenario/Outline name, or inside Background. This will handle tests where the steps do happen, based on that condition.
Important: Create more Scenario/Outline blocks to cover the tests where the steps don’t happen (i.e. you can’t have both possibilities in the same Scenario/Outline block) |
| I have a step that always happens but its Expected Result is different based on dynamic data (and I didn’t add Expected Result to the model) |
Apply the condition at the step level (the step content may or may not use <> syntax to include Parameters). Clone the step and change the Conditions to explicitly cover all unique possibilities.
See also Nuances, bullet #2. |
N/A |
| I have a step that sometimes happens and its Expected Result is different based on dynamic data (and I didn’t add Expected Result to the model) |
Apply the condition at the step level (the step content may or may not use <> syntax to include Parameters). Clone the step and change the Conditions to explicitly cover all unique possibilities.
See also Nuances, bullet #2. |
N/A |
Nuances
- To include one-off tests or a small set of negative testing scenarios or anything like that, you can intentionally avoid any parameterized syntax in the Scripts block (Manual or Automate->Scenario) and hardcode everything. That script will be disconnected from the model but will exist in a consolidated space with other scripts and will be exported.
- Keep in mind that, since Constraints support dependencies across any number of variables, you could add expected results to the Parameters screen, apply Constraints, then just pass the parameter name in the script without leveraging Conditional Logic described above. Depends on the specific rules involved and your comfort with Constraints.
- Make sure you understand the similarities and differences between Conditional steps with Expected Results in the Manual Scripts and Expected Outcomes in the Forced Interactions. There is a significant, yet subtle, difference:
- Conditional steps with Expected Results in the Manual Scripts treat the Scenarios data table as a “read-only precondition” - if the combination in the Condition never happens, the step will never be generated in export/will always be red in the preview.
- Expected Outcomes in Forced Interactions are guaranteed to happen because their conditions (forced values) “edit” the Scenarios table.
For instance, if you need to define an Expected Result that requires three or more specific values to appear in a single step (and you have generated a pairwise set of Scenarios), use the Forced Interactions feature (or a higher combination strength) to ensure the scenario is included in your suite. Then, use the Manual scripts feature to document the Expected Result for export.
If your Expected Result requires two or fewer specific values to appear in a single step (and you have generated a pairwise set of Scenarios), use the Manual scripts Condition feature without additional prep work.
- There are situations where certain testing ideas did not make it into your test model. Often these ideas represent expected outcomes or validation steps and may be necessary to complete the test script. By using the <Parameter Name> syntax in the Scenario Outline block, but intentionally not using the name of a parameter in the test model, you can include <Some Additional Idea> and Automate will add an empty column to the Gherkin Examples data table.
- Although all generated rows on the Scenarios screen are unique, it's possible to include significantly fewer parameters in a Scenario or Scenario Outline block than are present in the model, and for this subset of parameters to result in duplicate selections from the data table rows. One use case is in integration testing, where all parameters are used in system A to create e.g. a mortgage application, but then the validation in system B only needs a subset. You want your scripts to be modular enough, so you create 1 Scenario block for system A and another block for system B. If you just include those selected parameters in an Automate script for system B, all duplicate combinations will be removed and the export counts will mismatch. To force every scenario to be included in the Scenario or Scenario Outline export, you can add the @enumerate_all Gherkin tag on the line above the words Scenario or Scenario Outline.