- Slaying the Hydra: Parallel Execution of Test Automation
- Slaying the Hydra: Orchestration Overview and Setting a Clean Slate
- Slaying the Hydra: Run-Time State and Splitting Up the Execution
- Slaying the Hydra: Consolidation of Information and Reporting
- Slaying the Hydra: Modifications and Next Steps
In this third post of the blog series on parallel test execution, I explain how to execute distributed parallel test automation. The previous entry can be found here.
As discussed previously, The running stage (see below) within the pipeline context is set to execute three builds of the test_runner freestyle job in parallel. Each build is receiving the following parameters:
- browser – either equal to ‘ie’ or ‘chrome’
- total_number_of_builds – equal to ‘3’
- build_number – equal to ‘1’, ‘2’ or ‘3’

Freestyle Job Overview
In the following sections, I explain what freestyle components need utilized when constructing the test_runner job in Jenkins.
Parameters
As seen from the image above, parameters are being passed from the pipeline job into the freestyle job. We will update the freestyle job to be parameterized. This selection is made when configuring the Jenkins job (see below).

Next the freestyle job is configured with these parameter names:
- browser – the value received from the pipeline parameter value.
- total_number_of_builds – the value received from the pipeline parameter value.
- build_number – the value received from the pipeline parameter value.
- workspace_location – to show a different way of doing things, we can see from the image above that I did not pass a value for workspace location in the pipeline. When I configured the parameter (below), I set a default value in the freestyle job. This default value will be linked to the workspace_location parameter now unless I otherwise specify.

Node Selection
In this section we restrict where this build can execute to only machines associated with the @local tag only. This setting is located in the Manage Jenkins > Manage Nodes section of Jenkins. It provides us the ability to ensure we are not utilizing nodes that are otherwise utilized or not configured to run the cucumber tests in the steps below.

Version Control
In the Source Code Management section, we specify what testing suite to retrieve via version control and utilize for this effort, which will pull the suite down within the workspace. The “clean before checkout” additional behavior (Jenkins functionality) will remove any files in the workspace that are not in the Git repo before pulling the suite down. This allows for a clean slate for every execution.

Splitting Code
class Splitter
def total_builds
ENV['total_number_of_builds'].to_i
end
def build_number
ENV['build_number'].to_i
end
def main_run
scenarios = feature_iterator
splits = job_splitter(scenarios)
assignment = job_assigner(splits)
feature_mod_iterator(assignment, 'features', true)
end
def feature_mod_iterator(split_assignment, current_location = 'features', assign = true)
array = []
split_assignment.each do |value|
mod_value = value.gsub('@regression', '@split_builds')
regex = /#{value}$/
files = return_all_files(current_location, '*', 'feature')
files.each do |file|
output = File.open(file, 'r', &:read)
modified = output.gsub(regex, mod_value)
if assign
File.open(file, 'w+') { |f| f.print(modified) }
else
array.push(modified)
end
end
end
array
end
def feature_iterator(current_location = 'features')
files = return_all_files(current_location, '*', 'feature')
array = []
files.each do |file|
array.push(return_all_gherkin_scenarios(file))
end
array.flatten
end
def return_all_gherkin_scenarios(file)
output = File.open(file, 'r', &:read)
output.scan(/(@regression.*\n. (Scenario:|Scenario Outline:)?.*)/).map { |value| value[0] }
end
def return_all_files(current_location, filter = '*', file_type = '*')
Dir.glob("#{current_location}/**/#{filter}.#{file_type}")
end
def job_splitter(scenarios)
split = scenarios.length.to_i / total_builds.to_i
container = []
total_builds.times { container.push([]) }
mod_scenarios = scenarios.clone
total_builds.times do |index|
container[index].push(mod_scenarios[0..(split - 1)])
container[index].flatten!
(0..(split - 1)).to_a.length.times do
mod_scenarios.delete_at(0)
end
end
mod_scenarios.each_with_index do |value, index|
container[index].push(value)
end
container
end
def job_assigner(scenarios)
scenarios[(build_number.to_i - 1)]
end
end
one = Splitter.new
one.main_run
At a high level, the code block above is creating an array of arrays that split up the regression tests evenly between the number of executors. The build_number value is utilized to access the corresponding index value of the array. All of the tests in that location are re-tagged from @regression to @split_builds locally on the workspace that houses the Ruby/Cucumber code pulled down from version control.
You would have to change the @regression tag to whatever you are utilizing to tag your tests as regression on your team.
The cool thing is that this will run on each of the three workspaces and re-tag a unique subset of tests. Because the total_builds value is the same for all the jobs kicked off, it will create the same nested array structure on every workspace. The difference between workspaces comes about because of the build_number parameter that chooses which subset of tests to re-tag.
Running the Split Code
We should house the code above within our testing framework in version control. Within the Build section of Jenkins we then create a windows batch command. Next we set the environment variables that the code utilizes total_builds and build_number as being equal to the parameters set within the freestyle job. We can now run the ruby command passing the path to the .rb file that houses the code within the workspace (in reference to the code above).

Running the Tests
We set up another windows batch command to set environment variables for browser and or_tags, and in this instance, we kick off the tests utilizing a rake task. Cucumber Rake is a useful tool, but we could just as easily run a Cucumber command.
The important thing is that we are passing what will be the tag modified locally on each workspace(split_builds) to run only the tests assigned to that workspace. Additionally, we passed the browser variable set within the pipeline and passed to the freestyle job.

Storing Results
In our last batch command, we are extracting the json test results file and storing it on the workspace_location as a json file named with the build_number value (either 1, 2, or 3). This workspace location is the same as what we utilized in the clearing stage and what will be utilized in the consolidation stage.

Review and Next Steps
To review, in this post, we figured out how to build the freestyle job that is responsible for splitting, executing, and storing the results of our tests.
In the next post, we discuss how to consolidate the information from the freestyle job builds into a concise cucumber report.