Parallel Cucumber only outputs JUnit .xml for 1 of the scenarios in each feature file

[djangofan]

I am running parallel from a Rakefile using parallel_cucumber. What am I doing wrong if I only get 1 JUnit file for 1 of the scenarios in my .feature file?

I think both of my threads are getting the same file-name (named by feature file path: TEST-features-the_internet-basics.xml ) and one over-writes the other?

I get a process_0.log and a process_1.log file in my root folder but the JUnit .xml output files are NOT named this way, ending with the thread id.

Here is my project: https://github.com/djangofan/watir-cuke-starter
Here is my test output (with 2 .feature files containing 3 scenarios: resulting in only 2 junit .xml output files):

$ bundle exec rake test_parallel
0> Custom env: TEST=1 TEST_PROCESS_NUMBER=0; command: cucumber --format junit --out reports/win10_chrome/junit/ --format pretty features/the_internet/basics1.feature:6
1> Custom env: TEST=1 TEST_PROCESS_NUMBER=1; command: cucumber --format junit --out reports/win10_chrome/junit/ --format pretty features/the_internet/basics2.feature:6
2> Custom env: TEST=1 TEST_PROCESS_NUMBER=2; command: cucumber --format junit --out reports/win10_chrome/junit/ --format pretty features/the_internet/basics2.feature:13
0> Pid: 14744
1> Pid: 7308
2> Pid: 23348
1> # features/the_internet/the_internet.feature
1> Feature: Test The Internet
0> # features/the_internet/the_internet.feature
0> Feature: Test The Internet
2> # features/the_internet/the_internet.feature
2> Feature: Test The Internet
    Given I am on the internet                                       # features/step_definitions/the_internet_steps.rb:3
2>     When I see a heading on the page                                 # features/step_definitions/the_internet_steps.rb:7
2>     Then I see the text 'Welcome' contained in the heading           # features/step_definitions/the_internet_steps.rb:12
2>     And show me the test setup                                       # features/step_definitions/common_steps.rb:1
    Given I am on the internet                             # features/step_definitions/the_internet_steps.rb:3
0>     When I see a heading on the page                       # features/step_definitions/the_internet_steps.rb:7
0>     Then I see the text 'Welcome' contained in the heading # features/step_definitions/the_internet_steps.rb:12
0>     And show me the test setup                             # features/step_definitions/common_steps.rb:1
2>       Test Setup: Selenium::WebDriver::Remote::Capabilities, WIN10, chrome, latest
2> 1 scenario (1 passed)
2> 4 steps (4 passed)
2> 0m9.216s
2> Report output at: reports/win10_chrome
2> Exited with status 0
Thread 2 has finished. Remaining(2): 0, 1
    Given I am on the internet                             # features/step_definitions/the_internet_steps.rb:3
1>     When I see a heading on the page                       # features/step_definitions/the_internet_steps.rb:7
1>     Then I see the text 'Welcome' contained in the heading # features/step_definitions/the_internet_steps.rb:12
1>     And show me the test setup                             # features/step_definitions/common_steps.rb:1
1 scenario (1 passed)Selenium::WebDriver::Remote::Capabilities, WIN10, chrome, latest
0> 4 steps (4 passed)
0> 0m9.464s
0> Report output at: reports/win10_chrome
0> Exited with status 0
Thread 0 has finished. Remaining(1): 1
1>       Test Setup: Selenium::WebDriver::Remote::Capabilities, WIN10, chrome, latest
1> 1 scenario (1 passed)
1> 4 steps (4 passed)
1> 0m9.817s
1> Report output at: reports/win10_chrome
1> Exited with status 0
Thread 1 has finished. Remaining(0):
All threads are complete
************ FINAL SUMMARY ************
2 scenarios (2 passed)
12 steps (12 passed)
Took 0 Minutes, 16.74 Seconds

[djangofan]

If I split my single feature file into 2 separate .feature files , each with on Scenario, then I get the expected behavior of 2 Junit output files. So, it seems that parallel_cucumber will not log JUnit output if there is more than one scenario in a Feature file? Is this true or what am I misunderstanding? My Git repo is updated if you want to see proof.

[bayandin]

Hey @djangofan,

You're right JUNIT reports just overwrite each other.
You need to add to your cucumber.yaml config something like this:

<% test_batch_id = "#{ENV['TEST_BATCH_ID']}" %>

parallel_reports: >
  --format junit --out reports/junit_<%= test_batch_id %>/

Then you can run parallel cucumber with option --cucumber-options '--profile parallel_reports'.
Hope It'll help you.

[djangofan]

Thanks @bayandin I tried your suggestion and it partially works. With my output directory set at reports/win10_chrome/ , it creates a directory called reports/win10_chrome/junit_ . Here is my cucumber.yaml content:

# config/cucumber.yaml
<% report_out_dir = "#{ENV['OUT_DIR']}" %>
<% test_batch_id = "#{ENV['TEST_BATCH_ID']}" %>

parallel_reports: >
  --format html --out <%= report_out_dir %>/cukes_<%= test_batch_id %>.html
  --format junit --out <%= report_out_dir %>/junit_<%= test_batch_id %>/

This suggests to me that TEST_BATCH_ID is not getting set? What would cause that? I don't need to explicitly set it do I? If so, to what? It reads my OUT_DIR variable just fine.

[djangofan]

Ok, i was able to hook onto this variable. Is this what you really meant?
<% test_batch_id = "#{ENV['TEST_PROCESS_NUMBER']}" %>

[bayandin]

@djangofan It seems I know what's wrong. You use parallel_cucumber 0.1.22 from rubygems, but TEST_BATCH_ID was added in 0.2.x version which we haven't push to rubygems.

I've pushed 0.2.0.pre.36 version to rubygems. Internally we use this version. You can try it.
Version 0.2.x requires redis server or some redis replacement.
I'm ready to help you with setting up 0.2.x version if it's required, or you can stick with 0.1.22 and use TEST_PROCESS_NUMBER as you found it.

PS. Thanks for using paralelel_cucumber 👍

[djangofan]

I got this all working now: https://github.com/djangofan/cuke-parallel-starter

Except that I get a bunch of wierd warning output in the console that looks like:

10> Exception NameError' at /Users/me/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/cucumber-2.4.0/lib/cucumber/constantize.rb:41 - uninitialized constant Cucumber::Formatter::JsonException IO::EAGAINWaitReadable' at /Users/me/.rbenv/versions/2.1.3/lib/ruby/2.1.0/net/protocol.rb:153 - Resource temporarily unavailable - read would block