Concurrency possibly broken

[parak]

Hi again,

I recently started testing the latest master of this combined with latest master of test-kitchen (with some minor unrelated tweaks to both), and it seems that concurrency seems to have been broken by something. I'm trying to dig around where the problem originates, but if someone can confirm having the same issue that'd be great. Basically, it appears that concurrency doesn't actually start all kitchen containers, but a single one multiple times:

$ kitchen destroy all
-----> Starting Kitchen (v1.13.2)
-----> Destroying <ldap-ubuntu-1404>...
       Finished destroying <ldap-ubuntu-1404> (0m0.01s).
-----> Destroying <ldap-centos-68>...
       Deleting container ldap-centos-68
       Finished destroying <ldap-centos-68> (0m1.23s).
-----> Kitchen is finished. (0m1.37s)
$ kitchen converge all --concurrency=2 -l debug
-----> Starting Kitchen (v1.13.2)
-----> Creating <ldap-ubuntu-1404>...
-----> Creating <ldap-centos-68>...
D      Container ldap-centos-68 doesn't exist
D      Publish Image Name is kitchen-ldap-ubuntu-1404
D      Image Name is lxc-images:ubuntu1404-schrodinger
D      Container ldap-centos-68 doesn't exist
D      Publish Image Name is kitchen-ldap-centos-68
D      Image Name is lxc-images:centos68-schrodinger
D      Image lxc-images:centos68-schrodinger exists
D      security.privileged=true is added to Config Args, b/c its needed for mount binding
D      Config Args:  -c security.privileged=true
       Initializing container ldap-centos-68
D      run_local_command ran: lxc init lxc-images:centos68-schrodinger ldap-centos-68   -c security.privileged=true
D      Image lxc-images:ubuntu1404-schrodinger exists
D      security.privileged=true is added to Config Args, b/c its needed for mount binding
D      Config Args:  -c security.privileged=true
       Initializing container ldap-centos-68
D      run_local_command ran: lxc init lxc-images:ubuntu1404-schrodinger ldap-centos-68   -c security.privileged=true
error: The container already exists
D      Command finished: pid 13355 exit 1
D      Container ldap-centos-68 isn't running
       Starting container ldap-centos-68
D      run_local_command ran: lxc start ldap-centos-68 
error: Error calling 'lxd forkstart ldap-centos-68 /var/lib/lxd/containers /var/log/lxd/ldap-centos-68/lxc.conf': err='exit status 1'
  lxc 20161021122900.297 ERROR lxc_start - start.c:start:1439 - No such file or directory - failed to exec /sbin/init
  lxc 20161021162900.297 ERROR lxc_sync - sync.c:__sync_wait:57 - An error occurred in another process (expected sequence number 5)
  lxc 20161021162900.297 ERROR lxc_start - start.c:__lxc_start:1354 - failed to spawn 'ldap-centos-68'

Try `lxc info --show-log ldap-centos-68` for more info
D      Command finished: pid 13384 exit 1
D      Source path for the logs doesn't exist, creating /tmp/kitchen-logs/bluebird-chef/0/ldap-ubuntu-1404
D      run_local_command ran: lxc config device add ldap-centos-68 logs disk source=/tmp/kitchen-logs/bluebird-chef/0/ldap-ubuntu-1404 path=/var/log/chef-reports
D      Command finished: pid 13465 exit 0
error: Container is not running.
                                error: Container is not running.

The one created container does get provisioned, but the overall kitchen run results in a failure. Concurrency of 1 (or not specifying it) works just fine, and both instances are provisioned with no errors if ran sequentially. This also worked fine with test-kitchen 1.4.2, but I've had to upgrade in my test environment for gem compatibility reasons, so likely that's where things broke.

[bradenwright]

I did notice this as well. It was sometime around when I moved the lxd 2.0 (probably related to my gem but before I upgrade my gem to work with 2.0 concurrency worked. I'll have to spend a little time and see if I can figure it out. I never narrowed it down to version of kitchen but that makes sense.

I guess I'll spend sometime looking over their documentation for test-kitchen to see if anything changed, if you find anything I'll make sure to merge a PR quickly.

[parak]

@bradenwright I don't suppose you had a chance to poke at this any? I looked briefly a week or two ago and couldn't immediately find a culprit, but a more thorough investigation is currently on hold due to more pressing projects :(