Inconsistent Dome9 API behavior on AWS security group creation

[DanielRis]

When creating security groups using the dome9_aws_security_group resource, the underlaying Dome9 API shows inconsistent behavior.

Out of 100 security groups created via

locals {
    test_count = 100
}

resource "dome9_aws_security_group" "test" {
  count = local.test_count

  dome9_security_group_name = "test-securitygroup-${count.index}"
  description               = "Test description for ${count.index}"
  aws_region_id             = "us_east_1"
  dome9_cloud_account_id    = module.gatherer.dome9_cloud_account_id
  vpc_id = module.gatherer.vpc_id
  tags = {
    tag-key = "TAG-VALUE"
  }
}

10%-40% fail with the following API response:

"message": "All actions on inbound services of group <security-group-name> succeeded. The following outbound services of group <security-group-name> failed to add: . The following outbound services of group <security-group-name> failed to remove: Dome9.Web.Api.Models.CloudSecurityGroupServiceViewModel: The security group 'sg-<aws-id>' does not exist The following tags of group <security-group-name> failed to update: tag-key: The security group 'sg-<aws-id>' does not exist. The following tags of group <security-group-name> failed to remove: "

The error looks like there is a timing issue and some missing waits/retries on the Dome9 server side. It does not seem to wait long enough to apply tags/rules after the security group has been created.

The security group is created in AWS. If the creation was partial successful, should the API not respond with information of the security group or delete the partial created security group?

---

When the same terraform code is executed again, it tries to create the missing 13 security groups (they are not in the statefile because of the error response) and fails with a HTTP/1.1 500 Internal Server Error An error has occured. because a security group with the same name already exists in Dome9.

The only solution is then to manually delete the security group through the AWS or the Dome9 console.

[itzikYeret]

Hi @DanielRis
The team had deployed a fix for this issue, can you check please it is working now?

[dolevh-d9]

@DanielRis any update?
Does the issue has been solved?

[DanielRis]

@itzikYeret @dolevh-d9 sorry, it took me a while go get back to this. When running the same test code again, I can see better behavior (~97-99% of security groups are properly created) but I can still see error responses from the API like:

Error: FAILED: POST, https://api.dome9.com/v2/CloudSecurityGroup, 400, 400 Bad Request, {"message":"All actions on inbound services of group test-securitygroup-73 succeeded. The following outbound services of group test-securitygroup-73 failed to add: . The following outbound services of group test-securitygroup-73 failed to remove: Dome9.Web.Api.Models.CloudSecurityGroupServiceViewModel: The security group 'sg-0e84e7900aebf5657' does not exist All actions on tags of group test-securitygroup-73 succeeded."}