azapi_resource_list can return incomplete results

[lewis-jackson-bots]

We encountered an issue today with this data block:

data "azapi_resource_list" "subscription__postgresql_flexible_servers" {
  type      = "Microsoft.DBforPostgreSQL/flexibleServers@2022-12-01"
  parent_id = data.azurerm_subscription.current.id

  response_export_values = ["*"]
}

Where it returned all of our east US DBs and no UK south DBs. It looks like the cause of this is high latency in the Azure API.

This behaviour is undesirable as we'd want the data source to return an error if no UK DBs were found.

[ms-henglu]

Hi @lewis-jackson-bots ,

Thank you for taking time to report this issue.

Yes, it could be the latency in the Azure API caused this issue. A workaround is adding a time_sleep resource, hope this could help.

https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/sleep

[lewis-jackson-bots]

@ms-henglu do you mean a read timeout?

If not, can you give an example of time_sleep working around this?

[ms-henglu]

Hi @lewis-jackson-bots ,

I'm not sure if I understand the scenario correctly, if it's about getting the list of flexible servers after creating some and it returns incomplete results, please refer to below example:

resource "azapi_resource" "flexibleServer" {
  type      = "Microsoft.DBforMySQL/flexibleServers@2021-05-01"
  parent_id = azapi_resource.resourceGroup.id
  name      = "henglu624"
  location   = azapi_resource.resourceGroup.location
  body = {
   ...
  }
}

resource "time_sleep" "wait_1_min" {
  create_duration = "1m"
  depends_on = [azapi_resource.flexibleServer]
}

data "azapi_resource_list" "subscription__postgresql_flexible_servers" {
  type      = "Microsoft.DBforPostgreSQL/flexibleServers@2022-12-01"
  parent_id = data.azurerm_subscription.current.id

  response_export_values = ["*"]
  depends_on = [ time_sleep.wait_1_min ]
}

[lewis-jackson-bots]

I'm not sure if I understand the scenario correctly, if it's about getting the list of flexible servers after creating some and it returns incomplete results

Apologies for being unclear, but it's not a scenario like that.

The servers have existed for years and we're using this data block to pull a list of them down to monitor in DataDog. A plan recently showed only half of these servers (the East US replicas) and none of the UK primary servers, if we'd applied this we would have ended up not monitoring our primary servers.

When I tried to open the server list in the Azure portal it was also very slow there, so my hunch was that the Azure API was experiencing high latency / errors. In this scenario I would expect the data block to error out clearly in some way rather than silently returning a bad result.

[stemaMSFT]

@lewis-jackson-bots you can specify a reads time out block per the Terraform registry docs.
Just customize this limit and you should be good to go.

[lewis-jackson-bots]

I think the read timeout would work as mentioned above. But I still think that this is a bug where the provider has timed out and chosen to return partial results rather than giving feedback as an error.

What if I had a known slow query that I was happy to wait for as long as the full results were correct at the end?

[ms-henglu]

Thanks @lewis-jackson-bots , I'll check and improve the behavior that it should return an error in this case.