Update the data property in API responses

[mrchrisadams]

Acceptance criteria

• API docs are updated to explain what data means (link)

It turns out that the data field in the API responses needs to be updated to reflect that data really is referring to.

The data element originally referred to whether we have any information about green providers for a given tld.

The idea was that when the green web checker started out yeeears ago, we didn't have any information about providers for some top level domains.

For example, we didn't have much about the .cn for example, nor did we know of any green providers, so saying all were grey would be a bit unfair.

Rather than say every single website with a .cn domain was grey, we would say we didn't have information, and allow clients to say:

"we don't have evidence of being a green provider, but we don't know of any providers they could switch to either"

Some API clients, like our browser extensions would show this as a question mark when data and green were both false, to reflect this uncertainty, to show that there was no data about the site, but there was no data about the entire top level domain either.

In the earlier codebase, I think that we maintained a data structure that tracked whether there were enough green hosted domains to say that a) we had some green sites, and b) that we could point to examples to.

Once we had enough examples of websites on that top level domain that were hosted by providers using green energy, we would transition to showing data to be true, in those check results, rather than false.

However, this wasn't all that well understood when migrating from the older codebase, so along the way, data ended up being interpreted as meaning there is no data we can point to, which duplicates what the green field is showing anyway.

We need to either be more explicit about what the meaning of data is document it, updating the implementation if necessary, or remove the field in API responses, as in its current state it causes more confusion than it conveys useful information.

[mrchrisadams]

@arendjantetteroo I know you worked on this years ago, and this issue is basically a summary of a discussion we had in Zulip anyway.

I can update the docs to make this clearer what it refers to, but given this is more than ten years after the initial code was written, I wonder if we can get away with just updating the docs for the old API, and avoid reimplementing the older logic for cases where: data is true and green isfalse.

If we were to implement any of this logic, I think it would make sense to add to a new, versioned API for compatibility. WDYT?

[arendjantetteroo]

@mrchrisadams
I see the following flow for getting this in a proper state:

1. add data = true to every api response, saying we know something about every tld. Even if we don't, i think this claim is fine for where the TGWF is now. This turns all those green question marks into the grey smilies for hosts that we don't have green credentials for.
2. Update our extensions to remove the data logic, so they just check green == true or green == false. I guess a versioned api would be wise as we will have extensions that won't be updated by our users (like those in safari) and we don't really want to break those (right now they will show a green question mark).
3. Once we feel enough extensions users are upgraded, remove the data field entirely.

I'm not sure 2 and 3 is worth all the effort with store updates and lagging extension users though, i'm ok with just putting data = true everywhere and making a note on this in the api documentation this field is deprecated and will be true always.

[mrchrisadams]

Thanks @arendjantetteroo , I'll go for step one here, so we can break out the rest in other dev work for later.