Request frequency specifications not enforced

[3kt]

According to the documentation:

Both the fullHashes.find response and threatListUpdates.fetch response have a minimumWaitDuration field that clients must obey.

And

Automatic back-off applies to both the fullHashes.find response and threatListUpdates.fetch response.

This issue was first raised in #33, and marked as resolved in #49, however this only addresses the database update, and not fullHashes.find.

On a different note, enforcing minimumWaitDuration seems reasonable for both methods, but would back-off mode make sense for fullHashes.find? eg. do we really want to prevent any further hash confirmation for at least 15 minutes for a single 4xx or 5xx response?

[colonelxc]

The primary use case of the API is for client devices (phones, computers) that are each maintaining their local lists and checking them as someone browses the web. For those 3 billion devices, backing off for 15 minutes (even for fullHashes.find) or more will hopefully not have much negative impact. Most people wont visit a malicious site every hour, or every day. So yes, there is temporarily degraded coverage for users who get 500s.

A programming bug can easily turn millions/billions of clients into a large DDoS. Shedding even a small amount of the traffic (with a 5xx) can greatly relieve the server pressure by making these clients back off. This will happen naturally as our servers reach their limit.

Compared to the huge number of browser/mobile clients, the traffic from someone signing up for the API and using this client is relatively small. So there's just been lower priority in making this perfectly compliant. The simpler integration into the update requests has a similar effect, as a stale DB returns errors on lookups.

But I do agree this should be a representative client, as an example for other implementations.