The issue with throttling

[Marakai]

As some people here, but also in comments on various Youtube have reported, it appears that sending too many requests to the Google Maps servers from a single IP (i.e. yours while you fly) results in queries failing, probably due to throttling. This seems to be worse with faster Internet connections, though that's just anecdotally.

The issue is not helped by the fact that hostnames like mt.google.com are really just aliases that point to load balancers which do not round-robin from one actual map server to the next. Instead my tests show "stickiness" to one IP: a series of lookups to the published maps servers ended up with the same IP for different hostnames!

For example, running a query on mt.google.com, then mt1.google.com, then mt2, then mt3 (that's the highest number) gave me the same IP address! When you look closely that makes sense: you will see that those are all just DNS aliases for mt.l.google.com. If you keep doing lookups on that server you will continue to get the same IP until you pause for a while, then you finally get a different one.

Not sure what the work-around for this could be right now. It's interesting that Bing Maps don't suffer from this issues, but then maybe they whitelist rapid connection/downloads requests knowing somehow that they come from FS2020! Pure speculation on my part.

One random idea would be to crowdsource a list of IP addresses that people receive for the various hostname lookups and then maintain them in a round-robin table within the app. Provided the map lookups work when going by IP instead of hostname (not a given), regular updates would keep that list current.

I've submitted a PR that has some stubs for round-robin in it: the google servers are kept in a separate source file and the map lookups loop through that list. Still with the hostnames, which we know is pointless right now.

The code looks something like

In google_servers.py I put

GOOGLE_SERVERS = [
    'mt.google.com',
    "khm.google.com",
    "khms.google.com",
    # keep adding IP addresses here...
]

Then in the flask_server.py (formerly server.py) I put:

from google_servers import GOOGLE_SERVERS
_google_server_loop = itertools.cycle(GOOGLE_SERVERS)

and further down the data retrieval becomes

next_server = next(_google_server_loop)
url = url_mapping(next_server, tile_x, tile_y, level_of_detail)