Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace LimitedSizeDict with cachetools #40

Merged
merged 1 commit into from
May 24, 2024

Conversation

Dunedan
Copy link
Collaborator

@Dunedan Dunedan commented May 16, 2024

When receiving game reports from Pyrogenesis, it regularly happens that we never receive a report from both players of a rated game. This happens in the following situations:

  1. One of the player rage-quits by force-closing Pyrogenesis.
  2. One player looses lobby connectivity during the game.
  3. The rated game has only one human player and the other player is a bot (while Pyrogenesis shouldn't send reports for such games, here we are right now).

As we cache game reports in memory until we received and compared them from both players of a rated game, this means that the LimitedSizeDict used for that would reach its maximum size of 4096 items pretty fast. While that doesn't impact functionality, it's using up way more memory than necessary.

Additionally EcheLOn contained a bug which wouldn't even remove games from the LimitedSizeDict, where we did receive all reports for.

While we can (and actually do in this commit) fix the bug in EcheLOn, we have no control over the situations where we don't receive reports from both players. However, what we can do is to remove reports for games where we're pretty certain, that we'll never receive the missing report anymore.

To do so, this commit replaces the custom LimitedSizeDict class with a dependency on cachetools. For tracking reports from rated games a size-limited TTLCache is now used which evicts reports belonging to games after 1 hour, as we don't expect to receive reports later than a few seconds after the end of a rated game.

This should result in a significant reduction of the memory required by EcheLOn.

When receiving game reports from Pyrogenesis, it regularly happens that
we never receive a report from both players of a rated game. This
happens in the following situations:

1. One of the player rage-quits by force-closing Pyrogenesis.
2. One player looses lobby connectivity during the game.
3. The rated game has only one human player and the other player is a
   bot (while Pyrogenesis shouldn't send reports for such games, here we
   are right now).

As we cache game reports in memory until we received and compared them
from both players of a rated game, this means that the `LimitedSizeDict`
used for that would reach its maximum size of 4096 items pretty fast.
While that doesn't impact functionality, it's using up way more memory
than necessary.

Additionally EcheLOn contained a bug which wouldn't even remove games
from the `LimitedSizeDict`, where we did receive all reports for.

While we can (and actually do in this commit) fix the bug in EcheLOn, we
have no control over the situations where we don't receive reports from
both players. However, what we can do is to remove reports for games
where we're pretty certain, that we'll never receive the missing report
anymore.

To do so, this commit replaces the custom `LimitedSizeDict` class with a
dependency on `cachetools`. For tracking reports from rated games a
size-limited `TTLCache` is now used which evicts reports belonging to
games after 1 hour, as we don't expect to receive reports later than a
few seconds after the end of a rated game.

This should result in a significant reduction of the memory required by
EcheLOn.
@Dunedan Dunedan merged commit 30c459f into 0ad:master May 24, 2024
3 checks passed
@Dunedan Dunedan deleted the use-cachetools branch May 24, 2024 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant