Implement Caching Layer

[JosiahBull]

Description

Generating .wav files is extremely resource intensive. We need to cache commonly requested .wav files which should vastly improve performance.
The general requirements is that this stores files, limited by either number of files or total filesize. It must be able to be stopped/started easily, and be completely asynchronous.

Tasks

• Initial design of cache file, and figure out the backing data structures.
• Decide whether the cache is going to be a separate service, or be built into the app as a monolithic.
• Integration with Rocket, ideally through a simple request guard.
• Thread safe! Need to ensure we have implementation of send + sync.
• Able to handle large volume of requests and items with minimal overhead.
  • Benchmarks
  • Design to handle ~30k items. Heap allocated elements with Box is going to be a good idea.
  • Test throughput with 25 sources sending reqs, 10 getting cached elements, 15 generating unique elements.
• Generating new .wav files should be pushed to a separate thread (or a separate service entirely).
• Completely asynchronous design.
  • Handle serving cached .wav files while generating new ones for other reuqests.
  • Generate multiple .wav files at once with a configurable thread pool.
  • Queue generation requests and send them out as workers come free.