Skip to content

Commit

Permalink
Add indexed recipe, tidy no-longer-used modules
Browse files Browse the repository at this point in the history
  • Loading branch information
ajcr committed Jun 10, 2023
1 parent d1a41ce commit bffbb87
Show file tree
Hide file tree
Showing 8 changed files with 55 additions and 310 deletions.
2 changes: 1 addition & 1 deletion doc/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
## [0.5.0]
### Added
- New `rolling.ApplyPairwise` object.
- Add `"indexed"` window type to enable operations on windows with an index.
- Add `"indexed"` window type to enable operations on windows over `(index, value)` pairs (initial implementation work by @daviddavo)

## [0.4.0] - 2023-03-11
### Added
Expand Down
47 changes: 47 additions & 0 deletions doc/recipes.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,3 +124,50 @@ for ip_addresses, count in rolling.Mode(all_ip_addresses, WINDOW_SIZE, return_co
)
```
(This is obviously just a toy example. There are much better ways to do this on actual web servers.)

## Minute Sum

The charity appeal has been broadcast and donations are flooding in. How long does it take for the sum of donations made over the previous 60 seconds to exceed 888?

### Setup

The stream is made up of `(datetime, donation)` pairs:

```python
import itertools
import random
from datetime import datetime, timedelta

def donation_stream(current_time: datetime):
while True:
donation = random.choice([5, 10, 20, 50, 100])
# increment donation time by a random number of seconds
current_time += timedelta(seconds=random.randint(0, 10))
yield current_time, donation
```

### Solution

Use an iterator over the indexed values with a window size of 60 seconds:

```python
import rolling

TARGET = 888

start_time = datetime.now()

sixty_second_window_sums = rolling.Sum(
donation_stream(start_time),
window_size=timedelta(seconds=60),
window_type="indexed",
)

for window_sum in sixty_second_window_sums:
if window_sum > TARGET:
print(
f"It took {sixty_second_window_sums.index_buffer[-1] - start_time} "
f"to exceed {TARGET} in a 60 second window."
)
break
```
61 changes: 0 additions & 61 deletions rolling/apply_indexed.py

This file was deleted.

72 changes: 0 additions & 72 deletions rolling/arithmetic/nunique_indexed.py

This file was deleted.

14 changes: 7 additions & 7 deletions rolling/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,10 +44,10 @@ def __init__(self, iterable, window_size, window_type="fixed", **kwargs):
self._init_variable(iterable, window_size, **kwargs)

elif window_type == "indexed":
# keep track of all indexes that we encounter. Assumes that all
# Keep track of all indexes that we encounter. Assumes that all
# values we encounter will be stored in the same order. If not,
# the subtype will need to implement its own _next_indexed() method.
self._index_buffer = deque()
self.index_buffer = deque()
self._init_indexed(iterable, window_size, **kwargs)

else:
Expand Down Expand Up @@ -86,20 +86,20 @@ def _next_variable(self):
def _next_indexed(self):
new_index, new_value = next(self._iterator)

if self._index_buffer and new_index < self._index_buffer[-1]:
if self.index_buffer and new_index < self.index_buffer[-1]:
raise ValueError(
"Next index must be greater than or equal to last added index: "
f"{new_index} < {self._index_buffer[0]}"
f"{new_index} < {self.index_buffer[0]}"
)

self._index_buffer.append(new_index)
self.index_buffer.append(new_index)
self._add_new(new_value)

min_index = new_index - self.window_size

while self._index_buffer and self._index_buffer[0] <= min_index:
while self.index_buffer and self.index_buffer[0] <= min_index:
self._remove_old()
self._index_buffer.popleft()
self.index_buffer.popleft()

return self.current_value

Expand Down
90 changes: 0 additions & 90 deletions rolling/base_indexed.py

This file was deleted.

44 changes: 0 additions & 44 deletions tests/test_apply_indexed.py

This file was deleted.

Loading

0 comments on commit bffbb87

Please sign in to comment.