Skip to content

Commit

Permalink
(PE-38421) avoid stack overflow in event collection
Browse files Browse the repository at this point in the history
See https://stuartsierra.com/2015/04/26/clojure-donts-concat

As was seen in the field, the event concatenation can cause a stack
overflow for a large number of events.

This modifies the loop for retrieving events to use `into` instead
of `concat` which prevents the stack overflow by collecting the results
into a new vector with every iteration.
  • Loading branch information
jonathannewman committed May 22, 2024
1 parent e1a5d74 commit f28e2b1
Showing 1 changed file with 6 additions and 5 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -119,14 +119,15 @@
(map #(clojurize % (.watchable watch-key)) events)))

(schema/defn retrieve-events :- [Event]
"Blocks until an event the watcher is concerned with has occured. Will then
"Blocks until an event the watcher is concerned with has occurred. Will then
poll for a new event, waiting at least `window-min` for a new event to
occur. Will continue polling for as long as there are new events that occur
within `window-min`, or the `window-max` time limit has been exceeded."
[watcher :- (schema/protocol Watcher)]
(let [^WatchService watch-service (:watch-service watcher)
watch-key (.take watch-service)
initial-events (watch-key->events watch-key)
;; use `vec` to ensure the events sequence is correctly concatenated below
initial-events (vec (watch-key->events watch-key))
time-limit (+ (System/currentTimeMillis) window-max)
recursive @(:recursive watcher)]
(when recursive
Expand All @@ -140,14 +141,14 @@
(watch-new-directories! waiting-events watcher))
(.reset waiting-key)
(if (< (System/currentTimeMillis) time-limit)
(recur (concat events waiting-events))
(concat events waiting-events)))
(recur (into events waiting-events))
(into events waiting-events)))
events))
initial-events)))


(schema/defn process-events!
"Process for side-effects any events that occured for watcher's watch-key"
"Process for side effects any events that occurred for watcher's watch-key"
[watcher :- (schema/protocol Watcher)
events :- [Event]]
(let [callbacks @(:callbacks watcher)
Expand Down

0 comments on commit f28e2b1

Please sign in to comment.