Skip to content

Commit

Permalink
store/proxy: properly check if context has ended (#1082)
Browse files Browse the repository at this point in the history
How the code was before it could happen that we might receive some
series from the stream however by the time we'd send them back to the
reader, it would not read it anymore since the deadline would have been
exceeded.

Properly use a `select` here to get out of the goroutine if the deadline
has been exceeded.

Might potentially fix a problem where we see one goroutine hanging
constantly (and thus blocking from work being done):

```
goroutine profile: total 126
25 @ 0x42f62f 0x40502b 0x405001 0x404de5 0xe7435b 0x45cc41
	0xe7435a	github.com/improbable-eng/thanos/pkg/store.startStreamSeriesSet.func1+0x18a	/go/src/github.com/improbable-eng/thanos/pkg/store/proxy.go:318
```
  • Loading branch information
Giedrius Statkevičius authored and bwplotka committed Apr 26, 2019
1 parent 88544fc commit a4213b5
Showing 1 changed file with 8 additions and 5 deletions.
13 changes: 8 additions & 5 deletions pkg/store/proxy.go
Original file line number Diff line number Diff line change
Expand Up @@ -294,10 +294,6 @@ func startStreamSeriesSet(
return
}

if ctx.Err() != nil {
return
}

if err != nil {
wrapErr := errors.Wrapf(err, "receive series from %s", s.name)
if partialResponse {
Expand All @@ -315,7 +311,14 @@ func startStreamSeriesSet(
s.warnCh.send(storepb.NewWarnSeriesResponse(errors.New(w)))
continue
}
s.recvCh <- r.GetSeries()

select {
case s.recvCh <- r.GetSeries():
continue
case <-ctx.Done():
return
}

}
}()
return s
Expand Down

0 comments on commit a4213b5

Please sign in to comment.