Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collections API: RasterLinesJoin improvements #72

Open
kellyi opened this issue Sep 5, 2017 · 0 comments
Open

Collections API: RasterLinesJoin improvements #72

kellyi opened this issue Sep 5, 2017 · 0 comments

Comments

@kellyi
Copy link
Contributor

kellyi commented Sep 5, 2017

Spoke with @lossyrob a bit about how we might improve the performance of RasterLinesJoin and he pointed out two optimizations we could make:

Reducing the number of stream lines looped over per tile

Currently we loop over the whole set of MultiLines for each tile here: https://github.com/WikiWatershed/mmw-geoprocessing/blob/develop/api/src/main/scala/Geoprocessing.scala#L106

However, we could consider looping over only the subset of lines which actually intersect the tile. Depending on how many tiles are there for an AOI, this would reduce the number of times the lines loop executes since we'd only be dealing with lines with actual values.

We'd have to check whether improvements here would be offset by, presumably, looping over the lines to do the intersection operation before that.

Using Lines rather than MultiLines

Currently we do some processing on the input to transform the input stream vectors into MultiLines: https://github.com/WikiWatershed/mmw-geoprocessing/blob/develop/api/src/main/scala/Utils.scala#L120

However, apparently the MultiLines are unspooled by GT into Lines, so we could flatmap the stream vectors into a Seq[Line] and then try using something like a forEachByLineString method in the loop.

@kellyi kellyi added the BigCZ label Sep 5, 2017
@kellyi kellyi changed the title RasterLinesJoin improvements Collections API: RasterLinesJoin improvements Sep 22, 2017
@kellyi kellyi added enhancement and removed BigCZ labels Sep 22, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant