Best place to intercept tile rendering when there are no feature vertices within tile

[alpha-beta-soup]

I think I have an unusual use-case for QTiles, so I want to fork it and insert my own logic to speed up the rendering; I just don't know the best place to intercept the rendering process. This isn't either a feature request or a bug report.

I am generating a set of "mask" tiles for land and sea for my own tile server. Ultimately, a mask tile should be transparent over land and opaque over sea, and the alpha channel of the mask tile is then inserted into a tile rendered by my server.

I can prepare my mask tiles with QTiles relatively easily, but it is (understandably) slow to render down to zoom level 18. My own tile server is also a little slow to do the alpha substitution. To speed it up, I can avoid opening mask tiles that are either completely land (100% transparent) or completely oceanic (100% opaque). I do this with an additional pre-processing step: all mask tiles in the cache are inspected, and those that are completely land or completely ocean are substituted with files of 0 or 1 bytes respectively, and the remaining fraction of tiles are left untouched. Then, I can determine whether a tile is land, ocean, or both (i.e. coast) just by checking the file size with os.stat. I only need to do a complex alpha substitution for coastal tiles. This speeds up the average tile rendering time on my server considerably, and also massively reduces the size of the cache (I anticipate only 1-3 gigabytes even for 18 levels of zoom).

My remaining bottleneck then is in pre-rendering the mask tiles. Tiles that are completely opaque or transparent are being rendered by QTiles even though I'm just going to get rid of them in my pre-processing step. Can I speed up the process by skipping the rendering if there are no features detected within the bounds of a tile (ocean)? Can I also skip the rendering when there is a feautre, but there are no vertices (land)? That way, the only tiles that get rendered are those with feature vertices (i.e. those that contain coastlines). I expect this will make the production of mask tiles much faster, given only a small fraction of tiles intersect my coastlines, while at least 70% are oceanic, and most of the remainder are inland. Where is the best place to insert this short-circuiting logic within the QTiles application?

[F-JJTH]

Hi,
it looks like we are looking for the same feature: do not render a set of tiles within the selected extent.
I implemented something that I submitted in a pull request. In my case I want to render only tiles within my rasters extent. However in your case, you want to render only coastline from raster containing ocean+land.
I think you should look at a better solution: firstly instead of using a raster you would export coastline from OSM database. That way you can transform the resulting "coastline.osm" into any vector format readable by QGis.
Once it's done, you have to change my pull request to detect intersection with your vector features (lines) then QTiles will render only tiles who contains coastline.
I hope this hint will help you.
Regards

[alpha-beta-soup]

Nice, that's very helpful. I'm already using vector polygons for land. I'll adapt your fork to use intersection of the tile bounds with the vector features, rather than the extent of the layer (as my layer is global). That way I can at least avoid producing transparent oceanic tiles.

[F-JJTH]

Great, when you have implemented your patch I would be happy if you share it with the community. Myself I can be interested in a "render only tiles where polyline feature lives" in more or less future ;-)

[alpha-beta-soup]

I assumed installing a plugin from source would be easier than it seems to be. I've cloned the repo, ran make successifully after installing PyQT4, and then made a symlink within ~/.qgis2/python/plugins. However within the QGIS 2.8 plugins menu I can't seem to find reference to my local QTiles (only the experimental public version). What am I missing? It's been a while time since I played around with a QGIS plugin, I don't remember having problems getting set up.

[simgislab]

so you have two QTiles with the same names, toolbars etc? Do you really want that? Why don't you just leave one for development.

PS: PR by @F-JJTH that partly addresses your question was merged and new QTiles released yesterday. Generally I think that adding more logic for conditional tile generation based on feature presence won't hurt mainstream QTiles as well.

[alpha-beta-soup]

No, the only QTiles I can find when I attempt to add one to the QGIS interface is not my local version. How do I activate my development version within the QGIS interface?

My plan is to add something like @F-JJTH's PR allowing the specification of a vector feature: only tiles that intersect with this feature should then be generated. Separately I'd also like to add a control for skipping existing tiles, so I can interrupt tile generation.

[alpha-beta-soup]

Apologies, I have worked it out now. I'd forgotten about the plugin reloader.

[alpha-beta-soup]

Also just noticed #15, although I didn't think about working with a selection of features of a layer; only the entire layer.

[alpha-beta-soup]

Implemented this in #59 (successful for my purposes) so I'll close this question.