Error: pool is draining and cannot accept work

[springmeyer]

Generic pool has one throw inside its code here where it does throw new Error('pool is draining and cannot accept work').

This is unhandled in tilelive-bridge currently: we don't have a try/catch around acquire here.

We either need to handle this with a try/catch or fix our usage of generic-pool to ensure this condition is never hit.

It looks like this condition started to be hit after dd0c5ef at least in the case of Mapbox Studio Classic. What appears to be happening is that in the Studio Classic source editor a new data file is added and tile requests are made to tilelive-bridge.getTile. At the same time Studio Classic also calls update() which triggers re-creating the pool (in fact debugging indicates update is called twice when a file is saved for unknown reasons at https://github.com/mapbox/mapbox-studio-classic/blob/00c1a4e316a945d45732d8204e794168ea64720c/lib/source.js#L176-L177). The pool then gets drained and the in-flight tile requests then hit the unhandled pool is draining and cannot accept work and the whole process goes down.

TODO:

• spend more time confirming I'm understanding the above scenario right.
• ponder if this bug is related to the need to have cbaa67d

/cc @BergWerkGIS @yhahn

[springmeyer]

Good news: when I created this ticket I thought this was happening for every data format. But it actually is only happening when gpx is loaded, crashes (segfault), and triggers the app to close down. So I'm only able to replicate with gpx. What I saw before is that even after fixing the gpx crash I could still trigger the pool drain error on the gpx file. But perhaps I was seeing something wrong.

Take home: not going to make any changes to tilelive-bridge for now. Not until after the crasher (fixed in node-gdal 0.9.1) is fixed and upgraded in mapnik-omnivore - then I will revisit this. Refs mapbox/mapnik-omnivore#138

[springmeyer]

Turns out this does happen for other formats: @BergWerkGIS reports seeing this for shapefiles. So, this does likely need fixed. The stopgap of mapbox/mapbox-studio-classic@2478583 protects from studio crashing on this error and likely leads to some failed tile requests. These failed requests might leave broken tiles but can be fixed by reloading the map. But ideally we'd find a way to avoid these altogether.

[kriscarle]

Hi, In case it helps, I'm also seeing this when using a PostGIS datasource with tilelive-bridge inside https://github.com/mojodna/tilelive-cache If tilelive-cache purges a source, it is hung with the "pool is draining" error and never recovers.

I tried the stopgap branch and that didn't seem to make a difference. My workaround for now is to catch the error, recreate the source, and request the tile again.