Caching tiles not working as expected

[guidocioni]

I have a script making thousands of plots in parallel. The main function prepares the matplotlib axis and a background (using mapbox tiles), then every worker adds data on the plot and saves the picture.

Here is a pseudo-code example

def main():
  ...
  tile_source = cimgt.MapboxTiles(
      access_token=os.environ["MAPBOX_KEY"], map_id="dark-v10", cache=True
  )
  fig, axis = plt.subplots(
      1, 1, subplot_kw=dict(projection=tile_source.crs), figsize=(12, 12)
  )
  axis.add_image(tile_source, 6, interpolation="spline36")
  ...

# Spawns N concurrent processes with plotting function 
def plotting_function():
  ...
  axis.scatter()
  ...
  plt.savefig()

It seems that, even if I'm using caching, cartopy is trying to request the same tile multiple times, making a lot of requests to the server. I just realized this because of my mapbox account billing...

Why is this not working? what am I doing wrong? can't I just fetch the tile at the beginning when setting the axis and forget about it?

[greglucas]

At a quick glance that looks OK. A few things to possibly look into that may spark some ideas for you.

• Are the images actually saving to disk? If you have run your script already with the same extent and features, I would assume that a second time there would be no requests/downloading.
• Is multiprocessing required to reproduce this? Is there potentially a continual auto-lim update going on in the ax.scatter() calls that is growing the extents of the image and thus the requested tiles are changing with each new object added to the plot.
• Can you add in a fixed limit at the beginning and a fig.draw() after adding the image so that you retrieve that image right away outside of the potential concurrent plotting functions?
• Do you want the savefig in the concurrent calls instead of after all the workers are done? Each savefig will implicitly call a draw() of the current stack rather than the typical use which would be once at the very end of the main() function.

[guidocioni]

At a quick glance that looks OK. A few things to possibly look into that may spark some ideas for you.

• Are the images actually saving to disk? If you have run your script already with the same extent and features, I would assume that a second time there would be no requests/downloading.
• Is multiprocessing required to reproduce this? Is there potentially a continual auto-lim update going on in the ax.scatter() calls that is growing the extents of the image and thus the requested tiles are changing with each new object added to the plot.
• Can you add in a fixed limit at the beginning and a fig.draw() after adding the image so that you retrieve that image right away outside of the potential concurrent plotting functions?

Hey @greglucas, I didn't spend much time debugging because after the 3rd or 4th time I was running the script it finally started using the cache...I have no idea why.
The images were saved to disk and there were no continous change of the extents going on, so I guess it was just an intermittent problem.

• Do you want the savefig in the concurrent calls instead of after all the workers are done? Each savefig will implicitly call a draw() of the current stack rather than the typical use which would be once at the very end of the main() function.

I'm intrigued by this suggestion but I'm not sure I understand what you mean. How would you be able to only savefig at the end without caching every axis/intermediate figure? I'm using mutliprocessing because there is a big difference in producing every plot serially or doing it in parallel and I have a lot of figures to save.