Skip to content

Addition of Amazon S3 tile store module #122

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 30 commits into from

Conversation

estadtherr
Copy link
Contributor

I added support to mod_tile for using an Amazon S3 bucket for metatile storage, using the libs3 library to implement the S3 API communication. This allows a cluster of EC2 tile servers to share a tile store. The changes include:

  • new store_s3 module with all storage methods implemented. configuration of S3 storage is done using a connection string formatted as follows:

    s3://<S3 access key ID>:<S3 secret key>[@<S3 host>]/<bucket name>[/<base path>]

  • autoconf support for detecting presence of the libs3 library

  • S3 unit tests in the gen_tile_test program

If autoconf detects the libs3 library, the S3-related tests will be included in gen_tile_test. These tests require environment variables to be set containing the S3 access credentials: S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEY, and S3_BUCKET_NAME. If libs3 support is compiled in, and these environment variables are not defined, gen_tile_test will have failed assertions on that section of the test. I'm certainly open to other options for behavior, but this was simplest.

As a possibly optional change, I changed the parameters to the daemon() call to allow the working directory and stdout/stderr handles of renderd to be specified in the systemd unit specification (which is launching renderd in our setup). This is certainly not directly required for libs3 support, but should cause no change in behavior in the vast majority of instances. I'm more than willing to drop this change from the PR if you're concerned about the impacts.

Eric Stadtherr added 29 commits January 19, 2016 11:00
@sprutner
Copy link

+1

@pantierra
Copy link
Collaborator

Superseded by #143.

@pantierra pantierra closed this Feb 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants