Tilesmith: A GitHub Repository Pattern for Cloud-Native Geospatial Data Management

Introduction

Tilesmith is a repository pattern designed to enable efficient handling, transformation, and sharing of large geospatial datasets in cloud-native formats. By leveraging tools such as GitHub Pages for web hosting and Source Cooperative for cloud storage, Tilesmith provides a streamlined and reproducible workflow for managing geospatial information. Its focus is on simplicity, scalability, and sustainability, catering to the growing demand for open and accessible geospatial data solutions.

Core Principles

1. Cloud-Native Focus

Tilesmith assumes the use of cloud-native file formats like PMTiles, which are optimized for modern web-based applications. By offloading storage to Source Cooperative or similar services, Tilesmith minimizes dependencies on dedicated servers while maintaining scalability and accessibility.

2. GitHub as a Development Hub

The repository structure is designed to take full advantage of GitHub’s capabilities, including version control, collaboration, and GitHub Pages for hosting sample web applications. This reduces complexity and ensures the pattern remains accessible to a wide range of users.

3. Reproducibility and Automation

All processes — from data conversion to deployment — are defined in a Makefile, ensuring that tasks are automated, reproducible, and easy to execute. This fosters consistency across teams and projects.

4. Lightweight Repository Management

Large input and output files are excluded from GitHub using .gitignore, ensuring that the repository remains lightweight and focused on reproducible workflows. Data is stored externally in cloud services, streamlining repository maintenance.

Workflow Overview

1. Data Preparation

   • Input datasets, such as GeoPackage files, are placed in a src/ directory.
   • These datasets are too large for GitHub and are excluded via .gitignore.

2. Data Conversion

   • Using tools like felt/tippecanoe, the data is converted into PMTiles format.
   • The output is stored in a dst/ directory, ready for cloud hosting.

3. Demo Site Hosting

   • A lightweight web viewer built with MapLibre GL JS and Vite is hosted in the docs/ directory.
   • This viewer demonstrates the PMTiles dataset in action, providing an interactive interface for users.

4. Deployment

   • PMTiles files and a README.md file are uploaded to Source Cooperative using AWS CLI or similar tools.
   • The web viewer is deployed to GitHub Pages, offering a publicly accessible interface.

Repository Structure

.
├── .gitignore      # Excludes large data files
├── README.md       # Repository documentation
├── Makefile        # Automation of data conversion and deployment
├── src/            # Input data (excluded from GitHub)
├── dst/            # Output data (excluded from GitHub)
├── docs/           # Web application hosted on GitHub Pages
│   ├── index.html
│   ├── main.js
│   └── vite.config.js

Key Benefits

1. Scalability

   • By focusing on cloud-native formats, Tilesmith supports massive datasets without requiring significant local infrastructure.

2. Flexibility

   • The pattern is adaptable to various geospatial projects, from web mapping to data sharing.

3. Ease of Use

   • Automation through Makefile and the use of known tools like GitHub and AWS make the workflow accessible to a broad audience.

4. Open and Transparent

   • By leveraging open standards and open-source tools, Tilesmith aligns with modern principles of transparency and collaboration in geospatial data management.

Potential Use Cases

1. Geospatial Data Sharing

   • Organizations can use Tilesmith to share interactive datasets with stakeholders, combining the power of PMTiles and GitHub Pages.

2. Capacity Building

   • Tilesmith provides a reproducible and documented workflow, making it ideal for teaching and capacity-building initiatives in geospatial data management.

3. Disaster Response

   • Rapidly convert and share geospatial data in cloud-native formats to support disaster response efforts, ensuring scalability and accessibility.

Conclusion

Tilesmith represents a modern approach to geospatial data management, combining the best of cloud-native technology, automation, and open collaboration. By simplifying the handling and sharing of large datasets, Tilesmith empowers users to create, share, and utilize geospatial data effectively and sustainably. It is a practical and forward-looking solution for anyone working at the intersection of technology and geospatial information.