MapProxy
The caching proxy for web map services
Oliver Tonnhofer <
[email protected]>
From Oldenburg in Germany We focus on:
OpenSource WebGIS, Server development OpenStreetMap WMS MapProxy development and support
Content
What is it? About the project Examples News
What is it?
Clients
MapProxy
Server
Cache
Improve WMS Performance Tiled access
Change
Protocols/versions Image formats Projection
Enhance
Watermarks Attribution lines Combine Layer/Server
Map: Copyright David Liuzzo
Map: Copyright David Liuzzo
Sources
WMS
1.0.0/1.1.1/1.3.0
WMS-C
TMS
Custom Tiles
(Google Maps/Bing/etc.)
Services
WMS
1.0.0/1.1.1/1.3.0
WMS-C
TMS
KML
Super Overlays
About the project
History
Started late 2008 In production since mid 2009 OpenSource since early 2010
Releases
0.8.4 current 0.9.0 upcoming release
Written in Runs on License Tested Deployment
Python Linux/Unix Windows Affero GPL v3 lots of unit/system tests HTTP or FastCGI
Distributed SCM Issue tracker Mailing list Blog
Example-driven introduction
Getting started
% paster create --template \ mapproxy_conf foss4g_example % cd foss4g_example % paster serve etc/develop.ini --reload
http://localhost:8080/service?
Example 1
Enhance existing WMS server
BlueMarble
by NASA
Satellite imagery
Background
US population
by demo.opengeo.org
Vector data
Overlay
sources
sources: us_pop_wms: type: wms req: url: http://demo.opengeo.org/geoserver/wms? layers: topp:states transparent: true format: image/png bluemarble_wms: type: wms supported_srs: ['EPSG:4326'] req: url: http://neowms.sci.gsfc.nasa.gov/wms/wms? layers: BlueMarbleNG
caches & layers
caches: us_pop_cache: sources: [bluemarble_wms, us_pop_wms] grids: [GLOBAL_GEODETIC]
layers: world: title: BlueMarble + US Population sources: [us_pop_cache]
services
services: tms: kml: wms: srs: ['EPSG:900913', 'EPSG:4326'] image_formats: [image/png, image/jpeg] md: title: MapProxy WMS Demo abstract: FOSS4G Demo contact: person: Oliver Tonnhofer organization: Omniscale
WMS
GetCapabilities
http://localhost:8080/service?request=GetCapabilities&service=WMS
GetMap
EPSG:4326
EPSG:900913
TMS
http://localhost:8080/tms/1.0.0/world_EPSG4326
<?xml version="1.0" encoding="UTF-8" ?> <TileMap version="1.0.0"> <Title>BlueMarble + Country Borders</Title> <Abstract></Abstract> <SRS>EPSG:4326</SRS> <BoundingBox minx="-180.0" miny="-90.0" maxx="180.0" maxy="90.0" /> <Origin x="-180.0" y="-90.0" /> <TileFormat width="256" height="256" mime-type="image/jpeg" extension="jpeg" /> <TileSets profile="global-geodetic"> <TileSet href="http://localhost:8080/tms/1.0.0/world_EPSG4326/0" units-per-pixel="0.703125" order="0" /> <TileSet href="http://localhost:8080/tms/1.0.0/world_EPSG4326/1" units-per-pixel="0.3515625" order="1" /> ...
TMS
http://localhost:8080/tms/world_EPSG4326/3/4/5.jpeg
HTTP cache control
ETag: 5c4a1f9bcaf1b95b283a512357ff42fc Cache-control: max-age=259200 public ...
HTTP/1.1 304 Not Modied
TMS
http://localhost:8080/tms/world_EPSG4326/3/4/5.jpeg
KML
http://localhost:8080/kml/world_EPSG4326/0/0/0.kml
Spherical mercator?
Google Maps, Bing, OSM, etc.
caches: us_pop_cache: sources: [bluemarble_wms, us_pop_wms] grids: [GLOBAL_GEODETIC, GLOBAL_MERCATOR]
Reprojects BlueMarble from EPSG:4326 adds tms/1.0.0/world_EPSG900913
Featureinfo?
sources: us_pop_wms: type: wms wms_opts: featureinfo: true req: url: http://demo.opengeo.org/geoserver/wms? layers: topp:states transparent: true format: image/png
world layer is now queryable
GetFeatureInfo
Example 2
Improve WMS performance
OpenStreetMap WMS with CPU/IO intensive rendering
Large dataset Lots of details Rendering with antialiasing
sources
sources: osm_wms: type: wms req: url: http://localhost:81/mapnik? layers: osm
caches
caches: osm_cache: sources: [osm_wms] grids: [osm_germany] link_single_color_images: true
Caching
MapProxy allows free-zooming but caches at xed resolutions
Vector scaling
x2 x2
Vector scaling
grid
grids: osm_germany: srs: EPSG:25832 bbox: [-3.82, 46.18, 24.91, 55.23] bbox_srs: EPSG:4326 res_factor: sqrt2
More grid options
min_resolution max_resolution num_levels
tile_size
res
res_factor
Cache pre-generation
% mapproxy-seed -f etc/mapproxy.yaml \ etc/seed.yaml -c 8
Cache pre-generation
% mapproxy-seed -f etc/mapproxy.yaml \ etc/seed.yaml -c 8
Multiprocessing
seed.yaml
views: germany: bbox: [7.36, 46.33, 14.28, 56.09] bbox_srs: EPSG:4326 level: [0, 14] srs: ['EPSG:900913', 'EPSG:4326'] seeds: osm: views: ['germany'] remove_before: month: 1 days: 3
Colorado
not-Colorado
seed.yaml
Any OGR source (Shapele, PostGIS, etc.)
germany: ogr_datasource: 'shps/world_boundaries_m.shp' ogr_where: 'CNTRY_NAME = "Germany"' ogr_srs: 'EPSG:900913' level: [0, 14] srs: ['EPSG:900913', 'EPSG:4326']
WKT polygons http://mapproxy.org/static/polygons/
germany: polygons: 'polygons/GM.txt' polygons_srs: EPSG:900913 level: [0, 14] srs: ['EPSG:900913', 'EPSG:4326']
Seed strategy
Normal
level per level, top to bottom, left to right
Seed strategy
Normal
level per level, top to bottom, left to right
mapproxy-seed
recursive depth-rst traversal works with the OS/DB cache
PNG8 870x530 = 50ms
Reprojected PNG8 870x530 = 130ms
Single Tile < 1ms
News
Python Image Library
Really fast C-based imaging library slow PNG performance only Issues: no transparency PNG8
Python Image Library
Really fast C-based imaging library Now xed! 10x faster same as libjpeg to be included in next PIL release full transparency for PNG8
http://bitbucket.org/olt/pil-117-fastpng
Jython
Python implementation for JVM
no support for C extensions
Jython
Python implementation for JVM
no support for C extensions Jython wrapper with
Java ImageIO/JAI GeoTools
Jython
Python implementation for JVM
no support for C extensions Jython wrapper with
Java ImageIO/JAI GeoTools MapProxy as .war
alpha release
NEW
Thank you.
Oliver Tonnhofer <[email protected]>
Visit Join Read
http://mapproxy.org http://lists.osgeo.org/mailman/ listinfo/mapproxy http://blog.mapproxy.org
Backup tiles below
Level 3
Level 4
Level 3
Level 4
Level 3
Level 4
Level 3
Level 4
Level 3
Level 4
Level 3
Level 4
Level 4
Level 3
Level 2
Level 4
Level 3
Level 2
Level 4
Level 3
Level 2
Level 4
Level 3
Level 2
Level 4
Level 3
Level 2
Level 4
Level 3
Level 2
Level 4
Level 3
Level 2
Level 5
Level 14
Seed with avg. 150 tiles/second on a Quad Core ([email protected])
Deployment
Develop Production
Protocol
HTTP
FastCGI or HTTP Apache, nginx, lighttpd, varnish, squid, ...
Server
embedded
with auto reloading
Clients
Services
MapProxy
Sources
Server
reprojection on request before caching on-the-y
merge layers watermarks fast PNG encoding
free zooming advanced seed strategy custom color quantizer improved vector support KML WMS empty tiles handling TMS
seed polygon areas WMS-C shapeles WKT
