Introduction Technical overview Example Finally... OSMAlchemy. A bridge between SQLAlchemy and the OpenStreetMap API.

Transkript

1 OSMAlchemy A bridge between SQLAlchemy and the OpenStreetMap API Dominik George PyUnconf, TU Hamburg / 24

2 Overview 1 Introduction What is OSMAlchemy Side notes 2 Technical overview OSMAlchemy itself SQLAlchemy to Overpass bridge 3 Example Example 1: Getting OSMAlchemy ready in a Flask app Example 2: Bakery Watch 4 Finally... Feedback The End 2 / 24

3 What is OSMAlchemy How OSMAlchemy was born Development of Veripeditus, an AR game framework Objective: Bind game objects to real-world OpenStreetMap data Interact with real-world OpenStreetMap data in a game Issues: Accessing OSM directly is slow (and not very nice) Keeping planet.osm (e.g. in PostGIS) is expensive 3 / 24

4 What is OSMAlchemy Ideas Have OSM objects readily available in the ORM (SQLAlchemy) Necessities: All of OSM must be available Only relevant data must be loaded Data must be available locally for repeated use Cool addons: Completely transparent Developers only see SQLAlchemy just as they know it Easily linking non-osm data Simple (trivial, really) support for creating REST APIs Dream: Applications using arbitrary OSM data runnable on a cheap VPS 4 / 24

5 Side notes Who is creating OSMAlchemy? Dominik George: This is me ;-) Software developer and systems administrator Head of Teckids e.v., a FOSS youth organisation Eike Tim Jesinghaus: 15 year-old Python hacker Very advanced developer since his 11th birthday Known for his very successful PyGame workshops (starting at age 11) 5 / 24

6 Introduction Technical overview Example Finally... Side notes 6 / 24

7 Side notes Current state Very early development Highly experimental Very limited support for queries Flaky because of an SQLAlchemy bug with PYTHONHASHSEED But working ;-)! WARNING: Crazily advanced (stupidly exotic?) SQLAlchemy stuff, including monkey-patching the library ;-) 7 / 24

8 Overview 1 Introduction What is OSMAlchemy Side notes 2 Technical overview OSMAlchemy itself SQLAlchemy to Overpass bridge 3 Example Example 1: Getting OSMAlchemy ready in a Flask app Example 2: Bakery Watch 4 Finally... Feedback The End 8 / 24

9 OSMAlchemy itself Data model Only current OSM data is used, not historic Support for basic OSM elements: Node Way Relation Tags on everything Nodes in ways and members in relations Straight-forward ORM mapping (e.g. tags as dictionary) 9 / 24

10 OSMAlchemy itself Other components Overpass API client SQL to OverpassQL compiler OSM (XML) data importer Basic query caching 10 / 24

11 OSMAlchemy itself How it provides data Querying, e.g. nodes within a bounding box Intercept the query from SQLAlchemy Compile the query to OverpassQL Look when the query was last run, if it was, and if run online recently, skip and provide locally.... if run long ago, import data from OverpassQL. Accessing an object in ORM Intercept the loading event from SQLAlchemy Look when the object was last touched in the database, and if touched recently, skip and provide locally.... if touched long ago, import data from OverpassQL. 11 / 24

12 SQLAlchemy to Overpass bridge How OpenStreetMap is accessed Online queries to Overpass API SQL queries are compiled to OverpassQL XML data is imported into the ORM 12 / 24

13 SQLAlchemy to Overpass bridge SQL to OverpassQL compiler Example: All bakeries within a bounding box SQLAlchemy (creating query): session.query(node).filter(and (Node.latitude...,..., Node.tags.any(key=ßhop, value= bakery ))).all() Would compile to SQL: SELECT... FROM osm elements... WHERE latitude... AND... AND... Compiles to OverpassQL node(50.0,6.0,51.0,7.0)[ßhop- bakery ]... Credits to Eike, who wrote most of it :) 13 / 24

14 Overview 1 Introduction What is OSMAlchemy Side notes 2 Technical overview OSMAlchemy itself SQLAlchemy to Overpass bridge 3 Example Example 1: Getting OSMAlchemy ready in a Flask app Example 2: Bakery Watch 4 Finally... Feedback The End 14 / 24

15 Example 1: Getting OSMAlchemy ready in a Flask app Getting OSMAlchemy ready in a Flask app from f l a s k import F l a s k from f l a s k s q l a l c h e m y import SQLAlchemy from osmalchemy import OSMAlchemy APP = F l a s k ( n a m e ) DB = SQLAlchemy (APP) OSM = OSMAlchemy (DB, o v e r p a s s=true ) DB. c r e a t e a l l ( ) APP. run ( ) 15 / 24

16 Example 1: Getting OSMAlchemy ready in a Flask app Adding a REST API from f l a s k import F l a s k from f l a s k r e s t l e s s import APIManager from f l a s k s q l a l c h e m y import SQLAlchemy from osmalchemy import OSMAlchemy APP = F l a s k ( n a m e ) DB = SQLAlchemy (APP) OSM = OSMAlchemy (DB, o v e r p a s s=true ) API = APIManager (APP, OSM. c r e a t e a p i ( API ) f l a s k s q l a l c h e m y d b=db) DB. c r e a t e a l l ( ) APP. run ( ) 16 / 24

17 Example 2: Bakery Watch Bakery Watch Small review/rating application for bakeries A bit like Qype Simple REST API with OSM data and reviews JavaScript web-frontend 17 / 24

18 Example 2: Bakery Watch How to do the Review class c l a s s Review (DB. Model ) : i d = DB. Column (DB. I n t e g e r, p r i m a r y k e y=true ) osm node id = DB. Column (DB. B i g I n t e g e r, DB. ForeignKey ( osm nodes. e l e m e n t i d ) ) osm node = DB. r e l a t i o n s h i p (OSM. node, b a c k r e f= r e v i e w s ) s t a r s = DB. Column (DB. I n t e g e r ) comment = DB. Column (DB. Unicode ( ) ) 18 / 24

19 Example 2: Bakery Watch Add a review bakery = DB. s e s s i o n. query (OSM. node ). f i l t e r b y ( i d = ). one ( ) r e v i e w = Review ( osm node=bakery, s t a r s =4, comment= They do bake bread! ) DB. s e s s i o n. add ( r e v i e w ) DB. s e s s i o n. commit ( ) 19 / 24

20 Example 2: Bakery Watch Query the REST API $ c u r l h t t p : / / : / a p i / osm nodes /769 {... i d : , l a t i t u d e : , l o n g i t u d e : ,... r e v i e w s : [ { comment : They do bake bread!, i d : 1, osm node id : 769, s t a r s : 4 } ], 20 / 24

21 Overview 1 Introduction What is OSMAlchemy Side notes 2 Technical overview OSMAlchemy itself SQLAlchemy to Overpass bridge 3 Example Example 1: Getting OSMAlchemy ready in a Flask app Example 2: Bakery Watch 4 Finally... Feedback The End 21 / 24

22 Feedback Questions from us What do you think about the idea? What features do you like/dislike? What would you like to see added? What would you use OSMAlchemy for? 22 / 24

23 Feedback Our future Veripeditus, the first, free, pure-python framework for Augmented Reality mobile gaming OpenBuddyMap, a Qype/FourSquare-like, fully open check-in and review website 23 / 24

24 The End Further reading Flood me via or Jabber: 24 / 24

25 The End Thanks Thanks for listening and all your feedback! 25 / 24